Identification method and device, electronic equipment and computer medium

Abstract

The present disclosure discloses a recognition method and device, electronic equipment and computer medium. The method comprises: determining a plurality of to-be-detected images and first coordinate information of each to-be-detected image in the initial image based on the initial image and a sliding window sampling method; for each to-be-detected image, determining second coordinate information of a preset object in the to-be-detected image; based on the second coordinate information and the first coordinate information of the to-be-detected image in the initial image, determining third coordinate information of the preset object in the initial image, obtaining a plurality of sets of third coordinate information corresponding to the plurality of to-be-detected images; and determining a recognition result corresponding to the preset object in the initial image based on the initial image and the plurality of sets of third coordinate information. The present disclosure divides the initial image into a plurality of to-be-detected images by the sliding window sampling method, and respectively identifies the coordinate information, thereby reducing the system data processing amount and improving the data processing efficiency.

Description

TECHNICAL FIELD

[0001] The present disclosure belongs to the technical field of data processing, and particularly relates to a recognition method and device, electronic equipment and a computer medium. BACKGROUND

[0002] With the development and popularization of artificial intelligence technology, optical character recognition technology (OCR) has been applied in various industries, and has become one of the indispensable basic capabilities in many technical systems. The mainstream OCR technology in the related art can basically be divided into two steps: character detection and character recognition. The purpose of character detection is to detect the character region range in the image, so that the image in the character region range can be cut off from the image for subsequent input into the character recognition system for character recognition.

[0003] The input of the current character detection method is the original image to be detected, and the output is the position coordinates of the characters in the image. However, when the pixel size (resolution) of the image to be detected is too large, serious problems will occur. For example, when the side length reaches tens of thousands of pixels, the image to be detected is input into the neural network with a built-in character detection algorithm at one time, and the running process of the neural network will cause a huge video memory or memory occupation, and the risk of system video memory or memory overflow is large. SUMMARY

[0004] The embodiments of the present disclosure provide an implementation scheme different from the related art to solve the technical problem that when a large-size image to be detected is input into a neural network with a built-in character detection algorithm at one time, the running process of the neural network will cause a huge video memory or memory occupation, and the risk of system video memory or memory overflow is large.

[0005] In a first aspect, the present disclosure provides a recognition method, comprising:

[0006] determining a plurality of images to be detected and first coordinate information of each image to be detected in the initial image based on the initial image and a sliding window sampling method;

[0007] determining second coordinate information of a preset object in each image to be detected;

[0008] determining third coordinate information of the preset object in the initial image based on the second coordinate information and the first coordinate information of the image to be detected in the initial image, to obtain a plurality of sets of third coordinate information corresponding to the plurality of images to be detected;

[0009] determining a recognition result corresponding to the preset object in the initial image based on the initial image and the plurality of sets of third coordinate information.

[0010] In a second aspect, the present disclosure provides a recognition device, the device comprising:

[0011] a first determining module configured to determine a plurality of to-be-detected images and first coordinate information of each to-be-detected image in the initial image based on the initial image and a sliding window sampling method;

[0012] a second determining module configured to determine second coordinate information of a preset object in each to-be-detected image;

[0013] a third determining module configured to determine third coordinate information of the preset object in the initial image based on the second coordinate information and the first coordinate information of the to-be-detected image in the initial image, to obtain a plurality of sets of third coordinate information corresponding to the plurality of to-be-detected images;

[0014] a recognition module configured to determine a recognition result of the preset object in the initial image based on the initial image and the plurality of sets of third coordinate information.

[0015] In a third aspect, the present disclosure provides an electronic device, comprising:

[0016] a processor; and

[0017] a memory configured to store executable instructions of the processor;

[0018] wherein the processor is configured to execute any method in the possible implementation manners of the first aspect by executing the executable instructions.

[0019] In a fourth aspect, the present disclosure provides a computer readable storage medium having a computer program stored thereon, the computer program being executed by a processor to implement any method in the possible implementation manners of the first aspect.

[0020] By the identification method of the present disclosure, the initial image can be divided into a plurality of to-be-detected images by a sliding window sampling method, and first coordinate information of each to-be-detected image in the initial image is determined. By sampling the initial image into a plurality of to-be-detected images, the data processing amount can be greatly reduced. The second coordinate information of the preset object in the to-be-detected image is determined for each to-be-detected image, and then the third coordinate information of the preset object in the initial image is determined in combination with the first coordinate information of the to-be-detected image in the initial image, a plurality of sets of third coordinate information corresponding to the plurality of to-be-detected images are obtained, and finally the identification content corresponding to the preset object in the initial image is determined based on the initial image and the plurality of sets of third coordinate information. By obtaining the second coordinate information of the preset object in the to-be-detected image, and then calculating the third coordinate information of the preset object in the initial image, and finally identifying the identification result corresponding to the preset object according to the third coordinate information, not only the accuracy of the identification result of the preset object is ensured, but also the initial image is divided into a plurality of to-be-detected images for processing, which can control the memory or internal memory occupation in the running process of the neural network within a certain range, and reduce the risk of overflow of the system memory or internal memory. BRIEF DESCRIPTION OF DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure or the related art, the drawings needed to be used in the embodiments or related art description will be briefly introduced. Obviously, the drawings in the following description are some embodiments of the present disclosure, and other drawings can also be obtained by those skilled in the art without creative labor. In the drawings:

[0022] Figure 1 A flowchart of an identification method provided by an embodiment of the present disclosure is shown in the figure;

[0023] Figure 2 A sliding window longitudinal movement schematic diagram provided by an embodiment of the present disclosure is shown in the figure;

[0024] Figure 3 A sliding window transverse movement schematic diagram provided by an embodiment of the present disclosure is shown in the figure;

[0025] Figure 4 A display diagram of the second coordinate information of the preset object in the to-be-detected image provided by an embodiment of the present disclosure is shown in the figure;

[0026] Figure 5 A display diagram of the preset object in adjacent two to-be-detected images provided by an embodiment of the present disclosure is shown in the figure;

[0027] Figure 6 A display diagram of the preset object in one to-be-detected image of adjacent two to-be-detected images provided by an embodiment of the present disclosure is shown in the figure;

[0028] Figure 7 A display image of a preset object in another one of the two images to be detected is provided for an embodiment of the present disclosure.

[0029] Figure 8 A structural schematic diagram of a recognition device is provided for an embodiment of the present disclosure.

[0030] Figure 9 A schematic block diagram of an electronic device is provided for an embodiment of the present disclosure. DETAILED DESCRIPTION

[0031] Embodiments of the present disclosure are described in detail below with reference to the accompanying drawings. The embodiments described below by reference to the accompanying drawings are exemplary and are intended to explain the present disclosure, and cannot be understood as a limitation of the present disclosure.

[0032] The terms "first" and "second" and the like in the specification of the embodiments of the present disclosure, claims, and drawings are used to distinguish similar objects, and do not necessarily have to describe a specific order or chronological order. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the embodiments of the present disclosure described herein can be implemented, for example, in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product, or apparatus including a series of steps or units does not have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to such processes, methods, products, or apparatuses.

[0033] With the development and popularization of artificial intelligence technology, optical character recognition technology (OCR) has been applied in various industries, and has become one of the indispensable basic capabilities in many technical systems. The current mainstream OCR technology for Chinese and English can basically be divided into two steps: character detection and character recognition. The purpose of character detection is to detect the range of the character region in the image, so as to be able to cut out the character part image from the image and input it into the character recognition system.

[0034] The input of the current text detection method is an original image to be detected, and the output is the position coordinates of the text in the image. However, when the pixel size (resolution) of the image to be detected is too large, serious problems will occur. For example, when the edge length reaches tens of thousands of pixels, the image to be detected is input into the neural network embedded with the text detection algorithm at one time, and the running process of the neural network will cause a huge memory or memory occupation, and the risk of system memory overflow is large. A remedial method is to scale the original image to be detected to a fixed small size before inputting it into the neural network embedded with the text detection algorithm, which can control the huge memory or memory occupation caused by the running process of the neural network within a certain range. However, this operation has a significant defect that it may cause the imaging quality of part of the small text in the original image to be detected to decrease, affecting the subsequent text recognition effect, and even directly leading to the loss of the text detection target.

[0035] The technical solution of the present disclosure is mainly applied in the technical field of text recognition.

[0036] The technical solution of the present disclosure and how the technical solution solves the above technical problems will be described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of the present disclosure will be described below with reference to the accompanying drawings.

[0037] Figure 1 A flowchart of an identification method provided for an exemplary embodiment of the present disclosure is shown. The method can be applied to various electronic devices with text recognition function. The method comprises S101-S104:

[0038] S101, determining a plurality of images to be detected and first coordinate information of each image to be detected in the initial image based on the initial image and the sliding window sampling method.

[0039] In some embodiments, the first coordinate information mainly refers to the coordinate information of a preset vertex of the image to be detected in the initial image. For example, the preset vertex is the vertex of the upper left corner of the image to be detected.

[0040] In some embodiments, determining a plurality of images to be detected based on the initial image and the sliding window sampling method comprises:

[0041] Determining the height and width of the sliding window, according to the sliding window sampling method, the height and the width, moving the sliding window in the initial image according to the preset initial position and the moving step length for several times, taking the image in the area falling into the sliding window each time as an image to be detected, and obtaining a plurality of images to be detected.

[0042] In this system, the height and width of the sliding window are consistent with the height and width of the image to be detected. Taking a text recognition system as an example, the width and height of the image to be detected (the height and width of the sliding window) are the input dimensions of the text detection model in the text detection system, which mainly depend on the maximum capacity of the actual model in the production environment.

[0043] In some embodiments, the preset initial position mainly refers to the initial position at which the sliding window begins to move. For example, in an initial image, if the sliding window starts to move vertically or horizontally from the position of the upper left corner of the initial image, then the position of the upper left corner of the initial image is the preset initial position.

[0044] In some embodiments, the movement step size primarily refers to the distance the sliding window moves vertically or horizontally.

[0045] In practice, taking an initial image A as an example, after determining the height and width of the sliding window, the sliding window starts to move from the upper left corner of the initial image. When the sliding window moves horizontally or vertically once according to the moving step, the image that falls into the area of ​​the sliding window each time is taken as an image to be detected. The sliding window continues to move until the entire initial image A falls into the sliding window in sequence.

[0046] In some implementations, the first coordinate information of each image to be detected in the initial image is determined based on the initial image and the sliding window sampling method, including S1011-S1015:

[0047] S1011, Determine the number of the first set of images to be detected obtained by the vertically moving sliding window.

[0048] The number of images to be detected acquired by the vertically moving sliding window is determined according to the following formula:

[0049]

[0050] Where H is the height of the initial image, ceiling is the floor function, and n H Overlap is the number of images to be detected acquired by the vertically moving sliding window. H The height of the overlapping portion between two adjacent images to be detected is obtained by moving the sliding window vertically. `Sampling_` is the height of the sliding window. See [link to documentation] for details. Figure 2 As shown.

[0051] In some embodiments, the height of the overlapping portion between two adjacent images to be detected, obtained by vertically moving the sliding window, mainly refers to the overlapping pixel distance. Setting the overlapping portion ensures that text near the boundary of the sampled image to be detected appears completely in the image at least once, preventing text loss due to incomplete sampling.

[0052] S1012, Determine the ordinate information of the first group of images to be detected.

[0053] The ordinate information of the first group of images to be detected is determined according to the following formula:

[0054]

[0055]

[0056] Among them, y_top i Let y_bottom be the ordinate of the upper boundary of the image to be detected. i Let i be the ordinate of the lower boundary of the image to be detected, where i takes values ​​in the range [1, n]. H ]; where each vertical axis is as follows Figure 2 As shown.

[0057] In some embodiments, such as Figure 2 As shown, when the initial image height H-Overlap H It is not the stride. h If the nth integer multiple of the nth integer, then the nth integer multiple of ... H Stride size for each sampling h The stride of the previous n-1 samplings h If they are different, then the moving step size is Stride. h Simply set y_bottom to H. This ensures the consistency of the sampling size and the integrity of the preset object.

[0058] S1013, determine the number of the second set of images to be detected obtained by the horizontally moving sliding window.

[0059] The number of images to be detected acquired by the horizontally moving sliding window is determined according to the following formula:

[0060]

[0061] Where W is the width of the initial image, n W Overlap is the number of images to be detected acquired by the horizontally moving sliding window. WSampling_W is the width of the overlapping portion between two adjacent images to be detected, obtained by horizontally moving the sliding window. For details, please refer to [link to relevant documentation]. Figure 3 As shown.

[0062] In some embodiments, the width of the overlap between two adjacent images to be detected, obtained by horizontally moving the sliding window, mainly refers to the overlapping pixel distance. Setting the overlap is to ensure that text near the boundary of the sampled image to be detected appears completely in the image at least once, thus preventing text loss due to incomplete sampling.

[0063] S1014, Determine the horizontal coordinate information of the second group of images to be detected.

[0064] The horizontal coordinate information of the second set of images to be detected is determined according to the following formula:

[0065]

[0066]

[0067] Where x_left j x_right is the x-coordinate of the left boundary of the image to be detected. j Let i be the x-coordinate of the right boundary of the image to be detected, where i takes values ​​in the range [1, n]. w [The x-axis can be found in the image below;] Figure 3 As shown.

[0068] In some embodiments, such as Figure 3 As shown, when the width W of the initial image is overlap W It is not the lateral movement step size Stride h If the nth integer multiple of the nth integer, then the nth integer multiple of ... H Stride size for each sampling h The stride of the previous n-1 samplings h If they are different, then the moving step size is Stride. h We can simply take x_right as the location W. This ensures the consistency of the sampling size and the integrity of the preset objects.

[0069] S1015, determine the first coordinate information of each image to be detected in the initial image based on the vertical coordinate information and the horizontal coordinate information.

[0070] The first coordinate information set of each image to be detected in the initial image is determined based on the vertical and horizontal coordinate information according to the following formula:

[0071] R(i,j)

[0072] =[(x_left) j ,_top i ),(x_h j y_top i ),(x_left j ,y_ i ),(x_h j ,y_ i )], i∈[1,n H ],j∈[1,n W ]

[0073] Where R(i,j) is the first coordinate information set of the image to be detected.

[0074] In this embodiment, the coordinate information of vertices at the same position in the initial image for each image to be detected is selected from the first coordinate information set as the first coordinate information of each image to be detected in the initial image. For example, the first coordinate information Region(i,j):

[0075] Region(i,j)=[x_left j ,y_top i ], i∈[1,n H ],j∈[1,n W ].

[0076] S102, for each image to be detected, determine the second coordinate information of the preset object in the image to be detected.

[0077] In some embodiments, the preset object is an object that has the function of expressing information, such as text, symbols, or graphics.

[0078] In some embodiments, determining the second coordinate information of the preset object in the image to be detected includes steps S1021-S1022:

[0079] S1021, Obtain the first connected component and the first edge information of the first connected component corresponding to the preset object in the image to be detected.

[0080] In some embodiments, a connected region is defined as a region G on a plane in which any simple closed curve drawn in the plane always belongs to the interior of G.

[0081] In some embodiments, based on a position detection algorithm, the position of a preset object in the image to be detected is detected to obtain the first connected component of the preset object in the image to be detected and the first edge information of the first connected component.

[0082] For example, taking text as the preset object, the DBNet++ algorithm (text detection) is used. A segmentation model based on adaptive multi-scale feature fusion obtains the feature map of the text in the image to be detected. Simultaneously, the text and background are determined pixel-by-pixel on the text threshold map obtained by processing the image in the differentiable binarization module, resulting in accurate information about the first connected component and its first edge. Each first connected component includes at least one character.

[0083] S1022, based on the first edge information, determine the minimum envelope rectangle corresponding to the first connected region, and determine the vertex coordinate information of the minimum envelope rectangle as the second coordinate information of the preset object in the image to be detected.

[0084] In some embodiments, the minimum envelope rectangle is primarily the smallest bounding rectangle that encloses the primitive and is parallel to the x and y axes.

[0085] In some embodiments, based on the first connected component and the first edge information of the first connected component of the preset object in the image to be detected obtained in step S1021, the minimum envelope rectangle of the preset object in the image to be detected is obtained according to the predetermined function cv2.minAreaRect. The second coordinate information includes multiple first vertex coordinates, that is, one of the coordinate information of the four vertices (four first vertex coordinates) of the minimum envelope rectangle constitutes the second coordinate information of the preset object in the image to be detected.

[0086] like Figure 4 As shown, taking the preset object as text as an example, the minimum envelope rectangle of the text 1 "AB" is the rectangle formed by the dashed lines around "AB". The coordinates of the four vertices of this rectangle in the image to be detected constitute the second coordinate information of the text 1 "AB" in the image to be detected: [(x1,y1),(x2,y1),(x1,y2),(x2,y2)].

[0087] S103, based on the second coordinate information and the first coordinate information of the image to be detected in the initial image, determine the third coordinate information of the preset object in the initial image, and obtain several sets of third coordinate information corresponding to several images to be detected.

[0088] In some embodiments, determining the third coordinate information of a preset object in the initial image based on the second coordinate information and the first coordinate information of the image to be detected in the initial image includes:

[0089] For each first vertex coordinate, the first vertex coordinate is summed with the first coordinate information to obtain the second vertex coordinate, and then multiple second vertex coordinates corresponding to multiple first vertex coordinates are obtained; the multiple second vertex coordinates are used as the third coordinate information of the preset object in the initial image.

[0090] To ensure the accuracy of the first and second coordinate information, fixed points in both the initial and the image to be detected are used as the origin of the coordinate system during the acquisition process. For example, in the initial image, the top-left corner is used as the origin, with the positive x-axis to the right of the origin and the positive y-axis below the origin. In the image to be detected, the top-left corner is used as the origin, with the positive x-axis to the right of the origin and the positive y-axis below the origin.

[0091] Therefore, in this embodiment of the present disclosure, for different first coordinate information of the image to be detected, the first coordinate information of the image to be detected is summed with the coordinates of each first vertex in the second coordinate information corresponding to the preset object in the image to be detected, so as to obtain the third vertex coordinates corresponding to each first vertex coordinate, and the multiple second vertex coordinates are used as the third coordinate information of the preset object in the initial image.

[0092] For example, taking text as the preset object, if the first coordinate information of the image to be detected A is (1, 1), the second coordinate information of text 1 in the image to be detected A is [(1, 1), (2, 1), (1, 2), (2, 2)], the second coordinate information of text 2 is [(3, 4), (4, 4), (3, 5), (4, 5)], the second coordinate information of text 3 is [(6, 7), (7, 7), (6, 8), (7, 8)], and then the third coordinate information of text 1 in the initial image is [(2, 2), (3, 2), (2, 3), (3, 3)], the third coordinate information of text 2 in the initial image is [(4, 5), (5, 5), (4, 6), (5, 6)], and the third coordinate information of text 3 in the initial image is [(7, 8), (8, 8), (7, 9), (8, 9)].

[0093] If the first coordinate information of the image B to be detected is (4, 4), the second coordinate information of the text 1 in the image B to be detected is [(2,2), (3,2), (2,3), (3,3)], the second coordinate information of the text 2 is [(4,5), (5,5), (4,6), (5,6)], the second coordinate information of the text 3 is [(7,8), (8,8), (7,9), (8,9)], and then the third coordinate information of the text 1 in the initial image is [(6,6), (7,6), (6,7), (7,7)], the third coordinate information of the text 2 in the initial image is [(8,9), (9,9), (8,10), (9,10)], and the third coordinate information of the text 3 in the initial image is [(11,12), (12,12), (11,13), (12,13)].

[0094] In some embodiments, taking text as an example, such as... Figure 5As shown, two adjacent images to be detected (image C and image D) each contain their corresponding preset objects; that is, image C contains text 4, text 5, and text 6, and image D contains text 7, text 8, and text 9. In fact, in the overlapping area of ​​the two adjacent images ( Figure 5 In the area where the solid and dashed lines overlap, the following three situations may occur:

[0095] First, the text 4 in image C is exactly the same as the text 7 in image D. Second, the text 6 in image D is part of the text 5 in image C. Finally, the text 6 in image C and the text 9 in image D are both incomplete texts, and the text 6 and the text 9 can be combined to form a complete text.

[0096] In some embodiments, the method further includes: for two adjacent images to be detected, traversing the first preset objects in the two images to be detected, and determining the pixel area and pixel area intersection-union ratio of the first preset objects contained in each of the two images to be detected;

[0097] If the pixel area intersection-union ratio is greater than the first threshold, and the ratio of the largest pixel area among the two first preset objects contained in the two images to be detected to the pixel area of ​​the union of the two first preset objects is greater than the third threshold, then the third coordinate information corresponding to the second preset object with the smallest pixel area in the two images to be detected is deleted from several sets of third coordinate information.

[0098] In some embodiments, the pixel area intersection-union ratio mainly refers to the ratio of the intersection to the union of the pixel areas of two preset objects.

[0099] In some embodiments, if the pixel area intersection-union ratio of two first preset objects in two adjacent images to be detected is greater than a first threshold, it means that the two first preset objects have overlapped; if the pixel area intersection-union ratio is not greater than the first threshold, it means that the two preset objects have not overlapped.

[0100] For example, taking images C and D to be detected, with a first threshold of 0, and a third threshold of 0.99, as shown... Figure 5-7As shown, the pixel area of ​​the character 5 in the image to be detected is determined to be 8, the pixel area of ​​the character 8 in the image to be detected is determined to be 6, and the area of ​​the overlapping part of the pixels of the characters 5 and 8 is 6. The intersection-union ratio of the pixel areas of the characters 5 and 8 is determined to be 0.75. Since the intersection-union ratio of the pixel areas of the characters 5 and 8 is 0.75, which is greater than the first threshold 0, the ratio of the largest pixel area of ​​the characters 5 and 8 (the pixel area of ​​the character 5 is 8) to the pixel area of ​​the union of the characters 5 and 8 (the pixel area of ​​the union is 8) is calculated to be 1. Since the ratio 1 is greater than the third threshold 0.99, the third coordinate information corresponding to the character 8 with a pixel area of ​​6 will be deleted from several sets of third coordinate information.

[0101] In some embodiments, if the ratio of the largest pixel area among the pixel areas of two first preset objects contained in the two images to be detected to the pixel area of ​​the union of the two first preset objects is not greater than a third threshold, it indicates that the preset objects contained in the two adjacent images to be detected are not complete. That is, as follows Figure 6-7 In this embodiment, the text 6 in the image to be detected is “ABCDEFGHJKLM”, and the text 9 in the image to be detected is “FGHJKLMBFDE”. When the two adjacent detection images are combined, as shown... Figure 5 The text “ABCDEFGHJKLMBFDE” is a complete object.

[0102] For example, assuming the images to be detected are C and D, and the third threshold is 0.99, then... Figure 5-7 As shown, the pixel area of ​​the character 6 in the image to be detected is determined to be 12, the pixel area of ​​the character 9 in the image to be detected is 11, and the pixel area of ​​the union of the characters 6 and 9 is 16. The ratio of the largest pixel area of ​​the characters 6 and 9 (the pixel area of ​​the character 6 is 12) to the pixel area of ​​the union of the characters 6 and 9 (the pixel area of ​​the union is 16) is 0.75. Therefore, the ratio 0.75 is not greater than the third threshold of 0.99, indicating that the characters 6 and 9 are not complete.

[0103] In this embodiment, when the ratio of the largest pixel area among the pixel areas of the two first preset objects contained in the two images to be detected to the pixel area of ​​the union of the two first preset objects is not greater than a third threshold, the third coordinate information of the preset objects is determined as follows: steps S1031-S1035:

[0104] S1031, based on the initial image and the first preset objects contained in each of the two adjacent images to be detected, determine the third preset object in the initial image.

[0105] For example, taking the first preset object contained in image C as the text "6" and the first preset object contained in image D as the text "9", as follows: Figure 5-7 As shown, the text 6 in the image to be detected C is “ABCDEFGHJKLM”, and the text 9 in the image to be detected D is “FGHJKLMBFDE”. Based on the third preset object in the images to be detected C and D, namely… Figure 5 The symbol "ABCDEFGHJKLMBFDE" is shown in the image.

[0106] This disclosure ensures that the boundaries of local sampling do not cause omissions of preset objects by setting a reasonable overlapping sampling area between two adjacent images to be detected and merging preset objects in subsequent processing; furthermore, since the quality of the images to be detected each time is consistent with the initial image, the detection results of preset objects are accurate and reliable.

[0107] S1032, Obtain the second connected component and the second edge information of the second connected component corresponding to the third preset object in the initial image.

[0108] In some embodiments, the second connected component corresponding to the third preset object and the second edge information corresponding to the second connected component are determined. The specific determination method can be found in the above content and will not be repeated here.

[0109] S1033, based on the second edge information, determine the minimum envelope rectangle corresponding to the second connected region, and determine the vertex coordinate information of the minimum envelope rectangle corresponding to the second connected region as the fourth coordinate information of the third preset object in the initial image.

[0110] In some embodiments, the third preset object "ABCDEFGHJKLMBFDE" obtained in step S1031 is used. Figure 5 The dashed lines surrounding the third preset object "ABCDEFGHJKLMBFDE" represent the minimum envelope rectangle corresponding to the third preset object in the second connected region. The vertex coordinate information of the minimum envelope rectangle represents the fourth coordinate information of the third preset object in the initial image.

[0111] S1034, delete the third coordinate information corresponding to the first preset object contained in each of two adjacent images to be detected from several sets of third coordinate information.

[0112] S1035, add the fourth coordinate information as new third coordinate information to several groups of third coordinate information.

[0113] S104, based on the initial image and several sets of third coordinate information, determine the recognition result corresponding to the preset object in the initial image.

[0114] In some embodiments, taking text as an example, the third coordinate information of all text in the initial image is obtained. Then, the text region image can be extracted from the initial image according to the third coordinate information. For tilted text region images, the tilted text region image can be rotated and interpolated to obtain a non-tilted result. Subsequently, it is normalized to a uniform scale using a scaling method (such as cv2.resize) and then input into the neural network of the text detection algorithm to obtain the text content.

[0115] The recognition method disclosed herein divides the initial image into several images to be detected using a sliding window sampling method, and determines the first coordinate information of each image to be detected in the initial image. By dividing the initial image into multiple images to be detected, the amount of data processing can be greatly reduced. For each image to be detected, the second coordinate information of the preset object in the image to be detected is determined. Then, combined with the first coordinate information of the image to be detected in the initial image, the third coordinate information of the preset object in the initial image is determined, resulting in several sets of third coordinate information corresponding to several images to be detected. Finally, based on the initial image and the several sets of third coordinate information, the recognition content corresponding to the preset object in the initial image is determined. By obtaining the second coordinate information of the preset object in the image to be detected, the third coordinate information of the preset object in the initial image is calculated. Finally, the recognition result corresponding to the preset object is identified based on the third coordinate information. This not only ensures the accuracy of the preset object recognition result, but also, by dividing the initial image into multiple images to be detected for processing, the memory or video memory usage during the operation of the neural network can be controlled within a certain range, reducing the risk of system memory or video memory overflow.

[0116] Figure 8 This is a schematic diagram of the structure of an identification device provided for an exemplary embodiment of the present disclosure. The structure includes: a first determining module 201, a second determining module 202, a third determining module 203, and an identification module 204.

[0117] The first determining module 201 is used to determine a number of images to be detected and the first coordinate information of each image to be detected in the initial image based on the initial image and the sliding window sampling method;

[0118] The second determining module 202 is used to determine the second coordinate information of the preset object in each image to be detected;

[0119] The third determining module 203 is used to determine the third coordinate information of the preset object in the initial image based on the second coordinate information and the first coordinate information of the image to be detected in the initial image, so as to obtain several sets of third coordinate information corresponding to several images to be detected.

[0120] The recognition module 204 is used to determine the recognition result corresponding to the preset object in the initial image based on the initial image and several sets of third coordinate information.

[0121] In some embodiments, the first determining module 201 is further configured to determine the height and width of the sliding window;

[0122] Based on the sliding window sampling method, height, and width, and according to the preset initial position and moving step size, the sliding window is moved several times in the initial image. Each time, the image falling into the region of the sliding window is taken as a detection image, resulting in several detection images.

[0123] In some embodiments, the first determining module 201 is further configured to determine the number of the first set of images to be detected obtained by the vertically moving sliding window;

[0124] Determine the ordinate information of the first group of images to be detected;

[0125] Determine the number of the second set of images to be detected obtained by the horizontally moving sliding window;

[0126] Determine the x-coordinate information of the second group of images to be detected;

[0127] The first coordinate information of each image to be detected in the initial image is determined based on the vertical coordinate information and the horizontal coordinate information.

[0128] In some embodiments, the second determining module 202 is further configured to obtain the first connected component corresponding to the preset object in the image to be detected and the first edge information of the first connected component;

[0129] Based on the first edge information, the minimum envelope rectangle corresponding to the first connected region is determined, and the vertex coordinate information of the minimum envelope rectangle is determined as the second coordinate information of the preset object in the image to be detected.

[0130] In some embodiments, the second coordinate information includes the coordinates of multiple first vertices.

[0131] In some embodiments, the third determining module 203 is further configured to sum the first vertex coordinates and the first coordinate information for each first vertex coordinate to obtain the second vertex coordinates, and then obtain multiple second vertex coordinates corresponding to multiple first vertex coordinates;

[0132] The coordinates of multiple second vertices are used as the third coordinate information of the preset object in the initial image.

[0133] In some embodiments, the third determining module 203 is further configured to, for two adjacent images to be detected, traverse the first preset objects in the two images to be detected, and determine the pixel area and pixel area intersection ratio of the first preset objects contained in each of the two images to be detected.

[0134] If the pixel area intersection-union ratio is greater than the first threshold, and the ratio of the largest pixel area among the two first preset objects contained in the two images to be detected to the pixel area of ​​the union of the two first preset objects is greater than the third threshold, then the third coordinate information corresponding to the second preset object with the smallest pixel area in the two images to be detected is deleted from several sets of third coordinate information.

[0135] In some embodiments, the third determining module 203 is further configured to determine the third preset object in the initial image based on the initial image and the first preset objects contained in each of the two adjacent images to be detected, provided that the ratio of the largest pixel area among the pixel areas of the two first preset objects contained in the two images to be detected to the pixel area of ​​the union of the two first preset objects is not greater than a third threshold.

[0136] Obtain the second connected component and the second edge information of the second connected component corresponding to the third preset object in the initial image;

[0137] Based on the second edge information, the minimum envelope rectangle corresponding to the second connected region is determined, and the vertex coordinate information of the minimum envelope rectangle corresponding to the second connected region is determined as the fourth coordinate information of the third preset object in the initial image;

[0138] From several sets of third coordinate information, delete the third coordinate information corresponding to the first preset object contained in each of two adjacent images to be detected;

[0139] The fourth coordinate information is added as new third coordinate information to several groups of third coordinate information.

[0140] The execution principles and interaction processes of the constituent modules in this device embodiment, such as the first determining module 201, the second determining module 202, the third determining module 203, and the identification module 204, can be found in the descriptions of the above method embodiments.

[0141] The recognition device disclosed herein can divide an initial image into several images to be detected using a sliding window sampling method, and determine the first coordinate information of each image to be detected in the initial image. By dividing the initial image into multiple images to be detected, the amount of data processing can be greatly reduced. For each image to be detected, the second coordinate information of a preset object in the image to be detected is determined. Then, combined with the first coordinate information of the image to be detected in the initial image, the third coordinate information of the preset object in the initial image is determined, resulting in several sets of third coordinate information corresponding to several images to be detected. Finally, based on the initial image and the several sets of third coordinate information, the recognition content corresponding to the preset object in the initial image is determined. By obtaining the second coordinate information of the preset object in the image to be detected, the third coordinate information of the preset object in the initial image is calculated. Finally, the recognition result corresponding to the preset object is identified based on the third coordinate information. This not only ensures the accuracy of the preset object recognition result, but also, by dividing the initial image into multiple images to be detected for processing, the memory or video memory usage during the operation of the neural network can be controlled within a certain range, reducing the risk of system memory or video memory overflow.

[0142] It should be understood that the device embodiments and method embodiments can correspond to each other, and similar descriptions can be referred to the method embodiments.

[0143] It should be understood that the device embodiments and method embodiments can correspond to each other, and similar descriptions can be referred to the method embodiments. To avoid repetition, they will not be repeated here. Specifically, the device can execute the above method embodiments, and the foregoing and other operations and / or functions of each module in the device correspond to the corresponding processes in the various methods in the above method embodiments, which will not be repeated here for the sake of brevity.

[0144] The apparatus of this disclosure embodiment has been described above from the perspective of functional modules in conjunction with the accompanying drawings. It should be understood that this functional module can be implemented in hardware, in software instructions, or in a combination of hardware and software modules. Specifically, the steps of the method embodiments in this disclosure can be completed by integrated logic circuits in the processor's hardware and / or by software instructions. The steps of the method disclosed in this disclosure embodiment can be directly embodied as being executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. Optionally, the software module can reside in a mature storage medium in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps in the above method embodiments.

[0145] Figure 9This is a schematic block diagram of an electronic device provided in an embodiment of this disclosure. The electronic device may include:

[0146] The system includes a memory 301 for storing computer programs and a processor 302 for transferring program code to the processor 302. In other words, the processor 302 can retrieve and run the computer program from the memory 301 to implement the methods described in this embodiment.

[0147] For example, the processor 302 can be used to execute the above-described method embodiments according to instructions in the computer program.

[0148] In some embodiments of this disclosure, the processor 302 may include, but is not limited to:

[0149] General-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0150] In some embodiments of this disclosure, the memory 301 includes, but is not limited to:

[0151] Volatile memory and / or non-volatile memory. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), and Direct Rambus RAM (DR RAM).

[0152] In some embodiments of this disclosure, the computer program may be divided into one or more modules, which are stored in the memory 301 and executed by the processor 302 to perform the method provided in this disclosure. The one or more modules may be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program in the electronic device.

[0153] like Figure 9 As shown, the electronic device may further include:

[0154] Transceiver 303, which can be connected to processor 302 or memory 301.

[0155] The processor 302 can control the transceiver 303 to communicate with other devices; specifically, it can send information or data to other devices or receive information or data sent by other devices. The transceiver 303 may include a transmitter and a receiver. The transceiver 303 may further include antennas, and the number of antennas may be one or more.

[0156] It should be understood that the various components in the electronic device are connected through a bus system, which includes a data bus, a power bus, a control bus, and a status signal bus.

[0157] This disclosure also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a computer, enables the computer to perform the methods of the above-described method embodiments. Alternatively, embodiments of this disclosure also provide a computer program product containing instructions that, when executed by a computer, cause the computer to perform the methods of the above-described method embodiments.

[0158] When implemented using software, it can be implemented wholly or partially as a computer program product. This computer program product includes one or more computer instructions. When these computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this disclosure are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., digital video disc (DVD)), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0159] Those skilled in the art will recognize that the modules and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this disclosure.

[0160] In the embodiments provided in this disclosure, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or modules may be electrical, mechanical, or other forms.

[0161] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. For example, the functional modules in the various embodiments of this disclosure may be integrated into one processing module, or each module may exist physically separately, or two or more modules may be integrated into one module.

[0162] The above are merely specific embodiments of this disclosure, but the scope of protection of this disclosure is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this disclosure should be included within the scope of protection of this disclosure. Therefore, the scope of protection of this disclosure should be determined by the scope of the claims.

Claims

1. A recognition method, characterized in that, The method includes: Based on the initial image and the sliding window sampling method, several images to be detected and the first coordinate information of each image to be detected in the initial image are determined; For each image to be detected, determine the second coordinate information of the preset object in the image to be detected; Based on the second coordinate information and the first coordinate information of the image to be detected in the initial image, the third coordinate information of the preset object in the initial image is determined, and several sets of third coordinate information corresponding to the several images to be detected are obtained; Based on the initial image and the several sets of third coordinate information, the recognition result corresponding to the preset object in the initial image is determined.

2. The method according to claim 1, characterized in that, The method based on the initial image and sliding window sampling determines several images to be detected, including: Determine the height and width of the sliding window; Based on the sliding window sampling method, the height, and the width, the sliding window is moved several times in the initial image according to the preset initial position and the moving step size. Each image that falls into the area of ​​the sliding window is taken as a detection image, thus obtaining the several detection images.

3. The method according to claim 2, characterized in that, The method for determining the first coordinate information of each image to be detected in the initial image based on the initial image and the sliding window sampling method includes: Determine the number of the first group of images to be detected obtained by vertically moving the sliding window; Determine the ordinate information of the first group of images to be detected; Determine the number of the second set of images to be detected obtained by horizontally moving the sliding window; Determine the abscissa information of the second group of images to be detected; Based on the vertical coordinate information, the horizontal coordinate information is used to determine the first coordinate information of each image to be detected in the initial image.

4. The method according to claim 1, characterized in that, Determining the second coordinate information of the preset object in the image to be detected includes: Obtain the first connected component and the first edge information of the first connected component corresponding to the preset object in the image to be detected; Based on the first edge information, the minimum envelope rectangle corresponding to the first connected region is determined, and the vertex coordinate information of the minimum envelope rectangle is determined as the second coordinate information of the preset object in the image to be detected.

5. The method according to any one of claims 1 to 4, characterized in that, The second coordinate information includes the coordinates of multiple first vertices; Based on the second coordinate information and the first coordinate information of the image to be detected in the initial image, the third coordinate information of the preset object in the initial image is determined, including: For each of the first vertex coordinates, the first vertex coordinates are summed with the first coordinate information to obtain the second vertex coordinates, and then the multiple second vertex coordinates corresponding to the multiple first vertex coordinates are obtained; The coordinates of the plurality of second vertices are used as the third coordinate information of the preset object in the initial image.

6. The method according to claim 5, characterized in that, The method further includes: For two adjacent images to be detected, traverse the first preset objects in the two images to be detected, and determine the pixel area and pixel area intersection-union ratio of the first preset objects contained in each of the two images to be detected; If the pixel area intersection-union ratio is greater than the first threshold, and the ratio of the largest pixel area among the two first preset objects contained in the two images to the pixel area of ​​the union of the two first preset objects is greater than the third threshold, then the third coordinate information corresponding to the second preset object with the smallest pixel area in the two images to be detected is deleted from the plurality of sets of third coordinate information.

7. The method according to claim 6, characterized in that, The method further includes: If the ratio of the largest pixel area among the two first preset objects contained in the two images to be detected to the pixel area of ​​the union of the two first preset objects is not greater than a third threshold, a third preset object in the initial image is determined based on the initial image and the two first preset objects. Obtain the second connected component and the second edge information of the second connected component corresponding to the third preset object in the initial image; Based on the second edge information, the minimum envelope rectangle corresponding to the second connected component is determined, and the vertex coordinate information of the minimum envelope rectangle corresponding to the second connected component is determined as the fourth coordinate information of the third preset object in the initial image; From the plurality of sets of third coordinate information, delete the third coordinate information corresponding to the first preset object contained in each of the two adjacent images to be detected; The fourth coordinate information is added as new third coordinate information to the plurality of groups of third coordinate information.

8. An identification device, characterized in that, The device includes: The first determining module is used to determine a number of images to be detected and the first coordinate information of each image to be detected in the initial image based on the initial image and the sliding window sampling method; The second determining module is used to determine the second coordinate information of a preset object in the image to be detected for each image to be detected; The third determining module is used to determine the third coordinate information of the preset object in the initial image based on the second coordinate information and the first coordinate information of the image to be detected in the initial image, thereby obtaining several sets of third coordinate information corresponding to the several images to be detected. The recognition module is used to determine the recognition result corresponding to the preset object in the initial image based on the initial image and the several sets of third coordinate information.

9. An electronic device, characterized in that, include: processor; as well as Memory for storing the executable instructions of the processor; The processor is configured to perform the method of any one of claims 1-7 by executing the executable instructions.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements any one of the methods in 1-7.

Images