Image detection method, device, storage medium and equipment

Through pixel point matching technology, the problem of slow matching speed and poor accuracy of text lines and single word positions in existing multi-grained OCR detection is solved, and more efficient text detection results are achieved.

CN114332836BActive Publication Date: 2024-11-01BEIJING YOUZHUJU NETWORK TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202111604852.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-12-24
Publication Date
2024-11-01
Estimated Expiration
2041-12-24

AI Technical Summary

Technical Problem

In dense text scenes, the matching speed of text line positions and single word positions is slow, and the calculated IOU results are not accurate enough when the shape is irregular, resulting in poor matching effect and high-accurate text detection cannot be obtained. result.

Method used

By obtaining the target image, we can detect the binary map of the text line and the single word position, mark the text line, determine the pixel coordinate value, and calculate the pixel coordinate value of the center position of the single word, perform pixel point matching, and determine the text line identification to which the single word belongs. , realize the matching of text lines and single word positions.

Benefits of technology

It improves the matching effect of text lines and single word positions, reduces the computational complexity, and improves the accuracy and efficiency of text detection results.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114332836B_ABST
    Figure CN114332836B_ABST
Patent Text Reader

Abstract

The present application discloses an image detection method, device, storage medium and equipment, the method comprising: firstly detecting an acquired target image to obtain a binary image containing text line positions and a binary image containing single word positions, then marking the text lines in the binary image containing the text line positions to obtain the identifiers corresponding to each text line; and determining the coordinate values ​​of each pixel point in the position of each text line, and calculating the pixel point coordinate values ​​of the center position of each single word in the binary image containing the single word positions; then, matching the pixel point coordinate values ​​of the center position of each single word position with the coordinate values ​​of each pixel point in the position of each text line, determining the text line identifier to which each single word belongs, and then filling each single word into the text line of the corresponding identifier to achieve accurate matching of the text line position and the single word position, thereby improving the accuracy of the detection result and the detection efficiency.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image detection method, device, storage medium and equipment. Background Art

[0002] With the continuous breakthroughs and development of artificial intelligence (AI) technology, the demand for optical character recognition (OCR) recognition technology is gradually increasing. Among them, multi-granularity OCR detection refers to the simultaneous detection of text lines and single words in an image, which is equivalent to detecting text line position information and single word position information at the same time. Because some business scenarios require both text line information and single word information. Therefore, how to accurately match and integrate text line positions and single word positions is currently an important issue in related fields.

[0003] At present, when performing multi-granular OCR detection, the method for matching and integrating text line positions and single word positions usually adopts a grouping scheme based on the intersection-over-union (IOU) ratio of text line positions and single word positions. From the perspective of sets in mathematics, Figure 1 As shown in , IOU refers to dividing the intersection of two regions by the union of the two regions, and judging whether the positions of the two regions match based on the ratio. However, for dense text image scenes, when matching text line positions and single word positions, a large number of Figure 1 The box shown in the figure calculates IOU, which results in a slow matching speed. This is because for polygons, a toolkit such as Polygon is needed to calculate the IOU of two boxes, which makes the calculation complexity (the product of the number of single words and the number of text lines) high. In addition, for irregular shapes (such as concave polygons, etc.), considering the calculation speed, when calculating IOU, it will be converted into the minimum circumscribed rotated rectangle (minAreaRect) for calculation. For inclined, irregularly shaped, and containing position boxes, the calculated IOU result is not accurate enough, which leads to poor final matching effect and failure to obtain more accurate text detection results. Summary of the invention

[0004] The main purpose of the embodiments of the present application is to provide an image detection method, apparatus, storage medium and equipment, which can more accurately match and integrate text line position information and single word position information through pixel coordinate matching, improve matching effect, and thus improve the accuracy of text detection results.

[0005] The present application provides an image detection method, including:

[0006] Acquire a target image to be detected; and detect the target image to obtain a binary image containing text line positions and a binary image containing single word positions;

[0007] Marking the text lines in the binary image containing the text line positions to obtain the identifiers corresponding to the text lines; and determining the coordinate values ​​of the pixels at the positions of the text lines;

[0008] Calculate the pixel coordinate values ​​of the center position of each individual word position in the binary image containing the individual word position;

[0009] Matching the pixel coordinate values ​​of the center position of each word with the coordinate values ​​of each pixel in the position of each text line to determine the text line identifier to which each word belongs;

[0010] The individual words are filled into the text lines with corresponding marks, so as to match the positions of the text lines and the individual words, and obtain the detection results of the words in the target image.

[0011] In a possible implementation, the calculating of the pixel coordinate values ​​of the center position of each word in the binary image containing the word position includes:

[0012] In the coordinate system where the binary image containing the word positions is located, the coordinate values ​​of each pixel point at the position of each word are calculated;

[0013] The average value of the coordinate values ​​of each pixel point is calculated as the coordinate value of the pixel point at the center position of the corresponding word.

[0014] In a possible implementation, the calculating of the pixel coordinate values ​​of the center position of each word in the binary image containing the word position includes:

[0015] In the coordinate system where the binary image containing the word positions is located, the coordinate values ​​of each pixel point at the position of each word are calculated;

[0016] Calculate the average value of the coordinate values ​​of each pixel point as the coordinate value of the primary selected pixel point at the center position of the corresponding word;

[0017] According to the coordinate values ​​of the initially selected pixel points, the coordinate values ​​of N pixel points in the preset area of ​​the center position of the corresponding single word are determined as the coordinate values ​​of the pixel points at the center position of the corresponding single word; wherein N is a positive integer greater than 1.

[0018] In a possible implementation, matching the pixel point coordinate values ​​of the center positions of the positions of the individual words with the coordinate values ​​of the individual pixel points in the positions of the individual text lines to determine the text line identifiers to which the individual words belong includes:

[0019] Calculate the difference between the coordinate value of the pixel point at the center position of each word and the coordinate value of each pixel point at the position of each text line;

[0020] If the difference is lower than a preset difference threshold, the pixel point in the text row corresponding to the difference is used as the pixel point that successfully matches the pixel point at the center position of the corresponding word, and the identifier of the text row where the successfully matched pixel point is located is used as the text row identifier to which the corresponding word belongs.

[0021] In a possible implementation, matching the pixel point coordinate values ​​of the center positions of the positions of the individual words with the coordinate values ​​of the individual pixel points in the positions of the individual text lines to determine the text line identifiers to which the individual words belong includes:

[0022] Calculate the difference between the coordinate values ​​of the N pixel points and the coordinate value of each pixel point at the location of each text line;

[0023] If the difference value satisfies a preset difference value threshold, the N pixel points in the position of the text line corresponding to the difference value are used as the N pixel points that successfully match the N pixel points, and the identifier of the text line where the N pixel points that successfully match are located is determined as the identifier of the preliminary text line to which the corresponding single word belongs;

[0024] The majority of the initially selected text line identifiers is calculated as the text line identifier to which the corresponding single word belongs.

[0025] In a possible implementation, the method further includes:

[0026] If, after matching the pixel coordinate values ​​of the center positions of the respective words with the pixel coordinate values ​​of the respective pixel points in the respective text lines, an unmatched target word appears, a new target text line is created according to the position of the target word, and the target word is filled into the target text line.

[0027] In a possible implementation, the detecting of the target image to obtain a binary image containing text line positions and a binary image containing single word positions includes:

[0028] The target image is detected by a multi-granularity OCR detection model, and a binary image containing text line positions and a binary image containing single word positions are output.

[0029] The present application also provides an image detection device, the device comprising:

[0030] A detection unit is used to obtain a target image to be detected; and detect the target image to obtain a binary image containing text line positions and a binary image containing single word positions;

[0031] A marking unit, used to mark the text lines in the binary image containing the text line positions, obtain the identifier corresponding to each text line; and determine the coordinate value of each pixel point at the position of each text line;

[0032] A computing unit, for calculating the pixel coordinate values ​​of the center position of each individual word position in the binary image containing the individual word position;

[0033] A matching unit, used for matching the pixel point coordinate value of the center position of each word with the coordinate value of each pixel point in the position of each text line, to determine the text line identifier to which each word belongs;

[0034] The filling unit is used to fill each single word into the text line with the corresponding mark, so as to match the position of the text line with the position of the single word and obtain the detection result of the text in the target image.

[0035] In a possible implementation, the computing unit includes:

[0036] A first calculating subunit is used to calculate the coordinate values ​​of each pixel point at the position of each word in the coordinate system where the binary image containing the word position is located;

[0037] The second calculation subunit is used to calculate the average value of the coordinate values ​​of each pixel point as the coordinate value of the pixel point at the center position of the corresponding single word.

[0038] In a possible implementation, the computing unit includes:

[0039] A third calculating subunit is used for calculating the coordinate values ​​of each pixel point at the position of each word in the coordinate system where the binary image containing the word position is located;

[0040] A fourth calculation subunit is used to calculate the average value of the coordinate values ​​of each pixel point as the coordinate value of the primary pixel point corresponding to the center position of the single word;

[0041] The first determination subunit is used to determine the coordinate values ​​of N pixel points in a preset area of ​​the center position of the corresponding single word according to the coordinate values ​​of the initially selected pixel points, as the pixel point coordinate values ​​of the center position of the corresponding single word; N is a positive integer greater than 1.

[0042] In a possible implementation, the matching unit includes:

[0043] A fifth calculation subunit, used for calculating the difference between the coordinate value of the pixel point at the center position of each word and the coordinate value of each pixel point at the position of each text line;

[0044] A matching subunit is used to, if the difference is lower than a preset difference threshold, use the pixel point in the text row corresponding to the difference as the pixel point that successfully matches the pixel point at the center position of the corresponding word, and use the identifier of the text row where the successfully matched pixel point is located as the identifier of the text row to which the corresponding word belongs.

[0045] In a possible implementation, the matching unit includes:

[0046] a sixth calculation subunit, configured to calculate the difference between the coordinate values ​​of the N pixel points and the coordinate value of each pixel point at the location of each text line;

[0047] A second determination subunit is used for, if the difference satisfies a preset difference threshold, taking N pixel points in the position of the text line corresponding to the difference as N pixel points that successfully match the N pixel points, and determining an identifier of the text line where the N pixel points that successfully match are located as a preliminary text line identifier to which the corresponding word belongs;

[0048] The seventh calculation subunit is used to calculate the majority of the initially selected text line identifiers as the text line identifier to which the corresponding single word belongs.

[0049] In a possible implementation manner, the device further includes:

[0050] A new creation unit is used to create a new target text line according to the position of the target word, and fill the target word into the target text line if an unsuccessful match occurs after matching the pixel point coordinate values ​​of the center position of the position of each word with the coordinate values ​​of each pixel point in the position of each text line.

[0051] In a possible implementation, the detection unit is specifically used to:

[0052] The target image is detected by a multi-granularity OCR detection model, and a binary image containing text line positions and a binary image containing single word positions are output.

[0053] The embodiment of the present application also provides an image detection device, including: a processor, a memory, and a system bus;

[0054] The processor and the memory are connected via the system bus;

[0055] The memory is used to store one or more programs, and the one or more programs include instructions. When the instructions are executed by the processor, the processor executes any one of the implementations of the above-mentioned image detection method.

[0056] An embodiment of the present application also provides a computer-readable storage medium, in which instructions are stored. When the instructions are executed on a terminal device, the terminal device executes any one of the implementations of the above-mentioned image detection method.

[0057] An image detection method, apparatus, storage medium and device provided in an embodiment of the present application first acquire a target image to be detected; then detect the target image to obtain a binary image including text line positions and a binary image including single word positions, then mark the text lines in the binary image including the text line positions to obtain identifiers corresponding to each text line; and determine the coordinate values ​​of each pixel point in the position of each text line, and calculate the pixel point coordinate values ​​of the center position of each single word in the binary image including the single word positions; then, match the pixel point coordinate values ​​of the center position of each single word position with the coordinate values ​​of each pixel point in the position of each text line, determine the text line identifier to which each single word belongs, and then fill each single word into the text line with the corresponding identifier, so as to achieve matching of the text line position and the single word position, and obtain the detection result of the text in the target image. It can be seen that since the embodiment of the present application matches and integrates the text line position information and the single word position information more accurately through pixel coordinate matching, the matching effect is improved, and since the calculation complexity is only the number of pixels at the position of the single word when performing pixel matching, compared with the existing detection method, the calculation complexity is greatly reduced, the calculation efficiency is improved, and the accuracy and detection efficiency of the text detection results in the target image are improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0059] Figure 1 A schematic diagram of the IOU calculation process provided in an embodiment of the present application;

[0060] Figure 2 A schematic diagram of a flow chart of an image detection method provided in an embodiment of the present application;

[0061] Figure 3An example diagram of a binary image containing text line positions provided in an embodiment of the present application;

[0062] Figure 4 An example diagram of a binary diagram including a single word position provided in an embodiment of the present application;

[0063] Figure 5 A schematic diagram of marking text lines in a binary image containing text line positions provided in an embodiment of the present application;

[0064] Figure 6 An example diagram of establishing a coordinate system based on a binary image containing text line positions provided in an embodiment of the present application;

[0065] Figure 7 An example diagram of establishing a coordinate system based on a binary image containing a single word position provided in an embodiment of the present application;

[0066] Figure 8 An example diagram of coordinate values ​​of 9 pixels within a 3*3 area extending around a pre-selected pixel provided in an embodiment of the present application;

[0067] Fig. 9 An overall example diagram of image detection provided by an embodiment of the present application;

[0068] Fig.10 A schematic diagram of the composition of an image detection device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0069] As the application scenarios of OCR recognition technology increase, the demand for multi-granular OCR detection is also gradually increasing. This is because most OCR application scenarios not only need to complete the correct recognition of characters, but also need to display the recognized characters in order according to the text line position to which they belong. For example, for the detection of images such as ID cards and driver's licenses, the text line position and character position need to be accurately matched and integrated, so that the text content in the image can be displayed to the user in the form of specific text line classification, which is convenient for users to proofread the text detection results.

[0070] At present, when performing multi-granular OCR detection, the method for matching and integrating text line positions and single word positions usually adopts the IOU grouping method. However, for dense text image scenes, this method requires a large number of such as Figure 1The calculation complexity of the IOU calculated by the box shown is relatively high, resulting in a slow matching speed. In addition, for irregular shapes such as concave polygons, considering the calculation speed, they will be converted into the minimum circumscribed rotated rectangle for calculation when calculating the IOU. For inclined, irregularly shaped, and containing position boxes, the calculated IOU results are not accurate enough, which will also lead to poor final matching results and failure to obtain more accurate text detection results. Therefore, how to improve the matching effect of text line position and single word position to further improve the accuracy and efficiency of detection results is a technical problem that needs to be solved urgently.

[0071] To solve the above-mentioned defects, the present application provides an image detection method, which first acquires a target image to be detected; and detects the target image to obtain a binary image containing text line positions and a binary image containing single word positions, and then marks the text lines in the binary image containing the text line positions to obtain the identifiers corresponding to each text line; and determines the coordinate values ​​of each pixel point in the position of each text line, and calculates the pixel point coordinate values ​​of the center position of each single word in the binary image containing the single word position; then, matches the pixel point coordinate values ​​of the center position of each single word position with the coordinate values ​​of each pixel point in the position of each text line, determines the text line identifier to which each single word belongs, and then fills each single word into the text line with the corresponding identifier, realizes the matching of the text line position and the single word position, and obtains the detection result of the text in the target image. It can be seen that since the embodiment of the present application matches and integrates the text line position information and the single word position information more accurately through pixel coordinate matching, the matching effect is improved, and since the calculation complexity is only the number of pixels at the position of the single word when performing pixel matching, compared with the existing detection method, the calculation complexity is greatly reduced, the calculation efficiency is improved, and the accuracy and detection efficiency of the text detection results in the target image are improved.

[0072] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0073] First embodiment

[0074] See also Figure 2 , is a flow chart of an image detection method provided in this embodiment, the method comprising the following steps:

[0075] S201: Acquire a target image to be detected; and detect the target image to obtain a binary image containing text line positions and a binary image containing single word positions.

[0076] In this embodiment, any image that uses this embodiment to perform text detection is defined as a target image. In addition, it should be noted that this embodiment does not limit the type of the target image. For example, the target image can be a color image composed of three primary colors of red (R), green (G), and blue (B), or a grayscale image.

[0077] Furthermore, this embodiment does not limit the method for acquiring the target image. The target image can be obtained through screenshot, photography, etc. according to actual needs. For example, an image sent by a friend in a chat APP can be saved as a target image, or an image containing text taken by a camera can be used as a target image.

[0078] Furthermore, after acquiring the target image, it can be detected using existing or future image detection technology to obtain a binary image containing text line positions and a binary image containing single word positions, and then by executing subsequent steps S202-S204, accurate detection of the target image can be achieved.

[0079] Specifically, an optional implementation method is to detect the target image through a multi-granularity OCR detection model and output a binary map containing text line positions and a binary map containing single word positions. For example, the DBNet multi-granularity model can be used to simultaneously perform text line detection and single word detection on the entire target image to obtain a binary map containing text line positions and a binary map containing single word positions.

[0080] Among them, a binary image refers to an image with only two gray levels in the image, that is, the gray value of any pixel in the image is 0 or 255, representing black and white respectively.

[0081] In this embodiment, after using the DBNet multi-granularity model and other detection algorithms to simultaneously perform text line detection and single word detection on the entire target image, in the obtained binary map containing the text line position, "white" indicates the position of the text line in the binary image, and "black" indicates other positions of the image, for example, Figure 3 The example of a binary image containing text line positions shown in the figure, where the white area represents the position of the text line in the target image, and the black area represents other positions in the target image except the text line. In the binary image containing single word positions obtained, "white" represents the position of the single word in the binary image, and "black" represents other positions in the image, for example, Figure 4The example figure of the binary image containing the position of a single word is shown, in which the white area represents the position of the single word in the target image, and the black area represents the other positions in the target image except the single word.

[0082] S202: Mark the text lines in the binary image containing the text line positions to obtain the identifiers corresponding to the text lines; and determine the coordinate values ​​of the pixels at the positions of the text lines.

[0083] In this embodiment, after the binary image containing the text line position corresponding to the target image is obtained through step S201, in order to be able to more accurately match and integrate the text line position information and the single word position information, it is further necessary to mark the text lines in the binary image containing the text line position to obtain the unique identifier corresponding to each text line. The specific marking method is not limited in this application, and it is only necessary to ensure that each text line has an identifier different from other text lines. For example, each text line in the binary image containing the text line position can be numbered to obtain a unique serial number corresponding to each text line.

[0084] Example: Based on the above example, assume that the binary image containing the text line position is as follows Figure 3 As shown in the figure, the text lines in the eight "white" positions in the figure can be further numbered, such as Figure 5 As shown, the text lines in each "white" position can be numbered 1, 2, 3, 4, 5, 6, 7, and 8 in a clockwise manner from top to bottom.

[0085] Furthermore, in order to be able to more accurately match and integrate the text line position information and the single word position information, it is also necessary to establish a coordinate system based on the binary image containing the text line position, and in the coordinate system, determine the coordinate values ​​of each pixel point in the position of each text line, in order to execute the subsequent step S204. It should be noted that the present application does not limit the specific method of establishing a coordinate system based on the binary image containing the text line position. It is only necessary to ensure that for the same target image, the coordinate system established based on the binary image containing the text line position and the coordinate system established based on the binary image containing the single word position are the same coordinate system. For example, the coordinate system established based on the binary image containing the text line position can be as follows: Figure 6 As shown, the coordinate system is established with the upper left corner of the image as the coordinate origin, the upper edge width of the image as the x-axis, and the left edge length of the image as the y-axis. The coordinate values ​​of each pixel point in the eight text lines numbered 1, 2, 3, 4, 5, 6, 7, and 8 can then be determined. The specific determination process will not be repeated here.

[0086] S203: Calculate the pixel coordinate value of the center position of each word in the binary image containing the word position.

[0087] In this embodiment, after the binary image containing the single word position corresponding to the target image is obtained through step S201, in order to be able to more accurately match and integrate the text line position information and the single word position information, it is further necessary to calculate the pixel point coordinate values ​​of the center position of each single word in the binary image containing the single word position, so as to execute the subsequent step S204 and match it with the coordinate values ​​of each pixel point in the position of each text line determined by step S202.

[0088] Among them, an optional implementation manner is that the implementation process of this step S203 may specifically include the following steps A1-A2:

[0089] Step A1: In the coordinate system of the binary image containing the word position, the coordinate values ​​of each pixel point at the position of each word are calculated.

[0090] In this implementation, in order to be able to more accurately match and integrate text line position information and individual word position information, it is also necessary to establish a coordinate system based on the binary graph comprising the individual word position, and to ensure that for the same target image, the coordinate system established based on the binary graph comprising the text line position and the coordinate system established based on the binary graph comprising the individual word position are the same coordinate system, but the application does not limit the specific mode of establishing a coordinate system based on the binary graph comprising the individual word position. And, further, in the binary graph coordinate system comprising the individual word position, each pixel coordinate value of each individual word position can be calculated to perform subsequent step A2.

[0091] Example: Based on the above example, the binary image containing the text line position is constructed Figure 6 Similar to the coordinate system of , the coordinate system based on the binary image containing the position of the single word can be established as follows Figure 7 As shown, the coordinate system is also established with the upper left corner of the image as the coordinate origin, the upper edge width of the image as the x-axis, and the left edge length of the image as the y-axis. The coordinate values ​​of each pixel point in the white box where each word in the image is located can then be determined. The specific determination process will not be repeated here.

[0092] Step A2: Calculate the average value of the coordinate values ​​of each pixel point as the coordinate value of the pixel point at the center position of the corresponding word.

[0093] After calculating the pixel coordinate values ​​of each word position in the coordinate system of the binary image containing the word position through step A1, the pixel coordinate values ​​can be further averaged to determine the average value of the pixel coordinate values, and it is defined as (cx, cy) as the pixel coordinate value of the center position of the corresponding word to execute the subsequent step S204.

[0094] Another optional implementation manner is that the implementation process of the above step S203 may specifically include the following steps B1-B3:

[0095] Step B1: In the coordinate system of the binary image containing the word positions, the coordinate values ​​of each pixel point at the position of each word are calculated.

[0096] Step B2: Calculate the average value of the coordinate values ​​of each pixel point as the coordinate value of the primary pixel point corresponding to the center position of the single word.

[0097] It should be noted that the implementation process of steps B1-B2 is consistent with the above steps A1-A2, and will not be repeated here. The difference is that in this implementation, the average value (cx, cy) of the calculated coordinate values ​​of each pixel point is used as the coordinate value of the preliminary selected pixel point at the center position of the corresponding single word to execute the subsequent B3.

[0098] Step B3: According to the coordinate values ​​of the initially selected pixel points, the coordinate values ​​of N pixel points in the preset area corresponding to the center position of the single word are determined as the coordinate values ​​of the pixel points at the center position of the corresponding single word; wherein N is a positive integer greater than 1.

[0099] In this implementation, in order to improve the accuracy of the pixel coordinate values ​​of the center position of each word, after obtaining the coordinate value (cx, cy) of the preliminarily selected pixel at the center position of the word, the range can be further expanded at the center position of the word with the preliminarily selected pixel (cx, cy) as the center to reach the preset area, and the coordinate values ​​of N (N is a positive integer greater than 1) pixels contained in the preset area are determined, and then the coordinate values ​​of these N pixels are used together as the pixel coordinate values ​​of the center position of the corresponding word to perform the subsequent step S204.

[0100] The specific setting method of the preset area can be set according to the actual situation, and the embodiment of the present application does not limit this. For example, the preset area can be set to: the range of a 3*3 area expanded with the primary pixel point (cx, cy) as the center, so as to determine 9 pixel points therefrom, such as Figure 8 As shown, the coordinate values ​​of these 9 pixel points are (cx-1,cy-1), (cx,cy-1), (cx+1,cy-1), (cx-1,cy), (cx,cy), (cx+1,cy), (cx-1,cy+1), (cx,cy+1), and (cx+1,cy+1), respectively. The coordinate values ​​of these 9 pixel points can then be used as the pixel coordinate values ​​of the center position of the corresponding word.

[0101] Alternatively, the preset area can also be set to: the range of the cross-shaped area expanded with the initially selected pixel point (cx, cy) as the center, to determine 5 pixel points therefrom, the coordinate values ​​of these 5 pixel points are (cx, cy-1), (cx-1, cy), (cx, cy), (cx+1, cy), (cx, cy+1), and then the coordinate values ​​of these 5 pixel points can be used as the pixel point coordinate values ​​of the center position of the corresponding single word.

[0102] S204: Match the coordinate values ​​of the pixel points at the center of the positions where each single word is located with the coordinate values ​​of each pixel point at the positions where each text line is located, and determine the text line identifier to which each single word belongs.

[0103] In this embodiment, after determining the coordinate values ​​of each pixel point in each text line through step S202 and calculating the pixel point coordinate values ​​of the center position of each word through step S203, the pixel point coordinate values ​​of the center position of each word can be further matched one by one with the coordinate values ​​of each pixel point in each text line position, so as to determine the text line identifier to which each word belongs based on the matching result, so as to execute the subsequent step S205.

[0104] Specifically, an optional implementation method is that after the pixel coordinate values ​​(cx, cy) of the center position of each single word are calculated through the above steps A1-A2, the difference between the pixel coordinate values ​​(cx, cy) of the center position of each single word and the coordinate values ​​of each pixel in the position of each text line can be further calculated. If it is determined that the difference between the pixel coordinate value (cx, cy) of the center position of a certain single word and the coordinate value of a certain pixel in the position of a certain text line is lower than a preset difference threshold (the specific value can be limited according to the actual situation), the pixel in the position of the text line corresponding to the difference can be used as a pixel that successfully matches the pixel at the center position of the corresponding single word, and the identifier of the text line where the pixel that successfully matches is located is used as the identifier of the text line to which the corresponding single word belongs. It can be understood that the smaller the difference between the coordinate values ​​of two successfully matched pixel points, the closer the two are, and then the probability that the single word corresponding to the difference belongs to the corresponding text line is higher, and the matching effect is better.

[0105] Another optional implementation method is that after calculating the coordinate values ​​of the N pixel points of the center position of each single word through the above steps B1-B2, the difference between the coordinate values ​​of the N pixel points and the coordinate values ​​of each pixel point in the position of each text line can be further calculated. If it is determined that the difference between the coordinate values ​​of the N pixel points of the center position of a certain single word and the coordinate values ​​of the N pixel points in the text line is lower than the preset difference threshold (the specific value can be limited according to the actual situation), the pixel points corresponding to the N differences can be used as the pixel points that successfully match the N pixel points in the above single word, and the identifier of the text line where the N successfully matched pixel points are located is determined as the preliminary text line identifier to which the corresponding single word belongs. Further, the majority of the preliminary text line identifiers where the N successfully matched pixel points are located can be calculated, and the identifier corresponding to the majority can be used as the text line identifier to which the corresponding single word belongs.

[0106] Example: Based on the above example, assume that the eight text lines in the binary image containing the text line positions are marked as follows Figure 5 The binary graphs of the numbers 1, 2, 3, 4, 5, 6, 7, and 8 shown in FIG. 1 and FIG. 8 including the positions of the individual words are as follows: Figure 4 The figure shown in Figure 4 Take the single word corresponding to a white box in as an example, assuming that the coordinate values ​​of the 9 pixel points at the center position of the single word are determined by steps B1-B3 as follows: (cx-1,cy-1), (cx,cy-1), (cx+1,cy-1), (cx-1,cy), (cx,cy), (cx+1,cy), (cx-1,cy+1), (cx,cy+1), (cx+1,cy+1), and the preliminary text line identifiers of the pixel points that successfully match these 9 pixel points are calculated to be 1, 1, 1, 1, 1, 2, 2, 2. Then it can be calculated that the majority of these 9 text line identifiers is number 1, and then number 1 can be used as the text line identifier to which the corresponding single word belongs.

[0107] S205: Fill each single word into the corresponding marked text line to achieve the matching between the position of the text line and the position of the single word, and obtain the detection result of the text in the target image.

[0108] In this embodiment, after determining the preliminary text line identifier to which each word belongs through step S204, each word can be further filled into the text line of the corresponding identifier to achieve matching between the text line position and the word position, and obtain the detection result of the text in the target image.

[0109] Among them, an optional implementation method is that if in the above step S204, after the pixel point coordinate values ​​of the center position of each single word are matched with the coordinate values ​​of each pixel point in the position of each text line, there is a single word that is not successfully matched (here defined as a target single word), then in order to improve the accuracy of text detection, a new target text line can be further created according to the position of the target single word, and the target single word can be filled into the target text line to ensure the integrity of the text detection result.

[0110] In this way, by executing the above steps S201-S205 to perform pixel coordinate matching, the text line position information and the single word position information can be matched and integrated more accurately, so that the single words belonging to the same text line are divided into the same text line position, thereby improving the accuracy of the text detection results.

[0111] For example: Fig. 9 As shown, it shows an overall example diagram of image detection provided by an embodiment of the present application. In the figure, for the single words "big", "force", "1", "2", "shake", "sound", "b", "y", "t", "e", "fire", and "mountain" detected in the target image, through the above steps S201-S205, the pixel point coordinate values ​​of the center position of each single word can be determined (i.e. Fig. 9 After the dot position in each word box in the figure above is determined, the pixel coordinate value of the center position of each word is further matched with the coordinate value of each pixel point in the position of each text line, the text line identifier to which each word belongs is determined, and each word is filled into the text line with the corresponding identifier to obtain the detection result of the text in the target image, that is, Fig. 9 The contents of the rectangular boxes where the text lines are located: "Dali", "12", "Tik Tok", "byte", "Volcano".

[0112] In summary, the present embodiment provides an image detection method, which first acquires a target image to be detected; detects the target image to obtain a binary image including text line positions and a binary image including word positions, then marks the text lines in the binary image including text line positions to obtain identifiers corresponding to each text line; determines the coordinate values ​​of each pixel point in the position of each text line, and calculates the pixel point coordinate values ​​of the center position of each word in the binary image including word positions; then, matches the pixel point coordinate values ​​of the center position of each word position with the coordinate values ​​of each pixel point in the position of each text line, determines the text line identifier to which each word belongs, and then fills each word into the text line with the corresponding identifier, realizes the matching of the text line position and the word position, and obtains the detection result of the text in the target image. It can be seen that since the embodiment of the present application matches and integrates the text line position information and the single word position information more accurately through pixel coordinate matching, the matching effect is improved, and since the calculation complexity is only the number of pixels at the position of the single word when performing pixel matching, compared with the existing detection method, the calculation complexity is greatly reduced, the calculation efficiency is improved, and the accuracy and detection efficiency of the text detection results in the target image are improved.

[0113] Second embodiment

[0114] This embodiment will introduce an image detection device. For related content, please refer to the above method embodiment.

[0115] See also Fig.10 , is a schematic diagram of the composition of an image detection device provided in this embodiment, the device 1000 includes:

[0116] The detection unit 1001 is used to obtain a target image to be detected; and detect the target image to obtain a binary image containing text line positions and a binary image containing single word positions;

[0117] The marking unit 1002 is used to mark the text lines in the binary image containing the text line positions, obtain the identifier corresponding to each text line; and determine the coordinate value of each pixel point at the position of each text line;

[0118] A calculation unit 1003 is used to calculate the pixel coordinate values ​​of the center position of each individual word position in the binary image containing the individual word position;

[0119] A matching unit 1004 is used to match the pixel point coordinate value of the center position of each word with the coordinate value of each pixel point in the position of each text line to determine the text line identifier to which each word belongs;

[0120] The filling unit 1005 is used to fill each single word into the corresponding marked text line to achieve the matching of the position of the text line and the position of the single word, and obtain the detection result of the text in the target image.

[0121] In an implementation of this embodiment, the calculation unit 1003 includes:

[0122] A first calculating subunit is used to calculate the coordinate values ​​of each pixel point at the position of each word in the coordinate system where the binary image containing the word position is located;

[0123] The second calculation subunit is used to calculate the average value of the coordinate values ​​of each pixel point as the coordinate value of the pixel point at the center position of the corresponding single word.

[0124] In an implementation of this embodiment, the calculation unit 1003 includes:

[0125] A third calculating subunit is used for calculating the coordinate values ​​of each pixel point at the position of each word in the coordinate system where the binary image containing the word position is located;

[0126] A fourth calculation subunit is used to calculate the average value of the coordinate values ​​of each pixel point as the coordinate value of the primary pixel point corresponding to the center position of the single word;

[0127] The first determination subunit is used to determine the coordinate values ​​of N pixel points in a preset area of ​​the center position of the corresponding single word according to the coordinate values ​​of the initially selected pixel points, as the pixel point coordinate values ​​of the center position of the corresponding single word; N is a positive integer greater than 1.

[0128] In an implementation of this embodiment, the matching unit 1004 includes:

[0129] A fifth calculation subunit, used for calculating the difference between the coordinate value of the pixel point at the center position of each word and the coordinate value of each pixel point at the position of each text line;

[0130] A matching subunit is used to, if the difference is lower than a preset difference threshold, use the pixel point in the text row corresponding to the difference as the pixel point that successfully matches the pixel point at the center position of the corresponding word, and use the identifier of the text row where the successfully matched pixel point is located as the identifier of the text row to which the corresponding word belongs.

[0131] In an implementation of this embodiment, the matching unit 1004 includes:

[0132] a sixth calculation subunit, configured to calculate the difference between the coordinate values ​​of the N pixel points and the coordinate value of each pixel point at the location of each text line;

[0133] A second determination subunit is used for, if the difference satisfies a preset difference threshold, taking N pixel points in the position of the text line corresponding to the difference as N pixel points that successfully match the N pixel points, and determining an identifier of the text line where the N pixel points that successfully match are located as a preliminary text line identifier to which the corresponding word belongs;

[0134] The seventh calculation subunit is used to calculate the majority of the initially selected text line identifiers as the text line identifier to which the corresponding single word belongs.

[0135] In an implementation of this embodiment, the device further includes:

[0136] A new creation unit is used to create a new target text line according to the position of the target word, and fill the target word into the target text line if an unsuccessful match occurs after matching the pixel point coordinate values ​​of the center position of the position of each word with the coordinate values ​​of each pixel point in the position of each text line.

[0137] In an implementation of this embodiment, the detection unit 1001 is specifically used for:

[0138] The target image is detected by a multi-granularity OCR detection model, and a binary image containing text line positions and a binary image containing single word positions are output.

[0139] Furthermore, an embodiment of the present application also provides an image detection device, including: a processor, a memory, and a system bus;

[0140] The processor and the memory are connected via the system bus;

[0141] The memory is used to store one or more programs, and the one or more programs include instructions, and when the instructions are executed by the processor, the processor executes any one of the implementation methods of the above-mentioned image detection method.

[0142] Furthermore, an embodiment of the present application also provides a computer-readable storage medium, in which instructions are stored. When the instructions are executed on a terminal device, the terminal device executes any implementation method of the above-mentioned image detection method.

[0143] It can be known from the description of the above implementation mode that those skilled in the art can clearly understand that all or part of the steps in the above-mentioned embodiment method can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product can be stored in a storage medium such as ROM / RAM, a disk, an optical disk, etc., including several instructions for enabling a computer device (which can be a personal computer, a server, or a network communication device such as a media gateway, etc.) to execute the methods described in the various embodiments of the present application or certain parts of the embodiments.

[0144] It should be noted that the various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the various embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description.

[0145] It should also be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0146] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An image detection method, characterized in that: The method comprises: Acquire a target image to be detected; and detect the target image to obtain a binary image containing text line positions and a binary image containing single word positions; Marking the text lines in the binary image containing the text line positions to obtain the identifiers corresponding to the text lines; and determining the coordinate values ​​of the pixels at the positions of the text lines; Calculate the pixel coordinate values ​​of the center position of each individual word position in the binary image containing the individual word position; Matching the pixel coordinate values ​​of the center position of each word with the coordinate values ​​of each pixel in the position of each text line to determine the text line identifier to which each word belongs; The individual words are filled into the text lines with corresponding marks, so as to match the positions of the text lines and the individual words, and obtain the detection results of the words in the target image.

2. The method according to claim 1, characterized in that The step of calculating the pixel coordinate values ​​of the center position of each word in the binary image containing the word position includes: In the coordinate system where the binary image containing the word positions is located, the coordinate values ​​of each pixel point at the position of each word are calculated; The average value of the coordinate values ​​of each pixel point is calculated as the coordinate value of the pixel point at the center position of the corresponding word.

3. The method according to claim 1, characterized in that The step of calculating the pixel coordinate values ​​of the center position of each word in the binary image containing the word position includes: In the coordinate system where the binary image containing the word positions is located, the coordinate values ​​of each pixel point at the position of each word are calculated; Calculate the average value of the coordinate values ​​of each pixel point as the coordinate value of the primary selected pixel point at the center position of the corresponding word; According to the coordinate values ​​of the initially selected pixel points, the coordinate values ​​of N pixel points in the preset area of ​​the center position of the corresponding single word are determined as the coordinate values ​​of the pixel points at the center position of the corresponding single word; wherein N is a positive integer greater than 1.

4. The method according to claim 2, characterized in that: The step of matching the pixel coordinate values ​​of the center positions of the positions of the individual words with the coordinate values ​​of the individual pixels in the positions of the individual text lines to determine the text line identifiers to which the individual words belong includes: Calculate the difference between the coordinate value of the pixel point at the center position of each word and the coordinate value of each pixel point at the position of each text line; If the difference is lower than a preset difference threshold, the pixel point in the text row corresponding to the difference is used as the pixel point that successfully matches the pixel point at the center position of the corresponding word, and the identifier of the text row where the successfully matched pixel point is located is used as the text row identifier to which the corresponding word belongs.

5. The method according to claim 3, characterized in that: The step of matching the pixel coordinate values ​​of the center positions of the positions of the individual words with the coordinate values ​​of the individual pixels in the positions of the individual text lines to determine the text line identifiers to which the individual words belong includes: Calculate the difference between the coordinate values ​​of the N pixel points and the coordinate value of each pixel point at the location of each text line; If the difference value satisfies a preset difference value threshold, the N pixel points in the position of the text line corresponding to the difference value are used as the N pixel points that successfully match the N pixel points, and the identifier of the text line where the N pixel points that successfully match are located is determined as the identifier of the preliminary text line to which the corresponding single word belongs; The majority of the initially selected text line identifiers is calculated as the text line identifier to which the corresponding single word belongs.

6. The method according to claim 1, characterized in that The method further comprises: If, after matching the pixel coordinate values ​​of the center positions of the respective words with the pixel coordinate values ​​of the respective pixel points in the respective text lines, an unmatched target word appears, a new target text line is created according to the position of the target word, and the target word is filled into the target text line.

7. The method according to any one of claims 1 to 6, characterized in that: The target image is detected to obtain a binary image containing text line positions and a binary image containing single word positions, including: The target image is detected by a multi-granularity OCR detection model, and a binary image containing text line positions and a binary image containing single word positions are output.

8. An image detection device, characterized in that: The device comprises: A detection unit is used to obtain a target image to be detected; and detect the target image to obtain a binary image containing text line positions and a binary image containing single word positions; A marking unit, used to mark the text lines in the binary image containing the text line positions, obtain the identifier corresponding to each text line; and determine the coordinate value of each pixel point at the position of each text line; A computing unit, for calculating the pixel coordinate values ​​of the center position of each individual word position in the binary image containing the individual word position; A matching unit, used for matching the pixel point coordinate value of the center position of each word with the coordinate value of each pixel point in the position of each text line, to determine the text line identifier to which each word belongs; The filling unit is used to fill each single word into the text line with the corresponding mark, so as to match the position of the text line with the position of the single word and obtain the detection result of the text in the target image.

9. An image detection device, characterized in that: include: Processor, memory, system bus; The processor and the memory are connected via the system bus; The memory is used to store one or more programs, wherein the one or more programs include instructions, and when the instructions are executed by the processor, the processor executes the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, and when the instructions are executed on a terminal device, the terminal device executes the method according to any one of claims 1 to 7.