Text detection and recognition method, device, equipment and computer-readable storage medium

Abstract

The present invention discloses a text detection and recognition method, device, equipment and computer-readable storage medium, and belongs to the technical field of text recognition. The present invention obtains a text image, performs text line detection and single character detection on the text image, outputs the text line detection result and the single character detection result, and then performs text content recognition based on the text line detection result and the single character detection result, and generates a recognition result. The present invention performs two fine-grained text detections, text line and single character, in one text detection process, and is easy to handle the sorting and recognition of text with tilt angles, multi-directional text and curved text. Compared with only recognizing single characters, it has a higher character recognition accuracy and can improve the accuracy of text recognition.

Description

Technical Field

[0001] The present invention relates to the field of text recognition technology, and in particular to a text detection and recognition method, device, equipment and computer-readable storage medium. Background Art

[0002] Text detection and recognition has broad application prospects in the intelligent review of document images, document comparison, and image text retrieval. Intelligent review, document comparison, and image text retrieval all require that the text detection and recognition model can accurately detect the single characters in the image and recognize its text content, and finally organize the recognized multiple single characters into text lines as needed, and finally restore the document in order to determine the specific location of the text in the image document.

[0003] In the related art, when performing single-character text detection and recognition, a common text detection algorithm is usually used, which is trained with character-level annotation information to perform single-character text detection. Then, a series of single characters detected by the text detection algorithm are recognized, and the characters are sorted by line to restore the document. This solution uses a divide-and-conquer method to separate text detection and recognition, which is easy to implement, but it is easy to cause the disadvantages of inaccurate single-character detection and recognition and difficulty in sorting text lines. Summary of the invention

[0004] The main purpose of the present invention is to provide a text detection and recognition method, device, equipment and computer-readable storage medium, aiming to solve the problems of poor text detection and recognition accuracy and difficulty in text line sorting in the prior art.

[0005] To achieve the above object, the present invention provides a text detection and recognition method, which comprises the following steps:

[0006] Get text image;

[0007] Performing text line detection and single character detection on the text image, and outputting text line detection results and single character detection results;

[0008] Based on the text line detection result and the single character detection result, text content recognition is performed and a recognition result is generated.

[0009] Optionally, after the step of performing text line detection and single character detection on the text image and outputting the text line detection result and the single character detection result, the method further comprises:

[0010] According to the text line detection result and the single character detection result, grouping the single character detection frames in the single character detection result by text line;

[0011] According to the text line direction in the text line detection result, the single character detection frames are sorted according to the text line direction within the grouping group to generate a corrected single character detection result.

[0012] Optionally, the step of sorting the single character detection frames according to the text line direction in the text line detection result within the grouping group to generate a corrected single character detection result includes:

[0013] According to the text line direction in the text line detection result, sorting the single character detection frames in the grouping group according to the text line direction;

[0014] The size and direction of the sorted single-character detection frame are detected, and the size and direction are normalized row by row to generate a corrected single-character detection result.

[0015] Optionally, the step of sorting the single character detection frames according to the text line direction in the text line detection result within the grouping group to generate a corrected single character detection result includes:

[0016] confirming whether the text line detection result is curved text;

[0017] If yes, modify the size of the text line detection result to generate a reduced text line direction;

[0018] According to the text line direction, the single character detection frames are sorted in the grouping group according to the text line direction to generate a corrected single character detection result.

[0019] Optionally, the step of performing text content recognition based on the text line detection result and the single character detection result and generating a recognition result includes:

[0020] Performing text content recognition on the text line detection result to generate a text line character recognition result;

[0021] Determine whether the text line character recognition result is aligned with the position of the single character detection box in the single character detection result;

[0022] If so, a recognition result is generated based on the text line character recognition result.

[0023] Optionally, after the step of determining whether the text line character recognition result is aligned with the position of the single character detection box in the single character detection result, the method includes:

[0024] If not, performing text recognition based on the single character detection result to generate a single character recognition result;

[0025] Generate a single-character string according to the single-character detection frame and the single-character recognition result;

[0026] Calculating the edit distance between the single-character string and the corresponding text line string in the text line character detection result, and obtaining a corresponding edit operation;

[0027] The editing operation is performed to align the text line recognition result with the position of the single character detection box to generate a recognition result.

[0028] Optionally, the step of performing text line detection and single character detection on the text image and outputting the text line detection result and the single character detection result comprises:

[0029] Through the dual-branch text detection model, text line detection and single character detection are performed on the text image respectively, and a text line probability map and a threshold map, a single character probability map and a threshold map are predicted, and differentiable binarization is performed to obtain a text line binarization probability map and a single character binarization probability map respectively;

[0030] The text line binarization probability map and the single character binarization probability map are post-processed to obtain the text line detection result and the single character detection result.

[0031] In addition, to achieve the above-mentioned purpose, the present invention also provides a text detection and recognition device, the device comprising:

[0032] An acquisition module, used for acquiring text images;

[0033] A text detection module, used to perform text line detection and single character detection on the text image, and output text line detection results and single character detection results;

[0034] The text recognition module is used to perform text content recognition based on the text line detection result and the single character detection result, and generate a recognition result.

[0035] Optionally, the device further comprises:

[0036] A single character detection frame sorting correction module is used to group the single character detection frames in the single character detection result by text line according to the text line detection result and the single character detection result;

[0037] According to the text line direction in the text line detection result, the single character detection frames are sorted according to the text line direction within the grouping group to generate a corrected single character detection result.

[0038] Optionally, the single-character detection frame sorting correction module is further used for:

[0039] According to the text line direction in the text line detection result, sorting the single character detection frames in the grouping group according to the text line direction;

[0040] The size and direction of the sorted single-character detection frame are detected, and the size and direction are normalized row by row to generate a corrected single-character detection result.

[0041] Optionally, the single-character detection frame sorting correction module is further used for:

[0042] confirming whether the text line detection result is curved text;

[0043] If yes, modify the size of the text line detection result to generate a reduced text line direction;

[0044] According to the text line direction, the single character detection frames are sorted in the grouping group according to the text line direction to generate a corrected single character detection result.

[0045] Optionally, the device further comprises:

[0046] A character alignment module, used for performing text content recognition on the text line detection result to generate a text line character recognition result;

[0047] Determine whether the text line character recognition result is aligned with the position of the single character detection box in the single character detection result;

[0048] If so, a recognition result is generated based on the text line character recognition result.

[0049] Optionally, the character alignment module is further used for:

[0050] If not, performing text recognition based on the single character detection result to generate a single character recognition result;

[0051] Generate a single-character string according to the single-character detection frame and the single-character recognition result;

[0052] Calculating the edit distance between the single-character string and the corresponding text line string in the text line character detection result, and obtaining a corresponding edit operation;

[0053] The editing operation is performed to align the text line recognition result with the position of the single character detection box to generate a recognition result.

[0054] Optionally, the text detection module is further used for:

[0055] Through the dual-branch text detection model, text line detection and single character detection are performed on the text image respectively, and a text line probability map and a threshold map, a single character probability map and a threshold map are predicted, and differentiable binarization is performed to obtain a text line binarization probability map and a single character binarization probability map respectively;

[0056] The text line binarization probability map and the single character binarization probability map are post-processed to obtain the text line detection result and the single character detection result.

[0057] In addition, to achieve the above-mentioned purpose, the present invention also provides a text detection and recognition device, which includes: a memory, a processor, and a text detection and recognition program stored in the memory and executable on the processor, wherein the text detection and recognition program is configured to implement the steps of the text detection and recognition method as described above.

[0058] In addition, to achieve the above-mentioned purpose, the present invention also provides a computer-readable storage medium, on which a text detection and recognition program is stored, and when the text detection and recognition program is executed by a processor, the steps of the text detection and recognition method as described above are implemented.

[0059] The text detection and recognition method, device, equipment and computer-readable storage medium proposed in the embodiments of the present invention obtain a text image, perform text line detection and single character detection on the text image, output the text line detection result and the single character detection result, and then perform text content recognition based on the text line detection result and the single character detection result, and generate a recognition result. The present invention performs two fine-grained text detections, text line and single character, in one text detection process, and can easily handle the sorting and recognition of text with tilt angles, multi-directional text and curved text. Compared with recognizing only single characters, it has a higher character recognition accuracy and can improve the accuracy of text detection and recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 It is a structural diagram of a text detection and recognition device in a hardware operating environment involved in an embodiment of the present invention;

[0061] Figure 2 It is a flowchart of the first embodiment of the text detection and recognition method of the present invention;

[0062] Figure 3 A schematic diagram of a flow chart of sorting single character detection frames in an embodiment of a text detection and recognition method of the present invention;

[0063] Figure 4 This is a detailed flow chart of step S30 in an embodiment of the text detection and recognition method of the present invention;

[0064] Figure 5A schematic diagram of a character alignment process based on edit distance in an embodiment of a text detection and recognition method of the present invention;

[0065] Figure 6 A flowchart of single-character text detection and recognition in an embodiment of a text detection and recognition method of the present invention;

[0066] Figure 7 This is a detailed flow chart of step S20 in an embodiment of the text detection and recognition method of the present invention;

[0067] Figure 8 It is a structural diagram of a text detection model in an embodiment of a text detection and recognition method of the present invention;

[0068] Fig. 9 It is a schematic diagram of functional modules of an embodiment of a text detection and recognition device of the present invention.

[0069] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0070] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0071] Reference Figure 1 , Figure 1 The figure is a schematic diagram of the structure of a text detection and recognition device in the hardware operating environment involved in the embodiment of the present invention.

[0072] like Figure 1 As shown, the text detection and recognition device may include: a processor 1001, such as a central processing unit (CPU), a graphics processing unit (GPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Wherein, the communication bus 1002 is used to realize the connection communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a wireless fidelity (WIreless-FIdelity, WI-FI) interface). The memory 1005 may be a high-speed random access memory (Random Access Memory, RAM) memory, or a stable non-volatile memory (Non-Volatile Memory, NVM), such as a disk memory. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0073] Those skilled in the art will understand that Figure 1 The structure shown in the figure does not constitute a limitation on the text detection and recognition device, and may include more or less components than those shown in the figure, or combine certain components, or arrange the components differently.

[0074] like Figure 1 As shown, the memory 1005 as a computer-readable storage medium may include an operating system, a data storage module, a network communication module, a user interface module, and a text detection and recognition program.

[0075] exist Figure 1 In the text detection and recognition device shown, the network interface 1004 is mainly used for data communication with other devices; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and the memory 1005 in the text detection and recognition device of the present invention can be set in the text detection and recognition device, and the text detection and recognition device calls the text detection and recognition program stored in the memory 1005 through the processor 1001, and executes the text detection and recognition method provided by the embodiment of the present invention.

[0076] The embodiment of the present invention provides a method for detecting and recognizing text. Figure 2 , Figure 2 Schematic diagram of the flow of the first embodiment of the text detection and recognition method of the present invention.

[0077] In this embodiment, the text detection and recognition method includes:

[0078] Step S10, obtaining a text image;

[0079] Step S20, performing text line detection and single character detection on the text image, and outputting text line detection results and single character detection results;

[0080] Step S30, performing text content recognition based on the text line detection result and the single character detection result, and generating a recognition result.

[0081] The text detection and recognition method provided in this embodiment can be used in application scenarios such as intelligent auditing and recording of document images, document comparison, and image text retrieval. Existing single-character text detection and recognition technical solutions mainly include two schemes: ordinary text detection model + text recognition model, and single-character text detection model + character image classification. For the scheme of ordinary text detection model + text recognition model, it is difficult to accurately detect the position of a single character in an image, especially for half-width characters. For characters of the same font size, the character width of half-width characters is relatively narrow. In font design, its width is generally half of that of full-width characters. This is mainly reflected in English characters and symbols. Such as commas, semicolons, colons in Chinese and English. Due to the small character width, it is easy to cause missed detection or detection together with other characters. For full-width characters, due to their larger width, the probability of this situation is relatively low. Full-width characters are more likely to have missed detected characters, generally characters with fewer strokes, such as "一".

[0082] And when there is a large inclination in the text line of the document image or the text line has a curved layout, it is difficult to sort the characters in the text line by line, and the recognition accuracy of a single character is low. For the scheme of single-character text detection algorithm + character image classification, there are also problems of inaccurate detection of the position of a single character and difficulty in sorting the text line. And when processing Chinese documents, since the number of characters to be classified increases sharply compared with English documents, the single-character recognition is inaccurate.

[0083] Therefore, in order to solve the problem that the existing single-character text detection and recognition technology has poor single-character detection accuracy in specific scenarios and is prone to missed detection, misdetection and other situations. A character text detection and recognition method is proposed.

[0084] The following will explain each step in detail:

[0085] Step S10, obtain a text image;

[0086] In one embodiment, first, a text image needs to be obtained. Specifically, according to the specific application scenario, obtain the image to be detected input by the user, which can be various forms of images such as scanned images, photos, screenshots, etc. The format, size, and color of the text image are not limited. In order to improve the accuracy of subsequent image text detection, the obtained original image can be preprocessed, such as decolorization processing, resolution processing, etc., to obtain a text image.

[0087] Step S20, perform text line detection and single-character detection on the text image, and output the text line detection result and the single-character detection result;

[0088] In one embodiment, text line detection and single character detection are performed on a text image, and the corresponding text line detection results and single character detection results are output. In the related art, a common text detection algorithm is used to train with character-level annotation information to perform single character text detection. Then a series of single characters detected by the text detection algorithm are recognized, and the characters are sorted by line to restore the document. And the character area of ​​a single character is much smaller than the character area of ​​the entire text line. As a result, the detection accuracy of such text line detection algorithms for single characters is poor, and missed detection and wrong detection often occur, especially for the detection of half-width characters. Therefore, during the detection process, text line detection and single character detection are performed on the text image, and the position information of the two types of text lines and single characters in the image is output, and the text line detection results are used as a comparison of the single character detection results, so as to improve the integrity of the recognition results and solve the problem of missed detection that usually occurs in single character detection. Among them, the text line detection results and the single character detection results can be a text line text detection box and a single character text detection box.

[0089] Step S30, performing text content recognition based on the text line detection result and the single character detection result, and generating a recognition result.

[0090] In one embodiment, based on the text line detection results and single character detection results obtained by detection, text content recognition is further performed to generate recognition results. Specifically, the text line detection results (regional images) detected by the text detection module are recognized and the text content is output. At the same time, the single character regional images detected by the text detection module are recognized as needed, the text content is output, and the text line recognition results are matched with the single character recognition results. Since this solution adopts the line text recognition method, it has richer semantic features than single character recognition, and the recognition of text is more accurate.

[0091] This embodiment obtains a text image, performs text line detection and single character detection on the text image, outputs the text line detection result and the single character detection result, and then performs text content recognition based on the text line detection result and the single character detection result, and generates a recognition result. The present invention performs text detection of two different granularities, text line and single character, in one text detection process, and is easy to handle the sorting and recognition of text with tilt angles, multi-directional texts and curved texts. Compared with recognizing only single characters, it has a higher character recognition accuracy and can improve the accuracy of text detection and recognition.

[0092] Furthermore, based on the first embodiment of the text detection and recognition method of the present invention, a second embodiment of the text detection and recognition method of the present invention is proposed.

[0093] Reference Figure 3 , Figure 3The present invention is a flowchart of sorting single character detection frames in an embodiment of the text detection and recognition method. In a second embodiment, after performing text line detection and single character detection on the text image and outputting the text line detection result and the single character detection result, the method further includes:

[0094] Step S21, grouping the single character detection frames in the single character detection results by text line according to the text line detection results and the single character detection results;

[0095] Step S22, according to the text line direction in the text line detection result, sorting the single character detection frames in the grouping group according to the text line direction to generate a corrected single character detection result.

[0096] In one embodiment, based on the text line detection results and the single character detection results, the single character detection frames in the single character detection results are grouped by text line, and based on the text line direction in the text line detection results, the single character detection frames are sorted by the text line direction within the group to generate a revised single character detection result. It is understandable that since there may be text lines with large inclination angles or text lines with multiple text directions in the image, the single character frames detected by the text detection model may be of different sizes and directions, and each single character frame is independent and has not yet formed a row. It is difficult to sort the single character detection frames by row only. This solution combines the text line detection results with the single character detection results, first groups the single character detection frames by text line, and then sorts the single character detection frames by text line direction within the group.

[0097] Further, in one embodiment, the step of sorting the single character detection frames according to the text line direction in the text line detection result within the grouping group to generate a corrected single character detection result includes:

[0098] Step S221, according to the text line direction in the text line detection result, sorting the single character detection frames in the grouping group according to the text line direction;

[0099] Step S222, detecting the size and direction of the sorted single-character detection frame, normalizing the size and direction by row, and generating a corrected single-character detection result.

[0100] In one embodiment, the single-character detection frames are first sorted according to the text line direction in the text line detection results within the grouping group, and the size and direction of the single-character detection frames are normalized after sorting. It is understandable that sometimes a line of text consisting of characters of the same size will have problems such as the single-character detection frames of some characters being too large, too small, or having different directions. This situation is particularly common when the text line is tilted or curved. Therefore, this embodiment adjusts the size and direction of the single-character detection frames in the sorted text line to unify the size and direction to further improve the accuracy of subsequent recognition.

[0101] Further, in one embodiment, the step of sorting the single character detection frames according to the text line direction in the text line detection result within the grouping group to generate a corrected single character detection result includes:

[0102] Step S223, confirming whether the text line detection result is curved text;

[0103] Step S224: if yes, modify the size of the text line detection result to generate a reduced text line direction;

[0104] Step S225, according to the text line direction, sorting the single character detection frames in the grouping group according to the text line direction, and generating a corrected single character detection result.

[0105] In one embodiment, it is confirmed whether the text line detection result is a curved text. If so, the text line detection result is reduced to generate a reduced text line direction, and according to the text line direction, the single character detection frame is sorted in the grouping group according to the text line direction to generate a corrected single character detection result. In order to solve the problem that it is difficult to sort the single characters of the curved text line and it is difficult to correctly restore the order of the single characters according to the text line, in this embodiment, it is first determined whether the text line detection result is a curved text. Further, for the curved text, the text line detection result (regional image) is reduced to obtain a center line including the center of the single character. It can be considered that a curved rectangle is narrowed to a curved curve. The center line can be a curve or a broken line. According to the direction of the text line, the single character frames are grouped into the text line according to the position, and sorted corresponding to the above-mentioned reduced text line direction (center line). It can be understood that the curve is composed of a series of connected points, so it can be sorted according to the position of the single character on the curve to generate a corrected single character detection result.

[0106] This embodiment combines the text line detection results with the single character detection results, first groups the single character detection boxes by text line, then sorts the single character detection boxes within the group by text line direction, and further normalizes the size and direction of the single character detection boxes by line, thereby solving the problem that there may be text lines with large inclination angles or text lines with multiple text directions in the image, and the single character boxes detected by the text detection model may be of different sizes and directions.

[0107] Furthermore, based on the previous embodiments of the text detection and recognition method of the present invention, a third embodiment of the text detection and recognition method of the present invention is proposed.

[0108] Reference Figure 4 , Figure 4 The detailed flow chart of step S30 in the first embodiment of the text detection and recognition method of the present invention is as follows. In the third embodiment, the step of performing text content recognition based on the text line detection result and the single character detection result and generating a recognition result includes:

[0109] Step S31, performing text content recognition on the text line detection result to generate a text line character recognition result;

[0110] Step S32, determining whether the text line character recognition result is aligned with the position of the single character detection frame in the single character detection result;

[0111] Step S33: If yes, generate a recognition result based on the text line character recognition result.

[0112] This solution proposes a character alignment method based on edit distance, which is used to deal with single-character misdetection, missed detection and other problems caused by extreme situations such as unequal-width fonts, small text, and half-width symbols. It is the post-processing part of the present invention to further improve the accuracy.

[0113] Further, in one embodiment, after the step of determining whether the text line character recognition result is aligned with the position of the single character detection box in the single character detection result, the method includes:

[0114] Step S34: if not, perform text recognition based on the single character detection result to generate a single character recognition result;

[0115] In one embodiment, if the line text recognition result is not aligned with the position of the single character detection box, the single character detection result (text area) is first recognized separately to generate a single character recognition result. Specifically, the text recognition method is not limited here and can be implemented based on a deep learning algorithm.

[0116] Step S35, generating a single-character string according to the single-character detection frame and the single-character recognition result;

[0117] In one embodiment, a single-character string is generated based on the single-character detection frame and the single-character recognition result. It is understandable that the single-character recognition result and the single-character detection frame may not completely correspond. For example, if the recognition result in a single-character detection frame is two characters, the single-character detection frame is split into two single-character frames according to the character width, and then spliced ​​into a single-character string by line.

[0118] Step S36, calculating the edit distance between the single-character string and the corresponding text line string in the text line character detection result, and obtaining the corresponding edit operation;

[0119] In one embodiment, the edit distance between a single-character string and its corresponding row text string is calculated, and the corresponding edit operation is obtained. That is, the single-character string is compared with the row text string, and the corresponding edit operation is selected through the comparison result to correct the recognition result. Edit distance (Edit Distance): It is a string metric that measures the difference between two character sequences. The edit distance between two words is the minimum number of single-character edits (insertion, deletion or replacement) required to convert one word into another word. Generally speaking, the smaller the edit distance, the greater the similarity between the two strings. Specifically, the calculation of the edit distance can be achieved by the Levenshtein Distance algorithm.

[0120] Step S37, executing the editing operation, aligning the text line recognition result with the position of the single character detection box, and generating a recognition result.

[0121] In one embodiment, different processing steps are performed according to the processing steps corresponding to the editing operation to align the text line recognition result with the position of the single character detection box to generate the recognition result. The editing operation includes:

[0122] 1: If you need to insert the corresponding character into the single-character string, insert the corresponding character and insert the corresponding number of single-character detection boxes into two adjacent single-character detection boxes.

[0123] 2: If you need to replace characters or no operation is required, no modification will be made and the single character detection box will be retained.

[0124] 3: If you need to delete a character in a single-character string, delete the corresponding single-character detection box.

[0125] Reference Figure 5 , Figure 5This is a schematic diagram of the character alignment process based on edit distance in an embodiment of the text detection and recognition method of the present invention. The module first determines whether the line text recognition result is aligned with the position of the single character detection box based on the text line recognition result and the single character detection result. If the characters are already aligned, the result is directly output.

[0126] Reference Figure 6 , summarize the text detection and recognition method of the present invention, Figure 6 The flowchart of text detection and recognition in the first embodiment of the text detection and recognition method of the present invention is as follows: first, a text image is input, and single character detection is performed, and the single character detection result is output. The single character detection frame is organized according to the text line detection result, and the single character detection frame in the text line is sorted. After sorting, it is rotated, enlarged and aligned, and text recognition is performed on the text line image area (that is, the text line detection result), and it is determined whether the single character detection frame and the text line recognition result are aligned. If so, the single character and text line matching result (that is, the recognition result) are directly output. If not, text recognition is performed on the single character image area (single character detection result), and the single character detection frame and the text line recognition result are aligned by character matching based on the edit distance, and the result is output. By completing the detection of text lines and single characters at the same time in a text detection model, the accuracy of text recognition is improved. And the text line detection result is combined with the single character detection result to achieve the correct sorting of single characters in irregular texts such as tilted text, curved text, and multi-directional text. By combining the text line recognition result, the single character detection result and the single character recognition result, character alignment of the single character detection result and the text line recognition result is achieved.

[0127] This embodiment determines whether the line text recognition result is aligned with the position of the single character detection box based on the text line recognition result and the single character detection result. If the characters are aligned, the result is directly output; if not aligned, this scheme proposes a character alignment method based on edit distance, which calculates the edit distance between the single character string and the corresponding text line string in the text line character detection result, and obtains the corresponding editing operation, so as to align the line text recognition result with the position of the single character detection box to further improve the recognition accuracy.

[0128] Furthermore, based on the previous embodiments of the text detection and recognition method of the present invention, a fourth embodiment of the text detection and recognition method of the present invention is proposed.

[0129] Reference Figure 7 , Figure 7 The detailed flow diagram of step S20 in the first embodiment of the text detection and recognition method of the present invention is as follows. In the third embodiment, the steps of performing text line detection and single character detection on the text image and outputting the text line detection result and the single character detection result include:

[0130] Step S23, using a dual-branch text detection model, performing text line detection and single character detection on the text image, respectively, predicting a text line probability map and a threshold map, a single character probability map and a threshold map, and performing differentiable binarization to obtain a text line binarization probability map and a single character binarization probability map, respectively;

[0131] Step S24, post-processing the text line binarization probability map and the single character binarization probability map to obtain a text line detection result and a single character detection result.

[0132] In one embodiment, the text detection model used for detection adopts a dual-branch structure, and outputs text line detection results and single character detection results in one model at the same time. It can be understood that if only a single text is detected, the detection results of multiple single words are obtained, which are usually disordered, so it is difficult to sort the characters in the text line. Therefore, in this embodiment, text line recognition and single character recognition are performed simultaneously through a dual-branch structure. Figure 8 As shown, Figure 8 This is a structural diagram of a text detection model in an embodiment of a text detection and recognition method of the present invention. This model uses the ResNet residual neural network as the backbone network, combined with the FPN (Feature Pyramid Networks) feature pyramid network module to extract rich visual features of the text image to be detected; further, in order to simultaneously perform text detection at two levels, text line and single character, in one model, this model adopts a dual-branch structure, similar to DBNet (a text detection network), this model predicts the probability map and threshold map for each branch respectively, and performs differentiable binarization (DB), and obtains the text line binarization probability map (Textline map) and the single character binarization probability map (Text char map) respectively; and then obtains the text line text detection box and the single character text detection box through simple post-processing. Among them, the post-processing is to calculate the minimum bounding rectangle for a single text (text) instance (connected domain of a single white pixel) in the Text line map and the Text char map, and the minimum bounding rectangle is the detection box (result) of the text line / text. It should be noted that the pixel size, shortest side and other parameters in the post-processing can be adjusted according to the actual situation.

[0133] In this embodiment, the text detection model of the present invention is designed in combination with the downstream text recognition task. Through the dual-branch structure, text detection at two levels, text line and single character, is simultaneously performed, thereby generating text line detection results and single character detection results, thereby improving the accuracy of subsequent text recognition.

[0134] The present invention also provides a text detection and recognition device. Fig. 9 As shown, Fig. 9It is a functional module diagram of an embodiment of a text detection and recognition device of the present invention.

[0135] The text detection and recognition device of the present invention comprises:

[0136] An acquisition module 10, used for acquiring a text image;

[0137] A text detection module 20, used for performing text line detection and single character detection on the text image, and outputting text line detection results and single character detection results;

[0138] The text recognition module 30 is used to perform text content recognition based on the text line detection result and the single character detection result, and generate a recognition result.

[0139] Optionally, the device further comprises:

[0140] A single character detection frame sorting correction module is used to group the single character detection frames in the single character detection result by text line according to the text line detection result and the single character detection result;

[0141] According to the text line direction in the text line detection result, the single character detection frames are sorted according to the text line direction within the grouping group to generate a corrected single character detection result.

[0142] Optionally, the single-character detection frame sorting correction module is further used for:

[0143] According to the text line direction in the text line detection result, sorting the single character detection frames in the grouping group according to the text line direction;

[0144] The size and direction of the sorted single-character detection frame are detected, and the size and direction are normalized row by row to generate a corrected single-character detection result.

[0145] Optionally, the single-character detection frame sorting correction module is further used for:

[0146] confirming whether the text line detection result is curved text;

[0147] If yes, modify the size of the text line detection result to generate a reduced text line direction;

[0148] According to the text line direction, the single character detection frames are sorted in the grouping group according to the text line direction to generate a corrected single character detection result.

[0149] Optionally, the device further comprises:

[0150] A character alignment module, used for performing text content recognition on the text line detection result to generate a text line character recognition result;

[0151] Determine whether the text line character recognition result is aligned with the position of the single character detection box in the single character detection result;

[0152] If so, a recognition result is generated based on the text line character recognition result.

[0153] Optionally, the character alignment module is further used for:

[0154] Performing text content recognition on the text line detection result to generate a text line character recognition result;

[0155] Determine whether the text line character recognition result is aligned with the position of the single character detection box in the single character detection result;

[0156] If so, a recognition result is generated based on the text line character recognition result.

[0157] Optionally, the text detection module is further used for:

[0158] Through the dual-branch text detection model, text line detection and single character detection are performed on the text image respectively, and a text line probability map and a threshold map, a single character probability map and a threshold map are predicted, and differentiable binarization is performed to obtain a text line binarization probability map and a single character binarization probability map respectively;

[0159] The text line binarization probability map and the single character binarization probability map are post-processed to obtain the text line detection result and the single character detection result.

[0160] The present invention also provides a computer-readable storage medium.

[0161] The computer-readable storage medium of the present invention stores a text detection and recognition program, and when the text detection and recognition program is executed by a processor, the steps of the text detection and recognition method described above are implemented.

[0162] The method implemented when the management program running on the processor is executed can refer to the various embodiments of the management method of the present invention, and will not be described in detail here.

[0163] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprising a Raman spectroscopy data processing" does not exclude the presence of other identical elements in the process, method, article or system including the element.

[0164] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0165] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a computer-readable storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present invention.

[0166] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A text detection and recognition method, characterized in that: The text detection and recognition method comprises the following steps: Get text image; Performing text line detection and single character detection on the text image, and outputting text line detection results and single character detection results; Based on the text line detection result and the single character detection result, perform text content recognition and generate a recognition result; wherein, through the dual-branch text detection model, the text image is respectively subjected to text line detection and single-character detection, and a text line probability map and a threshold map, a single-character probability map and a threshold map are predicted, and differentiable binarization is performed to obtain a text line binarization probability map and a single-character binarization probability map, respectively; the text line binarization probability map and the single-character binarization probability map are post-processed to obtain the text line detection result and the single-character detection result, wherein the dual-branch text detection model adopts a dual-branch structure to realize the simultaneous output of the text line detection result and the single-character detection result in one model; Among them, after executing the steps of performing text line detection and single character detection on the text image and outputting the text line detection results and the single character detection results, the single character detection boxes in the single character detection results are grouped according to the text line detection results and the single character detection results; according to the text line direction in the text line detection results, the single character detection boxes are sorted according to the text line direction within the group to generate a corrected single character detection result.

2. The text detection and recognition method according to claim 1, characterized in that: The step of sorting the single character detection frames according to the text line direction in the text line detection result within the grouping group to generate a corrected single character detection result comprises: According to the text line direction in the text line detection result, sorting the single character detection frames in the grouping group according to the text line direction; The size and direction of the sorted single-character detection frame are detected, and the size and direction are normalized row by row to generate a corrected single-character detection result.

3. The text detection and recognition method according to claim 1, characterized in that: The step of sorting the single character detection frames according to the text line direction in the text line detection result within the grouping group to generate a corrected single character detection result comprises: confirming whether the text line detection result is curved text; If yes, modify the size of the text line detection result to generate a reduced text line direction; According to the text line direction, the single character detection frames are sorted in the grouping group according to the text line direction to generate a corrected single character detection result.

4. The text detection and recognition method according to claim 1, characterized in that: The step of performing text content recognition based on the text line detection result and the single character detection result and generating a recognition result comprises: Performing text content recognition on the text line detection result to generate a text line character recognition result; Determine whether the text line character recognition result is aligned with the position of the single character detection box in the single character detection result; If so, a recognition result is generated based on the text line character recognition result.

5. The text detection and recognition method according to claim 4, characterized in that: After the step of determining whether the text line character recognition result is aligned with the position of the single character detection box in the single character detection result, the method comprises: If not, performing text recognition based on the single character detection result to generate a single character recognition result; Generate a single-character string according to the single-character detection frame and the single-character recognition result; Calculating the edit distance between the single-character string and the corresponding text line string in the text line character detection result, and obtaining a corresponding edit operation; The editing operation is performed to align the text line recognition result with the position of the single character detection box to generate a recognition result.

6. A text detection and recognition device, characterized in that: The device comprises: An acquisition module, used for acquiring text images; A text detection module, used to perform text line detection and single character detection on the text image, and output text line detection results and single character detection results; A text recognition module, used to perform text content recognition based on the text line detection result and the single character detection result, and generate a recognition result; The text detection module is also used to perform text line detection and single character detection on the text image through a dual-branch text detection model, predict a text line probability map and a threshold map, a single character probability map and a threshold map, and perform differentiable binarization to obtain a text line binarization probability map and a single character binarization probability map, respectively; post-process the text line binarization probability map and the single character binarization probability map to obtain the text line detection result and the single character detection result, wherein the dual-branch text detection model adopts a dual-branch structure to achieve simultaneous output of the text line detection result and the single character detection result in one model; A grouping and sorting module is used to group the single-character detection frames in the single-character detection results according to the text line detection results and the single-character detection results; and to sort the single-character detection frames in the group according to the text line direction in the text line detection results to generate a revised single-character detection result.

7. A text detection and recognition device, characterized in that: The device comprises: a memory, a processor and a text detection and recognition program stored in the memory and executable on the processor, wherein the text detection and recognition program is configured to implement the steps of the text detection and recognition method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a text detection and recognition program, and when the text detection and recognition program is executed by a processor, the steps of the text detection and recognition method according to any one of claims 1 to 5 are implemented.

Images