Text recognition method and device

By establishing the positional relationship between the target characters and the text box, filtering and sorting, the problem of insufficient recognition accuracy in document format conversion is solved, and efficient and accurate text recognition is achieved.

CN113780229BActive Publication Date: 2024-11-22BEIJING KINGSOFT DIGITAL ENTERTAINMENT CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202111101994.2
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-09-18
Publication Date
2024-11-22
Estimated Expiration
2041-09-18

AI Technical Summary

Technical Problem

Existing OCR recognition technology is difficult to meet the high accuracy requirements in document format conversion scenarios, especially when dealing with special characters, headers, footers and overlapping text, which can easily lead to recognition errors and errors.

Method used

By obtaining the text to be recognized, establishing the positional relationship between the target characters and the text box, filtering and sorting the target characters, and generating accurate target text to ensure the accuracy and efficiency of the recognition results.

Benefits of technology

It improves the accuracy of text recognition, reduces the recognition error caused by character sorting errors, and ensures that the recognized text matches the original text with high degree.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113780229B_ABST
    Figure CN113780229B_ABST
Patent Text Reader

Abstract

The present application provides a text recognition method and device, wherein the text recognition method comprises: obtaining a text to be recognized; inputting the text to be recognized into a recognition module for processing, obtaining a target character in the text to be recognized, and a text box corresponding to the target character; establishing a positional relationship between the target character and the text box, and generating a target text corresponding to the text to be recognized based on the positional relationship.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a text recognition method. The present application also relates to a text recognition device, a computing device, and a computer-readable storage medium. Background Art

[0002] With the development of Internet technology, text recognition has become an indispensable technology in most business scenarios, such as photo search, file entry, paper information digitization, document format conversion, etc., all of which involve text recognition technology. The accuracy of text recognition is particularly important in each business scenario. In the prior art, in document format conversion scenarios, OCR recognition technology is mostly used to meet text recognition needs; however, as the demand for recognition accuracy increases, OCR recognition technology is difficult to meet the recognition accuracy requirements of most scenarios, so an effective solution is urgently needed to solve the above problems. Summary of the invention

[0003] In view of this, the embodiment of the present application provides a text recognition method to solve the technical defects existing in the prior art. The embodiment of the present application also provides a text recognition device, a computing device, and a computer-readable storage medium.

[0004] According to a first aspect of an embodiment of the present application, a text recognition method is provided, comprising:

[0005] Get the text to be recognized;

[0006] Input the text to be recognized into a recognition module for processing, and obtain target characters in the text to be recognized, and a text box corresponding to the target characters;

[0007] A positional relationship between the target character and the text box is established, and a target text corresponding to the text to be recognized is generated according to the positional relationship.

[0008] Optionally, the step of inputting the text to be recognized into a recognition module for processing to obtain target characters in the text to be recognized includes:

[0009] Input the text to be recognized into the recognition module, and process it through the character recognition unit in the recognition module to obtain the initial character and the character coordinates corresponding to the initial character;

[0010] Calculating the coordinate similarity between the character coordinates, and selecting the target character coordinates according to the calculation result;

[0011] The target character is selected from the initial characters based on the target character coordinates and outputted through the recognition module.

[0012] Optionally, the step of inputting the text to be recognized into a recognition module for processing to obtain a text box corresponding to the target character includes:

[0013] Input the text to be recognized into the recognition module, and process it through the text processing unit in the recognition module to obtain a text image and size information corresponding to the text image;

[0014] Detecting text elements contained in the text image, and creating a text box corresponding to the text elements based on the size information;

[0015] The text box corresponding to the text component element is used as the text box corresponding to the target character, and is output through the recognition module.

[0016] Optionally, the text component elements include at least one of the following: a header, a footer, a text line;

[0017] Correspondingly, creating a text box corresponding to the text component element based on the size information includes:

[0018] Determine, based on the size information, header coordinates corresponding to the header, footer coordinates corresponding to the footer, and text line coordinates corresponding to the text line;

[0019] Creating a header text box according to the header coordinates, creating a footer text box according to the footer coordinates, and creating a text line text box according to the text line coordinates;

[0020] The header text box, the footer text box and the text line text box are used as text boxes corresponding to the text component elements.

[0021] Optionally, before the step of establishing the positional relationship between the target character and the text box is performed, the step further includes:

[0022] Determine character coordinate information corresponding to the target character and text box coordinate information corresponding to the text box;

[0023] Accordingly, the establishing of the positional relationship between the target character and the text box includes:

[0024] A positional relationship between the target character and the text box is established based on the character coordinate information and the text box coordinate information.

[0025] Optionally, generating a target text corresponding to the to-be-recognized text according to the positional relationship includes:

[0026] Sort the target characters in the text box according to the position relationship and the character coordinate information to obtain a character text box containing the target characters;

[0027] The character text boxes are sorted according to the text box coordinate information, and the target text corresponding to the text to be recognized is obtained according to the sorting result.

[0028] Optionally, generating a target text corresponding to the to-be-recognized text according to the positional relationship includes:

[0029] Detecting whether there are remaining characters in the target characters according to the positional relationship;

[0030] If not, the target text corresponding to the text to be recognized is generated according to the positional relationship.

[0031] Optionally, if the result of detecting whether there are any remaining characters in the target characters according to the positional relationship is yes, the following steps are performed:

[0032] Extract the remaining characters from the target characters, and determine position information corresponding to the remaining characters;

[0033] Clustering the remaining characters based on the height information in the position information, and sorting the clustered remaining characters based on the width information in the position information;

[0034] Obtaining a supplementary text composed of the remaining characters according to the sorting result, and generating an intermediate text according to the positional relationship;

[0035] The supplementary text and the intermediate text are integrated to obtain the target text corresponding to the text to be recognized.

[0036] Optionally, after the step of generating a target text corresponding to the to-be-recognized text according to the positional relationship is performed, the method further includes:

[0037] When it is detected that the target text contains overlapping characters, the word spacing in the target text is adjusted.

[0038] According to a second aspect of an embodiment of the present application, a text recognition device is provided, including:

[0039] An acquisition module, configured to acquire text to be recognized;

[0040] A processing module is configured to input the text to be recognized into a recognition module for processing, and obtain target characters in the text to be recognized, and a text box corresponding to the target characters;

[0041] The generating module is configured to establish a positional relationship between the target character and the text box, and generate a target text corresponding to the text to be recognized according to the positional relationship.

[0042] According to a third aspect of an embodiment of the present application, a computing device is provided, including:

[0043] Memory and processor;

[0044] The memory is used to store computer-executable instructions, and the processor implements the steps of the text recognition method when executing the computer-executable instructions.

[0045] According to a fourth aspect of an embodiment of the present application, a computer-readable storage medium is provided, which stores computer-executable instructions, and when the instructions are executed by a processor, the steps of the text recognition method are implemented.

[0046] According to a fifth aspect of an embodiment of the present application, a chip is provided, which stores a computer program, and the computer program implements the steps of the text recognition method when executed by the chip.

[0047] The text recognition method provided by the present application, after obtaining the text to be recognized, in order to be able to accurately recognize the characters contained in the text to be recognized in the current format, the text to be recognized can be input into the recognition module for processing, so as to obtain the target characters in the text to be recognized output by the module according to the recognition result, and the text box corresponding to the target characters. At this time, the positional relationship between the target characters and the text box can be established, so as to determine which target characters are contained in the text box according to the positional relationship, thereby realizing the determination of the target text corresponding to the text to be recognized according to the positional relationship; after recognizing the target characters, the target characters contained in the text to be recognized can be located through the text box, so as to ensure that the recognized target text is obtained by mapping the text to be recognized, thereby effectively ensuring the recognition accuracy of the text to be recognized and reducing the recognition error caused by character sorting errors. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 is a schematic diagram of a text recognition method provided by an embodiment of the present application;

[0049] Figure 2 is a flowchart of a text recognition method provided by an embodiment of the present application;

[0050] Figure 3 is a schematic diagram of another text recognition method provided by an embodiment of the present application;

[0051] Figure 4 This is a processing flow chart of a text recognition method applied to a housing contract rental scenario provided by an embodiment of the present application;

[0052] Figure 5 is a schematic diagram of a target text provided by an embodiment of the present application;

[0053] Figure 6 is a schematic diagram of another target text provided by an embodiment of the present application;

[0054] Figure 7 is a structural schematic diagram of a text recognition device provided by an embodiment of the present application;

[0055] Figure 8 It is a structural block diagram of a computing device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0056] Many specific details are described in the following description to facilitate a full understanding of the present application. However, the present application can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the connotation of the present application, so the present application is not limited by the specific implementation disclosed below.

[0057] The terms used in one or more embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of the present application. The singular forms of "a", "said" and "the" used in one or more embodiments of the present application and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in one or more embodiments of the present application refers to and includes any or all possible combinations of one or more associated listed items.

[0058] It should be understood that, although the terms first, second, etc. may be used to describe various information in one or more embodiments of the present application, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of the present application, the first may also be referred to as the second, and similarly, the second may also be referred to as the first.

[0059] First, the terms involved in one or more embodiments of the present invention are explained.

[0060] OCR: (Optical Character Recognition) refers to the process of an electronic device (such as a scanner or digital camera) checking the characters printed on paper and then translating the shapes into computer text using character recognition methods; that is, the process of scanning text materials and then analyzing and processing the image files to obtain text and layout information. The main indicators for measuring the performance of an OCR system are: rejection rate, false recognition rate, recognition speed, user interface friendliness, product stability, ease of use and feasibility, etc.

[0061] PDF: (Portable Document Format) is a file format developed by Adobe Systems for file exchange in a way that is independent of applications, operating systems, and hardware. PDF files are based on the PostScript language image model and can guarantee precise colors and accurate printing results on any printer, that is, PDF will faithfully reproduce every character, color, and image of the original.

[0062] DBNet: A text detection network based on semantic segmentation algorithm. After receiving the input image, it obtains the feature map F after feature extraction, upsampling fusion and concat operation, and then uses the feature map F to predict the probability map (probability map) called P, and uses the feature map F to predict the threshold map (threshold map) called T. Finally, the binary map B is calculated through the probability map P and the threshold map T for output.

[0063] PDF Plumer: An open source Python-based PDF parsing package that can extract information such as characters and tables.

[0064] NMS: (Non-Maximum Suppression) is to suppress elements that are not maximum values. It is used in target detection to extract high-confidence target detection boxes and suppress low-confidence false detection boxes. Generally speaking, when the parsing model outputs the target box, there will be a lot of target boxes. The specific number is determined by the number of anchors. There are many duplicate boxes that locate the same target. NMS is used to remove these duplicate boxes and obtain the real target box.

[0065] YOLO: (You Only Look Once, object detection algorithm) is an object detector that uses features learned by deep convolutional neural networks to detect objects; YOLO v3, as the third version of YOLO, is mainly composed of 75 convolutional layers, which are most effective for analyzing object features. Without using a fully connected layer, the network can correspond to inputs of any size.

[0066] Faster-RCNN: A fully differentiable model whose input is represented as a tensor (multidimensional array) of Height×Width×Depth. After being processed by a pre-trained CNN model, a convolutional feature map is obtained. That is, CNN is used as a feature extractor and sent to the next part; then, RPN (Region Propose Network) processes the extracted convolutional feature map; RPN is used to find a predefined number of regions (regions, bounding boxes) that may contain objects; finally, the target detection process is completed based on the R-CNN module.

[0067] In the present application, a text recognition method is provided. The present application also relates to a text recognition device, a computing device, and a computer-readable storage medium, which are described in detail one by one in the following embodiments.

[0068] In practical applications, text recognition in PDF files is used in many scenarios, such as converting PDF to WORD format, which requires text recognition, or extracting text from PDF files for information entry, which requires text recognition. Usually, most PDF text extraction methods use OCR recognition technology, that is, converting PDF to images, and then using OCR recognition technology for detection and recognition; there are also toolkits such as PDF Plumer to extract PDF text information. However, OCR recognition technology has the problem of inaccurate character recognition, and the recognition error rate for special characters is high. The text information extraction tool cannot remove the header and footer for recognition, and the method of bolding text by overlapping the same text makes it easy to recognize multiple characters of the same type when recognizing bold text. Therefore, an effective solution is urgently needed to solve the above problems.

[0069] See also Figure 1 The text recognition schematic diagram shown in the figure, after obtaining the text to be recognized, in order to be able to accurately recognize the characters contained in the text to be recognized in the current format, the text to be recognized can be input into the recognition module for processing, so as to obtain the target characters in the text to be recognized output by the module according to the recognition result, and the text box corresponding to the target characters, and establish a positional relationship between the target characters and the text box, so that according to the positional relationship, it is possible to determine which target characters are contained in the text box, thereby realizing that according to the positional relationship, the target text corresponding to the text to be recognized can be determined; after recognizing the target characters, the target characters contained in the text to be recognized can be located through the text box, ensuring that the recognized target text is obtained by mapping the text to be recognized, thereby effectively ensuring the recognition accuracy of the text to be recognized and reducing the recognition error caused by character sorting errors.

[0070] Figure 2 A flowchart of a text recognition method provided according to an embodiment of the present application is shown, which specifically includes the following steps:

[0071] Step S202, obtaining the text to be recognized.

[0072] Specifically, the text to be recognized refers to the text that needs to identify the character units, image content, tables and / or formulas contained therein, and the text content needs to be extracted; it should be noted that the text to be recognized can be obtained from the client connected to the server, that is, after the user uploads the text to be recognized through the client connected to the server, the text recognition processing operation will be started to return the recognition result to the client held by the user, where the client held by the user includes but is not limited to mobile phones, computers, tablets and other smart devices, and this embodiment does not make any limitation here; accordingly, the text to be recognized includes but is not limited to PDF format, doc format, docx format, etc., and this embodiment does not make any limitation here.

[0073] In this embodiment, the text recognition method is explained by taking the text to be recognized in PDF format (the text to be recognized in this PDF format is a parsable PDF to support subsequent parsing and recognition of text content) as an example. The recognition process of text to be recognized in other formats can refer to the same or similar description content of this embodiment, and no further details will be given here.

[0074] Step S204: input the text to be recognized into a recognition module for processing, and obtain target characters in the text to be recognized, and a text box corresponding to the target characters.

[0075] Specifically, on the basis of the above-mentioned acquisition of the text to be recognized, further, in order to accurately recognize the content contained in the text to be recognized and avoid the problem of repeated recognition of characters or missed recognition of characters, the text to be recognized can be input into the recognition module for processing, that is, the characters in the text to be processed are recognized, and a text box is constructed for the content to be recognized in the text to be recognized. After the recognition is completed, the recognition result is output through the output layer of the recognition module, so as to obtain the target characters contained in the text to be recognized and the text box corresponding to the target characters according to the recognition result.

[0076] Among them, the recognition module specifically refers to an integrated module that integrates a text processing unit and a character recognition unit. When the text to be recognized is recognized, the text processing unit and the character recognition unit can complete the recognition processing operation at the same time, or they can complete the recognition processing operation in sequence. In order to improve the recognition processing rate, it is preferred that the text processing unit and the character recognition unit complete the recognition processing operation at the same time, that is, the recognition processing is performed in parallel. This embodiment does not make any restrictions here. Among them, the text processing unit can be implemented based on DBNet, YOLO v3 or Faster-RCNN, etc., and the character recognition unit can be implemented based on PDF Plumer or PDF Miner; during the specific implementation, it can be selected according to actual needs, and this embodiment does not make any restrictions here. Correspondingly, the target character specifically refers to the character unit contained in the text to be recognized, and a character unit represents a Chinese character, a data or a letter. The text box specifically refers to a rectangular box that selects each component in the text to be recognized, and the target character can be located through the text box.

[0077] That is to say, when locating the position of each target character, first determine the text box to which the target character belongs, and then sort according to the coordinates of each target character in the text box; it should be noted that the upper left corner of the text to be recognized can be used as the origin to establish an xy coordinate system, and the character coordinates of each character can be determined based on the coordinate system, that is, the height information is the y-axis coordinate of the character, and the width information is the x-axis coordinate of the character. Based on this, for example, a text box corresponds to a row of target characters in the text to be recognized. After determining the positional relationship between the target characters and the text box, you can sort according to the x-axis coordinate corresponding to each target character. There is no need to consider the y-axis coordinate to locate the position of each target character, which effectively improves the efficiency of character sorting to ensure the accuracy of the target text recognized subsequently, while avoiding confusion in character sorting.

[0078] In this process, in order to ensure that the recognition module can accurately complete the recognition processing operation, it is necessary to adequately train the text processing unit and the character recognition unit in the preparation stage. The training process of the text processing unit is as follows: after obtaining the sample to be recognized, the line, paragraph or language in the sample to be recognized is used as a label for the text box recognition unit, and the sample pair is formed based on the labeled sample. The initial text processing unit is trained until a text processing unit that meets the training stop condition is obtained and added to the recognition module. When adding labels to the sample to be recognized, only the x-direction coordinates of the single-line characters need to be considered for the line unit; for the paragraph unit, the y-direction coordinates need to be re-positioned based on the x-direction coordinates to achieve the recognition of the text box in the paragraph unit; adding text boxes in the language unit means: marking the continuous characters in the text that belong to the same language once, so that the trained model can output text boxes in the language unit.

[0079] In practical applications, different recognition scenarios can use text processing units trained in different ways to perform subsequent recognition processing operations. The specific recognition method can be selected according to the actual application scenario, and this embodiment does not make any limitation here.

[0080] Based on this, after obtaining the text to be recognized, the text to be recognized can be input into the recognition module for processing, the target characters can be recognized by the character recognition unit in the recognition module, and the text box can be recognized by the text processing unit, so as to facilitate the subsequent determination of the target text by combining the text box and the target characters, thereby achieving the goal of improving the recognition efficiency while ensuring the recognition accuracy.

[0081] Furthermore, in the process of identifying the target characters, considering that each character corresponds to a unique position in the text to be identified, in order to avoid the problem of position overlap (small character spacing, character overlap) caused by low recognition accuracy, thereby affecting the determination of the target characters, the target characters can be determined in combination with the coordinates during the recognition process. In this embodiment, the specific implementation method is as follows:

[0082] Step S2042, inputting the text to be recognized into the recognition module, processing it through the character recognition unit in the recognition module, and obtaining the initial character and the character coordinates corresponding to the initial character;

[0083] Step S2044, calculating the coordinate similarity between the character coordinates, and filtering the target character coordinates according to the calculation result;

[0084] Step S2046, filtering out the target character from the initial characters based on the target character coordinates, and outputting it through the recognition module.

[0085] Specifically, the character recognition unit specifically refers to a unit capable of recognizing character units contained in the text to be recognized and determining the coordinate information of each character; accordingly, the initial character is the recognized character unit, and the character coordinates are the coordinate information corresponding to each initial character, and the coordinate information indicates the position of the character in the text to be recognized; accordingly, the coordinate similarity specifically refers to a standard for characterizing the degree of overlap between the coordinates of each character. The higher the coordinate similarity, the greater the probability of character overlap, which further indicates that the possibility of recognizing two characters at the same position during the recognition process is greater. Conversely, the smaller the coordinate similarity, the smaller the probability of character overlap, which further indicates that the possibility of recognizing two characters at the same position during the recognition process is smaller; the coordinate similarity can be implemented using an IOU (Intersection over Union, overlap) algorithm, and the formula is as follows: IOU = (A∩B) / (A∪B), wherein A and B respectively represent the coordinates corresponding to two adjacent characters; when the IOU value is higher than a preset threshold, it is considered to be overlapped, otherwise it is not overlapped. The preset threshold can be set to 0.9 or 0.85, etc., and this embodiment does not make any limitation here. The target character coordinates specifically refer to the character coordinates determined after removing duplicate character coordinates. The target character coordinates can fully reflect the position of each character. On this basis, the target characters in the text to be recognized can be determined, and the errors caused by inaccurate recognition can be reduced by coordinate screening.

[0086] Among them, the character coordinates of the initial character will establish an xy coordinate system with the upper left corner of the text to be recognized as the origin, and the character coordinates of each initial character can be determined based on the coordinate system. That is, the height information is the y-axis coordinate of the character, and the width information is the x-axis coordinate of the character.

[0087] At the same time, when determining the coordinates of each initial character in the coordinate system, in order to ensure the accuracy of the coordinates of the positioned characters, it can be implemented by locating one character coordinate using four coordinates, that is, building a rectangular frame based on one initial character, using each vertex of the rectangular frame as a coordinate point to locate one initial character, that is, the coordinates corresponding to the four vertices are the character coordinates corresponding to the initial character. In practical applications, in order to efficiently determine the coordinates of the initial character, only the coordinates corresponding to the vertices on the diagonal of the rectangular frame where each initial character is located are determined. In addition, it can also be implemented by locating one character using one coordinate, that is, selecting the coordinates of the center point of the initial character as the character coordinates; in practical applications, the method of determining the character coordinates of the initial character can be selected according to the actual application scenario, and this embodiment does not make any restrictions here.

[0088] Based on this, after obtaining the text to be recognized, the text to be recognized can be input into the recognition module. The character recognition unit in the recognition module can determine the initial characters contained in the text to be recognized and the character coordinates corresponding to each initial character. In order to improve the recognition accuracy, the target characters can be screened based on the character coordinates at this time, specifically, calculating the coordinate similarity between the character coordinates corresponding to each initial character, and selecting character coordinates greater than a preset similarity threshold for deduplication according to the coordinate similarity, that is, eliminating the initial characters corresponding to the character coordinates whose character similarity is greater than the preset similarity threshold, thereby filtering out the target characters from the initial characters according to the target character coordinates determined after eliminating the duplicate character coordinates, and outputting the recognition module for subsequent creation of the target text corresponding to the text to be recognized to ensure recognition accuracy.

[0089] In practical applications, the process of calculating coordinate similarity and screening out target characters can use the NMS method to achieve deduplication, thereby ensuring that the screened out target characters are more accurate, thereby laying the foundation for the subsequent generation of target text and avoiding errors caused by recognition accuracy problems; in addition, the deduplication processing operation can also be implemented based on the preset deduplication rules, that is, determining the center point of each initial character, calculating the distance between adjacent center points, and when the distance is less than the preset distance, it is considered that the initial characters corresponding to the adjacent center points overlap, and then the overlapping initial characters are removed to obtain the target character. Correspondingly, character recognition can be completed using PDF Plumer or PDF Miner, etc., so that the characters and coordinate information in the text to be recognized in PDF format can be accurately extracted to assist in completing the recognition processing operation of the target characters. In specific implementation, character recognition and deduplication processing can also select other processing methods according to actual needs, and this embodiment does not make any restrictions here.

[0090] In summary, by screening target characters based on character coordinates, the errors caused by inaccurate recognition can be effectively reduced, thereby improving the recognition accuracy of the target characters, and ensuring that the target characters are closer to the word units contained in the text to be recognized, thereby effectively ensuring that the subsequently generated target text is more accurate.

[0091] Furthermore, in the process of identifying the text box corresponding to the target character, in order to accurately locate each target character through the text box, thereby ensuring the recognition accuracy of the target text, the text box recognition can be completed in combination with the text processing unit. In this embodiment, the specific implementation method is as follows:

[0092] Step S2142: input the text to be recognized into the recognition module, and process it through the text processing unit in the recognition module to obtain a text image and size information corresponding to the text image.

[0093] Specifically, the text recognition unit specifically refers to a unit that can convert the text to be recognized in PDF format into a picture format and can recognize the text picture size information; wherein, the text picture specifically refers to the picture obtained after converting the text to be recognized into a picture format, and correspondingly, the size information specifically refers to the length and width information corresponding to the text picture; in the process of the text processing unit processing the text to be recognized, it actually converts the text to be recognized into a general picture format, such as jpg or png format, for subsequent use.

[0094] Based on this, after obtaining the text to be recognized, the text to be recognized can be input into the recognition module. The text to be recognized can be processed by the text processing unit in the recognition module, and the text to be recognized can be converted into a picture format to obtain a text picture. At the same time, the size information of the text picture can be determined to facilitate the subsequent combination of the text picture and the size information to determine the text box corresponding to the target character, thereby assisting in the subsequent recognition of the target text.

[0095] Step S2144, detect the text components contained in the text image, and create a text box corresponding to the text components based on the size information.

[0096] Specifically, after obtaining the text image and the size information corresponding to the text image as described above, further, in order to accurately determine the text box corresponding to the target character, the text components of the text to be recognized can be detected in the text image, and a text box corresponding to each text component element can be created based on the size information. Among them, the text component element specifically refers to the basic elements that constitute the text to be recognized, including but not limited to headers, footers, and text lines; accordingly, the text box corresponding to the text component element specifically refers to a rectangular box that can select a header, footer, or text line. After determining the text box corresponding to each text component element, the target character can be located to assist in completing the recognition process of the target text.

[0097] It should be noted that the text box is used as the basis for the subsequent target text composition. Therefore, when creating a text box based on size information, it can be implemented in the following two ways. On the one hand, the text box is created with the edge of the character unit as the boundary, that is, the created text box will fit the character unit in the box; on the other hand, the text box is created with the size set by the edge of the character unit as the boundary, that is, the created text box will set the size of the distance from the character unit in the box, and this size will not affect other text boxes, that is, it will not overlap with other text boxes. In actual applications, the creation of text boxes can be selected according to needs, and this embodiment does not make any restrictions here.

[0098] In practical applications, when detecting text components from text images, algorithms such as DBNet, PSENet or PANNet can be used. That is, the text image is input into the trained model, and the text components, headers, footers and text lines of the text to be recognized can be obtained according to the recognition results of the model, so as to facilitate the subsequent recognition of the target text.

[0099] Based on this, in order to accurately detect the text components, during training, a large number of PDF files can be converted into images, and then the headers, footers, and text lines in each image can be annotated. A large number of sample pairs can be obtained based on the annotation results, and then the DBNet model can be trained using the sample pairs to obtain a DBNet model that meets the usage requirements based on the training results, so as to be used for the recognition of text components. In other words, the DBNet model can recognize text images and output the headers, footers, and text lines corresponding to the text to be recognized.

[0100] Further, after the text components are detected, since each text component element corresponds to a part of the target characters, the target text can be identified only by classifying each target character into a corresponding text box. Therefore, determining the text box corresponding to each text component element can assist in the subsequent positioning of the target characters. In this embodiment, the specific implementation method is as follows:

[0101] Step S21442, determining the header coordinates corresponding to the header, the footer coordinates corresponding to the footer, and the text line coordinates corresponding to the text based on the size information;

[0102] Step S21444, creating a header text box according to the header coordinates, creating a footer text box according to the footer coordinates, and creating a text line text box according to the text line coordinates;

[0103] Step S21446, taking the header text box, footer text box and text line text box as text boxes corresponding to the text component elements.

[0104] Specifically, the header coordinates specifically refer to the coordinate information corresponding to the header area in the text to be identified; the footer coordinates specifically refer to the coordinate information corresponding to the footer area in the text to be identified; the text line coordinates specifically refer to the coordinate information corresponding to the text area in the text to be identified; accordingly, the header text box specifically refers to the rectangular box for selecting the header area; the footer text box specifically refers to the rectangular box for selecting the footer area; the text line text box specifically refers to the rectangular box for selecting the text area; it should be noted that the text line text box can be a rectangular box for selecting the entire content contained in the text area; it can also be a plurality of rectangular boxes for selecting the entire content contained in the text area according to paragraphs; it can also be a plurality of rectangular boxes for selecting the entire content contained in the text area according to character units; it can also be a plurality of rectangular boxes for selecting the entire content contained in the text area according to line units; it can also be a plurality of rectangular boxes for selecting the entire content contained in the text area according to language types. During specific implementation, it can be selected according to needs, and this embodiment does not make any limitation here.

[0105] In practical applications, a document usually consists of text lines, headers and / or footers. Therefore, it is only necessary to determine each area and add the deduplicated target characters to each area to generate the target text corresponding to the text to be recognized.

[0106] Based on this, when the text components include headers, footers and text lines, after detecting the text components in the text to be recognized, the header coordinates corresponding to the header, the footer coordinates corresponding to the footer, and the text line coordinates corresponding to the text line can be located according to the size information of the text image; in this process, a reference coordinate positioning template that matches the size information of the text image can be selected, and then the four vertex coordinates corresponding to the header are read in the template as the header coordinates, the four vertex coordinates corresponding to the footer are read as the footer coordinates, and the four vertex coordinates corresponding to the text line are read as the text line coordinates; the coordinates corresponding to the text components are then connected to create a text box, that is, a header text box corresponding to the header area can be created according to the header coordinates, a footer text box corresponding to the footer area can be created according to the footer coordinates, and a text line text box corresponding to the text line area can be created according to the text line coordinates. At this time, the header text box, footer text box and text line text box can be used as the text boxes corresponding to the text components, so as to subsequently locate each target character, thereby recognizing the target text.

[0107] It should be noted that the reference coordinate positioning template is a pre-set template that matches the type of text to be recognized, and the reference coordinate positioning template contains the coordinates corresponding to each text component element; after the text component elements in the text to be recognized are detected, the reference coordinate positioning template can be used to determine the coordinates corresponding to the text component elements for subsequent creation of a text box.

[0108] In summary, by creating text boxes of text elements in units of regions, it can be ensured that there is no overlap of text boxes, and it can also assist in the subsequent positioning of target characters, thereby ensuring the accuracy of the recognized target text.

[0109] Step S2146, taking the text box corresponding to the text component element as the text box corresponding to the target character, and outputting it through the recognition module.

[0110] Specifically, after obtaining the text box corresponding to the text component element, it can be used as the text box corresponding to the target character, so as to facilitate the subsequent positioning of each target character according to the text box and ensure the accuracy of identifying the target text. Based on this, since each target character corresponds to unique coordinate information, each target character can be added to each text box according to the relationship between the coordinates contained in the text box, and then the target characters in the text box can be sorted to obtain the target text.

[0111] For example, after obtaining a paper in PDF format, the paper is input into the recognition module, and the text processing unit (such as OCR detection unit) in the recognition module converts the paper in PDF format into an image, and records the page size information of the paper, that is, the length and width information of the page, according to the conversion result; then, the trained DBNet model is used to detect the header, footer and text line in the paper, and the coordinate information corresponding to the header, footer and text line is recorded, and the header text box corresponding to the header, the footer text box corresponding to the footer, and the text line text box corresponding to the text line are determined based on the coordinate information.

[0112] Furthermore, while determining the text box corresponding to each component in the paper, the character recognition unit in the recognition module will be used to extract the characters contained in the paper, that is, PDF Plumer will be used to extract each character and determine the character coordinates corresponding to each character; then the NMF method will be used to calculate the similarity between the character coordinates, and the character coordinates with a similarity greater than a preset similarity threshold will be selected for deduplication, and the remaining character coordinates will be used as the target character coordinates. At this time, the target character will be selected from the recognized characters in combination with the target character coordinates as the recognition result of all the character units in the paper, so as to be used for the subsequent composition of the target text corresponding to the paper.

[0113] In addition, when the text to be recognized contains pictures, tables and / or formulas, in order to ensure that the pictures, tables and / or formulas can exist in the target text after the recognition process to avoid content loss, the pictures, tables and / or formulas can be processed by image recognition during the recognition process, that is, the contents contained in the pictures, tables and / or formulas are recognized by OCR recognition in the form of pictures for subsequent creation of the target text. In this embodiment, the specific implementation method is as follows:

[0114] When the content to be identified in the text to be identified includes image content, table content and / or formula content, the text processing unit in the recognition module can be used to select and identify the content to be identified to obtain the coordinate information of the content to be identified, so as to facilitate the subsequent determination of the position of the image, table and / or formula according to the coordinate information, so as to achieve the purpose of forming the target text.

[0115] It should be noted that when the text to be recognized contains pictures, text, tables and / or formulas, the above-mentioned recognition methods can be integrated to recognize each type of content to be recognized separately to ensure recognition accuracy and achieve a high degree of match between the target text created subsequently and the text to be recognized.

[0116] In addition, when recognizing the text to be recognized that contains a table, the text processing unit needs to recognize the structure frame of the table, and the character recognition unit needs to complete the recognition of the content within the structure frame. The two are combined as the recognition content of the table to facilitate the subsequent creation of the target text.

[0117] That is to say, when the content to be recognized contains image content, table content and / or formula content, the image, table and / or formula will be recognized as a recognition unit, which can not only avoid the impact of excessive character content contained therein, but also improve the recognition efficiency, thereby achieving the recognition of texts with richer content, improving the coverage of recognition types, and ensuring the recognition effect.

[0118] Continuing with the above example, when a paper in PDF format contains pictures, tables, and formulas, the OCR detection unit in the recognition module can be used to identify the length and width information of the pictures, tables, and formulas, and the coordinate information of the pictures, tables, and formulas can be determined in combination with the length and width information to facilitate the subsequent composition of the target text. The coordinate information can be added to the recognition results to form the target text containing characters, pictures, tables, and formulas.

[0119] Step S206, establishing a positional relationship between the target character and the text box, and generating a target text corresponding to the text to be recognized according to the positional relationship.

[0120] Specifically, after obtaining the target character and the text box corresponding to the target character, in order to ensure that there is a small error between the recognized target text and the text to be recognized, the target character can be positioned based on the text box, and the target characters can be reasonably and accurately sorted in each text box, so as to generate the target text according to the sorting result. Among them, the target text specifically refers to the recognition result obtained after the recognition processing of the text to be recognized; correspondingly, the position relationship specifically refers to the relationship between the text box containing the target character.

[0121] Based on this, after obtaining the target character, the positional relationship between the target character and the text box can be established through the back-end server, that is, the target character contained in the text box can be determined. When determining the target character contained in the text box, the area enclosed by the text box in the coordinate system can be preferentially determined, and then the target character whose character coordinates are located in the area is selected to establish a positional relationship, thereby determining the target character contained in the text box; in this process, the overlap between each target character and the text box can be calculated through IOU, and the target character whose overlap is greater than a preset threshold is selected to establish a positional relationship with the text box, thereby accurately determining which target characters are contained in the text box, so as to sort the target characters contained in the text box, and then generate the target text corresponding to the text to be recognized according to the sorting result, thereby ensuring the recognition accuracy.

[0122] Furthermore, since the text box corresponds to the text component element in the text to be recognized, the text box has a unique position in the text to be recognized, and the target character also has a unique position in the text to be recognized, when determining the positional relationship between the target character and the text box, it can be implemented based on the coordinate information of the target character and the text box. In this embodiment, the specific implementation method is as follows:

[0123] Step S2062, determining character coordinate information corresponding to the target character and text box coordinate information corresponding to the text box;

[0124] Step S2064: establishing a positional relationship between the target character and the text box based on the character coordinate information and the text box coordinate information.

[0125] Specifically, the character coordinate information specifically refers to the coordinates corresponding to each target character; the text box coordinate information specifically refers to the coordinates corresponding to each text box.

[0126] Based on this, in order to accurately determine the positional relationship between the text box and the target character to promote the accuracy of the subsequent generation of the target text, the character coordinate information corresponding to the target character and the text box coordinate information corresponding to the text box can be determined first, and then the positional relationship between the target character and the text box can be established based on the character coordinate information and the text box coordinate information. In other words, the target characters located in the text box are selected to establish a positional relationship, so as to determine which text box each target character is located in, and then the target characters contained in the text box are sorted according to the character coordinate information, so as to generate the target text.

[0127] In summary, by combining the coordinate information to establish the positional relationship between the text box and the target character, each target character can be accurately located, thereby effectively improving the recognition accuracy and the efficiency of generating the target text.

[0128] Furthermore, after determining the positional relationship between the target character and the text box based on the coordinate information, the target text can be generated according to the character coordinate information. In this embodiment, the specific implementation method is as follows:

[0129] Step S2162, sorting the target characters in the text box according to the position relationship and the character coordinate information to obtain a character text box containing the target characters;

[0130] Step S2164, sorting the character text boxes according to the text box coordinate information, and obtaining the target text corresponding to the text to be recognized according to the sorting result.

[0131] Specifically, the character text box specifically refers to a rectangular box obtained after adding the target character to the corresponding text box, and contains the target character.

[0132] Based on this, after obtaining the positional relationship between the target character and the text box, the target characters contained in the text box can be sorted based on the positional relationship and the character coordinate information to obtain the character text box containing the target character according to the sorting result, and then the character text boxes containing the target characters can be sorted according to the text box coordinate information to obtain the target text corresponding to the text to be recognized according to the sorting result.

[0133] Using the above example, after obtaining the header text box, footer text box and text line text box, the character coordinates corresponding to each character and the text box coordinates corresponding to each text box can be determined. Then, the relationship between each text box and each character is established by selecting the relationship with the character coordinates contained in the text box coordinates. For example, if the area corresponding to the header text box consists of coordinates (0, 0), (5, 5), (0, 5) and (5, 0), and the character coordinates are (1, 1) (1, 2) (6, 3) ..., then the characters corresponding to the character coordinates (1, 1) (1, 2) ... can be selected to establish the relationship with the header text box, and so on, until all are completed. According to the establishment results, it is determined that the header text box contains characters Z1 to Z10, the footer text box contains characters Z51 to Z55, and the text line text box contains characters Z11 to Z50.

[0134] Furthermore, at this time, the coordinates of each character can be sorted in each text box with a positional relationship based on the above positional relationship, that is, in the case where the selected area corresponds to each line of text content in the identified header text box, footer text box or text line text box, at this time, it is only necessary to locate each character in the x-axis direction according to the x-axis coordinate of each character to determine the arrangement order of each character in the corresponding text box. Alternatively, the initial positioning is performed in the x-axis direction according to the x-axis coordinate of each character, and then the secondary positioning is performed in the y-axis direction according to the y-axis coordinate of each character, and finally the sorting result of each character in the text box is determined according to the two positioning results.

[0135] Furthermore, characters Z1 to Z10 are sorted in the header text box according to the coordinates corresponding to characters Z1 to Z10, characters Z51 to Z55 are sorted in the footer text box according to the coordinates corresponding to characters Z51 to Z55, and characters Z11 to Z50 are sorted in the text line text box according to the coordinates corresponding to characters Z11 to Z50. Based on the sorting results, the target text corresponding to the paper in PDF format can be obtained, thereby completing the recognition processing of the paper in PDF format.

[0136] In summary, by using coordinates as a driving force to sort the target characters in the text box, the accuracy of character sorting can be guaranteed, thereby ensuring the accuracy of recognition of the text to be recognized.

[0137] In addition, when sorting the characters contained in the text box based on the position relationship, considering that the target character is the character obtained after screening, there may be other characters that are not sorted. If these characters are discarded, the recognition accuracy may be affected. Therefore, when there are other characters that are not sorted, clustering can be selected to process them. In this embodiment, the specific implementation method is as follows:

[0138] Detect whether there are remaining characters in the target character according to the positional relationship between the target character and the text box;

[0139] If not, a target text corresponding to the text to be recognized is generated according to the positional relationship.

[0140] If so, extract the remaining characters from the target characters and determine the position information corresponding to the remaining characters; cluster the remaining characters based on the height information in the position information, and sort the remaining characters that have completed clustering based on the width information in the position information; obtain the supplementary text composed of the remaining characters according to the sorting result, and generate the intermediate text according to the positional relationship between the target characters and the text box; integrate the supplementary text and the intermediate text to obtain the target text corresponding to the text to be recognized.

[0141] Specifically, the remaining characters specifically refer to the target characters that have not yet corresponded to the text box; correspondingly, the supplementary text specifically refers to the text composed of the remaining characters, and correspondingly, the intermediate text specifically refers to the text created based on the positional relationship between the text box and the target characters.

[0142] Based on this, after the positional relationship between the target character and the text box is established, it is possible to detect whether there are remaining characters in the target character. The specific detection method is to detect whether there are characters in the target character that have not established a positional relationship with the text box; if not, it means that all characters correspond to the corresponding text box, and then the target text generation process can be performed. If it exists, it means that there are still some characters that are not corresponding to the text box. In order to ensure the comprehensiveness of recognition, the position information of the remaining characters can be determined at this time, and then the remaining characters are clustered according to the height information in the position information to achieve clustering the remaining characters with consistent height information together, and at the same time, the remaining characters clustered together are sorted according to the width information in the position information. When sorting, if the text box is divided into lines, the remaining characters with the same height information can be selected to be sorted in the text box in the x-axis direction; or if the text box is not divided into lines, the remaining characters with the same height information can be selected to be initially sorted in the x-axis direction, and then sorted again in the y-axis direction according to the height information, so as to obtain the supplementary text composed of the remaining characters according to the sorting result.

[0143] Furthermore, after the supplementary text is obtained, an intermediate text is generated according to the positional relationship between the text box and the target character, and finally the supplementary text and the intermediate text are integrated to obtain the target text corresponding to the text to be recognized.

[0144] In addition, when detecting whether there are remaining characters, it is also possible to detect after the target text is created, that is, to create the initial target text according to the positional relationship between the target characters and the text box, and then detect whether there are remaining characters that have not been added to the initial target text. If it is detected that there are remaining characters, the supplementary text is generated according to the above processing operation and added to the initial target text, and the target text corresponding to the text to be recognized can be obtained. It should be noted that the creation process of the supplementary text can refer to the corresponding description content above, and no further elaboration is made here.

[0145] In one embodiment, assuming that after the paper is recognized, a header text box and a footer text box are obtained. At this time, it is determined through detection that the target characters also include remaining characters. In order to ensure the comprehensiveness of the recognition, the remaining characters can be classified according to the height information of the remaining characters, and the classified remaining characters can be sorted based on the width information of the remaining characters. When sorting, the remaining characters are connected according to the x-axis coordinates according to the width information to obtain the supplementary text according to the sorting result, and combined with the height information of the characters in the supplementary text, the bottom of the paper corresponding to the supplementary text is determined, and the footer content of the paper corresponding to the supplementary text can be determined. Then, the footer content is integrated with the header content and the main text content to obtain the target text corresponding to the paper in PDF format.

[0146] In summary, by detecting the remaining characters, the comprehensiveness of recognition can be effectively guaranteed, thereby improving the accuracy of identifying the text to be recognized, and achieving that the target text can fully reflect the content of the text to be recognized.

[0147] In addition, in order to avoid the existence of overlapping characters in the recognized target text and thus affect downstream business processing, overlapping character detection can be performed on the target text after the target text is generated. In this embodiment, the specific implementation method is as follows:

[0148] Step S2262: When it is detected that the target text contains overlapping characters, the character spacing in the target text is adjusted.

[0149] Specifically, overlapping characters refer to characters that overlap after being sorted in the text box, affecting reading. Based on this, when detecting whether the target text contains overlapping characters, it can be achieved by detecting the word spacing, that is, judging whether there is a situation in the target text where the word spacing is less than a preset threshold. If so, it means that the target characters overlap each other in the current target text. In order to facilitate downstream business use, the word spacing of the characters in the target text can be adjusted so that there is a certain distance between each character, ensuring that the characters in the target text are clearly visible and more convenient for downstream business use.

[0150] In actual applications, when using the target text, downstream businesses can extract information from the editable target text as needed to generate review text, edit text, or typeset text for user feedback to facilitate user use.

[0151] See also Figure 3 As shown in the schematic diagram, when the user submits the text to be recognized in PDF format through the front end (the front end is the interface visible to the user, such as the website front end interface and the APP user interface), the front end can parse the text to be recognized, obtain the PDF file stream and send it to the back end server; after receiving the text to be recognized, the back end server can input the PDF file stream corresponding to the text to be recognized into the recognition module in order to accurately recognize the characters contained in the text to be recognized in the current format, and process it through the text box recognition unit and the character recognition unit in the recognition module to obtain the target characters in the output text to be recognized according to the recognition results, and the text box corresponding to the target character, at this time, the position relationship between the target character and the text box can be established, so that the target characters contained in the text box can be determined according to the position relationship, so as to achieve the determination of the target file corresponding to the text to be recognized according to the position relationship; finally, in response to the processing operation of the downstream business, the file is extracted, and the text that meets the user's usage needs is created and feedback is given to the user. After the target characters are recognized, the target characters contained in the text to be recognized can be located through the text box to ensure that the recognized target text is obtained by mapping the text to be recognized, thereby effectively ensuring the recognition accuracy of the text to be recognized and reducing the recognition error caused by character sorting errors.

[0152] In conjunction with Appendix 4, the text recognition method provided by this application is further described by taking the application of the text recognition method in the housing contract leasing scenario as an example. Figure 4 A processing flow chart of a text recognition method applied to a housing contract rental scenario provided by an embodiment of the present application is shown, which specifically includes the following steps:

[0153] Step S402: Obtain the text to be recognized in PDF format.

[0154] In practical applications, text recognition in PDF files is used in many scenarios, such as converting PDF to WORD format, or extracting text from PDF files for information entry. Usually, most PDF text extraction methods use OCR recognition technology, that is, converting PDF to images, and then using OCR recognition technology for detection and recognition. There are also toolkits such as PDF plumer to extract PDF text information. OCR recognition technology has the problem of inaccurate character recognition, and the error rate for special characters is high. The text information extraction tool cannot remove the header and footer for recognition, and the method of bolding by overlapping characters is easy to repeatedly recognize multiple characters of the same type when recognizing text. Therefore, an effective solution is urgently needed to solve the above problems.

[0155] In this embodiment, the text to be identified is a housing contract, which includes {header-contract number: 123456789; text: housing lease contract, Party A: A, Party B: B, signing time: May 15, 2021}; and the housing contract is in PDF format.

[0156] Step S404: input the text to be recognized in PDF format into the recognition module for processing, and obtain the target characters in the text to be recognized and the text boxes corresponding to the target characters.

[0157] After obtaining the housing contract in PDF format, the housing contract can be input into the recognition module. Through the text recognition unit in the recognition module, such as the OCR detection unit, the housing contract in PDF format is first converted into an image, and the page size information of the housing contract, that is, the length and width information of the page, is recorded according to the conversion result; then the trained DBNet model (that is, the text processing unit) is used to detect the header, footer and text line in the housing contract, and the corresponding coordinate information is recorded according to the detection result. At this time, the DBNet model outputs the header text box corresponding to the header, the footer text box corresponding to the footer, and the text line text box corresponding to the text line, and each text box corresponds to the coordinate information, that is, the position of each text box, and the length and width information, which is convenient for subsequent text recognition. That is, determine that the header-contract number: 123456789 corresponds to the header text box, and the position information is S1; determine that the text: Housing Lease Contract, Party A: A, Party B: B, Signing Time: May 15, 2021 corresponds to the text line text box, and the position information is S2.

[0158] Further, while determining the text box corresponding to the text to be recognized, the tool extraction unit in the recognition module can also be used to extract each character. Optionally, the text information and coordinate information of each character are extracted using PDFplumer, and the coordinate information is the position of each character. Then, the NMS method is used to remove duplicates of the characters, so as to obtain all the character units in the text to be recognized for subsequent creation of the target text.

[0159] Step S406: Determine the text box that the target character matches.

[0160] Specifically, after determining the header text box with position information S1, the text line text box with position information S2, and the target character, the target character can be matched with the header text box and the text line text box to determine the character unit corresponding to each text box, that is, determine that the character units "Contract Number 1, 2, 3, 4, 5, 6, 7, 8, 9" correspond to the header text box, and the character units "Residential Lease Contract Party A A, Party B B, Signing Date May 15, 2021" correspond to the third text box.

[0161] Step S408: Sort the target characters according to the text box.

[0162] After determining the text box that each character matches, in order to generate the target format text that meets the requirements subsequently, the characters can be sorted in each text box according to the coordinate information corresponding to each character. That is, since the coordinate of the text box restricts the character sorting result, after sorting the characters according to their coordinate information in the text box, it can be ensured that the converted text matches the text in the PDF format. That is, the character units "Contract Number 1, 2, 3, 4, 5, 6, 7, 8, 9" are sorted in the header text box according to the coordinate information corresponding to each character, and the first text is obtained according to the sorting result. At the same time, the character units "Residential Lease Contract Party A A, Party B B, Signing Date May 15, 2021" are sorted in the text line text box according to the coordinate information corresponding to each character, and the second text is obtained according to the sorting result.

[0163] Step S410: Generate the target text according to the sorting result.

[0164] Specifically, after sorting each character into each text box, the target text as shown can be obtained according to the sorting result. Figure 5 Optionally, the target text is in an editable WORD format.

[0165] In addition, when matching text boxes and characters, individual characters may not be matched to text boxes. For example, the text to be recognized contains a footer, but the footer is not matched to the corresponding text box. In order to accurately recognize the footer, the characters in the unrecognized footer can be clustered according to the height information, that is, the remaining unrecognized target characters are clustered according to the height information, and the clustered characters are sorted according to the width information, that is, the characters in a single line of text are sorted in order according to the width information, and each line of text is sorted according to the height coordinate (height information), and the corresponding text line is added to the corresponding height position. In this embodiment, the following is obtained. Figure 6 The target text shown contains the header, footer and body.

[0166] The text recognition method provided by the present application, after obtaining the text to be recognized, in order to be able to accurately recognize the characters contained in the text to be recognized in the current format, the text to be recognized can be input into the recognition module for processing, so as to obtain the target characters in the text to be recognized output by the module according to the recognition result, and the text box corresponding to the target characters. At this time, the positional relationship between the target characters and the text box can be established, so as to determine which target characters are contained in the text box according to the positional relationship, so as to achieve the determination of the target text corresponding to the text to be recognized according to the positional relationship; after recognizing the target characters, the target characters contained in the text to be recognized can be located through the text box, so as to ensure that the recognized target text is obtained by mapping the text to be recognized, thereby effectively ensuring the recognition accuracy of the text to be recognized and reducing the recognition error caused by character sorting errors.

[0167] Corresponding to the above method embodiment, the present application also provides a text recognition device embodiment, Figure 7 FIG. 1 is a schematic diagram showing the structure of a text recognition device provided by an embodiment of the present application. Figure 7 As shown, the device comprises:

[0168] An acquisition module 702 is configured to acquire text to be recognized;

[0169] The processing module 704 is configured to input the text to be recognized into the recognition module for processing, and obtain the target characters in the text to be recognized, and the text box corresponding to the target characters;

[0170] The generating module 706 is configured to establish a positional relationship between the target character and the text box, and generate a target text corresponding to the text to be recognized according to the positional relationship.

[0171] In an optional embodiment, the processing module 704 is further configured to:

[0172] The text to be recognized is input into the recognition module, and the initial character and the character coordinates corresponding to the initial character are obtained by processing through the character recognition unit in the recognition module; the coordinate similarity between the character coordinates is calculated, and the target character coordinates are filtered according to the calculation result; the target character is filtered out from the initial characters based on the target character coordinates, and outputted through the recognition module.

[0173] In an optional embodiment, the processing module 704 is further configured to:

[0174] The text to be recognized is input into the recognition module, and processed by the text processing unit in the recognition module to obtain a text image and size information corresponding to the text image; the text component elements contained in the text image are detected, and a text box corresponding to the text component elements is created based on the size information; the text box corresponding to the text component elements is used as the text box corresponding to the target character, and outputted through the recognition module.

[0175] In an optional embodiment, the text component element includes at least one of the following: a header, a footer, and a text line; accordingly, the processing module 704 is further configured to:

[0176] Based on the size information, determine the header coordinates corresponding to the header, the footer coordinates corresponding to the footer, and the text row coordinates corresponding to the text row; create a header text box according to the header coordinates, create a footer text box according to the footer coordinates, and create a text row text box according to the text row coordinates; use the header text box, the footer text box, and the text row text box as text boxes corresponding to the text component elements.

[0177] In an optional embodiment, the text recognition device further includes:

[0178] A determination module, configured to determine character coordinate information corresponding to the target character and text box coordinate information corresponding to the text box;

[0179] Accordingly, the generating module 706 is further configured to:

[0180] A positional relationship between the target character and the text box is established based on the character coordinate information and the text box coordinate information.

[0181] In an optional embodiment, the generating module 706 is further configured to:

[0182] According to the positional relationship and the character coordinate information, the target characters are sorted in the text box to obtain a character text box containing the target characters; the character text box is sorted according to the text box coordinate information, and the target text corresponding to the text to be recognized is obtained according to the sorting result.

[0183] In an optional embodiment, the generating module 706 is further configured to:

[0184] According to the positional relationship, it is detected whether there are any remaining characters in the target characters; if not, the target text corresponding to the text to be recognized is generated according to the positional relationship.

[0185] In an optional embodiment, the generating module 706 is further configured to:

[0186] Extract the remaining characters from the target characters and determine the position information corresponding to the remaining characters; cluster the remaining characters based on the height information in the position information, and sort the clustered remaining characters based on the width information in the position information; obtain a supplementary text composed of the remaining characters according to the sorting result, and generate an intermediate text according to the position relationship; integrate the supplementary text and the intermediate text to obtain the target text corresponding to the text to be recognized.

[0187] In an optional embodiment, the text recognition device further includes:

[0188] The adjustment module is configured to adjust the word spacing in the target text when it is detected that the target text contains overlapping characters.

[0189] The text recognition device provided by the present application, after acquiring the text to be recognized, in order to be able to accurately recognize the characters contained in the text to be recognized in the current format, can input the text to be recognized into the recognition module for processing, so as to obtain the target characters in the text to be recognized output by the module according to the recognition result, and the text box corresponding to the target characters. At this time, the positional relationship between the target characters and the text box can be established, so as to determine which target characters are contained in the text box according to the positional relationship, so as to achieve the determination of the target text corresponding to the text to be recognized according to the positional relationship; after recognizing the target characters, the target characters contained in the text to be recognized can be located through the text box, so as to ensure that the recognized target text is obtained by mapping the text to be recognized, thereby effectively ensuring the recognition accuracy of the text to be recognized and reducing the recognition error caused by character sorting errors.

[0190] The above is a schematic scheme of a text recognition device of the present embodiment. It should be noted that the technical scheme of the text recognition device and the technical scheme of the above-mentioned text recognition method belong to the same concept. For details not described in detail in the technical scheme of the text recognition device, please refer to the description of the technical scheme of the above-mentioned text recognition method. In addition, each component in the device embodiment should be understood as a functional module that must be established to implement each step of the program flow or each step of the method, and each functional module is not an actual functional division or separation definition. The device claim defined by such a group of functional modules should be understood as a functional module architecture that mainly implements the solution through the computer program recorded in the specification, and should not be understood as a physical device that mainly implements the solution through hardware.

[0191] Figure 8 The structure block diagram of a computing device 800 provided according to an embodiment of the present application is shown. The components of the computing device 800 include but are not limited to a memory 810 and a processor 820. The processor 820 is connected to the memory 810 via a bus 830, and the database 850 is used to store data.

[0192] The computing device 800 also includes an access device 840 that enables the computing device 800 to communicate via one or more networks 860. Examples of these networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 840 may include one or more of any type of network interface (e.g., a network interface card (NIC)) that is wired or wireless, such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a World Wide Interoperability for Microwave Access (Wi-MAX) interface, an Ethernet interface, a universal serial bus (USB) interface, a cellular network interface, a Bluetooth interface, a near field communication (NFC) interface, and the like.

[0193] In one embodiment of the present application, the above components of the computing device 800 and Figure 8 Other components not shown in the figure may also be connected to each other, for example, via a bus. It should be understood that Figure 8 The computing device structure block diagram shown is only for the purpose of illustration, and is not intended to limit the scope of the present application. Those skilled in the art may add or replace other components as needed.

[0194] The computing device 800 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc.), a mobile phone (e.g., a smart phone), a wearable computing device (e.g., a smart watch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or PC. The computing device 800 may also be a mobile or stationary server.

[0195] The processor 820 is used to execute the following computer executable instructions:

[0196] Get the text to be recognized;

[0197] Input the text to be recognized into a recognition module for processing, and obtain target characters in the text to be recognized, and a text box corresponding to the target characters;

[0198] A positional relationship between the target character and the text box is established, and a target text corresponding to the text to be recognized is generated according to the positional relationship.

[0199] The above is a schematic scheme of a computing device of this embodiment. It should be noted that the technical scheme of the computing device and the technical scheme of the above text recognition method belong to the same concept, and the details not described in detail in the technical scheme of the computing device can be referred to the description of the technical scheme of the above text recognition method.

[0200] An embodiment of the present application further provides a computer-readable storage medium storing computer instructions, which are used when executed by a processor to:

[0201] Get the text to be recognized;

[0202] Input the text to be recognized into a recognition module for processing, and obtain target characters in the text to be recognized, and a text box corresponding to the target characters;

[0203] A positional relationship between the target character and the text box is established, and a target text corresponding to the text to be recognized is generated according to the positional relationship.

[0204] The above is a schematic scheme of a computer-readable storage medium of this embodiment. It should be noted that the technical scheme of the storage medium and the technical scheme of the above-mentioned text recognition method belong to the same concept, and the details not described in detail in the technical scheme of the storage medium can be referred to the description of the technical scheme of the above-mentioned text recognition method.

[0205] This embodiment discloses a chip storing a computer program, which implements the steps of the text recognition method when executed by the chip.

[0206] The above describes specific embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0207] The computer instructions include computer program codes, which may be in source code form, object code form, executable files or some intermediate forms, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium may be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0208] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described action sequence, because according to the present application, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0209] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0210] The preferred embodiments of the present application disclosed above are only used to help explain the present application. The optional embodiments do not describe all the details in detail, nor do they limit the invention to the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of the present application. The present application selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present application, so that those skilled in the art can understand and use the present application well. The present application is only limited by the claims and their full scope and equivalents.

Claims

1. A text recognition method, characterized in that: include: Get the text to be recognized; Input the text to be recognized into a character recognition unit in a recognition module for processing to obtain target characters in the text to be recognized, and input the text to be recognized into a text processing unit in the recognition module for processing to obtain text boxes corresponding to each text component element in the text to be recognized, wherein the text box is determined based on a text image and size information corresponding to the text to be recognized, and the text component element is a basic element constituting the text to be recognized; Determine the character coordinate information corresponding to the target character and the text box coordinate information corresponding to the text box. Based on the character coordinate information and the text box coordinate information, and in accordance with the positional relationship of the text box containing the target character, add the target character to the corresponding text box to generate the target text corresponding to the text to be recognized.

2. The text recognition method according to claim 1, characterized in that: The step of inputting the text to be recognized into a character recognition unit in a recognition module for processing to obtain target characters in the text to be recognized includes: Input the text to be recognized into the recognition module, and process it through the character recognition unit in the recognition module to obtain the initial character and the character coordinates corresponding to the initial character; Calculating the coordinate similarity between the character coordinates, and selecting the target character coordinates according to the calculation result; The target character is selected from the initial characters based on the target character coordinates and outputted through the recognition module.

3. The text recognition method according to claim 1, characterized in that: The step of inputting the text to be recognized into the text processing unit in the recognition module for processing to obtain a text box corresponding to each text component element in the text to be recognized includes: Input the text to be recognized into the recognition module, and process it through the text processing unit in the recognition module to obtain a text image and size information corresponding to the text image; Detecting text elements contained in the text image, and creating a text box corresponding to the text elements based on the size information; The text box corresponding to the text component element is used as the text box corresponding to the target character, and is output through the recognition module.

4. The text recognition method according to claim 3, characterized in that: The text component elements include at least one of the following: a header, a footer, and a text line; Correspondingly, creating a text box corresponding to the text component element based on the size information includes: Determine, based on the size information, header coordinates corresponding to the header, footer coordinates corresponding to the footer, and text line coordinates corresponding to the text line; Creating a header text box according to the header coordinates, creating a footer text box according to the footer coordinates, and creating a text line text box according to the text line coordinates; The header text box, the footer text box and the text line text box are used as text boxes corresponding to the text component elements.

5. The text recognition method according to claim 1, characterized in that: The step of adding the target character to a corresponding text box based on the character coordinate information and the text box coordinate information according to the positional relationship of the text box containing the target character, and generating a target text corresponding to the text to be recognized includes: Sort the target characters in the text box according to the position relationship and the character coordinate information to obtain a character text box containing the target characters; The character text boxes are sorted according to the text box coordinate information, and the target text corresponding to the text to be recognized is obtained according to the sorting result.

6. The text recognition method according to any one of claims 1 to 4, characterized in that: The step of adding the target character to a corresponding text box based on the character coordinate information and the text box coordinate information according to the positional relationship of the text box containing the target character, and generating a target text corresponding to the text to be recognized includes: Detecting whether there are remaining characters in the target characters according to the positional relationship; If not, the target text corresponding to the text to be recognized is generated according to the positional relationship.

7. The text recognition method according to claim 6, characterized in that: If the result of the detection of whether there are any remaining characters in the target characters according to the positional relationship is yes, the following steps are performed: Extract the remaining characters from the target characters, and determine position information corresponding to the remaining characters; Clustering the remaining characters based on the height information in the position information, and sorting the clustered remaining characters based on the width information in the position information; Obtaining a supplementary text composed of the remaining characters according to the sorting result, and generating an intermediate text according to the positional relationship; The supplementary text and the intermediate text are integrated to obtain the target text corresponding to the text to be recognized.

8. The text recognition method according to claim 1, characterized in that: After the step of generating the target text corresponding to the text to be recognized is performed, the method further includes: When it is detected that the target text contains overlapping characters, the word spacing in the target text is adjusted.

9. A text recognition device, characterized in that: include: An acquisition module, configured to acquire text to be recognized; a processing module configured to input the text to be recognized into a character recognition unit in a recognition module for processing, obtain target characters in the text to be recognized, and input the text to be recognized into a text processing unit in the recognition module for processing, obtain text boxes corresponding to each text component element in the text to be recognized, wherein the text boxes are determined based on a text image and size information corresponding to the text to be recognized, and the text component elements are basic elements constituting the text to be recognized; A generation module is configured to determine the character coordinate information corresponding to the target character and the text box coordinate information corresponding to the text box, and based on the character coordinate information and the text box coordinate information, add the target character to the corresponding text box in accordance with the positional relationship of the text box containing the target character, thereby generating a target text corresponding to the text to be recognized.

10. A computing device, characterized in that include: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the steps of any one of the methods of claims 1 to 8.

11. A computer-readable storage medium storing computer instructions, characterized in that: When the instruction is executed by a processor, the steps of the method described in any one of claims 1 to 8 are implemented.