A training data enhancement method and system for invoice recognition model

Abstract

The present invention provides a training data enhancement method and system for an invoice recognition model, comprising: inputting an invoice image into a general text detection model to obtain a text area; inputting the text area into a general text recognition model to obtain a recognized text; obtaining all field values ​​of the invoice from a ticketing system, and obtaining the real text after correction; matching the recognized text with the corresponding real text; determining the relative position of the recognized text in the real text, completing the unrecognized text, and calculating the position information of the completed complete text area; splicing the guide words in the recognized text with the corresponding real text to obtain the complete text information; taking the invoice image as input, the position information of the complete text area as a label, and constructing text detection training data; taking the complete text area as input, the complete text information as a label, and constructing text recognition training data. The present invention can automatically correct the recognition errors of the general model in the invoice scenario and generate high-quality labeled data.

Description

Technical Field

[0001] The present invention relates to the field of text recognition technology, and in particular to a training data enhancement method and system for an invoice recognition model. Background Art

[0002] OCR (Optical Character Recognition) technology has become a vital part of the field of artificial intelligence and computer vision. Its core goal is to structure the text information in the image for subsequent processing. OCR technology is widely used in document management, bill recognition, financial processing and other fields. Among them, invoice recognition is one of the important business scenarios. Through the automated processing of invoices, the financial and management efficiency of enterprises can be significantly improved.

[0003] In the invoice recognition system, text detection and text recognition are key technologies. The task of text detection is to locate the areas containing text in the invoice image and accurately calibrate the boundaries of these areas, usually expressed as four-point coordinates. Text recognition extracts the text content from these areas and ultimately converts the image information into recognized text. High-precision text detection and text recognition models rely on a large amount of high-quality training data. However, the construction process of training data is time-consuming and labor-intensive, becoming the main bottleneck of model development.

[0004] The current mainstream annotation methods include fully manual annotation and semi-automatic annotation tools:

[0005] The fully manual labeling method completely relies on the labelers to review the images one by one and label the coordinates and corresponding text content of each text area one by one. Although the labeling quality is high and it is suitable for various complex scenes, the labeling cost is high, it requires a lot of manpower, and requires the labelers to have high professionalism. The labeling efficiency is low. It takes several minutes or even longer to label a single image. It is difficult to generate a large-scale training data set in a short time. As the data scale grows, the manual labeling efficiency is difficult to keep up with the demand for training data scale for model optimization.

[0006] Semi-automatic annotation tools combine general text detection and text recognition models, and output preliminary results for manual correction. Although this reduces manual intervention to a certain extent and improves annotation efficiency, the application effect of semi-automatic annotation tools in invoice scenarios is not ideal: the general model accuracy is insufficient. Since the general model is not optimized for the invoice scenario, the initial results of detection and recognition often have large errors, resulting in high manual correction costs. Summary of the invention

[0007] In view of this, an embodiment of the present invention provides a method and system for enhancing training data for an invoice recognition model to eliminate or improve one or more defects existing in the prior art.

[0008] In one aspect, the present invention provides a method for enhancing training data for an invoice recognition model, the method comprising the following steps:

[0009] Input the invoice image into a general text detection model to obtain all text areas in the invoice image; after cropping the text area, input it into a general text recognition model to obtain recognized text; obtain all field values ​​of the corresponding invoice image through the ticketing system, annotate the recognized text, and obtain real text; wherein the real text does not contain the guide words on the invoice image;

[0010] The recognized texts are traversed in a loop, and the real texts corresponding to the recognized texts are matched by a text matching method; the relative position of the recognized texts in the corresponding real texts is determined by a text matching and edit distance method, and the unrecognized texts are completed to obtain the complete texts, and the position information of the complete text area is calculated; the guide words in the recognized texts are spliced ​​with the corresponding real texts to obtain the complete text information;

[0011] The invoice image is used as input, and the position information of the complete text area is used as a label to construct training data for text detection; the complete text area is obtained by cropping according to the position information of the complete text area, and the complete text area is used as input, and the complete text information is used as a label to construct training data for text recognition.

[0012] In some embodiments of the present invention, an invoice image is input into a general text detection model to obtain all text regions in the invoice image; after cropping the text region, the invoice image is input into a general text recognition model to obtain recognized text, including:

[0013] When the general text detection model is used to detect text regions, there are cases where a complete text region is detected, a partial text region is detected, and a text region is fragmented;

[0014] When the recognized text is obtained using the universal character recognition model, the recognized text may contain typos and missing characters.

[0015] In some embodiments of the present invention, all field values ​​of the corresponding invoice image are obtained through the ticketing system, and the recognized text is annotated, including:

[0016] Correct the invoice number, invoice code, invoice date, price and tax total and check code in the identified text.

[0017] In some embodiments of the present invention, the recognized texts are traversed in a loop, and a text matching method is used to match the real text corresponding to each recognized text, including:

[0018] Acquire the recognized text and the real text;

[0019] Loop through the recognized text, compare the recognized text with the real text, and calculate the ratio of the number of characters in the intersection of the two to the recognized text;

[0020] If the ratio is greater than a preset threshold, the match is successful.

[0021] In some embodiments of the present invention, the relative position of the recognized text in its corresponding real text is determined by using the edit distance method, and the unrecognized text is completed to obtain the complete text, including:

[0022] Initialize a matrix, wherein the rows of the matrix represent the characters of the recognized text and the columns represent the characters of the real text; calculate the edit distance between each character step by step through a dynamic programming algorithm to obtain the minimum edit distance between the recognized text and the real text;

[0023] According to the minimum edit distance, the longest common substring of the recognized text and the real text is obtained; the longest common substring represents the matching area of ​​the recognized text and the real text; based on the longest common substring, the relative position of the recognized text in its corresponding real text is determined, and the unrecognized text is completed.

[0024] In some embodiments of the present invention, calculating the position information of the complete text region includes:

[0025] Obtaining text area position information of the recognized text;

[0026] Calculating the average length of each character in the recognized text;

[0027] Calculating the total length of the characters to be padded according to the relative position of the recognized text in the corresponding real text;

[0028] The position information of the complete text area is calculated according to the text area position information of the recognized text and the total length of the characters to be padded, and the coordinate information of the four vertices of the complete text area is obtained.

[0029] In some embodiments of the present invention, when the recognized text is on the left side of the real text, the method for calculating the coordinate information of the complete text area includes:

[0030] ;

[0031] ;

[0032] ;

[0033] ;

[0034] ;

[0035] Similarly,

[0036] ;

[0037] ;

[0038] ;

[0039] ;

[0040] ;

[0041] The coordinates of the four vertices of the upper left corner, upper right corner, lower right corner and lower left corner of the complete text area are:

[0042] ;

[0043] in, The coordinates of the four vertices of the upper left corner, the upper right corner, the lower right corner and the lower left corner of the text area of ​​the recognized text; and Indicates the length of the recognized text; Represents the number of characters of the recognized text; The number of characters representing the complete text; and Indicates the average length of each character of the recognized text; and Indicates the total length of characters to be padded; Indicates error.

[0044] On the other hand, the present invention also provides a training data enhancement system for an invoice recognition model, characterized in that the system comprises:

[0045] The text extraction module is used to input the invoice image into the general text detection model to obtain all text areas in the invoice image; after cropping the text area, input it into the general text recognition model to obtain the recognized text; obtain all field values ​​of the corresponding invoice image through the ticketing system, annotate the recognized text, and obtain the real text;

[0046] The text comparison and correction module is used to loop through the recognized texts, match the real texts corresponding to the recognized texts by using the text matching method; determine the relative position of the recognized texts in the real texts corresponding to them by using the text matching and edit distance methods, and complete the unrecognized texts to obtain the complete texts, and calculate the position information of the complete text area; splice the guide words in the recognized texts with the real texts corresponding to them to obtain the complete text information;

[0047] The training data construction module is used to take the invoice image as input and the position information of the complete text area as a label to construct training data for text detection; the complete text area is obtained by cropping according to the position information of the complete text area, the complete text area is taken as input, and the complete text information is used as a label to construct training data for text recognition.

[0048] On the other hand, the present invention further provides a computer-readable storage medium having a computer program / instruction stored thereon, which implements the steps of any of the methods mentioned above when executed by a processor.

[0049] On the other hand, the present invention further provides a computer program product, comprising a computer program / instruction, characterized in that when the computer program / instruction is executed by a processor, the steps of any of the methods mentioned above are implemented.

[0050] The present invention provides a training data enhancement method and system for an invoice recognition model, comprising: inputting an invoice image into a general text detection model to obtain a text area; inputting the text area into a general text recognition model to obtain a recognized text; obtaining all field values ​​of the invoice from a ticketing system, and obtaining the real text after correction; matching the recognized text with the corresponding real text; determining the relative position of the recognized text in the real text, completing the unrecognized text, and calculating the position information of the complete text area after completion; splicing the guide words in the recognized text with the corresponding real text to obtain the complete text information; taking the invoice image as input, the position information of the complete text area as a label, and constructing text detection training data; taking the complete text area as input, the complete text information as a label, and constructing text recognition training data. The present invention can automatically correct the recognition error of the general model in the invoice scenario, generate high-quality annotated data, so as to improve the scale generation capability of the model training data, provide support for the optimization of the invoice recognition model, and thus achieve a higher model accuracy.

[0051] Additional advantages, purposes, and features of the present invention will be described in part in the following description, and will become apparent to those skilled in the art after studying the following, or may be learned from the practice of the present invention. The purposes and other advantages of the present invention may be achieved and obtained by the structures specifically indicated in the specification and the accompanying drawings.

[0052] Those skilled in the art will appreciate that the objectives and advantages that can be achieved with the present invention are not limited to the above specific description, and the above and other objectives that can be achieved by the present invention will be more clearly understood from the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] The drawings described herein are used to provide a further understanding of the present invention, constitute a part of the present application, and do not constitute a limitation of the present invention. In the drawings:

[0054] Figure 1 The figure is a schematic diagram of the steps of a training data enhancement method for an invoice recognition model in one embodiment of the present invention.

[0055] FIG. 2( a ) shows a situation when a general text detection model is used to detect a text region in an embodiment of the present invention.

[0056] FIG. 2( b ) shows a situation when a general text detection model is used to detect a text region in an embodiment of the present invention.

[0057] FIG. 2( c ) shows a situation when a general text detection model is used to detect a text region in an embodiment of the present invention.

[0058] Figure 3 Schematic diagram of a text area obtained by cropping in an embodiment of the present invention.

[0059] Figure 4 Schematic diagram of text area completion in one embodiment of the present invention. DETAILED DESCRIPTION

[0060] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments and the accompanying drawings. Here, the illustrative embodiments of the present invention and their descriptions are used to explain the present invention, but are not intended to limit the present invention.

[0061] It should also be noted that, in order to avoid obscuring the present invention due to unnecessary details, only structures and / or processing steps closely related to the solutions according to the present invention are shown in the accompanying drawings, while other details that are not closely related to the present invention are omitted.

[0062] It should be emphasized that the term “include / comprises” when used herein refers to the presence of features, elements, steps or components, but does not exclude the presence or addition of one or more other features, elements, steps or components.

[0063] It should also be noted that, unless otherwise specified, the term “connection” herein may refer not only to a direct connection but also to an indirect connection with an intermediate.

[0064] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In the accompanying drawings, the same reference numerals represent the same or similar components, or the same or similar steps.

[0065] In order to solve the problem that the existing labeling method has low accuracy of the general recognition model and is not suitable for invoice application scenarios, resulting in large errors in the recognition results and high manual correction costs, the present invention provides a training data enhancement method for the invoice recognition model, such as Figure 1 As shown, the present invention includes the following steps S101-S103:

[0066] Step S101: Input the invoice image into the general text detection model to obtain all text areas in the invoice image; after cropping the text area, input it into the general text recognition model to obtain the recognized text; obtain all field values ​​of the corresponding invoice image through the ticketing system, annotate the recognized text, and obtain the real text. The real text does not contain the guide words on the invoice image.

[0067] Step S102: Loop through the recognized texts, and use the text matching method to match the real text corresponding to each recognized text; use the text matching and edit distance method to determine the relative position of the recognized text in its corresponding real text, and complete the unrecognized text to obtain the complete text, and calculate the position information of the complete text area; splice the guide words in the recognized text with the corresponding real text to obtain the complete text information.

[0068] Step S103: Take the invoice image as input and the position information of the complete text area as a label to construct training data for text detection; crop the complete text area according to the position information of the complete text area, take the complete text area as input, and use the complete text information as a label to construct training data for text recognition.

[0069] In step S101, an open source general text detection model is used to detect all text areas in the invoice image, and the coordinates of the text areas are usually output in the form of a rectangular box or a quadrilateral box.

[0070] In some embodiments, the general text detection model may adopt an EAST model, a CTPN model, etc. The EAST model adopts a fully convolutional network (FCN) structure, which can directly predict the quadrilateral box (or rectangular box) of the text area. The CTPN model connects the text area through a long short-term memory (LSTM) network and finally outputs a rectangular box.

[0071] The detected text area is cropped to obtain a cropped image, and the open source general text recognition model is used to recognize the text in the cropped image to obtain the recognized text, namely the OCR (Optical Character Recognition) text.

[0072] In some embodiments, the general text recognition model may adopt a Tesseract OCR model, a CENN model, an EasyOCR model, or the like.

[0073] In some embodiments, when a general text detection model is used to detect a text region, taking the text "Name: XX (Tianjin) XXXX Co., Ltd." as an example, the following situations may exist:

[0074] As shown in Figure 2 (a), it is detected correctly and the text area of ​​“Name: XX (Tianjin) XXXX Co., Ltd.” is completely detected.

[0075] As shown in Figure 2 (b), only part of the text area is detected, namely "Name: XX (Tianjin)", while the latter half "XXXX Co., Ltd." is not detected.

[0076] As shown in Figure 2 (c), the detection results are fragmented, splitting the text area "Name: XX (Tianjin) XXXX Co., Ltd." that should be complete into two text areas: "Name: XX (Tianjin)" and "XXXX Co., Ltd.".

[0077] It is also possible that no text area is detected at all.

[0078] In some embodiments, after the general text detection model detects and obtains the text area, the text area is cropped out. Taking the text "Name: XX (Tianjin) XXXX Co., Ltd." as an example, Figure 3 As shown, the text area is cut out and input into the general text recognition model to be converted into corresponding text information to obtain the recognized text. Among them, the recognized text may have the following situations: recognition of wrong characters, missing characters in the middle, missing characters at the beginning, missing characters at the end, or mixed errors.

[0079] Based on the above description, it can be seen that the recognition accuracy of the recognized text (OCR text) obtained by the general model may be affected by factors such as fonts, typesetting, noise, etc., resulting in errors in some structured data. Therefore, all field values ​​of the corresponding invoice image are obtained from the ticketing system, and key fields such as invoice number, invoice code, invoice date, price and tax total, and check code are annotated and corrected to ensure the accuracy of the data, so as to serve as the real text of the invoice.

[0080] Among them, the real text obtained based on the ticketing system does not contain the guide word. Taking "Name: XX (Tianjin) XXXX Co., Ltd." as an example, the real text does not contain the guide word "Name:", but only contains "XX (Tianjin) XXXX Co., Ltd.".

[0081] In step S102, firstly, the real text corresponding to the recognized text is matched.

[0082] In some embodiments, the matching method includes the following steps:

[0083] Get the recognized text and all the real texts to be matched; loop through the recognized text, compare the recognized text with the real text, and calculate the ratio of the number of characters in the intersection of the two to the recognized text; if the ratio is greater than the preset threshold, the match is successful, and the real text is considered to be the real text corresponding to the recognized text. For example, the recognized text is "Name: XX (Tianjin)" and the corresponding real text is "XX (Tianjin) XXXX Co., Ltd.". For example, the threshold can be set to 70%.

[0084] Then the relative position of the recognized text in the real text is determined.

[0085] In some embodiments, the guide words in the recognized text are first removed. For example, if the recognized text is "Name: XX (Tianjin)", after removing the guide words, "XX (Tianjin)" is obtained, and the corresponding real text is "XX (Tianjin) XXXX Co., Ltd.". Through text matching and edit distance calculation, the first six characters of the recognized text in the real text can be obtained.

[0086] Edit distance is an algorithm that measures the similarity between two strings and is used to calculate the minimum number of edit operations required to convert one string into another. Edit operations usually include the following three types:

[0087] Insert: Insert a character into a string.

[0088] Delete: Delete a character from a string.

[0089] Replace: Replace one character in a string with another character.

[0090] For example:

[0091] To convert the string "kitten" to "sitting", we need to perform the following steps:

[0092] Replace k with s: kitten → sitten

[0093] Replace e with i: sitten → sittin

[0094] Insert g: sittin → sitting

[0095] So the edit distance is 3.

[0096] Based on the above description, the relative position of the recognized text in its corresponding real text is determined by using the edit distance method, including the following steps:

[0097] Initialize the matrix, where the rows of the matrix represent the characters of the recognized text and the columns represent the characters of the real text. Use the dynamic programming algorithm to gradually calculate the edit distance between each character to obtain the minimum edit distance between the recognized text and the real text. According to the minimum edit distance, find the longest common substring (LCS) of the recognized text and the real text. The longest common substring represents the matching area of ​​the recognized text and the real text. Based on the longest common substring, the relative position of the recognized text in its corresponding real text can be determined.

[0098] After determining the relative position of the recognized text in its corresponding real text, the unrecognized text is completed.

[0099] In some embodiments, Figure 4 As shown in the figure, since the invoice may be taken or scanned, the invoice image may be tilted. Take the detected text area as "Name: XX (Tianjin)" and the real text as "XX (Tianjin) XXXX Co., Ltd." as an example. Based on the above method, after determining the first six characters of the recognized text in the real text, the "XXXX Co., Ltd." that is not recognized is completed. Specifically:

[0100] Through the general text detection model, the text area is obtained, that is, Figure 4 The coordinates of the four vertices of the solid frame:

[0101] , which correspond to the four vertices of the upper left corner, upper right corner, lower right corner and lower left corner of the text area. Since the invoice is machine-printed, the size of the text in the same field is basically the same, that is, the spacing between characters is basically equal. Therefore, when the general text detection model only detects part of the text, the average length of each character is calculated to determine the length to be completed, so as to obtain the coordinates of the completed text area.

[0102] First, calculate the length of the recognized text, as shown in formula (1):

[0103] ; (1)

[0104] in, Represents the length of the recognized text, and formula (1) calculates the top right corner vertex With the upper left vertex

[0105] The Euclidean distance between .

[0106] Then calculate the average length of each character in the recognized text, as shown in formula (2):

[0107] ; (2)

[0108] in, Indicates the average length of each character in the recognized text; Indicates the length of the recognized text; Indicates the number of characters in the recognized text. For example, if the recognized text is "Name: XX (Tianjin)", then is 9.

[0109] After determining the number of characters to be padded, calculate the total length of the characters to be padded, as shown in formula (3):

[0110] ; (3)

[0111] in, Indicates the total length of characters to be padded; Indicates the average length of each character in the recognized text; Represents the number of characters of the complete text, Indicates the number of characters in the recognized text. Take the recognized text as "Name: XX (Tianjin)" and the complete text as "Name: XX (Tianjin) XXXX Co., Ltd." as an example. is 17, is 9, the number of characters to be filled is 8; Represents the error, for example, Can be set to 10.

[0112] Thus, the coordinates of the upper right corner of the complete text can be obtained, as shown in formula (4) and formula (5):

[0113] ; (4)

[0114] ; (5)

[0115] Similarly, the coordinates of the lower right corner of the complete text can be calculated using formulas (6) to (10):

[0116] ; (6)

[0117] ; (7)

[0118] ; (8)

[0119] ; (9)

[0120] ; (10)

[0121] in, Indicates the length of the recognized text; Indicates the number of characters of the recognized text; The number of characters representing the complete text; Indicates the average length of each character in the recognized text; Indicates the total length of characters to be padded; Indicates error.

[0122] In summary, the coordinates of the four vertices of the upper left corner, upper right corner, lower right corner and lower left corner of the complete text area are: .

[0123] Finally, the guide words in the recognized text are concatenated with the corresponding real text to obtain the complete text information. For example, the recognized text contains the guide word "Name:", and the corresponding real text is "XX (Tianjin) XXXX Co., Ltd.". The guide word and the real text are concatenated to obtain "Name: XX (Tianjin) XXXX Co., Ltd.", which is the complete text information.

[0124] In step S103, the invoice image is used as input, and the position information of the complete text area is used, as calculated above. For labels, build training data for text detection.

[0125] Based on the position information of the complete text area , crop the complete text area, take the complete text area as input, and use the complete text information, such as the "Name: XX (Tianjin) XXXX Co., Ltd." obtained above, as the label to construct text recognition training data.

[0126] Corresponding to the training data enhancement method for the invoice recognition model, the present invention also provides a training data enhancement system for the invoice recognition model, the system comprising:

[0127] The text extraction module is used to input the invoice image into the general text detection model to obtain all text areas in the invoice image; after cropping the text area, it is input into the general text recognition model to obtain the recognized text; all field values ​​of the corresponding invoice image are obtained through the ticketing system, and the recognized text is annotated to obtain the real text.

[0128] The text comparison and correction module is used to loop through the recognized texts and use the text matching method to match the real text corresponding to each recognized text; use the text matching and edit distance methods to determine the relative position of the recognized text in its corresponding real text, and complete the unrecognized text to obtain the complete text and calculate the position information of the complete text area; splice the guide words in the recognized text with its corresponding real text to obtain the complete text information.

[0129] The training data construction module is used to take the invoice image as input and use the position information of the complete text area as a label to construct training data for text detection; the complete text area is obtained by cropping according to the position information of the complete text area, the complete text area is taken as input, and the complete text information is used as a label to construct training data for text recognition.

[0130] Corresponding to the above method, the present invention also provides an electronic device, which includes a computer device, the computer device includes a processor and a memory, the memory stores computer instructions, the processor is used to execute the computer instructions stored in the memory, and when the computer instructions are executed by the processor, the electronic device implements the steps of the method described above.

[0131] The embodiment of the present invention also provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the steps of the aforementioned edge computing server deployment method are implemented. The computer-readable storage medium can be a tangible storage medium, such as a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a floppy disk, a hard disk, a removable storage disk, a CD-ROM, or any other form of storage medium known in the technical field.

[0132] It should be understood by those skilled in the art that the exemplary components, systems and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software or a combination of the two. Whether it is performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention. When implemented in hardware, it can be, for example, an electronic circuit, an application-specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present invention are programs or code segments used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link via a data signal carried in a carrier.

[0133] It should be clear that the present invention is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present invention.

[0134] In the present invention, features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, and / or combined with features of other embodiments or replace features of other embodiments.

[0135] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the embodiments of the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A training data enhancement method for an invoice recognition model, characterized in that: The method comprises the following steps: Input the invoice image into a general text detection model to obtain all text areas in the invoice image; after cropping the text area, input it into a general text recognition model to obtain recognized text; obtain all field values ​​of the corresponding invoice image through the ticketing system, annotate the recognized text, and obtain real text; wherein the real text does not contain the guide words on the invoice image; The recognized texts are traversed in a loop, and the real texts corresponding to the recognized texts are matched by a text matching method; the relative position of the recognized texts in the corresponding real texts is determined by a text matching and edit distance method, and the unrecognized texts are completed to obtain the complete texts, and the position information of the complete text area is calculated; the guide words in the recognized texts are spliced ​​with the corresponding real texts to obtain the complete text information; The invoice image is used as input, and the position information of the complete text area is used as a label to construct training data for text detection; the complete text area is obtained by cropping according to the position information of the complete text area, and the complete text area is used as input, and the complete text information is used as a label to construct training data for text recognition.

2. The training data enhancement method for invoice recognition model according to claim 1 is characterized in that: Input the invoice image into the general text detection model to obtain all text areas in the invoice image; after cropping the text area, input it into the general text recognition model to obtain the recognized text, including: When the general text detection model is used to detect text regions, there are cases where a complete text region is detected, a partial text region is detected, and a text region is fragmented; When the recognized text is obtained using the universal character recognition model, the recognized text may contain typos and missing characters.

3. The training data enhancement method for invoice recognition model according to claim 1, characterized in that: Obtain all field values ​​of the corresponding invoice image through the ticketing system and annotate the identified text, including: Correct the invoice number, invoice code, invoice date, price and tax total and check code in the identified text.

4. The training data enhancement method for invoice recognition model according to claim 1, characterized in that: The recognized texts are traversed in a loop, and the real text corresponding to each recognized text is matched by a text matching method, including: Acquire the recognized text and the real text; Loop through the recognized text, compare the recognized text with the real text, and calculate the ratio of the number of characters in the intersection of the two to the recognized text; If the ratio is greater than a preset threshold, the match is successful.

5. The training data enhancement method for invoice recognition model according to claim 1, characterized in that: Using the edit distance method, the relative position of the recognized text in its corresponding real text is determined, and the unrecognized text is completed to obtain a complete text, including: Initialize a matrix, wherein the rows of the matrix represent the characters of the recognized text and the columns represent the characters of the real text; calculate the edit distance between each character step by step through a dynamic programming algorithm to obtain the minimum edit distance between the recognized text and the real text; According to the minimum edit distance, the longest common substring of the recognized text and the real text is obtained; the longest common substring represents the matching area of ​​the recognized text and the real text; based on the longest common substring, the relative position of the recognized text in its corresponding real text is determined, and the unrecognized text is completed.

6. The training data enhancement method for invoice recognition model according to claim 1, characterized in that: Calculate the position information of the complete text area, including: Obtaining text area position information of the recognized text; Calculating the average length of each character in the recognized text; Calculating the total length of the characters to be padded according to the relative position of the recognized text in the corresponding real text; The position information of the complete text area is calculated according to the text area position information of the recognized text and the total length of the characters to be padded, and the coordinate information of the four vertices of the complete text area is obtained.

7. The training data enhancement method for invoice recognition model according to claim 6, characterized in that: When the recognized text is on the left side of the real text, the method for calculating the coordinate information of the complete text area includes: ； ； ； ； ； Similarly, ； ； ； ； ； The coordinates of the four vertices of the upper left corner, upper right corner, lower right corner and lower left corner of the complete text area are: ； in, The coordinates of the four vertices of the upper left corner, the upper right corner, the lower right corner and the lower left corner of the text area of ​​the recognized text; and Indicates the length of the recognized text; Represents the number of characters of the recognized text; The number of characters representing the complete text; and Indicates the average length of each character of the recognized text; and Indicates the total length of characters to be padded; Indicates error.

8. A training data enhancement system for an invoice recognition model, characterized in that: The system comprises: The text extraction module is used to input the invoice image into the general text detection model to obtain all text areas in the invoice image; after cropping the text area, input it into the general text recognition model to obtain the recognized text; obtain all field values ​​of the corresponding invoice image through the ticketing system, annotate the recognized text, and obtain the real text; The text comparison and correction module is used to loop through the recognized texts, match the real texts corresponding to the recognized texts by using the text matching method; determine the relative position of the recognized texts in the real texts corresponding to them by using the text matching and edit distance methods, and complete the unrecognized texts to obtain the complete texts, and calculate the position information of the complete text area; splice the guide words in the recognized texts with the real texts corresponding to them to obtain the complete text information; The training data construction module is used to take the invoice image as input and the position information of the complete text area as a label to construct training data for text detection; the complete text area is obtained by cropping according to the position information of the complete text area, the complete text area is taken as input, and the complete text information is used as a label to construct training data for text recognition.

9. A computer-readable storage medium having a computer program / instruction stored thereon, characterized in that: When the computer program / instructions are executed by a processor, the steps of the method as claimed in any one of claims 1 to 7 are implemented.

10. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of the method as claimed in any one of claims 1 to 7 are implemented.

Images