Letter of credit identification method, device, computer equipment and storage medium
Through the angle correction, text area segmentation and post-processing technology of letter of credit identification method, the problem of inaccurate letter of credit identification in the existing technology is solved, the accurate extraction and automatic identification of structured letter of credit information is achieved, and manual entry is reduced privacy risks.
Patent Information
- Application Number
- CN202111632352.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-12-28
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2041-12-28
AI Technical Summary
Existing letter of credit identification technology cannot accurately extract complex letter of credit layout and long text, and cannot accurately locate structured information in letter of credit, resulting in inaccurate identification and the risk of privacy leakage of manual entry.
Provide a method for identifying letters of credit, obtaining images of letters of credit to be identified, performing angle correction, text area segmentation and text recognition, combining preset rules for post-processing, extracting structured target information, and using paragraph classification model to distinguish paragraphs to achieve the purpose of Accurate extraction of structured information in letters of credit.
Accurate and efficient automatic identification of letters of credit, avoid privacy leakage during manual entry, and improve identification efficiency and accuracy.
Smart Images

Figure CN114463767B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of image recognition technology, and in particular to a letter of credit recognition method, device, computer equipment and storage medium. Background Art
[0002] Since the application scenarios of letters of credit are relatively special, there are almost no dedicated recognition engines for letters of credit in the industry. Existing letter of credit recognition usually adopts the method of manual entry and manual verification, or uses a general text recognition engine, such as the tesseract engine, or a general OCR recognition model based on deep learning.
[0003] However, since the information extracted from the letter of credit format is complex and text-intensive, and it is necessary to determine whether the checkbox is checked or blacked out, and there are two formats: tabular and non-tabular, the existing recognition technology solutions cannot accurately extract the complex structure of the letter of credit format, nor can they accurately locate the long text of the letter of credit. Summary of the invention
[0004] Based on this, it is necessary to provide a letter of credit identification method, device, computer equipment, computer readable storage medium and computer program product that can accurately identify letter of credit images in response to the above technical problems.
[0005] In a first aspect, the present application provides a method for identifying a letter of credit, the method comprising:
[0006] Acquire the letter of credit image to be identified;
[0007] Segment the letter of credit image to be recognized to obtain several text regions;
[0008] Perform text recognition on the text area to obtain the recognition result of the text area;
[0009] The recognition result of the text area is post-processed according to the preset rules to obtain the structured target information. In one embodiment, after obtaining the image of the letter of credit to be recognized, it also includes:
[0010] Perform angle correction on the letter of credit image to be recognized.
[0011] In one embodiment, the angle correction of the image of the letter of credit to be recognized includes:
[0012] Binarizing the letter of credit image to be identified to obtain a binary image;
[0013] Extract all table contours in the binary image;
[0014] Process all table contours to obtain the target contour of the table with the largest area;
[0015] Calculate the angle between adjacent straight lines in the target contour of the table with the largest area to obtain the rotation angle;
[0016] The image of the letter of credit to be identified is corrected according to the rotation angle.
[0017] In one embodiment, all table outlines are processed to obtain a target outline of a table with a maximum area, including:
[0018] Filter according to the contour area and obtain the table with the smallest contour area as the current table;
[0019] Get the pending table adjacent to the current table;
[0020] The to-be-processed table and the current table are merged as a new current table, and the to-be-processed tables adjacent to the current table are continuously obtained until the table with the largest area is obtained, and the target contour of the table with the largest area is extracted.
[0021] In one embodiment, before performing text recognition on a text area, the process includes:
[0022] Make repairs to the text area.
[0023] In one embodiment, repairing the text area includes:
[0024] Obtain the format corresponding to the letter of credit image to be identified;
[0025] Repair the text area according to the layout.
[0026] In one embodiment, repairing the text area according to the layout includes:
[0027] When the letter of credit image to be recognized is in a table format, extracting the boundary to be processed in the target direction in the text area;
[0028] Obtain the boundary to be processed that is closest to the text area of the target type in the text area as the target boundary;
[0029] Moves the bounds of the text area of the target type to the target bounds.
[0030] In one embodiment, repairing the text area according to different layouts includes:
[0031] When the letter of credit image to be recognized is in a non-tabular format, obtaining the coordinates of the text area of the target type in the text area;
[0032] Segment the letter of credit image to be identified according to the coordinates and the width of the letter of credit image to be identified, and obtain text slices corresponding to the text area of the target type;
[0033] Perform corresponding image operations on different directions of the text slices to obtain the content contour;
[0034] Repair the text area of the target type according to the horizontal coordinate of the content outline.
[0035] In one embodiment, post-processing the recognition result of the text area according to a preset rule to obtain structured target information includes:
[0036] Extracting a set of candidate fields in the recognition result according to a preset layout structure;
[0037] Matching is performed according to the preset configuration rules corresponding to the preset layout to obtain the field position and field content corresponding to each field in the candidate field set;
[0038] Obtain a candidate set of field information through the positional relationship between fields and field information;
[0039] Filter the field information candidate set to obtain the target information.
[0040] In one embodiment, before extracting the candidate field set in the recognition result according to the preset layout structure, the method further includes:
[0041] Get the layout distinguishing field;
[0042] The recognition result is matched with the format distinguishing field to determine the format of the letter of credit image to be recognized.
[0043] In one embodiment, before obtaining the field information candidate set based on the positional relationship between the field and the field information, the method further includes:
[0044] When the letter of credit image to be identified is a plurality of letter of credit images, the plurality of letter of credit images are merged according to the order in which the plurality of letter of credit images appear.
[0045] In one embodiment, filtering the field information candidate set before obtaining the target information further includes:
[0046] When the field information is target field information, the field information is input into the paragraph classification model; and the target field information belonging to the same paragraph is extracted through the paragraph classification model.
[0047] In one embodiment, the paragraph classification model is obtained by pre-training, and the training process of the paragraph classification model includes:
[0048] Get the text of the letter of credit paragraph;
[0049] Combine and mark the letter of credit paragraph text according to a preset method to obtain letter of credit training data;
[0050] Preprocessing the letter of credit training data to obtain preprocessed letter of credit training data;
[0051] The preprocessed letter of credit training data is input into the first machine learning model for training to obtain a paragraph classification model.
[0052] In a second aspect, the present application further provides a letter of credit identification device, the device comprising:
[0053] An acquisition module, used for acquiring the image of the letter of credit to be identified;
[0054] A text region detection module is used to segment the letter of credit image to be identified to obtain a number of text regions;
[0055] A recognition module is used to perform text recognition on the text area and obtain a recognition result of the text area;
[0056] The post-processing module is used to post-process the recognition results of the text area according to preset rules to obtain structured target information.
[0057] In a third aspect, the present application further provides a computer device, which includes a memory and a processor, wherein the memory stores a computer program, and the processor implements the method in any one of the above embodiments when executing the computer program.
[0058] In a fourth aspect, the present application further provides a computer-readable storage medium, wherein a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the method in any one of the above embodiments is implemented.
[0059] In a fifth aspect, the present application further provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the method in any one of the above embodiments is implemented.
[0060] The above-mentioned letter of credit recognition method, device, equipment, computer storage medium and program product first obtain the letter of credit image to be recognized, then segment the letter of credit image to be recognized to obtain several text areas, and perform text recognition on the text areas to obtain the results to be recognized in the text areas, and finally post-process the recognition results of the text areas according to preset rules to obtain structured target information, thereby realizing accurate and efficient automatic extraction of structured target information in the letter of credit, and avoiding privacy leakage in the manual input process. BRIEF DESCRIPTION OF THE DRAWINGS
[0061] Figure 1 A diagram of an application environment of a method for identifying a letter of credit in an embodiment;
[0062] Figure 2 A schematic diagram of a flow chart of a method for identifying a letter of credit in one embodiment;
[0063] Figure 3 A schematic diagram of a letter of credit image to be identified in an embodiment;
[0064] Figure 4 A schematic diagram of letter of credit image correction in one embodiment;
[0065] Figure 5 A schematic diagram of a process of correcting the angle of a letter of credit in one embodiment;
[0066] Figure 6 A schematic diagram of inaccurate text area detection in one embodiment;
[0067] Figure 7 is a schematic diagram of a boundary to be processed in a target direction in one embodiment;
[0068] Figure 8 A schematic diagram of repairing a text area of a table-format letter of credit in one embodiment;
[0069] Fig. 9 A schematic diagram of a repair process for a text area of a table-format letter of credit in one embodiment;
[0070] Fig.10 A schematic diagram of vertical binarization in one embodiment;
[0071] Fig.11 A schematic diagram of text slices of a text area of a target type in one embodiment;
[0072] Fig.12 A schematic diagram of a lateral closing operation in one embodiment;
[0073] Fig.13 A schematic diagram of a repair process for a text area of a non-tabular letter of credit in one embodiment;
[0074] Fig.14 A schematic diagram of extracting a candidate field set from a recognition result in one embodiment;
[0075] Fig.15 A schematic diagram of rule configuration of a field in an embodiment;
[0076] Fig.16 A schematic diagram of collecting letter of credit corpus in one embodiment;
[0077] Fig.17 A schematic diagram of a letter of credit training set data in an embodiment;
[0078] Fig.18 A schematic diagram of training data preprocessing in one embodiment;
[0079] Fig.19A schematic diagram of model training of a paragraph classification model in one embodiment;
[0080] Fig. 20 A flowchart of a method for identifying a letter of credit in one embodiment;
[0081] Fig.21 A schematic diagram of a post-processing process in an embodiment;
[0082] Fig. 22 A structural block diagram of a letter of credit identification device in one embodiment;
[0083] Fig.23 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION
[0084] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.
[0085] The letter of credit identification method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers. Obtain the letter of credit image to be identified; segment the letter of credit image to obtain several text areas; perform text recognition on the several text areas obtained by segmentation to obtain the recognition results of the text areas; finally, post-process the recognition results of the text areas according to preset rules to obtain structured target information. Among them, the terminal 102 can be but is not limited to various personal computers, laptops, smart phones, tablets, Internet of Things devices and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart car-mounted devices, etc. Portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The server 104 can be implemented with an independent server or a server cluster consisting of multiple servers.
[0086] In one embodiment, Figure 2 As shown, a method for identifying a letter of credit is provided, which is applied to Figure 1 Taking the server 104 in the example as an example, the following steps are included:
[0087] S202, obtaining the letter of credit image to be identified.
[0088] The letter of credit image to be identified refers to the letter of credit image that needs to be recognized, which can be obtained by scanning, taking a photo, etc. and uploaded to the server. The letter of credit is a special bill, which includes two long fields: description_goods and document_required. Figure 3 As shown, Figure 3 The figure is a schematic diagram of a letter of credit image to be identified in an embodiment, in which the long field document_required includes multiple lines of text and multiple check boxes. Optionally, the server can select part or all of the bill images to be identified from the terminal as needed and upload them to the server, so that the server can identify the bill images to be identified according to the instructions, which can reduce the workload of manual input and verification, improve efficiency, and help ensure data security.
[0089] S204, segmenting the letter of credit image to be recognized to obtain a plurality of text regions.
[0090] The text area refers to the area corresponding to a text detection box in the letter of credit image to be identified. Specifically, Figure 3 As shown in the figure, the box outside each line of text is the text area.
[0091] Specifically, after the server obtains the letter of credit image to be identified, it detects the text area in the letter of credit image to be identified, and segments the detected text area to obtain a number of text areas. In other embodiments, the text area in the letter of credit image to be identified can be identified by a pre-trained text detection model, and the text area can be segmented to obtain a number of text areas.
[0092] S206, performing text recognition on the text area to obtain a recognition result of the text area.
[0093] The recognition result refers to the text content obtained by performing text recognition on the text area. For example, if the text area includes three words "letter of credit", the recognition result of the text area is letter of credit.
[0094] Specifically, after obtaining the plurality of text regions, the server recognizes the text regions to obtain recognition results of the text regions. In other embodiments, the server may input the text regions into a pre-trained text recognition model, and recognize the text regions through the text recognition model to obtain recognition results corresponding to the text regions.
[0095] S208: Post-process the recognition result of the text area according to preset rules to obtain structured target information.
[0096] Among them, the preset rule refers to the rule for post-processing the recognition structure of the pre-set text area, which can be set according to the specific usage scenario. After post-processing the recognition result of the text area according to the preset rule, the structured target information can be obtained, wherein the target information refers to the information to be extracted, for example, it can be the content corresponding to the document_required field in the letter of credit image to be identified. The structured target information refers to the target information that conforms to the format of the preset rule, such as "Name: Zhang San".
[0097] Specifically, after obtaining the recognition results of each text area, the server will post-process the recognition results according to the preset rules. For example, the post-processing can be to extract the texts with the same content in the letter of credit, that is, the long fields in the letter of credit as a paragraph, and finally output the structured target information. In other embodiments, the post-processing can also be to perform post-processing verification on the structured target information, and unnecessary information in the structured target information can be filtered, and finally the structured target information is output.
[0098] In the above-mentioned letter of credit recognition method, the image of the letter of credit to be recognized is first obtained, and then the image of the letter of credit to be recognized is segmented to obtain several text areas, and text recognition is performed on the text areas to obtain the results to be recognized in the text areas. Finally, the recognition results of the text areas are post-processed according to preset rules to obtain structured target information, thereby realizing accurate and efficient automatic extraction of structured target information in the letter of credit, and avoiding privacy leakage in the manual input process.
[0099] In one embodiment, after acquiring the image of the letter of credit to be identified, the method further includes: performing angle correction on the image of the letter of credit to be identified.
[0100] Among them, the angle correction is to rotate the rotated letter of credit image to be identified so as to meet the standard for processing the letter of credit image to be identified.
[0101] Specifically, after obtaining the image of the letter of credit to be identified, it is also necessary to perform image correction on the image of the letter of credit to be identified so that the rotation angle of the image of the letter of credit to be identified meets the standard for processing the image of the letter of credit to be identified, wherein the angle correction can be performed using radial transformation according to the coordinates of the corner points of the largest table in the image of the letter of credit to be identified. Figure 4 As shown, Figure 4 Schematic diagram of letter of credit image correction in one embodiment, in which the left side of the figure is the letter of credit image to be identified which needs angle correction, and the right side is the letter of credit image to be identified after angle correction is completed.
[0102] In the above embodiment, by performing angle correction on the image of the letter of credit to be identified, the image of the letter of credit to be identified can be better processed for the next step.
[0103] In one embodiment, the angle correction of the letter of credit image to be identified includes: binarizing the letter of credit image to be identified to obtain a binarized image; extracting all table contours in the binarized image; processing all table contours to obtain a target contour of the table with the largest area; calculating the angle between adjacent straight lines in the target contour of the table with the largest area to obtain a rotation angle; and performing angle correction on the letter of credit image to be identified according to the rotation angle.
[0104] Among them, binarization refers to setting the grayscale value of the pixel points on the image to 0 or 255, that is, the process of making the entire image present an obvious black and white effect. The binarized image refers to the image obtained after binarization processing of the letter of credit image to be identified; the target contour refers to the table contour obtained after processing the table contour, for example, it can be the table contour obtained by merging adjacent table frames.
[0105] Specifically, firstly, the image of the letter of credit to be identified is binarized to obtain a binary image corresponding to the image of the letter of credit to be identified, so as to facilitate the extraction of the table contour; then, all the table contours in the binary image are extracted to obtain all the table contours in the binary image; after obtaining the table contours, the contour areas of all the tables can be obtained, and optionally, the contour areas of the tables can be calculated by the table contours, for example, by using the length and width of the table contours for calculation; further, all the contour areas are processed to obtain the target contour of the table with the largest area, wherein optionally, the target contour of the table with the largest area can be obtained by merging adjacent tables; the rotation angle is calculated by calculating the angle between adjacent straight lines in the target contour of the table with the largest area, wherein optionally, the rotation angle can be calculated by calculating the angle between adjacent straight lines in the target contour by the Pythagorean theorem; finally, the angle of the letter of credit image to be identified is corrected according to the calculated rotation angle.
[0106] In the above embodiment, after calculating the target contour of the largest table by calculating all the table contours, the angle between adjacent straight lines in the target contour of the largest table is calculated to obtain the rotation angle, and finally the angle correction of the letter of credit image to be recognized is performed according to the rotation angle. In this way, the tilted letter of credit image in the actual scene can be corrected to avoid the phenomenon of inaccurate regression in the subsequent processing of the letter of credit image.
[0107] In one embodiment, all table contours are processed to obtain a target contour of a table with the largest area, including: filtering according to contour area to obtain a table with the smallest contour area as the current table; obtaining a table to be processed adjacent to the current table; merging the table to be processed with the current table as a new current table, and continuing to obtain tables to be processed adjacent to the current table until the table with the largest area is obtained, and extracting the target contour of the table with the largest area.
[0108] The contour area is the area calculated based on the table contour. Specifically, after all the table contours in the binary image are obtained, the contour area is calculated based on the table contours. Optionally, after all the table contours are obtained, the contour area of each table can be calculated based on the length and width of the table contours. The current table refers to the table with the smallest contour area; the table to be processed refers to the table adjacent to the current table that needs to be merged.
[0109] Specifically, after all the table contours in the binary image are obtained, the contour area is calculated according to the table contour, and filtering is performed according to the contour area to obtain the table with the smallest contour area as the current table. Then, the table to be processed adjacent to the current table, i.e., the table with the smallest contour area, is obtained, wherein optionally, the contour can be filtered in parallel according to the upper and lower and left and right table lines to obtain the table to be processed adjacent to the current table; the current table and the table to be processed are merged to obtain a new current table, and the table to be processed adjacent to the current table is continuously obtained, and the current table and the table to be processed adjacent to the current table are merged until the table with the largest area is obtained, and finally the target contour of the table contour with the largest area is extracted.
[0110] In other embodiments, specifically combined with Figure 5 As shown, Figure 5 The present invention is a flowchart of angle correction of a letter of credit in an embodiment. First, the image of the letter of credit to be identified is input, and the image of the letter of credit to be identified is binarized to obtain a binarized image; the binarized image is filtered to obtain all table contours, the contour area is calculated based on all table contours, and filtering is performed based on the contour area to obtain the contour area with the smallest area; then filtering is performed based on the parallelism of the upper and lower and left and right table lines to obtain the current table, i.e., the table to be processed adjacent to the contour area with the smallest area, the current table is merged with the table to be processed to obtain a new current table, and filtering is continued based on the parallelism of the upper and lower and left and right table lines to obtain the new table to be processed adjacent to the current table, and the new current table is merged with the table to be processed until the table with the largest area is obtained, and finally the rotation angle is calculated through the angle between the adjacent straight lines of the table, and the angle correction of the letter of credit image to be identified is performed based on the rotation angle.
[0111] In the above embodiment, the rotation angle is calculated by calculating the angle between adjacent straight lines in the maximum table, so as to obtain the rotation angle of the letter of credit image to be processed, and then perform angle correction on the letter of credit image to be processed.
[0112] In one embodiment, before performing text recognition on a text region, the text region is repaired.
[0113] Among them, repair refers to adjusting the width or width of the text area so that the text area can include the complete content to be recognized, so as to avoid extraction errors when performing subsequent text recognition on the text area.
[0114] Specifically, since there are many long and dense text lines in the letter of credit image, inaccurate recognition may occur in the process of identifying the text area in the letter of credit image. For example, the beginning and end of the text line may be missed or the recognition may be wrong, which may cause errors in the subsequent text recognition in the text area. Figure 6 As shown, Figure 6 This is a schematic diagram of inaccurate text area detection in an embodiment. The first character of the text line is missing at the place indicated by the arrow in the figure. Therefore, the text area needs to be repaired before character recognition is performed on the text area to make the subsequent character recognition results in the text area more accurate. Optionally, different repair methods can be used for letters of credit in tabular and non-tabular formats to ensure more accurate text area repair.
[0115] In the above embodiment, repairing the text area can make the result of subsequent character recognition on the text area more accurate.
[0116] In one embodiment, repairing the text area includes: obtaining a layout corresponding to the letter of credit image to be identified; and repairing the text area according to the layout.
[0117] The layout refers to the layout corresponding to the letter of credit image to be identified. All letters of credit are generated according to the corresponding layout, which can be a tabular layout or a non-tabular layout.
[0118] Specifically, the server first obtains the layout corresponding to the image of the letter of credit to be identified, and repairs it in different ways according to the layout corresponding to the letter of credit. For example, the image of the letter of credit to be identified in a tabular layout can be repaired according to the table line boundary, and the image of the letter of credit to be identified in a non-tabular layout can be repaired according to the contour search method based on the text line. Specifically, before obtaining the image of the letter of credit to be identified, the server will judge the layout of the image of the letter of credit to be identified to determine the layout corresponding to the image of the letter of credit to be identified and use the corresponding repair method to repair the text area to be repaired in the image of the letter of credit to be identified. Optionally, the layout of the image of the letter of credit to be identified can be distinguished according to the length of the field and whether it is a table inside the image of the letter of credit to be identified. In other embodiments, it is also possible to judge whether the image of the letter of credit to be identified is a letter of credit in a tabular layout by judging whether the image of the letter of credit to be identified includes a large number of table boundary lines, otherwise it is a letter of credit in a non-tabular layout.
[0119] In the above embodiment, by acquiring the format corresponding to the letter of credit image to be identified, the text area is repaired in different ways to achieve accurate repair of the text area.
[0120] In one embodiment, the text area is repaired according to the layout, including: when the letter of credit image to be identified is in a table layout, extracting the to-be-processed boundary in the target direction in the text area; obtaining the to-be-processed boundary closest to the text area of the target type in the text area as the target boundary; and moving the boundary of the text area of the target type to the target boundary.
[0121] The target direction refers to a direction set according to a preset rule, which can be horizontal or vertical, and can be selected according to a specific usage scenario. In this embodiment, vertical is selected as the target direction because there are many long and dense text lines in the image of the letter of credit to be identified, so the beginning and the end may be missed, so using vertical as the target direction can avoid the phenomenon of missing the beginning and the end. In other embodiments, horizontal or both horizontal and vertical directions can be selected as the target direction according to actual conditions.
[0122] Among them, the boundary to be processed refers to any table frame line in the letter of credit image to be processed; the text area of the target type refers to a text area that meets the preset requirements, for example, it can be a text area with a text area width greater than a preset threshold, that is, a long text area, and the target boundary refers to the boundary line of the target type area being moved to the boundary that meets the preset requirements, such as the boundary of the nearest vertical table line of the long field.
[0123] Specifically, when the letter of credit image to be recognized is in a table format, the target direction in the text area is extracted as the boundary line to be processed. Figure 7 As shown, Figure 7 is a schematic diagram of a boundary to be processed in the target direction in one embodiment, Figure 7 The continuous and dark vertical straight line in the text area is the boundary line to be processed. Then, before obtaining the boundary to be processed closest to the text area of the target type in the text area as the target boundary, the text area of the target type is also obtained, wherein optionally, a threshold can be pre-set. If the width of the current text area is greater than the preset threshold, the current text area is determined to be the text area of the target type. Then, the boundary to be processed closest to the text area of the target type is obtained as the target boundary, and the boundary of the text area of the target type is moved to the target boundary. Figure 8 As shown, Figure 8 The figure is a schematic diagram of repairing the text area of the table-type letter of credit in one embodiment. In other embodiments, the letter of credit image to be identified may be firstly subjected to binarization processing, and then the boundary to be processed in the target direction of the text area may be extracted.
[0124] In other embodiments, in combination Fig. 9 As shown, Fig. 9 FIG. 1 is a schematic diagram of a repair process of a text area of a table-type letter of credit in an embodiment. First, the letter of credit image to be recognized is binarized, and then the vertical core structure is expanded and eroded to obtain a vertical binary image. Fig.10 As shown, Fig.10 The figure is a schematic diagram of vertical binarization in an embodiment; then, whether the text area belongs to a long field text area is determined according to the width of the text area, wherein optionally, a threshold value can be pre-set, and if the width of the text area is greater than the threshold value, it is determined to be a long field text area; according to the boundary of the nearest vertical table line of the long field text area, the coordinates of the two ends of the long field are repaired as the vertical boundary position.
[0125] In the above embodiment, by moving the boundary of the text area of the target type to the target boundary, the table format letter of credit image to be identified is repaired, which can avoid the phenomenon of missing the beginning and end of the text line or incorrect identification, so as to ensure the subsequent accurate identification of the text area.
[0126] In one embodiment, the text area is repaired according to different layouts, including: when the letter of credit image to be identified is a non-tabular layout, obtaining the coordinates of the text area of the target type in the text area; segmenting the letter of credit image to be identified according to the coordinates and the width of the letter of credit image to be identified to obtain text slices corresponding to the text area of the target type; performing corresponding image operations on different directions of the text slices to obtain content contours; and repairing the text area of the target type according to the horizontal coordinate of the content contour.
[0127] Specifically, when the letter of credit image to be identified is a letter of credit in a non-tabular format, firstly, the coordinates of the text area of the target type in the text area are obtained, wherein the determination of the text area of the target type can refer to the specific implementation of the above embodiment, and will not be described repeatedly here. Then, the coordinates of the text area of the target type in the text area are obtained, and the letter of credit image to be identified is segmented according to the coordinates of the text area of the target type and the width of the letter of credit image to be identified, and the text slice corresponding to the text area of the target type is obtained. Specifically, the text slice corresponding to the text area of the target type is obtained. Fig.11 As shown, Fig.11Schematic diagram of text slices of a target type of text area in one embodiment. Next, corresponding image operations are performed on the text slices of the target type of text area in different directions, wherein optionally, the text slices of the target type of text area can be opened in the vertical direction and closed in the horizontal direction to obtain the content outline in the text slices. In other embodiments, before corresponding image operations are performed on the text slices in different directions, the text slices can be binarized, and after obtaining the binarized text slices, corresponding image operations are performed on the text slices in different directions. Specifically combined Fig.12 , Fig.12 FIG. 1 is a schematic diagram of a horizontal closing operation in an embodiment, in which the white area is the content outline. Finally, the target type of text area is repaired according to the horizontal coordinate of the content outline, and the repair can be performed according to the minimum and maximum horizontal coordinates of the content outline.
[0128] In other embodiments, in combination Fig.13 As shown, Fig.13 The figure is a flowchart of repairing text area of non-tabular letter of credit in one embodiment. First, the coordinate position of the text area of the target type is obtained, and the coordinate position (location[0], location[1], location[2], location[3]) is recorded, and the slice data is cut out according to the width of the original image and the height of the text area of the target type; then, the vertical opening operation and the horizontal closing operation are performed on the text area of the target type respectively. Specifically, the vertical opening operation is used for the case where the text and the vertical table line are connected together, and the vertical table line is removed; the horizontal closing operation is used for the horizontal large core structure to perform a horizontal closing operation, and the text area content is connected into blocks for contour detection, so as to obtain the text lines of this text area, such as Fig.12 As shown, after obtaining the content contour by performing corresponding image operations on different directions of the text slice, the minimum horizontal coordinate and the maximum horizontal coordinate are found as the boundary of the long text, and finally, the minimum and maximum horizontal coordinates (min_x, location[1], max_x, location[2]) of the above contour are statistically corrected.
[0129] In the above embodiment, by performing image operations in different directions on the text slice corresponding to the target type, the content outline in the text slice can be obtained, and finally the text area of the target type can be repaired according to the horizontal coordinate of the content outline to ensure that the text area can be accurately identified subsequently.
[0130] In one embodiment, the recognition results of the text area are post-processed according to preset rules to obtain structured target information, including: extracting a set of candidate fields in the recognition results according to a preset layout structure; matching according to preset configuration rules corresponding to the preset layout to obtain the field position and field content corresponding to each field in the field set; obtaining a candidate set of field information through the positional relationship between the field and the field information; and filtering the candidate set of field information to obtain the target information.
[0131] Among them, the preset layout refers to the preset letter of credit layout, which can be a letter of credit in a tabular layout or a letter of credit in a non-tabular layout; the candidate field set refers to the set of all fields in the letter of credit image to be identified; the preset configuration rule refers to the preset field matching rule, through which the field position and field content corresponding to each field in the candidate field set can be obtained, and the field information refers to the content corresponding to the field, for example, "Name: Zhang San", where the field is "Name" and the field information is "Zhang San"; the field information candidate set refers to the set of field information corresponding to a certain field in the letter of credit image to be identified; the target information refers to a field information of a person in the set of all field information that needs to be extracted.
[0132] Specifically, the server first extracts a set of candidate fields in the recognition result according to a preset layout structure. For example, in a letter of credit in a table layout, the left side is mostly a field, and the right side is the corresponding field information. The set of candidate fields can be extracted according to the left-right parallel relationship. Optionally, before extracting the set of candidate fields in the recognition result according to the preset layout structure, the layout of the letter of credit image to be recognized can be determined by matching the recognition result with the layout distinguishing field, and then extracting the set of candidate fields in the recognition result. Fig.14 As shown, Fig.14 A schematic diagram of extracting a candidate field set from a recognition result in an embodiment. Fig.14 The format of the letter of credit in the table format letter of credit can be extracted to obtain a set of candidate fields based on the left-right parallel relationship.
[0133] Specifically, after obtaining the candidate field combination, matching is performed according to the preset setting rules corresponding to the preset layout to obtain the field position and field content corresponding to each field in the candidate field set. For example, if the letter of credit image to be identified is in a table layout, the preset configuration rules corresponding to the table layout are used for matching, otherwise the preset configuration rules corresponding to the non-table layout are used for matching. In one embodiment, the preset setting rules can be regular searches based on field configurations. Regular searches based on field configurations are matched according to the scores corresponding to the rule content, wherein the rule content is all the fields used in the letter of credit, and the scores corresponding to the rule content are predetermined. Optionally, the rule content with the highest score can be matched first. When traversing all the rule contents, the rule content with the highest score is traversed first. If the field in the candidate field set matches the rule content, no matching is required. Otherwise, matching is required according to the rule content with a lower score. Specifically, Fig.15 , Fig.15 A schematic diagram of the rule configuration of a field in an embodiment. In other embodiments, the preset setting rule may be a matching method based on the full text and full name. If the regular search based on the field configuration does not match the corresponding field, a text similarity search will be performed based on the matching method of the full text and full name to find the most likely field as a benchmark.
[0134] Specifically, after obtaining the field position and field content, the candidate set of field information corresponding to the field can be obtained through the positional relationship between the field and the field information. Since the distribution pattern of fields and field information in the letter of credit can be divided into top-bottom structure, left-right structure, and non-top-bottom structure, the candidate field information set can be matched and searched based on the candidate field position information by vertical search, horizontal search, and value content search.
[0135] Specifically, after obtaining the field information candidate set, the field information candidate set can be filtered to obtain the target information. Optionally, the filtering method can be based on rules. For example, there are fixed rules for dates, amounts, etc., so filtering can be performed according to the rules of some fields. In one embodiment, filtering can be performed by field value, because the field information candidate set may mistakenly extract other field keywords, and this part obviously does not need to be extracted, so it can be determined whether there are these fields in the field information candidate set, and if so, filtering can be performed. Optionally, before filtering the field information candidate set, long fields belonging to the same content in the field information candidate set can also be extracted as paragraphs.
[0136] In the above embodiment, a set of candidate fields in the recognition result is extracted through a preset layout structure, and matched according to the preset configuration rules corresponding to the preset layout to obtain the field position and field content corresponding to each field in the candidate field set, and then the field information candidate set is obtained through the positional relationship between the field and the field information, and finally the field information candidate set is filtered to obtain the target information, which can achieve accurate extraction of the target information and solve problems such as multiple layouts of letters of credit and paragraph distinction.
[0137] In one embodiment, before extracting the candidate field set in the recognition result according to the preset layout structure, it also includes: obtaining the layout distinguishing field; matching the recognition result with the layout distinguishing field to determine the layout of the letter of credit image to be recognized.
[0138] The format distinguishing field refers to a field used to distinguish between a tabular format letter of credit and a non-tabular format letter of credit. Specifically, the format distinguishing field is first obtained, and then the text area is subjected to text recognition to obtain a recognition result and matched with the format distinguishing field to determine the format of the letter of credit image to be recognized. For example, a field that does not exist in a non-tabular format letter of credit can be used as a field for distinguishing between a tabular format letter of credit and a non-tabular identification letter of credit. If the recognition result includes the field, the letter of credit image to be recognized is judged to be a tabular format letter of credit, otherwise it is a non-tabular format letter of credit.
[0139] In the above implementation, the layout of the letter of credit image to be identified can be determined by matching the layout-distinguishing fields, and the corresponding preset configuration rules are selected for matching according to the different layouts of the letter of credit images to be identified to obtain the field position and field content corresponding to each field in the candidate field set.
[0140] In one embodiment, before obtaining the field information candidate set through the positional relationship between the fields and the field information, the method further includes: when the letter of credit image to be identified is a plurality of letter of credit images, merging the plurality of letter of credit images according to the order in which the plurality of letter of credit images appear.
[0141] Specifically, since there are multiple pages of the letter of credit image to be identified, especially the letter of credit image to be identified in a non-tabular format, when the letter of credit image to be identified is multiple letter of credit images, that is, there are multiple pages, therefore, by using the positional relationship between the fields and the field information, before obtaining the field information candidate set, it is necessary to merge the multiple letter of credit images according to the order of their appearance.
[0142] In the above embodiment, structured extraction can be achieved by merging multiple letter of credit images.
[0143] In one embodiment, filtering the field information candidate set to obtain the target information also includes: when the field information is the target field information, inputting the field information into a paragraph classification model; and extracting the target field information belonging to the same paragraph through the paragraph classification model.
[0144] The target field information is the field information that meets the conditions in the field information candidate set. For example, when the preset condition is whether the description_goods corresponds to the document_required field, the target field information is a long field.
[0145] Specifically, when the field information is target field information, the field information is input into a paragraph classification model, wherein the paragraph classification model is a pre-trained machine learning model that can extract texts with line breaks and belonging to the same content as a paragraph. Through the paragraph classification model, the target field information in the field information candidate set can be extracted as a paragraph.
[0146] In the above embodiment, the target field information in the field information candidate set can be extracted as a paragraph through the paragraph classification model, so as to solve the problem of paragraph distinction that cannot be solved in the related art.
[0147] In one embodiment, the paragraph classification model is obtained through pre-training, and the training process of the paragraph classification model includes: obtaining the paragraph text of the letter of credit; combining and marking the paragraph text of the letter of credit according to a preset method to obtain letter of credit training data; preprocessing the letter of credit training data and expanding the letter of credit training data; inputting the preprocessed letter of credit training data into the first machine learning model for training to obtain the paragraph classification model.
[0148] The paragraph text of the letter of credit is based on two long fields of the real letter of credit. The text belonging to a paragraph is collected, and the line break is used to distinguish whether it is the same paragraph. Fig.16 As shown, Fig.16 A schematic diagram of collecting letter of credit corpus in an embodiment.
[0149] Specifically, first obtain the paragraph text of the letter of credit, and then combine the paragraph text of the letter of credit according to a preset method, wherein optionally, the paragraph text of the letter of credit can be randomly combined, and all the paragraph texts can be marked, for example, the text belonging to the same paragraph is marked as 1, otherwise it is marked as 0, so as to obtain the training data of the letter of credit. Fig.17 As shown, Fig.17 Schematic diagram of letter of credit training set data in one embodiment.
[0150] Specifically, the letter of credit training data is preprocessed, for example, the preprocessing can adopt a random replacement strategy to improve the generalization of the model. This is because the letter of credit text terms are professional, and the training data obtained by processing the two long field corpus of the real letter of credit will make the model very easy to fit, so the training data must be preprocessed. In other embodiments, a random replacement strategy is adopted to achieve the purpose of automatic expansion of the training corpus, in which 80% is randomly replaced [mask], 10% is randomly replaced with words, and 10% remains unchanged. Fig.18 As shown, Fig.18 FIG. 4 is a schematic diagram of training data preprocessing in one embodiment.
[0151] Specifically, the preprocessed letter of credit training data is input into the first machine learning model for training, wherein the first machine learning model refers to a machine learning model that can judge whether adjacent lines of text are the same paragraph, wherein the Bert language model can be optionally used. The first machine learning model can learn a large amount of letter of credit training data to obtain a paragraph classification model that can classify whether adjacent text lines in the recognition result are the same paragraph. Specifically combined Fig.19 As shown, Fig.19 The figure is a schematic diagram of model training of a paragraph classification model in one embodiment, wherein the first machine learning model is selected as the Bert language model.
[0152] In the above embodiment, the generalization of the model can be improved by preprocessing the letter of credit training data, and a paragraph classification model can be obtained by training the letter of credit training data to classify whether adjacent text lines in the recognition results are the same paragraph, so as to realize that texts with line breaks and belonging to the same content need to be extracted as a paragraph.
[0153] In one embodiment, a method for identifying a letter of credit is provided, specifically combining Fig. 20 , Fig. 20 The figure is a flow chart of a method for identifying a letter of credit in an embodiment, which can be specifically divided into three parts: angle correction, detection and identification, and post-processing.
[0154] Specifically, since the image of the letter of credit to be identified uploaded by the business scan may be tilted at a small angle, which will lead to inaccurate recognition of the text area by the subsequent text detection model, the image of the letter of credit to be identified will be angle corrected after the image of the letter of credit to be identified is obtained. Specifically, in this embodiment, the largest form of the letter of credit is retained through image processing operations, and the angle correction is performed using affine transformation according to the corner point coordinates of the largest form, which can solve the problem of small angle correction of the letter of credit. Specifically, it can be combined with Figure 5 shown.
[0155] Specifically, a pre-trained text detection model is used to perform single-category text detection on the image, the image size is unified, the required fields are detected, and the coordinate points of the text detection box are output. The training process of the text detection model is as follows: 1. Annotate the real data set of letters of credit, in which the foreground text is marked as text; 2. Expand based on the real data set of letters of credit, enrich the training samples while ensuring the distribution of real data as much as possible, and ensure that the ratio of real data to expanded data is not less than 1 / 10; 3. Input the training sets in steps 1 and 2 into the second machine learning model, wherein the second machine learning model refers to a machine learning model that can recognize text areas. In this embodiment, the RetinaNet model is selected to train the training set to obtain a text detection model based on the RetinaNet model. The text detection model based on the RetinaNet model will count the Anchor aspect ratio based on the real data in 1 and the synthetic data in 2; considering the large aspect ratio of text, when generating Anchors in Feature Map, only boxes with an aspect ratio greater than or equal to 1 are generated according to the text width; according to the statistical Anchor aspect ratio, the Retinanet detection model is trained in combination with training parameters such as image enhancement strategy; according to the quality of the model reasoning results, the training parameters are continuously optimized, the data enhancement strategy is adjusted, and the data is increased until the model reaches the ideal accuracy. The text detection model based on the RetinaNet model can accurately detect the text in the letter of credit image to be identified, and then the letter of credit image to be identified is segmented to obtain a number of text regions according to the detection result. It should be noted that in this embodiment, the second machine learning model adopts the Retinanet model, and in other embodiments, any machine learning model can be used to train and learn the training set to obtain a text detection model that can accurately identify the text region in the letter of credit image to be identified.
[0156] Specifically, the long field text area is repaired. This is because the letter of credit is a document-intensive scenario with many long and dense text lines. The Anchor-based detection model will have inaccurate regression in the long field, which will cause the beginning and end of the text line to be missed or recognized incorrectly during recognition, resulting in extraction errors. Therefore, the detection results need to be corrected after the detection model is completed. The letter of credit format has two types: tabular and non-tabular. For the tabular format, the repair process is as follows: delimitation is performed based on the table line boundary. When the long field text area regression is inaccurate, that is, there are incomplete boxes at both ends; before recognition, the long field text area will be judged and then forcibly pulled to the vicinity of the table line. The judgment here refers to whether the text area is a long field text area. The key to this method is to first obtain the vertical table line; then use the position information of the vertical table line in the figure as the boundary, and for those text lines that belong to the long field, the left and right ends are post-processed as the table line position. The specific method can be combined with Fig. 9 As shown, first, an expansion and corrosion operation is performed on the vertical core structure of the binarization setting to obtain a vertical binarization image, obtain the contour coordinates of the binarization image, and then according to the boundary of the nearest vertical table line of the long field text area, the coordinates at both ends of the long field are repaired to the vertical boundary position.
[0157] Specifically, for non-tabular format, the repair process is as follows: Non-tabular format letter of credit data adopts the contour search method based on text lines. The boundary is obtained according to rule statistics: First, the coordinate position in the long field text area is obtained, and the coordinate position (location[0], location[1], location[2], location[3]) is recorded, and the slice data is cut out according to the width of the original image and the height of the long field text area; finally, the contour search is performed to find the minimum horizontal coordinate and the maximum horizontal coordinate as the boundary of the long field text area. Specifically, obtain the long field text area: record (location[0], location[1], location[2], location[3]); cut the image: cut the image according to the longitudinal coordinate of the long field text area, and then crop it according to the original width of the image to ensure that the content of the long field text area is in the cut image; binarization: binarize the image; vertical opening operation: used to remove the situation where the text and the vertical table line are connected together, and remove the vertical table line; horizontal closing operation: the horizontal large core structure performs a horizontal closing operation to connect the text area content into blocks for contour detection, so as to obtain the text line of this text area; repair the long field: according to the statistical minimum and maximum horizontal coordinates of the above-mentioned contour, correct (min_x, location[1], max_x, location[2]). Optionally, before repairing the long field text area, the long field text area in the several text areas obtained by segmentation is judged, and if the text area is a long field text area, it is repaired. It should be noted that in other embodiments, all text areas obtained by segmentation can be repaired, not limited to the long field text area.
[0158] Specifically, after repairing the long field text area in several text areas, the text area is identified. Specifically, a pre-trained text recognition model is used to identify the text in the text area. Among them, the training process of the text recognition model is as follows: 1. Based on the real data and synthetic data coordinates, all data text slices are obtained as the training set of the letter of credit recognition model; synthetic data refers to the text content filled into the pre-generated letter of credit template, and synthetic data can reduce the manual expenditure on data annotation; 2. Adjust the data enhancement strategy and the training set in 1, and input the training set into the third machine learning model, wherein the third machine learning model refers to a machine learning model that can recognize the text in the image. In this embodiment, Seq2Seq is used as the second machine learning model; 3. According to the quality of the model reasoning results, the training parameters are continuously optimized, the data enhancement strategy is adjusted, and the data is increased until the model reaches the ideal accuracy, and a Seq2Seq-based text recognition model that can accurately recognize the text is obtained. The text recognition model based on Seq2Seq can be used to perform text recognition on the text area to obtain the recognition result of the text area. It should be noted that in other embodiments, any machine learning model can be used to train and learn the training set to obtain a text recognition model that can accurately recognize the text in the letter of credit image to be recognized.
[0159] Specifically, after obtaining the recognition result, the recognition result is post-processed, specifically in combination with Fig.21 As shown, Fig.21It is a schematic diagram of the post-processing process in an embodiment. The specific process is described as follows: Layout classification: distinguish between table and non-table layouts. The two layout extraction methods and extracted fields are different, and the loaded configuration content is also different; there are differences in the extraction of table and non-table layouts, so classification is required before extraction, distinguishing the layout differences and then performing structured extraction. In this embodiment, the key fields unique to the table class configured first will be fuzzy matched before extraction to match whether the current data is a table class; specifically, all text line contents are traversed, and if one of them is matched, it means that it is a table layout, otherwise it is a non-table layout; the layout is internally distinguished according to the length of the field and whether it is a table to distinguish the extraction method. For example, the short fields of the table class are mostly fields on the left and the corresponding field information on the right, which can be extracted according to the parallel relationship between the left and right. Field key matching: The layout of the letter of credit is relatively special, the number of pages is not fixed, the field content spans pages and is long, and it cannot be matched according to the template matching method; before extracting the field information, the position and text content of the field candidate must be found (matched according to the regular and full text similarity). In this embodiment, the field position and field content of the corresponding extraction field are first found; due to the difference in quality of different pictures, the recognized text will more or less have individual character recognition errors, which affects the search to find the field; in order to improve the accuracy of the search field, this embodiment uses two methods to search: 1) Regular search based on key configuration, before matching the field, first load the field rule configuration corresponding to the table and non-table, each txt file corresponds to the configuration rule of each field (the rule content and the score corresponding to the rule content, the highest score is given priority as the match). Text similarity search based on the full name of the key configuration If the regular search based on the above 1) field configuration does not find the corresponding field, a text similarity search will be performed based on 2) the full text and full name matching method to find the most likely field as a benchmark. Field information matching: Letter of credit data has multiple pages and single pages. Most of the table-type data is single page, and there are also multiple pages. Most of the non-table-type data is at least 2 pages. Therefore, when extracting and identifying, it is not possible to extract only single pages. Therefore, in this embodiment, the entire letter of credit image to be identified is used as a whole for structured extraction; there are certain rules and laws for the fixed fields and non-fixed field information (that is, the content to be extracted) in the letter of credit image to be identified; the field-field information rules or laws can be summarized into three: up-down structure, left-right structure, and non-up-down-left-right structure; based on this rule or law, differentiated structured extraction is performed. One point to be explained is that before extraction, the multi-page letter of credit is merged in the order of page appearance, including the merging of images and the merging of detection and recognition results.Long field paragraph classification: It is impossible to obtain the semantic continuity of sentences between long field sentences based on the position rule information, so the Bert-based language model is used to determine whether the long field context sentences belong to the same paragraph and extract them. Specifically, the Bert-based language model first obtains all candidate sets of long fields and sorts them according to the y coordinate; the sorted text content is sent to the Bert model for judgment every two lines to determine whether the two lines of input text are a sentence (the same paragraph); according to the judgment result, the long field information is output. Before the Bert-based language model first obtains all candidate sets of long fields, it is also necessary to judge the candidate field information. If the candidate field information is a long field text, the Bert-based language model is input, otherwise the post-processing verification is directly performed. Optionally, it can be distinguished whether the candidate field information is a long field by judging whether the field corresponding to the candidate field information refers to the two long fields of description_goods and document_required. Post-processing verification: The candidate field information set content is screened and filtered, and finally structured output. Specifically, the screening and filtering of the candidate field information set content can include the following two methods: 1. Filtering according to rules: Some fields have certain rules, such as dates. 2. Filter field content: The candidate field information may mistakenly extract other field content keywords, which is obviously not what we need. Therefore, we can determine whether there are these fields in the candidate field information set, and filter them if so.
[0160] In the above embodiment, in view of the fact that the text of the letter of credit is rich and complex, a letter of credit angle detection method based on image processing is proposed, which is simple and efficient; secondly, in view of the requirement that the two long fields of the letter of credit need to be distinguished into paragraphs, a letter of credit long field paragraph classification strategy based on the Bert language model is proposed; in view of the fact that the text of the letter of credit is rich and the business logic is complex, a post-processing process including layout classification, field matching, field information matching, long field paragraph classification, and post-processing verification can solve the problems of multiple layouts of the letter of credit, cross-page identification information, background interference, paragraph distinction, etc.
[0161] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.
[0162] Based on the same inventive concept, the embodiment of the present application also provides a letter of credit identification device for implementing the letter of credit identification method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more letter of credit identification device embodiments provided below can refer to the limitations of the letter of credit identification method above, and will not be repeated here.
[0163] In one embodiment, Fig. 22 As shown, a device for identifying a letter of credit is provided, comprising: an acquisition module 100, a text area detection module 200, an identification module 300 and a post-processing module 400, wherein:
[0164] The acquisition module 100 is used to acquire the image of the letter of credit to be identified.
[0165] The text region detection module 200 is used to segment the letter of credit image to be identified to obtain a number of text regions.
[0166] The recognition module 300 is used to perform text recognition on the text area to obtain a recognition result of the text area.
[0167] The post-processing module 400 is used to post-process the recognition result of the text area according to preset rules to obtain structured target information.
[0168] In one embodiment, the above-mentioned letter of credit identification device further comprises:
[0169] The angle correction module is used to perform angle correction on the letter of credit image to be identified.
[0170] In one embodiment, the angle correction module includes:
[0171] The binarization unit is used for binarizing the letter of credit image to be identified to obtain a binarized image.
[0172] The table outline extraction unit is used to extract all table outlines in the binary image.
[0173] The table outline processing unit is used to process all table outlines to obtain the target outline of the table with the largest area.
[0174] The rotation angle calculation unit is used to calculate the angle between adjacent straight lines in the target contour of the maximum area table to obtain the rotation angle.
[0175] The correction unit is used to perform angle correction on the letter of credit image to be identified according to the rotation angle.
[0176] In one embodiment, the table outline extraction unit includes:
[0177] The filtering subunit is used to filter according to the contour area and obtain the table with the minimum contour area as the current table.
[0178] The to-be-processed table acquisition unit is used to acquire the to-be-processed table adjacent to the current table.
[0179] The merging unit is used to merge the to-be-processed table with the current table as a new current table, and continue to obtain the to-be-processed tables adjacent to the current table until the table with the largest area is obtained, and extract the target contour of the table with the largest area.
[0180] In one embodiment, the above-mentioned letter of credit identification device further comprises:
[0181] The repair module is used to repair the text area.
[0182] In one embodiment, the repair module further includes:
[0183] The format acquisition unit is used to acquire the format corresponding to the letter of credit image to be identified.
[0184] The text area repair unit is used to repair the text area according to the layout.
[0185] In one embodiment, the text region repair unit includes:
[0186] The to-be-processed boundary acquisition subunit is used to extract the to-be-processed boundary in the target direction in the text area when the letter of credit image to be identified is in a table format.
[0187] The target boundary acquisition subunit is used to acquire the boundary to be processed closest to the text area of the target type in the text area as the target boundary.
[0188] The border moving subunit is used to move the border of the text area of the target type to the target border.
[0189] In one embodiment, the text area repair unit further includes:
[0190] The coordinate acquisition subunit is used to acquire the coordinates of the text area of the target type in the text area when the letter of credit image to be identified is in a non-tabular format.
[0191] The text slice acquisition subunit is used to segment the letter of credit image to be identified according to the coordinates and the width of the letter of credit image to be identified, and obtain the text slice corresponding to the text area of the target type.
[0192] The content contour acquisition subunit is used to perform corresponding image operations on different directions of the text slice to obtain the content contour.
[0193] The region repair subunit is used to repair the text region of the target type according to the horizontal coordinate of the content contour.
[0194] In one embodiment, the post-processing module 400 further includes:
[0195] The candidate field set acquisition unit is used to extract the candidate field set in the recognition result according to the preset layout structure.
[0196] The field matching unit is used to match the preset configuration rules corresponding to the preset format to obtain the field position and field content corresponding to each field in the candidate field set.
[0197] The field information candidate set acquisition unit is used to acquire the field information candidate set according to the positional relationship between the field and the field information.
[0198] The target information acquisition unit is used to filter the field information candidate set to obtain the target information.
[0199] In one embodiment, the post-processing module further includes:
[0200] The distinguishing field obtaining unit is used to obtain the layout distinguishing field.
[0201] The format matching unit is used to match the recognition result with the format distinguishing field to determine the format of the letter of credit image to be recognized.
[0202] In one embodiment, the post-processing module 400 further includes:
[0203] The image merging unit is used to merge the multiple letter of credit images according to the order of appearance of the multiple letter of credit images when the letter of credit image to be identified is multiple letter of credit images.
[0204] In one embodiment, the post-processing module 400 further includes:
[0205] The paragraph classification unit is used to input the field information into the paragraph classification model when the field information is the target field information; and extract the target field information belonging to the same paragraph through the paragraph classification model.
[0206] In one embodiment, the post-processing module 400 further includes:
[0207] The paragraph text acquisition unit is used to acquire the paragraph text of the letter of credit.
[0208] The training data acquisition unit is used to combine and mark the letter of credit paragraph text according to a preset method to obtain letter of credit training data.
[0209] The preprocessing unit is used to preprocess the letter of credit training data to obtain the preprocessed letter of credit training data.
[0210] The model training unit is used to input the preprocessed letter of credit training data into the first machine learning model for training to obtain a paragraph classification model.
[0211] Each module in the above-mentioned letter of credit identification device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the corresponding operations of each module.
[0212] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Fig.23 As shown. The computer device includes a processor, a memory and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store the image data of the letter of credit to be identified. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a letter of credit identification method is implemented.
[0213] Those skilled in the art will understand that Fig.23 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.
[0214] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.
[0215] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.
[0216] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.
[0217] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.
[0218] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0219] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.
Claims
1. A method for identifying a letter of credit, characterized in that: The method comprises: Acquire the letter of credit image to be identified; Segmenting the letter of credit image to be identified to obtain a plurality of text regions; Performing text recognition on the text area to obtain a recognition result of the text area; Post-processing the recognition result of the text area according to preset rules to obtain structured target information; After obtaining the image of the letter of credit to be identified, the method further includes: Performing angle correction on the letter of credit image to be identified; The step of performing angle correction on the image of the letter of credit to be identified comprises: Binarizing the letter of credit image to be identified to obtain a binary image; Extracting all table contours in the binary image; Processing all the table outlines to obtain a target outline of the table with the largest area; Calculate the angle between adjacent straight lines in the target contour of the maximum area table to obtain a rotation angle; The image of the letter of credit to be identified is angle-corrected according to the rotation angle.
2. The method according to claim 1, characterized in that The step of processing all the table outlines to obtain the target outline of the table with the largest area includes: Filter according to the contour area and obtain the table with the smallest contour area as the current table; Obtain a to-be-processed table adjacent to the current table; The to-be-processed table and the current table are combined as a new current table, and the to-be-processed tables adjacent to the current table are continuously acquired until the table with the largest area is acquired, and the target contour of the table with the largest area is extracted.
3. The method according to claim 1, characterized in that Before performing text recognition on the text area, the method includes: Repair is performed on the text area.
4. The method according to claim 3, characterized in that: The repairing of the text area includes: Obtaining the format corresponding to the letter of credit image to be identified; The text area is repaired according to the layout.
5. The method according to claim 4, characterized in that The repairing of the text area according to the layout includes: When the letter of credit image to be identified is in a table format, extracting the boundary to be processed in the target direction of the text area; Acquire a to-be-processed boundary closest to a text region of a target type in the text region as a target boundary; Move the boundary of the text area of the target type to the target boundary.
6. The method according to claim 4, characterized in that The repairing of the text area according to different layouts includes: When the letter of credit image to be identified is in a non-tabular format, obtaining the coordinates of a text area of a target type in the text area; Segmenting the letter of credit image to be identified according to the coordinates and the width of the letter of credit image to be identified, and obtaining text slices corresponding to the text area of the target type; Performing corresponding image operations on different directions of the text slices to obtain content contours; The text area of the target type is repaired according to the horizontal coordinate of the content outline.
7. The method according to claim 1, characterized in that The post-processing of the recognition result of the text area according to the preset rule to obtain structured target information includes: Extracting a set of candidate fields in the recognition result according to a preset format structure; Matching according to the preset configuration rule corresponding to the preset format obtains the field position and field content corresponding to each field in the candidate field set; Obtaining a candidate set of field information according to the positional relationship between the field and the field information; The field information candidate set is filtered to obtain target information.
8. The method according to claim 7, characterized in that Before extracting the candidate field set in the recognition result according to the preset layout structure, the method further includes: Get the layout distinguishing field; The recognition result is matched with the format distinguishing field to determine the format of the letter of credit image to be recognized.
9. The method according to claim 7, characterized in that: Before obtaining the field information candidate set according to the positional relationship between the field and the field information, the method further includes: When the letter of credit image to be identified is a plurality of letter of credit images, the plurality of letter of credit images are merged according to the order in which the plurality of letter of credit images appear.
10. The method according to claim 7, characterized in that The filtering of the field information candidate set to obtain the target information further includes: When the field information is target field information, the field information is input into a paragraph classification model; and the target field information belonging to the same paragraph is extracted through the paragraph classification model.
11. The method according to claim 10, characterized in that The paragraph classification model is obtained through pre-training, and the training process of the paragraph classification model includes: Get the text of the letter of credit paragraph; Combining and marking the letter of credit paragraph texts according to a preset method to obtain letter of credit training data; Preprocessing the letter of credit training data to obtain preprocessed letter of credit training data; The preprocessed letter of credit training data is input into the first machine learning model for training to obtain the paragraph classification model.
12. A letter of credit identification device, characterized in that: The device comprises: An acquisition module, used for acquiring the image of the letter of credit to be identified; A text region detection module, used for segmenting the letter of credit image to be identified to obtain a plurality of text regions; A recognition module, used for performing text recognition on the text area to obtain a recognition result of the text area; A post-processing module, used for post-processing the recognition result of the text area according to preset rules to obtain structured target information; An angle correction module, used for performing angle correction on the letter of credit image to be identified; The angle correction module comprises: A binarization unit, used for binarizing the letter of credit image to be identified to obtain a binarized image; A table outline extraction unit, used to extract all table outlines in the binary image; A table outline processing unit, used for processing all the table outlines to obtain a target outline of a table with a maximum area; A rotation angle calculation unit, used for calculating the angle between adjacent straight lines in the target contour of the maximum area table to obtain a rotation angle; A correction unit is used to perform angle correction on the letter of credit image to be identified according to the rotation angle.
13. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 11 are implemented.
14. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 11 are implemented.
15. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 11 are implemented.
