Method, device, equipment and storage medium for identifying bill form structure

Through deep learning technology and Transformer model, automatic identification and optimization of bill table structure is achieved, solving the problems of inefficiency and low accuracy in traditional methods, and improving the accuracy and efficiency of bill table recognition.

CN118470736BActive Publication Date: 2024-11-26YOUDINGTE TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410677167.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-05-29
Publication Date
2024-11-26
Estimated Expiration
2044-05-29

AI Technical Summary

Technical Problem

The traditional method of identification of bill table structures relies on manual processing, resulting in inefficiency and low accuracy, which cannot meet the needs of modern bill processing.

Method used

采用一种票据表格结构识别方法,包括失真部位检测、超分辨率重构、单元格识别、文字布局分析和相邻单元格合并,通过深度学习技术和Transformer模型,自动识别和优化票据表格结构。

Benefits of technology

It improves the clarity and recognition accuracy of the bill image, simplifies the table structure, reduces duplicate information, and improves the overall recognition efficiency and readability.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118470736B_ABST
    Figure CN118470736B_ABST
Patent Text Reader

Abstract

The present invention relates to the field of bill recognition, and in particular to a bill table structure recognition method, device, equipment and storage medium. The method comprises the following steps: obtaining a bill image to be recognized; detecting the distorted part of the bill image to be recognized to generate spatial structure information of the distorted part; restoring the structural details based on the spatial structure information of the distorted part to obtain the distorted part detail repair data; reconstructing the bill image to be recognized with super-resolution based on the distorted part detail repair data to obtain a super-resolution bill image; recognizing the outer contour of the table on the super-resolution bill image to obtain the outer contour line of the table; recognizing the cells of the super-resolution bill image based on the outer contour line of the table to obtain the cell area; obtaining the cell row and column topological structure data based on the cell area; obtaining the table text layout data based on the cell row and column topological structure data. The present invention realizes efficient and accurate bill table structure recognition.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of bill recognition, and in particular to a bill table structure recognition method, device, equipment and storage medium. Background Art

[0002] With the continuous development and application of information technology, bill form structure recognition has become increasingly important in various industries and fields. Trade bills, such as invoices, bills of lading, customs declarations, etc., are important documents used to record transaction information in international trade. These bills usually contain a large amount of structured data, such as commodity name, quantity, price, date, etc. Traditional bill form structure recognition methods usually rely on manual processing and manual observation, and have problems of low work efficiency and low accuracy. In order to meet the needs of modern bill processing, it is necessary to develop an intelligent bill form structure recognition method to improve recognition accuracy and processing efficiency. Summary of the invention

[0003] In order to solve the above technical problems, the present invention proposes a bill table structure recognition method, device, equipment and storage medium to solve at least one of the above technical problems.

[0004] To achieve the above object, the present invention provides a method for identifying a bill table structure, comprising the following steps:

[0005] Step S1: Acquire a bill image to be identified; perform distortion detection on the bill image to be identified to generate spatial structure information of the distorted part; perform structural detail restoration based on the spatial structure information of the distorted part to obtain detailed restoration data of the distorted part;

[0006] Step S2: Based on the detailed restoration data of the distorted part, the bill image to be identified is reconstructed with super resolution to obtain a super resolution bill image; the outer contour of the table is identified on the super resolution bill image to obtain the outer contour line of the table;

[0007] Step S3: performing cell recognition on the super-resolution bill image based on the outer contour of the table, thereby obtaining a cell area; and obtaining cell row and column topological structure data based on the cell area;

[0008] Step S4: obtaining table text layout data based on the cell row and column topological structure data; performing incompletely covered text recognition on the text layout data, and marking the incompletely covered text position when incompletely covered text is detected; performing content area fitting optimization on the cell area based on the incompletely covered text position to obtain a corrected cell area;

[0009] Step S5: obtaining adjacent cell relationship features and semantic similarity based on the corrected cell area; inputting the adjacent cell relationship features and semantic similarity into a preset Transformer-based model to perform cell merging calculation to obtain an adjacent cell merging value;

[0010] Step S6: Compare the adjacent cell merge values ​​based on the preset cell merge threshold. When the preset cell merge threshold is less than or equal to the adjacent cell merge value, merge the adjacent cells in the correction cell area to obtain merged cells; perform table structure recognition on the merged cells to obtain the bill table structure recognition result.

[0011] The present invention also provides a bill table structure recognition device, which is used to execute the bill table structure recognition method as described above, comprising:

[0012] The distortion repair module is used to obtain the bill image to be identified; detect the distorted part of the bill image to be identified to generate spatial structure information of the distorted part; restore the structural details based on the spatial structure information of the distorted part to obtain the detailed repair data of the distorted part;

[0013] The table contour module is used to perform super-resolution reconstruction on the bill image to be identified based on the detailed restoration data of the distorted part to obtain a super-resolution bill image; and perform table outer contour recognition on the super-resolution bill image to obtain the table outer contour line;

[0014] A topological structure module is used to identify cells of the super-resolution bill image based on the outer contour of the table, thereby obtaining the cell area; and obtain the cell row and column topological structure data based on the cell area;

[0015] The cell correction module is used to obtain table text layout data based on the cell row and column topological structure data; perform incompletely covered text recognition on the text layout data, and when incompletely covered text is detected, mark the incompletely covered text position; perform content area fitting optimization on the cell area based on the incompletely covered text position to obtain a corrected cell area;

[0016] A cell merging module is used to obtain adjacent cell relationship features and semantic similarity based on the corrected cell area; the adjacent cell relationship features and semantic similarity are input into a preset Transformer-based model to perform cell merging calculation to obtain adjacent cell merging values;

[0017] The table recognition module is used to compare the adjacent cell merge values ​​based on a preset cell merge threshold. When the preset cell merge threshold is less than or equal to the adjacent cell merge value, the adjacent cells in the correction cell area are merged to obtain merged cells; the table structure of the merged cells is recognized to obtain the bill table structure recognition result.

[0018] The present invention also provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of any one of the above-mentioned bill form structure recognition methods when executing the computer program.

[0019] The present invention also provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of any of the above-mentioned bill form structure recognition methods are implemented.

[0020] The bill form structure recognition method, device, equipment and storage medium provided by the present invention have the following beneficial effects:

[0021] The present invention determines the distorted or damaged areas in the bill image to be identified through distorted area detection, which helps to repair these areas in a targeted manner in subsequent processing, thereby improving the quality and clarity of the image. Structural detail recovery targets the distorted areas and restores the lost structural details to make these areas more visible, reduce information loss and blur, and provide better input data for subsequent processing. Super-resolution reconstruction improves the resolution of the image to make the bill image clearer and richer in details, which helps to improve the accuracy and recognition effect of subsequent processing. Table outer contour recognition accurately determines the boundaries of the table in the bill image, providing accurate positioning and reference for subsequent cell recognition and layout analysis. Cell recognition divides the table into independent cells by detecting and locating the cell area in the bill image, which provides accurate areas for subsequent text layout and content recognition. Row and column topological structure data describes the relationship and arrangement between cells, including the order and relative positions of rows and columns, which helps to establish the structure and layout information of the table, and provides a basis for subsequent text recognition and merging operations. The layout data describes the text content and position information in each cell. By identifying the text layout data, the text information in the table is extracted. The marks and content areas that are not fully covered by the text are optimized so that in the text recognition process, even if some text is not fully covered, this information can be identified and corrected, thereby improving the overall recognition accuracy. The adjacent cell relationship feature describes the connection between cells, including positional relationships, boundary information, etc., which helps to determine the interaction and adjacent relationship between cells. The semantic similarity measures the similarity between the contents of adjacent cells. By calculating the semantic similarity, it is determined whether the contents in the cells are similar, so as to decide whether to merge them. The adjacent cell merging uses a preset merging threshold to merge similar adjacent cells, thereby simplifying the table structure and reducing duplicate information, which helps to improve the readability and overall layout of the table. The table structure recognition analyzes the merged cells to identify the overall structure and layout of the table, including headers, rows, columns and other information, which provides a basis for subsequent data extraction and analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 A schematic diagram of the steps of a bill table structure recognition method of the present invention;

[0023] Figure 2 Detailed implementation flow chart of step S1;

[0024] Figure 3 Detailed implementation flow chart of step S2;

[0025] Figure 4 Detailed implementation flow chart of step S3. DETAILED DESCRIPTION

[0026] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0027] The present application example provides a bill form structure recognition method, device, equipment and storage medium. The execution subjects of the bill form structure recognition method, device, equipment and storage medium include but are not limited to the following: mechanical equipment, data processing platform, cloud server node, network upload device, etc. equipped with the system can be regarded as the general computing node of the present application, and the data processing platform includes but is not limited to: at least one of an audio image management system, an information management system, and a cloud data management system.

[0028] In the embodiment of the present invention, refer to Figure 1 , is a schematic flow chart of the steps of a bill table structure recognition method of the present invention. In this example, the steps of the bill table structure recognition method include:

[0029] Step S1: Acquire a bill image to be identified; perform distortion detection on the bill image to be identified to generate spatial structure information of the distorted part; perform structural detail restoration based on the spatial structure information of the distorted part to obtain detailed restoration data of the distorted part;

[0030] In this embodiment, after obtaining authorization, a high-resolution scanner or digital camera is used, or the original image data of the bill to be identified is obtained from a database, and an image segmentation technology based on deep learning is used to perform pixel-by-pixel distortion detection on the bill image. A convolutional neural network and other models are used to identify distorted areas such as wrinkles, stains, and blur in the bill image, and the structural information such as the spatial position, size, and shape of these distorted areas is extracted to generate a spatial structural description of the distorted parts. Combined with the spatial structural information of the distorted parts, an image restoration technology based on a generative adversarial network (GAN) is used to learn prior knowledge of the bill image through the GAN network, and the contextual information around the distorted area is used to synthesize the detailed restoration content corresponding to the distorted area. The repaired detailed information is fused with the original bill image to generate a bill image in which the distorted part has been repaired.

[0031] Step S2: Based on the detailed restoration data of the distorted part, the bill image to be identified is reconstructed with super resolution to obtain a super resolution bill image; the outer contour of the table is identified on the super resolution bill image to obtain the outer contour line of the table;

[0032] In this embodiment, a super-resolution reconstruction technology based on deep learning is adopted, and the distorted part detail restoration data generated in step S3 is used to train a super-resolution generative adversarial network (SRGAN) to learn the mapping relationship from low-resolution bill images to high-resolution bill images. The original bill image to be identified is input into the SRGAN model to obtain a super-resolution bill image with significantly improved resolution. The instance segmentation technology based on deep learning is adopted to perform fine edge detection on the super-resolution bill image. A convolutional neural network model, such as Mask R-CNN, is used to identify the outer contour of the table area in the bill image. Through further analysis of the contour, the precise coordinate information of the table area is extracted to prepare for positioning for subsequent table content recognition.

[0033] Step S3: performing cell recognition on the super-resolution bill image based on the outer contour of the table, thereby obtaining a cell area; and obtaining cell row and column topological structure data based on the cell area;

[0034] In this embodiment, based on the outer contour information of the table and combined with the super-resolution bill image, the cell area is identified, and a method based on structural analysis is used to identify the intersection of the vertical and horizontal directions along the table contour, and locate the precise boundaries of each cell. By analyzing the cell boundaries, a data structure containing the coordinate information of all cell areas is generated. By clustering and sorting the cell coordinates, the rows and columns of each cell are identified, and a topological structure data containing cell row and column indexes is constructed to describe the layout information of the entire bill table.

[0035] Step S4: obtaining table text layout data based on the cell row and column topological structure data; performing incompletely covered text recognition on the text layout data, and marking the incompletely covered text position when incompletely covered text is detected; performing content area fitting optimization on the cell area based on the incompletely covered text position to obtain a corrected cell area;

[0036] In this embodiment, based on the cell row and column topological structure, the text distribution inside each cell is further analyzed, and a deep learning-based text detection technology, such as EAST or PSENET, is used to locate the text area inside each cell. The detected text area coordinate information is associated with the cell topological structure to construct the text layout data of the entire table. Optical character recognition (OCR) technology is used to extract text content from each text area in the text layout data. During the text recognition process, it is detected whether there is text that is not completely covered by the cell boundary. For such incompletely covered text, their relative position coordinates within the cell are recorded.

[0037] Step S5: obtaining adjacent cell relationship features and semantic similarity based on the corrected cell area; inputting the adjacent cell relationship features and semantic similarity into a preset Transformer-based model to perform cell merging calculation to obtain an adjacent cell merging value;

[0038] In this embodiment, based on the correction of cell area information, the spatial relationship features between adjacent cells are analyzed, and the spatial relationship features such as the distance, alignment, and boundary overlap between adjacent cells are extracted. At the same time, a pre-trained natural language processing model, such as BERT, is used to calculate the semantic similarity of the contents of adjacent cells, and the spatial relationship features and semantic similarity are combined into a multidimensional feature vector to represent the degree of association between adjacent cells. A deep learning model based on the Transformer architecture is constructed to model and evaluate the relationship between adjacent cells. The obtained feature vector is used as the input of the Transformer model, and the potential relationship between adjacent cells is learned through a multi-layer attention mechanism. The output of the model is a scalar value, which represents the confidence score of whether adjacent cells should be merged. According to this score value, it is determined which adjacent cells should be merged into a new cell area.

[0039] Step S6: Compare the adjacent cell merge values ​​based on the preset cell merge threshold. When the preset cell merge threshold is less than or equal to the adjacent cell merge value, merge the adjacent cells in the correction cell area to obtain merged cells; perform table structure recognition on the merged cells to obtain the bill table structure recognition result.

[0040] In this embodiment, a predetermined cell merging threshold is set as a standard for determining whether adjacent cells need to be merged. The obtained adjacent cell merging value is compared with the preset merging threshold. If the merging value is greater than or equal to the threshold, the corresponding adjacent cells are merged into a new cell area. By iteratively comparing and merging all adjacent cells, a set of merged cell data is finally obtained. Combined with information such as the cell position, size, and arrangement, the row and column index relationship of the table is identified, and the identification results are organized into a structured table data model to describe the complete structural information of the bill table.

[0041] In this embodiment, refer to Figure 2 , is a flowchart of detailed implementation steps of step S1. In this embodiment, the detailed implementation steps of step S1 include:

[0042] Step S11: Acquire the image of the bill to be identified;

[0043] Step S12: Detect the distorted part of the bill image to be identified to extract the distorted part image;

[0044] Step S13: performing multi-scale deep feature analysis on the distorted part image to generate spatial structure information of the distorted part;

[0045] Step S14: performing detail texture feature learning processing on the bill image to be identified to obtain clear detail texture feature data;

[0046] Step S15: restoring the structural details of the spatial structural information of the distorted part based on the clear detail texture feature data to obtain the detail restoration data of the distorted part.

[0047] In this embodiment, after obtaining authorization, a high-resolution scanner or digital camera is used, or the original image data of the bill to be identified is obtained from a database, and a deep learning-based image segmentation technology is used to perform pixel-by-pixel distortion detection on the bill image. A convolutional neural network and other models are used to identify distorted areas such as wrinkles, stains, blur, etc. in the bill image, and these distorted parts are extracted from the original bill image to generate a set of distorted part images. A multi-scale deep learning network, such as U-Net or FPN, is used to extract features from the distorted part images, and the spatial structural features of the distorted parts, such as shape, Texture, edge, etc., fuse multi-scale features to generate a feature vector that richly describes the spatial structure of the distorted part, use pre-trained deep learning models, such as VGG or ResNet, extract features of the original bill image, and extract rich detailed texture features such as texture, edge, shape, etc. from the middle layer of the model. Aggregate and encode these feature data to generate a set of feature vectors that describe the clear details of the bill. Use technologies such as generative adversarial networks (GANs) to learn the mapping relationship from the spatial structure of the distorted part to the clear details, apply the learned mapping to the distorted part, and generate a set of detail repair data for the distorted part.

[0048] In this embodiment, refer to Figure 3 , is a flowchart of detailed implementation steps of step S2. In this embodiment, the detailed implementation steps of step S2 include:

[0049] Step S21: performing super-resolution reconstruction on the bill image to be identified based on the detailed restoration data of the distorted part to obtain a super-resolution bill image;

[0050] Step S22: performing table structure detection on the super-resolution bill image to obtain the bill table structure position;

[0051] Step S23: performing bounding box segmentation based on the bill table structure position data to obtain a table bounding box image;

[0052] Step S24: Recognize the outer contour of the table on the table boundary frame image to obtain the outer contour line of the table.

[0053] In this embodiment, a super-resolution generative adversarial network (SRGAN) is trained to learn the mapping relationship from low-resolution bill images to high-resolution bill images, and the original bill image to be identified is input into the SRGAN model to obtain a super-resolution bill image with significantly improved resolution. A target detection technology based on deep learning, such as FasterR-CNN or YOLO, is used to perform table detection on the super-resolution bill image. The position coordinates and size information of the table in the bill image are identified by training the target detection model, and the detection results are organized into a structured representation containing the table position data to prepare for subsequent table content analysis. Based on the table position coordinate information, bounding box cropping is performed on the original super-resolution bill image to extract a sub-image of each table area and generate a set of table bounding box image data. An instance segmentation technology based on deep learning, such as Mask R-CNN, is used to perform fine edge detection on the table bounding box image to identify the outer contour line of the table area in each table image and obtain a set of table outer contour line data.

[0054] In this embodiment, refer to Figure 4 , is a flowchart of detailed implementation steps of step S3. In this embodiment, the detailed implementation steps of step S3 include:

[0055] Step S31: performing table connected domain analysis based on the table outer contour to obtain contour endpoints;

[0056] Step S32: performing cell recognition on the super-resolution bill image based on the contour endpoints, thereby obtaining a cell area;

[0057] Step S33: Perform row and column topological structure analysis on the cell area to obtain cell row and column topological structure data.

[0058] In this embodiment, based on the outer contour line data of the table, a connected domain analysis is performed on it, and the key turning point positions on the outer contour line of the table are identified through contour line endpoint detection. A rule-based cell detection method is used to find vertical and horizontal intersections along the contour line endpoints. These intersections define the precise boundaries of each cell, and a data containing all cell areas is generated. By clustering and sorting the cell coordinates, the rows and columns of each cell are identified, and a topological structure data containing cell row and column indexes is constructed to describe the layout information of the entire bill table.

[0059] In this embodiment, step S4 includes the following steps:

[0060] Step S41: performing text layout analysis on the cell area based on the cell row and column topological structure data to obtain table text layout data;

[0061] Step S42: performing incompletely covered text recognition on the text layout data, and marking the position of the incompletely covered text when incompletely covered text is detected;

[0062] Step S43: performing contour expansion processing on the cell area based on the position of the text that is not completely covered to obtain an enlarged rectangular area;

[0063] Step S44: optimizing the content area fit of the enlarged rectangular area based on the text layout data to obtain a corrected cell area.

[0064] In this embodiment, a text detection technology based on deep learning, such as EAST or PSENET, is used to identify the position of the text area inside the cell, and the detected text area coordinate information is associated with the cell topological structure to generate text layout data for the entire table. During the text recognition process, it is detected whether there is text that is not completely covered by the cell boundary. For such incompletely covered text, their relative position coordinates in the cell are recorded, and the cell boundary is appropriately expanded so that the text area can be completely contained in the cell area. A set of enlarged rectangular cell area data is generated to prepare for subsequent content area optimization. A deep learning segmentation network, such as U-Net, is used to learn the mapping relationship from the enlarged area to the text area, and the learned mapping is applied to the enlarged rectangular area to generate the final corrected cell area data.

[0065] In this embodiment, step S5 includes the following steps:

[0066] Step S51: extracting the geometric information of adjacent cells in the correction cell area to obtain the adjacent cell relationship characteristics;

[0067] Step S52: Calculating text semantic similarity for the correction cell area to generate semantic similarity;

[0068] Step S53: inputting the adjacent cell relationship features and semantic similarity into the preset Transformer-based model to perform cell merging calculation to obtain the adjacent cell merging value.

[0069] In this embodiment, based on the corrected cell area data, the geometric relationship between each adjacent cell is analyzed, and geometric features such as the distance, overlap, and alignment between adjacent cells are extracted to describe their spatial position relationship. A pre-trained natural language processing model, such as BERT or RoBERTa, is used to semantically encode the text content inside each cell, and the semantic similarity of the text content between adjacent cells is calculated to reflect their relevance at the semantic level. This semantic similarity feature indicates the degree of association between cells at the content level, and is also an important basis for merging judgments. The potential association relationship between adjacent cells is learned through the self-attention mechanism of the Transformer model, and the output of the model is a scalar value, indicating a confidence score of whether the adjacent cells should be merged.

[0070] In this embodiment, step S6 includes the following steps:

[0071] Step S61: comparing adjacent cell merge values ​​based on a preset cell merge threshold, and when the preset cell merge threshold is less than or equal to the adjacent cell merge value, merging adjacent cells in the correction cell area to obtain a merged cell;

[0072] Step S62: performing boundary smoothing processing on the merged cells to generate a smooth cell area;

[0073] Step S63: Perform table structure recognition on the smooth cell area to obtain the bill table structure recognition result.

[0074] In this embodiment, the obtained merge value of adjacent cells is compared with a preset merge threshold. If the merge value is greater than or equal to the preset threshold, the corresponding adjacent cells are merged into a new cell area. By iteratively comparing and merging all adjacent cells, a set of merged cell data is finally obtained. This ensures that only adjacent cells with a sufficiently high merge confidence level are merged, retaining the integrity of the table structure. The merged cell area is subjected to boundary smoothing optimization processing, and a boundary smoothing algorithm based on mathematical morphology is used to eliminate jagged boundaries that may be generated during the merging process. The row and column index relationship of the table is identified by combining information such as the relative position, size, and arrangement of the cells, and the identification results are organized into a structured table data model to describe the complete structural information of the bill table.

[0075] The present invention also provides a bill table structure recognition device, which is used to execute the bill table structure recognition method as described above, comprising:

[0076] The distortion repair module is used to obtain the bill image to be identified; detect the distorted part of the bill image to be identified to generate spatial structure information of the distorted part; restore the structural details based on the spatial structure information of the distorted part to obtain the detailed repair data of the distorted part;

[0077] The table contour module is used to perform super-resolution reconstruction on the bill image to be identified based on the detailed restoration data of the distorted part to obtain a super-resolution bill image; and perform table outer contour recognition on the super-resolution bill image to obtain the table outer contour line;

[0078] A topological structure module is used to identify cells of the super-resolution bill image based on the outer contour of the table, thereby obtaining the cell area; and obtain the cell row and column topological structure data based on the cell area;

[0079] The cell correction module is used to obtain table text layout data based on the cell row and column topological structure data; perform incompletely covered text recognition on the text layout data, and when incompletely covered text is detected, mark the incompletely covered text position; perform content area fitting optimization on the cell area based on the incompletely covered text position to obtain a corrected cell area;

[0080] A cell merging module is used to obtain adjacent cell relationship features and semantic similarity based on the corrected cell area; the adjacent cell relationship features and semantic similarity are input into a preset Transformer-based model to perform cell merging calculation to obtain adjacent cell merging values;

[0081] The table recognition module is used to compare the adjacent cell merge values ​​based on a preset cell merge threshold. When the preset cell merge threshold is less than or equal to the adjacent cell merge value, the adjacent cells in the correction cell area are merged to obtain merged cells; the table structure of the merged cells is recognized to obtain the bill table structure recognition result.

[0082] The present invention also provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of any one of the above-mentioned bill form structure recognition methods when executing the computer program.

[0083] The present invention also provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of any of the above-mentioned bill form structure recognition methods are implemented.

[0084] Those skilled in the art clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described systems, systems and units refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0085] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it is stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or partly or all or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions for a computer device (a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media for storing program codes such as USB flash drives, mobile hard disks, read-only memory (ROM), random access memory (RAM), magnetic disks or optical disks.

[0086] Therefore, the embodiments should be regarded as illustrative and non-restrictive from all points, and the scope of the present invention is limited by the appended claims rather than the above description, and it is therefore intended that all changes falling within the meaning and range of equivalent elements of the application documents are included in the present invention.

[0087] The above is only a specific embodiment of the present invention, so that those skilled in the art can understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but should conform to the widest scope consistent with the principles and novel features invented herein.

Claims

1. A method for identifying the structure of a bill form, characterized in that: The following steps are involved: Step S1: Acquire a bill image to be identified; perform distortion detection on the bill image to be identified to generate spatial structure information of the distorted portion; Restore structural details based on the spatial structure information of the distorted part to obtain detailed restoration data of the distorted part; Step S2: performing super-resolution reconstruction on the bill image to be identified based on the detailed restoration data of the distorted part to obtain a super-resolution bill image; performing table outer contour recognition on the super-resolution bill image to obtain the table outer contour line; Step S3: performing cell recognition on the super-resolution bill image based on the outer contour of the table, thereby obtaining a cell area; and obtaining cell row and column topological structure data based on the cell area; Step S4: obtaining table text layout data based on the cell row and column topology data; Perform incompletely covered text recognition on the text layout data. When incompletely covered text is detected, mark the location of the incompletely covered text; optimize the content area fit of the cell area based on the location of the incompletely covered text to obtain a corrected cell area; Step S5: Obtaining adjacent cell relationship features and semantic similarity based on the corrected cell area; inputting the adjacent cell relationship features and semantic similarity into a preset Transformer-based model to perform cell merging calculation to obtain an adjacent cell merging value; Step S6: comparing adjacent cell merge values ​​based on a preset cell merge threshold; when the preset cell merge threshold is less than or equal to the adjacent cell merge value, merging adjacent cells in the correction cell area to obtain a merged cell; Perform table structure recognition on the merged cells to obtain the bill table structure recognition result.

2. The bill table structure recognition method according to claim 1, characterized in that: The specific steps of step S1 are: Step S11: Acquire the image of the bill to be identified; Step S12: Detecting the distorted portion of the bill image to be recognized to extract the distorted portion image; Step S13: performing multi-scale deep feature analysis on the distorted part image to generate spatial structure information of the distorted part; Step S14: performing detail texture feature learning processing on the bill image to be identified to obtain clear detail texture feature data; Step S15: recovering the structural details of the spatial structural information of the distorted part based on the clear detail texture feature data to obtain detail restoration data of the distorted part.

3. The bill table structure recognition method according to claim 1, characterized in that: The specific steps of step S2 are: Step S21: performing super-resolution reconstruction on the bill image to be identified based on the detailed restoration data of the distorted part to obtain a super-resolution bill image; Step S22: performing table structure detection on the super-resolution bill image to obtain the bill table structure position; Step S23: performing bounding box segmentation based on the bill table structure position data to obtain a table bounding box image; Step S24: performing table outer contour recognition on the table boundary frame image to obtain the table outer contour line.

4. The bill table structure recognition method according to claim 1, characterized in that: The specific steps of step S3 are: Step S31: performing table connected domain analysis based on the table outer contour to obtain contour endpoints; Step S32: performing cell recognition on the super-resolution bill image based on the contour endpoints, thereby obtaining a cell area; Step S33: performing row and column topological structure analysis on the cell area to obtain cell row and column topological structure data.

5. The bill table structure recognition method according to claim 1, characterized in that: The specific steps of step S4 are: Step S41: performing text layout analysis on the cell area based on the cell row and column topology data to obtain table text layout data; Step S42: performing incompletely covered text recognition on the text layout data, and marking the position of the incompletely covered text when incompletely covered text is detected; Step S43: performing contour expansion processing on the cell area based on the position of the text that is not completely covered to obtain an expanded rectangular area; Step S44: performing content area fitting optimization on the enlarged rectangular area based on the text layout data to obtain a corrected cell area.

6. The bill table structure recognition method according to claim 1, characterized in that: The specific steps of step S5 are: Step S51: extracting geometric information of adjacent cells in the correction cell area to obtain adjacent cell relationship features; Step S52: performing text semantic similarity calculation on the correction cell area to generate semantic similarity; Step S53: inputting the adjacent cell relationship features and semantic similarity into the preset Transformer-based model to perform cell merging calculation to obtain the adjacent cell merging value.

7. The bill table structure recognition method according to claim 1, characterized in that: The specific steps of step S6 are: Step S61: comparing adjacent cell merge values ​​based on a preset cell merge threshold, and when the preset cell merge threshold is less than or equal to the adjacent cell merge value, merging adjacent cells in the correction cell area to obtain a merged cell; Step S62: performing boundary smoothing on the merged cells to generate a smooth cell area; Step S63: performing table structure recognition on the smooth cell area to obtain a bill table structure recognition result.

8. A bill form structure recognition device, characterized in that: include: The distortion repair module is used to obtain the bill image to be identified; detect the distorted parts of the bill image to be identified to generate spatial structure information of the distorted parts; Restore structural details based on the spatial structure information of the distorted part to obtain detailed restoration data of the distorted part; The table outline module is used to perform super-resolution reconstruction on the bill image to be identified based on the detailed restoration data of the distorted part to obtain a super-resolution bill image; and perform table outer contour recognition on the super-resolution bill image to obtain the table outer contour line; A topology structure module is used to identify cells in the super-resolution bill image based on the outer contour of the table, thereby obtaining the cell area; and obtain the cell row and column topology structure data based on the cell area; A cell correction module is used to obtain table text layout data based on cell row and column topology data; Perform incompletely covered text recognition on the text layout data. When incompletely covered text is detected, mark the location of the incompletely covered text; optimize the content area fit of the cell area based on the location of the incompletely covered text to obtain a corrected cell area; The cell merging module is used to obtain the adjacent cell relationship features and semantic similarity based on the corrected cell area; the adjacent cell relationship features and semantic similarity are input into the preset Transformer-based model to perform cell merging calculation to obtain the adjacent cell merging value; A table recognition module is used to compare adjacent cell merge values ​​based on a preset cell merge threshold. When the preset cell merge threshold is less than or equal to the adjacent cell merge value, adjacent cells in the correction cell area are merged to obtain a merged cell. Perform table structure recognition on the merged cells to obtain the bill table structure recognition result.

9. A computer device comprising a memory and a processor, wherein a computer program is stored in the memory, wherein: When the processor executes the computer program, the steps of the bill form structure recognition method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the bill form structure recognition method according to any one of claims 1 to 7 are implemented.