A table document alignment recognition method and device

Abstract

The application discloses a table document comparison and recognition method and device, and the method comprises the following steps: positioning a paper table document to be compared through a table document positioning model, optimizing the background of the image to be compared through a table image enhancement model, obtaining a table region picture through a table detection model, recognizing the table region picture by using a table structure recognition model and a character recognition model, and finally sending the recognition result into a difference comparison model to output the comparison and recognition result of the table cells and characters; the paper table document and the electronic table document having a corresponding relationship are extracted for text content and table structure, the difference between the table content is recognized, and the table document is automatically audited, so that the efficiency of the table document auditing can be improved and the auditing cost can be reduced.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of artificial intelligence technology, and in particular to a method and apparatus for comparing and recognizing table documents. Background Technology

[0002] When comparing and reviewing electronic documents, such as PDF files, images, and other table documents that cannot be edited, manual verification of the table content is required. This consumes a huge amount of manpower and resources when comparing a large number of table document data.

[0003] Traditional automated difference comparison methods have solved some of the comparison problems. Traditional pixel-based difference comparison methods can accurately locate differences. However, pixel-based document comparison and recognition methods typically have high requirements for two tables, ensuring that the layout of the two tables remains unchanged. When comparing paper and electronic spreadsheets, the acquisition of paper documents can cause table position shifts, compromising layout consistency and leading to significant errors in pixel-based comparison methods.

[0004] Currently, OCR technology is relatively mature. It's used to recognize table documents and then compare the differences between the recognized text content. When recognizing table content, OCR technology outputs the coordinates of the text and its corresponding text content. By associating the text in two tables through these coordinates, differences are compared between the text content. However, OCR-based table comparison suffers from the same problem as pixel-based comparison methods: during the acquisition of paper tables, distortion occurs, causing the coordinates recognized by OCR to not accurately match the corresponding text in the electronic document, resulting in incorrect matching and significant comparison errors.

[0005] The above content is only used to help understand the technical solution of the present invention and does not represent an admission that the above content is prior art. Summary of the Invention

[0006] This invention provides a method and apparatus for comparing and recognizing table documents, which solves the problem that existing pixel difference comparison and single OCR recognition methods have large comparison errors when comparing paper table documents due to pixel offset and table distortion during the acquisition process.

[0007] The table document comparison and recognition method provided by this invention includes the following steps:

[0008] Receive images of paper form documents to be compared and corresponding standard spreadsheet documents, and convert the standard spreadsheet documents into images of standard form documents;

[0009] The image of the paper form document to be compared is input into the form document positioning model to obtain the position-corrected image of the paper form document.

[0010] The position-corrected paper table document image is input into the table image enhancement model to obtain a background-optimized paper table document image to be compared.

[0011] The background-optimized image of the paper table document to be compared is input into the table detection model to obtain the image of the table region to be compared; the standard table document image is input into the table detection model to obtain the standard table region image.

[0012] The image of the table region to be compared is input into the table structure recognition model to obtain the coordinates of the intersection points of the cells to be compared; the image of the standard table region is input into the table structure recognition model to obtain the coordinates of the standard cell intersection points.

[0013] The image of the table region to be compared is input into the text recognition model to obtain the coordinates of the text rectangle to be compared and the text; the image of the standard table region is input into the text recognition model to obtain the coordinates of the standard text rectangle and the text.

[0014] The coordinates of the text rectangle to be compared and the text are input into the difference comparison model. At the same time, the coordinates of the standard text rectangle and the text are input into the difference comparison model to obtain the comparison and recognition results of the table cell and the text.

[0015] Preferably, the table document localization model uses a semantic segmentation-based table document localization model to locate and correct areas of paper table documents;

[0016] Accordingly, the specific steps for inputting the image of the paper form document to be compared into the form document positioning model to obtain the position-corrected image of the paper form document include:

[0017] Step S11: Segment the pixel region of the paper form document image to be compared to obtain the mask image of the paper form document to be compared.

[0018] Step S12: Calculate the coordinates of the four corner points of the mask image of the paper form document to be compared;

[0019] Step S13: By performing perspective transformation on the coordinates of the four document corner points, the document area is cropped to obtain a paper table document image with position correction.

[0020] Preferably, the table image enhancement model uses a method based on light removal to enhance the image of the corrected paper table document;

[0021] Accordingly, the specific steps for inputting the position-corrected paper table document image into the table image enhancement model to obtain a background-optimized paper table document image for comparison include:

[0022] Step S21: Apply Gaussian blur to the image of the paper form document with the corrected position to obtain a background image with uniform illumination;

[0023] Step S22: Perform a division operation between the image of the paper form document with the corrected position and the background image to obtain a weighted image;

[0024] Step S23: Scale the weighted graph to obtain a paper table document image with the lighting removed;

[0025] Step S24: Perform contrast enhancement and sharpening processing on the light-removed paper form document image to obtain an image-enhanced paper form document image, and use the image-enhanced paper form document image as a comparison paper form document image for background optimization.

[0026] Preferably, the table detection model uses a deep learning-based table detection model to locate table regions;

[0027] Accordingly, the steps of inputting the background-optimized image of the paper table document to be compared into the table detection model to obtain the image of the table region to be compared, and inputting the standard table document image into the table detection model to obtain the standard table region image, specifically include:

[0028] Step S31: Perform table detection on the image-enhanced paper table document image to obtain the coordinates of the table region rectangle and crop out the table region image to be compared;

[0029] Step S32: Perform table detection on the standard table document image to obtain the coordinates of the table area rectangle, and crop out the standard table area image;

[0030] Step S33: By matching the table coordinates, obtain the matching pairs between the table to be compared and the standard table.

[0031] Preferably, step S33, which involves obtaining matching pairs between the table to be compared and the standard table through table coordinate matching, specifically includes:

[0032] Step S311: Select the coordinates of a rectangle in the table to be compared and calculate the intersection-union ratio (IU) of all rectangles in the standard table. Select the rectangle coordinates of the standard table with the largest IU that meets the set threshold and match the current rectangle coordinates of the table to be compared.

[0033] Step S312: Perform matching calculations on the subsequent rectangle coordinates of the table to be compared to obtain the coordinate matching relationship between the table to be compared and the standard table;

[0034] Step S313: Mark the coordinates of the rectangles that are not matched in the table to be compared as the newly added table coordinates, and mark the coordinates of the rectangles that are not matched in the standard table as the missing table coordinates;

[0035] Step S314: Based on the detected rectangle coordinates, crop out the table image, and construct a table image pair by matching the cropped table image. The image pair consists of the table region image to be compared and the standard table region image.

[0036] Preferably, the table structure recognition model uses a semantic segmentation model to obtain table borders, and obtains table cells through the intersections of the borders. Specific steps include:

[0037] Step S41: The semantic segmentation model segments the table image pairs described in step S314 to obtain the table frame lines of the table image pairs; the table frame lines of the table image pairs include: the table frame lines of the table region image to be compared, and the table frame lines of the standard table region image.

[0038] Step S42: Perform intersection detection on the table frame lines of the table image pair to obtain the coordinates of the table cells;

[0039] Step S43: Perform four-sided validation on the obtained table cells and filter out unclosed table cells.

[0040] Preferably, the method for performing four-sided verification on table cells in step S43 includes the following steps:

[0041] Step S51: For the table cells obtained in step S52, find the cell with the smallest side length;

[0042] Step S52: Set the minimum side length as the minimum sampling interval length;

[0043] Step S53: Set sampling points on the four sides of the table cell obtained in step S52 using the minimum sampling interval;

[0044] Step S54: Check whether there is a closed cell by determining whether the sampling point is within the table frame obtained in step S41.

[0045] Preferably, the steps of inputting the table region image to be compared into the text recognition model to obtain the coordinates of the text rectangle to be compared and the text; and inputting the standard table region image into the text recognition model to obtain the standard text rectangle coordinates and the text, specifically include:

[0046] The table image pair in step S314 is identified using an OCR model to obtain the text content and text coordinates of the table image pair.

[0047] Preferably, the method for comparing and recognizing table documents further includes:

[0048] The text content and text coordinates of the obtained table image pairs are correlated to the corresponding table cells by calculating the intersection-union ratio (OCR) of the text coordinates and cell coordinates, resulting in a table to be compared and a standard table. By comparing the differences in the text content within the cells, the coordinates of the differing cells and the corresponding differing text content are obtained. The specific steps include:

[0049] Step S61: Calculate the intersection-union ratio (IUU) of the table cell coordinates obtained in step S43 and the text coordinates obtained in step S54;

[0050] Step S62: When the intersection-union ratio is greater than the set threshold, associate the text content with the cell;

[0051] Step S63: When the intersection-union ratio is less than the set threshold, calculate the distance from the center point of the text coordinates to the center points of all cell coordinates, and associate the text content with the cell with the smallest distance;

[0052] Step S64: Compare the text content of corresponding cells in the table to be compared with the standard table one by one to obtain the coordinates of cells with content differences and the corresponding text content of the differences.

[0053] Furthermore, the present invention also proposes a device for comparing and recognizing table documents, the device comprising: a comparison data acquisition unit, a document positioning unit, a document image enhancement unit, a table detection unit, a table structure recognition unit, a table text recognition unit, a table difference comparison unit, and a comparison and recognition display unit;

[0054] The comparison data acquisition unit is used to acquire images of paper form documents and standard electronic spreadsheet documents to be compared, and to convert the standard electronic spreadsheet documents into images of standard form documents.

[0055] The document positioning unit is used to position and crop paper table document images;

[0056] The document image enhancement unit is used to remove the uneven background problem in the position-corrected paper form document image;

[0057] The table detection unit is used to detect the table positions and relationships between tables in paper table document images and standard table document images, and to construct table image pairs based on the relationships.

[0058] The table structure recognition unit is used to recognize the table cell structure of the table image pair;

[0059] The table text recognition unit is used to recognize the text content and position coordinates of table image pairs;

[0060] The table difference comparison unit is used to identify the differences in content and location coordinates between the paper table document image to be compared and the standard table document image.

[0061] The comparison and identification display unit is used to display the differences between the table to be compared and the standard table.

[0062] The beneficial effects of this invention are as follows: it enables end-to-end comparison and recognition of paper form documents and standard electronic spreadsheet documents without manual correction; using table structure and semantic content for comparison and recognition has high robustness and accuracy; and it optimizes the background of paper document images by removing illumination, reducing background interference and further improving the accuracy of text recognition, thereby improving the accuracy of comparison and recognition. Attached Figure Description

[0063] Figure 1 This is a schematic diagram of the main implementation flow of the table document comparison and recognition method in this invention.

[0064] Figure 2 This is a schematic diagram illustrating the cell closure determination in an embodiment of the present invention;

[0065] Figure 3 This is a schematic diagram illustrating the matching of cells and text content in an embodiment of the present invention;

[0066] Figure 4 This is a schematic diagram of the structure of the table document comparison and recognition device in an embodiment of the present invention;

[0067] Figure 5 This is a schematic diagram of the unit components in an embodiment of the table document comparison and recognition device of the present invention. Detailed Implementation

[0068] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, the accompanying drawings show only the parts relevant to the present invention, and not all of the structures.

[0069] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Please refer to... Figure 1 , Figure 1 This is a flowchart of a table document comparison and recognition method provided by the present invention. The method includes:

[0070] Step S10: Receive the image of the paper form document to be compared and the corresponding standard spreadsheet document (such as a PDF document), and convert the standard spreadsheet document into a standard form document image;

[0071] Step S20: Input the image of the table document to be compared into the table document positioning model to obtain the position-corrected paper table document image;

[0072] Step S30: Input the position-corrected paper table document image into the table image enhancement model to obtain a background-optimized paper table document image to be compared;

[0073] Step S40: Input the background-optimized image of the paper table document to be compared and the standard table document image into the table detection model to obtain the image of the table region to be compared and the standard table region image;

[0074] Step S50: Input the table area image of the table to be compared and the table area image of the standard table into the table structure recognition model to obtain the cell intersection coordinates of the table to be compared and the cell intersection coordinates of the standard table.

[0075] Step S60: Input the table region image to be compared and the standard table region image into the text recognition model to obtain the coordinates of the text rectangle to be compared and the text, as well as the coordinates of the standard text rectangle and the text.

[0076] Step S70: Input the coordinates and text of the text rectangle to be compared, as well as the coordinates and text of the standard text rectangle, into the difference comparison model to obtain the table cells and text content that have differences.

[0077] Step S10 involves capturing an image of the table document to be compared using a mobile device, and then converting it into a standard electronic spreadsheet image by having the user upload a standard table PDF file. It should be noted that there is no specific order in which the image of the paper table is captured and the standard table PDF file is uploaded.

[0078] In step S20, the table document localization model adopts a semantic segmentation-based model, which is used to locate and correct document regions. Specific steps include:

[0079] Step S11: The segmentation model calculates the pixel region of the document image and obtains the mask image of the document region;

[0080] Step S12: Calculate the coordinates of the four corner points of the document mask image;

[0081] Step S13: By performing perspective transformation on the coordinates of the four document corner points, the document area is cropped to obtain a table document image without background.

[0082] In step S11, the segmentation model is pre-trained, and the training process includes: annotation of real-scene data, augmentation of synthetic scene data, and model training. Annotation of real-scene data involves pre-collecting document images using a handheld camera and then annotating the document regions in the images to obtain real-world labeled data. Synthetic scene data is generated by synthesizing a large amount of labeled data. Training uses a UNet network model based on an Encoder-Decoder architecture as the segmentation model, trained on the labeled data.

[0083] The method for generating the synthetic scene data includes: a data acquisition unit collecting electronic document files, including images or PDF files, and converting the electronic documents into image formats for storage; selecting appropriate background images according to actual scene requirements, such as desktop or wall images, and collecting background images of different colors and under different lighting conditions according to diverse requirements; randomly selecting an electronic document image and a background image, randomly selecting four coordinate points on the background image according to rules, calculating a perspective transformation matrix using the four corner points of the electronic document image and the four randomly selected coordinate points, and superimposing the electronic document image onto the background image based on the perspective transformation matrix to generate labeled data for the synthetic scene.

[0084] During the model training process, due to the scarcity of real-world scene data and the abundance of synthetic data, the model is prone to overfitting to the synthetic scene data. Therefore, the model is first iteratively trained on the abundant synthetic scene data to obtain a baseline model, and then fine-tuned on real-world scenes to obtain the final document localization model.

[0085] Step S12 involves calculating the location of the paper document using a document region mask image, where the document region pixels are 1 and the background pixels are 0. The document's position coordinates within the captured image are obtained by calculating the pixel positions of the four corner points of the document region in the mask image. .

[0086] Step S13 involves taking the four vertices of the standard table image. As the target point of perspective transformation .pass and The perspective transformation matrix is ​​calculated using the following formula:

[0087]

[0088] in, It is the perspective transformation matrix, represented as:

[0089]

[0090] By using perspective transformation, the image of the paper form is cropped from the photograph to obtain an image of the same size as the standard form.

[0091] Step S30 involves enhancing the image of the paper table document obtained after positioning and correction. During actual shooting, inconsistent lighting and other interference can easily cause errors in subsequent text and table structure recognition. By removing the background from the corrected paper table image, a white background with black text image is generated that conforms to visual perception, making it easier for users to observe and improving the accuracy of subsequent text and table structure recognition models. The specific steps for enhancing the paper table include:

[0092] Step S21: Apply Gaussian blur to the image of the table document to obtain a background image with uniform lighting;

[0093] Step S22: Perform a division operation on the paper table image and the background image to obtain the weighted image;

[0094] Step S23: Scale up the weight map to obtain the image of the table document after removing the lighting;

[0095] Step S24: Perform contrast enhancement and sharpening processing on the light-removed table image to obtain an enhanced image of the paper table document.

[0096] In step S21, the text area and background area are usually relatively large in a document image. By using a 50x50 Gaussian kernel to perform Gaussian blur operation on the paper table image, a blurred image that is close to the document background can be obtained.

[0097] In step S22, paper form images are used. and Perform a division operation on the background image to obtain a weighted image of the original image relative to the background image. In the weighting, values ​​close to 1 can be represented as background pixels, and values ​​close to 0 can be represented as text areas.

[0098] Step S23 involves processing the weight map. Further scaling to a pixel range of 0-255 yields a table image with the background removed. .

[0099] In step S24, the contrast enhancement and sharpening processing of the table image after removing the light can enhance the visual appeal of the text and borders, making it easier for users to view the table image and improving the clarity of the text and table borders, thereby increasing the accuracy of text recognition and table structure recognition.

[0100] In step S40, the table detection model uses the Faster R-CNN network, a deep learning-based object detection model, to detect table regions, obtaining matching pairs between the table image to be compared and the corresponding positions in the standard table. Specific steps include:

[0101] Step S31: The image enhancement of the table to be compared is input into the table detection model to obtain the coordinates of the table region rectangle, and the table region image to be compared is cropped out;

[0102] Step S32: Input the standard table image into the table detection model to obtain the coordinates of the table region rectangle, and crop out the standard table region image;

[0103] Step S33: By matching table coordinates, obtain the image matching pairs corresponding to the table image to be compared and the standard table image.

[0104] The table detection model uses Faster R-CNN as the object detection model. The model is pre-trained, and the training process includes: table object data annotation, synthesized table object data, and model training. Specifically, table object data annotation is performed by collecting images of table documents from real-world scenarios; synthesized table object data is obtained by parsing spreadsheets; and model training uses a mixture of manually annotated and synthesized data to train the Faster R-CNN network model. The model's confidence parameter is set to 0.8, meaning that when the confidence of a detected object is greater than 0.8, the object is considered a table.

[0105] In step S33, table coordinate matching involves pairing the detection boxes of the table to be compared with the detection boxes of the standard table by calculating the intersection-union ratio (IUU) between the detection boxes. Specific steps include:

[0106] Step S311: Select the coordinates of a rectangle in the table to be compared and calculate the intersection-union ratio (IU) of all rectangles in the standard table. Select the rectangle coordinates of the standard table with the largest IU that meets the set threshold and match the current rectangle coordinates of the table to be compared.

[0107] Step S312: Perform the matching calculation of the rectangle coordinates of the table to be compared in step S311 on the subsequent rectangle coordinates of the table to be compared, and obtain the rectangle coordinate matching relationship between the table to be compared and the standard table.

[0108] Step S313: Mark the coordinates of the rectangles that are not matched in the table to be compared as the newly added table coordinates, and mark the coordinates of the rectangles that are not matched in the standard table as the missing table coordinates;

[0109] Step S314: Based on the detected rectangle coordinates, crop out the table image, and construct table image pairs from the cropped table images through matching relationships.

[0110] In step S311, the rectangular coordinates of the table to be compared are represented as follows: The standard table represents it as follows The Intersection over Union (IOU) ratio of a rectangle is calculated using the following formula:

[0111]

[0112] The cross-union ratio (CUI) threshold for matching rectangles is set to 0.5. A successful match is indicated when the calculated CUI is greater than or equal to 0.5, and a failure is indicated when it is less than 0.5. If no matching rectangle is found for a table to be compared, the table is marked as a new table. When all tables to be compared are matched, the remaining rectangles in the standard table image are marked as the coordinates of the missing table.

[0113] The table matching relationship includes two parts: 1. The rectangle coordinates of the table to be compared; 2. The rectangle coordinates of the standard table. Specifically, when a new table is added, the rectangle coordinates of the standard table in the matching relationship are None; when a missing table exists, the rectangle coordinates of the table to be compared in the matching relationship are None. The three cases of the matching relationship can be represented as: when... Matching relation M={"src": ,“dst”: }; When a new table is added, the matching relation M = {"src": When a missing table exists, the matching relation M = {"src": None, "dst": None}; }

[0114] In step S50, the table structure recognition model uses a semantic segmentation model to obtain table borders, and obtains table cells through the intersections of the borders. Specific steps include:

[0115] Step S41: The semantic segmentation model segments the table image pairs described in step S314 to obtain the table borders of the table image pairs;

[0116] Step S42: Perform intersection detection on the table borders to obtain the coordinates of the table cells;

[0117] Step S43: Perform four-sided validation on the table cells and filter out unclosed table cells.

[0118] The table structure recognition model in step S41 employs the DeepLabV3 model, a semantic segmentation model based on deep learning. This model is pre-trained, and the training process includes: table structure data annotation, table structure data synthesis, and model training. The table structure includes row lines and column lines. Table structure data annotation involves annotating table borders in the actual scene, including row and column lines. Due to the high cost of table border annotation, a large amount of annotated data is synthesized by fusing real table data with background document images. Model training uses a mixture of real-scene annotated data and synthesized data to train the DeepLabV3 model.

[0119] Step S42 involves using the DeepLabV3 model to segment the table row line mask and column line mask. Connectivity analysis is then performed on the row and column line masks to obtain the table line regions, resulting in the coordinates of the row and column lines. The row and column lines are represented using the coordinates of their two endpoints. The corner coordinates of the cells are obtained by calculating the intersection points of the row and column lines.

[0120] In step S43, please refer to Figure 2 The method for performing quadruple validation on table cells includes the following steps:

[0121] Step S51: For the table cells obtained in step S52, find the cell with the smallest side length;

[0122] Step S52: Set the minimum side length as the minimum sampling interval length;

[0123] Step S53: Set sampling points on the four sides of the table cells obtained in step S52 using the minimum sampling interval;

[0124] Step S54: Check whether there is a closed cell by determining whether the sampling point is within the table frame obtained in step S41.

[0125] The minimum side length in step S51 needs to distinguish between the minimum side length of the row line and the minimum side length of the column line. When determining whether a cell is closed, it is necessary to simultaneously determine whether the direction of the row line and column line of the cell is closed. Figure 2 In the middle, take the smallest row line edge of the cell as The smallest column line side is . Figure 2 It should be noted that the solid lines are the table lines identified by the segmentation model. In reality, the table may contain line segments that are not part of the table lines, which could cause interference. Figure 2 middle, The line segment is in the cell. This leads to The constructed straight line will intersect with the actual solid line, forming an unclosed cell. As an interval, in Sampling is performed on the line segment to obtain Six sampling points. Determine whether the x-axis coordinates of each of these six points are correct. Within the line segment,

[0126] Since points P1, P5, and P6 are all outside the range of the line segment, it can be determined that the line segment... The resulting cells are not closed cells.

[0127] Step S60 involves inputting the matching image pairs from step S314 into the text recognition model to obtain the coordinates of the text rectangles and the text of the table to be compared, as well as the coordinates of the text rectangles and the text of the standard table. The text recognition model includes two parts: text detection and text recognition. The text detection model uses a DBNet network for text detection, and the text recognition model uses a CRNN network for text recognition, ultimately obtaining a text recognition result with both location and text information.

[0128] The difference comparison model described in step S70 identifies the text content and location of table image pairs. The text content is associated with the corresponding table cell by calculating the intersection-union ratio of the text coordinates and cell coordinates, thus obtaining the table to be compared and the standard table. By comparing the differences in the text content within the cells, the coordinates of the differing cells and the corresponding differing text content are obtained. The specific steps include:

[0129] Step S61: Calculate the intersection-union ratio (IUU) of the table cell coordinates obtained in step S43 and the text coordinates obtained in step S54;

[0130] Step S62: When the intersection-union ratio is greater than the set threshold, associate the text content with the cell;

[0131] Step S63: When the intersection-union ratio is less than the set threshold, calculate the distance from the center point of the text coordinates to the center points of all cell coordinates, and associate the text content with the cell with the smallest distance;

[0132] Step S64: Compare the text content of corresponding cells in the table to be compared with the standard table one by one to obtain the coordinates of cells with content differences and the corresponding text content of the differences;

[0133] Please see Figure 3 , Figure 3 The table has two cells. and The table image contains two rows of recognized text. and The Intersection over Union (IOU) threshold is set to 0.5, where the text content... Location coordinates and table cell If the intersection-union ratio (IUU) of the location coordinates is equal to 1, which satisfies the set threshold, then... Associated with cell Chinese text content With cells and The calculated IOU is 0, which does not meet the set threshold.

[0134] In step S63, when the intersection-union ratio does not meet the threshold, the center point distance is used to associate the text content. Figure 3 Chinese text content center point Distance from the center point of the two cells and The distances are respectively and ,because The minimum distance is used to associate the text content with the table cell. .

[0135] Please see Figure 4 To achieve the aforementioned objectives, this invention provides an embodiment of a table document comparison and recognition device, comprising: a camera 101, a processor 102, a user interface 103, a network interface 104, a display interface 105, a communication bus 106, and a memory 107. The camera 101 may include a fixed camera or a handheld camera. The processor 102 may include a CPU processor or a GPU processor. The user interface 103 may include an electronic document storage device, and optionally may include a wireless interface or a wired interface. The network interface 104 may include a standard Ethernet interface (RJ45) or a wireless interface (WIFI). The display interface 105 may include a display. The communication bus 106 is used to realize the transmission and communication needs between the various functional components. The memory 107 may include a RAM memory or a ROM memory. Optionally, the memory may be a storage device independent of the aforementioned processor 102, and the memory includes: a data acquisition unit, a document positioning unit, a document image enhancement unit, a table detection unit, a table structure recognition unit, a table text recognition unit, a table difference comparison unit, and a comparison and recognition display unit. Camera 101 is used to capture images of paper forms to be compared. User interface 103 is used to collect standard spreadsheet documents. Processor 102 performs table comparison and recognition by calling the data acquisition unit, document positioning unit, document image enhancement unit, table detection unit, table structure recognition unit, table text recognition unit, table difference comparison unit, and comparison recognition and display unit in memory 107.

[0136] Please see Figure 5 The unit components in the embodiment of the table document comparison and recognition device of the present invention include: a comparison data acquisition unit, a document positioning unit, a document image enhancement unit, a table detection unit, a table structure recognition unit, a table text recognition unit, a table difference comparison unit, and a comparison and recognition display unit;

[0137] The comparison data acquisition unit is used to acquire images of paper forms to be compared and PDF files of standard forms;

[0138] The document positioning unit is used to locate and crop paper table images;

[0139] The document image enhancement unit is used to remove background interference from paper form images;

[0140] The table detection unit is used to detect the table positions and relationships in paper table images and standard tables;

[0141] The table structure recognition unit is used to recognize the table cell structure of a table image;

[0142] The table text recognition unit is used to recognize the text content and position coordinates in the table image;

[0143] The table difference comparison unit is used to identify the differences and location coordinates between the collected paper table to be compared and the standard table;

[0144] The comparison and identification display unit is used to display the differences between the table to be compared and the standard table.

[0145] Note that the above description is merely a preferred embodiment of the present invention and the technical principles employed. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments, and may include many other equivalent embodiments without departing from the concept of the present invention, the scope of which is determined by the scope of the appended claims.

Claims

1. A method for comparing and recognizing tabular documents, characterized in that, The method includes the following steps: Receive images of paper form documents to be compared and corresponding standard spreadsheet documents, and convert the standard spreadsheet documents into images of standard form documents; The image of the paper form document to be compared is input into the form document positioning model to obtain the position-corrected image of the paper form document. The position-corrected paper table document image is input into the table image enhancement model to obtain a background-optimized paper table document image to be compared. The background-optimized image of the paper table document to be compared is input into the table detection model to obtain the image of the table region to be compared; the standard table document image is input into the table detection model to obtain the standard table region image. The image of the table region to be compared is input into the table structure recognition model to obtain the coordinates of the intersection points of the cells to be compared; the image of the standard table region is input into the table structure recognition model to obtain the coordinates of the standard cell intersection points. The image of the table region to be compared is input into the text recognition model to obtain the coordinates of the text rectangle to be compared and the text; the image of the standard table region is input into the text recognition model to obtain the coordinates of the standard text rectangle and the text. The coordinates of the text rectangle to be compared and the text are input into the difference comparison model. At the same time, the coordinates of the standard text rectangle and the text are input into the difference comparison model to obtain the comparison and recognition results of the table cell and the text.

2. The method for comparing and recognizing table documents according to claim 1, characterized in that, The table document localization model uses a semantic segmentation-based table document localization model to locate and correct areas of paper table documents; Accordingly, the specific steps for inputting the image of the paper form document to be compared into the form document positioning model to obtain the position-corrected image of the paper form document include: Step S11: Segment the pixel region of the paper form document image to be compared to obtain the mask image of the paper form document to be compared. Step S12: Calculate the coordinates of the four corner points of the mask image of the paper form document to be compared; Step S13: By performing perspective transformation on the coordinates of the four document corner points, the document area is cropped to obtain a paper table document image with position correction.

3. The method for comparing and recognizing table documents according to claim 1, characterized in that, The table image enhancement model uses a method based on light removal to enhance the image of the corrected paper table document. Accordingly, the specific steps for inputting the position-corrected paper table document image into the table image enhancement model to obtain a background-optimized paper table document image for comparison include: Step S21: Apply Gaussian blur to the image of the paper form document with the corrected position to obtain a background image with uniform illumination; Step S22: Perform a division operation between the image of the paper form document with the corrected position and the background image to obtain a weighted image; Step S23: Scale the weighted graph to obtain a paper table document image with the lighting removed; Step S24: Perform contrast enhancement and sharpening processing on the light-removed paper form document image to obtain an image-enhanced paper form document image, and use the image-enhanced paper form document image as a comparison paper form document image for background optimization.

4. The method for comparing and recognizing table documents according to claim 3, characterized in that, The table detection model uses a deep learning-based table detection model to locate table regions. Accordingly, the background-optimized image of the paper table document to be compared is input into the table detection model to obtain the image of the table region to be compared; The steps of inputting the standard table document image into the table detection model to obtain a standard table region image specifically include: Step S31: Perform table detection on the image-enhanced paper table document image to obtain the coordinates of the table region rectangle and crop out the table region image to be compared; Step S32: Perform table detection on the standard table document image to obtain the coordinates of the table area rectangle, and crop out the standard table area image; Step S33: By matching the table coordinates, obtain the matching pairs between the table to be compared and the standard table.

5. The method for comparing and recognizing tabular documents according to claim 4, characterized in that, Step S33, which involves obtaining matching pairs between the table to be compared and the standard table through table coordinate matching, specifically includes: Step S311: Select the coordinates of a rectangle in the table to be compared and calculate the intersection-union ratio (IU) of all rectangles in the standard table. Select the rectangle coordinates of the standard table with the largest IU that meets the set threshold and match the current rectangle coordinates of the table to be compared. Step S312: Perform matching calculations on the subsequent rectangle coordinates of the table to be compared to obtain the coordinate matching relationship between the table to be compared and the standard table; Step S313: Mark the coordinates of the rectangles that are not matched in the table to be compared as the newly added table coordinates, and mark the coordinates of the rectangles that are not matched in the standard table as the missing table coordinates; Step S314: Based on the detected rectangle coordinates, crop out the table image, and construct a table image pair by matching the cropped table image. The image pair consists of the table region image to be compared and the standard table region image.

6. The method for comparing and recognizing tabular documents according to claim 5, characterized in that, The table structure recognition model uses a semantic segmentation model to obtain table borders, and obtains table cells through the intersections of the borders. Specific steps include: Step S41: The semantic segmentation model segments the table image pairs described in step S314 to obtain the table frame lines of the table image pairs; the table frame lines of the table image pairs include: the table frame lines of the table region image to be compared, and the table frame lines of the standard table region image. Step S42: Perform intersection detection on the table frame lines of the table image pair to obtain the coordinates of the table cells; Step S43: Perform four-sided validation on the obtained table cells and filter out unclosed table cells.

7. The method for comparing and recognizing tabular documents according to claim 6, characterized in that, In step S43, the method for performing four-sided verification on the table cells includes the following steps: Step S51: For the table cells obtained in step S52, find the cell with the smallest side length; Step S52: Set the minimum side length as the minimum sampling interval length; Step S53: Set sampling points on the four sides of the table cell obtained in step S52 using the minimum sampling interval; Step S54: Check whether there is a closed cell by determining whether the sampling point is within the table frame obtained in step S41.

8. The method for comparing and recognizing tabular documents according to any one of claims 5-7, characterized in that, The table area image to be compared is input into the text recognition model to obtain the coordinates of the text rectangle and the text to be compared. The standard table area image is input into the text recognition model to obtain the coordinates of the standard text rectangle and the text. The specific steps include: The table image pair in step S314 is identified using an OCR model to obtain the text content and text coordinates of the table image pair.

9. The method for comparing and recognizing tabular documents according to claim 8, characterized in that, The method for comparing and recognizing table documents also includes: The text content and text coordinates of the obtained table image pairs are correlated to the corresponding table cells by calculating the intersection-union ratio (OCR) of the text coordinates and cell coordinates, resulting in a table to be compared and a standard table. By comparing the differences in the text content within the cells, the coordinates of the differing cells and the corresponding differing text content are obtained. The specific steps include: Step S61: Calculate the intersection-union ratio (IUU) of the table cell coordinates obtained in step S43 and the text coordinates obtained in step S54; Step S62: When the intersection-union ratio is greater than the set threshold, associate the text content with the cell; Step S63: When the intersection-union ratio is less than the set threshold, calculate the distance from the center point of the text coordinates to the center points of all cell coordinates, and associate the text content with the cell with the smallest distance; Step S64: Compare the text content of corresponding cells in the table to be compared with the standard table one by one to obtain the coordinates of cells with content differences and the corresponding text content of the differences.

10. A device for comparing and recognizing tabular documents, characterized in that, The device includes: a comparison data acquisition unit, a document positioning unit, a document image enhancement unit, a table detection unit, a table structure recognition unit, a table text recognition unit, a table difference comparison unit, and a comparison recognition display unit; The comparison data acquisition unit is used to acquire images of paper form documents and standard electronic spreadsheet documents to be compared, and to convert the standard electronic spreadsheet documents into images of standard form documents. The document positioning unit is used to position and crop paper table document images; The document image enhancement unit is used to remove the uneven background problem in the position-corrected paper form document image; The table detection unit is used to detect the table positions and relationships between tables in paper table document images and standard table document images, and to construct table image pairs based on the relationships. The table structure recognition unit is used to recognize the table cell structure of the table image pair; The table text recognition unit is used to recognize the text content and position coordinates of table image pairs; The table difference comparison unit is used to identify the differences in content and location coordinates between the paper table document image to be compared and the standard table document image. The comparison and identification display unit is used to display the differences between the table to be compared and the standard table.

Images