Method and system for intelligently extracting and comparing document information

Abstract

The invention provides a document information intelligent extraction and comparison method and system, and relates to the technical field of document intelligent processing analysis, and the method comprises the following steps: obtaining a comparison document and a to-be-compared document, and converting the comparison document and the to-be-compared document into PNG images; selecting a comparison area and a to-be-compared area, and preprocessing the PNG image to obtain a comparison area image and a to-be-compared area image; performing table recognition based on the comparison region image and the to-be-compared region image; according to the table recognition result, extracting a comparison text and a to-be-compared text; semantic understanding is carried out based on the comparison text and the to-be-compared text; and performing retrieval and text difference comparison according to the semantic understanding result to obtain row-level difference and character-level difference, and generating a visual comparison report based on the row-level difference and the character-level difference. The method can adapt to processing of multiple types of documents, is high in precision of complex table recognition and semantic understanding, supports analysis and comparison of specified areas or contents, and is visual in comparison result difference display.

Description

Technical Field

[0001] This invention relates to the field of intelligent document processing and analysis technology, and in particular to a method and system for intelligent extraction and comparison of document information. Background Technology

[0002] With the increasing prevalence of digital office work, the demand for document information extraction and verification is growing in various scenarios such as e-commerce, trade, law, and finance. Document content is typically stored and represented in the form of table templates and transmitted between multiple business stakeholders (such as e-commerce contracts). These stakeholders need to verify and check key information in the documents (such as order numbers and recipients) to identify discrepancies and avoid inconsistencies.

[0003] Existing intelligent document information extraction and comparison methods have the following limitations: First, they are relatively weak in processing multiple document types, such as PDF, DOCX, XLSX, and PNG, which require separate adaptation, resulting in cumbersome and inefficient processing. Second, they are weak in recognizing complex table template structures in documents, such as those with merged cells or cells spanning multiple rows and columns, which can easily lead to cell extraction errors and low information extraction accuracy. Third, the accuracy of text information recognition is low, and the ability to understand the semantics of text information is insufficient, leading to errors in the verification results. In addition, user interaction capabilities are lacking, as they lack the ability to accurately verify local information of documents that meet certain conditions based on user-specified areas or input words. Finally, the differences in the comparison results are not displayed intuitively; the comparison results are mostly plain text lists, lacking fine-grained distinctions at the row and character levels, making it difficult for users to quickly locate the differences.

[0004] How to solve the above-mentioned technical problems is the challenge facing this invention. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a method and system for intelligently extracting and comparing document information that is adaptable to processing various document types, has high accuracy in recognizing complex tables and understanding semantics, supports analysis and comparison of specified areas or content, and provides intuitive display of differences in comparison results.

[0006] The technical solution adopted by this invention to solve its technical problem is as follows: This invention provides a method for intelligent extraction and comparison of document information, comprising the following steps: Obtain the comparison document and the document to be compared, and convert the formats of the comparison document and the document to be compared to obtain the comparison document image and the document to be compared image; Select the comparison region and the region to be compared, and preprocess the comparison document image and the region to be compared based on the comparison region and the region to be compared to obtain the comparison region image and the region to be compared image; Table recognition is performed based on the comparison region image and the region to be compared image to obtain a list of comparison text cells and a list of text cells to be compared. Extract text information from the list of comparison text cells and the list of text cells to be compared to obtain the comparison text and the text to be compared. Based on semantic understanding of the comparison text and the text to be compared, the structured semantic results of the comparison text and the text to be compared are obtained; Based on the structured semantic results of the comparative text and the structured semantic results of the text to be compared, the retrieval and text difference comparison are performed to obtain the line-level differences and character-level differences, and a visual comparison report is generated based on the line-level differences and character-level differences.

[0007] Preferably, the document formats of the comparison document and the document to be compared include images, PDF documents, and Office documents; the format conversion includes not performing format conversion on images, converting PDF documents to PNG image format, and converting Office documents first to PDF format and then to PNG image format.

[0008] Preferably, the selection of the comparison area and the area to be compared includes recording the start position and end position of the user's mouse click and slide events, and using a coordinate transformation algorithm to transform the start position and the end position from the screen coordinate system to the original image coordinate system to obtain the comparison area and the area to be compared after coordinate transformation.

[0009] Preferably, the preprocessing of the comparison document image and the document image to be compared includes: Based on the comparison area and the area to be compared after coordinate transformation, the comparison document image and the document image to be compared are cropped to obtain the cropped image. Convert the cropped image to BGR format to obtain a BGR format image; The BGR format image is converted to a grayscale image, the grayscale image is bitwise inverted to obtain the inverted grayscale image, and the inverted grayscale image is subjected to adaptive threshold binarization to obtain the comparison region image and the region to be compared image.

[0010] Preferably, the table recognition includes: The binary images of the horizontal and vertical lines of the table are extracted from the comparison region image and the image to be compared region, respectively. The horizontal and vertical lines of the table are then subjected to erosion and dilation operations to obtain the extended horizontal and vertical lines of the table. The extended horizontal and vertical binary images of the table are added pixel-level to obtain a grid image of the table lines. Perform a bitwise AND operation on the grid image to generate the original intersection mask of the horizontal and vertical lines, and then perform a dilation operation on the intersections in the original intersection mask to obtain the intersection region; The intersection region is located by contour detection, and the centroid coordinates of the intersection region are calculated using image moments to obtain the intersection coordinate set; Points in the same column and row of the intersection coordinate set are clustered into one cluster using DBSCAN. The average value of the x-coordinate and the average value of the y-coordinate within the cluster are taken as the corrected x-coordinate and corrected y-coordinate of all points in the cluster, thus obtaining the corrected intersection coordinate set. The points in the corrected intersection coordinate set are taken as the first vertex of the cell, and the points in the corrected intersection coordinate set whose coordinate values ​​are greater than the first vertex of the cell and are closest to the first vertex of the cell are taken as the second vertex of the cell. The cells are determined, and a list of text cells to be compared and a list of text cells to be compared are obtained.

[0011] Preferably, the step of extracting text information based on the comparison text cell list and the text cell list to be compared includes: Based on each cell in the list of comparison text cells and the list of text cells to be compared, crop the comparison area image and the comparison area image to obtain the comparison text cell image and the comparison text cell image; The text regions are located and marked in the comparison text cell image and the text cell image to be compared, respectively. The bounding box information of the text regions is output. The bounding box information of the text regions is the coordinates of the four vertices of the quadrilateral containing the text regions. Text is extracted from the text region, and combined with the bounding box information of the text region, the text in the text region is converted into editable text through image processing and deep learning algorithms, resulting in several editable text blocks and the bounding box information of each text block. The bounding box information of the text block is the coordinates of the four vertices of the quadrilateral containing the text block. The text blocks are reorganized based on their bounding box information to obtain the comparison text and the text to be compared.

[0012] Preferably, the semantic understanding based on the comparison text and the text to be compared includes constructing prompt words for the comparison text and the text to be compared, calling the LLM model to perform semantic segmentation on the comparison text and the text to be compared, and generating category keys.

[0013] Preferably, the step of performing a retrieval and text difference comparison based on the structured semantic results of the comparative text and the structured semantic results of the text to be compared, obtaining line-level differences and character-level differences, and generating a visual comparison report based on the line-level differences and character-level differences includes: Construct the query input based on the content to be retrieved and compared from the structured semantic results of the comparative text; The Embedding model is used to vectorize the structured semantic results of the query input and the text to be compared, resulting in the query vector and the text to be compared vector. The similarity between the query vector and the text vector to be compared is calculated using cosine distance, and the text vector to be compared with the highest similarity is selected as the text to be compared. The difflib library was used to perform line-level and character-level comparison analysis on the retrieved and compared texts to obtain line-level and character-level differences. Use background colors to distinguish line-level differences, use inline tags to mark character-level differences, and add a summary panel that includes line count statistics to generate an HTML report with styled tags.

[0014] This invention also provides a document information intelligent extraction and comparison system, including a front-end system and a back-end system. The front-end system provides user interaction through a browser webpage, allowing users to select comparison areas and areas to be compared using a mouse or to input queries through text boxes, and is used for visual display of difference comparison results. The back-end system includes a document processing module for acquiring comparison documents and areas to be compared uploaded by the front-end system, and uniformly converting documents of different formats into PNG image format; an image preprocessing module for receiving the comparison areas and areas to be compared selected by the front-end system, and preprocessing the comparison document images and areas to be compared based on the comparison areas and areas to be compared, outputting comparison area images and areas to be compared images; and a table recognition module for... The system performs table recognition on the comparison region image and the region to be compared image. This involves extracting horizontal and vertical lines, extending and merging lines to form a grid, calculating and correcting intersection coordinates, sorting intersections, and extracting cells. The output is a list of comparison text cells and a list of text cells to be compared. The semantic understanding module constructs prompt words for the comparison text and the text to be compared, calls an LLM model to perform semantic segmentation on the comparison text and the text to be compared, and generates category keys. The output is a structured semantic result for the comparison text and the text to be compared. The difference comparison module retrieves and compares the structured semantic results of the comparison text and the text to be compared, generates a styled HTML visual comparison report, and feeds the report back to the front-end system for display.

[0015] The beneficial effects of this invention are as follows: it can adapt to the processing of multiple document types, has high accuracy in complex table recognition and semantic understanding, supports analysis and comparison of specified areas or content, and displays the differences in comparison results intuitively. By converting multiple document types into a unified PNG format, the cumbersome process of adapting different formats individually is avoided, achieving efficient and compatible processing of multiple document types. Image preprocessing enhances the contrast of table lines; combined with erosion and dilation operations to extend broken lines, DBSCAN clustering to correct intersection coordinate deviations, and vertex filtering to extract cells, it effectively solves the recognition errors of complex table structures such as merged cells and cross-row / column structures, improving the completeness and accuracy of table information extraction. Text block recombination improves the coherence of text recognition; combined with the LLM model, it performs semantic segmentation of text and generates category keys, achieving structured semantic annotation of text information. By recording user mouse operations and transforming coordinates to obtain comparison areas and areas with comparison, combined with the embedding model's vector retrieval to match user queries, it supports users in accurately locating the local information to be compared according to their needs. The difflib library enables a two-tiered analysis of line-level differences (marked with background color) and character-level differences (marked with inline tags), which, combined with an HTML report containing line count statistics, visually presents the differences. Attached Figure Description

[0016] Figure 1 This is a diagram illustrating the method steps of the present invention.

[0017] Figure 2 This is a system module diagram of the present invention. Detailed Implementation

[0018] To clearly illustrate the technical features of this solution, the following detailed implementation method will be used to explain the solution.

[0019] Example 1: See Figure 1 As shown, this embodiment is a method for intelligent extraction and comparison of document information, including the following steps: S1. Obtain the comparison document and the document to be compared, and convert the format of the comparison document and the document to be compared to obtain the comparison document image and the document to be compared image; The document formats of the comparison document and the document to be compared include images, PDF documents, and Office documents; the format conversion includes not converting images, converting PDF documents to PNG image format, and converting Office documents to PDF format first and then to PNG image format.

[0020] It should be noted that users upload two disk files to be compared via the "Upload Files" button on the browser page. The backend system then converts these different document types into PNG image format for easier subsequent visual processing and OCR optical recognition. After processing, the converted PNG image documents are returned to the browser page for preview, allowing users to select interactive comparison areas and areas to be compared.

[0021] S2. Select the comparison area and the area to be compared, and preprocess the comparison document image and the area to be compared based on the comparison area and the area to be compared to obtain the comparison area image and the area to be compared image; The selection of the comparison area and the area to be compared includes recording the start and end positions of the user's mouse click and slide events. A coordinate transformation algorithm is used to transform the start and end positions from the screen coordinate system to the original image coordinate system, resulting in the coordinate-transformed comparison area and the area to be compared.

[0022] It should be noted that the target area is selected through a rectangle displayed by the user's mouse click and swipe events. Specifically, when the user clicks the mouse, the front-end system records the starting position information of the mouse click. Then, as the user presses and drags the mouse, the front-end system continuously updates the current position information of the mouse. When the user releases the mouse, the front-end system records the ending position information of the mouse release. The starting and ending position information represent the upper left and lower right corner positions of the rectangle displayed by the mouse click and swipe events, respectively, both represented by two-dimensional coordinates. A coordinate transformation algorithm is used to convert from the screen coordinate system to the original image coordinate system. This transformation process involves the following four coordinate systems:

[0023] Table 1. Coordinate systems involved in the coordinate transformation process The specific conversion process is as follows (taking the starting point as an example, the conversion method for the ending point is the same): 1. From screen coordinate system to container coordinate system containerX=selectStartX-containerLeft containerY=selectStartY-containerTop Where (selectStartX, selectStartY) are the pixel coordinates in the screen coordinate system when the user clicks the mouse. (containerLeft, containerTop) are the pixel coordinates of the top-left corner of the front-end container in the screen coordinate system.

[0024] 2. From container coordinate system to image display coordinate system imageX=containerX-(imageLeft-containerLeft) imageY=containerY-(imageTop-containerTop) Here, (imageLeft, imageTop) are the pixel coordinates of the top-left corner of the preview image in the screen coordinate system.

[0025] 3. Image display coordinate system → Original image coordinate system naturalStartX=imageX*(naturalWidth / displayWidth) naturalStartY=imageY*(naturalHeight / displayHeight) Wherein, naturalWidth / naturalHeight are the pixel width and height of the original image, and displayWidth / displayHeight are the pixel width and height of the preview image.

[0026] The above process yields the pixel coordinates (naturalStartX, naturalStartY) of the top-left corner of the selected area in the original image coordinate system. The same process can be used to calculate the pixel coordinates (naturalEndX, naturalEndY) of the bottom-right corner in the original image coordinate system. These represent the location information of the document area selected by the user.

[0027] Preprocessing of the target document image includes: Based on the comparison area and the area to be compared after coordinate transformation, the comparison document image and the document image to be compared are cropped to obtain the cropped image. Convert the cropped image to BGR format to obtain a BGR format image; The BGR format image is converted to a grayscale image, the grayscale image is bitwise inverted to obtain the inverted grayscale image, and the inverted grayscale image is subjected to adaptive threshold binarization to obtain the comparison region image and the region to be compared image.

[0028] It should be noted that the comparison document image and the document image to be compared are cropped according to the rectangle formed by the comparison area and the area to be compared after coordinate transformation, that is, the starting point and the ending point after coordinate transformation, to obtain the cropped image. The cropped image is a byte stream in RGB format. The original target document image (PNG format) and the cropped image (RGB format) are converted into BGR format images by the Pillow library to ensure that subsequent image processing based on OpenCV (such as adaptive threshold binarization, table line detection, intersection coordinate calculation, etc.) can be performed based on the correct color channel information.

[0029] Convert a BGR format image to a grayscale image, represented as Gray. The conversion method uses a weighted average to combine the RGB three channels into a single channel. Gray = 0.299 * R + 0.587 * G + 0.114 * B The grayscale image Gray is bitwise inverted (white becomes black, black becomes white) to obtain ~Gray, and then adaptive thresholding binarization is performed on ~Gray. The specific algorithm is as follows:

[0030]

[0031]

[0032] 0.3*((ksize-1)*0.5-1)+0.8

[0033] in, For pixels The local weighted average gray value at that location. It represents the coordinates (center point) of the pixel currently being processed in the image. Representing coordinates pixel values, For pixels Adaptive threshold at the location, Indicates the weighting coefficient; , is the neighborhood coordinate offset, representing the position offset relative to the center pixel (x,y). Indicates horizontal offset. Indicates vertical offset. It represents the size of the neighborhood; C is the standard deviation of the Gaussian function; C is the constant offset. For binarized pixels The value of .

[0034] After the above processing, a binary image suitable for table line detection is obtained, denoted as binary.

[0035] S3. Perform table recognition based on the comparison area image and the area to be compared image to obtain the comparison text cell list and the text cell to be compared list; Table recognition includes: The binary images of the horizontal and vertical lines of the table are extracted from the comparison region image and the image to be compared region, respectively. The horizontal and vertical lines of the table are then subjected to erosion and dilation operations to obtain the extended horizontal and vertical lines of the table. The extended horizontal and vertical binary images of the table are added pixel-level to obtain a grid image of the table lines. Perform a bitwise AND operation on the grid image to generate the original intersection mask of the horizontal and vertical lines, and then perform a dilation operation on the intersections in the original intersection mask to obtain the intersection region; The intersection region is located by contour detection, and the centroid coordinates of the intersection region are calculated using image moments to obtain the intersection coordinate set; Points in the same column and row of the intersection coordinate set are clustered into one cluster using DBSCAN. The average value of the x-coordinate and the average value of the y-coordinate within the cluster are taken as the corrected x-coordinate and corrected y-coordinate of all points in the cluster, thus obtaining the corrected intersection coordinate set. The points in the corrected intersection coordinate set are taken as the first vertex of the cell, and the points in the corrected intersection coordinate set whose coordinate values ​​are greater than the first vertex of the cell and are closest to the first vertex of the cell are taken as the second vertex of the cell. The cells are determined, and a list of text cells to be compared and a list of text cells to be compared are obtained.

[0036] It should be noted that, for the binary black and white image "binary" obtained in step S2, a horizontal stripe kernel is created. The kernel is rectangular in shape, with a height of 1 and a width equal to the image width / scale, where scale is an adjustable scaling factor. This horizontal stripe kernel is used to erode the binary image, meaning only horizontal lines with a width greater than the width of the horizontal stripe kernel are retained, while small noise or non-horizontal structures are removed. Then, the scale factor is reduced to create a larger horizontal stripe kernel, which is used to dilate the previously obtained horizontal lines. This dilation with the larger kernel extends the horizontal lines to the left and right, connecting the previously broken parts, resulting in a black and white binary image of the horizontal line detection result, denoted as `extended_h`. Similarly, a black and white binary image of the vertical line detection result is obtained, denoted as `extended_v`.

[0037] Perform pixel-level addition on the two binary images extended_h and extended_v, and merge them into a single image, thus creating a complete grid line image, denoted as combined_lines. combined_lines[i,j]=min(extended_h[i,j]+extended_v[i,j],255).

[0038] For the grid image of the table lines, the intersection points of the horizontal and vertical lines are first accurately extracted using a bitwise AND operation (cv2.bitwise_and) to generate the original intersection mask. Then, a 3×3 rectangular kernel is used to dilate the intersection points, expanding single pixels into smaller regions to enhance visibility and detection robustness. Next, these dilated intersection regions are located using contour detection (cv2.findContours). Finally, image moments (cv2.moments) are used to calculate the centroid coordinates of each contour, obtaining a precise set of integer coordinate points. The data structure of the coordinate point set is shown below: points=[ (x1, y1), # Coordinates of the first intersection point (x2, y2), # Coordinates of the second intersection point (x3,y3),#... ... ] S4. Extract text information based on the list of comparison text cells and the list of text cells to be compared, to obtain the comparison text and the text to be compared. Extracting text information from a list of text-based cells includes: Based on each cell in the list of comparison text cells and the list of text cells to be compared, crop the comparison area image and the comparison area image to obtain the comparison text cell image and the comparison text cell image; The text regions are located and marked in the comparison text cell image and the text cell image to be compared, respectively. The bounding box information of the text regions is output. The bounding box information of the text regions is the coordinates of the four vertices of the quadrilateral containing the text regions. Text is extracted from the text region, and combined with the bounding box information of the text region, the text in the text region is converted into editable text through image processing and deep learning algorithms, resulting in several editable text blocks and the bounding box information of each text block. The bounding box information of the text block is the coordinates of the four vertices of the quadrilateral containing the text block. The text blocks are reorganized based on their bounding box information to obtain the comparison text and the text to be compared.

[0039] It should be noted that the PP-OCRv5_server_det detection model from the PaddleOCR library is used to locate and mark regions containing text in the image, and then output the bounding box information of the text regions. The PP-OCRv5_server_rec recognition model from the PaddleOCR library extracts text information from the text regions in the image, receiving the bounding boxes of the text regions output by the text detection module as input. Through image processing and deep learning algorithms, the text in the image is converted into editable text. The output data structure is as follows: {box:[[[x1,y1],[x2,y2],[x3,y3],[x4,y4]],...],text:['Text block 1',...]} Multiple text blocks are regrouped and sorted according to visual line structure to achieve text organization that conforms to human reading order (top to bottom, left to right). The specific process is as follows: Parse the text block information, analyze the obtained data structure to obtain the bounding box vertex coordinates of each text block, as well as the text information of the text block; Calculate the average line height. For all text blocks, calculate the height of the text block height = y2 - y1, and then calculate the average height avg_height of all text blocks. Text block sorting: Sort all text blocks in ascending order according to the y1 coordinate; The line grouping process iterates through the sorted text blocks, compares the vertical center distance between the current text block and the last text block of the previous line, and if it is less than the threshold (0.5 times the average height), it is added to the current line; otherwise, a new line is created. Inline sorting: Sort the text blocks within each line in ascending order horizontally according to the x1 coordinate. Line aggregation concatenates the text blocks grouped and sorted by line to obtain the target text.

[0040] S5. Based on the comparison text and the text to be compared, perform semantic understanding to obtain the structured semantic results of the comparison text and the structured semantic results of the text to be compared. Prompt words are constructed for the comparison text and the text to be compared. The LLM model is then called to perform semantic segmentation on the comparison text and the text to be compared, and category keys are generated.

[0041] It should be noted that building prompts for the input text sequence requires the model to perform semantic segmentation on each text sequence and generate category keys (for Chinese characters). At the same time, it is necessary to ensure that the concatenated segmented text fragments are completely consistent with the original input text sequence. In particular, for newline characters, spaces, and other symbols, these special symbols also need to be verified to maintain the integrity of the information and avoid errors in subsequent information verification due to information loss.

[0042] The prompt format constructed in this embodiment is as follows: prompt=""" Please perform semantic segmentation on the following text object: {{text}} ##Forced Commands## The text is semantically segmented into multiple sequences, each of which must be a 100% complete copy of the original input text fragment. To ensure that all segmented sequence values ​​are concatenated, they must be exactly equal to the original input values, including newline characters, spaces, and other symbols. Summarize each segmented text sequence and generate Chinese category keys. ##Output Format## {{ "Category Key A":{{ "Subcategory key 1":"Original text fragment 1", "Subcategory key 2":"Original text fragment 2", ... }}, "Category Key B":{{ "Subcategory Key 3":"Original Text Fragment 3", "Subcategory key 4":"Original text fragment 4", ... }}, ... }} """ Wherein, text is the target text obtained in step S4.

[0043] Here is a text example: text = [ "Shipper\nZHEJIANG RISE-START IMPORT AND EXPORT CO.,LTD\nROOM 427,BUILDING 2, ZHEJIANG WEALTH CENTER, NO.97 GUDUN RD,HANGZHOU, CHINA""", "Consignee\nTO THE ORDER OF STANDARD BANK PLC.\nTOPKHANA ROAD BRANCH,DHAKA, BANGLADESH\nBIN 000001437-0202""", "Notify Party\nKALIGANJ AGRO-PROCESSING LIMITED.\nEKUTA, SAURAIDBAZAR, KALIGANJ,\nGAZIPUR DHAKA 1720 BANGLADESH\nBIN 003648831-0306\nSTANDARDBANK PLC.\nTOPKHANA ROAD BRANCH, 36 TOPKHANA ROAD, DHAKA-1000, BANGLADESH.\nBIN 000001437-0202""" ] By accessing the local Ollam API through requests, and calling the DeepSeek-R1:32b model, the following parameters are set: param= { "model": "deekseek-r1:32b", "prompt": prompt, "max_tokens": 16384, # Increase token limit to ensure complete output "temperature": 0.01, # Reduce randomness } The returned data format is as follows: <think> Okay, now I need to process the user's query. The user provided a JSON object and requested that each value be semantically segmented to generate multiple sequences, ensuring that these sequences completely replicate the original text, including newlines, spaces, and other symbols. Then, each segmented text fragment should be summarized to generate Chinese category keys.

[0044] In short, I need to meticulously analyze the value structure of each key, accurately divide it into meaningful parts, assign appropriate Chinese category keys to each part, and ensure that the output format meets the requirements.

[0045] < / think> json { "Shipper": { Company Name: ZHEJIANG RISE-START IMPORT AND EXPORT CO.,LTD Detailed Address: ROOM 427, BUILDING 2, ZHEJIANG WEALTH CENTER, NO.97 GUDUN RD, HANGZHOU, CHINA }, "Consignee": { "Consignee Header": "TO THE ORDER OF STANDARD BANK PLC." "Address": "TOPKHANA ROAD BRANCH, DHAKA, BANGLADESH", "BIN Number": "BIN 000001437-0202" }, "Notifying party": { Notifying Company Name: KALIGANJ AGRO-PROCESSING LIMITED. Notifying party address: "EKUTA, SAURAID BAZAR, KALIGANJ,\nGAZIPUR DHAKA 1720BANGLADESH", Notifying Party BIN Number: "BIN 003648831-0306", "Bank Name": "STANDARD BANK PLC." Bank Address: TOPKHANA ROAD BRANCH, 36 TOPKHANA ROAD, DHAKA-1000, BANGLADESH. Bank BIN Number: BIN 000001437-0202 } ``` Because the DeepSeek-R1 model has strong reasoning capabilities, the returned results include " <think>The structured JSON result is wrapped in the tag `json{result}`, where `result` is the structured semantic understanding result we want to obtain. The `result` can be extracted using regular expressions, as shown in the example above.

[0046] S6. Based on the structured semantic results of the comparative text and the structured semantic results of the text to be compared, perform retrieval and text difference comparison to obtain line-level differences and character-level differences, and generate a visual comparison report based on the line-level differences and character-level differences.

[0047] The structured semantic results of the target text are retrieved and compared with textual differences to obtain line-level and character-level differences, and a visual comparison report is generated, including: Construct the query input based on the content to be retrieved and compared from the structured semantic results of the comparative text; The Embedding model is used to vectorize the structured semantic results of the query input and the text to be compared, resulting in the query vector and the text to be compared vector. The similarity between the query vector and the text vector to be compared is calculated using cosine distance, and the text vector to be compared with the highest similarity is selected as the text to be compared. The difflib library was used to perform line-level and character-level comparison analysis on the retrieved and compared texts to obtain line-level and character-level differences. Use background colors to distinguish line-level differences, use inline tags to mark character-level differences, and add a summary panel that includes line count statistics to generate an HTML report with styled tags.

[0048] It should be noted that the formula for calculating similarity using cosine distance is as follows:

[0049] Taking the result obtained in step S5 as an example, if the user wants to compare the "recipient address", they can enter Query="recipient address" on the front-end page. After the above information retrieval steps, the cell containing "recipient" and the original text information "TOPKHANA ROADBRANCH, DHAKA, BANGLADESH" corresponding to the "address" attribute can be retrieved from the two documents to be compared.

[0050] Two-level text difference analysis is performed using the difflib library: First, text block operations (equality / deletion / insertion / replacement) are identified through line-level comparison. For replacement blocks, character-level comparison is further performed. Background colors are used to distinguish line-level differences (red for deletion / green for addition / yellow for modification), and inline tags are used to mark character-level differences (pink for deletion / green for addition).

[0051] The hardware and software environment configuration required for this embodiment is as follows: Hardware configuration: A Dell Precision 3680 server with a 24-core CPU, 128GB of RAM, and an Nvidia 4090 graphics card. Software configuration: The front-end uses the Vue framework, the main server uses the Spring Boot framework, and the core components are deployed as a web application using Python Flask + Docker, providing core algorithm services. Basic model: The OCR algorithm uses the PaddleOCR pre-trained library, and the LLM large model uses a DeepSeek-R1:32b model deployed locally on ollama, which is called via API. The implementation process of this invention is as follows: First, the user sends an HTTP request to the Vue.js service deployed on the server via a browser on their local computer, selects two local documents to be extracted and compared, and uploads them to the server. The front-end Vue service requests the uploaded documents from the back-end SpringBoot service via API. Then, the SpringBoot service obtains the uploaded documents, calls the document parsing module of Python Flask to convert the document format, and uniformly converts non-image formats such as PDF, DOC, and XLSX into PNG image format. The converted PNG images are then returned to the front-end Vue service for image rendering and displayed in the user's browser window. Then, users can select a rectangle by clicking and sliding the mouse in the image preview window. The Vue service uses a coordinate transformation algorithm to convert the coordinates of the user's preview window to the coordinates of the original PNG image, which is the location information of the document area the user is interested in. At the same time, users can also enter a query phrase, which is the document content attribute information the user is interested in. Next, these two pieces of information are assembled into the user's interactive query, and the converted PNG image is sent again to the backend Spring Boot service via API. The Spring Boot service sequentially calls five algorithm function modules: image preprocessing, table recognition, information extraction, semantic understanding, and information comparison, to achieve intelligent extraction and accurate comparison of document information.

[0052] Example 2: See Figure 2 As shown, this embodiment is a document information intelligent extraction and comparison system, including a front-end system and a back-end system. The front-end system provides user interaction through a browser webpage, allowing users to select comparison areas and areas to be compared using a mouse or to enter a query through a text box, and is used for visual display of the difference comparison results. The back-end system includes a document processing module, used to acquire the comparison documents and documents to be compared uploaded by the front-end system, and uniformly convert documents of different formats into PNG image format; an image preprocessing module, used to receive the comparison areas and areas to be compared selected by the front-end system, and preprocess the comparison document images and the images to be compared based on the comparison areas and the areas to be compared, outputting the comparison area images and the images to be compared area images; and a table recognition module, used for... The system performs table recognition on the comparison region image and the image to be compared region image. This involves extracting horizontal and vertical lines, extending and merging lines to form a grid, calculating and correcting intersection coordinates, sorting intersections, and extracting cells. The output is a list of comparison text cells and a list of text cells to be compared. The semantic understanding module constructs prompt words for the comparison text and the text to be compared text. It then calls an LLM model to perform semantic segmentation on the comparison text and the text to be compared text and generates category keys. The output is a structured semantic result for both the comparison text and the text to be compared text. The difference comparison module retrieves and compares the structured semantic results of the comparison text and the text to be compared text, generates a styled HTML visual comparison report, and feeds the report back to the front-end system for display.

[0053] The technical features of this invention not described can be implemented by or using existing technology, and will not be repeated here. Of course, the above description is not a limitation of this invention, and this invention is not limited to the examples above. Any changes, modifications, additions or substitutions made by those skilled in the art within the scope of this invention should also be within the protection scope of this invention.< / think>

Claims

1. A document information intelligent extraction comparison method, characterized in that, The method comprises the following steps: obtaining a comparison document and a document to be compared, and performing format conversion on the comparison document and the document to be compared to obtain a comparison document image and a document to be compared image; selecting a comparison region and a region to be compared, and performing preprocessing on the comparison document image and the document to be compared image based on the comparison region and the region to be compared to obtain a comparison region image and a region to be compared image; performing table recognition based on the comparison region image and the region to be compared image to obtain a comparison text cell list and a region to be compared text cell list; extracting text information based on the comparison text cell list and the region to be compared text cell list to obtain comparison text and region to be compared text; performing semantic understanding based on the comparison text and the region to be compared text to obtain comparison text structured semantic results and region to be compared text structured semantic results; performing retrieval and text difference comparison based on the comparison text structured semantic results and the region to be compared text structured semantic results to obtain line-level differences and character-level differences, and generating a visual comparison report based on the line-level differences and the character-level differences. 2.The method of claim 1, wherein, The document formats of the comparison document and the document to be compared include pictures, PDF documents and Office documents; the format conversion includes no format conversion processing for pictures, conversion of the PDF documents into PNG image format, and conversion of the Office documents into PDF format and then into PNG image format. 3.The method of claim 2, wherein, The selection of the comparison region and the region to be compared includes recording the start position and the end position of a user mouse click sliding event, and converting the start position and the end position from a screen coordinate system to an original image coordinate system by using a coordinate conversion algorithm to obtain the comparison region and the region to be compared after coordinate conversion. 4.The method of claim 3, wherein, The preprocessing of the comparison document image and the document to be compared image includes: performing image cropping on the comparison document image and the document to be compared image according to the comparison region and the region to be compared after coordinate conversion to obtain a cropped image; converting the cropped image into BGR format to obtain a BGR format image; converting the BGR format image into a grayscale image, performing bitwise negation on the grayscale image to obtain a negated grayscale image, and performing adaptive threshold binarization processing on the negated grayscale image to obtain the comparison region image and the region to be compared image.

5. The method of claim 4, wherein, The table recognition includes: extracting table horizontal line binary images and table vertical line binary images from the comparison region image and the region to be compared image, respectively, and performing erosion and dilation operations on the table horizontal line binary images and the table vertical line binary images, respectively, to obtain extended table horizontal line binary images and extended table vertical line binary images; performing pixel-level addition operation on the extended table horizontal line binary images and the extended table vertical line binary images to obtain a table line grid image; performing bitwise AND operation on the table line grid image to generate a horizontal and vertical line original intersection mask, and performing dilation operation on the intersections in the original intersection mask to obtain an intersection region; locating the intersection region by contour detection, and calculating the centroid coordinates of the intersection region by using image moment to obtain an intersection coordinate set; The points in the same column and the same row in the intersection point coordinate set are clustered into a cluster by DBSCAN, and the average value of the x coordinate and the average value of the y coordinate in the cluster are taken as the corrected x coordinate and the corrected y coordinate of all points in the cluster, to obtain a corrected intersection point coordinate set; The points in the corrected intersection point coordinate set are sequentially taken as the first vertex of a cell, and the point in the corrected intersection point coordinate set with a coordinate value greater than the first vertex of the cell and closest to the first vertex of the cell is taken as the second vertex of the cell, to determine the cell, and obtain a comparison text cell list and a text to be compared cell list. 6.The method of claim 5, wherein, The text information is extracted based on the comparison text cell list and the text to be compared cell list, which includes: According to each cell in the comparison text cell list and the text to be compared cell list, the comparison region image and the text to be compared region image are cropped to obtain a comparison text cell image and a text to be compared cell image; The text region is located and marked in the comparison text cell image and the text to be compared cell image respectively, and the bounding box information of the text region is output, wherein the bounding box information of the text region is the four vertex coordinates of the quadrilateral containing the text region; The text is extracted from the text region, and the text in the text region is converted into editable text by image processing and deep learning algorithm combined with the bounding box information of the text region, to obtain a plurality of editable text blocks and the bounding box information of each text block, wherein the bounding box information of the text block is the four vertex coordinates of the quadrilateral containing the text block; The text blocks are reorganized based on the bounding box information of the text blocks to obtain comparison text and text to be compared.

7. The method of claim 6, wherein, The semantic understanding based on the comparison text and the text to be compared includes constructing a prompt prompt word for the comparison text and the text to be compared, calling an LLM model to perform semantic segmentation on the comparison text and the text to be compared, and generating a category key. 8.The method of claim 7, wherein, The retrieval and text difference comparison based on the comparison text structured semantic result and the text to be compared structured semantic result to obtain line-level differences and character-level differences, and the generation of a visual comparison report based on the line-level differences and the character-level differences include: Constructing a Query input according to the content that needs to be compared in the comparison text structured semantic result; Using an Embedding model to vectorize the Query input and the text to be compared structured semantic result to obtain a Query vector and a text to be compared vector; Calculating the similarity of the Query vector and the text to be compared vector using cosine distance, and selecting the text to be compared vector with the highest similarity as the comparison text to be retrieved; Performing line-level and character-level comparison analysis on the comparison text to be retrieved using a difflib library to obtain line-level differences and character-level differences; Using background color to distinguish line-level differences, using inline tags to mark character-level differences, and adding an abstract panel containing line number statistics to generate a style-marked HTML report.

9. A document information intelligent extraction comparison system, characterized by, The document information intelligent extraction comparison method for executing any one of claims 1-8 comprises a front-end system and a back-end system. The front-end system provides interaction for users through a browser webpage, supports users to select comparison areas and comparison areas to be compared through a mouse or input queries through a text box, and is used for visual display of difference comparison results. The back-end system comprises a document processing module, which is used for obtaining comparison documents and comparison documents to be compared uploaded by the front-end system, and uniformly converts documents in different formats into a PNG image format. An image preprocessing module is used for receiving comparison areas and comparison areas to be compared selected by the front-end system, and pre-processing comparison document images and comparison document images to be compared based on the comparison areas and the comparison areas to be compared, and outputting comparison area images and comparison area images to be compared. A table recognition module is used for recognizing comparison area images and comparison area images to be compared, extracting table horizontal lines and vertical lines, extending lines, merging lines to form a grid, calculating and correcting intersection coordinates, sorting intersections, and extracting cells, and outputting comparison text cell lists and comparison text cell lists to be compared. A semantic understanding module is used for constructing a prompt word for comparison text and comparison text to be compared, calling an LLM model to perform semantic segmentation on the comparison text and the comparison text to be compared and generate a category key, and outputting comparison text structured semantic results and comparison text structured semantic results to be compared. A difference comparison module is used for searching and comparing differences between comparison text structured semantic results and comparison text structured semantic results to be compared, generating an HTML visual comparison report with style marks, and feeding back the report to the front-end system for display.

Images