Intelligent receipt auditing method, system and device and medium

Abstract

The invention provides an intelligent receipt auditing method, system and device and a medium, and belongs to the technical field of artificial intelligence. The method comprises the steps of collecting scanning receipt files in different formats, and generating receipt pictures after format unified processing; performing seal area detection and separation, table structure identification, text content extraction and key information anchoring and integration on the document picture by calling a pre-training model and an interface so as to extract basic information, list table information and seal information of the document and generate structured document information; comparing and auditing the basic information of the document by using a basic information auditing agent, and auditing the consistency of the basic information; auditing the content of each row in the list table based on the list table information by utilizing the list table information auditing agent; checking the consistency of the seal information and other related information by using a seal information checking agent; and performing optimization processing on audit results of the three agents by using a language model to form an audit report.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence technology, and more specifically relates to an intelligent document review method, system, device and medium. Background Technology

[0002] In enterprise project management processes, a large number of documents are generated during key stages such as procurement, delivery, and acceptance, including delivery receipts, resource lists, acceptance reports, and settlement vouchers. These documents not only record core business data such as material flow and service delivery, but also carry key commercial content such as supplier information, contract terms, and quality indicators, serving as important carriers of the project execution process. However, in the context of current enterprise digital transformation, the hybrid management model of paper and electronic documents presents a severe challenge to information integrity verification. Omissions, errors, or version inconsistencies in document information can easily lead to supply chain disruptions, financial disputes, or audit risks. Therefore, establishing a systematic document review mechanism, verifying the compliance, consistency, and timeliness of documents through multi-dimensional comparison, has become an indispensable part of modern enterprise risk management systems, playing a decisive role in ensuring the standardized operation of project lifecycle management. In traditional enterprise management models, document review has long relied on manual operations, resulting in significant efficiency bottlenecks and potential quality risks. From an efficiency perspective, manual verification of each data item in a document is limited by the worker's working hours and concentration, making it difficult to meet the high-frequency, high-volume business processing needs of modern enterprises. In terms of cost, enterprises need to establish professional review teams and invest significant human resources in repetitive tasks, leading to high operating costs. More importantly, during manual review, reviewers are inevitably prone to oversights and misjudgments due to subjective factors such as fatigue and distraction. These human errors can potentially trigger subsequent business disputes or financial risks, causing significant losses to the enterprise. In real-world business operations, invoices and forms exhibit a high degree of heterogeneity. Structurally, document layouts vary across different business scenarios, potentially employing tree, matrix, or nested structures. Content-wise, a single document often contains structured data, semi-structured text, and unstructured notes. Furthermore, customized header designs and merged cell layouts by different companies further complicate document processing. This complexity makes it difficult to directly apply traditional Optical Character Recognition (OCR) technology to document processing, hindering the accurate extraction and verification of key information within the documents. In conclusion, the existing document review methods have significant shortcomings in terms of efficiency, cost, accuracy, and the ability to handle complex documents. Summary of the Invention

[0003] To address the above problems, the present invention aims to provide an intelligent document review method, system, device, and medium. By adopting a multi-vision model and intelligent agent collaboration approach, it achieves automatic review of document files, effectively improving the accuracy and reliability of the review.

[0004] To achieve the above objectives, the present invention employs the following technical solution: In a first aspect, embodiments of this application provide an intelligent document review method, including: Collect scanned document files of different formats, perform format unification processing, and generate document images; By calling pre-trained models and interfaces, the system performs seal area detection and separation, table structure recognition, text content extraction, and key information anchoring and integration on document images to extract basic document information, list table information, and seal information, thereby generating structured document information. Using a pre-set basic information review agent, the basic information of the documents is compared and reviewed to ensure consistency of the unit name, contact person, contact information, and project type. The intelligent agent is reviewed using a pre-set list table information. Based on the list table information, the content of each row in the list table is reviewed. The system uses pre-set seal information to audit the intelligent agent and verify the consistency of the seal information with other relevant information. A large-scale language model is used to optimize the review results of the three agents and generate a review report.

[0005] In one optional implementation, the step of collecting scanned document files of different formats, performing format unification processing, and generating document images includes: Collect scanned document files in PDF, PNG, and JPEG formats, and use image processing tools to convert them into PNG document images.

[0006] In an optional implementation, the step of calling a pre-trained model and interface to perform seal area detection and separation, table structure recognition, text content extraction, and key information anchoring and integration on the document image, in order to extract basic document information, list table information, and seal information, and generate structured document information, including: Using a pre-defined stamp area detection model, the coordinate areas of all stamps in the document image are identified to determine the stamp area. Separate the stamp area from the document image and extract the stamp image; For document images with the stamp area removed, a preset table structure recognition model is used to detect the table structure of the document image, locate and obtain the area coordinates of each cell in the table structure.

[0007] In an optional implementation, the step of calling a pre-trained model and interface to perform seal area detection and separation, table structure recognition, text content extraction, and key information anchoring and integration on the document image to extract basic document information, list table information, and seal information, and generate structured document information, further includes: Using a pre-defined text region detection model, the coordinate regions of all text in a document image with the stamp area removed are accurately identified, and the coordinate data of the four endpoints of each text line region are obtained to determine the text region. The coordinates of the four endpoints of each text line area are associated with the area coordinates of each cell in the table structure. Redundant parts of the text area are deleted and the area is expanded. The text area is then updated. A pre-defined text content recognition model is used to identify the text content contained in the text area and extract the text line content. Based on the text area and text line content, a recall operation is performed on redundant text areas caused by recognition errors to generate the text content of the document image; The coordinates of each cell in the table structure are associated with the text content of the document image to match the cell position of the text row area and establish the relationship between the table structure and the text content in terms of spatial coordinates. Using key syntax matching technology, table areas containing preset key values ​​are anchored to identify header rows containing basic information and determine the coordinate information of the header rows.

[0008] In an optional implementation, the step of calling a pre-trained model and interface to perform seal area detection and separation, table structure recognition, text content extraction, and key information anchoring and integration on the document image to extract basic document information, list table information, and seal information, and generate structured document information, further includes: Based on the coordinate information of the header row, and combined with the relationship between the table structure and the text content in the spatial coordinate plane, the specific values ​​in the header row are matched as the basic information of the document. Based on the position information of all cells in the header row, and by leveraging the relationship between the table structure and the text content in spatial coordinates, the coordinate information and specific values ​​of all cells corresponding to each column area are located, thus determining the coordinate information and specific values ​​of all cells in the list table. Iterate through the coordinates of all cells in the list table to obtain the position information of the first cell in each row of the list table, and combine the coordinates and specific values ​​of all cells in the list table to generate the list table information. For the seal image, a preset seal text recognition model is used to identify the valid information contained therein and generate seal information; The basic information, list and table information, and seal information of the documents are integrated and processed to generate structured document information.

[0009] In an optional implementation, after generating the structured document information, the method further includes: Information items are extracted from the list table information, and the information items are compared with the preset product types to identify the product type of the document; the product types of the document include server, operating system, storage, and network. The document information is reconstructed according to the current system input data format, and its data format is converted into the current system input data format.

[0010] In an optional implementation, the intelligent agent that reviews the information in a preset checklist table reviews the content of each row in the checklist table based on the checklist table information, including: Based on the list table information and combined with the relevant data of the current system, the pre-set list table information auditing agent audits the content of each row in the list table, checking whether the configuration information, quantity information, time information, and name are completely consistent with the relevant data of the system.

[0011] In an optional implementation, the step of using a preset seal information verification agent to verify the consistency of the seal information with other relevant information includes: Using a pre-set intelligent agent to review seal information, the agent checks the seal information against the basic information of the document and the unit name and subject name recorded in the list table to verify whether the information is consistent.

[0012] Secondly, embodiments of this application also provide an intelligent document review system, including: The document acquisition module is used to acquire scanned document files of different formats, perform format unification processing, and generate document images; The document information extraction module is used to extract basic information, list table information and seal information from document images by calling pre-trained models and interfaces, and to generate structured document information. The basic information verification module is used to compare and verify the basic information of documents using a preset basic information verification intelligent agent, and to verify the consistency of unit name, contact person, contact information, and project type. The list table information review module is used to review the contents of each row in the list table based on the pre-set list table information review intelligent agent. The seal information verification module is used to verify the consistency of seal information with other relevant information using a pre-set seal information verification intelligent agent. The audit summary module is used to optimize the audit results of the three agents using a large language model and generate an audit report.

[0013] Thirdly, embodiments of this application also provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the intelligent document review method described in any of the above descriptions.

[0014] Fourthly, embodiments of this application also provide a storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the intelligent document review method as described in any of the above.

[0015] As can be seen from the above technical solutions, the present invention has the following advantages: The intelligent document review method provided in this application employs a collaborative approach based on multi-visual models and intelligent agents to achieve automatic review of document files. In the crucial stage of document information extraction, this method comprehensively utilizes five visual foundational models with specific functions, including a seal area detection model, a table structure detection model, a text area detection model, a text content recognition model, and a seal content recognition model. Compared to large multimodal models, these visual foundational models possess greater specialization in the specific domain of document review, are simpler to train and optimize, easier to deploy, and have a faster response speed, effectively improving the efficiency and accuracy of document review. Specifically, in the table structure recognition task, the visual foundational models used in this method can accurately detect the coordinate information of table lines within document images, a capability currently lacking in mainstream large multimodal models on the market. Furthermore, in seal content recognition, the visual foundational models used in this method can effectively adapt to curved and deformed text within seals, thereby achieving a high recognition accuracy rate, a task that is also difficult for large multimodal models to accomplish. Through the collaborative mechanism of five basic visual models, the document information extraction method in this paper can adapt to complex and ever-changing document structures, and also demonstrates strong adaptability and processing capabilities for document images with poor image quality.

[0016] This application proposes a method combining "visual foundational models from specific professional fields with a comprehensive, long-thinking model" for document review. This approach effectively ensures the controllability of the analysis results, making them logically closer to real human review logic, thereby improving the accuracy and reliability of the review. Specifically, this application first comprehensively utilizes multiple visual foundational models from specific professional fields to accurately extract three types of key information from documents through collaborative work: basic information, list / table information, and seal information. This information is then organized into structured data for subsequent summarization and analysis. This organized data is passed to three specially designed document review agents, each reviewing and comparing different types of data. After obtaining the structured review results output by each agent, this application introduces a comprehensive, long-thinking language model to summarize and optimize the output results of each agent. This language model can identify and correct omissions and misjudgments in the review process, ensuring the accuracy and completeness of the review results. Finally, the optimized review results are formatted in a way that is easy for business personnel to read and use, thereby assisting them in improving the efficiency and accuracy of document review.

[0017] The method proposed in this application designs a highly flexible and usable document content extraction model by combining multiple basic visual models. This method fully considers the diversity and complexity of expense reimbursement documents, aiming to accurately and efficiently extract various key information from scanned document files. In practical application scenarios, the method proposed in this patent innovatively integrates five types of highly professional and targeted basic visual models, including a table structure detection model, a text region detection model, a text content recognition model, a seal region detection model, and a seal content recognition model. Through the synergistic effect of these models, accurate and efficient recognition of various document contents can be achieved. The recognized information includes three categories: basic document information, such as project name and company name; detailed content within the list table, such as configuration, quantity, product type, and date; and seal information, i.e., the seal's main information. These various types of information are interconnected and constitute the overall architecture of the document information, providing a comprehensive and accurate data foundation for the efficient review of the subsequent document review intelligent agent.

[0018] This application designs three collaborative intelligent agents: a basic information verification agent, a list / table information verification agent, and a seal information verification agent, to accurately compare and verify the structured information extracted from the visual basic model. Unlike verification using a multimodal large model, verification based on structured information can achieve higher accuracy and reliability, and can effectively solve the problem of inaccurate judgments caused by poor image quality in multimodal large models.

[0019] This application introduces a large-scale language model with long-term thinking capabilities in the final processing stage to comprehensively summarize and refine the agent's review results, generating a complete review report. This processing significantly optimizes the presentation of the review results, improving their readability. Unlike directly using the raw and crude API interface returns, the review results refined by the long-term thinking model more closely align with the reviewers' actual business logic. This improvement not only greatly enhances the readability of the review results but also significantly strengthens their usability in practical business applications. Attached Figure Description

[0020] To more clearly illustrate the technical solution of the present invention, the accompanying drawings used in the description will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 A flowchart illustrating the intelligent document review method provided in this application.

[0022] Figure 2 A flowchart illustrating another intelligent document review method provided in this application.

[0023] Figure 3 A flowchart illustrating the document data extraction method provided in this application.

[0024] Figure 4 A schematic diagram of the structure of the intelligent document review system provided in this application.

[0025] Figure 5 A schematic diagram of the structure of the electronic device provided in this application. Detailed Implementation

[0026] The various embodiments of this disclosure will be described more fully in the detailed steps of the intelligent document review method described below. This disclosure may have various embodiments, and adjustments and changes may be made therein. However, it should be understood that there is no intention to limit the various embodiments of this disclosure to the specific embodiments disclosed herein, but rather this disclosure should be understood to cover all adjustments, equivalents, and / or alternatives falling within the spirit and scope of the various embodiments of this disclosure.

[0027] In the following, the terms “comprising” or “may include”, which may be used in various embodiments of this disclosure, indicate the presence of the disclosed functions, operations, or elements, and do not limit the addition of one or more functions, operations, or elements. Furthermore, as used in various embodiments of this disclosure, the terms “comprising,” “having,” and their cognates are intended only to indicate a particular feature, number, step, operation, element, component, or combination of the foregoing, and should not be construed as primarily excluding the presence of one or more other features, numbers, steps, operations, elements, components, or combinations of the foregoing, or the possibility of adding one or more combinations of the foregoing.

[0028] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0029] Please see Figure 1 The diagram shows a flowchart of a smart document review method in a specific embodiment. The method includes: S1: Collect scanned document files of different formats, perform format unification processing, and generate document images.

[0030] In a specific implementation, an external tool is invoked to convert scanned document files in different formats such as PDF, PNG, and JPEG into PNG format document images for subsequent extraction and processing.

[0031] S2: By calling pre-trained models and interfaces, the system performs seal area detection and separation, table structure recognition, text content extraction, and key information anchoring and integration on document images to extract basic document information, list table information, and seal information, thereby generating structured document information.

[0032] It should be noted that after generating structured document information, this method still requires data summarization and grouping, as well as data format reconstruction.

[0033] Specifically, the first step is to extract information items from the list table information. Based on the product type, the product type of the document is divided into multiple categories such as server, operating system, storage, and network, in order to adapt to different audit logics in the future. This step mainly relies on syntax matching to achieve classification accuracy.

[0034] Then, the document information is reconstructed according to the current system input data format, and the document information is reconstructed to the current system input data format, so as to resolve the difference between the document information and the system input data format and provide convenience for subsequent processing steps.

[0035] S3: Using a pre-set basic information auditing agent, the basic information of the document is compared and audited to ensure consistency of the unit name, contact person, contact information, and project type.

[0036] In a specific implementation, a pre-set basic information review agent is used to compare and review the basic information of the document based on the extracted results of the reconstructed document and the information input by the system. This process verifies whether the contents of fields such as project name, unit name, contact person, contact information, and project type are consistent between the document and the system.

[0037] S4: Use the pre-set checklist information to review the intelligent agent, and review the content of each row in the checklist based on the checklist information.

[0038] In a specific implementation, based on the list table information and combined with the relevant data of the current system, a preset list table information auditing agent is used to audit the content of each row in the list table, checking whether the configuration information, quantity information, time information and name are completely consistent with the relevant data of the system.

[0039] For example, the intelligent agent uses list table information to audit the data. Based on the restructured structured list table information and the relevant information input by the system, it conducts a detailed check on the content of each row in each table to check whether the specific information such as configuration, quantity, time and name are completely consistent between the document and the system.

[0040] S5: Use preset seal information to audit the intelligent agent and verify the consistency of the seal information with other relevant information.

[0041] In a specific implementation, a pre-set intelligent agent for verifying seal information is used to check the seal information against the basic information of the document and the unit name and subject name recorded in the list table information to verify whether the information is consistent.

[0042] For example, using the seal information verification agent, the seal information obtained by the visual basic model is checked one by one against the unit name, subject name and other related information in other locations on the document to verify whether the above information in different locations is strictly consistent.

[0043] S6: Use a large language model to optimize the review results of the three agents and generate a review report. In a specific implementation, a large language model with long-term thinking ability is used to optimize the review results generated by the three intelligent agents, forming a review report that can be easily read and referenced by business personnel. At the same time, omissions in the review process are corrected, and the review results of the intelligent agents are processed.

[0044] In an exemplary embodiment, step S2 of the above method extracts the basic information of the document, the list table information, and the seal information, specifically including the following sub-steps: S201: Using a preset seal area detection model, identify the coordinate areas of all seals in the document image to determine the seal area.

[0045] For example, the seal region detection uses a pre-trained seal region detection model to identify the coordinate regions of all seals in the input document image. The training corpus of this model covers more than 1,500 document images and corresponding manually annotated seal position information.

[0046] S202: Perform a separation operation on the seal area in the document image and extract the seal image.

[0047] For example, the seal area separation operation separates the area containing the seal on the document, reducing the adverse effects of seal obscuring on the subsequent table structure recognition and text content recognition process, and ensuring accurate and efficient seal information recognition.

[0048] S203: For document images where the stamp area has been removed, a preset table structure recognition model is used to detect the table structure of the document image, locate and obtain the area coordinates of each cell in the table structure.

[0049] For example, table structure detection uses a pre-trained table structure recognition model to detect the table structure within a document image where the stamp area has been removed. This accurately locates and obtains the coordinates of each cell in the table, providing basic data for subsequent processing.

[0050] S204: Using a preset text region detection model, accurately identify the coordinate regions of all text in the document image where the stamp region has been removed, obtain the coordinate data of the four endpoints of each text line region, and determine the text region.

[0051] For example, text region detection uses a pre-trained text region detection model to accurately identify the coordinate regions of all text in a document image where the stamp region has been removed, and obtains the coordinate data of the four endpoints of each text line region.

[0052] S205: Associate the coordinate data of the four endpoints of each text line area with the area coordinates of each cell in the table structure, perform redundant part deletion and area expansion processing on the text area, and update the text area.

[0053] For example, text region supplementation correlates the recognition results with the coordinates of the table structure, processes the recognized text regions, removes redundant parts, supplements missing text regions, compensates for the limitations of the text region detection model, and improves the accuracy of subsequent text content recognition.

[0054] S206: Using a preset text content recognition model, the text content contained in the text area is recognized and the text line content is extracted.

[0055] For example, text content recognition uses a pre-trained text content recognition model to identify the text content contained in the text region obtained after the first two sub-steps.

[0056] S207: Based on the text area and text line content, perform a recall operation on redundant text areas caused by recognition errors, and generate the text content of the document image.

[0057] For example, text region recall, based on the defined text line region and the recognized text line content, performs a recall operation on redundant text regions caused by recognition errors, thus solving the text duplication problem caused by overlapping text regions.

[0058] S208: Associate the area coordinates of each cell in the table structure with the text content of the document image, match the cell position where the text row area is located, and establish the relationship between the table structure and the text content in terms of spatial coordinates.

[0059] For example, the detection region association operation associates the detection results of the table structure with the recognition results of the text content, matches the cell position where the text line area is located, and establishes a connection between the table structure and the text content at the spatial coordinate level.

[0060] S209: Employ key syntax matching technology to anchor table areas containing preset key values, anchor the header row containing basic information, and determine the coordinate information of the header row.

[0061] For example, key area anchoring uses key syntax matching technology to anchor table areas containing key values. The anchored areas include name-type content in the basic information and the header row of the list table. By using the coordinate positions of these key areas, the basic information in the document image is accurately matched and associated with the specific content in the list table to integrate complete information.

[0062] S210: Based on the coordinate information of the header row, and combined with the relationship between the table structure and the text content in spatial coordinates, match the specific values ​​in the header row as the basic information of the document.

[0063] For example, the basic information value is located by matching the specific value corresponding to the basic information based on the key area coordinates anchored in step S209 and the coordinate relationship between the text line area and the table cell area.

[0064] S211: Based on the position information of all cells in the header row, and by leveraging the relationship between the table structure and the text content in spatial coordinates, locate the coordinate information and specific values ​​of all cells corresponding to each column area, and determine the coordinate information and specific values ​​of all cells in the list table.

[0065] For example, in the list table numerical positioning, based on the position information of all cells in the list table header row anchored in step S209, the content of all cells corresponding to each column area is located by using the correlation between spatial coordinates.

[0066] S212: Traverse the coordinate information of all cells in the list table to obtain the position information of the first cell of each row in the list table, and generate the list table information by combining the coordinate information and specific values ​​of all cells in the list table.

[0067] For example, the system iterates through the cell positions in the located list table to obtain the coordinates of the smallest unit, or "atomic row," in each row. By iterating through the detected cell positions based on the list table structure, the system accurately obtains the coordinates of the "atomic row" in each row, avoiding errors in table content recognition caused by merging cells.

[0068] S213: For the seal image, a preset seal text recognition model is used to identify the valid information contained therein and generate seal information.

[0069] For example, in the seal content recognition, a pre-trained seal text recognition model is used to identify the valid information contained in the seal image obtained in step S202. This model has a high recognition accuracy for curved text areas and deformed characters within the seal.

[0070] S214: Integrate and process the basic information, list table information and seal information of the document to generate structured document information.

[0071] For example, information extraction and integration involves combining the basic information, list table information, and seal information of the identified documents and outputting them in a unified manner.

[0072] In this embodiment, a mechanism employing multiple visual basic models working collaboratively is first proposed to accurately extract key information from document files. Then, multiple collaborative document review agents are used to review and compare the extracted key information. Finally, a long-thinking large-scale model is used to refine and summarize the review results to generate a complete document review report. Compared to traditional document review methods that primarily rely on manual review, this method effectively improves document review efficiency, reduces the risk of omissions or misjudgments caused by human error, and significantly reduces labor costs.

[0073] Compared to other intelligent document review solutions, this method innovatively utilizes five types of basic visual models working collaboratively to comprehensively extract key information from documents. This information includes basic information, list / table information, and seal information. Compared to using multimodal large models to extract information, the collaborative approach of multiple basic visual models not only has advantages such as greater operational flexibility, lower cost, and easier optimization, but also effectively avoids the limitations of the visual understanding network structure in multimodal large models. Furthermore, this method employs three agents with collaborative working capabilities to review the basic information, list / table information, and seal information extracted from the formatted document, respectively. Finally, a large language model with long-term thinking capabilities is used to comprehensively summarize and deeply optimize the review results of these three agents to improve the readability and accuracy of the review.

[0074] Furthermore, as a refinement and extension of the specific implementation methods of the above embodiments, in order to fully illustrate the specific implementation process in this embodiment, as follows: Figure 2 As shown, this application also provides another intelligent document review method, including the following steps: Step 1 (File Format Conversion): By calling an external tool, the scanned document files in different formats such as PDF, PNG, and JPEG are uniformly converted into PNG format to facilitate subsequent extraction and processing.

[0075] Step 2 (Document Information Extraction): Call a separate document information extraction interface to extract the structured document content. The extracted content covers three categories of key information: basic information, list / table information, and seal information.

[0076] Step 3 (Data Summarization and Grouping): Based on the extracted list of information items, categorize them into multiple categories according to product type, such as server, operating system, storage, and network. These categories will be used to adapt to different review logics later. This step is mainly implemented using syntax matching to ensure the accuracy of the classification.

[0077] Step 4 (Data Format Reconstruction): Given the significant differences in format between the extracted document file information and the system input data, this step reconstructs the information from different sources into a unified structure, thereby facilitating subsequent processing steps.

[0078] Step 5 (Basic Information Verification Agent): Based on the extracted results of the reconstructed documents, the basic information of the documents is compared and verified in conjunction with the information input into the system. Specifically, this step is mainly used to verify whether the content of fields such as project name, unit name, contact person, contact information, and project type is consistent between the document and the system.

[0079] Step 6 (Intelligent Agent for Reviewing List Table Information): Based on the structured list table information formed after reconstruction, and combined with the relevant information input by the system, conduct a detailed check on the content of each row in each table to verify whether the specific information such as configuration, quantity, time, and name are completely consistent between the document and the system.

[0080] Step 7 (Seal Information Verification Agent): For the seal information obtained by the visual basic model, check it one by one with the unit name, subject name and other related information appearing in other locations on the document to verify whether the above information in different locations is strictly consistent.

[0081] Step 8 (Summary of Review Results): Using a large-scale language model with long-term thinking capabilities, the review results generated by the three agents are processed and optimized to create a review report that is easy for business personnel to read and reference. Simultaneously, the large-scale language model corrects any oversights that occur during the review process and further aligns the agents' review results with the actual review procedures of business personnel, thereby improving the accuracy and ease of use of the review results.

[0082] In one embodiment of the present invention, based on step 2, the following will provide a possible embodiment and describe its specific implementation in a non-limiting manner.

[0083] refer to Figure 3 As shown in the figure, this embodiment discloses a method for extracting document data, which specifically includes the following steps: Step 2.1 (Seal Region Detection): For the input document image, this step uses a pre-trained seal region detection model to identify the coordinate regions of all seals in the document image. The training corpus for this seal detection model covers more than 1,500 document images and their corresponding manually labeled seal location information.

[0084] Step 2.2 (Seal Area Separation): After identifying the seal location information in the previous step, this step separates the area containing the seal on the document. This separation effectively reduces the adverse effects of seal obscuring on subsequent table structure and text content recognition processes. Furthermore, this measure helps ensure more accurate and efficient seal information recognition.

[0085] Step 2.3 (Table Structure Extraction): The table structure extraction step includes table structure detection, which is a sub-step.

[0086] Step 2.3-1 (Table Structure Detection): For the document image with the stamp area removed, a pre-trained table structure recognition model is used to detect the table structure within the document, accurately locate and obtain the area coordinates of each cell in the table, thereby providing basic data for further processing of the table information in the document.

[0087] Step 2.4 (Text Content Extraction): The text content extraction step includes four sub-steps: text region detection, text region completion, text content recognition, and text region recall. Step 2.4-1 (Text Region Detection): For document images with the stamp area removed, a pre-trained text region detection model is used to accurately identify the coordinate regions of all text within the document. During the recognition process, the acquired information specifically covers the coordinate data of the endpoints around each text line region. This coordinate data provides accurate location data for subsequent text content recognition, thus ensuring the effectiveness and accuracy of the text content recognition.

[0088] Step 2.4-2 (Text Region Supplementation): When processing document images, due to significant differences in image quality, some low-quality images may cause problems in text region recognition. Specifically, this manifests as redundant information in the recognition results or the omission of some text regions. To address this, this step processes the identified text regions by associating the recognition results with the coordinates of the table structure, removing redundant parts and supplementing any missing text regions. This method effectively compensates for the limitations of the text region detection model, thereby improving the accuracy of subsequent text content recognition.

[0089] Step 2.4-3 (Text Content Recognition): For the text regions obtained after processing in the previous two sub-steps, this step uses a pre-trained text content recognition model to recognize the text content contained in the identified text line regions.

[0090] Step 2.4-4 (Text Region Recall): Based on the defined text line regions and the text line content obtained after recognition, further recall operations are performed on redundant text regions caused by recognition errors. The core purpose of this step is to solve the text duplication problem caused by overlapping text regions. Specifically, the method used is to determine whether the text regions are disjoint, intersecting, or overlapping based on their spatial relationships. For multiple text regions that intersect or overlap, recall is performed according to the specific text recognition situation.

[0091] Step 2.5 (Detection Region Association): The detection results of the table structure are associated with the recognition results of the text content to match the cell position of the text line area, thereby establishing a connection between the table structure and the text content in terms of spatial coordinates.

[0092] Step 2.6 (Key Area Anchoring): Based on the table structure and text content established in the previous step, key syntax matching technology is used to anchor table areas containing key values. In this process, the key table areas of interest mainly cover two categories: first, name-related content in basic information, such as "Project Name" and "Unit Name," which are crucial for accurately identifying the main information of the document; second, the header rows of the list table, such as "Configuration," "Name," and "Quantity," which are key identifiers for understanding the table's content structure. By using the coordinate positions of these key table areas, the basic information contained in the document image can be further precisely matched with the specific content in the list table, thereby integrating and reconciling the complete information contained in the document.

[0093] Step 2.7 (Basic Information Recognition): The basic information recognition step includes basic information numerical location, which is one sub-step.

[0094] Step 2.7-1 (Basic Information Value Location): Based on the key area coordinates anchored in Step 2.6, and combined with the coordinate relationship between the text line area and the table cell area, match the specific values ​​corresponding to the basic information, such as the specific values ​​corresponding to the project name and the unit name.

[0095] Step 2.8 (List Table Recognition): The list table recognition step includes three sub-steps: list table value localization, list table edge detection, and list table row value traversal.

[0096] Step 2.8-1 (List Table Value Location): Based on the position information of all cells in the list table header row anchored in Step 2.6, and by using the relationship between spatial coordinates, locate all cell contents corresponding to each column area.

[0097] Step 2.8-2 (List Table Edge Detection): Traverse the cell positions in the located list table to obtain the coordinate position of the smallest unit in each row of the list table, i.e., the "atomic row".

[0098] Step 2.8-3 (List Table Row Value Traversal): For the list table structure, traverse the detected cell positions. Through this traversal process, accurately obtain the coordinate information of the smallest unit in each row of the list table, that is, the "atomic row". This step can effectively avoid table content recognition errors caused by merging cells and improve adaptability to complex table types.

[0099] Step 2.9 (Seal Information Extraction): Seal information extraction includes seal content recognition, which is one sub-step.

[0100] Step 2.9-1 (Seal Content Recognition): For the seal image obtained in Step 2.2, a pre-trained seal text recognition model is used to identify the valid information it contains. This seal text recognition model has been optimized and performs well in recognizing curved text areas within the seal, and also has a high recognition accuracy for deformed text on the seal.

[0101] Step 2.10 (Information Extraction and Integration): The key information identified in Steps 2.7, 2.8 and 2.9 is integrated and processed, and then output in a unified manner.

[0102] like Figure 4 As shown, the following are embodiments of the intelligent document review system provided in this disclosure. This system and the intelligent document review methods in the above embodiments belong to the same inventive concept. For details not described in detail in the embodiments of the intelligent document review system, please refer to the embodiments of the intelligent document review methods described above.

[0103] An intelligent document approval system includes: The document acquisition module is used to acquire scanned document files of different formats, perform format unification processing, and generate document images; The document information extraction module is used to extract basic information, list table information and seal information from document images by calling pre-trained models and interfaces, and to generate structured document information. The basic information verification module is used to compare and verify the basic information of documents using a preset basic information verification intelligent agent, and to verify the consistency of unit name, contact person, contact information, and project type. The list table information review module is used to review the contents of each row in the list table based on the pre-set list table information review intelligent agent. The seal information verification module is used to verify the consistency of seal information with other relevant information using a pre-set seal information verification intelligent agent. The audit summary module is used to optimize the audit results of the three agents using a large language model and generate an audit report.

[0104] The intelligent document review system provided in this embodiment collects documents of different formats and unifies them into PNG format. It utilizes a pre-trained model to achieve seal separation, table structure recognition, and accurate text content extraction. Combined with key syntax matching technology, it anchors key information and generates structured document information. After product type identification and data format reconstruction to adapt to system requirements, three intelligent agents review basic information, list table content, and seal consistency. Finally, a large-scale language model optimizes the review results and generates a review report. This system achieves standardized processing of multi-format documents, improves the accuracy and efficiency of seal, table, and text information extraction, ensures the consistency and accuracy of data such as unit names and configuration information through agent-based review, enhances the reliability of review results through large-scale language model optimization, and allows document information to better adapt to system requirements through data format reconstruction. This effectively improves the automation level, accuracy, and processing efficiency of document review, while reducing manual review costs and error probability.

[0105] Figure 5 A schematic diagram of the hardware structure of an electronic device for implementing various embodiments of the present invention.

[0106] The intelligent document verification method provided in this application can be applied to electronic devices. Those skilled in the art will understand that the electronic device structure involved in the embodiments of this invention does not constitute a limitation on the electronic device. An electronic device may include more or fewer components than illustrated, or combine certain components, or have different component arrangements. In the embodiments of this invention, the electronic device includes, but is not limited to, laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the embodiments of this application described and / or claimed herein.

[0107] Electronic devices may include processors, external memory interfaces, internal memory, universal serial bus (USB) interfaces, charging management modules, power management modules, batteries, wireless communication modules, audio modules, speakers, microphones, sensor modules, buttons, cameras, displays, and SIM card interfaces, etc.

[0108] A processor may include one or more processing units, such as: a central processing unit (CPU), an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural network processing unit (NPU). Different processing units may be independent devices or integrated into one or more processors.

[0109] The processor can serve as the nerve center and command center of an electronic device. The controller can generate operation control signals based on the instruction opcode and timing signals to control the fetching and execution of instructions.

[0110] The processor may also include memory for storing instructions and data. In some embodiments, the memory in the processor is a cache memory. This memory can store instructions or data that the processor has just used or that are used repeatedly. If the processor needs to use the instruction or data again, it can retrieve it directly from this memory. This avoids repeated accesses, reduces processor latency, and thus improves system efficiency.

[0111] An external storage interface (ESI) can be used to connect external memory cards, such as microSD cards, to expand the storage capacity of electronic devices. The external memory card communicates with the processor through the ESI to perform data storage functions, such as saving music and video files on the external memory card.

[0112] Internal memory can be used to store computer executable program code, which includes instructions. The processor executes various functional applications and data processing of electronic devices by running the instructions stored in internal memory. Internal memory can include a program storage area and a data storage area. Internal memory can include high-speed random access memory, and can also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc.

[0113] Wireless communication functionality in electronic devices can be achieved through antennas, wireless communication modules, modem processors, and baseband processors.

[0114] Wireless communication modules can provide solutions for wireless communication applications in electronic devices, including wireless local area networks (WLANs) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technologies.

[0115] Electronic devices can implement audio functions through audio modules, speakers, receivers, microphones, headphone jacks, and application processors.

[0116] Electronic devices can achieve shooting functions through ISPs, cameras, video codecs, GPUs, displays, and application processors.

[0117] Electronic devices can achieve display functions through GPUs, displays, and application processors.

[0118] A GPU is a microprocessor for image processing, connected to the display screen and application processor. GPUs are used to perform mathematical and geometric calculations for graphics rendering. A processor may include one or more GPUs, which execute program instructions to generate or modify display information.

[0119] A display screen is used to display images, videos, etc. A display screen includes a display panel.

[0120] The aforementioned electronic device realizes the intelligent document review method of this application. It collects scanned documents of different formats and processes them into PNG format. Combined with a pre-trained model, it performs seal detection and separation, table structure recognition, text extraction, and key information anchoring and integration to generate structured document information. After product type recognition and data format reconstruction, three intelligent agents are used to review basic information, list table content, and seal consistency. Finally, a large-scale language model is used to optimize the review results. This achieves the beneficial effects of standardizing multi-format document processing, improving the accuracy and efficiency of information extraction, ensuring data consistency and accuracy, enhancing the reliability of review results, and adapting to system data format requirements. As a result, it effectively improves the automation level, accuracy, and processing efficiency of document review, and reduces labor costs and error probability.

[0121] The storage medium provided in this application stores a program product capable of implementing an intelligent document review method.

[0122] Intelligent document approval methods include: Collect scanned document files of different formats, perform format unification processing, and generate document images; By calling pre-trained models and interfaces, the system performs seal area detection and separation, table structure recognition, text content extraction, and key information anchoring and integration on document images to extract basic document information, list table information, and seal information, thereby generating structured document information. Using a pre-set basic information review agent, the basic information of the documents is compared and reviewed to ensure consistency of the unit name, contact person, contact information, and project type. The intelligent agent is reviewed using a pre-set list table information. Based on the list table information, the content of each row in the list table is reviewed. The system uses pre-set seal information to audit the intelligent agent and verify the consistency of the seal information with other relevant information. A large-scale language model is used to optimize the review results of the three agents and generate a review report. In some possible implementations, the document review method of this disclosure can be implemented as a program product that includes program code that, when the program product is run on a terminal device, causes the terminal device to perform the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of this disclosure.

[0123] The storage medium disclosed herein may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof.

[0124] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An intelligent document review method, characterized in that, include: Collect scanned document files of different formats, perform format unification processing, and generate document images; By calling pre-trained models and interfaces, the system performs seal area detection and separation, table structure recognition, text content extraction, and key information anchoring and integration on document images to extract basic document information, list table information, and seal information, thereby generating structured document information. Using a pre-set basic information review agent, the basic information of the documents is compared and reviewed to ensure consistency of the unit name, contact person, contact information, and project type. The intelligent agent is reviewed using a pre-set list table information. Based on the list table information, the content of each row in the list table is reviewed. The system uses pre-set seal information to audit the intelligent agent and verify the consistency of the seal information with other relevant information. A large-scale language model is used to optimize the review results of the three agents and generate a review report.

2. The intelligent document review method according to claim 1, characterized in that, The process of collecting scanned document files in different formats, performing format unification processing, and generating document images includes: Collect scanned document files in PDF, PNG, and JPEG formats, and use image processing tools to convert them into PNG document images.

3. The intelligent document review method according to claim 2, characterized in that, The process involves calling pre-trained models and interfaces to perform seal area detection and separation, table structure recognition, text content extraction, and key information anchoring and integration on document images. This extracts basic document information, list table information, and seal information, generating structured document information, including: Using a pre-defined stamp area detection model, the coordinate areas of all stamps in the document image are identified to determine the stamp area. Separate the stamp area from the document image and extract the stamp image; For document images with the stamp area removed, a preset table structure recognition model is used to detect the table structure of the document image, locate and obtain the area coordinates of each cell in the table structure.

4. The intelligent document review method according to claim 3, characterized in that, The process involves calling pre-trained models and interfaces to perform seal area detection and separation, table structure recognition, text content extraction, and key information anchoring and integration on document images to extract basic document information, list table information, and seal information, generating structured document information. This also includes: Using a pre-defined text region detection model, the coordinate regions of all text in a document image with the stamp area removed are accurately identified, and the coordinate data of the four endpoints of each text line region are obtained to determine the text region. The coordinates of the four endpoints of each text line area are associated with the area coordinates of each cell in the table structure. Redundant parts of the text area are deleted and the area is expanded. The text area is then updated. A pre-defined text content recognition model is used to identify the text content contained in the text area and extract the text line content. Based on the text area and text line content, a recall operation is performed on redundant text areas caused by recognition errors to generate the text content of the document image; The coordinates of each cell in the table structure are associated with the text content of the document image to match the cell position of the text row area and establish the relationship between the table structure and the text content in terms of spatial coordinates. Using key syntax matching technology, table areas containing preset key values ​​are anchored to identify header rows containing basic information and determine the coordinate information of the header rows.

5. The intelligent document review method according to claim 4, characterized in that, The process involves calling pre-trained models and interfaces to perform seal area detection and separation, table structure recognition, text content extraction, and key information anchoring and integration on document images to extract basic document information, list table information, and seal information, generating structured document information. This also includes: Based on the coordinate information of the header row, and combined with the relationship between the table structure and the text content in the spatial coordinate plane, the specific values ​​in the header row are matched as the basic information of the document. Based on the position information of all cells in the header row, and by leveraging the relationship between the table structure and the text content in spatial coordinates, the coordinate information and specific values ​​of all cells corresponding to each column area are located, thus determining the coordinate information and specific values ​​of all cells in the list table. Iterate through the coordinates of all cells in the list table to obtain the position information of the first cell in each row of the list table, and combine the coordinates and specific values ​​of all cells in the list table to generate the list table information. For the seal image, a preset seal text recognition model is used to identify the valid information contained therein and generate seal information; The basic information, list and table information, and seal information of the documents are integrated and processed to generate structured document information.

6. The intelligent document review method according to claim 5, characterized in that, After generating the structured document information, the process also includes: Information items are extracted from the list table information, and the information items are compared with the preset product types to identify the product type of the document; the product types of the document include server, operating system, storage, and network. The document information is reconstructed according to the current system input data format, and its data format is converted into the current system input data format.

7. The intelligent document review method according to claim 6, characterized in that: The intelligent agent that uses a preset list table information to review the content of each row in the list table, including: Based on the list table information and combined with the relevant data of the current system, the pre-set list table information auditing agent audits the content of each row in the list table, checking whether the configuration information, quantity information, time information and name are completely consistent with the relevant data of the system. The intelligent agent that uses preset seal information to verify the consistency of seal information with other relevant information includes: Using a pre-set intelligent agent to review seal information, the agent checks the seal information against the basic information of the document and the unit name and subject name recorded in the list table to verify whether the information is consistent.

8. An intelligent document review system, characterized in that, The system employs the intelligent document review method as described in any one of claims 1 to 7; The system includes: The document acquisition module is used to acquire scanned document files of different formats, perform format unification processing, and generate document images; The document information extraction module is used to extract basic information, list table information and seal information from document images by calling pre-trained models and interfaces, and to generate structured document information. The basic information verification module is used to compare and verify the basic information of documents using a preset basic information verification intelligent agent, and to verify the consistency of unit name, contact person, contact information, and project type. The list table information review module is used to review the contents of each row in the list table based on the pre-set list table information review intelligent agent. The seal information verification module is used to verify the consistency of seal information with other relevant information using a pre-set seal information verification intelligent agent. The audit summary module is used to optimize the audit results of the three agents using a large language model and generate an audit report.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the intelligent document review method as described in any one of claims 1 to 7.

10. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the intelligent document review method as described in any one of claims 1 to 7.

Images