A document element extraction and review system based on a large model

Abstract

This invention relates to a document element extraction and review system based on a large model, comprising a document element extraction module, a main text element extraction module, and an element review module. The document element extraction module is used to automatically identify and extract key elements of the document. The main text element extraction module is used to parse the main text of the document to extract ordered and structured information. The element review module is used to review each element of the document item by item to ensure the accuracy and completeness of each element. This invention has the advantages of automated and intelligent document processing, improving the efficiency and accuracy of document element extraction and review.

Description

Technical Field

[0001] This invention relates to the field of information processing technology, and in particular to a document element extraction and review system based on a large model. Background Technology

[0002] In government and business management, document processing is a fundamental and crucial task, involving the transmission of information, the release of policies, and the recording of decisions. Traditional document processing methods mainly rely on manual operation, including reading, understanding, and extracting key information from documents, as well as subsequent review processes. These processes are not only time-consuming and labor-intensive, but also prone to errors and omissions due to human factors.

[0003] With the acceleration of informatization, the quantity and complexity of official documents are constantly increasing, which brings greater challenges to document processing. Existing document processing technologies have achieved automation to a certain extent, and model technology has made significant progress in the application of natural language processing, providing possible solutions for automated document processing. However, the existing document structures and contents are often complex and diverse, and there is still room for improvement in processing efficiency and accuracy. Summary of the Invention

[0004] In view of the shortcomings of the prior art, the technical problem to be solved by this patent application is how to provide an automated and intelligent document processing system that improves the efficiency and accuracy of document element extraction and review.

[0005] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:

[0006] A document element extraction and review system based on a large model includes a document element extraction module, a main text element extraction module, and an element review module. The document element extraction module is used to automatically identify and extract key elements of the document. The main text element extraction module is used to parse the main text of the document to extract ordered and structured information. The element review module is used to review each element of the document to ensure the accuracy and completeness of each element.

[0007] As an optimization, the document element extraction module is the part of this system used to automatically identify and extract key elements of documents. The document element extraction module includes the steps of data preprocessing, named entity recognition, element extraction and verification, output and optimization.

[0008] As an optimization, data preprocessing first collects various styles of official document texts, then unifies the format and cleans the data, segmenting the entire document text into independent sentences or paragraphs; Named Entity Recognition (NER) first defines entity categories, then extracts text features such as word features, character features, position features, and part-of-speech features, trains a NER model, and then uses the NER model to identify each entity and its category in the document text; Element extraction and verification extracts each element from the document based on the output of the NER model, performs rule verification, and uses context information to verify the rationality of the extracted entities; Output and optimization steps first output the extracted elements and their categories in the form of structured data, then provide a user interface for user feedback on the extraction results, the system adjusts the NER model based on the feedback, then fine-tunes it based on the pre-trained large model through transfer learning, performs entity disambiguation through context information and entity relationship networks, and finally combines multiple different NER models through a weighted average method.

[0009] As an optimization, the text element extraction module parses the main body of the document to extract ordered and structured information, including the main body subheadings, the content corresponding to the main body subheadings, the attachment titles and the attachment content. This includes steps such as text segmentation and identification, extraction of the content corresponding to the main body subheadings, extraction of the attachment titles and content, and special format processing.

[0010] As an optimization, text segmentation and identification first uses regular expressions to divide the main body of the document into independent paragraphs. The system uses contextual information to understand the content under each subheading, and then uses a large model to identify the subheading of each paragraph. When extracting the content corresponding to the main body subheading, the system locates the content under each subheading, extracts the main body content corresponding to the subheading, and associates it with the subheading to form structured data. When extracting attachment titles and content, the system uses keyword search to identify attachments or match the content adjacent to the attachment, locates the starting position of the attachment, and then extracts the attachment title. When extracting attachment content, for text attachments, the system will extract all text content after the attachment title; for non-text attachments, the system will extract the attachment file name information. Finally, special formats are processed. For tables in the main body or attachments, they are first processed into Markdown format, and then the large model is used to identify the table structure and extract the information in the table. For attachments containing images, the system will extract the descriptive text of the image or the image file name.

[0011] As an optimization, the element review module uses a large model combined with a rule engine to review each element of the document, ensuring the accuracy and completeness of each element, including steps such as title review, content review, and review of other elements.

[0012] As an optimization, the title review first establishes a standard for judging correct titles based on common errors in official document titles. Then, according to the standard, an online model is used to generate negative data for official document titles to ensure coverage of all possible errors. The data is then checked and fine-tuned using LoRA. After fine-tuning, the LM-Cocktail tool is used to merge the fine-tuned model with the base model. The merged model is then used to review common errors in official document titles. Finally, based on the semantic capabilities of the large model, the consistency between the theme of each title and its corresponding content is reviewed. For subheadings in the main text, the semantic relationships between subheadings are also reviewed. The content review first examines the grammar, semantics, logic, and standard usage of the main text and attachments, establishing a positive... First, establish clear standards. Then, select representative positive and negative examples from the historical document database or call online models. The online model labels errors in the dataset and verifies, corrects, and scores the data. The model is then fine-tuned and merged. A trained syntax review model is used to review the main text and attachments of the official documents. For other elements, regular expression matching is used, combined with the issuing authority information, to verify the correctness of the document number. The accuracy of the addressee's name is verified by matching the organization name, responsibilities, and level information in the knowledge base. The accuracy of the issuing authority's name is also verified by matching the organization name in the knowledge base. Finally, a date parsing library is used to check whether the document issuance time conforms to logical order and whether the time format is correct.

[0013] The present invention has the following advantages:

[0014] (1) Improve document processing efficiency: This method and system can quickly extract key elements from documents and review them, which can shorten the document processing time and improve office efficiency.

[0015] (2) Ensure the accuracy of official documents: Through in-depth understanding and intelligent review of the large model, this method and system effectively ensure the accuracy of document elements and reduce the probability of document errors.

[0016] (3) Reduce manual workload: Automated extraction and review of document elements reduces the amount of manual work involved in tedious review tasks, allowing staff to devote more energy to other tasks.

[0017] (4) Improve the level of automation in document processing: The implementation of this patent is conducive to the development of document processing towards automation and intelligence, and also conducive to improving the modernization level of the entire document processing process.

[0018] (5) Strong applicability: The system can perform differentiated review for different types of official document elements, adapting to diverse official document processing needs and having wide applicability.

[0019] (6) Promote standardized management: Through unified review standards and methods, this patent helps to achieve standardized management of official document processing and improve the quality and standardization of official documents. Attached Figure Description

[0020] Figure 1 This is a framework diagram of a document element extraction and review system based on a large model, as described in this invention. Detailed Implementation

[0021] The present invention will now be described in further detail with reference to the accompanying drawings. In the description of the present invention, it should be understood that directional terms such as "upper," "lower," "top," and "bottom" indicate directions or positional relationships based on the directions or positional relationships shown in the accompanying drawings. These terms are used only for the convenience of describing the present invention and simplifying the description. Unless otherwise stated, these directional terms do not indicate or imply that the device or element referred to must have a specific orientation or be constructed and operated in a specific orientation, and therefore should not be construed as limiting the scope of protection of the present invention. The directional terms "inner" and "outer" refer to the inner or outer contours relative to the outline of each component itself.

[0022] like Figure 1 As shown, a document element extraction and review system based on a large model includes a document element extraction module, a main text element extraction module, and an element review module. The document element extraction module is used to automatically identify and extract key elements of the document. The main text element extraction module is used to parse the main text of the document to extract ordered and structured information. The element review module is used to review each element of the document to ensure the accuracy and completeness of each element.

[0023] In this embodiment, the document element extraction module is the part of the system used to automatically identify and extract key elements of documents. The document element extraction module includes the steps of data preprocessing, named entity recognition, element extraction and verification, output and optimization.

[0024] In this embodiment, data preprocessing first collects various styles of official document texts, then unifies the format and cleans the data, segmenting the entire document text into independent sentences or paragraphs; Named Entity Recognition (NER) first defines entity categories, then extracts text features such as word features, character features, position features, and part-of-speech features, trains a NER model, and then identifies each entity and its category in the document text through the NER model; Element extraction and verification extracts each element in the document based on the output of the NER model, performs rule verification, and uses context information to verify the rationality of the extracted entities; Output and optimization steps first output the extracted elements and their categories in the form of structured data, then provide a user interface for user feedback on the extraction results, the system adjusts the NER model based on the feedback, then fine-tunes it based on the pre-trained large model through transfer learning, performs entity disambiguation through context information and entity relationship networks, and finally combines multiple different NER models through a weighted average method.

[0025] Specifically, during data preprocessing, document collection involves gathering official document texts of different formats and types, with as many styles as possible; format unification involves converting official document texts of different formats into a unified text format to facilitate subsequent processing; document text cleaning involves removing special characters, whitespace, and redundant spaces from the official document texts to ensure the neatness of the document content; and sentence segmentation involves dividing the entire document text into independent sentences or paragraphs to provide basic units for NER.

[0026] Specifically, in Named Entity Recognition (NER), entity category definition is based on the characteristics of official documents, defining entity categories, i.e., the various elements of the document, including title, document number, addressee, issuing authority, issuing time, and attachment titles; feature engineering extracts text features, including word features, character features, positional features, and part-of-speech features, to provide input for the NER model; model training uses a labeled document dataset to train the NER model, which can use BiLSTM-CRF or BERT models; entity recognition inputs the preprocessed document text into the trained NER model to identify the various entities in the text and their categories.

[0027] Specifically, in the element extraction and validation process, element extraction involves extracting various elements from the document based on the output of the NER model; rule validation involves performing rule validation on the extracted elements, for example, document numbers usually contain specific formats, which can be validated using regular expressions; and context validation involves using contextual information to validate the rationality of the extracted entities.

[0028] Specifically, in the output and optimization process, structured output involves outputting the extracted elements and their categories in the form of structured data, such as JSON or XML format; error feedback provides a user interface that allows users to provide feedback on the extraction results, and the system adjusts the NER model based on the feedback to continuously improve recognition accuracy; model optimization uses transfer learning techniques to fine-tune a pre-trained large model to adapt to the specific language characteristics of official documents; entity disambiguation disambiguates potentially ambiguous entities using contextual information and entity relationship networks; and multi-model fusion combines multiple NER models with different structures and improves entity recognition accuracy through weighted averaging.

[0029] In this embodiment, the text element extraction module parses the main body of the official document to extract ordered and structured information, including main body subheadings, content corresponding to the main body subheadings, attachment titles and attachment content, including text segmentation and identification, extraction of content corresponding to main body subheadings, extraction of attachment titles and content, and special format processing steps.

[0030] In this embodiment, text segmentation and identification first uses regular expressions to divide the main body of the document into independent paragraphs. The system uses contextual information to understand the content under each subheading, and then uses a large model to identify the subheading of each paragraph. When extracting the content corresponding to the main body subheading, the system locates the content under each subheading, extracts the main body content corresponding to the subheading, and associates it with the subheading to form structured data. When extracting the title and content of attachments, the system uses keyword search to identify attachments or match the content adjacent to the attachment, locates the starting position of the attachment, and then extracts the attachment title. When extracting the attachment content, for text attachments, the system will extract all text content after the attachment title; for non-text attachments, the system will extract the attachment file name information. Finally, special formats are processed. For tables in the main body or attachments, they are first processed into Markdown format, and then the large model is used to identify the table structure and extract the information in the table. For attachments containing images, the system will extract the descriptive text of the image or the image file name.

[0031] In this embodiment, the element review module uses a large model combined with a rule engine to review each element of the document to ensure the accuracy and completeness of each element, including steps such as title review, content review, and other element review.

[0032] In this embodiment, the title review first establishes a standard for judging correct titles based on common errors in official document titles. Then, according to the standard, an online model is used to generate negative data for official document titles to ensure coverage of all possible errors. The data is then checked and fine-tuned using LoRA. After fine-tuning, the LM-Cocktail tool is used to merge the fine-tuned model and the base model. The merged model is then used to review common errors in official document titles. Finally, based on the semantic capabilities of the large model, the consistency between the theme of each title and its corresponding content is reviewed. For subheadings in the main text, the semantic relationships between subheadings are also reviewed. The content review first examines the grammar, semantics, logic, and standard usage of the main text and attachments of the official document to establish a correct title. First, establish clear standards. Then, select representative positive and negative examples from the historical document database or call online models. The online model labels errors in the dataset and verifies, corrects, and scores the data. The model is then fine-tuned and merged. A trained syntax review model is used to review the main text and attachments of the official documents. For other elements, regular expression matching is used, combined with the issuing authority information, to verify the correctness of the document number. The accuracy of the addressee's name is verified by matching the organization name, responsibilities, and level information in the knowledge base. The accuracy of the issuing authority's name is also verified by matching the organization name in the knowledge base. Finally, a date parsing library is used to check whether the document issuance time conforms to logical order and whether the time format is correct.

[0033] This patent aims to deeply understand and accurately extract key information from official documents, including titles, issuing units, and subheadings and content within the main text, thereby reducing manual operations and lowering error rates. Furthermore, this patent aims to ensure the completeness and accuracy of extracted elements through the intelligent review function of a large-scale model, reducing the workload of manual review and improving the automation level of document processing. In summary, this patent aims to provide an efficient and reliable document element extraction and review system to improve the office efficiency of government and enterprises, reduce human resource costs, enhance the quality of document processing and user satisfaction, and promote the modernization and intelligentization of document processing workflows.

[0034] Finally, it should be noted that those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A document element extraction and review system based on a large model, characterized in that, It includes a document element extraction module, a main text element extraction module, and an element review module; The document element extraction module is used to automatically identify and extract key elements of documents. The text element extraction module is used to parse the text of the official document in order to extract ordered and structured information. The element review module is used to review each element of the document to ensure the accuracy and completeness of each element. The document element extraction module is the part of this system used to automatically identify and extract key elements of documents. The document element extraction module includes the steps of data preprocessing, named entity recognition, element extraction and verification, output and optimization. Data preprocessing first collects various styles of official document texts, then standardizes the format and cleans the data, and divides the entire official document text into independent sentences or paragraphs; Named entity recognition first defines entity categories, then extracts text features such as word features, character features, position features, and part-of-speech features, trains a NER model, and then uses the NER model to identify each entity and its category in the official document text. Element extraction and verification: Based on the output of the NER model, the various elements in the document are extracted, rule verification is performed, and the reasonableness of the extracted entities is verified by using context information. The output and optimization steps first output the extracted elements and their categories in the form of structured data, then provide a user interface for users to provide feedback on the extraction results. The system adjusts the NER model based on the feedback, then fine-tunes it on the basis of the pre-trained large model through transfer learning, performs entity disambiguation through contextual information and entity relationship network, and finally combines multiple different NER models by weighted averaging. The main text element extraction module parses the main text of the official document to extract ordered and structured information, including main text subheadings, the content corresponding to the main text subheadings, attachment titles and attachment content, including text segmentation and identification, extraction of content corresponding to main text subheadings, extraction of attachment titles and content, and special format processing steps; Text segmentation and identification: First, the main body of the document is divided into independent paragraphs using regular expressions. The system uses contextual information to understand the content under each subheading, and then uses a large model to identify the subheading of each paragraph. When extracting content corresponding to subheadings in the main text, the system locates the content under each subheading, extracts the main text content corresponding to the subheading, and associates it with the subheading to form structured data; When extracting attachment titles and content, the system identifies attachments or matches content adjacent to attachments using keyword search, locates the beginning of the attachment, and then extracts the attachment title. When extracting attachment content, for text attachments, the system extracts all text content after the attachment title; for non-text attachments, the system extracts the attachment file name information. Finally, special formatting is handled. For tables in the main text or attachments, they are first processed into Markdown format, and then the table structure is identified through a large model to extract the information from the tables. For attachments containing images, the system will extract the descriptive text of the images or the image file names.

2. The document element extraction and review system based on a large model according to claim 1, characterized in that, The element review module uses a large model combined with a rule engine to review each element of the document, ensuring the accuracy and completeness of each element. This includes steps such as title review, content review, and review of other elements.

3. The document element extraction and review system based on a large model according to claim 2, characterized in that, The title review process first establishes criteria for judging correct titles based on common errors in official document titles. Then, according to the criteria, an online model is used to generate negative data for official document titles to ensure coverage of all possible errors. The data is then checked and fine-tuned using LoRA. After fine-tuning, the LM-Cocktail tool is used to merge the fine-tuned model and the base model. The merged model is then used to review common errors in official document titles. Finally, based on the semantic capabilities of the large model, the consistency between each title and the corresponding content is reviewed. For subheadings in the main text, the semantic relationships between subheadings are also reviewed. The content review first examines the grammar, semantics, logic, and standard usage of the main body and attachments of the official documents to establish correct standards. Then, representative positive and negative examples are generated by selecting from the historical document database or calling online models. The online model marks the errors in the dataset and verifies, corrects, and scores the data. Then, the model is fine-tuned and merged. The trained grammar review model is used to review the content of the main body and attachments of the official documents. The verification of the other elements involves using regular expression matching, combined with the issuing authority information, to verify the correctness of the document number; verifying the accuracy of the addressee name by matching the organization name, organization responsibilities, and organization level information in the knowledge base; verifying the accuracy of the issuing authority name by matching the organization name in the knowledge base; and using a date parsing library to check whether the document issuance time conforms to the logical order and whether the time format is correct.

Images