Intelligent document checking method and system of visual workflow

Abstract

The invention belongs to the technical field of natural language processing and intelligent document processing, and relates to an intelligent document checking method and system of a visual workflow, and the method comprises the steps: generating the visual workflow according to an obtained verification demand; executing the workflow, and based on the extracted document information of the to-be-reviewed document, in combination with the review rule and the domain knowledge base, reviewing the to-be-reviewed document through a preset large language model to obtain a review result; and generating a verification report according to the verification result. The problems that the verification accuracy is poor, the verification rule configuration is not flexible, and the verification process cannot be visualized are solved.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing and intelligent document processing, and specifically discloses an intelligent document review method and system for a visualized workflow. Background Art

[0002] Document review is a core part of the information processing process. It aims to ensure the accuracy, professionalism and compliance of documents through content accuracy verification, format standardization correction, logical consistency inspection and language quality optimization. With the rapid development of informatization and digitalization, document types are becoming more and more complex, such as technical reports, legal contracts, academic papers, etc., and their review needs are multi-dimensional, high-precision and compliant. Traditional text checking and grammar checking tools are largely limited by rules and known vocabulary, and it is difficult to cope with the deep semantic understanding, complex knowledge, and professional text content review needs. It is urgent to achieve automation and intelligent upgrades through technological innovation.

[0003] Existing document review technologies have the following disadvantages: 1. Inflexible configuration of review rules. Traditional review tools rely on code to write rule templates, making it difficult for ordinary users to directly participate in rule definition, resulting in high collaboration costs between business experts and technical personnel.

[0004] 2. Insufficient intelligence: Traditional proofreading solutions based on regular expressions or keyword matching cannot understand the contextual semantics of documents and have difficulty identifying deep-seated problems such as logical contradictions and terminology ambiguity, resulting in low proofreading accuracy.

[0005] 3. Poor adaptability to the field. Documents in professional fields need to be verified in conjunction with the industry knowledge base. For example, petroleum engineering documents have high professional requirements. If they are not verified in conjunction with the industry knowledge base, large verification errors are likely to occur, resulting in low verification accuracy. The existing verification scheme is not suitable for highly professional fields, such as drilling design and logging reports in the petroleum industry.

[0006] 4. The review process cannot be visualized. The existing review rules lack the ability to visualize the execution order, conditional branches and other logical relationships, making it difficult to build and maintain complex review processes. Summary of the invention

[0007] The purpose of the present invention is to provide an intelligent document review method and system with a visualized workflow to solve the problems of poor review accuracy, inflexible review rule configuration, and the inability to visualize the review process.

[0008] The specific scheme of the present invention is as follows: An intelligent document review method with a visual workflow, comprising: Generate visual workflow based on the obtained verification requirements; Execute the workflow, extract the document information of the document to be reviewed, combine the review rules and domain knowledge base, and review the document to be reviewed through the preset large language model to obtain the review result; Generate a review report based on the review results.

[0009] In some embodiments, the workflow for generating a visualization includes: According to verification requirements, users can select or drag nodes and connecting lines in the node library through the visual editor, define the order and conditions of the review steps, build a visual review process and review rules, and generate a visual workflow.

[0010] In some embodiments, the workflow for generating a visualization includes: According to the verification requirements, the corresponding workflow template in the workflow template library is automatically matched to generate a visual workflow.

[0011] In some embodiments, based on extracting document information of the document to be reviewed, combined with review rules and domain knowledge base, the document to be reviewed is reviewed by a preset large language model to obtain a review result, including: The review rules include semantic-level rules; Based on the document information, semantic-level rules and LLM prompt words generated by the domain knowledge base, the document to be reviewed is reviewed by a preset large language model to obtain the review result.

[0012] In some embodiments, the LLM prompt words generated based on document information, semantic level rules and domain knowledge base include: Dynamically convert semantic-level rules in workflow into LLM prompt words; Gain expertise based on document information and domain knowledge base; Combine professional knowledge with relevant text segments of the document to be reviewed into LLM prompt words.

[0013] In some embodiments, obtaining expertise based on document information and a domain knowledge base includes: A retrieval-enhanced generation algorithm is used to obtain professional knowledge that is similar in vector based on document information through context matching in the domain knowledge base of the document to be reviewed through vector retrieval.

[0014] In some embodiments, based on extracting document information of the document to be reviewed, combining review rules and domain knowledge base, reviewing the document to be reviewed by using a preset large language model to obtain a review result, further comprising: The review rules also include structural rules; According to the document information and structural rules, the document to be reviewed is reviewed by using a preset large language model to obtain the review result.

[0015] In some embodiments, extracting document information of the document to be reviewed includes: After the document to be reviewed is uploaded, it is parsed into a text document through an optical character recognition algorithm; The text document is parsed through a large language model to extract the document information of the document to be reviewed.

[0016] In some embodiments, the verification requirements obtained include: Enter natural language instructions in the language user interface (LUI) and obtain verification requirements through the large language model.

[0017] The present invention also relates to an intelligent document review system for a visual workflow, which is used for the above-mentioned intelligent document review method for a visual workflow, comprising: The document parsing module is used to extract document information of the document to be reviewed through the optical character recognition algorithm and the large language model; A verification requirement acquisition module is used to obtain verification requirements through a language user interface; Visual workflow arrangement module, which is used to select or drag nodes and connection lines in the node library through the visual editor to build a visual review process and review rules to generate workflows according to verification requirements, or automatically match the corresponding workflow template in the workflow template library to generate workflows; The intelligent review module is used to execute the workflow. Based on the extracted document information, combined with the review rules and domain knowledge base, the review results are obtained by reviewing the documents to be reviewed through the preset large language model. The report generation module is used to generate a review report based on the review results of the document to be reviewed.

[0018] Compared with the prior art, the present invention has the following advantages and beneficial effects: Through the user's self-construction of visual workflows, the visual configuration of user-defined review rules, review priority settings and exception handling mechanisms is realized, which reduces the difficulty of building and maintaining complex review processes and improves the flexibility of review rule configuration; based on document information, semantic-level rules and LLM prompt words generated by the domain knowledge base, the review results are obtained through the preset large language model, and the semantic-level rules are integrated with the domain knowledge base to achieve semantic-level error detection, solve the problem of deep semantics such as difficulty in identifying logical contradictions and terminology ambiguity, and improve the accuracy and professionalism of review; at the same time, the dynamically expanded domain knowledge base enables the domain knowledge base to better meet verification needs and improve the accuracy of review. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 This is a flow chart of an intelligent document review method with a visualized workflow in Example 1 of the present invention.

[0020] Figure 2 This is a schematic diagram of an intelligent document review system with a visualized workflow in Example 1 of the present invention. DETAILED DESCRIPTION

[0021] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0022] Example 1 An intelligent document review method with visual workflow, such as Figure 1 As shown, including: S1. Extract document information of the document to be reviewed; After the document to be reviewed is uploaded, it is parsed into a text document through the optical character recognition algorithm, namely the OCR algorithm; the text document is parsed through the large language model to extract the document information of the document to be reviewed.

[0023] The types of documents to be reviewed include: PDF documents, word documents, etc. Document information includes text content, paragraph levels, table data, and metadata; for example, text content includes the contract signatories, and table data includes drilling parameters.

[0024] S2. Obtain verification requirements through the language user interface; Enter natural language commands or voice commands in the language user interface (LUI) and obtain verification requirements through the large language model.

[0025] After the natural language instructions input by the user in the language user interface enter the large language model, the large language model first understands the natural language instructions according to the intent categories set in the intent classification list of the prompt word to classify the problem, then outputs the classification category, and finally outputs the verification requirements in the language user interface, and the user confirms the output verification requirements. The prompt words in the large language model include: role, responsibility, intention classification list, requirements, and thinking steps.

[0026] For example, in the language user interface, enter the natural language command "check whether the well depth data in the drilling plan matches the formation pressure"; after "check whether the well depth data in the drilling plan matches the formation pressure" enters the large language model, the role of the prompt word in the large language model is an expert in well engineering field intention recognition; The responsibility is to understand the natural language instructions currently input by the user according to the context, identify the user's specific intentions, and classify them; The intent classification list includes: query drilling time efficiency and index statistical analysis, and its typical problem is to query the drilling time efficiency of a certain well; query detailed time efficiency statistical analysis of multiple wells, and its typical problem is to query detailed time efficiency statistical analysis of multiple wells; query block drilling index statistics, and its typical problem is to query drilling index statistics of a certain well; query annual average index comparison, and its typical problem is to query annual average index comparison of a certain well; query annual average index statistical analysis, and its typical problem is to query average index statistical analysis of a certain well; query annual average index and template comparison, and its typical problem is to query average index and template comparison of a certain block; query block drilling index difference statistics, and its typical problem is to query drilling index difference statistics of a certain well; query drilling time efficiency index detailed statistics, and its typical problem is to query drilling time efficiency index detailed statistics of a certain well every day; It is required to list the text and intent in the natural language instructions input by the user separately, and to perform rigorous classification based on the understanding of the natural language instructions input by the user; The thinking steps are to obtain and understand the natural language instructions input by the user, extract the keywords in the natural language instructions input by the user; match the intent according to the keywords and the intent classification list; and combine the natural language instructions and keywords input by the user to perform the final intent classification.

[0027] S3, generate a visual workflow based on verification requirements; Visual workflow refers to the ability to display the steps and content of the entire workflow through a visual interface. Users can use the graphical interface of the visual editor to independently build workflows based on verification requirements, and display the steps and content of the entire workflow in the graphical interface. The graphical interface provides selectable node libraries and connecting lines; or automatically match the corresponding workflow in the workflow template library based on verification requirements, and display the steps and content of the entire matching workflow through the graphical interface.

[0028] According to the verification requirements, users select or drag the pre-stored nodes in the node library to the canvas through the graphical interface of the visual editor, and use the pre-stored connection lines to connect the nodes, define the order and conditions of the review steps, build a visual review process and review rules, and generate a visual workflow; the workflow constructed by the user is stored as a workflow template in the workflow template library.

[0029] The nodes in the node library include input nodes, output nodes, large model nodes, assistant nodes, QA knowledge base retrieval nodes, document knowledge base question and answer nodes, report nodes, code nodes, conditional branch nodes and end nodes.

[0030] Among them, the conditional branch node includes a conditional expression and at least one branch, and different branches are executed according to different conditions such as selection variables, selection conditions, input values, etc. in the conditional expression.

[0031] A connection line is a line that connects an upstream node with a downstream node. It is used to represent the operation logic of a process and to indicate that the output of an upstream node is transmitted to a downstream node so that the downstream node can use the output of the upstream node.

[0032] For example, a workflow that users can independently build includes four review processes: format verification, term matching, logic verification, and compliance review: Select a code node as the format check, and check the title and paragraph format by writing the format check code and binding the format check rules. Format check is used to ensure that the structure of the document to be reviewed completely matches the template, such as title matching and paragraph format matching.

[0033] Select the large model node as the term matcher, and write prompt words to bind the term match rules to detect whether the document to be reviewed contains the required terms. Term matching is used to ensure the professionalism and compliance of the content of the document to be reviewed. For example, in contract review, check whether the "force majeure" in the legal mandatory clause is omitted.

[0034] The code node is connected to the large model node through a connecting line, and the output of the code node can be used as the input of the large model node.

[0035] Select the assistant node as the logic verification, bind the logic verification rules by writing the corresponding prompt words and adding the "calculator" tool to check whether the argument, reasoning or conclusion conforms to the logical rules. For example, check whether the numerical calculation in the reasoning is correct, and determine whether the logical relationship between propositions such as cause and effect, parallelism, and conditions is correct.

[0036] The large model node is connected to the assistant node through a connecting line, and the output of the large model node can be used as the input of the assistant node.

[0037] Select the document knowledge base question and answer node as the compliance review, upload original files such as "Compliance Manual.pdf" and "Latest Laws and Regulations.docx" to the domain knowledge base of the document to be reviewed, select the domain knowledge base in the search scope in the document knowledge base question and answer node, and write prompt words to bind the compliance review rules to check whether the documents, behaviors or processes to be reviewed comply with mandatory or recommended rules such as laws and regulations, industry standards, and internal policies.

[0038] The assistant node is connected to the document knowledge base question and answer node through a connecting line, and the output of the assistant node can be used as the input of the document knowledge base question and answer node.

[0039] Or according to the verification requirements, the corresponding workflow template in the workflow template library is automatically matched to generate a visual workflow. Users can adjust the parameters of each node in the workflow through the graphical interface of the visual editor.

[0040] For example, according to the verification requirement of "checking whether the well depth data in the drilling plan matches the formation pressure", it is identified that the review process that meets the verification requirement includes numerical comparison, engineering specification retrieval, and logical consistency verification in sequence. The numerical comparison is performed through the large model node and the code node, the engineering specification retrieval is performed through the document knowledge base question and answer node, and the logical consistency is performed through the assistant node; based on the large model node, code node, document knowledge base question and answer node, and assistant node connected in sequence, the corresponding workflow template is automatically matched in the workflow template library to generate a visual workflow; users can adjust the parameters of each node in the workflow according to the verification requirements to make the workflow better match the verification requirements.

[0041] S4, execute the workflow, based on the extracted document information, combined with the review rules and domain knowledge base, review the document to be reviewed through the preset large language model to obtain the review result; Review rules include structured rules and semantic-level rules; structured rules include format rules, numerical rules, etc., which are used to solve problems such as format errors and numerical out-of-bounds; semantic-level rules include quality assurance QA rules, logic rules, terminology rules, etc., which are used to solve problems such as semantic conflicts of clauses, improper use of terminology, logical contradictions, and terminology compliance.

[0042] The review includes structured rule review and semantic rule review.

[0043] Structured rule review includes: based on the document information and structured rules, the pre-set large language model is used to directly verify the format errors, value out-of-bounds and other issues of the document to be reviewed to obtain the review results. For example, the format rules and value rules are used to directly verify the missing contract number in the format errors of the document to be reviewed and the pressure value exceeding the safety threshold in the value out-of-bounds.

[0044] Semantic-level rule review includes: LLM prompt words generated based on document information, semantic-level rules and domain knowledge base, and reviewing the document to be reviewed through a preset large language model to obtain the review result.

[0045] Among them, the preset large language model is a large language model that has been trained and adjusted through a large number of samples of documents to be reviewed. The preset large language model has better performance in the review task.

[0046] The domain knowledge base can upload the required files according to the verification requirements, and dynamically expand the content of the domain knowledge base, so that the domain knowledge base can better meet the verification requirements and adapt to the verification requirements of different industries; for example, for petroleum documents awaiting review, it is necessary to upload files such as equipment parameter standards, safety specification items, engineering calculation formulas related to the petroleum industry to the domain knowledge base to expand the professional knowledge of the petroleum industry so that the domain knowledge base can better meet the verification requirements.

[0047] The generation of LLM prompt words includes: Dynamically convert the semantic-level rules of the review rules in the workflow into prompts that can be understood by the large language model LLM, that is, dynamically convert the semantic-level rules into LLM prompts; The retrieval-enhanced generation algorithm (RAG algorithm) is used to obtain professional knowledge similar in vectors through context matching in the domain knowledge base of the document to be reviewed based on the document information extracted from the document to be reviewed. Combine the retrieved expertise with relevant text segments of the document to be reviewed into LLM prompt words.

[0048] For example, in the "User Prompt Words" of the "Document Knowledge Base Question and Answer Node", the prompt words include: “Given the [text] of the document to be reviewed, combined with the [expertise] of the domain knowledge base, determine whether the [text] meets the [condition].” Here, [text] and [condition] are known variables input into this node. [Expertise] is the relevant content that is closest to the [text] of the document to be reviewed in terms of vector, retrieved from the domain knowledge base by running the “Document Knowledge Base Question and Answer Node”.

[0049] Fill in the actual values ​​of [text], [condition] and [expertise] in the prompt word. For example, "Given the [text] of the document to be reviewed contains: Both parties should keep the cooperation information confidential, and the breaching party should compensate for the loss [text paragraph]; combined with the [expertise] of the domain knowledge base, it contains: "Standard confidentiality obligation clauses should include: definition of confidentiality scope, confidentiality period, liability for breach of contract, etc.", determine whether the "confidential information" of the [text paragraph] meets the "confidentiality obligation clause" of the [condition].

[0050] The preset large language model gives the proofreading results based on the above prompt words with its own reasoning and generation capabilities; if the proofreader finds logical contradictions or non-compliant content, it will be marked as an error and the original text and correction suggestions will be output.

[0051] The review results support click-through tracing, where users can view error triggering rules, modification correction suggestions, and reference basis, allowing users to quickly correct documents to be reviewed.

[0052] S5. Generate a review report based on the review results of the document to be reviewed.

[0053] The review report includes highlighted error locations, correction suggestions, and classification of error types.

[0054] Through the user's self-construction of visual workflows, the visual configuration of user-defined review rules, review priority settings and exception handling mechanisms is realized, which reduces the difficulty of building and maintaining complex review processes, improves the flexibility of review rule configuration, and can improve the efficiency of review rule maintenance by more than 60%; based on document information, semantic-level rules and LLM prompt words generated by the domain knowledge base, the review results are obtained through the preset large language model, and the semantic-level rules are integrated with the domain knowledge base to realize semantic-level error detection, solve deep-seated problems such as difficult-to-identify logical contradictions and terminology ambiguity, and improve review accuracy and review professionalism; at the same time, the dynamically expanded domain knowledge base enables the domain knowledge base to better meet verification needs and improve review accuracy.

[0055] The present invention also relates to an intelligent document review system with a visual workflow, such as Figure 2 As shown, including: The document parsing module is used to extract document information of the document to be reviewed through the optical character recognition algorithm and the large language model; A verification requirement acquisition module is used to obtain verification requirements through a language user interface; Visual workflow arrangement module, which is used to select or drag nodes and connection lines in the node library through the visual editor to build a visual review process and review rules to generate workflows according to verification requirements, or automatically match the corresponding workflow template in the workflow template library to generate workflows; The intelligent review module is used to execute the workflow. Based on the extracted document information, combined with the review rules and domain knowledge base, the review results are obtained by reviewing the documents to be reviewed through the preset large language model. The report generation module is used to generate a review report based on the review results of the document to be reviewed.

[0056] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. An intelligent document review method for a visual workflow, characterized in that: include: Generate visual workflow based on the obtained verification requirements; Execute the workflow, extract the document information of the document to be reviewed, combine the review rules and domain knowledge base, and review the document to be reviewed through the preset large language model to obtain the review result; Generate a review report based on the review results.

2. The method for intelligent document review based on a visual workflow according to claim 1, characterized in that: The workflow for generating visualization includes: According to verification requirements, users can select or drag nodes and connecting lines in the node library through the visual editor, define the order and conditions of the review steps, build a visual review process and review rules, and generate a visual workflow.

3. The method for intelligent document review based on a visual workflow according to claim 1, characterized in that: The workflow for generating visualization includes: According to the verification requirements, the corresponding workflow template in the workflow template library is automatically matched to generate a visual workflow.

4. The method for intelligent document review of a visual workflow according to claim 1, characterized in that: The method extracts document information of the document to be reviewed, combines review rules and domain knowledge base, and reviews the document to be reviewed by a preset large language model to obtain a review result, including: The review rules include semantic level rules; Based on the document information, semantic-level rules and LLM prompt words generated by the domain knowledge base, the document to be reviewed is reviewed by a preset large language model to obtain the review result.

5. The method for intelligent document review based on a visual workflow according to claim 4, characterized in that: The LLM prompt words generated based on document information, semantic level rules and domain knowledge base include: Dynamically convert semantic-level rules in workflow into LLM prompt words; Gain expertise based on document information and domain knowledge base; Combine professional knowledge with relevant text segments of the document to be reviewed into LLM prompt words.

6. The method for intelligent document review based on a visual workflow according to claim 5, characterized in that: The obtaining of professional knowledge based on document information and domain knowledge base includes: A retrieval-enhanced generation algorithm is used to obtain professional knowledge that is similar in vector based on document information through context matching in the domain knowledge base of the document to be reviewed through vector retrieval.

7. The method for intelligent document review based on a visual workflow according to claim 1, characterized in that: The method extracts document information of the document to be reviewed, combines review rules and a domain knowledge base, and reviews the document to be reviewed by a preset large language model to obtain a review result, and further includes: The review rules also include structured rules; According to the document information and structural rules, the document to be reviewed is reviewed by using a preset large language model to obtain the review result.

8. The method for intelligent document review based on a visual workflow according to claim 1, characterized in that: The document information of the document to be reviewed includes: After the document to be reviewed is uploaded, it is parsed into a text document through an optical character recognition algorithm; The text document is parsed through a large language model to extract the document information of the document to be reviewed.

9. The method for intelligent document review based on a visual workflow according to claim 1, characterized in that: The obtained verification requirements include: Enter natural language instructions in the language user interface (LUI) and obtain verification requirements through the large language model.

10. An intelligent document review system with visual workflow, characterized in that: An intelligent document review method for a visual workflow according to any one of claims 1 to 9, comprising: The document parsing module is used to extract document information of the document to be reviewed through the optical character recognition algorithm and the large language model; A verification requirement acquisition module is used to obtain verification requirements through a language user interface; Visual workflow arrangement module, which is used to select or drag nodes and connection lines in the node library through the visual editor to build a visual review process and review rules to generate workflows according to verification requirements, or automatically match the corresponding workflow template in the workflow template library to generate workflows; The intelligent review module is used to execute the workflow. Based on the extracted document information, combined with the review rules and domain knowledge base, the review results are obtained by reviewing the documents to be reviewed through the preset large language model. The report generation module is used to generate a review report based on the review results of the document to be reviewed.

Images