Document auditing method and device based on cooperation of large model and rule engine

Abstract

The application discloses a document auditing method and device based on large model and rule engine cooperation, the method comprises the following steps: analyzing an original document to form an intermediate representation file; extracting entities from the intermediate representation file to form an entity candidate set; inputting a thinking chain prompt word into a large model to obtain a first triple, a first confidence and a first risk level, and calculating a first traceable score of the large model according to a reasoning path node; using a rule engine to make a deterministic judgment to obtain a second triple, a second confidence, a risk level and a second traceable score; calculating two weighted voting values and obtaining a conflict difference value by taking the difference; when the conflict difference value is less than a preset judgment threshold, taking the conclusion corresponding to the high weighted voting value as the final conclusion; otherwise, starting an artificial review process; and generating an auditing report in multiple formats. The method takes advantage of the strong language understanding ability of the large model and the strong certainty of the rule engine, and avoids the problems of missed review or excessive marking.

Description

Technical Field

[0001] This application relates to the field of intelligent review technology for highly compliant documents, and in particular to a document review method and apparatus based on the collaboration of a large model and a rule engine. Background Technology

[0002] With the accelerated digitalization of highly compliant industries such as finance, law, government affairs, and healthcare, the number of mixed documents such as contracts, approvals, reports, and invoices is exploding. Traditional manual review can no longer meet the regulatory requirements of "zero delay, zero omissions, and traceability." Although single-point technologies such as optical character recognition, natural language processing, and rule engines are gradually maturing, pure rule engines struggle to handle semantic ambiguity when conducting document compliance reviews, while large-scale models lack determinism. When the conclusions of the two approaches conflict, there is no reliable arbitration mechanism, leading to omissions or over-marking. Summary of the Invention

[0003] This application provides a document review method and apparatus based on the collaboration of a large model and a rule engine. It enables the large model and the rule engine to review the original document simultaneously and has a reliable arbitration mechanism, thereby improving the accuracy of the review.

[0004] In a first aspect, embodiments of this application provide a document review method based on the collaboration of a large model and a rule engine. The method includes: parsing the original document to form an intermediate representation file; extracting entities from the intermediate representation file using a large model based on entity definitions in a preset review rule base to form an entity candidate set; constructing a thought chain prompt using the entity candidate set and the preset review rule base; inputting the thought chain prompt into the large model to obtain a first triplet, a first confidence level, and a first risk level; and calculating a first traceability score for the large model based on inference path nodes; wherein the first triplet includes a first conclusion, a first basis, and a first reference coordinate; and using a rule engine to perform deterministic analysis on the entity candidate set according to the preset review rule base. The process involves determining and obtaining a second triplet, a second confidence level, a second risk level, and a second traceability score. The second triplet includes a second conclusion, a second basis, and a second reference coordinate. Two weighted voting values ​​are obtained by weighting the first confidence level and the first traceability score, as well as the second confidence level and the second traceability score. The difference between these two weighted voting values ​​is then calculated to obtain a conflict difference. This conflict difference is compared to a preset judgment threshold. If the conflict difference is less than the preset judgment threshold, the conclusion corresponding to the higher weighted voting value is taken as the final conclusion. Otherwise, a manual review process is initiated, and the manual review conclusion is taken as the final conclusion. The final conclusion, reference coordinate, and revision history are written into a structured template file to generate audit reports in various formats.

[0005] In conjunction with the first aspect, in one possible implementation, parsing the original document to form an intermediate representation file includes: performing geometric correction on each page of the original document and dividing it according to content type, outputting the bounding rectangle coordinates and a third confidence score for each type of content area; performing font recognition on text-type areas and generating a reading order list with hierarchical labels to obtain a first parsing result; performing structural recognition on table-type areas to obtain a second parsing result; detecting the authenticity of seals and signatures and calculating a fourth confidence score to obtain a third parsing result; and serializing the first parsing result, the second parsing result, and the third parsing result to form the intermediate representation file.

[0006] In conjunction with the first aspect, in one possible implementation, the step of extracting entities from the intermediate representation file using a large model based on the entity definitions in a preset review rule base to form an entity candidate set includes: pre-annotating the intermediate representation file based on the preset review rule base; inputting the intermediate representation file into the large model to obtain the output result of the large model; and performing deduplication, uniform formatting of amounts and dates, and merging of similar entities on the output result of the large model to generate the entity candidate set including entity address, standard name, type, value, coordinate list, and original part.

[0007] In conjunction with the first aspect, in one possible implementation, the step of constructing the entity candidate set and the preset review rule base into a thought chain prompt, and inputting the thought chain prompt into the large model to obtain the first triplet, the first confidence level, and the first risk level includes: constructing the entity candidate set and the preset review rule base into the thought chain prompt of a preset structure according to a preset template; inputting the thought chain prompt of the preset structure into the large model, and restricting the large model to output the first triplet, the first confidence level, and the first risk level.

[0008] In conjunction with the first aspect, in one possible implementation, calculating the first traceability score of the large model based on the inference path nodes includes: splitting the thought chain output by the large model into inference nodes at the sentence level; the fifth confidence level of each inference node is obtained by multiplying the average attention weight of the sentence in the original context by the logical coherence score; mapping the first risk level to a quantized weight; if the inference node involves cross-paragraph, cross-table, or numerical calculations, determining the node complexity weight to be 2.0; otherwise, determining the node complexity weight to be 1.0; multiplying the fifth confidence level, the quantized weight, and the node complexity weight of each inference node sequentially to obtain a single-node traceability score; the first traceability score is the 95th percentile of all single-node traceability scores.

[0009] In conjunction with the first aspect, in one possible implementation, the step of using a rule engine to perform deterministic judgment on the entity candidate set according to the preset audit rule base to obtain a second triplet, a second risk level, and a second traceability score includes: using the rule engine to execute rigid prohibition type rules in the preset audit rule base; when the rigid prohibition type rule is met, rejection is taken as the second conclusion, and the second triplet, the second confidence level, and the second risk level are output, terminating subsequent judgments; using the rule engine to execute risk warning type rules in the preset audit rule base; when the risk warning type rule is met, a warning is taken as the second conclusion, and the second triplet, the second confidence level, and the second risk level are output; if the risk warning type rule is triggered by the rigid prohibition type rule, conflict arbitration is initiated: the conclusion of the rigid prohibition type rule is taken as the standard, and the address of the risk warning type rule is written into an additional prompt field for front-end highlighting; for each triggered rule, the rule complexity weight is multiplied by the second confidence level as the second traceability score.

[0010] In conjunction with the first aspect, in one possible implementation, the method further includes: parsing the reasoning steps of the large model and constructing a reasoning path graph, and generating an interactive visual interface that integrates logical relationships and risk distribution by marking the nodes of the reasoning path graph with risk levels and mapping the node positions back to the highlighted original document.

[0011] In conjunction with the first aspect, in one possible implementation, the method further includes: writing entries with the second traceability score above a preset score into a high traceability whitelist and issuing an audit report; marking entries with the second traceability score below the preset score as requiring manual review and linking them to the inference path graph for highlighting.

[0012] In conjunction with the first aspect, in one possible implementation, the method further includes: obtaining at least one audit constraint description and updating the preset audit rule base.

[0013] Secondly, embodiments of this application provide a document review device based on the collaboration of a large model and a rule engine. The document review device includes: a parsing module for parsing the original document to form an intermediate representation file; an extraction module for extracting entities from the intermediate representation file using a large model based on entity definitions in a preset review rule base, forming an entity candidate set; a first evaluation module for constructing a thought chain prompt using the entity candidate set and the preset review rule base, inputting the thought chain prompt into the large model to obtain a first triplet, a first confidence level, and a first risk level, and calculating a first traceability score for the large model based on inference path nodes; wherein the first triplet includes a first conclusion, a first basis, and a first reference coordinate; and a second evaluation module for evaluating the document according to the preset review rules. The system uses a rule engine to perform deterministic judgments on the candidate entity set, obtaining a second triplet, a second confidence level, a second risk level, and a second traceability score. The second triplet includes a second conclusion, a second basis, and a second reference coordinate. A voting module is used to perform weighted calculations on the first confidence level and the first traceability score, as well as the second confidence level and the second traceability score, to obtain two weighted voting values, and to calculate the difference between the two weighted voting values ​​to obtain a conflict difference. A conclusion module is used to compare the conflict difference with a preset judgment threshold, and if the conflict difference is less than the preset judgment threshold, the conclusion corresponding to the higher weighted voting value is taken as the final conclusion; otherwise, a manual review process is initiated. A reporting module is used to write the final conclusion, reference coordinates, and revision traces into a structured template file to generate review reports in various formats.

[0014] Thirdly, embodiments of this application provide an electronic device, the electronic device comprising: one or more processors; and a memory storing computer-executable instructions, which, when executed by the one or more processors, cause the one or more processors to perform the document review method based on the collaboration of a large model and a rule engine as described in the first aspect or any possible manner of the first aspect.

[0015] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer-readable instructions, which, when executed by a computer, implement the document review method based on the collaboration of a large model and a rule engine as described in the first aspect or any possible implementation of the first aspect.

[0016] This application provides a document review method based on the collaboration of a large model and a rule engine. When reviewing original documents using a preset review rule base, this method employs both a large model and a rule engine for joint review. The final conclusion is determined by the discrepancy between the conclusions of the large model and the rule engine, forming a reliable arbitration mechanism. This method utilizes both a large model and a rule engine to review the original document simultaneously, leveraging the advantages of the large model's strong language understanding capabilities and the rule engine's high determinism. Furthermore, manual review is performed when the discrepancy between the large model and the rule engine reaches a preset judgment threshold, avoiding issues of missed reviews or over-marking. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A flowchart illustrating the document review method based on the collaboration of a large model and a rule engine provided in this application embodiment;

[0019] Figure 2 This is a schematic diagram of a document review device based on the collaboration of a large model and a rule engine, provided in an embodiment of this application. Detailed Implementation

[0020] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0021] This application provides a document review method based on the collaboration of a large model and a rule engine, which includes steps S101 to S107.

[0022] S101. Parse the original document to form an intermediate representation file.

[0023] The original document can be a scanned copy, a PDF file, an OFD file, an image, or a handwritten document.

[0024] The intermediate representation file generated after parsing the original document in step S101 serves as the data basis for judgment by the large model and rule engine. For example, the intermediate representation file can be a Markdown format file.

[0025] S102. Based on the definition of entities in the preset audit rule base, use the large model to extract entities from the intermediate representation file to form an entity candidate set.

[0026] For example, the pre-defined audit rule base uses domain-specific language to describe audit rules for high-compliance scenarios such as finance, law, government affairs, and engineering. The audit rules include business scenarios, priorities, entity definitions, verification types, triggering actions, and prompts.

[0027] S103. Construct the entity candidate set and the preset review rule base into a thinking chain prompt word, input the thinking chain prompt word into the large model, obtain the first triplet, the first confidence level and the first risk level, and calculate the first traceability score of the large model based on the reasoning path nodes; wherein, the first triplet includes the first conclusion, the first basis and the first reference coordinate.

[0028] S104. Based on the preset audit rule base, use the rule engine to make a deterministic judgment on the entity candidate set and obtain the second triplet, the second confidence level, the second risk level and the second traceability score; wherein, the second triplet includes the second conclusion, the second basis and the second reference coordinate.

[0029] S105. The first confidence level and the first traceable score, as well as the second confidence level and the second traceable score, are weighted to obtain two weighted voting values, and the difference between the two weighted voting values ​​is obtained to obtain the conflict difference.

[0030] S106. Compare the conflict difference with the preset judgment threshold.

[0031] If the conflict difference is less than a preset judgment threshold, proceed to step S107 and take the conclusion corresponding to the higher weighted vote value as the final conclusion. Otherwise, proceed to step S108, initiate the manual review process, and take the manual review conclusion as the final conclusion.

[0032] S109. Write the final conclusion, reference coordinates, and revision history into a structured template file to generate audit reports in various formats.

[0033] This method uses a large model and a rule engine to review the original document simultaneously, leveraging the advantages of the large model's strong language understanding ability and the rule engine's strong determinism. Furthermore, when the conflict difference between the large model and the rule engine reaches a preset judgment threshold, manual review is performed to avoid problems such as missed reviews or over-marking.

[0034] Step S101 specifically includes steps S201 to S205.

[0035] S201. Perform geometric correction on each page of the original document and divide it according to content type. Output the coordinates of the bounding rectangle of each content area and the third confidence level.

[0036] Performing geometric correction on each page of the original document can eliminate distortions such as perspective distortion and curvature, restoring text, tables, seals, etc. on the page to their standard form, making the recognition in subsequent steps S202 to S204 more accurate and efficient. For example, firstly, the quadrilateral contour of the page is extracted using the PaddlePaddle edge detection model, and then subpixel-level geometric correction is completed through homography transformation.

[0037] S202. Perform font recognition on the text type region and generate a reading order list with hierarchical labels to obtain the first parsing result.

[0038] The hierarchy tags include headings, body text, and footnotes, and headings can include multiple levels, such as headings 1-6.

[0039] For example, firstly, an optical character recognition model is run within the text type region to recognize the text, while simultaneously using a PaddlePaddle font classification model to output character encoding, font category, and confidence score; then, using the center coordinates of the text type region, font size, and font weight as input, the PaddlePaddle reading order model is invoked to generate a reading order list that conforms to human visual habits based on topological sorting, and hierarchical labels are added.

[0040] S203. Perform structural identification on the table-type region to obtain the second parsing result.

[0041] For example, firstly, the PaddlePaddle table structure recognition model is used to output an equivalent Excel structure containing cross-row and column merging information; and simultaneously, the semantics of the table header are extracted for subsequent field-level verification; then, for a small number of tables that fail to be recognized, they are automatically converted to manual annotation.

[0042] S204. Detect the authenticity of the seal and signature and calculate the fourth confidence level to obtain the third analysis result.

[0043] For example, for the seal sub-image, firstly, the PaddlePaddle seal detection model is used to detect the three elements of the seal sub-image: outer circle, five-pointed star, and text ring. If any element is missing, it is judged as "forged". Then, after confirming that the elements of the seal sub-image are complete, the PaddlePaddle seal comparison model is used to calculate the feature cosine similarity with the registered seal image and output the fourth confidence level in the range of 0-1.

[0044] For example, for the signature area, the stroke sequence is first obtained using the PaddlePaddle signature key point model, and the writing dynamic features are extracted using the PaddlePaddle signature dynamics model. Then, the stroke sequence and dynamic features are compared with the reserved handwriting template to output the authenticity of the signature and the fourth confidence level.

[0045] S205. Serialize the first parsing result, the second parsing result, and the third parsing result to form an intermediate representation file.

[0046] For example, the intermediate representation file can be a Markdown file.

[0047] In this embodiment of the application, step S102 specifically includes steps S301 to S303.

[0048] S301. Based on the preset audit rule base, pre-annotate intermediate representation files.

[0049] S302. Input the intermediate representation file into the large model and obtain the output result of the large model.

[0050] Specifically, in step S302, the intermediate representation file is sliced ​​and then input into the large model. For example, it can be sliced ​​according to a three-level granularity of "page-segment-sentence".

[0051] Furthermore, the output format can be specified in the prompts input to the large model. For example, the large model can be required to output a five-tuple of "entity-type-value-sentence offset-page coordinates". The large model can also be required to concatenate cross-page entities, that is, to combine the last sentence on the previous page with the first sentence on the current page to form a context, so as to ensure continuous semantics.

[0052] S303. Deduplicatize the output of the large model, uniformly format the amount and date, merge similar entities, and generate a candidate set of entities including entity address, standard name, type, value, coordinate list, and original part.

[0053] For example, when performing step S303, MinHash and LSH can be used to remove duplicates, and two synonymous entities can be merged when their BERT similarity is greater than 0.95. The entity candidate set generated in step S303 can be a structured JSON file.

[0054] In this embodiment of the application, step S103 involves constructing a thought chain prompt word from the entity candidate set and the preset audit rule base, inputting the thought chain prompt word into the large model, and obtaining the first triplet, the first confidence level, and the first risk level. Specifically, this includes steps S401 to S402.

[0055] S401. Construct the entity candidate set and the preset review rule base into a preset structure of mind chain prompts according to the preset template.

[0056] For example, the preset structure can be "rule restatement - fact listing - thought chain instruction", where rule restatement includes links to legal provisions, fact listing is accompanied by coordinate anchors, and thought chain instruction includes: step-by-step reasoning and outputting conclusions, basis and reference coordinates.

[0057] S402. Input the pre-structured thought chain prompts into the large model, and restrict the output of the large model to the first triplet, the first confidence level, and the first risk level.

[0058] Step S103 calculates the first traceability score of the large model based on the inference path nodes, including steps S501 to S505.

[0059] S501. The thought chain output by the large model is split into inference nodes at the sentence level. The fifth confidence of each inference node is obtained by multiplying the average attention weight of the sentence in the original context with the logical coherence score.

[0060] When performing step S501, the fifth confidence level is calculated according to the following formula:

[0061] ;

[0062] in, Indicates the fifth confidence level; This represents the average attention weight of the sentence in its original context; for example, it represents the average attention value of the sentence from the last layer of attention heads in a large model. This represents the logical coherence score output by pre-trained coherence classifiers such as RoBERTa-large-mnli, ranging from 0 to 1.

[0063] S502, Map the first risk level to a quantitative weight.

[0064] For example, the first risk level may include low risk, medium risk and high risk, and the corresponding quantitative weights may be 1.0, 2.0 and 4.0.

[0065] If the reasoning node involves cross-paragraph, cross-table, or numerical calculations, proceed to step S503 and determine the node complexity weight as 2.0; otherwise, proceed to step S504 and determine the node complexity weight as 1.0.

[0066] S505. Multiply the fifth confidence score, quantization weight, and node complexity weight of each inference node in sequence to obtain the traceability score of a single node.

[0067] When performing step S505, the traceability score of a single node is calculated according to the following formula:

[0068] ;

[0069] in, Indicates a single-node traceable score. This indicates the fifth confidence level. Indicates quantization weights, Indicates node complexity.

[0070] S506, The first traceable score is the 95th percentile of all single-node traceable scores.

[0071] When performing step S506, the first traceability score is calculated according to the following formula:

[0072] .

[0073] in, This indicates the first traceable score.

[0074] Step S104 specifically includes steps S601 to S604.

[0075] S601. Use the rule engine to execute the rigid prohibition type rules in the preset audit rule base. If the rigid prohibition type rules are met, reject the case as a conclusion and obtain the second triplet, the second confidence level and the second risk level, and terminate the subsequent judgment.

[0076] The preset review rule base includes both rigid prohibition type rules and risk warning type rules. When executing step S601, the rigid prohibition type rules are used to judge the content in the candidate entity set. When content that conforms to the rigid prohibition type rules is found, it is rejected as the second conclusion, and a second triplet, a second confidence level, and a second risk level are obtained. Then, steps S602 and S603 are terminated.

[0077] S602. Use the rule engine to execute the risk warning type rules in the preset audit rule base. When the risk warning type rules are met, the warning is taken as the second conclusion, and the second triplet, the second confidence level, and the second risk level are obtained.

[0078] S603. If a risk warning type rule is triggered by a rigid prohibition type rule, enter conflict arbitration: the conclusion of the rigid prohibition type rule shall prevail, and the address of the risk warning type rule shall be written into the additional prompt field for the front end to highlight.

[0079] Front-end highlighting can conspicuously remind staff of risks, enabling them to reassess the highlighted information and avoid undetected errors.

[0080] S604. For each triggering rule, the product of the rule complexity weight and the second confidence level is used as the second traceability score.

[0081] The rule complexity is derived from static analysis using the domain-specific language, with the basic regularization rule being 1.0, cross-table aggregation being 2.5, and machine learning model invocation being 4.0.

[0082] When performing step S604, calculate the following formula: Traceability score = Rule complexity × Entity confidence.

[0083] Furthermore, the document review method based on the collaboration of a large model and a rule engine provided in this application embodiment also includes: parsing the reasoning steps of the large model and constructing a reasoning path graph, and generating an interactive and visual interface that integrates logical relationships and risk distribution by marking the nodes of the reasoning path graph with risk levels and mapping the node positions back to the highlighted original documents.

[0084] Furthermore, the document review method based on the collaboration of a large model and a rule engine provided in this application embodiment also includes steps S701 and S702.

[0085] S701. Add entries with a second traceability score above the preset score to the high traceability whitelist and issue an audit report.

[0086] S702. Entries with a second traceable score less than a preset score are marked as requiring manual review and are highlighted in the reasoning path diagram.

[0087] For example, the preset score can be 3.

[0088] The document review method based on the collaboration of a large model and a rule engine provided in this application embodiment further includes: obtaining at least one review constraint description and updating the preset review rule library.

[0089] By updating the preset audit rule base, it is possible to adapt to changing compliance policies, thereby enabling rapid response to adjustments in regulatory details.

[0090] For example, the auditor can drag and drop or edit the web canvas using a domain-specific language to generate an audit constraint description of "Contract - Amount ≥ 10 million - Must be signed by both parties" and add the audit constraint description to the preset audit rule library.

[0091] This application also provides a document review device based on the collaboration of a large model and a rule engine, such as... Figure 2 As shown, the document review device based on the collaboration of a large model and a rule engine includes a parsing module 201, an extraction module 202, a first evaluation module 203, a second evaluation module 204, a voting module 205, a conclusion module 206, and a reporting module 207.

[0092] The parsing module 201 parses the original document to form an intermediate representation file. The extraction module 202 extracts entities from the intermediate representation file using a large model based on the entity definitions in the preset review rule base, forming an entity candidate set. The first evaluation module 203 constructs a thought chain prompt using the entity candidate set and the preset review rule base, inputs the thought chain prompt into the large model, obtains the first triplet, the first confidence level, and the first risk level, and calculates the first traceability score of the large model based on the inference path nodes; wherein, the first triplet includes the first conclusion, the first basis, and the first reference coordinate. The second evaluation module 204 performs a deterministic judgment on the entity candidate set using a rule engine based on the preset review rule base, obtaining the second triplet, the second confidence level, the second risk level, and the second traceability score; wherein, the second triplet includes the second conclusion, the second basis, and the second reference coordinate. The voting module 205 performs weighted calculations on the first confidence level and the first traceability score, as well as the second confidence level and the second traceability score, to obtain two weighted voting values, and calculates the difference between the two weighted voting values ​​to obtain a conflict difference value. The conclusion module 206 compares the conflict difference with a preset judgment threshold, and if the conflict difference is less than the preset judgment threshold, the conclusion corresponding to the higher weighted voting value is taken as the final conclusion; otherwise, a manual review process is initiated. The report module 207 writes the final conclusion, reference coordinates, and revision traces into a structured template file to generate review reports in various formats.

[0093] The parsing module 201 is specifically used for: geometrically correcting each page of the original document and dividing it according to content type, outputting the bounding rectangle coordinates and third confidence score of each content type region; performing font recognition on text type regions and generating a reading order list with hierarchical labels to obtain the first parsing result; recognizing the structure of table type regions to obtain the second parsing result; detecting the authenticity of seals and signatures and calculating the fourth confidence score to obtain the third parsing result; and serializing the first, second, and third parsing results to form an intermediate representation file.

[0094] The extraction module 202 is specifically used for: pre-annotating the intermediate representation file based on the preset audit rule library; inputting the intermediate representation file into the large model to obtain the output result of the large model; deduplicating the output result of the large model, uniformly formatting the amount and date, merging similar entities, and generating an entity candidate set including entity address, standard name, type, value, coordinate list, and original part.

[0095] The first evaluation module 203 is specifically used to: construct the thought chain prompt words in the preset structure form according to the preset template and the preset review rule base; input the thought chain prompt words in the preset structure form into the large model, and restrict the output of the large model to the first triplet, the first confidence level and the first risk level.

[0096] The second evaluation module 204 is specifically used for: executing rigid prohibition type rules in the preset audit rule library using the rule engine; when the rigid prohibition type rule is met, rejection will be taken as the second conclusion, and the second triplet, second confidence level, and second risk level will be output, terminating subsequent judgments; executing risk warning type rules in the preset audit rule library using the rule engine; when the risk warning type rule is met, a warning will be taken as the second conclusion, and the second triplet, second confidence level, and second risk level will be output; if a risk warning type rule is triggered by a rigid prohibition type rule, conflict arbitration will be initiated: the conclusion of the rigid prohibition type rule will be taken as the standard, and the address of the risk warning type rule will be written into the additional warning field for front-end highlighting; for each triggered rule, the product of the rule complexity weight and the entity confidence level will be used as the second traceability score.

[0097] The document review device 200 based on the collaboration of a large model and a rule engine also includes a classification module. The classification module is used to: write items with a second traceability score above a preset score into a high traceability whitelist and generate a review report; mark items with a second traceability score below a preset score as requiring manual review and link them to the inference path graph for highlighting.

[0098] The document review device 200 based on the collaboration of a large model and a rule engine also includes an update module, which is used to obtain at least one review constraint description and update the preset review rule library.

[0099] The apparatus or module described in the above embodiments can be implemented by a computer chip or physical entity, or by a product with a certain function. For ease of description, the above apparatus is described by dividing it into various modules according to their functions. When implementing the embodiments of this application, the functions of each module can be implemented in one or more software and / or hardware. Of course, a module that implements a certain function can also be implemented by combining multiple sub-modules or sub-units.

[0100] This application also provides an electronic device, including one or more processors and a memory storing computer-executable instructions. When executed by the one or more processors, the computer-executable instructions cause the one or more processors to perform the document review method based on the collaboration of a large model and a rule engine described above in this application.

[0101] This application also provides a computer-readable storage medium storing computer-readable instructions. When the computer-readable instructions are executed by a computer, they implement the document review method based on the collaboration of a large model and a rule engine described above in this application.

[0102] The aforementioned storage media include, but are not limited to, random access memory (RAM), read-only memory (ROM), cache, hard disk drive (HDD), or memory card.

[0103] The various embodiments in this specification are described in a progressive manner. For the same or similar parts between the various embodiments, please refer to each other. Each embodiment focuses on describing the differences from other embodiments.

[0104] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit this application. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of this application.

Claims

1. A document review method based on large model and rule engine cooperation, characterized in that, The method comprises the following steps: parsing the original document to form an intermediate representation file; based on the definition of the entity in the preset audit rule library, using a large model to extract entities from the intermediate representation file to form an entity candidate set; constructing the entity candidate set and the preset audit rule library into a thinking chain prompt word, inputting the thinking chain prompt word into the large model to obtain a first triple, a first confidence and a first risk level, and calculating a first traceable score of the large model according to the reasoning path node; wherein the first triple comprises a first conclusion, a first basis and a first reference coordinate; According to the preset audit rule library, the certainty of the entity candidate set is judged by using a rule engine to obtain a second triple, a second confidence, a second risk level and a second traceable score; wherein the second triple comprises a second conclusion, a second basis and a second reference coordinate; The first confidence and the first traceable score, and the second confidence and the second traceable score are calculated respectively to obtain two weighted voting values, and the difference between the two weighted voting values is obtained to obtain a conflict difference value; Compare the conflict difference value with the preset judgment threshold, and when the conflict difference value is less than the preset judgment threshold, the conclusion corresponding to the high weighted voting value is taken as the final conclusion; otherwise, an artificial review process is started, and the artificial review conclusion is taken as the final conclusion; The final conclusion, the reference coordinate and the revision trace are written into a structured template file to generate an audit report in multiple formats.

2. The method of claim 1, wherein, The parsing of the original document to form an intermediate representation file comprises: Geometric correction is performed on each page of the original document, and the content is divided according to the content type, and the bounding rectangle coordinates and the third confidence of each type of content area are output; Font recognition is performed on the text type area, and a reading order list with hierarchical labels is generated to obtain a first analysis result; The structure of the table type area is recognized to obtain a second analysis result; The authenticity of the seal and signature is detected and the fourth confidence is calculated to obtain a third analysis result; The first analysis result, the second analysis result and the third analysis result are serialized to form the intermediate representation file.

3. The method of claim 1, wherein, The method comprises the following steps: Based on the preset audit rule library, pre-annotate the intermediate representation file; The intermediate representation file is input into the large model to obtain the result output by the large model; The result output by the large model is de-duplicated, the amount and date are uniformly formatted, and the similar entities are merged to generate the entity candidate set including entity address, standard name, type, value, coordinate list and original part.

4. The method of claim 1, wherein, The entity candidate set and the preset audit rule library are constructed into a thinking chain prompt word according to a preset template, and the thinking chain prompt word is input into the large model to obtain a first triple, a first confidence and a first risk level. ​ The thought chain prompt word in the preset structure form is input into the large model, and the large model is limited to output the first triple, the first confidence and the first risk level.

5. The method of claim 1, wherein, The first traceable score of the large model is calculated according to the reasoning path node, including: The thought chain output by the large model is split into reasoning nodes at the sentence level granularity, and the fifth confidence of each reasoning node is obtained by multiplying the average value of the attention weight of the sentence in the original context and the logical coherence score; The first risk level is mapped to a quantitative weight; If the reasoning node involves cross-paragraph, cross-table or numerical calculation, the node complexity weight is determined as 2.0; otherwise, the node complexity weight is determined as 1.0; The fifth confidence, the quantitative weight and the node complexity weight of each reasoning node are multiplied in sequence to obtain a single-node traceable score; The first traceable score is the 95th percentile value of all single-node traceable scores.

6. The method of claim 1, wherein, The second triple, the second risk level and the second traceable score are obtained by using a rule engine to make deterministic judgment on the entity candidate set according to the preset audit rule library, including: The rule engine is used to execute the rigid prohibition type rule in the preset audit rule library, and when the rigid prohibition type rule is met, the second conclusion is rejected, and the second triple, the second confidence and the second risk level are output, and the subsequent judgment is terminated; The rule engine is used to execute the risk prompt type rule in the preset audit rule library, and when the risk prompt type rule is met, the second conclusion is prompted, and the second triple, the second confidence and the second risk level are output; If the risk prompt type rule is triggered by the rigid prohibition type rule, conflict arbitration is entered: the conclusion of the rigid prohibition type rule is used as the standard, and the address of the risk prompt type rule is written into an additional prompt field for front-end highlighting display; For each triggered rule, the product of the rule complexity weight and the second confidence is used as the second traceable score.

7. The method of claim 1, wherein, Further comprising: The reasoning steps of the large model are parsed and constructed into a reasoning path graph, and a node of the reasoning path graph is marked with a risk level, and the node position is reversely mapped to the original document for highlighting display, to generate an interactive visual interface that integrates logical relationships and risk distribution.

8. The method of claim 7, wherein, Further comprising: Entries with the second traceable score above a preset score are written into a high-traceability white list, and an audit report is issued; Entries with the second traceable score less than the preset score are marked as needing manual review, and are linked to the reasoning path graph for highlighting display.

9. The method of claim 1, wherein, Further comprising: At least one audit constraint description is obtained, and the preset audit rule library is updated.

10. A document review apparatus based on large model and rule engine collaboration, characterized by, Comprising: A parsing module is configured to parse an original document to form an intermediate representation file; An extraction module is configured to extract entities from the intermediate representation file using a large model based on definitions of the entities in a preset audit rule library to form an entity candidate set; The first evaluation module is configured to construct the entity candidate set and the preset audit rule library into a thinking chain prompt word, input the thinking chain prompt word into the large model, obtain a first triple, a first confidence and a first risk level, and calculate a first traceable score of the large model according to a reasoning path node; wherein the first triple includes a first conclusion, a first basis and a first reference coordinate; The second evaluation module is configured to determine the entity candidate set using a rule engine according to the preset audit rule library to obtain a second triple, a second confidence, a second risk level and a second traceable score; wherein the second triple includes a second conclusion, a second basis and a second reference coordinate; The voting module is configured to respectively perform weighted calculation on the first confidence and the first traceable score, and the second confidence and the second traceable score to obtain two weighted voting values, and obtain a conflict difference value by subtracting the two weighted voting values; The conclusion module is configured to compare the conflict difference value with a preset judgment threshold, and when the conflict difference value is less than the preset judgment threshold, take the conclusion corresponding to the high weighted voting value as a final conclusion; otherwise, start an artificial review process; The report module is configured to write the final conclusion, the reference coordinate and the revision trace into a structured template file to generate an audit report in multiple formats.

11. An electronic device, comprising: One or more processors; and a memory storing computer executable instructions that, when executed by the one or more processors, cause the one or more processors to perform the document review method based on the large model and the rule engine as claimed in any one of claims 1 to 9. The computer readable storage medium stores computer readable instructions which are executed by a computer to implement the document review method based on the large model and the rule engine as claimed in any one of claims 1 to 9. ​ 12. A computer-readable storage medium, characterized in that, ​

Images