A data auditing method, a data auditing device, an electronic device, and a storage medium

Abstract

The application discloses a material auditing method, a material auditing device, an electronic device and a storage medium, relates to the technical field of computers, and does not hand over complete multi-layer nested logic to a large language model for unified reasoning, but strips complex logic combination operations to a logic operation expression, so that the large language model only participates in local atomic node determination, can inhibit hallucinations generated when the large language model processes complex logic from the root, and distinguishes different types of atomic nodes to configure a differential execution strategy, retains semantic reasoning capability of the large language model, and relies on the logic operation expression to guarantee that a logic execution path is fixed and a result is reproducible, and overcomes defects such as the need for research and development intervention for rule modification of a hard coding scheme and insufficient flexibility for adaptation to multiple types of auditing rules.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a data review method, data review device, electronic device and storage medium. Background Technology

[0002] In business scenarios such as health insurance claims, medical expense review, liability determination, and risk control review, the review conclusion is usually not determined by a single condition, but by a large number of natural language rules. For example, whether a claim liability is established may involve multiple types of information, such as the name of the disease, doctor's diagnosis, examination report, treatment items, scope of drugs, completeness of materials, exclusion clauses, waiting period, pre-existing conditions, and liability labels.

[0003] These types of rules are typically characterized by complex origins, complex expressions, frequent changes, and the need for semantic understanding. Different insurance companies, products, and liability clauses have different requirements for claims review, and rules often exist in the form of clause texts, insurance company review requirement documents, risk control requirement documents, and operational configuration instructions. Natural language rules often contain logical expressions such as "and," "or," "any," "simultaneously satisfy," "not belonging to," "except," "reaching a certain threshold," and "within a certain time range," and these logical expressions may be nested in multiple layers.

[0004] With the development of large language models, existing solutions have begun to attempt to directly input complex review rules and case materials into the large language model, which then outputs the final review conclusion. However, in scenarios with high accuracy requirements, such as insurance claims, large language models are prone to instability when directly processing complex logic. For example, a large language model may miss a certain exemption condition, interpret an "OR" relationship as an "AND" relationship, or omit a branch in multi-level nested conditions, leading to an unreliable final review conclusion.

[0005] Another approach is to hardcode all audit rules into program logic or traditional rule engine rules. While this approach can improve the determinism of logic execution, it lacks the ability to quickly adapt to different insurance companies, products, and liability items. Adding or modifying rules usually requires the intervention of R&D personnel, making it difficult to meet the needs of business scenarios with frequent rule changes. Summary of the Invention

[0006] This application provides a document review method, document review device, electronic device, and storage medium to solve the following problems in the related technology: In claims scenarios, reasoning often relies on complex natural language logic descriptions containing a large number of AND and OR relationships and with nesting. When the whole is handed over to a large language model for processing, the model is very prone to semantic logic errors that do not follow the internal logical structure, resulting in illusions and uncontrollable accuracy. If it is changed to hard coding, although the execution accuracy can be guaranteed, it lacks flexibility and scalability and is difficult to quickly adapt to changing logic.

[0007] This application provides a method for document review, including: Obtain the rule document, parse the logical description content in the rule document, convert the logical description content into a logical operation expression, the logical operation expression includes several atomic nodes and operators used to associate each atomic node, and configure an execution strategy for each atomic node; The large language model receives case materials to be reviewed, extracts information from the case materials, and maps the extracted information to the input information of each atomic node. The large language model performs logical judgments based on the input information corresponding to each atomic node according to the execution strategy configured for each atomic node, and determines the logical judgment value of each atomic node. The large language model substitutes the logical decision value of each atomic node into the logical operation expression to complete the operation and obtain the output variable; The audit conclusion is generated based on the output variables.

[0008] Furthermore, the logical description is transformed into a logical operation expression, including: The logical description is decomposed into several atomic conditions. Each atomic condition is a logical description expressed by only one operator. The atomic conditions are used as atomic nodes, and an execution strategy is configured for each atomic node to obtain the logical judgment value of each atomic node by judging whether the corresponding atomic condition is satisfied. Determine the dependencies between the atomic nodes according to the logical syntax of the logical description, express the AND / OR / nesting relationships between the atomic nodes according to the dependencies using operators, and generate at least one logical operation expression; For each logical operation expression, perform logical operation syntax verification and manual logical verification, and revise the logical operation expression based on the verification results.

[0009] Furthermore, the logical operation syntax verification for each logical operation expression includes: A logic operation execution engine is configured, and the logic operation execution engine supports at least one of the following logical relationships: AND, OR, NOT, nested combination, intermediate variable, threshold judgment, quantity judgment, time window judgment, and set inclusion judgment. The logical operation execution engine performs logical syntax checks between atomic nodes, including YAML (another markup language) structure, field integrity, variable references, circular dependencies, logical operators, execution strategies, and final output variable checks.

[0010] Furthermore, when configuring the execution strategy for each atomic node, the following is included: When configuring the execution strategy, the deterministic conditions for structured field judgment, numerical threshold judgment, time range judgment, and material existence judgment are configured as rule engine rules, and the conditions that need to understand the semantic relationships of medical record text, diagnosis description, and examination report are configured as large language model semantic reasoning rules (llm).

[0011] Furthermore, the step of performing logical judgments based on the input information corresponding to each atomic node according to the execution strategy configured for each atomic node, and obtaining the logical judgment value of each atomic node, includes: Obtain the atomic conditions corresponding to each atomic node, and classify the atomic nodes into rule engine rule nodes or large language model semantic reasoning rule nodes according to the atomic conditions; Based on the input information corresponding to each atomic node, the rule engine rule node is invoked to determine whether the input information satisfies the corresponding atomic condition; the large language model semantic reasoning rule node is invoked to identify the semantics of the input information and determine whether it satisfies the corresponding atomic condition. Based on the determination result of whether the input information corresponding to each atomic node satisfies the corresponding atomic condition, the logical determination value of each atomic node is obtained.

[0012] Furthermore, when configuring the execution strategy for each atomic node, the following is also included: Based on the dependencies between multi-atomic nodes, the operators between multi-atomic nodes are combined into intermediate nodes, and an execution strategy is configured for the intermediate nodes. When obtaining the logical decision value for each atomic node, the following is also included: If intermediate nodes exist, the logic operation execution engine calculates intermediate variables for each intermediate node based on the logical judgment values ​​and logical relationships of each atomic node.

[0013] Furthermore, generating the audit conclusion based on the output variables includes: The output variables include at least one of the following: multi-level nested logical operation results, material integrity identifiers, semantic matching markers, threshold judgment markers, time window judgment markers, set inclusion judgment markers, and intermediate node operation results; The generated review conclusion includes at least one of the following: review passed, review rejected, risk control triggered, materials missing, and transfer to manual review.

[0014] This application also provides a document review device, the device comprising: The rule logic orchestration module is used to obtain rule documents, parse the logical description content in the rule documents, convert the logical description content into logical operation expressions, the logical operation expressions include several atomic nodes and operators used to associate each atomic node, and configure an execution strategy for each atomic node. The case information extraction module is used to receive case materials to be reviewed by the large language model, extract information from the case materials to be reviewed, and map the extracted information to the input information of each atomic node. The hybrid atomic decision execution module is used to perform logical decision based on the input information corresponding to each atomic node according to the execution strategy configured for each atomic node, and obtain the logical decision value of each atomic node. The result aggregation module is used to substitute the logical decision value of each atomic node into the logical operation expression to complete the operation and obtain the output variable; The audit conclusion output module is used to generate audit conclusions based on the output variables.

[0015] This application also provides an electronic device, including: a memory for storing a computer program; and a processor for executing the computer program to implement the steps of any of the above-described data review methods.

[0016] This application also provides a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the steps of any of the above-described document review methods.

[0017] This application first obtains the rule document and parses its internal logical description, transforming it into a logical expression containing several atomic nodes and associative operators. Simultaneously, an execution strategy is configured for each atomic node. Then, a large language model receives the case materials to be reviewed and extracts information, mapping the extracted information to the input information of each atomic node. Next, based on the execution strategy configured for each atomic node and the corresponding input information, logical judgments are performed to obtain the logical judgment value for each atomic node. Finally, all logical judgment values ​​are substituted into the logical expression to obtain the output variable, and the review conclusion is generated based on the output variable. This solution no longer relies on the large language model for unified reasoning of complete multi-layered nested logic, but instead decouples complex logical combinations and operations to the logical expression. The large language model only participates in the judgment of local atomic nodes, fundamentally suppressing the illusions that arise when the large language model processes complex logic. Furthermore, it differentiates the configuration of execution strategies for different types of atomic nodes, preserving the semantic reasoning capabilities of the large language model while ensuring a fixed logical execution path and reproducible results through the logical expression. This overcomes the shortcomings of hard-coded solutions, such as the need for R&D intervention for rule modifications and insufficient flexibility in adapting to multiple types of review rules. Attached Figure Description

[0018] To more clearly illustrate the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a flowchart illustrating a document review method in one embodiment of this application; Figure 2 This is a logic diagram of a document review method in one embodiment of this application; Figure 3 This is a flowchart of the offline DSL rule asset generation process in one embodiment of this application; Figure 4 This is a flowchart of online case execution in one embodiment of this application; Figure 5 This is a schematic diagram of the liability review logic for serious illness in one embodiment of this application; Figure 6 This is a structural block diagram of a data review device in one embodiment of this application; Figure 7 This is an internal structural diagram of a computer device in one embodiment of this application. Detailed Implementation

[0020] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of this application.

[0021] It should be noted that, in the description of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. The terms "first," "second," etc., in this application are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0022] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0023] Based on the background technology, when directly using large language models to process complex rules, the complex logical structure is easily weakened or misunderstood by the model. In insurance claims scenarios, review rules often contain multiple layers of "AND / OR / NOT" relationships. If these conditions are input into a large language model as a whole and the model directly gives the final conclusion, the model must understand the facts of the case and simultaneously complete complex logical combinations, which can easily lead to omissions, misjudgments, or missing logical branches.

[0024] When directly using large language models to execute complex logic, the determinism of the logical path cannot be guaranteed. The output of large language models is affected by factors such as prompt words, context length, and the order of input information. For the same type of complex rules, slight changes in the input material may lead to different inference paths generated by the model, resulting in uncontrollable review conclusions.

[0025] Traditional rule engines lack semantic understanding capabilities. Insurance claims review involves numerous judgments requiring natural language understanding, such as "Does the medical record description support a certain diagnosis?", "Do the examination results meet the disease definition requirements?", and "Is the treatment related to the disease?". Traditional rule engines typically rely on structured fields and struggle to directly process unstructured textual information such as medical records, diagnostic descriptions, and examination reports.

[0026] Hard-coded methods are costly to maintain. Rules vary significantly across different insurance companies, products, and liability categories. If all rules are developed using code, adding or adjusting business logic requires developers to modify the code and redeploy, making it difficult to support rapid expansion with multiple insurance companies, products, and rules.

[0027] Plain text or JSON configurations are insufficient for expressing complex logic diagrams and execution strategies. Plain text descriptions lack a semantic structure that allows for deterministic execution. While JSON configurations can hold some fields, without domain-specific language operators, variable mechanisms, intermediate variable mechanisms, and execution strategy binding mechanisms, they still struggle to express complex rule structures such as multi-level nested logic, variable reuse, and branch execution method selection.

[0028] Therefore, current research on improving the accuracy of natural language logical descriptions is limited to how to analyze and identify complex logical combinations, which leads to increased costs or inaccurate logic identification causing illusions when processing complex logic.

[0029] To address the aforementioned shortcomings, embodiments of this application provide a data verification method. By shifting the approach, the large language model no longer directly undertakes complex logical combination reasoning, but is only responsible for local atomic semantic judgments. Complex logical combinations are deterministically completed by the DSL execution engine, thereby reducing the illusion problem in the process of complex logic processing.

[0030] DSL stands for Domain Specific Language, a specialized language distinct from general-purpose programming languages ​​like Java and Python. To implement conditional judgments and multi-rule combination filtering in various scenarios, all mainstream DSLs are equipped with a unified and universal basic logical operation system. The core includes three basic operations: logical AND, logical OR, and logical NOT. These are the core foundation for DSLs to implement business condition judgments, adapting to the expression writing, rule configuration, and condition filtering requirements of various DSLs. Table 1 provides core information on the three common basic logical operations of DSLs, covering operation names, symbols, operation rules, operation characteristics, and application scenarios.

[0031] Table 1

[0032] The three basic logical operations in a DSL have fixed operation priorities, which are crucial for standardizing expression writing and avoiding logical ambiguity. The priorities, from highest to lowest, are logical NOT, logical AND, and logical OR. That is, during computation, the NOT operation is performed first, followed by the AND operation with multiple conditions, and finally the OR operation. If it is necessary to adjust the order of operations or customize logical combination rules in complex business scenarios, the corresponding conditions can be enclosed in parentheses "()" to artificially increase the priority of the operations within the parentheses. This ensures that the computational logic of the DSL expression is completely consistent with the business requirements and avoids logical deviations caused by default priorities.

[0033] like Figure 1 As shown, an embodiment of this application provides a data review method, including the following steps: Step S1: Obtain the rule document, parse the logical description content in the rule document, convert the logical description content into a logical operation expression, the logical operation expression includes several atomic nodes and operators used to associate each atomic node, and configure an execution strategy for each atomic node; Step S2: The large language model receives the case data to be reviewed, extracts information from the case data, and maps the extracted information to the input information of each atomic node. Step S3: The large language model performs logical judgment based on the input information corresponding to each atomic node according to the execution strategy configured for each atomic node, and determines the logical judgment value of each atomic node. Step S4: The large language model substitutes the logical decision value of each atomic node into the logical operation expression to complete the operation and obtain the output variable; Step S5: Generate an audit conclusion based on the output variables.

[0034] The process begins by acquiring the rule document and parsing its internal logical description, transforming it into a logical expression containing several atomic nodes and associated operators. An execution strategy is then configured for each atomic node. Next, the large language model receives the case data to be reviewed and extracts information, mapping the extracted information to the input information for each atomic node. Following this, logical judgments are performed based on the configured execution strategies of each atomic node and the corresponding input information to obtain the logical judgment value for each atomic node. Finally, all logical judgment values ​​are substituted into the logical expression to obtain the output variable, and a review conclusion is generated based on the output variable. This solution no longer entrusts the complete multi-layered nested logic to a large language model for unified reasoning. Instead, it decouples complex logical combination operations and performs them through logical operation expressions. The large language model only participates in the determination of local atomic nodes, which can fundamentally suppress the illusions that arise when the large language model processes complex logic. At the same time, it distinguishes between different types of atomic nodes and configures differentiated execution strategies. This retains the semantic reasoning capabilities of the large language model while relying on logical operation expressions to ensure that the logical execution path is fixed and the results are reproducible. It overcomes the shortcomings of hard-coded solutions, such as the need for R&D intervention for rule modification and insufficient flexibility in adapting to multiple types of audit rules. It is suitable for business scenarios with complex and frequently updated rules, such as health insurance claims and risk control audits.

[0035] Among them, Large Language Model (LLM) refers to a deep learning model trained on a large amount of text data, which enables the model to generate natural language text or understand the meaning of language text.

[0036] The large language model used in the steps can be a relevant open-source multimodal large model, or it can be a multimodal large model trained or fine-tuned based on the labeled data of the document to be identified.

[0037] Before using the large language model, the process includes training the large language model, which involves: collecting training data, including image-text pairs corresponding to each training task; wherein the training tasks include image classification, image-text matching, and image description generation; performing an alignment pre-training phase based on the image classification task and its corresponding image-text pairs, and the image-text matching task and its corresponding image-text pairs; and performing a generation pre-training phase on the large language model based on the results of the alignment pre-training phase, combined with the image description generation task and its corresponding image-text pairs.

[0038] The training data can be in the form of image-text pairs, which can be in the form of image-text pairs such as images and text pairs corresponding to the images, or images and text pairs based on image questions and answers, depending on the task.

[0039] The large language model performs logical judgments based on the input information corresponding to each atomic node according to the execution strategy configured for each atomic node. The process of determining the logical judgment value of each atomic node is to use the large language model to recognize whether the document image information meets the conditions configured for the corresponding atomic node. The execution strategy is to give the logical judgment value of each atomic node for various conditions that are met.

[0040] Specifically, compared to step S3, the large language model performs the following steps: Acquire the document image to be identified, perform structural recognition on the document image to obtain the structural information of the document image to be identified; The cropping position of the document is determined based on the structured information corresponding to the obtained document image to be identified. The original document image to be identified is cropped to obtain at least one region patch (region image) that includes different structured region information. Each region patch can include one type of structured region information. The text content in each region's map tiles is recognized based on the character recognition model, resulting in unformatted text information in each region's map tiles; The system compares the region tiles and unformatted text information with the conditions configured for the corresponding atomic nodes to determine whether the conditions are met. If the conditions are met, the system outputs the logical judgment value of the corresponding atomic node as one; otherwise, it outputs the logical judgment value of the corresponding atomic node as zero.

[0041] like Figure 2 As shown, the logic of the data review method consists of two parts: offline DSL rule asset generation and online case execution. Figure 2 This demonstrates the overall relationship between offline DSL rule asset generation and online case execution.

[0042] Figure 3 Flowchart for generating offline DSL rule assets; Figure 3Demonstrates the complete offline process from insurance company rule documents to DSL asset publication.

[0043] The offline rule asset construction phase includes S101-S105.

[0044] S101: Obtain the insurance company's rule document and generate a DSL (Logical Operation) expression.

[0045] The system retrieves input materials from insurance companies, including claims requirements documents, product terms, risk control rules, and audit rule descriptions. These materials may include structured rule tables, semi-structured configuration documents, and natural language text.

[0046] The system generates a DSL expression based on candidate atomic conditions. The DSL is carried out in YAML format, but the internal logic syntax, variable mechanism, execution strategy fields, and node dependencies of YAML are defined by the custom DSL.

[0047] S102: Bind the execution policy to the DSL node.

[0048] In DSL, each atomic node or intermediate node can be configured with an execution strategy. The execution strategy is manually configured and includes rules and LLM. For deterministic conditions such as structured field judgments, numerical threshold judgments, time range judgments, and material existence judgments, rules can be configured; for conditions that require understanding the semantic relationships of medical record text, diagnostic descriptions, and examination reports, LLM can be configured.

[0049] S103: DSL syntax check.

[0050] After generating the DSL, the system performs syntax validation through the DSL execution engine, including validation of YAML structure, field integrity, variable references, circular dependencies, logical operators, execution strategies, and final output variables.

[0051] S104: Manual logic verification.

[0052] After the syntax check passes, the business logic expressed by the DSL is manually verified. The focus of manual verification is not on checking the DSL syntax, but on whether the DSL accurately expresses the meaning of the company's original rules.

[0053] S105: DSL Asset Release.

[0054] After passing DSL syntax verification and manual logic verification, the DSL is published as an offline rule asset corresponding to the insurance company, product, liability item, or risk control rule. Subsequently, online cases under that insurance company or liability item are reviewed and executed according to this DSL asset.

[0055] The logical relationships supported by DSL include AND, OR, NOT, nested combinations, intermediate variables, threshold judgments, quantity judgments, time window judgments, and set inclusion judgments. DSL supports the combination of any variables to form new variables. For example: A = Does the disease name meet the definition of a critical disease?; B = Does the doctor's diagnosis support a critical disease?; C = Does the examination report meet the indicator requirements?; D = A AND B AND C; E = Does a disclaimer exist?; Result = D AND NOT E.

[0056] Figure 4 A flowchart for online case execution; Figure 4 The process demonstrates the steps of case input, information extraction, DSL loading, hybrid execution, result output, and feedback attribution.

[0057] The online case execution phase includes: S201-S205.

[0058] S201: Obtain case input.

[0059] The system receives input for cases awaiting review. Case input consists of a mix of structured data and unstructured text, including applicant information, insured information, date of visit, invoice amount, hospital name, expense details, medical records, discharge summary, examination report, diagnosis certificate, and claim application instructions.

[0060] S202: Case information extraction and node input trimming.

[0061] Before executing DSL, the system extracts information from the case materials, converting the original case materials into local inputs that each atomic node needs to consume. For nodes that require judgment by a large language model, the full text of the case is not directly input; instead, relevant information after extraction and trimming is input.

[0062] S203: Execute DSL determination.

[0063] The system executes DSL nodes according to the execution plan. For rule nodes, the system calls the rule execution module; for LLM nodes, the system calls the large language model semantic execution module. At this stage, the large language model is only responsible for atomic condition judgments and not for the overall complex logic combination.

[0064] S204: Calculation of intermediate variables.

[0065] After the atomic nodes output their results, the DSL execution engine calculates intermediate variables based on the logical relationships defined in the DSL. These intermediate variables can be formed by combining atomic nodes or by combining other intermediate variables.

[0066] S205: Final audit result output.

[0067] The system generates review conclusions based on the final output variables of the DSL, such as review passed, review rejected, risk control triggered, materials missing, and transfer to manual review.

[0068] In this embodiment, converting the logical description into a logical operation expression includes: The logical description is decomposed into several atomic conditions. Each atomic condition is a logical description expressed by only one operator. The atomic conditions are used as atomic nodes, and an execution strategy is configured for each atomic node to obtain the logical judgment value of each atomic node by judging whether the corresponding atomic condition is satisfied. Determine the dependencies between the atomic nodes according to the logical syntax of the logical description, express the AND / OR / nesting relationships between the atomic nodes according to the dependencies using operators, and generate at least one logical operation expression; For each logical operation expression, perform logical operation syntax verification and manual logical verification, and revise the logical operation expression based on the verification results.

[0069] This process involves breaking down logical descriptions into atomic conditions and constructing atomic nodes. Dependencies between atomic nodes are determined based on the logical syntax of the logical description. Operators are used to establish AND / OR / nested relationships between atomic nodes to generate logical operation expressions. Simultaneously, logical operation syntax verification and manual logical verification are performed on these expressions, and the expressions are corrected based on the verification results. This step completely transforms natural language rules into standardized, executable logical operation expressions, proactively identifying logical deviations and structural errors that may occur during the logical translation process. It ensures that the logical operation expressions fully reproduce the logical relationships of the original rules, avoiding errors in subsequent atomic node logic judgments and overall expression calculations due to expression distortion. This further reduces underlying logical interference in the atomic node judgment stage of the large language model, improving the rule restoration accuracy of the entire review process. Business personnel can directly adjust rules based on the dual verification process, completing rule iterations without requiring developers to modify the underlying code.

[0070] In this embodiment, performing logical operation syntax verification on each logical operation expression includes: A logic operation execution engine is configured, and the logic operation execution engine supports at least one of the following logical relationships: AND, OR, NOT, nested combination, intermediate variable, threshold judgment, quantity judgment, time window judgment, and set inclusion judgment. The logical operation execution engine performs logical syntax checks between atomic nodes, including YAML (another markup language) structure, field integrity, variable references, circular dependencies, logical operators, execution strategies, and final output variable checks.

[0071] Relying on a unified logic operation execution engine to carry out all types of business logic operations, it automates the entire syntax verification process. It can batch intercept syntax defects, variable errors, operator misuse, and circular dependency issues at the configuration level of logic operation expressions, unify the execution standards of all logic operation expressions, fix the operation priority of multi-level nested logic, reduce the workload of manual line-by-line verification, and provide stable and unified underlying operation support for the logical judgment of each atomic node and the operation of intermediate nodes, avoiding abnormal output variable results caused by engine capability deficiencies or syntax configuration errors.

[0072] In this embodiment, configuring the execution strategy for each atomic node includes: When configuring the execution strategy, the deterministic conditions for structured field judgment, numerical threshold judgment, time range judgment, and material existence judgment are configured as rule engine rules, and the conditions that need to understand the semantic relationships of medical record text, diagnosis description, and examination report are configured as large language model semantic reasoning rules (llm).

[0073] Specifically, all atomic nodes are divided into two execution strategies. Only atomic conditions requiring semantic understanding are handled by the large language model, significantly reducing the reasoning scope of the large language model. This allows the large language model to perform semantic recognition of a single atomic node without undertaking multi-layered complex logical deductions. This effectively reduces output fluctuations caused by factors such as context length and input information order, and reduces the probability of hallucination generation. At the same time, the stable and unbiased advantages of the rule engine rules are used to handle all structured deterministic conditions, achieving complementarity between the rule engine rules and the semantic reasoning rules of the large language model. This balances the high-speed judgment of structured information with the semantic recognition capability of medical text, adapting to business scenarios where structured forms and unstructured medical texts coexist in case materials.

[0074] In this embodiment, the step of performing logical judgment based on the input information corresponding to each atomic node according to the execution strategy configured for each atomic node, and obtaining the logical judgment value of each atomic node, includes: Obtain the atomic conditions corresponding to each atomic node, and classify the atomic nodes into rule engine rule nodes or large language model semantic reasoning rule nodes according to the atomic conditions; Based on the input information corresponding to each atomic node, the rule engine rule node is invoked to determine whether the input information satisfies the corresponding atomic condition; the large language model semantic reasoning rule node is invoked to identify the semantics of the input information and determine whether it satisfies the corresponding atomic condition. Based on the determination result of whether the input information corresponding to each atomic node satisfies the corresponding atomic condition, the logical determination value of each atomic node is obtained.

[0075] This solution achieves complete decoupling of the two types of atomic node judgment processes, with different types of nodes executing judgments independently without interfering with each other. It can accurately output standardized Boolean logic judgment values ​​for each atomic node, providing a unified and unbiased basic variable for logical expression operations. This avoids single-node judgment distortion caused by node type confusion and judgment logic crosstalk. The judgment process of each atomic node is independent and traceable, which facilitates subsequent location of atomic conditions corresponding to abnormal audit results and improves the efficiency of case problem investigation.

[0076] In this embodiment, when configuring the execution strategy for each atomic node, the following is also included: Based on the dependencies between multi-atomic nodes, the operators between multi-atomic nodes are combined into intermediate nodes, and an execution strategy is configured for the intermediate nodes. When obtaining the logical decision value for each atomic node, the following is also included: If intermediate nodes exist, the logic operation execution engine calculates intermediate variables for each intermediate node based on the logical judgment values ​​and logical relationships of each atomic node.

[0077] By adding intermediate nodes to split excessively long, multi-branch logical operation expressions, the overall complex logic is broken down into multiple intermediate variables for step-by-step calculation. This reduces the computational complexity of a single logical operation expression, avoids the problem of missing branches in a single operation of multi-layered nested logic, and allows intermediate variables to be reused in multiple logical branches. This simplifies the configuration difficulty of complex rule expressions. At the same time, relying on the logic operation execution engine to uniformly calculate intermediate variables ensures that the results of segmented operations are consistent with the overall logical operation results, improves the readability and maintainability of complex multi-branch rules, and lowers the operational threshold for rule configuration personnel.

[0078] The logic operation execution engine supports combining logic from any variables to form new variables. For example: A = Does the disease name meet the definition of a critical disease?; B = Does the doctor's diagnosis support a critical disease?; C = Does the examination report meet the indicator requirements?; D = A AND B AND C; E = Does a disclaimer exist?; Result = D AND NOT E.

[0079] like Figure 5 As shown, Figure 5 This is a schematic diagram illustrating the logic of liability review for serious illnesses. Figure 5 This demonstrates the logical relationship between atomic conditions, intermediate variables, and final results in the review of critical illnesses.

[0080] In this embodiment, generating the audit conclusion based on the output variable includes: The output variables include at least one of the following: multi-level nested logical operation results, material integrity identifiers, semantic matching markers, threshold judgment markers, time window judgment markers, set inclusion judgment markers, and intermediate node operation results; The generated review conclusion includes at least one of the following: review passed, review rejected, risk control triggered, materials missing, and transfer to manual review.

[0081] This solution aggregates all computational markers of atomic nodes and intermediate nodes to form complete output variables, fully retaining the judgment status of all subdivided conditions across the entire chain. Relying on the combination relationships of various markers within the output variables, it accurately matches multiple standardized review conclusions, covering the business results of the entire claims review scenario. It fully retains all judgment traces to achieve full-chain traceability of review conclusions, making it easier for business personnel to review cases, optimize review rules and risk control standards. Standardized review conclusions can be directly connected to the business system to achieve automatic case diversion, reduce the amount of manual review intervention, and improve the implementation effect of review automation.

[0082] In the above-mentioned document review method, the rule document is first obtained and its internal logical description is parsed. The logical description is then transformed into a logical operation expression containing several atomic nodes and associated operators. At the same time, an execution strategy is configured for each atomic node. Then, the large language model receives the case documents to be reviewed and completes information extraction. The extracted information is mapped to the input information of each atomic node. Next, according to the execution strategy configured for each atomic node, the logical judgment is performed in combination with the corresponding input information to obtain the logical judgment value of each atomic node. Finally, all logical judgment values ​​are substituted into the logical operation expression to obtain the output variable, and the review conclusion is generated based on the output variable. This solution no longer entrusts the complete multi-layered nested logic to a large language model for unified reasoning. Instead, it decouples complex logical combination operations and performs them through logical operation expressions. The large language model only participates in the determination of local atomic nodes, which can fundamentally suppress the illusions that arise when the large language model processes complex logic. At the same time, it distinguishes between different types of atomic nodes and configures differentiated execution strategies. This retains the semantic reasoning capabilities of the large language model while relying on logical operation expressions to ensure that the logical execution path is fixed and the results are reproducible. It overcomes the shortcomings of hard-coded solutions, such as the need for R&D intervention for rule modification and insufficient flexibility in adapting to multiple types of audit rules. It is suitable for business scenarios with complex and frequently updated rules, such as health insurance claims and risk control audits.

[0083] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method.

[0084] In one embodiment, such as Figure 6 As shown, a data review device 10 is provided, including: a rule logic arrangement module 1, a case information extraction module 2, a hybrid atomic judgment execution module 3, a result aggregation module 4, and a review conclusion output module 5.

[0085] The rule logic orchestration module 1 is used to obtain a rule document, parse the logical description content in the rule document, convert the logical description content into a logical operation expression, the logical operation expression includes several atomic nodes and operators used to associate each atomic node, and configure an execution strategy for each atomic node.

[0086] The case information extraction module 2 is used to receive case materials to be reviewed, extract information from the case materials to be reviewed, and map the extracted information into the input information of each atomic node.

[0087] The hybrid atomic decision execution module 3 is used to perform logical decision based on the input information corresponding to each atomic node according to the execution strategy configured for each atomic node, and to determine the logical decision value of each atomic node.

[0088] The result aggregation module 4 is used to substitute the logical decision value of each atomic node into the logical operation expression to complete the operation and obtain the output variable.

[0089] The audit conclusion output module 5 is used to generate an audit conclusion based on the output variables.

[0090] Understandably, the data review device 10 includes a DSL execution engine and a large language model. The DSL execution engine includes a rule logic orchestration module 1, and the large language model includes a case information extraction module 2, a hybrid atomic judgment execution module 3, a result aggregation module 4, and a review conclusion output module 5. The DSL execution engine can also be integrated into the large language model.

[0091] In this embodiment, the rule logic orchestration module 1 includes: An atomic condition generation module is used to decompose the logical description content into several atomic conditions. Each atomic condition is a logical description expressed by only one operator. The atomic conditions are used as atomic nodes, and an execution strategy is configured for each atomic node to obtain the logical judgment value of each atomic node by judging whether the corresponding atomic condition is satisfied. The logic operation generation module is used to determine the dependency relationship between the atomic nodes according to the logic syntax of the logic description content, express the AND / OR / nesting relationship between the atomic nodes with operators according to the dependency relationship, generate at least one logic operation expression, and combine the multi-atomic nodes and the operators between the multi-atomic nodes into an intermediate node according to the dependency relationship of the multi-atomic nodes. The execution strategy configuration submodule is used to configure the deterministic conditions of structured field judgment, numerical threshold judgment, time range judgment, and material existence judgment as rule engine rules when configuring the execution strategy, configure the conditions that need to understand the semantic relationship of medical record text, diagnosis description, and examination report as large language model semantic reasoning rules, and configure the execution strategy for the intermediate node. The logical operation syntax verification module is used to set up the logical operation execution engine. The logical operation execution engine supports at least one of the following logical relationships: AND, OR, NOT, nested combination, intermediate variables, threshold judgment, quantity judgment, time window judgment, and set inclusion judgment. The logical operation execution engine performs logical syntax verification between atomic nodes, including markup language structure, field integrity, variable reference, circular dependency, logical operators, execution strategy, and final output variable verification. The manual logic verification module is used to perform manual logic verification on each logical operation expression. The logical operation asset publishing module is used to encapsulate and publish logical operation expressions that have been verified and corrected.

[0092] In this embodiment, the case information extraction module 2 includes: a case data receiving submodule, a case information extraction module, and a standardized mapping submodule; The case data receiving submodule is used to receive case data to be reviewed, which includes structured forms, medical record texts, examination reports, and diagnostic documents. The case information extraction module is used to extract information from the case materials to be reviewed, and extract structured field information and unstructured medical text information. The standardized mapping submodule is used to uniformly convert the extracted information into a format and map it into the input information of each atomic node.

[0093] In this embodiment, the hybrid atomic decision execution module 3 includes: a logic operation execution plan generation module, a hybrid execution scheduling module, a node type classification submodule, a rule execution module, a large language model semantic execution module, and an intermediate variable calculation submodule; The logic operation execution plan generation module is used to read the execution strategies and variable dependencies of each atomic node and intermediate node, and generate a node execution plan; The hybrid execution scheduling module is used to distinguish different types of atomic nodes according to the execution plan and to schedule the execution of atomic nodes with no dependencies in parallel. The node type classification submodule is used to obtain the atomic conditions corresponding to each atomic node, and classify the atomic nodes into rule engine rule nodes or large language model semantic reasoning rule nodes according to the atomic conditions. The rule execution module is used to call rules to determine whether the input information satisfies the corresponding atomic conditions based on the input information corresponding to each atomic node. The large language model semantic execution module is used to call the large language model to recognize the semantics of the input information and determine whether the corresponding atomic conditions are met. The intermediate variable calculation submodule is used to calculate intermediate variables based on the logical judgment values ​​and logical relationships of each atomic node through the logical operation execution engine if intermediate nodes exist, and to obtain the logical judgment value of each atomic node based on the judgment result of whether the input information corresponding to each atomic node satisfies the corresponding atomic condition.

[0094] In this embodiment, the result aggregation module 4 includes: a judgment value loading submodule, a result calculation module, and an output variable encapsulation submodule; The judgment value loading submodule is used to receive the logical judgment values ​​of all atomic nodes and intermediate nodes; The result calculation module is used to substitute the logical judgment value of each atomic node and intermediate node into the logical operation expression to complete the calculation; The output variable encapsulation submodule is used to summarize the calculation results and generate output variables that include multi-layer nested logical operation results, material integrity identifiers, semantic matching markers, threshold judgment markers, time window judgment markers, set inclusion judgment markers, and intermediate node operation results.

[0095] In this embodiment, the audit conclusion output module 5 includes: an output variable parsing submodule, a business conclusion matching submodule, and a result output logging submodule; The output variable parsing submodule is used to read various judgment flags in the output variable; The business conclusion matching submodule is used to generate an audit conclusion based on the output variables. The generated audit conclusion includes at least one of the following: audit passed, audit rejected, risk control hit, materials missing, transfer to manual review, inconsistent material information, waiting period not met, pre-existing conditions exempt from liability, treatment items not covered by compensation, and diagnosis mismatch with compensation liability. The result output traceability submodule is used to output standardized audit conclusions and simultaneously retain atomic node judgment details and logical operation path records.

[0096] In the aforementioned document review device, the system first acquires the rule document and parses its internal logical description, transforming the logical description into a logical expression containing several atomic nodes and associated operators. Simultaneously, an execution strategy is configured for each atomic node. Next, the large language model receives the case documents to be reviewed and extracts information, mapping the extracted information to the input information of each atomic node. Then, based on the execution strategy configured for each atomic node and the corresponding input information, logical judgments are performed to obtain the logical judgment value for each atomic node. Finally, all logical judgment values ​​are substituted into the logical expression to obtain the output variable, and a review conclusion is generated based on the output variable. This solution no longer entrusts the complete multi-layered nested logic to a large language model for unified reasoning. Instead, it decouples complex logical combination operations and performs them through logical operation expressions. The large language model only participates in the determination of local atomic nodes, which can fundamentally suppress the illusions that arise when the large language model processes complex logic. At the same time, it distinguishes between different types of atomic nodes and configures differentiated execution strategies. This retains the semantic reasoning capabilities of the large language model while relying on logical operation expressions to ensure that the logical execution path is fixed and the results are reproducible. It overcomes the shortcomings of hard-coded solutions, such as the need for R&D intervention for rule modification and insufficient flexibility in adapting to multiple types of audit rules. It is suitable for business scenarios with complex and frequently updated rules, such as health insurance claims and risk control audits.

[0097] For a description of the features in the embodiment corresponding to the document review device, please refer to the relevant descriptions in the embodiment corresponding to the document review method; they will not be repeated here. It is understood that the document review method can also be improved based on the descriptions of the features in the document review device.

[0098] Embodiments of this application also provide an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to perform the steps in any of the above-described data review method embodiments.

[0099] In one embodiment, the electronic device may be a server, and its internal structure diagram may be as follows: Figure 7 As shown, this electronic device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The database stores data for document review. The network interface communicates with external terminals via a network connection. When the computer program is executed by the processor, it implements a document review method.

[0100] Embodiments of this application also provide a computer-readable storage medium storing a computer program, wherein the computer program is configured to execute the steps in any of the above-described data review method embodiments when it is run.

[0101] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0102] The embodiments of this application also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in any of the above-described data review method embodiments.

[0103] Embodiments of this application also provide another computer program product, including a non-volatile computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps in any of the above-described data review method embodiments.

[0104] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0105] The foregoing has provided a detailed description of the document review method, document review device, electronic device, and storage medium provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are only intended to aid in understanding the method and core ideas of this application. It should be noted that those skilled in the art can make various improvements and modifications to this application without departing from its principles, and these improvements and modifications also fall within the protection scope of the claims of this application.

Claims

1. A method for document verification, characterized in that, include: Obtain the rule document, parse the logical description content in the rule document, convert the logical description content into a logical operation expression, the logical operation expression includes several atomic nodes and operators used to associate each atomic node, and configure an execution strategy for each atomic node; The large language model receives case materials to be reviewed, extracts information from the case materials, and maps the extracted information to the input information of each atomic node. The large language model performs logical judgments based on the input information corresponding to each atomic node according to the execution strategy configured for each atomic node, and determines the logical judgment value of each atomic node. The large language model substitutes the logical decision value of each atomic node into the logical operation expression to complete the operation and obtain the output variable; The audit conclusion is generated based on the output variables.

2. The document review method according to claim 1, characterized in that, Transforming the logical description into a logical operation expression includes: The logical description is decomposed into several atomic conditions. Each atomic condition is a logical description expressed by only one operator. The atomic conditions are used as atomic nodes, and an execution strategy is configured for each atomic node to obtain the logical judgment value of each atomic node by judging whether the corresponding atomic condition is satisfied. Determine the dependencies between the atomic nodes according to the logical syntax of the logical description, express the AND / OR / nesting relationships between the atomic nodes according to the dependencies using operators, and generate at least one logical operation expression; For each logical operation expression, perform logical operation syntax verification and manual logical verification, and revise the logical operation expression based on the verification results.

3. The document review method according to claim 2, characterized in that, The logical operation syntax verification for each logical operation expression includes: A logic operation execution engine is set up, and the logic operation execution engine supports at least one of the following logical relationships: AND, OR, NOT, nested combination, intermediate variable, threshold judgment, quantity judgment, time window judgment, and set inclusion judgment. The logical operation execution engine performs logical syntax verification between atomic nodes, including statement structure, field integrity, variable references, circular dependencies, logical operators, execution strategies, and final output variable verification.

4. The document review method according to claim 1, characterized in that, When configuring the execution strategy for each atomic node, the following is included: When configuring the execution strategy, the deterministic conditions for structured field judgment, numerical threshold judgment, time range judgment, and material existence judgment are configured as rule engine rules, and the conditions that require understanding the semantic relationships of medical record text, diagnosis description, and examination report are configured as semantic reasoning rules of large language model.

5. The document review method according to claim 4, characterized in that, The step of performing logical judgments based on the input information corresponding to each atomic node according to the execution strategy configured for each atomic node, and obtaining the logical judgment value of each atomic node, includes: Obtain the atomic conditions corresponding to each atomic node, and classify the atomic nodes into rule engine rule nodes or large language model semantic reasoning rule nodes according to the atomic conditions; Based on the input information corresponding to each atomic node, the rule engine rule node is invoked to determine whether the input information satisfies the corresponding atomic condition; the large language model semantic reasoning rule node is invoked to identify the semantics of the input information and determine whether it satisfies the corresponding atomic condition. Based on the determination result of whether the input information corresponding to each atomic node satisfies the corresponding atomic condition, the logical determination value of each atomic node is obtained.

6. The document review method according to claim 5, characterized in that, When configuring the execution strategy for each atomic node, the following is also included: Based on the dependencies between multi-atomic nodes, the operators between multi-atomic nodes are combined into intermediate nodes, and an execution strategy is configured for the intermediate nodes. When obtaining the logical decision value for each atomic node, the following is also included: If intermediate nodes exist, the logic operation execution engine calculates intermediate variables for each intermediate node based on the logical judgment values ​​and logical relationships of each atomic node.

7. The document review method according to claim 1, characterized in that, The step of generating an audit conclusion based on the output variables includes: The output variables include at least one of the following: multi-level nested logical operation results, material integrity identifiers, semantic matching markers, threshold judgment markers, time window judgment markers, set inclusion judgment markers, and intermediate node operation results; The generated review conclusion includes at least one of the following: review passed, review rejected, risk control triggered, materials missing, and transfer to manual review.

8. A document verification device, characterized in that, The device includes: The rule logic orchestration module is used to obtain rule documents, parse the logical description content in the rule documents, convert the logical description content into logical operation expressions, the logical operation expressions include several atomic nodes and operators used to associate each atomic node, and configure an execution strategy for each atomic node. The case information extraction module is used to receive case materials to be reviewed by the large language model, extract information from the case materials to be reviewed, and map the extracted information to the input information of each atomic node. The hybrid atomic decision execution module is used to perform logical decision based on the input information corresponding to each atomic node according to the execution strategy configured for each atomic node, and obtain the logical decision value of each atomic node. The result aggregation module is used to substitute the logical decision value of each atomic node into the logical operation expression to complete the operation and obtain the output variable; The audit conclusion output module is used to generate audit conclusions based on the output variables.

9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, configured to implement the steps of the data review method as described in any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, it implements the steps of the data review method as described in any one of claims 1 to 7.

Images