Bidding compliance auditing method and system based on double-checking mechanism

Abstract

The application discloses a bidding compliance auditing method and system based on a double-checking mechanism. The method comprises the following steps: receiving a bidding document; performing document analysis on the bidding document and generating evidence atoms; constructing an auditing matter evidence chain; performing rule engine checking based on the evidence chain, performing enhanced large language model checking based on the evidence chain, and outputting first and second auditing results; performing multi-dimensional cross-validation on the first and second auditing results, outputting the auditing result when the cross-validation is consistent, identifying the conflict reason when the cross-validation is inconsistent, triggering supplementary evidence extraction according to the identified conflict type, repeatedly performing checking based on the updated evidence chain until the convergence condition is met or the auditing result that needs manual review is generated, and structuring and writing the manually reviewed auditing result into the corresponding checking configuration library to complete the closed-loop calibration. The application can improve the accuracy of bidding compliance auditing.

Description

Technical Field

[0001] This application relates to the field of bidding and tendering review technology, specifically to a bidding and tendering compliance review method and system based on a dual verification mechanism. Background Technology

[0002] Compliance review of bidding and tendering processes is crucial in commercial activities. It involves examining numerous documents, including tender notices, instructions to bidders, qualification requirements, evaluation methods, commercial response forms, technical response forms, tender appendices, contract terms, letters of authorization, security deposit certificates, qualification certificates, and performance records. These documents are lengthy, vary widely in format, contain numerous tables and attachments, and have extensive cross-references between clauses. With the increasing complexity of commercial activities and the continuous development of bidding and tendering practices, higher demands are placed on the accuracy and efficiency of compliance review.

[0003] In existing technologies, the following methods are commonly used to address compliance review issues in bidding and tendering processes: First, a keyword- and rule-engine-based review scheme first extracts text from the document, then configures various rules to automatically determine matters such as the bid security amount and submission deadline. Second, a large language model-based review scheme inputs document fragments into the model, which then determines whether the text contains problems and outputs a natural language explanation. Third, a dual review scheme that simply combines rule engines and large language models, simultaneously applying rule and model judgments to the same document; if the conclusions are inconsistent, it is transferred to manual review.

[0004] However, existing technologies all have limitations. Keyword-based and rule-engine-based solutions require manual rule pre-setting, making it difficult to cover implicit risks, lacking legal basis and evidence chain interpretation, exhibiting poor adaptability to special cases, and making it difficult to trace rule base updates. Large language model-based solutions suffer from unstable conclusions when input context is insufficient, difficulty in binding output to original text and regulations, inferior judgment of certain matters compared to rule engines, and lack of historical sample and rule version recall mechanisms. Simple parallel dual-review solutions lack unified evidence chain constraints and conflict resolution mechanisms. Summary of the Invention

[0005] To improve the accuracy of bidding compliance audits and reduce false alarms and omissions, this application provides a bidding compliance audit method and system based on a dual verification mechanism.

[0006] Firstly, this application provides a method for verifying the compliance of bidding and tendering processes based on a dual verification mechanism, including: Receive bidding documents; The bidding documents are parsed to generate evidence atoms; each evidence atom, as the smallest unit of evidence, includes at least a unique identifier, source, page number, chapter, original text fragment, context, hash value, and timestamp. Construct an evidence chain for audit items; the construction steps include: using audit items as an index, linking the clauses to be audited, the content of the bid response, the legal basis, local regulatory rules, historical review cases and rule versions, so as to support the tracing of each audit conclusion back to the original documents and sources of evidence; Based on the evidence chain, the rule engine performs verification and outputs a first review result; based on the evidence chain, the large language model with enhanced retrieval performs verification and outputs a second review result. The first and second review results are cross-validated from multiple dimensions. When the cross-validation is consistent, the review result is output. When the cross-validation is inconsistent, the cause of the conflict is identified, and supplementary evidence is extracted based on the identified conflict type. The rule engine verification and large language model verification are repeatedly executed based on the updated evidence chain until the convergence condition is met and the review result requiring manual review is output or generated. The manually reviewed review result is then structured and written into the corresponding verification configuration library to complete the closed-loop calibration.

[0007] By adopting the above approach, bidding documents, regulations, historical review cases, and rule versions are constructed into a traceable chain of evidence, serving as common input for the rule engine and the large language model. This avoids the two making incomparable judgments based on different text ranges. Through multi-dimensional cross-validation and conflict cause identification, simple result comparison is upgraded to multi-round cross-validation convergence, identifying rule omissions and reducing mechanical rule false alarms. Through versioned closed-loop calibration of manual review results, similar risks can be automatically optimized in subsequent audits, improving the system's continuous learning and auditability capabilities.

[0008] Preferably, the rule engine verification includes performing deterministic verification based on a preset rule base and the type of audit item to generate a first audit result; The enhanced large language model verification includes performing keyword retrieval, vector retrieval, rule number retrieval, knowledge graph path retrieval, or case similarity retrieval in the legal rule base, local regulatory base, enterprise rule base, historical review case base, and project document evidence base, based on the review items and evidence atoms, to obtain the basis package; and taking the evidence atoms to be reviewed, the associated context, the retrieved basis package, the rule engine's interpretable result summary, and the output format constraints as input to the large language model to generate a second review result.

[0009] By adopting the above scheme, the rule engine can verify deterministic matters such as numerical values, dates, signatures, and qualifications, making up for the instability of the large language model in terms of accurate calculation and format verification. At the same time, the enhanced large language model verification incorporates legal basis, local regulatory rules, and historical review cases into the model context, reducing false alarms caused by the model's arbitrary reasoning and insufficient evidence. The combination of the two can comprehensively and accurately complete the compliance review of bidding.

[0010] Preferably, the conflict types of the cross-validation include: generating different conflict types based on combinations of high-risk conclusions, low-risk conclusions, uncertain conclusions, and conclusions cited as evidence, based on the outputs of the rule engine and the large language model; The convergence conditions include any one of the following: at least two consecutive rounds of cross-validation yield the same risk conclusion; the completeness of the evidence chain reaches a preset threshold; there is no logical contradiction between the rule engine's reference to the rule and the large language model's reference to the rule; the change in overall confidence is less than a preset threshold; the number of cross-validation rounds reaches a preset maximum value; and the conclusion is confirmed by a human reviewer. The overall confidence is obtained through weighted calculation, and the weight coefficients include rule hit strength, model confidence, completeness of the evidence chain, consistency of legal basis, similarity of historical cases, and conflict penalty.

[0011] By adopting the above scheme, the generation method of conflict types in cross-validation is clarified, enabling the system to accurately identify conflicts based on different combinations of output conclusions from the rule engine and the large language model. At the same time, multiple convergence conditions are determined, including consistency of risk conclusions, completeness of the evidence chain, contradictory situations, changes in overall confidence, cross-validation rounds, and manual confirmation. Furthermore, the overall confidence is calculated using weighted averages to ensure that the review process can converge under reasonable conditions, thereby improving the accuracy and reliability of the review.

[0012] Preferred options also include: The constructed evidence chain for audit items is stored in the form of a knowledge graph, generating an evidence network for audit items. The generation steps include: defining the node types of the evidence network; the node types include: evidence atom nodes, entity nodes, legal clause nodes, and audit item nodes; defining the association edges of the evidence network; the association edges represent the relationships between the nodes; and assigning weights to the association edges that reflect the strength and credibility of the relationships between nodes. Based on the stage and risk type of the current audit task, subgraphs are extracted from the evidence network, and structured conditional views for use by the rule engine and semantic evidence path views for use by the large language model are generated respectively, so that the rule engine and the large language model can perform verification based on the generated corresponding views. When supplementary evidence extraction is triggered, the new evidence is added to the evidence network as an incremental evidence atomic node, and the logical relationship of the evidence network is updated; after the review converges and the results are output, the final review conclusion is connected to the relevant entity nodes and legal clause nodes as a historical review node.

[0013] By adopting the above scheme, the evidence network is generated by storing the evidence chain of the audit items in the form of a knowledge graph. The node types and associated edges are clearly defined and weighted, which helps to clearly present the relationships and credibility between the evidence. Subgraphs are extracted and different views are generated according to the audit task stage and risk type, so that the rule engine and large language model can accurately utilize the evidence. The logical relationship of the evidence network is incrementally updated when supplementing evidence, and the final audit conclusion is connected after the audit is completed, which ensures the dynamic updating and integrity of the evidence network, thereby improving the accuracy and traceability of the audit process.

[0014] Preferred options also include: When the cross-validation of the first review result and the second review result is inconsistent, a pre-trained reinforcement learning strategy network is used to select an action from a preset action set according to the current verification state to drive the extraction of supplementary evidence and subsequent re-verification until the convergence condition is met. The current verification status includes at least the consistency of the two audit results, the overlap rate of evidence citations, and the type of conflict identified; the preset action set includes: requesting supplementary evidence of a specified type, re-searching regulations, adjusting rule parameters, and triggering a large language model review; the reinforcement learning strategy network is trained with the correctness of the audit conclusion as a positive reward and the number of verification rounds as a negative reward.

[0015] By adopting the above scheme, when the cross-validation of the first review result and the second review result is inconsistent, the pre-trained reinforcement learning policy network selects actions from the preset action set according to the current verification state, which drives the extraction of supplementary evidence and subsequent re-verification more intelligently, reduces unnecessary verification rounds, improves the efficiency of reaching the convergence condition, and thus improves the review efficiency and accuracy.

[0016] Preferred options also include: Based on the metadata of bidding documents, the execution behavior process is mined, and the third-party review results containing behavioral patterns are output. Before performing multi-dimensional cross-validation on the first audit result and the second audit result, determine whether to include the third audit result in the cross-validation based on the risk type of the current audit matter and / or the risk score of the third audit result; if included, perform multi-dimensional cross-validation on the first audit result, the second audit result and the third audit result, and use the consistency of the three audit results as one of the criteria for convergence judgment.

[0017] By adopting the above scheme, behavioral process mining is performed based on the metadata of bidding documents to obtain the third review result. The risk type of the review item and the risk score of the third review result are used to determine whether to include it in cross-validation, thereby supplementing the review information from the perspective of bidding behavior process and improving the accuracy of the review. At the same time, the consistency of the three review results is used as one of the convergence judgment criteria to improve the reliability of the convergence judgment.

[0018] Preferred options also include: The audit results of manual review are structured and written into the corresponding verification configuration library, including the generated result tags; Closed-loop calibration generates suggestions based on result label generation rules, rule exception suggestions, prompt template optimization suggestions, threshold adjustment suggestions, or retrieval index update suggestions, and synchronously updates and records the calibration version number.

[0019] By adopting the above scheme, the audit results of manual review are structured and labeled, various calibration suggestions are generated based on the labels, and the calibration version number is updated and recorded at the same time, so as to realize the version-based closed-loop calibration of the system and improve the system's automatic optimization capability and auditability for similar risks.

[0020] Secondly, this application provides a bidding compliance review system based on a dual verification mechanism, including: a document acquisition module for receiving bidding documents; The document parsing module is used to parse bidding documents and generate evidence atoms; the evidence atom, as the smallest unit of evidence, includes at least a unique identifier, source, page number, chapter, original text fragment, context, hash value, and timestamp. The evidence chain construction module is used to build the evidence chain for the audit items. The construction steps include: using the audit items as an index, associating the clauses to be audited, the content of the bid response, the legal basis, local regulatory rules, historical review cases and rule versions, so as to support the tracing of each audit conclusion back to the original documents and sources of evidence. The rule engine verification module is used to perform rule engine verification based on the evidence chain and output the first audit result; The retrieval-enhanced large language model verification module is used to perform retrieval-enhanced large language model verification based on the evidence chain and output a second verification result. The cross-validation and result output module is used to perform multi-dimensional cross-validation on the first and second audit results. When the cross-validation is consistent, the audit result is output. When the cross-validation is inconsistent, the conflict cause is identified, and supplementary evidence is extracted based on the identified conflict type. The rule engine verification and large language model verification are repeatedly executed based on the updated evidence chain until the convergence condition is met and the audit result requiring manual review is output or generated. The manually reviewed audit result is then structured and written into the corresponding verification configuration library to complete the closed-loop calibration.

[0021] By adopting the above solution, the receipt, parsing, evidence chain construction, rule engine verification, and large language model verification of bidding documents can be realized. The audit results are obtained through multi-dimensional cross-validation and conflict handling, and the closed-loop calibration of the manual review results is completed. This improves the accuracy, consistency, and traceability of bidding compliance audits, effectively reduces false alarms and omissions, and enhances continuous learning and auditability capabilities.

[0022] Thirdly, this application provides a computer-readable storage medium including a stored computer program, wherein the computer program, when running, controls the device where the computer-readable storage medium is located to perform the method described above.

[0023] Fourthly, this application provides a computer device, the computer device including a memory, a processor and a program stored in the memory and executable thereon, the program being executed by the processor to implement the steps of the method described above.

[0024] In summary, this application has the following beneficial effects: 1. By receiving bidding documents, parsing and generating evidence atoms and constructing evidence chains, the system performs rule engine and large language model verification based on these, followed by multi-dimensional cross-validation. In case of inconsistencies, conflicts are identified, supplementary evidence is added for re-verification, and finally closed-loop calibration is performed to improve the accuracy of audit results and reduce false positives and false negatives. The system also sets the rule engine to perform deterministic verification, the large language model to enhance output results through retrieval, and various convergence conditions. Furthermore, the system writes the results of manual review into the configuration library in a structured manner to optimize continuous learning and auditability. 2. Construct an evidence chain for review items and store it in a knowledge graph to generate an evidence network. Subgraphs can be extracted for use by the rule engine and the large language model. The evidence network is updated when supplementary evidence is provided. When the verification results are inconsistent, reinforcement learning strategy network selection drives supplementary evidence and re-verification. This enables the rule engine and the large language model to perform verification based on more complete evidence, thereby improving the accuracy and efficiency of the review. 3. Based on the metadata of the bidding documents, the behavior process is mined to output the third review results. Before cross-validation, the risk type and the risk score of the third review results can be used to determine whether to include them in the cross-validation. The consistency of the three review results is used as one of the criteria for convergence judgment, which further improves the accuracy and reliability of the review and makes the review process more comprehensive and scientific. Attached Figure Description

[0025] Figure 1 A schematic diagram of the overall architecture of the bidding compliance review method based on the dual verification mechanism described in the specific embodiment; Figure 2 This is a flowchart of the bidding compliance review method based on a dual verification mechanism as described in a specific embodiment; Figure 3 This is a schematic diagram illustrating the construction of the evidence chain in the bidding compliance review method based on a dual verification mechanism as described in a specific embodiment. Figure 4 This is a schematic diagram of the evidence atom and audit result data structure in the bidding compliance review method based on the dual verification mechanism described in a specific embodiment; Figure 5 This is a flowchart illustrating the conflict cause identification and automatic correction process in the bidding compliance review method based on a dual verification mechanism as described in a specific embodiment. Figure 6 This is a state transition diagram of multiple rounds of cross-validation in the bidding compliance review method based on dual verification mechanism described in the specific embodiment; Figure 7 This is a diagram illustrating the manual review feedback and version-based closed-loop calibration in the bidding compliance review method based on a dual verification mechanism described in a specific embodiment. Figure 8 This is a schematic diagram of the bidding compliance review system based on a dual verification mechanism as described in a specific embodiment. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0027] The overall design concept of this application is as follows: Figure 1 As shown, this paper presents a method for improving the accuracy of bidding compliance review based on a dual verification mechanism. This method does not simply superimpose keyword rules, a large language model, and manual review; instead, it uses a traceable evidence chain as a common input, ensuring that both the rule engine verification channel and the retrieval-enhanced large language model verification channel output review results based on the same evidence boundary. Subsequently, a cross-validation module performs consistency verification in dimensions such as risk conclusions, cited evidence, legal basis, risk type, and confidence level. When the results of the two channels are inconsistent, the cause of the conflict is first identified, and then at least one of the following actions is automatically executed: supplementary evidence extraction, re-retrieval of legal basis, rule engine recalculation, and large language model review. Cross-validation is then repeated based on the updated evidence chain until a preset convergence condition is met or a review result requiring manual review is generated. The manual review results are further written into the rule base, rule exception base, prompt template base, retrieval index base, threshold parameter base, and historical case sample pool, forming a versioned closed-loop calibration. A specific embodiment is used to illustrate the method below.

[0028] like Figure 2 As shown in the figure, this application discloses a bidding compliance review method based on a dual verification mechanism, including: document reception, evidence atom generation, evidence chain construction, rule engine verification, large language model verification, cross-validation, conflict handling, and closed-loop calibration. By adopting a dual verification mechanism, the accuracy of bidding compliance review is improved, thereby reducing false alarms and omissions in compliance review and forming a closed-loop calibration effect. The following is a further detailed description of this application.

[0029] S1. Receive bidding documents.

[0030] Specifically, the document receiving step is equivalent to accepting the review task and is the starting point of the entire review process. Bidding documents can be received via network interfaces, storage devices, etc. Bidding documents are quite diverse, including files directly transmitted from the bidding party's system or stored on a local server. The received bidding documents cover various types, such as tender announcements, instructions to bidders, and tender documents. In practical applications, to ensure the stability and accuracy of document reception, methods such as file format verification and file integrity checks can be used. For example, for received files, check whether they are in the specified format, such as PDF or DOCX, and also check whether the file size and number of bytes meet expectations.

[0031] S2. Parse the bidding documents and generate evidence atoms.

[0032] Specifically, document parsing of bidding documents can employ natural language processing technology to identify chapters, tables, attachments, and other content within the documents. For example, it can identify chapters such as the bidding announcement, instructions to bidders, qualification requirements, evaluation methods, contract terms, bid appendices, commercial response forms, and technical response forms. For tables, the system records row and column coordinates, table headers, cell values, and page numbers. For main text clauses, the system records chapter numbers, original text snippets, context, and page positions. For attachments and certificates, the system records file names, certificate types, validity periods, and identification confidence levels.

[0033] The parsed content is broken down into the smallest units of evidence, including source, page number, chapter, original text, context, and hash value, generating evidence atoms. Each evidence atom includes at least a unique identifier, source, page number, chapter, original text fragment, context, hash value, and timestamp. The unique identifier can be generated using UUIDs or similar methods to ensure the uniqueness of each evidence atom. The hash value is used to verify the integrity of the evidence atom and prevent data tampering. The timestamp records the time the evidence atom was generated, facilitating subsequent traceability and management. Multi-threading can be used to generate evidence atoms to improve processing efficiency.

[0034] S3. Construct a chain of evidence for the audit items.

[0035] Specifically, the evidence chain construction process uses the review items as an index to link the clauses to be reviewed, the content of the bid response, the legal basis, local regulatory rules, historical review cases, and rule versions. The construction of the evidence chain allows each review conclusion to be traced back to the original documents and sources of evidence, improving the traceability of the review results. During the construction of the evidence chain, the data can be updated and maintained regularly to ensure its accuracy and completeness. The evidence chain can also be stored using relational databases, graph databases, knowledge graphs, or other methods.

[0036] like Figure 3 As shown, for the audit item Q, the evidence includes the tender conditions and terms (E_bid), the tender response (E_tender), the legal basis (k_law), the local regulation (k_local), historical cases (C_case), and the rule version (R_ver). An evidence index chain is generated: E_bid-E_tender-k_law-k_law-C_case-R_ver, thus forming a traceable evidence package for use by both the rule engine and the large language model.

[0037] S4. Based on the evidence chain, execute the rule engine to verify and output the first audit result.

[0038] Specifically, the rule engine verification steps include: performing deterministic verification based on a preset rule base and the type of audit item. The rules stored in the preset rule base can be expressed using rule expressions, decision tables, condition trees, scripted rules, or visual rule configuration methods. Rule content includes, but is not limited to, margin ratios, submission deadlines, certificate validity periods, qualification levels, years of performance experience, consortium requirements, completeness of signatures, required fields for each chapter, format consistency, and rules regarding prohibited and restricted words.

[0039] The rule engine verification includes deterministic checks such as numerical verification, date verification, qualification level verification, signature integrity verification, chapter consistency verification, and prohibited / restricted word verification. To further ensure the accuracy of rule verification, the content of the rule base can be dynamically adjusted according to the project type and regulatory requirements of different audit items during the rule engine verification process.

[0040] like Figure 4 As shown, the rule engine outputs the first review result. The first review result includes: review item number, matched rule number, rule version, matched evidence atoms, risk level, rule confidence level, rule application conditions, and rule explanation text. For items that do not match explicit rules but have candidate risks, the rule engine can output a low-risk or uncertain status for further processing through cross-validation.

[0041] S5. Perform enhanced large language model verification based on the evidence chain and output the second audit result.

[0042] Specifically, the enhanced large language model verification steps include: based on the audit items and evidence atoms, performing keyword retrieval, vector retrieval, rule number retrieval, knowledge graph path retrieval, or case similarity retrieval in the legal rule base, local regulatory base, enterprise rule base, historical review case base, and project document evidence base to obtain the evidence package related to the current audit items.

[0043] The second review result is generated by taking the atomic evidence to be reviewed, the associated context, the retrieved evidence package, similar historical cases, the rule engine's interpretable result summary, and the output format constraints as input to the large language model. Figure 4 As shown, the second review result should include at least the risk conclusion, cited evidence, cited basis, reasoning summary, risk type, model confidence level, and reasons for uncertainty.

[0044] By enhancing constraints through retrieval, model judgments are confined to traceable evidence and legal basis, thereby reducing the risk of arbitrary reasoning, omitting key context, or neglecting applicable exceptions. When selecting a large language model, one can choose a locally deployed model, a private model, an industry-specific model, a general model interface, or a combination of multiple models, depending on actual needs. These large language models are pre-trained using historical input data and output audit data.

[0045] S6. Perform multi-dimensional cross-validation, and perform supplementary evidence extraction, repeated validation, and convergence determination based on the cross-validation results.

[0046] Specifically, the multi-dimensional cross-validation step involves comparing the first and second audit results across multiple dimensions. In this embodiment, the comparison dimensions include: whether the risk conclusions are consistent, whether the risk types are consistent, whether the cited evidence covers the same chain of evidence, whether the legal basis is consistent, whether the confidence level difference exceeds a threshold, whether there are contradictory clauses, and whether there are supporting historical review samples.

[0047] When the multi-dimensional comparison results of the two are consistent, the audit result is output, including: there is a risk of violation or there is no risk of violation. For example, when both audit results output high risk and the evidence chain is consistent, the cross-validation result is that there is a confirmed risk of violation.

[0048] When the multi-dimensional comparison results of the two are inconsistent, the cause of the conflict is identified, and supplementary evidence is extracted based on the identified conflict type. The rule engine verification and large language model verification are repeatedly executed based on the updated evidence chain until the convergence condition is met and output or generate an audit result that requires manual review.

[0049] In the process of conflict cause identification, a cross-validation matrix can be constructed based on the rule engine, the big oracle model, and the evidence chain to identify and determine the specific conflict type, and trigger supplementary evidence extraction based on the identified conflict type. Table 1 below shows a partial cross-validation matrix of rules, models, and evidence chains: Table 1 .

[0050] like Figure 5 As shown, different conflict types are obtained by combining high-risk conclusions, low-risk conclusions, or model uncertainty conclusions, as well as cited evidence conclusions, based on the output of the rule engine and the large language model. According to different conflict types, the completeness of the evidence chain, the consistency of legal basis, the similarity of historical review samples, and the confidence difference are referenced simultaneously to complete the pre-configured appropriate evidence supplementation strategy, thereby completing the extraction of supplementary evidence.

[0051] For example: If the rule engine outputs low risk but the large model outputs high risk, it is judged as a suspected missed report. For suspected missed reports, the strategy for obtaining appropriate supplementary evidence includes: prioritizing the search for regulations related to implicit restrictions, similar historical cases, and related contextual chapters. If the rule engine outputs high risk but the large model outputs low risk, it is judged as a suspected false positive. For suspected missed reports, the strategy for obtaining appropriate supplementary evidence includes: prioritizing the search for exception conditions, project scale, technical complexity, regulatory scope, and historical rejection samples. If both the rule engine output and the large model output are high risk, and the evidence cited by the two is different, it is judged as evidence discrepancy. For evidence discrepancy, the strategy for obtaining appropriate supplementary evidence includes: re-extracting preceding and following paragraphs, same-level chapters, table footnotes, and attachment content. If both the rule engine output and the large model output are risk uncertain, and the evidence cited by both is insufficient, it is judged as insufficient evidence. For insufficient evidence, the system expands the search scope and marks the missing evidence types that require manual review.

[0052] In this process, after supplementary evidence is added to the new evidence chain, the rule engine recalculates the verification based on the updated evidence, the large language model re-infers and verifies based on the updated retrieval-enhanced evidence, and then the two review results are compared again. This process is repeated until the preset convergence condition is met or a review result requiring manual review is generated.

[0053] Specifically, the convergence condition can be set to include any of the following: at least two consecutive rounds of cross-validation yield the same risk conclusion; the completeness of the evidence chain reaches a preset threshold; there is no logical contradiction between the rule engine's cited rules and the large language model's cited rules; the change in overall confidence is less than a preset threshold; the number of cross-validation rounds reaches a preset maximum value; and the conclusion is confirmed by a human reviewer. The overall confidence is obtained through weighted calculation, using the following formula: ; In the formula, w1 to w6 are configurable weight coefficients that can be adjusted according to project type, regulatory scope and historical review results, corresponding to rule hit strength, model confidence, evidence chain completeness, consistency of legal basis, similarity of historical cases and conflict penalty items, respectively.

[0054] The updated evidence chain is used to repeatedly execute rule engine verification and large language model verification until the convergence condition is met to output or generate audit results that require manual review. The manually reviewed audit results are then structured and written into the corresponding verification configuration library to complete the closed-loop calibration.

[0055] S7. Determine whether the cross-validation has converged based on the results, and decide whether to proceed to the next round of validation based on the convergence result.

[0056] Specifically, if the current round of cross-validation results in non-convergence, then conflict identification, correction, and re-search are performed, repeating steps S3-S6 to complete the next round of validation, and the convergence result of the next round is output. If the current round of cross-validation results in convergence, the convergence result is output.

[0057] S8. Based on the convergence result, output the audit result or the manual review result.

[0058] Specifically, when the preset convergence conditions are met, the final audit result is output, a violation risk report for the current audit item is generated, and the process stops proceeding to the next step. If the preset number of supplementary verifications is reached and the convergence conditions are still not met, a manual review request is generated, and the manual review result is obtained, along with the reviewer's confirmation conclusion. Figure 6 As shown.

[0059] S9. Write the manual verification results into the calibration library.

[0060] Specifically, after obtaining the audit results from manual review, closed-loop calibration is performed. The closed-loop calibration step involves structuring the audit results from manual review and writing them into the corresponding verification configuration library. For example, the manual review results are structured into result labels such as confirmed risk, rejection risk, reasonable exception, insufficient evidence, missing rules, model misjudgment, and extraction error.

[0061] Closed-loop calibration generates suggestions based on result label generation rules, rule exception suggestions, prompt template optimization suggestions, threshold adjustment suggestions, or retrieval index update suggestions, and synchronously updates and records the calibration version number.

[0062] like Figure 7 As shown, to ensure system auditability, all updates record the version number, applicable project type, approver, release time, grayscale range, rollback flag, and associated review sample. Subsequent review tasks using the new configuration can use the version number to explain differences in review results.

[0063] In a specific embodiment, the difference from the above embodiments lies in that the constructed evidence chain for audit matters is stored in the form of a knowledge graph, generating an evidence network for audit matters. This more intuitively displays the relationships between various elements, facilitating knowledge reasoning and querying, and improving the accuracy and efficiency of verification. The method further includes: The constructed evidence chain for audit items is stored in the form of a knowledge graph, generating an evidence network for audit items. Specifically, the steps for generating the evidence network for audit items include: First, define the node types of the evidence network. These node types include: evidence atom nodes, entity nodes, legal clause nodes, and review item nodes. Evidence atom nodes are the smallest units of evidence, carrying attributes such as the original text, highlighted position, document source, timestamp, and initial credibility score. Entity nodes are standardized entities extracted from atoms, and are strongly correlated with the external knowledge graph. Legal clause nodes contain the full text of the clause, its effective date, scope of application, and basis for penalties; they can be connected to evidence atoms and risk conclusions via "support / violation" edges. Review item nodes are specific review checkpoints, such as "project managers must not have projects under construction," serving as convergence points for evidence combinations.

[0064] Define the relational edges of the evidence network. These edges represent the relationships between nodes, including: Constitutive edges (evidence atom-entity): representing the entities mentioned in the atom; Relationship edges (entity-entity): such as "same social security payment unit" or "shareholding relationship," with weights indicating the strength of the relationship; Logical edges (set of evidence atoms-review item): such as "Evidence Atom A (project manager's name) + Evidence Atom B (list of projects under construction) + Evidence Atom C (letter of commitment)" jointly supporting the determination "the project manager has no projects under construction," and recording the logical combination function (AND / OR / weight); Legal citation edges (review item / evidence → legal clause): marking the specific applicable legal provisions.

[0065] Weights are assigned to the edges that reflect the strength and credibility of the relationships between nodes. These edge weights can be set as the product of a base weight, evidence credibility, a time decay factor, and a spatial relevance factor. The base weight can be pre-set based on the relationship type; evidence credibility is derived from the credibility score of the evidence atoms; the time decay factor decreases the weight for relationships with high time sensitivity; and the spatial relevance factor increases the weight for spatially related relationships with closer proximity.

[0066] Secondly, based on the stage and risk type of the current audit item, subgraphs are extracted from the evidence network. The current audit item can be divided into audit stages according to the entire bidding process: bidding stage audit, tendering stage audit, bid opening stage audit, bid evaluation stage audit, and bid awarding stage audit; thus determining the stage of the audit task corresponding to the current audit item; the risk type of the current audit item can be classified according to typical risk types in the bidding process, such as: qualification compliance type, technical compliance type, bid rigging type, and price reasonableness type. Therefore, corresponding subgraphs can be extracted from the evidence network based on the stage and risk type, completing subgraph pruning.

[0067] Then, a structured conditional view for the rule engine and a semantic evidence path view for the large language model are generated, respectively, so that the rule engine and the large language model can perform verification based on the generated corresponding views. Specifically, to facilitate verification between the rule engine and the large language model, a structured conditional view for the rule engine is generated: the edge logic in the extracted subgraph is logically mapped to decision tables or rule fact objects. The rule engine only needs to calculate based on the fact objects, and the result is then written back to the risk conclusion node of the evidence network. A semantic evidence path view for the large language model is generated: it is output in the form of a natural language evidence outline. Text is generated around the matters to be reviewed according to the structure of "claim (review matter) - evidence chain - legal citation". In addition, a unified graph query language interface is provided, allowing the rule engine to query the evidence status when conditions match, and also allowing the LLM to actively call functions to obtain extended evidence during the reasoning process.

[0068] Finally, based on the above verification, the audit results are obtained, the audit results are cross-validated and supplementary evidence is triggered. That is, when supplementary evidence is triggered, the new evidence is added to the evidence network as an incremental evidence atomic node and the logical relationship of the evidence network is updated. After the audit converges and the results are output, the final audit conclusion is connected to the relevant entity nodes and legal clause nodes as a historical audit node.

[0069] A specific embodiment, differing from the above embodiments, utilizes a reinforcement learning policy network to automatically select appropriate actions based on the current verification state, enabling more intelligent conflict handling and evidence supplementation. This allows for adaptive responses to different conflict situations, further optimizing the review process. The method also includes: When the cross-validation results of the first and second audits are inconsistent, a pre-trained reinforcement learning strategy network selects an action from a preset action set based on the current verification state to drive the extraction of supplementary evidence and subsequent re-verification until the convergence condition is met.

[0070] The pre-constructed reinforcement learning policy network includes the following steps: First, defining the state: the current verification state, including the consistency of the two review results, the overlap rate of evidence citations, and the type of conflict identified. Second, defining actions: a pre-set set of actions, including supplementary evidence, re-retrieval, and recalculation, such as requesting supplementary evidence of a specified type, re-retrieval of regulations, adjusting rule parameters, and triggering a large language model review. Third, defining rewards: guided by convergence efficiency and accuracy, including training with positive rewards for the correctness of the review conclusion and negative rewards for the number of verification rounds.

[0071] Specifically, the first round of data acquisition involves obtaining the first review results (risk level, triggering rule, cited evidence ID, and regulatory clause) and the second review results (risk level, confidence level, argument summary, cited evidence ID, and regulatory clause) from the rule engine and large model, and then calculating the initial verification status. This includes: consistency of conclusions (degree of disagreement), overlap rate of evidence citations, and preliminary classification of conflict types, if the initial verification status... If the audit conclusion is not reached, the audit conclusion will be output directly; otherwise, the RL control loop will be entered.

[0072] When entering a loop verification, the initial verification state will be changed. The encoding is used as the input vector for the reinforcement learning policy network, and the policy network outputs an action probability distribution. Inactive actions are masked using an action mask, and the action with the highest probability is selected. Based on the action type, the system automatically supplements evidence, updates the evidence network, re-searches regulations, or adjusts parameters; for affected audit items, it re-executes rule engine verification and LLM verification to obtain updated audit conclusions; based on the updated conclusions, it calculates the new status. Calculate instant rewards If the evidence gap narrows, add +x1 (e.g., 0.2); if convergence occurs directly, add +x2 (e.g., 10); if the action does not substantially improve the situation, add -x3 (e.g., 0.5). If convergence is achieved, the process terminates and outputs the conclusion; if convergence is not achieved and the maximum number of rounds has not been reached, the tuple ( Store it in the experience pool, and let As the current state, the reinforcement learning policy network is re-inputted, and the steps of action selection, action execution, evidence update and re-verification, re-verification and new state generation are repeated; if convergence is still not achieved after the maximum number of rounds, manual intervention is forced.

[0073] A specific embodiment, differing from the above embodiments, involves: mining the behavioral flow based on the metadata of the bidding documents to output a third review result containing behavioral patterns; and considering multi-dimensional cross-validation of the three review results to make the review process more flexible and comprehensive, improve the accuracy and reliability of the review results, and reduce false positives and false negatives; the method further includes: First, based on the metadata of the bidding documents, the execution behavior process is mined, and the third review result containing the behavior pattern is output.

[0074] Specifically, behavioral feature vectors are extracted from the bidding metadata event logs, including: individual anomalies: the number of times the bid documents were modified, the proximity of the upload time to the deadline, the concentration of banks issuing the guarantee, etc.; and relationship anomalies: different bidders sharing IP / MAC addresses, regular intervals in the upload time series, and the cosine similarity of the bid composition. These features are then input into a pre-trained risk prediction model to obtain the final risk prediction result as the third review result. For example, the pre-trained risk prediction model may employ a collusion risk prediction model to predict and output a review result indicating the presence of collusion risk and a risk score.

[0075] Secondly, before performing multi-dimensional cross-validation of the first audit result and the second audit result, it is determined whether to include the third audit result in the cross-validation based on the risk type of the current audit matter and / or the risk score of the third audit result.

[0076] Considering that not all audit items have valid outputs of third-party audit results, third-party audit results that generally include behavioral patterns are specialized in bid rigging and collusion, and may have no signal or low signal confidence for scenarios such as pure qualification document compliance and technical solution review. Therefore, for the risk type of the audit item and / or the risk score combination of the third-party audit results, an option to include the third-party audit results in cross-validation is pre-configured; for example, if the risk category of the audit item is bid rigging, then regardless of whether the risk score of the third-party audit result is lower than the preset risk score, it is set to participate in validation. If the risk category of the audit item is pricing reasonableness, then if the risk score of the third-party audit result is not lower than the preset risk score, it is included in validation.

[0077] Then, when inclusion, the first review result, the second review result, and the third review result are cross-validated in multiple dimensions, and whether the three review results are consistent is used as one of the criteria for convergence judgment.

[0078] Specifically, the first, second, and third review results are compared across multiple dimensions. If the multi-dimensional comparison results of the first, second, and third review results are consistent, the review result is output. If the multi-dimensional comparison results of the three are inconsistent, the cause of the conflict is identified, and supplementary evidence is extracted based on the identified conflict type. The rule engine verification and large language model verification are repeatedly executed based on the updated evidence chain until the convergence condition is met and a review result requiring manual review is output.

[0079] like Figure 8As shown, this application discloses a bidding compliance review system based on a dual verification mechanism, which can be implemented by a server, cloud platform, private deployment platform, or enterprise internal review system. The system specifically includes: a file receiving module 100, a file parsing and evidence atom generation module 200 (including a document parsing module 210 and an evidence atom generation module 220), an evidence chain construction module 300, a rule engine verification module 400, a retrieval-enhanced large language model verification module 500, and a cross-validation and result output module 600 (including a cross-validation module 610, a conflict cause identification module 620, a supplementary evidence extraction module 630, a convergence determination module 640, a risk report output module 650, and a versioned closed-loop calibration module 660). Details are as follows: The document acquisition module 100 is used to receive bidding documents. The document parsing and evidence atom generation module 200 is used to parse bidding documents and generate evidence atoms. Specifically, it includes a document parsing module 210 and an evidence atom generation module 220. The document parsing module 210 is used to parse the document, identify chapters, tables, attachments, page numbers, fields, and values, and generate structured parsing results. The evidence atom generation module 220 is used to break down the document content into the smallest evidence units containing source, page number, chapter, original text, context, and hash value.

[0080] The evidence chain construction module 300 is used to construct the evidence chain for audit items. The construction steps include: using the audit items as an index, associating the clauses to be audited, the content of the bid response, the legal basis, the local regulatory rules, the historical review cases and the rule versions, so as to support the tracing of each audit conclusion back to the original documents and sources of evidence.

[0081] The rule engine verification module 400 is used to perform rule engine verification based on the evidence chain and output the first audit result.

[0082] The retrieval-enhanced large language model verification module 500 is used to perform retrieval-enhanced large language model verification based on the evidence chain and output a second verification result.

[0083] The cross-validation and result output module 600 is used to perform cross-validation between the first audit result output by the rule engine and the second audit result output by the large model, output the validation result, and output the audit result or the audit result of manual review; and to write the versioned manual review result into the rule base, prompt template, threshold parameter and case sample pool.

[0084] Specifically, the cross-validation module 610 is used to perform multi-dimensional cross-validation on the first and second audit results. When the cross-validation is consistent, the audit result is output and a risk report is generated and sent to the risk report output module. When the cross-validation is inconsistent, the conflict cause identification module is invoked. The conflict cause identification module 620 is used to identify the conflict cause when the cross-validation is inconsistent and input the identification result into the supplementary evidence extraction module. The supplementary evidence extraction module 630 is used to trigger supplementary evidence extraction according to the identified conflict type and input the extracted supplementary evidence into the convergence determination module. The convergence determination module 640 is used to repeatedly execute rule engine verification and large language model verification based on the updated evidence chain until the convergence condition is met and an audit result requiring manual review is output or generated. The risk report output module 650 is used to output a risk report. The versioned closed-loop calibration module 660 is used to structure the manually reviewed audit result and write it into the corresponding verification configuration library to complete the closed-loop calibration.

[0085] The following will demonstrate a real-time example of performing dual verification on bidding documents by adopting the series of steps described in this embodiment and leveraging the advantages of rule engines and large language models. Specifically, it includes: Case Study 1: Handling Suspected Omissions of Implicit Restrictive Clauses. Upon receiving the tender documents and bids for an engineering project, the document parsing module identifies sections such as "Instructions to Bidders," "Qualification Review Conditions," "Business Response Form," and "Evaluation Methods," and extracts evidence atoms such as "Experience in Similar Projects in the Region," "Local Service Response Capability Scoring Items," and "Similar Performance in the Past Three Years." In the first round of review, the rule engine did not hit explicit rules such as "limited to local enterprises." The large language model, based on enhanced evidence from retrieval, judged that this combination might constitute an implicit regional restriction. The cross-validation module marked it as a suspected omission, triggering supplementary searches of adjacent sections, evaluation methods, and regulatory basis. In the second round of review, the rule engine hit non-essential regional condition candidate rules, but the large language model still output high risk, and the system generated high-priority manual review items accordingly.

[0086] Case Study 2: Handling Suspected False Alarms in Qualification Level Rules. A project's rule engine identified a high-risk scenario based on "qualification level exceeding typical project size requirements." The large language model, within the enhanced retrieval context, identified that the project involved special processes, safety levels, and local regulatory permits, and retrieved evidence allowing for higher qualification requirements for specific projects. The cross-validation module identified this as a suspected false alarm, supplemented the retrieval with information on project size, technical complexity, and regulatory scope, and output reasonable exception candidates. After manual verification, this sample was added to the rule exception library for that specific project type.

[0087] Case Study 3: Handling the Risk of Inconsistent Deadlines. For the same project, the tender notice, the instructions to bidders, and the appendix to the tender letter all contain different bid deadlines. The rule engine identified three inconsistencies using date field consistency rules; the large language model explained, based on the chain of evidence, that these inconsistencies might affect the bidders' response deadlines and the bid opening arrangements, and that the evidence cited by both documents overlapped. The cross-validation module determined this to be a confirmed risk and directly output a risk report.

[0088] Case Study 4: Handling Evidence Disagreements Regarding Designated Brand Clauses. A tender document contained the phrase "Use Brand A or equivalent products." The rule engine flagged a designated brand risk due to the presence of the brand name; the large language model, however, considered the phrase "or equivalent products" to be insufficient and potentially not constituting absolute exclusivity. The cross-validation module identified this as an evidence disagreement and further retrieved technical parameters, alternative solutions, and scoring details, rather than directly classifying it as a risk or false alarm.

[0089] Case Study 5: Closed-Loop Calibration Driven by Manual Review Results. The system outputs 100 risk items for a batch of projects. Manual review confirms 60 as genuine risks, 25 as false alarms, 10 as reasonable exceptions, and 5 as insufficient evidence. The versioned closed-loop calibration module structurally attributes the review results according to risk type and conflict cause, generating suggestions for rule revisions, rule exceptions, prompt template optimizations, search index updates, and threshold adjustments.

[0090] This application also discloses a computer-readable storage medium.

[0091] Specifically, the computer-readable storage medium stores a computer program that can be loaded by a processor and executed, such as the bidding compliance review method based on the dual verification mechanism described above. The computer-readable storage medium includes, for example, various media that can store program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0092] This application also discloses a computer device.

[0093] Specifically, the computer device includes a memory and a processor, and the memory stores a computer program that can be loaded by the processor and executed to perform the bidding compliance audit method based on the dual verification mechanism described above.

[0094] The above are all preferred embodiments of this application and are not intended to limit the scope of protection of this application. Any feature disclosed in this specification (including the abstract and drawings) may be replaced by other equivalent or similar features unless specifically stated otherwise. That is, unless specifically stated otherwise, each feature is only one example of a series of equivalent or similar features.

Claims

1. A method for reviewing the compliance of bidding and tendering processes based on a dual verification mechanism, characterized in that, include: Receive bidding documents; The bidding documents are parsed and evidence atoms are generated. The evidence atom, as the smallest unit of evidence, includes at least a unique identifier, source, page number, chapter, original text fragment, context, hash value, and timestamp; Construct an evidence chain for audit items; the construction steps include: using audit items as an index, linking the clauses to be audited, the content of the bid response, the legal basis, local regulatory rules, historical review cases and rule versions, so as to support the tracing of each audit conclusion back to the original documents and sources of evidence; Based on the evidence chain, the rule engine performs verification and outputs a first review result; based on the evidence chain, the large language model with enhanced retrieval performs verification and outputs a second review result. The first and second review results are cross-validated from multiple dimensions. When the cross-validation is consistent, the review result is output. When the cross-validation is inconsistent, the cause of the conflict is identified, and supplementary evidence is extracted based on the identified conflict type. The rule engine verification and large language model verification are repeatedly executed based on the updated evidence chain until the convergence condition is met and the review result requiring manual review is output or generated. The manually reviewed review result is then structured and written into the corresponding verification configuration library to complete the closed-loop calibration.

2. The bidding compliance review method based on a dual verification mechanism according to claim 1, characterized in that, The rule engine verification includes performing deterministic verification based on a preset rule base and the type of audit item, and generating a first audit result; The enhanced large language model verification includes performing keyword retrieval, vector retrieval, rule number retrieval, knowledge graph path retrieval, or case similarity retrieval in the legal rule base, local regulatory base, enterprise rule base, historical review case base, and project document evidence base, based on the review items and evidence atoms, to obtain the basis package; and taking the evidence atoms to be reviewed, the associated context, the retrieved basis package, the rule engine's interpretable result summary, and the output format constraints as input to the large language model to generate a second review result.

3. The bidding compliance review method based on a dual verification mechanism according to claim 1, characterized in that, The conflict types of the cross-validation include: different conflict types generated by combining high-risk conclusions, low-risk conclusions, uncertain conclusions, and conclusions based on cited evidence, based on the outputs of the rule engine and the large language model; The convergence conditions include any one of the following: at least two consecutive rounds of cross-validation yield the same risk conclusion; the completeness of the evidence chain reaches a preset threshold; there is no logical contradiction between the rule engine's reference to the rule and the large language model's reference to the rule; the change in overall confidence is less than a preset threshold; the number of cross-validation rounds reaches a preset maximum value; and the conclusion is confirmed by a human reviewer. The overall confidence is obtained through weighted calculation, and the weight coefficients include rule hit strength, model confidence, completeness of the evidence chain, consistency of legal basis, similarity of historical cases, and conflict penalty.

4. The bidding compliance review method based on a dual verification mechanism according to claim 1, characterized in that, Also includes: The constructed evidence chain for audit items is stored in the form of a knowledge graph, generating an evidence network for audit items; The generation steps include: defining the node types of the evidence network; the node types include: evidence atom nodes, entity nodes, legal clause nodes, and review item nodes; defining the association edges of the evidence network; the association edges represent the relationships between the nodes; and assigning weights to the association edges that reflect the strength and credibility of the relationships between nodes. Based on the stage and risk type of the current audit task, subgraphs are extracted from the evidence network, and structured conditional views for use by the rule engine and semantic evidence path views for use by the large language model are generated respectively, so that the rule engine and the large language model can perform verification based on the generated corresponding views. When supplementary evidence extraction is triggered, the new evidence is added to the evidence network as an incremental evidence atomic node, and the logical relationship of the evidence network is updated; after the review converges and the results are output, the final review conclusion is connected to the relevant entity nodes and legal clause nodes as a historical review node.

5. The bidding compliance review method based on a dual verification mechanism according to claim 1, characterized in that, Also includes: When the cross-validation of the first review result and the second review result is inconsistent, a pre-trained reinforcement learning strategy network is used to select an action from a preset action set according to the current verification state to drive the extraction of supplementary evidence and subsequent re-verification until the convergence condition is met. The current verification status includes at least the consistency of the two review results, the overlap rate of evidence citations, and the type of conflict identified; the preset action set includes: requesting supplementary evidence of a specified type, re-searching regulations, adjusting rule parameters, and triggering a large language model review; The reinforcement learning strategy network is trained with positive rewards for the correctness of the audit conclusion and negative rewards for the number of verification rounds.

6. The bidding compliance review method based on a dual verification mechanism according to claim 1, characterized in that, Also includes: Based on the metadata of bidding documents, the execution behavior process is mined, and the third-party review results containing behavioral patterns are output. Before performing multi-dimensional cross-validation on the first audit result and the second audit result, determine whether to include the third audit result in the cross-validation based on the risk type of the current audit matter and / or the risk score of the third audit result; if included, perform multi-dimensional cross-validation on the first audit result, the second audit result and the third audit result, and use the consistency of the three audit results as one of the criteria for convergence judgment.

7. The bidding compliance review method based on a dual verification mechanism according to claim 1, characterized in that, Also includes: The audit results of manual review are structured and written into the corresponding verification configuration library, including the generated result tags; The closed-loop calibration generates rule extension suggestions, rule exception suggestions, prompt template optimization suggestions, threshold adjustment suggestions, or retrieval index update suggestions based on the result label, and synchronously updates and records the calibration version number.

8. A bidding compliance review system based on a dual verification mechanism, characterized in that, include: The document acquisition module is used to receive bidding documents; The document parsing module is used to parse bidding documents and generate evidence atoms. The evidence atom, as the smallest unit of evidence, includes at least a unique identifier, source, page number, chapter, original text fragment, context, hash value, and timestamp; The evidence chain construction module is used to build the evidence chain for the audit items. The construction steps include: using the audit items as an index, associating the clauses to be audited, the content of the bid response, the legal basis, local regulatory rules, historical review cases and rule versions, so as to support the tracing of each audit conclusion back to the original documents and sources of evidence. The rule engine verification module is used to perform rule engine verification based on the evidence chain and output the first audit result; The retrieval-enhanced large language model verification module is used to perform retrieval-enhanced large language model verification based on the evidence chain and output a second verification result. The cross-validation and result output module is used to perform multi-dimensional cross-validation on the first and second audit results. When the cross-validation is consistent, the audit result is output. When the cross-validation is inconsistent, the conflict cause is identified, and supplementary evidence is extracted based on the identified conflict type. The rule engine verification and large language model verification are repeatedly executed based on the updated evidence chain until the convergence condition is met and the audit result requiring manual review is output or generated. The manually reviewed audit result is then structured and written into the corresponding verification configuration library to complete the closed-loop calibration.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored computer program, wherein, when the computer program is executed, it controls the device on which the computer-readable storage medium is located to perform the method as described in any one of claims 1 to 7.

10. A computer device, characterized in that, The computer device includes a memory, a processor, and a program stored in and executable on the memory, the program being executed by the processor to implement the steps of the method as described in any one of claims 1 to 7.

Images