Multi-document multi-stage retrieval and verification method based on large model thinking chain reasoning

Abstract

The application provides a multi-document multi-stage retrieval and verification method based on large model thinking chain reasoning, and belongs to the technical field of retrieval enhancement generation and acceptance verification. The method receives and analyzes user verification requests, obtains a target document set including technical specification documents and quality evidence documents, and then performs three-stage hierarchical progressive retrieval and analysis. The authoritative standard documents and core content boundaries are extracted from the technical specification documents. The authoritative specification item set is obtained from the authoritative standard documents according to the core content boundaries, and the actual measurement quality verification data is obtained from the quality evidence documents according to the to-be-verified demands. Then, the progressive thinking chain reasoning prompt sentence is constructed and introduced into the large language model, the correlation comparison and multi-step reasoning are completed, and the compliance judgment conclusion and traceable deduction process description are output. The method of the application can realize verification automation, precision and traceability, and provides reliable support for multi-type document verification in complex scenarios.

Description

Technical Field

[0001] This invention relates to the field of retrieval enhancement generation and acceptance verification technology, and in particular to a multi-document, multi-stage retrieval and verification method based on large-model thinking chain reasoning. Background Technology

[0002] Against the backdrop of digital transformation, industries such as high-end manufacturing, energy and chemical engineering, and infrastructure are placing higher demands on the quality and compliance of materials and equipment. The acceptance process often requires item-by-item verification, focusing on whether parameter requirements meet relevant standards and supported by actual quality data. The goal is to ensure that deliverables meet the mandatory or recommended clauses of design specifications, national / industry / international standards.

[0003] Traditional methods rely heavily on manual processes: reviewers, based on the structured parameters in the goods list, first consult technical specifications (Type A documents), then locate authoritative standard documents (Type C documents) based on the standard numbers cited in Type A documents, find clauses / limits or test methods in Type C documents, and subsequently compare the measured data in quality verification documents (Type B documents) with the aforementioned clauses one by one to reach a conclusion. However, due to the cumbersome cross-type association retrieval of technical specifications, quality verification documents, and cited authoritative standard documents, and the fact that standard clause location is affected by version and aliases, the efficiency of data retrieval and verification is low. It is difficult to achieve efficient cross-source association retrieval, standardized fusion reasoning, and full evidence chain traceability verification in a multi-type document set containing technical specifications, quality verification, and cited standard documents. Summary of the Invention

[0004] In view of this, the present invention provides a multi-document, multi-stage retrieval and verification method based on large model thinking chain reasoning, in order to improve the efficiency of cross-source association retrieval of multiple types of documents.

[0005] The technical solution adopted in this invention is: This invention provides a multi-document, multi-stage retrieval and verification method based on large-scale model thinking chain reasoning, including: The system receives and parses user-submitted verification requests to obtain the verification requirements and a set of target documents. The set of target documents includes at least technical specification documents and quality verification documents, and the set of target documents supports association with authoritative standard documents. Based on the verification requirements, a first-level progressive search is performed in technical specification documents, and a large language model is called to perform semantic parsing on the first-level search results to generate structured technical parameter text. Based on the structured technical parameter text, the authoritative standard documents to be searched and the boundaries of the core content are determined. Based on the requirements to be verified and the boundaries of the core content, a second-level progressive search is performed in authoritative standard documents to obtain the corresponding set of authoritative norm entries. Based on the verification request, a third-level progressive search is performed in the quality verification documents to obtain the measured quality verification data that has been standardized in format. Based on the verification request, structured technical parameter text, authoritative specification item set, and actual quality verification data, a progressive thinking chain reasoning prompt statement is constructed. Import the progressive thinking chain reasoning prompts into the large language model, perform cross-source compliance verification and multi-dimensional logical reasoning, and output compliance judgment conclusions and traceable deduction process descriptions; Based on the compliance determination conclusion, a full-chain traceability verification file is created and persistently stored, and a set of supporting information on which the compliance determination conclusion depends is generated in a structured association manner.

[0006] Furthermore, the first-level progressive retrieval is performed in technical specification documents based on the requirements to be verified, and a large language model is invoked to perform semantic parsing on the first-level retrieval results to generate structured technical parameter text. Based on the structured technical parameter text, the boundaries of the authoritative standard documents to be retrieved and their core content are determined, including: Using the request to be verified as the search query, a semantic similarity-weighted search is performed in technical specification documents to extract the unstructured knowledge fragments with the highest relevance priority; The system integrates the request to be verified with the unstructured knowledge fragments to construct a progressive thinking chain reasoning prompt, and submits the progressive thinking chain reasoning prompt to a large language model for deep semantic parsing, outputting the structured technical parameter text, which includes key parameter items and the standard number identifiers referenced by the key parameter items; The standard number identifier is extracted from the structured technical parameter text. The standard number identifier is then formatted and standardized according to the preset standard name regular expression matching rules and normalization mapping mechanism. The processing results are deduplicated to determine the core content boundaries of the authoritative standard documents to be retrieved.

[0007] Furthermore, based on the requirements to be verified and the boundaries of core content, a second-level progressive search is performed in authoritative standard documents to obtain the corresponding set of authoritative standard entries, including: Based on the core content boundaries of the verification request and authoritative standard documents, hierarchical knowledge retrieval is performed in the authoritative standard documents to determine the target standard document. When there are standard aliases or version differences, a keyword hierarchical matching fallback mechanism is used to locate the target standard document. In addition, the target standard document is anchored at the item level semantic context by combining the core keywords to obtain multiple item-level semantic fragments. Multiple entry-level semantic fragments are dereduplicated and aggregated according to their original order in the target standard document to form an authoritative set of standard entries.

[0008] Furthermore, based on the request to be verified, a third-level progressive retrieval is performed in the quality supporting documents to obtain standardized measured quality verification data, including: A multimodal parsing tool was used to extract the original content text of quality-supporting documents, source identifiers were added to the original content text, and a reliability score was assigned to the original content text. Based on the requirements to be verified, a structured query vector is constructed, and the target paragraph is located in the original content text by querying the structured query vector. The target paragraph is then semantically filtered by combining the structured technical parameter text to remove segments without actual measured values. The measured data points are identified and located in the target paragraph after semantic filtering by using rule templates and a large language model. The measured data values ​​are extracted from the measured data points, and the data units of the measured data values ​​are unified by a pre-built unit converter. The measured data values ​​are then converted into a structured verification data package in JSON format. The structured verification data package includes parameter name, measured value, unit, data source, and confidence level fields. The structured verification data package is verified for its data source. When there are multiple data sources for the same parameter, the structured verification data package is cross-validated by weighted voting or large language model reasoning to determine the optimal measured data. The optimal measured data is then output as the measured quality verification data.

[0009] Furthermore, the step of using a multimodal parsing tool to extract the original content text of quality-supporting documents, adding source identifiers to the original content text, and configuring reliability scores for the original content text includes: A multimodal parsing tool is used to extract the original content text of quality-supporting documents. For the terminology variations of the same technical parameter in the original content text, terminology alignment is performed by combining a pre-built thesaurus with regular expressions and a semantic embedding model. A source identifier is added to each data point of the extracted original content text. The source identifier includes the document type and version number. At the same time, a reliability score is configured for each data point of the original content text based on the authority of the source and the integrity of the data.

[0010] Furthermore, the progressive thought chain reasoning prompt statements constructed based on the request to be verified, structured technical parameter text, authoritative specification item set, and measured quality verification data include: Based on semantic correspondence, the original verification request, structured technical parameter text, authoritative specification item set and actual quality verification data are logically paired to form a set of multi-source heterogeneous reasoning text containing technical parameter requirements, authoritative specification items and actual verification data; Data partitions are set in the multi-source heterogeneous inference text to respectively carry the original verification request, technical parameter requirements, authoritative specification items and actual verification data, resulting in the original verification request partition, technical parameter partition, authoritative specification partition and actual data partition; The multi-source heterogeneous reasoning texts are integrated into the thought chain reasoning prompts to form progressive thought chain reasoning prompts. The logical order and format conventions of the reasoning output of the large language model are set. The output order and format conventions are used to instruct the large language model to perform compliance analysis and generate conclusions for each set of multi-source heterogeneous reasoning texts in the order of technical parameter requirements, authoritative standard items and actual verification data.

[0011] Furthermore, the process of importing the progressive thought chain reasoning prompts into a large language model, performing cross-source compliance verification and multi-dimensional logical reasoning, and outputting compliance judgment conclusions and traceable deduction process descriptions includes: The progressive thinking chain reasoning prompts are imported into a large language model. The large language model is then used to perform text parsing on the data in the technical parameter partition and the authoritative norm partition in the multi-source heterogeneous reasoning text. Compliance benchmarks from the authoritative norm partition are extracted. The compliance benchmarks include, but are not limited to, numerical threshold ranges, text description specifications, or compliance judgment conditions. The large language model is used to perform structured text parsing on the data in the measured data partition, and the actual measurement parameters or validity descriptions from the measured quality verification data are extracted as measured supporting data. The compliance benchmark and the actual test data are used to perform multi-dimensional compliance verification through a large language model, and the verification results are output. The multi-dimensional compliance verification includes one or more of the following: numerical accuracy verification, text semantic fit analysis, and logical consistency judgment. Based on the verification results, a compliance judgment conclusion for the verification request is generated through a large language model. The compliance benchmark, actual test evidence data and multi-dimensional verification process description are integrated into a traceable deduction process description with a visual marker of the reasoning path according to a preset format.

[0012] Furthermore, the method also includes: creating an independent execution context based on the unique identity of the current operating user and the unique code of the current task to be verified. The independent execution context is used to isolate the intermediate processing data and the final output result of the current task to be verified, and at the same time, recording the context operation log to support full process backtracking.

[0013] In summary, the beneficial effects of the present invention are as follows: This invention provides a multi-document, multi-stage retrieval and verification method based on a large-scale model's thought chain reasoning. The method parses the verification request to obtain the verification request and a target document set, including at least technical specification documents and quality verification documents. It then performs a three-stage hierarchical progressive retrieval and analysis. After completing the three-stage retrieval, it integrates the verification requests from different document types, structured technical parameter texts, authoritative specification sets, and measured quality verification data. It constructs a logically coherent progressive thought chain reasoning prompt statement and inputs it into a large-scale model for multi-step reasoning analysis. The method determines whether each measured quality verification data point meets the consistency requirements of technical specifications and standards. Finally, it generates a compliance judgment conclusion and a traceable deduction process description. This achieves implicit semantic association mining and complete information chain tracing across document types and knowledge sources, providing efficient and reliable technical support for intelligent compliance review under complex document systems. Attached Figure Description

[0014] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the embodiments of the present invention will be briefly introduced below. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort, and these are all within the protection scope of the present invention.

[0015] Figure 1 This is a flowchart of the multi-document, multi-stage retrieval and verification method based on large-model thinking chain reasoning of the present invention; Figure 2 This is a schematic diagram of the complete multi-stage knowledge retrieval and verification process of the present invention. Detailed Implementation

[0016] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Unless otherwise specified, the present invention and the various features in the embodiments can be combined with each other, all of which are within the protection scope of the present invention.

[0017] Currently, when industries such as high-end manufacturing, energy and chemical engineering, and infrastructure conduct acceptance inspections of the quality and compliance of materials and equipment, inspectors first review technical specifications (such as technical specifications, drawings, and product data sheets) based on the structured parameter requirements in the goods list. Then, they locate authoritative standard documents (such as GB / T and ASTM) based on the standard numbers cited in the technical specifications, identify the clauses / limits or test methods within these authoritative standard documents, and subsequently compare the measured data from quality supporting documents (such as factory inspection reports, third-party test reports, and material quality certificates) with the aforementioned clauses one by one to reach a final conclusion. This process has the following prominent pain points: (1) Long citation chains and low efficiency in locating clauses: The cost of locating clauses in scenarios such as technical specification documents → authoritative standard documents → clauses is high and prone to errors; (2) Difficulty in managing standard versions and aliases: Authoritative standard documents have different clauses and testing methods in different versions, aliases / same names, and manual processing is prone to version confusion, leading to inaccurate judgment; (3) Significant semantic differences: Different types of documents, such as technical specification documents, quality verification documents, and authoritative standard documents, have variations in the terminology for the same technical concept (such as "tensile strength / tensile strength / fracture strength"), which makes automatic extraction and alignment difficult. (4) Insufficient cross-source deep reasoning: The multi-step reasoning from “parameter requirements → standard definition → quality data” relies on deep semantic understanding. Traditional rule-based or keyword matching methods are difficult to reproduce stably in complex contexts. (5) Insufficient evidence chain and process traceability: The manual "qualified / unqualified" conclusion lacks a readable reasoning chain and structured evidence binding, which is not conducive to auditing, review and accountability; (6) Scalability and consistency challenges: Different people's manual interpretation of different types of documents is prone to deviation, making it difficult to ensure consistency and scalability across projects and batches.

[0018] Based on the above analysis, there is an urgent need in this field for an intelligent parameter verification method that can comprehensively solve the above problems. This method should possess the following technical features: (1) Automated processing of long knowledge reference chains: It can automatically identify and track standard references in documents, and realize the automatic association from the original verification requirements to specific standard clauses; (2) Multi-source and multi-type knowledge fusion: It can uniformly process knowledge information from different sources and of different types, and realize cross-type knowledge retrieval and association; (3) Deep semantic understanding and reasoning: Based on the powerful understanding capabilities of large language models, it enables complex logical reasoning and knowledge association analysis; (4) Complete evidence chain construction: It can record and store all key information in the verification process to form a traceable and auditable evidence chain.

[0019] Therefore, this invention proposes a multi-document, multi-stage retrieval and verification method based on large language model thinking chain reasoning. The detailed implementation process of the method is shown in the following embodiments.

[0020] Reference Figure 1 and Figure 2 As shown in the figure, an embodiment of the present invention provides a multi-document, multi-stage retrieval and verification method based on large-model thinking chain reasoning, including: S1: Receive the verification request submitted by the user, parse the verification request, and obtain the request to be verified and the target document set. The target document set includes at least technical specification documents and quality verification documents, and the target document set supports association with authoritative standard documents. In this embodiment, for ease of describing document types, technical specification documents, quality verification documents, and authoritative standard documents are all pre-stored in a vector database.

[0021] S2: Based on the requirements to be verified, perform a first-level progressive search in technical specification documents, and call the large language model to perform semantic parsing on the first-level search results to generate structured technical parameter text. Based on the structured technical parameter text, determine the authoritative standard documents to be searched and the boundaries of core content. S3: Based on the requirements to be verified and the boundaries of core content, perform a second-level progressive search in authoritative standard documents to obtain the corresponding set of authoritative norm entries; S4: Based on the requirements to be verified, perform a third-level progressive search in the quality verification documents to obtain the measured quality verification data that has been standardized in format; S5: Construct progressive thought chain reasoning prompts based on the requirements to be verified, structured technical parameter text, authoritative specification item set, and measured quality verification data; S6: Import the progressive thinking chain reasoning prompts into the large language model, perform cross-source compliance verification and multi-dimensional logical reasoning, and output compliance judgment conclusions and traceable deduction process descriptions; S7: Create and persistently store a full-chain traceability verification file based on the compliance judgment conclusion, and generate a set of supporting information on which the compliance judgment conclusion depends in a structured association manner.

[0022] Technical specifications documents specifically include technical specifications, design drawings, and product data sheets. Quality verification documents specifically include factory inspection reports, third-party test reports, and material quality certificates. Authoritative standard documents specifically include the original texts of publicly available or authorized national standards (such as GB / T) and international standards (such as ASTM).

[0023] Taking the automated verification process of hot-bent pipe material parameters as an example, suppose a certain acceptance verification task includes a batch of hot-bent pipes with the specification model "φ168.3*12" and the material grade "L360-PSL2". To complete this verification task, the set of documents that need to be processed includes the "Hot-Bent Pipe Technical Specification" (as a technical specification document) and the "Hot-Bent Pipe Material Re-inspection Report" (as a quality verification document).

[0024] In this embodiment, step S1 receives a user-submitted verification request, which includes: a set of verification requests represented in a structured form, and multiple manually specified target document set identifiers used to limit the initial knowledge retrieval scope. The target document set can be determined based on the target document set identifiers. The target document set includes at least technical specification documents (such as product technical specification documents) and quality verification documents (such as supplier quality certification documents), and supports subsequent automatic association with authoritative standard documents (such as relevant standard specification documents) identified by references in the technical specification documents.

[0025] Based on the above-mentioned assumptions regarding the automated verification scenario, the specific implementation process of step S1 is as follows: A verification request submitted by a user is received. This request contains a structured requirement to be verified, such as a data structure containing "Verification Item: Tensile Strength" and "Requirement Description: Whether it meets the technical specifications." The request also includes the identifier of the target document set required for this verification, such as the "Hot Bending Pipe Technical Specifications" and the "Hot Bending Pipe Material Re-inspection Report."

[0026] In this embodiment, S2 performs a first-level progressive search in technical specification documents based on the requirements to be verified, and calls a large language model to perform semantic parsing on the first-level search results to generate structured technical parameter text. Based on the structured technical parameter text, the authoritative standard documents to be searched and the boundaries of the core content are determined, including: Using the request to be verified as the search query, a semantic similarity-weighted search is performed in technical specification documents to extract the unstructured knowledge fragments with the highest relevance priority.

[0027] The system integrates the requirements to be verified with the unstructured knowledge fragments to construct a progressive thought chain reasoning prompt. This prompt is then submitted to a large language model for deep semantic analysis, outputting a structured technical parameter text. This structured technical parameter text includes key parameter items and the standard numbers referenced by those items. The structured technical parameter text can be stored in the form of a technical requirements table.

[0028] The standard number identifier is extracted from the structured technical parameter text. The standard number identifier is then formatted and standardized according to the pre-set standard name regular expression matching rules and normalization mapping mechanism. The results are deduplicated to determine the core content boundaries of the authoritative standard documents to be retrieved.

[0029] Semantic similarity-weighted retrieval is a precise retrieval method adapted to multi-document, multi-parameter verification scenarios. Its core is to combine text semantic relevance with custom weight factors to filter target information: First, the verification request (such as "tensile strength compliance") and the document content are converted into semantic vectors respectively. The semantic matching degree (i.e., similarity) between the two is calculated by the algorithm. Then, weight factors such as the relevance of the document to the verification topic, the authority of the data fragment, and the importance of the field are introduced to perform weighted calculation on the similarity to obtain a comprehensive score. Finally, the scores are sorted from high to low, and the highly relevant and high-quality knowledge fragments with the highest scores are extracted first. This avoids the semantic bias of simple keyword retrieval and can focus on core information through weight tilting, thereby improving the accuracy and efficiency of retrieval.

[0030] This embodiment uses processes such as semantic similarity weighted retrieval, mind chain hint fusion parsing, and standard identifier cleaning and deduplication in standardized technical specification documents to accurately extract parameter requirements and standard identifiers, ensuring the structured nature and accuracy of structured technical parameter texts and providing a reliable basis for the accurate positioning of authoritative standard documents.

[0031] Based on the above-mentioned automated verification scenario, the specific process of performing the first stage of knowledge retrieval in step S2 is as follows: S21: Using the technical requirement to be verified, "tensile strength", as the query input, perform a semantic similarity weighted search in the document "Technical Specification for Hot Bending Pipes" to obtain the most relevant text fragments to the query, such as: "...the tensile strength of hot bending pipes must meet the requirements of ASTM A370 standard...".

[0032] S22: Construct a CoT (CoT) reasoning prompt, which integrates the technical requirement to be verified and the text fragment retrieved in S21. This prompt is submitted to the large language model. Based on the pre-defined parsing instructions in the prompt, the model performs deep parsing of the text fragment, generating a structured parsed technical requirement table, i.e., structured technical parameter text. This table not only confirms that the specific requirement for "tensile strength" is "meets ASTM A370 standard," but also successfully identifies and extracts the explicit identifier "ASTM A370" from the standard specification.

[0033] S23: Extract the standard specification identifier "ASTM A370" from the "Technical Requirements Table", and then perform format verification and standardization on the identifier according to a series of pre-set regular expressions (e.g., `r'ASTM\s*[A-Z0-9.]+'`) to obtain the authoritative standard document "ASTM A370" and its core content boundaries (e.g., "L360-PSL2 steel grade tensile strength related clauses") for subsequent retrieval.

[0034] In this embodiment, in step S3, based on the requirements to be verified and the boundaries of core content, a second-level progressive search is performed in authoritative standard documents to obtain the corresponding set of authoritative specification entries, including: Based on the core content boundaries of the verification request and authoritative standard documents, hierarchical knowledge retrieval is performed in authoritative standard documents to determine the target standard document. When there are standard aliases or version differences, a keyword hierarchical matching fallback mechanism is used to locate the target standard document. In addition, the target standard document is anchored at the item level semantic context by combining core keywords to obtain multiple item-level semantic fragments. Multiple entry-level semantic fragments are dereduplicated and aggregated according to their original order in the target standard document to form an authoritative set of standard entries.

[0035] Among them, the keyword hierarchical matching fallback mechanism is an adaptation mechanism in the retrieval of C-type documents (authoritative standard documents) to address the difficulty in locating due to standard aliases and version differences. The core is to design keyword sets according to the hierarchy from precise to broad. When the matching of the previous level fails, it will automatically fall back to the next level to ensure that the target standard document is not missed and the location is accurate. The specific process is as follows: First, the standard number (e.g., ASTM A370) and the core parameter to be verified (e.g., tensile strength) extracted from the first-level search are used as precise keywords to match the full name of the standard document, the specification number, or the title of the core clause. If no match is found (e.g., due to version differences between "ASTM A370-2020" and "ASTM A370-2018", or aliases for "tensile strength" and "tensile strength"), the search returns to the second level, retaining the main standard number (e.g., ASTM A370) and the set of parameter synonyms for fuzzy matching. If the target document is still not located, the search returns to the third level, retaining only the standard category identifier (e.g., ASTM) and keywords in the core technical field (e.g., strength testing of metallic materials) for range searching. At the same time, the search scope is further narrowed by combining the classification directory of Category C documents, ultimately achieving stable location of the target standard document and avoiding search failures due to inconsistent naming or version differences.

[0036] This embodiment effectively solves the retrieval difficulties caused by standard file aliases and version differences by using a restricted keyword matching fallback mechanism, entry-level context positioning, and fragment deduplication and aggregation. It efficiently obtains complete and authoritative standard entry content, thus consolidating the standard basis for verification.

[0037] Specifically, based on the above-mentioned hypothetical scenario, step S3 is implemented as follows: Based on the authoritative standard document "ASTM A370" obtained from S2 and combined with the keyword "tensile strength", an entry-level contextual search was performed within the standard document to obtain specific standard entry-level semantic fragments, such as: "...the minimum tensile strength (Rm) of L360-PSL2 steel grade steel pipe is 460MPa..."; The multiple entry-level semantic fragments obtained from the location are dereduplicated and aggregated in the order of the text to form standard entry content.

[0038] In this embodiment, step S4 involves performing a third-level progressive retrieval in quality supporting documents based on the verification request to obtain standardized measured quality verification data, including the following steps: A multimodal parsing tool was used to extract the original text content of quality-supporting documents, add source identifiers to the original text content, and assign reliability scores to the original text content. The multimodal parsing tool included an OCR recognition tool and a PDF parser; the OCR recognition tool was used to scan reports, and the PDF parser was used to process structured tables.

[0039] Based on the requirement to be verified, a structured query vector is constructed. This structured query vector is then used to locate the target paragraph within the original content text. Semantic filtering of the target paragraph is performed using structured technical parameter text, eliminating segments without measured values, such as text containing only theoretical descriptions without actual measurements. Specifically, the requirement to be verified can be broken down into key elements (such as the parameter name "tensile strength" and the constraint condition "≥460MPa") to generate the structured query vector. The rule template in this embodiment is primarily used for locating key values, such as matching the numerical pattern "\d+.?\d* MPa" using regular expressions. Through collaborative identification with the large model using the rule template, measured data points, such as "532MPa," are located.

[0040] By using rule templates and a large language model, measured data points are identified and located in the semantically filtered target paragraph. Measured data values ​​are extracted from these points, and a pre-built unit converter standardizes the units of these values, transforming them into a structured verification data package in JSON format. This structured verification data package includes parameter names, measured values, units, data sources, and confidence levels. The unit converter primarily standardizes the units of the measured data values, such as converting "psi" to "MPa," ensuring data comparability.

[0041] The structured verification data package is verified for its data source. When there are multiple data sources for the same parameter, the structured verification data package is cross-validated by weighted voting or large language model reasoning to determine the optimal measured data. The optimal measured data is then output as the measured quality verification data.

[0042] In multi-source cross-validation, the confidence score of the original content text is used as a weighting factor. Based on the confidence scores of each data packet, the measured values ​​are weighted and averaged to avoid misjudgments caused by homogenization. The specific validation process is as follows: Aggregate all available data packets for the same parameter (e.g., tensile strength data from the "Factory Inspection Report" (confidence level 0.75) and the "Third-Party Test Report" (confidence level 0.95)). Ensure consistent units and standardized parameter names, and generate a temporary dataset containing the measured values, confidence levels, and source metadata for each data packet.

[0043] Calculate the numerical differences between data packets (e.g., whether the relative deviation exceeds a preset threshold, such as 5%). If all data packets have the same value (deviation < threshold), directly output the consensus value; otherwise, mark it as "conflicting data" and enter the weighted voting decision process. Example: Report A's measured value is 532 MPa, and Report B's measured value is 520 MPa, with a deviation of 2.3% < 5%, requiring no complex verification; if Report C's measured value is 480 MPa, with a deviation > 5%, then conflict handling is triggered.

[0044] In the weighted voting decision-making process, the confidence score of each data packet is used as the basis for weighting, but a dynamic adjustment factor is introduced: Base weight = Confidence score (e.g., 0.95, 0.75). Adjustment factor: Considers source authority (third-party report weight + 0.1) and data freshness (recent data weight + 0.05). The weight allocation calculation formula is: Final weight = Confidence score + ∑(Adjustment factor).

[0045] Calculate the weighted average of the measured values: ; Example: Data packet A (value 532, weight 0.95), data packet B (value 520, weight 0.75), fused value ≈ 528.6 MPa. Finally, output the preliminary results (fused value) and a summary of the weight distribution.

[0046] For example, in the hot bending of pipe scenario, if two reports give measured values ​​of 532MPa and 520MPa respectively, the system will prioritize the data with higher confidence (e.g., 0.95) and output a consistency conclusion (e.g., "data deviation <5%, verified").

[0047] In addition, the method of using a large language model to perform multi-source cross-validation of structured verification data packets involves inputting a list of multi-source data packets (including values, weights, and sources), relevant context, and validation instructions into the large language model, which then performs multi-step reasoning according to the preset validation instructions and finally outputs the validation results.

[0048] Specifically, a multimodal parsing tool is used to extract the original content text of quality-supporting documents, source identifiers are added to the original content text, and reliability scores are assigned to the original content text. This process includes the following steps: Multimodal parsing tools are used to extract the original content text of quality-supporting documents. For terminology variations of the same technical parameter in the original content text, terminology alignment is performed using a pre-built thesaurus, combined with regular expressions and a semantic embedding model. For example, for terminology variations (such as "tensile strength" vs. "tensile strength"), a thesaurus is built, and terminology alignment is achieved using regular expressions and a semantic embedding model (such as BERT).

[0049] A source identifier is added to each data point of the extracted original content text. The source identifier includes the document type and version number. At the same time, a reliability score is configured for each data point of the original content text based on the authority of the source and the integrity of the data.

[0050] Specifically, based on the above-mentioned assumptions in the automated verification scenario, step S4 executes the third stage of knowledge retrieval as follows: The system uses the keyword "tensile strength" as the query input, performs a search in the document "Re-inspection Report of Hot-Bent Pipe Material" and obtains relevant text content. For example, it extracts "...tensile strength MPa TS Rm | Measured Value: 532..." from the "Mechanical Properties" table of the material re-inspection report, and uses this content as product quality certification data, i.e., measured quality verification data.

[0051] In this embodiment, S5 constructs a progressive thought chain reasoning prompt based on the request to be verified, structured technical parameter text, authoritative specification item set, and measured quality verification data, including: Based on semantic correspondence, the original verification request, structured technical parameter text, authoritative specification item set, and measured quality verification data are logically paired to form a set of multi-source heterogeneous reasoning text containing technical parameter requirements, authoritative specification items, and measured verification data.

[0052] Data partitions are set in the multi-source heterogeneous inference text to respectively carry the original verification request, technical parameter requirements, authoritative specification items and actual verification data, resulting in the original verification request partition, technical parameter partition, authoritative specification partition and actual data partition.

[0053] The multi-source heterogeneous reasoning texts are integrated into the thought chain reasoning prompts to form progressive thought chain reasoning prompts. The logical order and format conventions of the reasoning output of the large language model are set. The output order and format conventions are used to instruct the large language model to perform compliance analysis and generate conclusions for each set of multi-source heterogeneous reasoning texts in the order of technical parameter requirements, authoritative standard items and actual verification data.

[0054] This embodiment forms multi-source heterogeneous reasoning text through logical pairing of multi-source information, and sets clear output order and format constraints to standardize the construction logic of progressive thinking chain reasoning prompts, ensuring the coherence and consistency of reasoning in the large language model and avoiding information fusion chaos.

[0055] In step S5, a progressive thought chain reasoning prompt statement is constructed to generate the final verification conclusion. Its multi-source heterogeneous reasoning text is logically divided into multiple regions, which respectively carry: the original verification request, technical parameter requirements, authoritative standard items and actual test verification data, so as to form a multi-dimensional and logically coherent information fusion reasoning context.

[0056] Specifically, based on the above-mentioned hypothetical scenario, step S5 is implemented as follows: Once the retrieval tasks S3 and S4 are completed, the system constructs a final progressive thought chain reasoning prompt statement to generate the verification conclusion. This process specifically includes: S51: Based on the correspondence, the original verification request, the technical requirements table generated in S2, the standard clause content obtained in S3 ("...the minimum tensile strength (Rm) is 460MPa...") are logically paired with the product quality certification data obtained in S4 ("...the measured value: 532...") to form a multi-dimensional reasoning context containing "technical requirements - standard clauses - quality data"; S52: Integrate the multi-dimensional reasoning context into the thought chain reasoning prompts, and set clear output order and format constraints. Instruct the large language model to strictly follow the order of "technical requirements → standard clauses → quality data" to perform item-by-item comparison analysis and conclusion generation for each set of contexts.

[0057] In this embodiment, step S6 imports the progressive thought chain reasoning prompts into the large language model, performs cross-source compliance verification and multi-dimensional logical reasoning, and outputs a compliance judgment conclusion and a traceable deduction process description, including: The progressive thinking chain reasoning prompts are imported into a large language model. The large language model is then used to perform text parsing on the data in the technical parameter partition and the authoritative norm partition in the multi-source heterogeneous reasoning text. Compliance benchmarks from the authoritative norm partition are extracted. The compliance benchmarks include, but are not limited to, numerical threshold ranges, text description specifications, or compliance judgment conditions. The large language model is used to perform structured text parsing on the data in the measured data partition, and the actual measurement parameters or validity descriptions from the measured quality verification data are extracted as measured supporting data. The compliance benchmark and the actual test data are used to perform multi-dimensional compliance verification through a large language model, and the verification results are output. The multi-dimensional compliance verification includes one or more of the following: numerical accuracy verification, text semantic fit analysis, and logical consistency judgment. Based on the verification results, a compliance judgment conclusion for the verification request is generated through a large language model. The compliance benchmark, actual test evidence data and multi-dimensional verification process description are integrated into a traceable deduction process description with a visual marker of the reasoning path according to a preset format.

[0058] This embodiment refines the reasoning process of identifying compliance criteria, extracting evidence to be evaluated, comparing multiple dimensions of logic, and generating interpretable descriptions, ensuring the accuracy of compliance judgment conclusions, while making the reasoning process clear and traceable, and easy to understand and verify.

[0059] Specifically, based on the aforementioned assumptions in the automated verification scenario, step S6 submits the progressive thought chain reasoning prompt to the large language model. Upon receiving this highly integrated thought chain reasoning prompt, the large language model performs multi-step logical reasoning, specifically implemented as follows: S61: The large language model first integrates information from the technical parameter partitioning and authoritative specification partitioning in the context, and determines the qualified standard as "tensile strength not less than 460MPa"; S62: The large language model finds the actual measured value of "532MPa" in the quality data area of ​​the context as evidence to be evaluated; S63: The large language model compares the pass / fail standard with the evidence to be evaluated and determines that 532MPa is greater than 460MPa. S64: Based on the comparison results, the large language model generates a structured final parameter verification result, i.e., a compliance judgment conclusion, which reads: "Qualified. Reasons: 1. The technical specifications require compliance with ASTM A370 standard; 2. ASTM A370 standard specifies that the minimum tensile strength of L360-PSL2 steel grade material is 460MPa; 3. The material re-inspection report shows that the measured value is 532MPa, which is greater than 460MPa, therefore it meets the requirements."

[0060] In this embodiment, step S7 creates and persistently stores a traceable verification record. The record is associated in a structured manner with all the evidence information on which the conclusion depends, including at least: the target document set identifier (technical specification documents and quality evidence documents), the technical requirements table, the product quality certification data, and authoritative standard documents and corresponding standard clauses that may be involved.

[0061] Specifically, in the automated verification scenario based on the above assumptions, step S7 is implemented as follows: A full-chain traceability verification record is created, containing the compliance judgment conclusion and a structured association of all supporting information relied upon to generate the result, including the target document set identifiers ("Hot Bending Pipe Technical Specification" and "Hot Bending Pipe Material Re-inspection Report"), the parsed technical requirements table, and the identified standard document "ASTM A370". This record is persistently stored in a database for subsequent auditing and traceability.

[0062] Through the above embodiments, the method of the present invention can automatically complete a complex parameter verification task that requires traversing three different types of document sources and performing deep logical reasoning, significantly improving the efficiency and accuracy of the verification, and ensuring complete transparency and traceability of the process.

[0063] In this embodiment, before executing step S1, the method further includes: creating an independent execution context based on the unique identity of the current operating user and the unique code of the current task to be verified. The independent execution context is used to isolate the intermediate processing data and the final output result of the current task to be verified, and at the same time, record the context operation log to support full-process backtracking, ensuring that the retrieval range between multiple concurrent or continuous verifications does not interfere with each other and avoids data pollution.

[0064] In summary, the method of this embodiment is suitable for automated and traceable knowledge verification of related information in heterogeneous document sets in complex business scenarios, such as compliance verification of multi-source documents involving product technical specifications, supplier quality certificates, and external standards and specifications in procurement scenarios. It has the following technical advantages: (1) Improved verification automation and efficiency: Through the architecture of three-stage progressive retrieval and dual LLM call reasoning, the complex verification process that originally required manual review and comparison of multiple types of documents is automated, which significantly shortens the verification cycle and reduces labor costs. (2) Ensures the accuracy and consistency of verification: Based on standard name normalization, item-level context positioning and templated progressive thinking chain reasoning prompts, it reduces version confusion and misunderstanding bias, and improves consistency across projects and batches; (3) Complete evidence chain tracing is achieved: all evidence information on which the verification conclusion is based is associated in a structured manner, which facilitates quality auditing and problem review; (4) Enhanced scalability and maintainability: Supports the expansion of standard systems and retrieval strategies by industry category, supports collaboration and version management of multiple language models, and adapts to the continuous evolution of standards and scenarios.

[0065] This embodiment's method parses the verification request to obtain the verification request and a target document set, including at least technical specification documents and quality verification documents, from the verification request. It then performs a three-stage hierarchical progressive retrieval and analysis. After completing the three-stage retrieval, the verification requests from different document types, structured technical parameter texts, authoritative specification sets, and measured quality verification data are integrated to construct a logically coherent progressive thought chain reasoning prompt. This prompt is input into a large model for multi-step reasoning analysis to determine whether each measured quality verification data point meets the consistency requirements of technical specifications and standards. Finally, a compliance judgment conclusion and a traceable deduction process description are generated. This achieves implicit semantic association mining and complete information chain tracing across document types and knowledge sources, providing efficient and reliable technical support for intelligent compliance review under complex document systems. This invention, by constructing a knowledge processing architecture of "staged retrieval—progressive focusing—collaborative reasoning," achieves implicit semantic association mining and complete information chain tracing across document types and knowledge sources, providing efficient and reliable technical support for intelligent compliance review under complex document systems.

[0066] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A multi-document, multi-stage retrieval and verification method based on large-scale model thinking chain reasoning, characterized in that, include: Receive the verification request submitted by the user, parse the verification request, and obtain the request to be verified and the target document set; The target document set includes at least technical specification documents and quality verification documents, and the target document set supports association with authoritative standard documents; Based on the request to be verified, a first-level progressive retrieval is performed in technical specification documents. A large language model is then invoked to perform semantic parsing on the first-level retrieval results, generating structured technical parameter text. Based on this structured technical parameter text, the boundaries of the authoritative standard documents to be retrieved and their core content are determined. This includes: using the request to be verified as the search query, performing a semantic similarity-weighted retrieval in technical specification documents to extract unstructured knowledge fragments with the highest relevance priority; fusing the request to be verified with the unstructured knowledge fragments to construct progressive thought chain reasoning prompts, and submitting these prompts to a large language model for deep semantic parsing, outputting the structured technical parameter text, which includes key parameter items and the standard number identifiers referenced by these key parameter items; extracting the standard number identifiers from the structured technical parameter text, performing format verification and identifier standardization processing on the standard number identifiers according to preset standard name regular expression matching rules and a normalization mapping mechanism, and deduplicating the processing results to determine the boundaries of the authoritative standard documents to be retrieved and their core content. Based on the verification request and core content boundaries, a second-level progressive retrieval is performed in authoritative standard documents to obtain the corresponding set of authoritative normative entries. This includes: performing hierarchical knowledge retrieval in authoritative standard documents based on the verification request and the core content boundaries of authoritative standard documents to determine the target standard document; when there are standard aliases or version differences, a keyword hierarchical matching fallback mechanism is used to locate the target standard document; and the target standard document is anchored at the entry level semantic context by combining core keywords to obtain multiple entry-level semantic fragments; the multiple entry-level semantic fragments are deredundantized and aggregated according to the original order in the target standard document to form the set of authoritative normative entries. Based on the verification request, a third-level progressive retrieval is performed on the quality supporting documents to obtain standardized measured quality verification data. This includes: extracting the original content text of the quality supporting documents using a multimodal parsing tool, adding source identifiers to the original content text, and configuring reliability scores for the original content text; constructing a structured query vector based on the verification request, and using the structured query vector to locate the target paragraph in the original content text; performing semantic filtering on the target paragraph in conjunction with structured technical parameter text to remove segments without measured values; and using rule templates and a large language model to analyze the semantically filtered target paragraph. The system identifies and locates measured data points, extracts measured data values ​​from these points, and uses a pre-built unit converter to unify the data units of the measured data values. The measured data values ​​are then converted into a structured verification data packet in JSON format. This structured verification data packet includes parameter name, measured value, unit, data source, and confidence level fields. The system then verifies the data source of the structured verification data packet. When multiple data sources exist for the same parameter, a weighted voting method or a large language model inference method is used to perform multi-source cross-validation of the structured verification data packet to determine the optimal measured data. This optimal measured data is then output as the measured quality verification data. Based on the verification request, structured technical parameter text, authoritative standard item set, and actual quality verification data, a progressive thinking chain reasoning prompt statement is constructed. Import the progressive thinking chain reasoning prompts into the large language model, perform cross-source compliance verification and multi-dimensional logical reasoning, and output compliance judgment conclusions and traceable deduction process descriptions; Based on the compliance determination conclusion, a full-chain traceability verification file is created and persistently stored, and a set of supporting information on which the compliance determination conclusion depends is generated in a structured association manner.

2. The multi-document, multi-stage retrieval and verification method based on large-model thinking chain reasoning as described in claim 1, is characterized in that, The process involves using a multimodal parsing tool to extract the original content text of quality-supporting documents, adding source identifiers to the original content text, and configuring reliability scores for the original content text, including: A multimodal parsing tool is used to extract the original content text of quality-supporting documents. For the terminology variations of the same technical parameter in the original content text, terminology alignment is performed by combining a pre-built thesaurus with regular expressions and a semantic embedding model. A source identifier is added to each data point of the extracted original content text. The source identifier includes the document type and version number. At the same time, a reliability score is configured for each data point of the original content text based on the authority of the source and the integrity of the data.

3. The multi-document, multi-stage retrieval and verification method based on large-model thinking chain reasoning according to claim 1, characterized in that, The progressive thought chain reasoning prompts, constructed based on the request to be verified, structured technical parameter text, authoritative standard item set, and measured quality verification data, include: Based on semantic correspondence, the original verification request, structured technical parameter text, authoritative specification item set and actual quality verification data are logically paired to form a set of multi-source heterogeneous reasoning text containing technical parameter requirements, authoritative specification items and actual verification data; Data partitions are set in the multi-source heterogeneous inference text to respectively carry the original verification request, technical parameter requirements, authoritative specification items and actual verification data, resulting in the original verification request partition, technical parameter partition, authoritative specification partition and actual data partition; The multi-source heterogeneous reasoning texts are integrated into the thought chain reasoning prompts to form progressive thought chain reasoning prompts. The logical order and format conventions of the reasoning output of the large language model are set. The output order and format conventions are used to instruct the large language model to perform compliance analysis and generate conclusions for each set of multi-source heterogeneous reasoning texts in the order of technical parameter requirements, authoritative standard items and actual verification data.

4. The multi-document, multi-stage retrieval and verification method based on large-model thinking chain reasoning according to claim 1, characterized in that, The process of importing progressive thought chain reasoning prompts into a large language model, performing cross-source compliance verification and multi-dimensional logical reasoning, and outputting compliance judgment conclusions and traceable deduction process descriptions includes: The progressive thinking chain reasoning prompts are imported into a large language model. The large language model is then used to perform text parsing on the data in the technical parameter partition and the authoritative norm partition of the multi-source heterogeneous reasoning text. Compliance benchmarks from the authoritative norm partition are extracted. The compliance benchmarks include numerical threshold ranges, text description specifications, or compliance judgment conditions. The large language model is used to perform structured text parsing on the data in the measured data partition, and the actual measurement parameters or validity descriptions from the measured quality verification data are extracted as measured supporting data. The compliance benchmark and the actual test data are used to perform multi-dimensional compliance verification through a large language model, and the verification results are output. The multi-dimensional compliance verification includes one or more of the following: numerical accuracy verification, text semantic fit analysis, and logical consistency judgment. Based on the verification results, a compliance judgment conclusion for the verification request is generated through a large language model. The compliance benchmark, actual test evidence data and multi-dimensional verification process description are integrated into a traceable deduction process description with a visual marker of the reasoning path according to a preset format.

5. The multi-document, multi-stage retrieval and verification method based on large-model thinking chain reasoning according to claim 1, characterized in that, Also includes: An independent execution context is created based on the unique identifier of the current user and the unique code of the task to be verified. The independent execution context is used to isolate the intermediate processing data and the final output result of the task to be verified, and at the same time, the context operation log is recorded to support full process backtracking.

Images