A test data analysis method and system based on a large language model

Abstract

The application provides a test data analysis method and system based on a large language model, belonging to the technical field of intelligent analysis. The method analyzes the field semantics, table field relationships and natural language descriptions in standard document format data using a large language model to generate candidate structured data, and generates structure relationship candidate data in combination with standard image format data. A candidate value set is constructed, and cross-modal semantic alignment is performed, including unit normalization, symbol mapping, synonymous expression merging and semantic equivalence determination. The analysis result is determined according to the consistency score, evidence weight and candidate value credibility, and the consistency component coefficient and evidence weight component coefficient are updated through constrained reinforcement learning. When the credibility difference value is less than the preset threshold, manual review and / or supplementary retrieval are triggered. The analysis result is constructed as a traceable test data three-dimensional account and responds to natural language queries. The application can improve the analysis accuracy, result credibility and retrieval efficiency of multi-source heterogeneous test data.

Description

Technical Field

[0001] This invention relates to the field of experimental data processing and intelligent analysis technology, and in particular to an experimental data analysis method and system based on a large language model. Specifically, it relates to a technical solution for cross-modal analysis, semantic alignment, consistency verification, credibility assessment, review triggering, and three-dimensional ledger construction of text, tables, drawings, structural diagrams, and natural language descriptions in multi-source experimental data. Background Technology

[0002] In product development, engineering testing, performance verification, inspection and testing, and operational evaluation, a large amount of experimental data is typically generated. This data comes from diverse sources, including text documents such as test reports, test record sheets, technical specifications, standards and standards, and test conclusions, as well as structural diagrams, test drawings, screenshots, graphs, table screenshots, and other image-based materials. Because different types of data vary significantly in their source, recording methods, expression habits, data formats, and semantic granularity, the same attribute of the same test object often appears repeatedly in different forms, and problems such as numerical conflicts, inconsistent units, inconsistent names, inconsistent semantic expressions, or missing context may arise. This increases the difficulty of collecting, verifying, tracing, and reusing experimental data.

[0003] In existing technologies, the processing of experimental data typically relies on manual review, rule matching, template extraction, or single-modal information parsing. These approaches are highly dependent on document format, field naming, and layout structure, making them prone to issues such as missed or incorrect candidate value extraction, or inaccurate positioning when faced with nested tables, natural language descriptions, cross-page expressions, mixed text and graphics, and non-standard experimental records. Furthermore, existing solutions often focus on extracting explicit fields, lacking a deep understanding of field semantics, table field relationships, and the implicit experimental information within natural language descriptions. Consequently, they struggle to accurately generate candidate structured data suitable for subsequent merging and comparison.

[0004] Furthermore, when faced with multiple candidate values ​​from different documents, modalities, time periods, and sources, existing technologies often can only perform simple string comparisons, numerical comparisons, or regularized unit conversions, making it difficult to make stable judgments on whether candidate values ​​are semantically equivalent or point to the same experimental meaning. Especially when there are aliases for technical terms, abbreviations, contextual omissions, changes in description methods, or implicit correspondences between text and images, relying solely on traditional rules or shallow similarity algorithms is insufficient to achieve consistency verification across modalities and expression forms.

[0005] Furthermore, existing experimental data integration systems typically lack a unified modeling mechanism for the credibility of candidate values. For multiple mutually supporting or conflicting candidate values, it is difficult to comprehensively consider factors such as analytical confidence level, experimental time stage, data modality type, data source credibility, and consistency relationships among candidate values ​​to select the most credible analytical result. In cases with ambiguous boundaries or severe evidence conflicts, there is often a lack of adaptive review triggering mechanisms and analytical parameter update mechanisms, resulting in insufficient stability of analytical results, high costs for manual review, and difficulty in forming a continuous optimization loop based on historical review results.

[0006] Meanwhile, existing experimental data management methods mostly rely on tabular archiving or document storage, lacking the ability to uniformly organize and correlate analytical results across time, logic, and knowledge dimensions. This makes it difficult to create a comprehensive, traceable, searchable, and reasonable ledger of experimental data. When users submit queries in natural language, existing systems typically cannot effectively convert these queries into structured query instructions, nor can they return experimental data results or summary information with evidence chains, thus limiting the in-depth utilization and intelligent service capabilities of experimental data.

[0007] Therefore, there is an urgent need to propose a new experimental data parsing method and system to solve technical problems such as insufficient candidate data generation capability, insufficient cross-modal semantic alignment capability, insufficient consistency verification, inaccurate credibility assessment, imperfect verification loop, and weak subsequent retrieval and question answering capabilities for multi-source heterogeneous experimental data. Summary of the Invention

[0008] The purpose of this invention is to provide a method and system for parsing experimental data based on a large language model, in order to solve the problems existing in the parsing of multi-source experimental data, such as insufficient generation of candidate structured data, insufficient ability to determine cross-modal semantic equivalence, low accuracy of consistency verification, unreliable credibility assessment, low efficiency of manual review, and difficulty in forming a traceable three-dimensional ledger of experimental data. This invention aims to achieve high-quality parsing, reliable fusion, closed-loop optimization, and intelligent retrieval and question answering of multi-source heterogeneous experimental data.

[0009] To achieve the aforementioned objective, this invention provides a method for parsing experimental data based on a large language model, executed by a processor. The method includes: extracting candidate data from standard document format data, wherein a large language model parses the field semantics, table field relationships, and natural language descriptions in the standard document format data to generate candidate structured data; parsing standard image format data to generate candidate structural relationship data; merging the candidate structured data and candidate structural relationship data according to the experimental object identifier and attribute type identifier to construct a set of candidate values ​​corresponding to the same attribute of the same experimental object; and performing cross-modal semantic alignment on the candidate values ​​in the candidate value set, wherein the cross-modal semantic alignment includes at least unit normalization, symbol mapping, and synonymy. The process involves: expressing merging and semantic equivalence determination performed by a large language model; determining the parsing result based on the consistency score between candidate values, the evidence weight corresponding to the candidate value, and the credibility of the candidate value; jointly modeling the update process of the consistency component coefficient vector and the evidence weight component coefficient vector as a constrained reinforcement learning decision process, and updating the consistency component coefficient vector and the evidence weight component coefficient vector according to the feedback result; selecting the candidate value with the highest credibility as the parsing result, and triggering manual review and / or supplementary retrieval when the difference between the highest credibility and the second highest credibility is less than a preset threshold; organizing the parsed experimental data according to the time dimension, logical dimension, and knowledge dimension to form a traceable three-dimensional ledger of experimental data, and responding to natural language queries based on the three-dimensional ledger of experimental data.

[0010] To achieve the aforementioned objective, this invention also provides an experimental data parsing system based on a large language model, comprising: a text-based experimental data parsing module for extracting candidate data from standard document format data, wherein the large language model parses the field semantics, table field relationships, and natural language descriptions in the standard document format data to generate candidate structured data; a drawing and structural diagram parsing module for parsing standard image format data to generate candidate structural relationship data; and a multimodal semantic fusion and consistency verification module for merging the candidate structured data and candidate structural relationship data according to the experimental object identifier and attribute type identifier, constructing a candidate value set corresponding to the same attribute of the same experimental object, performing cross-modal semantic alignment on the candidate values ​​in the candidate value set, and determining the parsing result based on the consistency score between candidate values, the evidence weight corresponding to the candidate values, and the credibility of the candidate values. The cross-modal semantic alignment includes at least unit normalization, symbol mapping, synonym merging, and semantic equivalence determination performed by a large language model; a joint parameter update module is used to jointly model the update process of the consistency component coefficient vector and the evidence weight component coefficient vector as a constrained reinforcement learning decision process, and update the consistency component coefficient vector and the evidence weight component coefficient vector according to the feedback result; a parsing decision and review triggering module is used to select the candidate value with the highest credibility as the parsing result, and trigger manual review and / or supplementary retrieval when the difference between the highest credibility and the second highest credibility is less than a preset threshold; a three-dimensional experimental data ledger construction module is used to associate and organize the parsed experimental data according to the time dimension, logical dimension, and knowledge dimension to form a traceable three-dimensional experimental data ledger; and an intelligent retrieval and question answering module based on a large language model is used to respond to natural language queries based on the three-dimensional experimental data ledger.

[0011] Compared with the prior art, the present invention has at least the following beneficial effects: Firstly, this invention introduces a large language model in the standard document candidate structured data generation stage. The large language model parses the field semantics, table field relationships, and natural language descriptions, thereby breaking through the dependence on fixed templates, rule dictionaries, and explicit fields, and improving the completeness and accuracy of candidate data extraction in complex documents, non-standard documents, and documents with mixed text and graphics.

[0012] Secondly, this invention introduces semantic equivalence determination performed by a large language model during the semantic alignment process, enabling the system to not only handle unit differences, symbol differences, and synonymous expressions, but also to identify whether different candidate values ​​point to the same or equivalent meaning at the experimental semantic level, thereby improving the consistency and reliability of cross-document, cross-modal, and cross-expression data merging.

[0013] Third, by constructing an evaluation chain of "candidate value set - consistency score - evidence weight - credibility", this invention can comprehensively consider the degree of consistency between candidate values ​​and the evidence quality of the candidate values ​​themselves, thereby achieving a refined comparison of multiple conflicting candidate values ​​and selection of credible analysis results, thus improving the accuracy and stability of the final analysis results.

[0014] Fourth, this invention models the update process of the consistency component coefficient and the evidence weight component coefficient as a constrained reinforcement learning decision process, and introduces feedback information such as parsing correctness, verification cost and coefficient volatility, forming a closed-loop update mechanism of parsing-verification-feedback-re-optimization. This enables the system to continuously optimize according to the actual task distribution and historical verification experience, reducing the dependence on manual experience parameter tuning.

[0015] Fifth, the present invention sets up a review triggering mechanism based on the difference between the maximum confidence level and the second-highest confidence level. In cases where there is a large conflict of evidence, unclear boundaries, or insufficient confidence, manual review and / or supplementary retrieval can be automatically triggered, thereby improving the controllability and reliability of key analysis results while ensuring the overall level of automation.

[0016] Sixth, this invention organizes the parsed experimental data according to the time dimension, logical dimension, and knowledge dimension to form a traceable three-dimensional ledger of experimental data. This makes the experimental data no longer limited to scattered documents and isolated fields, but can support source tracing, logical association, knowledge reasoning, and evidence chain output, thereby improving the systematicness and usability of experimental data management.

[0017] Seventh, this invention further utilizes a large language model to parse user natural language queries into structured query instructions for the three-dimensional ledger, and returns the corresponding experimental data results or summary information, thereby improving the accessibility and interaction efficiency of experimental data for business personnel, R&D personnel and management personnel, and enhancing the intelligent service capabilities of experimental data.

[0018] In summary, this invention can effectively achieve intelligent analysis, reliable fusion, closed-loop optimization, and traceable management of multi-source heterogeneous experimental data. Attached Figure Description

[0019] Figure 1 This is a flowchart of the experimental data parsing method based on a large language model provided by the present invention.

[0020] Figure 2 This is a line graph comparing different schemes under various evaluation indicators in the comparative experiment of this invention.

[0021] Figure 3 This is a line graph comparing different ablation schemes under various evaluation indicators in the ablation experiment of this invention. Detailed Implementation

[0022] The technical solution of the present invention will be further described in detail below with reference to specific embodiments. It should be noted that the following embodiments are only for illustrating the present invention and are not intended to limit the scope of protection of the present invention. Without departing from the concept of the present invention, those skilled in the art can make equivalent substitutions or conventional modifications to the technical features, all of which should fall within the scope of protection of the present invention.

[0023] Figure 1 This is a flowchart of the experimental data parsing method based on a large language model provided by the present invention, such as... Figure 1 As shown, this invention provides a method for parsing experimental data based on a large language model, executed by a processor calling program instructions from memory. The method is applicable to multi-source heterogeneous experimental data, performing unified parsing, candidate value construction, cross-modal semantic alignment, consistency verification, evidence weighting, credibility assessment, adaptive parameter updates, review triggering, and three-dimensional ledger construction on text, tables, drawings, structural diagrams, and natural language descriptions. It also supports subsequent natural language queries and evidence chain return.

[0024] S1. Acquisition and Standardization of Multi-Source Experimental Data Specifically, this includes: acquiring multi-source test data to be analyzed, wherein the multi-source test data includes at least one or more of the following: test reports, test record sheets, standard specifications, inspection conclusions, structural diagrams, drawings, image screenshots, graphs, table screenshots, and annotations; preprocessing and format standardizing the multi-source test data, converting editable documents, scanned documents, image documents, and mixed-type test data into standard document format data and / or standard image format data, respectively.

[0025] Specifically, for text-based or editable experimental data, chapter hierarchy, title information, table structure, field order, page number information, and paragraph position information are preserved; for scanned documents and image-based data, layout detection, region segmentation, character recognition, chart region positioning, and structural line extraction are performed to generate standard image format data that can be used for subsequent analysis; for mixed text and image data, standard document format data and standard image format data are output simultaneously, and page-level, region-level, or object-level index associations are established between the two.

[0026] Furthermore, a unified data index primary key is generated for the transformed data. The data index primary key includes at least the source document identifier, page number identifier, region identifier, modality type identifier, and collection time identifier, so that it can be referenced in subsequent steps when merging candidate values, calculating consistency, and generating evidence chains.

[0027] In this step, the present invention unifies multi-source experimental data with different sources, formats, and expressions into standard document format data and / or standard image format data, while retaining the layout structure and source location information required for subsequent traceability. This provides a unified, standardized, and traceable data foundation for candidate data extraction, cross-modal alignment, and evidence chain output, avoiding problems such as source loss, context breakage, or missing structural information in subsequent processing.

[0028] S2. Candidate Data Extraction and Candidate Value Set Construction Specifically, this includes: extracting candidate data from the standard document format data, wherein a large language model parses the field semantics, table field relationships, and natural language descriptions in the standard document format data to generate candidate structured data; parsing the standard image format data to generate candidate structural relationship data; and then merging the candidate structured data and candidate structural relationship data according to the test object identifier and attribute type identifier to construct a set of candidate values ​​corresponding to the same attribute of the same test object.

[0029] The parsing of standard document format data specifically includes: identifying the test object name, sample number, component identifier, parameter name, attribute value, numerical unit, test time, test stage, test conditions, context title, table header fields, paragraph description, and page positioning information; using a large language model combined with chapter titles, table header context, field adjacency relationships, natural language descriptions, and domain-specific semantics to uniformly understand explicit and implicit fields, thereby outputting candidate structured data. The candidate structured data includes at least one or more of the following: test object identifier, attribute name, attribute value, unit, test time, source page number, table header information, chapter title, paragraph position, or box coordinates.

[0030] The parsing of standard image format data specifically includes: identifying annotation text, dimension text, structural connection relationships, arrow pointing relationships, legend relationships, curve correspondence relationships, and local explanatory information in the image area, and converting them into candidate structural relationship data. This candidate structural relationship data is used to characterize the relationships between test objects, between attributes and objects, and between structural components and measurement results.

[0031] After extracting candidate structured data and candidate data related to structural relationships, the data is merged according to the experimental object identifier and attribute type identifier to construct a set of candidate values ​​corresponding to the same attribute of the same experimental object: , in, For the same attribute of the same test subject, For the first One candidate value, The number of candidate values, and each candidate value The associated source document identifier, data modality type, test time phase identifier, and field location information; the field location information includes at least one or more of the following: page number, table header, chapter title, paragraph position, table cell position, or box coordinates.

[0032] In this invention, the training of the large language model can be carried out by combining domain-specific corpus-supervised fine-tuning with instruction-based fine-tuning. Specifically, a training corpus consisting of test reports, inspection records, standard specifications, technical specifications, and manually annotated field samples can be constructed to supervise and fine-tune the pre-trained language model, enabling it to learn the extraction rules for test objects, attribute names, attribute values, units, and contextual relationships. Furthermore, task templates such as "extracting attribute values ​​from natural language descriptions," "inferring field meanings based on table headers and context," and "identifying implicit attributes based on chapters and paragraphs" can be added through instruction construction to enhance the model's ability to parse non-standard expressions, implicit fields, and complex table relationships. Incremental fine-tuning can also be continuously performed using manually reviewed samples to improve the model's adaptability in the target test domain.

[0033] In this step, the present invention introduces a large language model into the standard document candidate structured data generation process, enabling the system to not only extract explicit fields, but also understand the experimental information implicit in the relationships between table fields, natural language descriptions, and contextual semantics, thereby improving the completeness and accuracy of candidate value extraction. At the same time, by unifying and merging text candidate information with structural relationship candidate information obtained from image parsing, a more comprehensive evidence set can be established for the same experimental object and the same attribute, laying the foundation for subsequent semantic alignment and credibility assessment.

[0034] S3, Candidate value cross-modal semantic alignment Specifically, this includes performing cross-modal semantic alignment on the candidate values ​​in the candidate value set. The cross-modal semantic alignment includes at least unit normalization, symbol mapping, synonym expression merging, and semantic equivalence determination performed by a large language model to determine whether different candidate values ​​point to the same or equivalent experimental meaning at the semantic level.

[0035] Among these, unit normalization is used to convert candidate values ​​from different unit systems to a unified dimensional expression; symbol mapping is used to map abbreviations, symbols, codes, and engineering notations used in different documents to a unified semantic identifier; synonym expression merging is used to map semantically similar expressions such as "rated output power," "nominal output power," and "output rating" to a unified attribute name. The semantic equivalence determination performed by the large language model is used to identify whether candidate values ​​still correspond to the same experimental meaning despite differences in surface strings, expression forms, and syntactic structures.

[0036] Specifically, any two candidate values ​​can be... and The textual expression, context headings, table header information, paragraph descriptions, adjacent field relationships, and test subject descriptions are jointly input into a large language model. The large language model outputs a judgment result indicating whether the two are semantically equivalent, partially equivalent, conflicting, or require further verification. The semantic equivalence judgment result is used for subsequent semantic consistency components. The generation of .

[0037] In this invention, the training method for the large language model used for semantic equivalence determination can be carried out by combining supervised training with semantic matching samples, contrastive learning training, and manual review feedback reinforcement training. Specifically, a candidate value pair sample set can be constructed, with each sample pair including candidate value text, field context, source information, and manually labeled "equivalent / inequivalent / conditionally equivalent". The decision head of the large language model is trained through supervised learning. Further, positive and negative sample pairs can be used for contrastive learning to make semantically equivalent samples closer in the embedding space and semantically conflicting samples more separated. Subsequent review results can also be used as incremental training samples to continuously update the model, thereby improving the model's ability to recognize aliases of technical terms, abbreviations, contextual omissions, and cross-modal mapping relationships.

[0038] In this step, the present invention introduces unit normalization, symbol mapping, synonym expression merging, and semantic equivalence determination performed by a large language model in the semantic alignment process. This can solve the problem of multiple expressions, multiple modal recording methods, and multiple professional aliases for the same experimental attribute in multi-source data. This reduces false conflicts and false splits caused by expression differences and improves the accuracy of candidate value merging and subsequent consistency verification.

[0039] S4. Consistency Score Calculation Specifically, this includes: for any two candidate values and Calculate the consistency score: , in, Candidate values With candidate values Consistency score between them For numerical consistency, For consistency of units, For semantic consistency, For context consistency, These are the learnable coefficients corresponding to the consistency components; Wherein, the numerical consistency It can be calculated as follows: , in, and Candidate values With candidate values The corresponding value, The scaling parameter is used to adjust the degree of influence of numerical differences on numerical consistency; when and When the difference is small, Larger; when the difference between the two is large, Decrease.

[0040] The unit consistency This is used to reflect whether the units of candidate values ​​are consistent after normalization; when two candidate values ​​have different original units but are consistent in dimensions and can be corresponded after conversion, the unit consistency takes the higher value; when the units conflict and cannot be converted, the unit consistency takes the lower value.

[0041] semantic consistency Candidate values ​​from a large language model With candidate values The semantic equivalence is determined to reflect the degree of correspondence between the two candidate values ​​at the experimental semantic level.

[0042] Context consistency The calculation is based on the field location information of the candidate values ​​in the document. The field location information includes at least one or more of the following: page number, table header or chapter title, paragraph position, table cell position or box coordinates. If two candidate values ​​come from the context area corresponding to similar chapters, similar table headers, adjacent paragraphs or the same test phase, then their contextual consistency is high.

[0043] In practical implementation, we can first test the same set of candidate values. For each pair of candidate values, calculate numerical consistency, unit consistency, semantic consistency, and contextual consistency, and then use the current coefficient vector. Calculate the consistency score matrix to characterize the mutual support and conflict relationships within the candidate value set.

[0044] In this step, the present invention jointly models numerical consistency, unit consistency, semantic consistency and contextual consistency, which can avoid the bias of judgment caused by relying solely on single numerical comparison or string matching, making the evaluation of the relationship between candidate values ​​more comprehensive, stable and in line with the actual experimental semantics, thereby providing a more reliable basis for subsequent credibility calculation.

[0045] S5. Calculation of the weight of evidence Specifically, this includes calculating the evidence weight of each candidate value based on its analytical confidence level, trial time phase, data modality type, and data source. , in, Candidate values Weight of evidence Candidate values The analytical confidence score is used to characterize the credibility of candidate values ​​obtained from text parsing, image parsing, or semantic extraction processes of large language models. Candidate values During the trial period, This is a time-stage weighting function, used to assign different weights to candidate values ​​based on whether they are in the pre-test, mid-test, or post-test stage, or the formal testing, retesting, or re-verification stage. Candidate values Data modality type, This is a modal weighting function used to assign different weights based on modal types such as text, tables, images, structural diagrams, or drawings; Candidate values Data source This is a source credibility function used to assign different credibility levels based on authoritative documents, original records, derived reports, or external citation sources; These are the learnable coefficients corresponding to the evidence weight components.

[0046] Wherein, the time stage weight function The modal weighting function can be preset according to the test business rules or obtained from historical review samples. For example, the weight of the formal testing stage is greater than that of the pre-test stage, and the weight of the review and confirmation stage is greater than that of the initial recording stage. The source credibility function can be set according to the stability and interpretability of different modalities in the target domain; It can be set according to document-level trust level, source department, issuance status, or version status.

[0047] To ensure that different factors can be compared together, before calculating the weight of evidence, the following can be considered: , , and Normalize to a uniform numerical range, and then perform weighted summation.

[0048] In this step, the present invention incorporates the analytical confidence level of the candidate value itself, the experimental time stage, the data modality type, and the credibility of the data source into a unified evidence weight model. This enables subsequent credibility assessment to consider not only whether the candidate values ​​are consistent with each other, but also the quality differences of the evidence on which each candidate value is based. This avoids the undue influence of a large number of low-quality candidate values ​​on the final result and improves the credibility of the analytical results.

[0049] S6. Calculation of Candidate Value Credibility Specifically, this includes: calculating the credibility of each candidate value based on the consistency score and the evidence weight. , in, Candidate values Credibility, Candidate values With candidate values Consistency score between them Candidate values Weight of evidence Candidate values The weight of evidence. This formula represents: candidate values... The credibility of a candidate value is determined by its consistency with other candidate values ​​and the quality of evidence it provides for itself and related candidate values; a candidate value that is supported by more high-weight, highly consistent candidate values ​​has higher credibility.

[0050] Furthermore, after calculating the confidence level of all candidate values, a candidate value confidence level sequence can be formed. They are then sorted by their credibility level for use in subsequent analysis, decision-making, and review triggering.

[0051] In practice, when a candidate value significantly conflicts with most high-weight candidate values, its corresponding consistency score is low, thus reducing its credibility. Conversely, when a candidate value has a single source but high credibility of that source, high analytical confidence, and consistency with other supporting evidence, its credibility can still be relatively high. This allows for a balanced handling of "numerous but weak" evidence and "few but strong" evidence.

[0052] In this step, the present invention couples the consistency relationship between candidate values ​​with the evidence quality of each candidate value itself to form a unified credibility index, which can more accurately distinguish the support relationship and conflict relationship between candidate values, thereby improving the rationality and stability of the final analysis result selection and avoiding the bias caused by relying solely on a single high-confidence candidate value or solely on majority voting.

[0053] S7. Joint parameter update of consistency component coefficient and evidence weight component coefficient Specifically, this includes: converting the consistency component coefficient vector... and evidence weight component coefficient vector The update process is jointly modeled as a constrained reinforcement learning decision-making process, outputting coefficient increments based on the current parsing task state. and And after constrained projection, the updated version is obtained. and Reuse the updated and Recalculate the consistency score, evidence weight, and candidate value credibility.

[0054] The state vector of the constrained reinforcement learning decision-making process can be represented as: , in, For the first The state vector corresponding to the sub-analysis task. The average analytical confidence level of the candidate evidence for the current task. For attribute type tags, As an indicator of candidate value conflict, For consistency component statistics, This represents the weight value for the current experimental time phase. The current data modality weight value. This represents the current data source's credibility value. This is a flag indicating whether a review has been triggered.

[0055] The action can be represented as: , in, For the first The reinforcement learning action corresponding to the next parsing task. For the increment of the consistency component coefficient, This represents the increment of the evidence weight component coefficient.

[0056] The updated coefficient vector satisfies: , in, and These are the updated coefficient vectors, and These are the constraint projection operators, and The constraint domains are the consistency component coefficient and the evidence weight component coefficient, respectively. The constraint domains are used to ensure that each coefficient falls within a preset range and satisfies nonnegativity, upper bound constraints and / or normalization constraints, so as to avoid numerical divergence or unreasonable bias in the coefficient update process.

[0057] Furthermore, the reinforcement learning reward function is defined as follows: , in, For the first The reward value for each analysis task. This serves as an indicator of consistency between the analytical results and those verified manually or by authoritative sources. To review the triggered penalty items, To calculate the cost term, This is a penalty term for coefficient volatility. The reward weight coefficient is used to encourage improved parsing accuracy, reduce unnecessary manual review, control computational costs, and suppress drastic fluctuations in the coefficient.

[0058] In this invention, the constrained reinforcement learning model can be trained through a combination of offline pre-training and online fine-tuning. Specifically, offline training data can be constructed first using historical trial analysis samples, manual review records, authoritative annotation results and corresponding candidate value sets, consistency scores, evidence weights, and final analysis results to pre-train the policy model and value model, enabling them to learn the coefficient adjustment rules under different analysis states. Subsequently, during the execution of real analysis tasks, online fine-tuning is performed based on actual review feedback, correctness results, and cost information, thereby improving the model's adaptability to the current business scenario. During training, methods such as experience replay, target network, policy constraint update, entropy regularization, or gradient pruning can be used to improve training stability. Correspondingly, the policy model is trained using historical analysis trajectories and online feedback trajectories to output coefficient increment actions, and the value model is trained using reward prediction to evaluate the contribution of the current state action to the long-term analysis effect.

[0059] In this step, the present invention models the update process of the consistency component coefficient and the evidence weight component coefficient as a constrained reinforcement learning decision process. It can adaptively adjust the scoring and weighting strategies according to different attribute types, different conflict levels, different modal combinations and different source quality, thereby reducing manual experience in parameter setting and improving the system's adaptability, generalization ability and continuous optimization ability in complex experimental data scenarios.

[0060] S8, Analysis Decision, Review Triggering and Feedback Flow Specifically, this includes: selecting the candidate value with the highest confidence level as the parsing result for the same attribute; when the difference between the highest confidence level and the second highest confidence level is less than a preset threshold... When manual review and / or supplementary retrieval are triggered, the review results, parsing correctness, review cost, and coefficient volatility are used as reward feedback for subsequent reinforcement learning updates.

[0061] Let the confidence level of the candidate value with the highest confidence level in the candidate value set be . The second highest credibility is Then when When the current attribute analysis is deemed to have unclear evidence boundaries or significant candidate conflicts, manual review and / or supplementary retrieval are required; among these, This is a preset threshold used as the switching boundary between automatic analysis and manual intervention in the control system.

[0062] The supplementary retrieval includes at least: performing secondary character recognition on the original document's location area and / or performing related retrieval on adjacent pages; if necessary, adjacent chapters, related charts, footnotes, and other historical records of the same test subject can also be called for supplementary comparison to improve the sufficiency of the analytical decision basis.

[0063] After manual review, the correct candidate values ​​confirmed by the review, their source document identifiers, field location information, trial time information, and corresponding parsing correctness labels are written back to the training sample pool. Simultaneously, the review process is calculated to determine whether this round of parsing triggered a review, the review time, computational cost, and coefficient change magnitude, and these are used as factors in the reward function. , , and It is a component used to update the reinforcement learning model.

[0064] In this step, the present invention establishes a dynamic balance between the system's automatic parsing capability and manual careful verification by setting a threshold mechanism for the difference between the maximum and second-highest credibility. For cases where the credibility distinction is obvious, the results can be automatically output to improve processing efficiency. For cases where there is obvious evidence conflict, unclear boundaries, or semantic instability, manual review and supplementary retrieval are automatically triggered to improve the reliability of key parsing results. Furthermore, the review feedback continuously feeds back into the model update, forming a closed-loop optimization.

[0065] S9. Construction of a Three-Dimensional Ledger for Experimental Data and Natural Language Query S9 specifically includes: organizing the parsed experimental data according to the time dimension, logical dimension, and knowledge dimension to form a traceable three-dimensional ledger of experimental data; parsing the user's natural language query into a structured query instruction for the three-dimensional ledger; and returning the corresponding experimental data results or summary information.

[0066] The time dimension describes the temporal relationships of experimental data generation, recording, retesting, verification, and updating; the logical dimension describes the logical relationships between experimental objects, attributes, experimental conditions, results, conclusions, and source records; and the knowledge dimension describes the knowledge graph-like connections between experimental objects, attributes, structural relationships, and causal relationships. The resulting three-dimensional ledger of experimental data can be stored using a structured ledger table, a graph-structured ledger, or a hybrid of both.

[0067] When performing path reasoning on a graph-structured ledger, the candidate path score can be represented as: , in, Candidate paths The rating, Relationship edges in the path Relevance score For entity nodes in the path Credibility, This is an adjustment coefficient for the node credibility component; the entity node credibility is at least partially derived from the candidate value credibility obtained in the preceding steps. This path scoring mechanism allows for the selection of more relevant and credible evidence paths within multi-hop relationships, enabling the answering of complex queries or the generation of summaries.

[0068] Output the chain of evidence along with the query results: , in, For the chain of evidence, For source document identification, Provide field location information. This field location information may further include one or more of the following: page number, chapter title, table header information, paragraph position, table cell position, or box coordinates, to allow users to trace back the basis for parsing.

[0069] Furthermore, the user's natural language query can be parsed into structured query instructions by the large language model. Specifically, this includes: identifying the test objects, target attributes, time range, comparison conditions, source constraints, and summary intent in the query, and mapping them into query statements oriented towards structured ledgers or graph-structured ledgers; for query tasks requiring multi-hop association, condition filtering, or summary summarization, the large language model can also combine entity relationships and evidence chain information in the three-dimensional ledger to generate user-oriented result descriptions.

[0070] This invention organizes the analysis results according to the time dimension, logical dimension, and knowledge dimension, which can transform scattered, heterogeneous, and fragmented experimental information into a traceable, searchable, and reasonable three-dimensional ledger. At the same time, by converting natural language queries into structured query instructions and returning results or summary information with evidence chains, it can significantly improve the reuse efficiency, retrieval efficiency, and decision support capabilities of experimental data.

[0071] The present invention also provides an experimental data parsing system based on a large language model, including: a multi-source experimental data acquisition and format conversion module, a text-based experimental data parsing module, a drawing and structural schematic parsing module, a multimodal semantic fusion and consistency verification module, a joint parameter update module, a parsing decision and review triggering module, an experimental data three-dimensional ledger construction module, and an intelligent retrieval and question answering module based on a large language model.

[0072] The system includes the following modules: a multi-source experimental data acquisition and format conversion module for acquiring multi-source experimental data and converting it into standard document format data and / or standard image format data; a text-based experimental data parsing module for extracting candidate data from the standard document format data, wherein a large language model parses the field semantics, table field relationships, and natural language descriptions in the standard document format data to generate candidate structured data; a drawing and structural diagram parsing module for parsing the standard image format data to generate candidate structural relationship data; a multimodal semantic fusion and consistency verification module for merging the candidate structured data and candidate structural relationship data according to the experimental object identifier and attribute type identifier, constructing a candidate value set corresponding to the same attribute of the same experimental object, and performing cross-modal semantic alignment, calculating consistency scores, evidence weights, and candidate value credibility on the candidate values ​​in the candidate value set; and a joint parameter update module for updating the consistency component coefficient vector. and evidence weight component coefficient vector The update process is jointly modeled as a constrained reinforcement learning decision process, outputting coefficient increments and updating them after constraint projection. and The analysis decision and review triggering module is used to select the candidate value with the highest credibility as the analysis result, and trigger manual review and / or supplementary retrieval when the difference between the highest credibility and the second highest credibility is less than a preset threshold; the experimental data three-dimensional ledger construction module is used to associate and organize the analyzed experimental data according to the time dimension, logical dimension and knowledge dimension to form a traceable experimental data three-dimensional ledger; the intelligent retrieval and question answering module based on the large language model is used to parse the user's natural language query into a structured query instruction for the three-dimensional ledger, and return the corresponding experimental data results or summary information.

[0073] The modules of the above system work together through data interfaces or message mechanisms. The functional implementation process corresponds to the aforementioned method implementation method, and will not be repeated here.

[0074] This implementation introduces a large language model into the standard document candidate structured data generation stage, and further incorporates semantic equivalence judgment performed by the large language model in the cross-modal semantic alignment of candidate values. This enables the system to more accurately identify the corresponding information of the same experimental object and the same attribute in multi-source heterogeneous experimental data. By constructing a joint evaluation mechanism of consistency scoring, evidence weight, and credibility, it achieves credible comparison and robust selection of multiple candidate values. By modeling the update process of consistency component coefficients and evidence weight component coefficients as a constrained reinforcement learning decision process, it achieves adaptive optimization of analytical parameters. By setting a review trigger mechanism based on credibility difference, it improves the reliability of key analytical results. By constructing a traceable three-dimensional ledger of experimental data and supporting natural language queries, it enhances the organization, retrieval, traceability, and intelligent service capabilities of experimental data. Therefore, this invention can effectively solve the problems of insufficient multi-source experimental data analysis, inaccurate semantic alignment, insufficient credible fusion capability, and low subsequent utilization efficiency in the prior art.

[0075] To verify the effectiveness of the experimental data parsing method and system based on large language models described in this invention in the parsing of multi-source heterogeneous experimental data, the following comparative experiment can be used for illustration.

[0076] I. Experimental Objective The improvements of this invention over the prior art are verified in the following aspects: (1) Verify whether the present invention improves the completeness of candidate data extraction and parsing accuracy in standard documents, non-standard documents, mixed descriptions of tables and natural language, and mixed text and image data; (2) Verify whether the present invention improves the accuracy of cross-modal semantic alignment, semantic equivalence determination and conflict candidate value screening; (3) Verify whether the present invention is superior to existing solutions in terms of manual review trigger rate, reliability of final parsing results and availability of natural language queries.

[0077] II. Experimental Data Experimental data can be selected from multi-source test data under a certain product testing scenario as the test set. The test set includes test reports, test record sheets, test drawings, structural diagrams, image screenshots, table screenshots, and annotations, forming a sample set of test data. To ensure the representativeness of the experiment, the test set can simultaneously cover the following scenarios: (1) Standard documents with uniform attribute field names and standardized formats; (2) Non-standard documents with changes in table headers, paragraph descriptions, and implicit expressions; (3) Mixed data including drawing annotations, structural arrows, table screenshots, and text-figure correspondences; (4) Data on the same attribute of the same test subject from different sources, where there are differences in units, aliases, abbreviations, and candidate values.

[0078] Furthermore, domain experts can manually annotate the test object identifiers, attribute names, attribute values, units, test times, source location information, and final correct parsing results in the test set to form an authoritative annotation set, which can serve as a benchmark for comparison and evaluation.

[0079] III. Comparison Scheme The comparative experiment can be set up with the following three schemes: Comparison with Solution 1: The traditional rule parsing solution uses a parsing method based on keyword matching, fixed templates, regular expressions, and unit conversion rules to extract fields, merge results, and output results from the experimental data.

[0080] Comparison Scheme 2: The baseline scheme without large language models and reinforcement learning updates uses layout parsing, table recognition, OCR recognition, similarity matching, and a consistency scoring method with fixed weights to complete candidate value extraction, merging, and decision-making. However, it does not introduce large language models in the generation of standard document candidate structured data, nor does it introduce large language models in semantic equivalence determination. Furthermore, the consistency component coefficient and the evidence weight component coefficient adopt manually preset fixed values.

[0081] Experimental scheme: The scheme of this invention adopts the method described in this invention, introduces a large language model in the standard document candidate structured data generation stage, introduces semantic equivalence judgment performed by the large language model in the cross-modal semantic alignment of candidate values, and uses consistency scoring, evidence weight, credibility joint modeling and constrained reinforcement learning update mechanism to complete parsing decision and review triggering.

[0082] IV. Evaluation Indicators The following indicators can be used to evaluate the experimental results: (1) Candidate value extraction recall rate, which measures the proportion of candidate values ​​that should be extracted that are correctly extracted; (2) Candidate value extraction accuracy, which measures the proportion of correct candidate values ​​among the extracted candidate values; (3) Final parsing accuracy, which measures the proportion of the output results that are consistent with the authoritative annotation results; (4) Semantic equivalence determination accuracy, which is used to measure the accuracy of determining whether candidate values ​​of different expression forms are semantically equivalent or not; (5) Manual review trigger rate, which measures the proportion of samples that need to enter the manual review process; (6) Average parsing time per attribute, used to measure the system's parsing efficiency; (7) Evidence chain integrity rate, which measures the proportion of the output results that can be traced back to the source document identifier and field location information; (8) Natural language query hit rate, which measures the proportion of correct results or correct summaries returned after a user’s natural language query.

[0083] V. Experimental Procedure Under the same test set and hardware environment, Comparison Scheme 1, Comparison Scheme 2, and the scheme of this invention were run respectively. For each target attribute of each test object, candidate value extraction, candidate value merging, parsing result output, and evidence chain generation were performed; for the scheme of this invention, semantic equivalence determination, credibility calculation, review triggering, and reinforcement learning update were also performed. Then, the output results of each scheme were compared with the authoritative annotation set, and various evaluation indicators were statistically analyzed.

[0084] To verify the role of the large language model in two key stages of this invention, an ablation experiment can be further set up: First, "large language model parsing in the generation of candidate structured data for standard documents" was removed; Second, remove the "large language model in semantic equivalence determination"; Third, the "constrained reinforcement learning update mechanism" was removed and replaced with a fixed coefficient.

[0085] Ablation experiments can further illustrate the contribution of each key technical feature to the overall improvement.

[0086] VI. Experimental Results In one embodiment, the experimental results shown in Table 1 can be obtained.

[0087] Table 1 Comparative experimental results of different schemes

[0088] A comparison chart of suture removal was drawn based on Table 1, as shown below. Figure 2 As shown, Figure 2 In the diagram, the horizontal axis symbols A1 to A8 represent different evaluation metrics, where A1 represents recall, A2 represents precision, A3 represents parsing accuracy, A4 represents semantic determination rate, A5 represents review trigger rate, A6 represents parsing time, A7 represents evidence chain integrity rate, and A8 represents query hit rate. The legend symbols L1 to L3 represent different experimental schemes, where L1 represents the traditional rule parsing scheme, L2 represents the baseline scheme without large language model and reinforcement learning updates, and L3 represents the present invention.

[0089] Furthermore, in the ablation experiment, the results are shown in Table 2.

[0090] Table 2 Ablation Experiment Results of Key Technical Features of the Invention

[0091] Based on Table 2, a comparison chart of the suture removal diagrams was drawn as follows: Figure 3 As shown, Figure 3 In the diagram, the horizontal axis symbols B1 to B4 represent different ablation experiment schemes, where B1 represents the complete scheme, B2 represents document LLM removal (removing the large language model in the generation of standard document candidate structured data), B3 represents semantic LLM removal (removing the large language model in semantic equivalence determination), and B4 represents RL update removal (removing the constrained reinforcement learning update mechanism). The legend symbols M1 to M3 represent different evaluation metrics, where M1 represents parsing accuracy, M2 represents semantic decision rate, and M3 represents review trigger rate.

[0092] VII. Analysis of Experimental Results From Table 1 and Figure 2 It can be seen that, compared with traditional rule parsing schemes and fixed-weight baseline schemes, the present invention significantly improves the recall rate of candidate value extraction, the precision rate of candidate value extraction, the final parsing accuracy, the semantic equivalence determination accuracy, the evidence chain integrity rate, and the natural language query hit rate. At the same time, the manual review trigger rate is significantly reduced, indicating that the present invention not only improves the parsing quality of multi-source heterogeneous experimental data, but also reduces the cost of manual intervention.

[0093] From Table 2 and Figure 3 It can be seen that after removing the large language model in the generation of candidate structured data for standard documents, the completeness of candidate value extraction decreases, which in turn leads to a decrease in the final parsing accuracy. After removing the large language model in semantic equivalence determination, the accuracy of semantic equivalence determination decreases significantly, indicating that the ability to unify and merge candidate values ​​of different expressions is weakened. After removing the constrained reinforcement learning update mechanism, both the final parsing accuracy and the review trigger rate deteriorate, indicating that the adaptive update of the consistency component coefficient and the evidence weight component coefficient helps to improve the system's adaptability to different parsing scenarios.

[0094] Therefore, it can be demonstrated that by introducing a large language model in the standard document candidate structured data generation stage and in the semantic equivalence determination stage, and combining consistency scoring, evidence weight, credibility joint modeling and constrained reinforcement learning update mechanism, this invention can effectively improve the parsing accuracy, semantic merging ability, result credibility and retrieval service capability of multi-source heterogeneous experimental data, and has significant technical effects.

[0095] The comparative experiments above show that, compared with the prior art, the present invention can achieve at least the following beneficial effects: improve the completeness of candidate data extraction in standard and non-standard documents, improve the accuracy of cross-modal semantic equivalence determination, improve the credibility and accuracy of the final parsing results, reduce the trigger rate of manual review, improve the completeness of experimental data evidence chain and the hit rate of natural language queries, thereby verifying that the present invention has outstanding substantive technical effects.

[0096] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, or combinations made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for parsing experimental data based on a large language model, characterized in that, Executed by the processor, including: Candidate data extraction is performed on standard document format data, wherein a large language model parses the field semantics, table field relationships and natural language descriptions in the standard document format data to generate candidate structured data; Standard image format data is parsed to generate candidate data for structural relationships; According to the test object identifier and attribute type identifier, the candidate structured data and the candidate structural relationship data are merged to construct a set of candidate values ​​corresponding to the same attribute of the same test object; Perform cross-modal semantic alignment on the candidate values ​​in the candidate value set. The cross-modal semantic alignment includes at least unit normalization, symbol mapping, synonym merging, and semantic equivalence determination performed by a large language model. The analysis results are determined based on the consistency score among candidate values, the evidence weight corresponding to the candidate values, and the credibility of the candidate values. The update process of the consistency component coefficient vector and the evidence weight component coefficient vector is jointly modeled as a constrained reinforcement learning decision process, and the consistency component coefficient vector and the evidence weight component coefficient vector are updated according to the feedback results. The candidate value with the highest credibility is selected as the parsing result, and manual review and / or supplementary retrieval are triggered when the difference between the highest credibility and the second highest credibility is less than a preset threshold. The parsed experimental data is organized according to the time dimension, logical dimension, and knowledge dimension to form a traceable three-dimensional ledger of experimental data, and natural language queries are responded to based on the three-dimensional ledger of experimental data.

2. The experimental data parsing method based on a large language model according to claim 1, characterized in that, The candidate structured data includes one or more of the following: test object identifier, attribute name, attribute value, unit, test time, source page number, table header information, chapter title, paragraph position, or box coordinates.

3. The experimental data parsing method based on a large language model according to claim 1, characterized in that, The set of candidate values ​​corresponding to the same attribute of the same test subject is represented as follows: ， in, For the set of candidate values, For the first One candidate value, This represents the number of candidate values, and each candidate value is associated with the source document identifier, data modality type, experiment time phase identifier, and field location information.

4. The experimental data parsing method based on a large language model according to claim 3, characterized in that, The consistency score among the candidate values ​​is calculated as follows: ， in, Candidate values With candidate values Consistency score between them For numerical consistency, For consistency of units, For semantic consistency, For context consistency, , , , These are the learnable coefficients corresponding to the consistency components.

5. The experimental data parsing method based on a large language model according to claim 4, characterized in that, Numerical consistency Calculate as follows: ， in, and Candidate values With candidate values The corresponding value, The scale parameter; the semantic consistency Candidate values ​​from a large language model With candidate values The semantic equivalence is determined; the context consistency is obtained. Calculated based on the field location information of candidate values ​​in the document.

6. The experimental data parsing method based on a large language model according to claim 1, characterized in that, The evidence weights corresponding to the candidate values ​​are calculated as follows: ， in, Candidate values Weight of evidence Candidate values The resolution confidence level, Candidate values During the trial period, For the time phase weight function, Candidate values Data modality type, For modal weighting functions, Candidate values Data source For the source credibility function, , , , These are the learnable coefficients corresponding to the evidence weight components.

7. The experimental data parsing method based on a large language model according to claim 1, characterized in that, The confidence level of the candidate value is calculated as follows: ， in, Candidate values Credibility, Candidate values With candidate values Consistency score between them Candidate values Weight of evidence Candidate values The weight of evidence.

8. The experimental data parsing method based on a large language model according to claim 1, characterized in that, The state vector of the constrained reinforcement learning decision-making process is: ， in, For the first The state vector corresponding to the sub-analysis task. The average analytical confidence level of the candidate evidence for the current task. For attribute type tags, As an indicator of candidate value conflict, For consistency component statistics, This represents the weight value for the current experimental time phase. The current data modality weight value. This represents the current data source's credibility value. This serves as a flag indicating whether a review has been triggered. The actions of the constrained reinforcement learning decision-making process are as follows: ， in, For the first The reinforcement learning action corresponding to the next parsing task. For the increment of the consistency component coefficient, This represents the increment of the evidence weight component coefficient.

9. The experimental data parsing method based on a large language model according to claim 8, characterized in that, The updated coefficient vector satisfies: ， in, and These are the updated coefficient vectors, and These are the constraint projection operators, and These are the constraint domains for the consistency component coefficient and the evidence weight component coefficient, respectively. The reward function for the constrained reinforcement learning decision-making process is: ， in, For the first The reward value for each analysis task. This serves as an indicator of consistency between the analytical results and those verified manually or by authoritative sources. To review the triggered penalty items, To calculate the cost term, This is a penalty term for coefficient volatility. , , , This is the reward weighting coefficient.

10. A test data parsing system based on a large language model, characterized in that, include: The text-based experimental data parsing module is used to extract candidate data from standard document format data. The large language model parses the field semantics, table field relationships, and natural language descriptions in the standard document format data to generate candidate structured data. The drawing and structural diagram parsing module is used to parse standard image format data to generate candidate data for structural relationships; The multimodal semantic fusion and consistency verification module is used to merge the candidate structured data and structural relationship candidate data according to the test object identifier and attribute type identifier, construct a candidate value set corresponding to the same attribute of the same test object, perform cross-modal semantic alignment on the candidate values ​​in the candidate value set, and determine the parsing result based on the consistency score between candidate values, the evidence weight corresponding to the candidate values, and the credibility of the candidate values. The cross-modal semantic alignment includes at least unit normalization, symbol mapping, synonym expression merging, and semantic equivalence judgment performed by the large language model. The joint parameter update module is used to jointly model the update process of the consistency component coefficient vector and the evidence weight component coefficient vector as a constrained reinforcement learning decision process, and update the consistency component coefficient vector and the evidence weight component coefficient vector according to the feedback results. The analysis decision and review trigger module is used to select the candidate value with the highest credibility as the analysis result, and to trigger manual review and / or supplementary retrieval when the difference between the highest credibility and the second highest credibility is less than a preset threshold. The experimental data three-dimensional ledger construction module is used to organize the parsed experimental data according to the time dimension, logical dimension and knowledge dimension to form a traceable experimental data three-dimensional ledger. The intelligent retrieval and question-answering module based on the large language model is used to respond to natural language queries based on the three-dimensional ledger of the experimental data.

Images