Material preparation process data causal quality evaluation method and system under large language model collaboration

Abstract

The invention discloses a material preparation process data causal quality evaluation method and system under large language model collaboration, and the method comprises the steps: building a whole-flow process directed graph of "component-structure-process-performance", and achieving the structuralization and semantic representation of data; then fusing a graph neural network and causal reasoning, quantifying an individual causal effect of process parameters on material performance based on anti-fact intervention, and accurately identifying key quality control points; a large language model three-stage cooperation mechanism is introduced, domain knowledge priori generation, reasonable anti-fact construction and result semantic calibration are achieved, and intelligence and reliability of the evaluation process are ensured; according to the method, the limitation of traditional correlation analysis is broken through, the crossing from shallow anomaly detection to deep causal diagnosis is realized, the material process data quality management level and the optimization decision efficiency are improved, and data support and quality guarantee are provided for digital twinning construction.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of intelligent management and quality evaluation of material data, more specifically, relates to a material preparation process data causal quality evaluation method and system under the cooperation of a large language model. BACKGROUND

[0002] With the deepening of national strategies such as the material genome project and intelligent manufacturing, the new mode of data-driven material research and development has become an industry trend. Material preparation process data, as a key bridge connecting material "composition-structure" and "performance", directly determines the accuracy of computational prediction and the efficiency of design iteration. However, the current quality management and evaluation methods of material process data are seriously lagging behind the explosive growth of data size, mainly existing the following bottlenecks and challenges: First, the problem of multi-source heterogeneity and structural deficiency of process data is prominent. Material preparation process data usually comes from experimental records, production logs, scientific literature and various databases, with different formats, granularities and specifications. The existing storage and management methods are mainly based on unstructured text description or simple table, which cannot effectively express the time sequence logic between process steps, the causal relationship between parameters and states, and the complex constraint relationship across processes. This chaotic state of structure makes it difficult for machines to deeply understand and automatically process data, greatly hindering the sedimentation, reuse and intelligent mining of process knowledge.

[0003] Second, the existing quality evaluation methods are shallow and lack causal insight. Currently, the quality evaluation of process data mainly relies on rule-based verification (such as threshold judgment, format checking) or traditional statistical methods (such as outlier detection). Although these methods can identify obvious and superficial data anomalies (such as numerical out-of-range, record missing), they cannot deeply diagnose the rationality of the process logic behind the data. For example, they cannot judge whether "a certain heat treatment heating rate is matched with the target phase change mechanism" or "a certain component adjustment necessarily leads to performance changes". This "correlation rather than causality" evaluation mode cannot find deep and logical quality defects, and cannot meet the internal requirements of high-reliability material research and development for data quality.

[0004] Third, it is difficult to effectively integrate domain knowledge into the automated evaluation process. Material process design contains deep scientific principles and expert experience, but these valuable knowledge exists in an implicit form, which is difficult to formalize and quantify. Although large language models (LLM) represent artificial intelligence technology that exhibits strong natural language processing capabilities, it still has limitations in understanding graph structure data and complex scientific reasoning. Directly applying LLM for quality evaluation is easily disturbed by the "illusion" problem, resulting in content that contradicts facts or scientific principles, and the reasoning process is uncontrollable, making it difficult to ensure the reliability and credibility of the evaluation results.

[0005] Fourth, the evaluation results lack explainability and are difficult to guide practice. Existing data quality evaluation tools often output simple "pass / fail" conclusions or abstract quality scores, lacking depth of explanation and specific optimization guidance for problem sources. Material engineers cannot learn "why is this process parameter setting unreasonable" and "how to adjust to improve data quality and process level" from the results, making it difficult to form a quality evaluation closed loop and fully release the value of data.

[0006] In summary, there is an urgent need for an intelligent quality evaluation method that can deeply integrate material domain knowledge, understand the internal logic of the process, and provide causal explanations to solve the above problems and enable high-quality management and efficient use of material data. SUMMARY

[0007] The present application provides a large language model collaborative material preparation process data causal quality evaluation method and system to address the problems mentioned in the background art in any aspect of structured characterization, causal quality insight, domain knowledge integration, and result explainability of material preparation process data. The method breaks through the limitations of traditional quality evaluation methods based on correlation analysis, achieving a technical leap from shallow anomaly detection to deep causal diagnosis, significantly improving the quality management level of material process data and the decision-making efficiency of process optimization.

[0008] To solve the above technical problems, the present application provides a large language model collaborative material preparation process data causal quality evaluation method, including the following steps: Step S1, pre-processing the raw material preparation process data in the material field, and constructing a process directed graph based on the pre-processed data; Step S2, training a graph neural network model based on the process directed graph constructed in step S1, constructing a counterfactual process directed graph through counterfactual intervention, comparing the performance prediction difference between the original graph (i.e. the process directed graph constructed in step S1) and the counterfactual process directed graph, quantifying individual causal effects and generating causal evaluation indicators; Step S3, construct a three-stage evaluation system under the cooperation of a large model, generate semantic prior knowledge through LLM-I, construct a reasonable counterfactual sample through LLM-II, and perform semantic calibration on the causal evaluation result through LLM-III; Step S4, integrate the quantitative results and semantic analysis content of the foregoing steps to generate a structured quality report containing multi-dimensional evaluation information, visual elements and interactive design, and provide problem positioning, causal explanation and optimization suggestions.

[0009] As a further improvement of the first aspect of the application, the step of constructing a directed graph of the process is as follows: 1) Establish a standardized four-layer node classification system, including process step nodes, process parameter nodes, material state nodes and performance index nodes; 2) Use rule engine and natural language processing technology to parse the original data, and automatically or semi-automatically extract instances of the above four types of nodes; 3) On the basis of node instantiation, according to the principles of materials science and process logic, establish the directed connection between nodes to form a graph structure; 4) Convert the graph structure into an adjacency matrix and a node feature matrix to complete digital storage and serialization.

[0010] As a further improvement of the first aspect of the application, the step S2 specifically comprises the following steps: Step S2.1, select a suitable graph neural network model, and train the graph neural network using the process directed graph and corresponding performance data constructed in step S1; Step S2.2, select an intervention target based on domain knowledge or feature importance analysis, modify the target parameter node feature value through do operation to generate a counterfactual process directed graph; Step S2.3, input the process directed graph constructed in step S1 and the counterfactual process directed graph constructed in step S2.2 into the model trained in step S2.1, respectively, to obtain corresponding performance prediction values, calculate the difference between the two performance prediction values, and quantify the individual causal effect; Step S2.4: statistically significant evaluation and reasonableness verification, domain knowledge verification and causal path analysis are performed on the calculated causal effect.

[0011] As a further improvement of the first aspect of the application, the graph neural network model is selected from GCN and GAT.

[0012] As a further improvement of the first aspect of the application, the step S3 of generating semantic prior knowledge through LLM-I is as follows: Step S3.1.1, structured Prompt construction, design instructions with clear roles, tasks and output formats; Step S3.1.2, submit the Prompt to the large language model for knowledge generation and extraction, use the vast amount of materials science literature and knowledge in LLM-I to generate the required analysis results rich in field semantics; Step S3.1.3, analyze the output of LLM-I, convert the key assertions in it into structured knowledge triples, and store them in the temporary priori knowledge base to provide query basis for LLM-II and LLM-III.

[0013] As a further improvement of the first aspect of the application, the step S3, the step of LLM-II constructing a rationalized counterfactual sample is as follows: Step S3.2.1, combine the semantic priori generated by LLM-I with specific material preparation process data to construct a new Prompt; Step S3.2.2, LLM-II makes inferences under the constraints of field knowledge according to the instructions, generates modification suggestions, and provides a brief rationalized explanation for each suggestion for LLM-III to check; Step S3.2.3, automatically accept the suggestions of LLM-II, and modify the characteristic values of the corresponding process parameter nodes based on the original process directed graph according to the do-algorithm rules, thereby instantiating one or more counterfactual process directed graphs.

[0014] As a further improvement of the first aspect of the application, the step S3, the step of LLM-III performing semantic calibration on the causal evaluation results is as follows: Step S3.3.1, integrate the preliminary results to be calibrated, the corresponding process directed graph and counterfactual process directed graph, and the priori knowledge of LLM-I and the rationalized explanation of LLM-II together and submit them to LLM-III; Step S3.3.2, LLM-III cross-comparisons the causal conclusions with the field common sense to determine whether the conclusions conflict with the known scientific laws, and reviews whether the rationalized explanation of LLM-II is valid; Step S3.3.3, if the verification is passed, LLM-III outputs a "confirmation" signal and an enhanced explanation, and the causal relationship is marked as high confidence; if conflicts or ambiguities are found, LLM-III will output specific calibration opinions, and feed them back to step 3.2.3 to trigger a new round of counterfactual construction and evaluation, or note them in the form of "warning" in the final report for experts to focus on.

[0015] As a further improvement of the first aspect of the application, the step S4 specifically comprises the following steps: Step S4.1, through data aggregation and key information extraction, systematically integrate the outputs of all upstream modules to prepare raw materials for report generation; Step S4.2, LLM-assisted narrative generation through structured report template definition, utilizing pre-designed report templates and the copywriting capabilities of large language models to transform data into natural language narratives; Step S4.3, graphical elements to enhance the readability and interactivity of the report through causal graph visualization, effect comparison charts, and interactive element embedding. Step S4.4, through versioned storage and optimized iteration tracking, ensure that the report can continuously evolve and synchronize with the process optimization process.

[0016] On the other hand, the present application also provides a material preparation process data causal quality evaluation system under the cooperation of a large language model, which includes the following functional modules: Process directed graph construction module: used for preprocessing the original material preparation process data, and converting the "composition-structure-process-performance" whole process of material preparation into a machine understandable graph structure representation, i.e. process directed graph. The process directed graph contains four types of nodes: process step nodes, process parameter nodes, material state nodes and performance index nodes, and three types of edge relationships: time sequence edges, association edges and causal edges. The module finally outputs a preparation process directed graph represented by a node feature matrix. This graph not only records the whole process of the process, but also reveals the internal relationship between parameters, states and performance, laying a solid data foundation for intelligent evaluation of the whole system.

[0017] Causal reasoning and quantitative evaluation module: based on the process directed graph, the embedding representation of nodes and graphs is learned through graph neural network, the causal reasoning theory is introduced and counterfactual intervention is implemented, the individual causal effect of each key process parameter on material performance is quantified, and a set of causal evaluation indexes is output. The causal evaluation index set clearly indicates which process parameters are the key causal factors affecting material performance, and how their influence direction and strength are, providing core and quantitative causal evidence for the final quality evaluation report.

[0018] Large language model collaborative evaluation module: this module realizes the integration of domain knowledge and the calibration of evaluation results through a three-stage collaborative mechanism, which specifically includes the LLM-I unit for generating semantic priors, the LLM-II unit for constructing counterfactuals, and the LLM-III unit for semantic calibration.

[0019] Report generation module: integrates the output results of the aforementioned modules to automatically generate a structured quality report containing quality scores, problem positioning, causal analysis, optimization suggestions and confidence evaluation, and supports multi-modal visualization and interactive presentation.

[0020] Compared with the prior art, the present application has the following significant advantages: This invention provides a causal quality evaluation method for materials and processes data under the collaboration of a large language model. It achieves structured and semantic representation of data by constructing a directed graph of the entire process flow from "composition to structure to process to performance." It integrates graph neural networks and causal reasoning theory, quantifies the individual causal effects of process parameters on performance through counterfactual intervention, identifies key quality control points, and introduces a three-stage collaborative mechanism of the large language model to sequentially complete the generation of domain knowledge priors, the construction of rational counterfactual evidence, and the semantic calibration of results. Finally, it generates a quality report with semantic interpretation capabilities, accurately locating problems, elucidating causal mechanisms, and providing actionable process optimization suggestions. This method overcomes the shortcomings of traditional quality evaluation methods in causal inference and knowledge fusion, achieving a leap from shallow anomaly detection to deep causal diagnosis, significantly improving the level of process quality management and the efficiency of optimization decisions. Attached Figure Description

[0021] Figure 1 This is a schematic diagram of the overall process of the material preparation process data causal quality evaluation method under the large language model collaboration of the present invention. Figure 2 Directed graph construction for the material preparation process of this invention; Figure 3 This is a flowchart of the causal quality evaluation model of the present invention; Figure 4 This is a schematic diagram of the multi-stage collaborative evaluation process of the large language model of the present invention; Detailed Implementation To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings and specific embodiments. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0022] like Figure 1 As shown, the present invention provides a causal quality evaluation method for material process data under the collaboration of a large language model. Its core process covers the entire chain of processing from raw data input to final quality report generation, specifically including the following steps: Step S1: Data preprocessing and normalization construction of the directed process graph.

[0023] First, the raw data from experimental records, production logs, and databases underwent data cleaning, feature engineering, and standardization. Specifically: Data cleaning involves addressing missing values, outliers, and inconsistencies in the raw data. For example, it includes filling in or marking missing key process parameters and standardizing numerical units and terminology.

[0024] Feature engineering includes deep processing of data, including one-hot encoding of categorical features (such as heat treatment type), converting text descriptions into numerical vectors, and generating derived features based on domain knowledge (such as calculating phase transition critical temperature).

[0025] Standardization mainly includes anomaly detection, and the application uses statistical methods and machine learning algorithms to preliminarily identify potential outliers in the data, providing targets for subsequent focused analysis.

[0026] Subsequently, based on the pre-processed standard data, the "composition-structure-process-performance" whole process of material preparation is mapped into a standardized process directed graph.

[0027] The standardization construction method of the process directed graph includes the following steps: Node instantiation: According to the pre-defined four-layer node system, create node instances from the data. Including process step nodes (such as melting, heat treatment), process parameter nodes (such as temperature, time), material state nodes (such as grain size, microstructure) and performance index nodes (such as strength, plasticity).

[0028] Edge relationship establishment and graph structure generation: According to the principle of materials science, establish the directed connection between nodes, the edge relationship includes time sequence edge, which is used to represent the execution order of material preparation process steps, association edge, which is used to represent the parameters of the step, and causal edge, which is used to represent the driving relationship of parameters to material state or performance, such as "quenching rate → residual stress → fatigue performance"). Based on node instantiation, according to the principle of materials science and process logic, the directed connection between nodes is established to form the graph structure.

[0029] The construction of the process directed graph is the data basis and premise of the method of the application. The construction of the graph structure can convert multi-source heterogeneous, unstructured original process data into machine understandable, semantically rich structured representation. This conversion process solves the long-standing "data island" and "semantic gap" problems in the field of material data, providing a unified, standardized data carrier for subsequent causal reasoning and intelligent evaluation. In addition, for easy calculation and processing, the constructed process directed graph is converted into a standard mathematical representation, including the construction of an adjacency matrix, the construction of a node feature matrix, storage and serialization, and finally outputting a preparation process directed graph represented by an adjacency matrix and a node feature matrix. This graph not only records the whole process of the process, but also reveals the internal relationship between parameters, states and performance, laying a solid data foundation for the intelligent evaluation of the whole system.

[0030] Step S2, causal reasoning framework quantitative evaluation.

[0031] This step aims to go beyond traditional correlation analysis quality evaluation methods by introducing causal inference theory, which can accurately identify and quantify the real causal effect of process parameters on material properties, providing scientific and reliable quantitative basis for root cause diagnosis of quality problems and process optimization. Specifically, refer to Figure 3 , comprising the following steps: Step S2.1, graph neural network model construction and training.

[0032] The user selects a suitable graph neural network model (such as GCN, GAT) from the pre-defined model library, and trains the model using the process directed graph constructed in step S1 and its corresponding performance data. The model training goal is to enable GNN to accurately learn the complex structure and attribute relationship in the process directed graph, and realize accurate prediction of material performance (such as strength, elongation). After training, the GNN model becomes a reliable predictor that can understand the process-performance correlation, thus a graph neural network model facing the process directed graph can be constructed as the infrastructure for prediction and representation.

[0033] Step S2.2, counterfactual intervention and sample construction. Based on the trained graph neural network model, counterfactual samples are constructed through intervention operations to simulate the scenario of "if a parameter is changed, how will the performance change". Among them, the intervention target selection: based on domain knowledge (which can interact with LLM-I module) or feature importance analysis, select the key process parameter node to be evaluated as the intervention target (such as "solid solution temperature"). Perform do operation: apply do operation (such as do (solid solution temperature = 400℃)) to the selected intervention variable, that is, modify the feature value of the target node while keeping all other node features and structures in the graph unchanged, thus generating a counterfactual process directed graph.

[0034] Step S2.3, individual causal effect calculation and quantification. By comparing the output difference of the original graph and the counterfactual process directed graph in the GNN model, the individual causal effect is quantified. Obtain the prediction result: input the original process directed graph and the counterfactual process directed graph into the trained GNN model to obtain the corresponding performance prediction value. Calculate the causal effect: for continuous performance indicators, calculate the individual causal effect (ICE), the calculation formula is as follows: ICE = MI(do (intervened parameter); ) - MI(do (original parameter; ), Where MI is mutual information, is the performance prediction value. This process can accurately identify the key quality control points that have a real causal effect on the final material performance. In addition, the mutual information change (ΔMI) can be calculated to measure the change in information quantity brought by the intervention, and the causal strength can be evaluated from another dimension.

[0035] Step S2.4, Causal Evaluation Index Generation and Output. The system summarizes, sorts and performs significance tests on the ICE and AMI calculated for all evaluated process parameters, and finally outputs a set of causal evaluation indexes. This index set clearly indicates which process parameters are key causal factors affecting material performance, how their influence direction and strength are, and provides core, quantitative causal evidence for the final quality evaluation report.

[0036] Through step S2, the deep causal relationship in the material preparation process data is mined and measured, and the quality evaluation is improved from the "what" correlation to the "why" causality.

[0037] Step S3, construction of three-stage evaluation system under the cooperation of large model.

[0038] Step S3 aims to safely and controllably integrate domain knowledge into the automated evaluation process, and avoid the "illusion" risk of large models. In combination with Figure 4 , the present application creatively integrates the semantic understanding and reasoning ability of large language models into the quality evaluation process, effectively solving the bottleneck problem of difficulty in formalizing the integration of domain knowledge and lack of semantic explanation of evaluation results in traditional methods, and ensuring that the entire evaluation process is both data-driven and knowledge-guided. Referring to Figure 4 , the method for constructing a three-stage evaluation system under the cooperation of a large model includes the following steps: Step S3.1, semantic prior generation (LLM-I).

[0039] This step converts the structured information of the process directed graph into natural language description as a prompt word input to the large language model. The specific processing process: LLM based on its internalized material science knowledge, performs deep analysis on the process path. The output result: generate structured semantic prior knowledge including key process control point identification, material organization evolution prediction and potential quality risk warning. These outputs are parsed and stored in the domain knowledge base of the system, laying a foundation for subsequent analysis, including the following steps: 1) Construction of structured Prompt. Design instructions with clear roles, tasks and output formats.

[0040] 2) Knowledge generation and extraction. Submit the Prompt to the large language model. LLM-I generates analysis results that meet the requirements and are rich in domain semantics based on its internalized massive material science literature and knowledge.

[0041] 3) Priori knowledge base update. The output of LLM-I is parsed, and the key assertions (e.g., "The solution temperature of AZ91D alloy is usually between 400-430°C") are converted into structured knowledge triples and stored in a temporary priori knowledge base, providing query basis for LLM-II and LLM-III.

[0042] Step S3.2, counterfactual construction (LLM-II).

[0043] This step is based on the semantic priori generated by LLM-I in step S3.1 and the specific user quality problem, guiding the large language model to conduct counterfactual reasoning. Processing process: LLM conducts rationalized reasoning according to the logic of "if changing X parameter, it may have Z influence on Y performance". Output result: generate process parameter modification suggestions with physical and chemical rationality (for example, "adjust the solution temperature from 415°C to 400°C"), and provide brief mechanism explanation for each suggestion. These suggestions are automatically converted by the system into executable counterfactual intervention instructions.

[0044] Specifically includes the following steps: 1) Intervention target prompt. Combine the semantic priori generated by LLM-I (such as the list of key parameters) with specific process data to construct a new Prompt.

[0045] 2) Rationalized counterfactual generation. LLM-II conducts reasoning according to the instructions under the constraints of domain knowledge, generating modification suggestions (such as "400°C"). The system will require LLM-II to provide a brief rationalization explanation (such as "lower temperature can inhibit grain growth") for each suggestion, in preparation for LLM-III verification.

[0046] 3) Counterfactual graph instantiation. The system automatically receives the suggestions of LLM-II, and according to the do-calculus rules, modifies the characteristic values of the corresponding process parameter nodes based on the original process directed graph, thereby instantiating one or more counterfactual process directed graphs.

[0047] Step S3.3, semantic calibration (LLM-III).

[0048] Integrate the causal evaluation preliminary results output in step S2 with the counterfactual suggestions of LLM-II, and submit them to the large language model for final verification. Processing process: LLM plays the role of "field expert reviewer", and judges the rationality of the causal relationship based on the principles of materials science. Output result: give confirmation signal and supplementary mechanism explanation for scientific and reasonable causal relationship; give calibration opinions (such as "this modification may lead to excessive loss of strength, suggest adjusting the aging process simultaneously") for conclusions with doubts or conflicts; give confidence evaluation for all conclusions. These calibrated results will be the core basis for the final quality report.

[0049] The objective of this step is to perform a final check on the preliminary causal assessment results, identify and correct potential scientific errors or LLM "hallucinations", and improve the reliability of the conclusions. This includes the following steps: 1) Multi-source information integration and presentation. The preliminary results to be calibrated, the corresponding original / counterfactual process maps, and the prior knowledge of LLM-I and the rationalized explanations of LLM-II are integrated together and submitted to LLM-III.

[0050] 2) Semantic consistency check. LLM-III plays the role of "reviewer" and cross-checks the causal conclusions with domain common sense. It will determine whether the conclusions conflict with known scientific laws (for example, the general law that strength and ductility are inversely related), and whether the rationalized explanations of LLM-II are valid.

[0051] 3) Feedback optimization and report generation. If the check is passed, LLM-III outputs a "confirmation" signal and enhanced explanations, and the causal relationship is marked as high confidence. If conflicts or ambiguities are found, LLM-III will output specific calibration suggestions (such as "this modification may result in excessive loss of strength, it is recommended to consider adjusting the time-effective process simultaneously"). The system will feed back this suggestion to step S3.2, triggering a new round of counterfactual construction and evaluation, or noting it in the form of a "warning" in the final report for experts to focus on.

[0052] Step S4, quality report generation with semantic interpretation capability.

[0053] This step aims to solve the problem of poor interpretability of traditional quality evaluation results and inability to guide practice. The system automatically integrates the quantitative results of causal effects and the semantic analysis generated by LLM to generate a structured comprehensive quality evaluation report. This report not only contains quantitative quality scores, but also clearly explains the following in natural language form: problem positioning, causal analysis, optimization suggestions, and confidence evaluation.

[0054] Specifically, the present application converts the complex data, causal analysis and semantic insights generated by the previous modules into a comprehensive quality evaluation report that is clear in structure, clear in conclusion, and has high interpretability and strong action guidance significance. The report directly faces material engineers and decision makers, seamlessly connecting the results of intelligent analysis to actual R&D and production optimization processes. Report generation includes the following steps: Step S4.1, multi-source information integration and key insight extraction. The system automatically gathers and correlates all key outputs from steps S1 to S3 to prepare raw materials for report generation, including: Information Integration: Receive the standardized process directed graph from step S1 processing; the causal effect quantification indicators (such as ICE values of each parameter, significance levels) from step S2 processing; and the semantic analysis generated by LLM from step S3 processing (including prior knowledge, counterfactual suggestions, calibration opinions and confidence evaluation).

[0055] Insight Extraction: Based on preset rules and algorithms, automatically extract core conclusions from integrated information, for example: (1) Identify key problems: Locate process parameters with high ICE values, statistical significance, and high LLM confidence as main quality problem points.

[0056] (2) Extract core causal chains: Summarize the most significant causal paths, such as "insufficient solid solution temperature → insufficient γ' phase dissolution → decreased yield strength".

[0057] (3) Summarize feasible suggestions: Organize reasonable counterfactual modification schemes generated by LLM-II and confirmed by LLM-III.

[0058] Step S4.2, structured report automatic generation. Based on pre-defined, comprehensive report templates, the system automatically generates the final quality evaluation report, which includes the following aspects: Template-driven: The report adopts a standardized chapter structure to ensure the completeness and standardization of the content. The standardized chapter structure mainly includes: (1) Executive summary: Summarize the overall quality level, the most core quality problems and their expected impact in the most concise language.

[0059] (2) Problem positioning and causal analysis: Elaborate on key process parameters, their causal effect size and direction, and their role in the process graph, and embed the deep machine interpretation provided by LLM.

[0060] (3) Data quality score: Give a quantitative comprehensive quality score (such as 85 / 100) and sub-item scores from multiple dimensions such as "causal consistency", "semantic rationality" and "process specification".

[0061] (4) Optimization suggestions and expected benefits: List specific and executable process parameter adjustment schemes (such as "extend the solid solution time from 12 hours to 16 hours"), and attach the performance change value predicted based on GNN counterfactual prediction (such as "expected strength can be improved by about 15MPa").

[0062] (5) Risk warning and confidence: Clearly point out the uncertainties and assumptions in the analysis, and show the calibration records and overall confidence provided by LLM-III.

[0063] Step S4.3, multimodal visualization and interactive presentation. To enhance the readability and utility of the report, the system encapsulates key information in graphical form. Specifically, it includes: Causal diagram highlights: automatically generate subgraphs of the process directed graph and highlight the most critical problem paths using arrows of different colors and thicknesses.

[0064] Effect comparison chart: integrate bar charts to visually display the causal effect ranking of different process parameters.

[0065] Prediction comparison chart: use radar charts or column charts to display the performance prediction comparison before and after the implementation of the optimization scheme.

[0066] Interactive design: in the digital report interface, users can click on professional terms (such as "γ' phase"), parameters or chart elements in the report to view more detailed data sources, definitions or LLM extended explanations in the form of a floating card, achieving deep drilling of knowledge.

[0067] Step S4.4, report output and iterative tracking.

[0068] Report output: the system finally outputs a complete and standardized report (supporting formats such as PDF, HTML, etc.) and automatically archives it to the database, associating it with the corresponding process directed graph, model parameters and LLM dialogue records, ensuring traceability throughout the process.

[0069] Iterative tracking: when users implement process optimization based on the report recommendations, they can feed back new experimental data to the system.

[0070] The system can generate an iterative comparison report to clearly show the improvement of quality problems and the evolution of new and old causal patterns, forming a closed-loop quality management process of "evaluation-optimization-verification-re-evaluation".

[0071] In addition, the present application also provides a material preparation process data causal quality evaluation system for performing the above steps, which includes the following functional modules: Process directed graph construction module: used for preprocessing the original material preparation process data and converting the "composition-structure-process-performance" whole process of material preparation into a machine understandable graph structure representation, i.e. process directed graph. The process directed graph contains four types of nodes: process step nodes, process parameter nodes, material state nodes and performance index nodes, as well as three types of edge relationships: time sequence edges, association edges and causal edges. The module finally outputs a preparation process directed graph represented by a node feature matrix. This graph not only records the whole process of the process, but also reveals the internal relationship between parameters, states and performance, laying a solid data foundation for intelligent evaluation of the whole system.

[0072] Causal inference and quantitative evaluation module: based on the process directed graph, learn the embedding representation of nodes and graphs through graph neural network, introduce causal inference theory and implement counterfactual intervention, quantify the individual causal effect of each key process parameter on material performance, and output a set of causal evaluation indexes. The causal evaluation index set clearly points out which process parameters are the key causal factors affecting material performance, how their influence direction and strength are, and provides core, quantitative causal evidence for the final quality evaluation report.

[0073] Large language model collaborative evaluation module: this module realizes the integration of domain knowledge and the calibration of evaluation results through a three-stage collaborative mechanism, including the LLM-I unit for generating semantic priors, the LLM-II unit for counterfactual construction, and the LLM-III unit for semantic calibration.

[0074] Report generation module: integrates the output results of the preceding modules, automatically generates a structured quality report containing quality scores, problem positioning, causal analysis, optimization suggestions and confidence evaluation, and supports multi-modal visualization and interactive presentation.

[0075] The system of the present application, the process directed graph construction module can effectively solve the problem that the internal logic and correlation of data are difficult for machines to understand. Secondly, in the causal inference and quantitative evaluation module, the causal inference framework is introduced to go beyond surface anomaly detection and realize quantitative evaluation of the causal contribution of process parameters. Thirdly, in the large language model collaborative evaluation module, the LLM three-stage collaborative mechanism is constructed to safely and reliably integrate domain knowledge into the automated evaluation process. Finally, the report generation module integrates all the output results of the preceding modules to generate a quality report with semantic interpretation ability, providing direct and feasible action guidelines for process optimization, thereby bridging the "last mile" of material data quality management.

[0076] Although embodiments of the present application have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made therein without departing from the principles and spirit of the present application, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A causal quality evaluation method for material preparation process data under the collaborative framework of large language models, characterized by: The method includes the following steps: Step S1: Preprocess the raw material preparation process data in the field of materials, and construct a directed process graph based on the preprocessed data; Step S2: Based on the directed process graph constructed in Step S1, train a graph neural network model, construct a counterfactual process directed graph through counterfactual intervention, compare the performance prediction differences between the directed process graph and the counterfactual process directed graph, quantify individual causal effects, and generate causal evaluation indicators. Step S3: Construct a three-stage evaluation system under the collaboration of large models. Generate semantic prior knowledge through LLM-I, construct rationalized counterfactual samples through LLM-II, and perform semantic calibration on causal evaluation results through LLM-III. Step S4: Integrate the quantitative results and semantic analysis content from the previous steps to generate a structured quality report containing multi-dimensional evaluation information, visualization elements, and interactive design, providing problem identification, causal explanation, and optimization suggestions.

2. The method for evaluating the causal quality of material preparation process data under the collaborative approach of a large language model as described in claim 1, characterized in that: The steps to construct a directed graph of the process are as follows: 1) Establish a standardized four-level node classification system, including process step nodes, process parameter nodes, material state nodes, and performance index nodes; 2) Using rule engines and natural language processing technology, the raw data is parsed to automatically or semi-automatically extract instances of the above four types of nodes; 3) Based on node instantiation, and in accordance with the principles of materials science and process logic, directed connections are established between nodes to form a graph structure; 4) Transform the graph structure into an adjacency matrix and a node feature matrix to complete digital storage and serialization.

3. The method for evaluating the causal quality of material preparation process data under the collaborative approach of a large language model as described in claim 1, characterized in that: Step S2 specifically includes the following steps: Step S2.1: Select a suitable graph neural network model and train the graph neural network using the process directed graph and corresponding performance data constructed in step S1. Step S2.2: Select intervention targets based on domain knowledge or feature importance analysis, modify the feature values ​​of target parameter nodes through do operations, and generate a counterfactual process directed graph; Step S2.3: Input the directed process graph constructed in step S1 and the counterfactual directed process graph constructed in step S2.2 into the model trained in step S2.1 respectively, obtain the corresponding performance prediction values, calculate the difference between the two performance prediction values, and quantify the individual causal effect. Step S2.4: Perform statistical significance assessment and rationality verification, domain knowledge verification, and causal path analysis on the calculated causal effects.

4. The method for evaluating the causal quality of material preparation process data under the collaborative approach of a large language model as described in claim 3, characterized in that: The graph neural network models used are GCN and GAT.

5. The method for evaluating the causal quality of material preparation process data under the synergy of large language models according to any one of claims 1-4, characterized in that: Step S3, which involves generating semantic prior knowledge using LLM-I, is as follows: Step S3.1.1: Construct a structured Prompt, designing instructions with clearly defined roles, tasks, and output formats; Step S3.1.2: Submit the Prompt to the large language model for knowledge generation and extraction. Utilize the massive amount of materials science literature and knowledge within LLM-I to generate analysis results that meet the requirements and are rich in domain semantics. Step S3.1.3: Parse the output of LLM-I, convert the key assertions into structured knowledge triples, and store them in a temporary prior knowledge base to provide a basis for querying LLM-II and LLM-III.

6. The method for evaluating the causal quality of material preparation process data under the collaborative approach of a large language model as described in claim 5, characterized in that: Step S3, the steps of LLM-II in constructing rationalized counterfactual samples, are as follows: Step S3.2.1: Combine the semantic priors generated by LLM-I with the specific material preparation process data to construct a new Prompt; Step S3.2.2: LLM-II, based on the instructions and constrained by domain knowledge, performs reasoning to generate modification suggestions and provides a brief rational explanation for each suggestion for LLM-III verification. Step S3.2.3: Automatically accept the LLM-II suggestion and, based on the do-calculus rules, modify the eigenvalues ​​of the corresponding process parameter nodes on the basis of the original process directed graph, thereby instantiating and generating one or more counterfactual process directed graphs.

7. The method for evaluating the causal quality of material preparation process data under the collaborative approach of a large language model as described in claim 6, characterized in that: Step S3, the semantic calibration of the causal evaluation results by LLM-III, is as follows: Step S3.3.1: Integrate the preliminary results to be calibrated, the corresponding process directed graph and counterfactual process directed graph, as well as the prior knowledge of LLM-I and the rationalization explanation of LLM-II, and submit them to LLM-III. Step S3.3.2: Cross-compare the causal conclusions with common sense in the field using LLM-III to determine whether the conclusions conflict with known scientific laws and examine whether the rational explanations of LLM-II are valid. Step S3.3.3: If the verification passes, LLM-III outputs a "confirmation" signal and an enhanced interpretation, and the causal relationship is marked as high confidence. If a conflict or ambiguity is found, LLM-III will output specific calibration opinions and feed these opinions back to step 3.2.3, triggering a new round of counterfactual construction and evaluation, or it may be noted in the final report as a "warning" for experts to review.

8. The method for evaluating the causal quality of material preparation process data under the synergy of large language models according to any one of claims 1-4, characterized in that: Step S4 specifically includes the following steps: Step S4.1: Through data aggregation and key information extraction, systematically integrate the outputs of all upstream modules to prepare raw materials for report generation; Step S4.2: Through the definition of structured report templates and LLM-assisted narrative generation, data is transformed into natural language narratives by utilizing pre-designed report templates and the copywriting generation capabilities of large language models. Step S4.3: Enhance the readability and interactivity of the report by using cause-and-effect diagram visualization, effect comparison charts, and interactive element embedding; Step S4.4: Ensure that the report can continuously evolve and keep pace with the process optimization process by versioned storage and optimized iteration tracking.

9. A causal quality evaluation system for material preparation process data under the collaboration of a large language model, characterized in that: The system for performing the method as described in any one of claims 1-8 includes the following functional modules: Process Directed Graph Construction Module: Used to preprocess the raw material preparation process data and transform the entire process of material preparation, from "composition-structure-process-performance", into a machine-understandable graph structure representation, i.e., a process directed graph; Causal reasoning and quantitative evaluation module: Based on the directed graph of the process, the embedding representation of nodes and graph is learned through graph neural network, causal reasoning theory is introduced and counterfactual intervention is implemented to quantify the individual causal effects of each key process parameter on material properties and output a set of causal evaluation indicators. Large Language Model Collaborative Evaluation Module: This module achieves the integration of domain knowledge and calibration of evaluation results through a three-stage collaborative mechanism, specifically including the LLM-I unit for generating semantic priors, the LLM-II unit for constructing counterfactual information, and the LLM-III unit for semantic calibration. Report generation module: Integrates the output of the aforementioned modules to automatically generate a structured quality report that includes quality scores, problem identification, causal analysis, optimization suggestions, and confidence assessment, and supports multimodal visualization and interactive presentation.

Images