Solid waste cementing material performance prediction method based on instruction fine tuning large language model

Abstract

The invention discloses a solid waste cementing material performance prediction method based on an instruction fine-tuning large language model, and the method comprises the steps: converting components, process parameters and performance indexes in collected solid waste material training data into standardized text description, and taking the standardized text description as the input of a model; calculating the similarity between the materials based on the material component fingerprints and the microstructure information, and selecting a sample with high similarity as a few-sample example; constructing a hybrid instruction set; an instruction fine tuning technology is adopted to adapt a pre-trained large language model, so that the pre-trained large language model deduces a prediction result of target performance in a few-sample or zero-sample scene; and the precision and the stability of model output are optimized through structure perception example sorting and a post-processing module. The method aims to solve the problems of scarcity of annotated data and insufficient model generalization ability, and is suitable for material performance prediction tasks in few-sample and zero-sample scenes.

Description

Technical Field

[0001] The present invention relates to the technical field of solid waste gelling material performance prediction, and in particular to a solid waste gelling material performance prediction method based on an instruction fine-tuning large language model. Background Art

[0002] In the field of solid waste cementitious material performance prediction, traditional technologies mainly rely on traditional machine learning models such as support vector machines and decision trees, or physics-based models such as finite element analysis. However, these methods have significant limitations: First, they are highly data-dependent and require a large amount of labeled data. However, solid waste material experiments are expensive and data acquisition is difficult, which seriously restricts model training efficiency. Second, their generalization ability is weak, making it difficult to adapt to the actual application scenarios of solid waste materials with complex and variable compositions and widely varying preparation processes. Third, they lack multi-task adaptability, requiring separate training models for different performance prediction tasks such as strength and durability, resulting in high R&D costs and low efficiency. In recent years, large language models (LLMs) have demonstrated powerful few-shot / zero-shot learning capabilities in natural language processing. However, their application in materials science remains exploratory, particularly in the performance prediction of solid waste cementitious materials in few-shot and zero-shot scenarios, lacking targeted optimization for material domain characteristics. Among existing technologies, instruction fine-tuning can guide models to learn specific tasks through task instructions, but it has not yet been combined with structural feature analysis of solid waste cementitious materials, making it difficult to fully realize the model's potential for predicting material properties. Similarity retrieval technology has some applications in recommendation systems and molecular property prediction, but it has not yet been introduced to the solid waste material performance prediction scenario, preventing effective correlation analysis of complex material systems. Given the shortcomings of traditional methods in data utilization efficiency, scenario adaptability and multi-tasking capabilities, as well as the limitations of the combination of existing language model technology and materials science, there is an urgent need to propose an innovative method that integrates instruction fine-tuning technology and material field characteristics to break through the technical bottleneck of solid waste cementitious material performance prediction. Summary of the Invention

[0003] The purpose of the present invention is to provide a solid waste cementitious material performance prediction method based on an instruction fine-tuning large language model, to solve the problems of strong data dependence, weak generalization ability, poor multi-task adaptability and insufficient numerical regression accuracy of traditional models, to achieve performance prediction in few-sample / zero-sample scenarios through instruction fine-tuning technology, to enhance adaptability to complex scenarios by combining material composition and structural similarity retrieval, to achieve multi-task unified modeling by using hybrid instruction set design, and to improve the accuracy and stability of numerical regression results through structure-aware examples and post-processing modules.

[0004] The technical solution adopted by the present invention is: a method for predicting the properties of solid waste cementitious materials based on instruction fine-tuning of a large language model, comprising the following steps: Step 1, data collection, collecting solid waste material training data; Step 2: Data preprocessing: converting the composition, process parameters, and performance indicators of solid waste materials into standardized text descriptions as input to the model; Step 3, structural similarity retrieval, calculates the similarity between materials based on material composition fingerprints and microstructure information, and selects samples with high similarity as minority sample examples; Step 4: Construct a hybrid instruction set, including zero-shot instructions and few-shot instructions, where the zero-shot instructions contain task descriptions, and the few-shot instructions combine similar material examples with target material task instructions; Step 5: Model fine-tuning: Use instruction fine-tuning technology to adapt the pre-trained large language model so that it can infer the target performance prediction results in few-shot or zero-shot scenarios; Step 6, inference optimization, optimizes the accuracy and stability of the model output through structure-aware example sorting and post-processing modules.

[0005] Furthermore, the training data in step 1 is derived from experimental data, literature data or a public material database, and covers at least one type of solid waste of fly ash, slag and heavy metal-containing waste residue.

[0006] Furthermore, the process parameters in step 2 include at least one of calcination temperature, mixing ratio, water-binder ratio, alkali activator concentration, curing temperature and curing time; and the performance indicators include at least one of compressive strength, acid resistance, durability and frost resistance.

[0007] Furthermore, the small number of sample examples in step 3 are sorted based on at least one of material composition similarity, process parameter similarity, or microstructure similarity, and samples with high similarity are input first.

[0008] Furthermore, the material composition fingerprint in step 3 includes at least one of element ratio, chemical bond type, and mineral phase composition, and is input into the model through standardized natural language text.

[0009] Furthermore, the proportion of the mixed instruction set in step 4 is: zero-sample instructions account for 30%-50%, and few-sample instructions account for 50%-70%.

[0010] Furthermore, the instruction fine-tuning in step 5 adopts QLoRA technology to freeze most of the model parameters and only fine-tune the low-rank adapter.

[0011] Furthermore, the base model of the large language model in step 5 is LLaMA2 or its variant, which supports the generation of long text input and numerical regression tasks.

[0012] Furthermore, the method supports multi-task unified modeling, and a single model can simultaneously predict at least two performance indicators among compressive strength, acid resistance, durability and frost resistance.

[0013] Furthermore, in the inference phase of step 5, the optimal number of examples is determined through sensitivity analysis, and the number of examples ranges from 2-shot to 4-shot.

[0014] The beneficial effects of the present invention are as follows: in a small sample scenario with only 5-10 labeled samples, the prediction error (RMSE) is reduced by 20%-30%, significantly improving data utilization efficiency; a single model can simultaneously predict multiple performance indicators such as strength and durability, avoiding repeated training and reducing training costs; for solid waste materials with complex components such as heavy metal waste slag, the prediction accuracy is improved by more than 15%, enhancing the model's adaptability to complex and changing scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 Flow chart of the method of the present invention. DETAILED DESCRIPTION

[0016] The present invention will be further described below with reference to the accompanying drawings.

[0017] like Figure 1 As shown, the present invention is a method for predicting the properties of solid waste cementitious materials based on a large language model fine-tuned by instructions, comprising the following steps: Step 1: Data collection: collect solid waste material training data; the training data comes from experimental data, literature data or public material databases, covering at least one type of solid waste among fly ash, slag, and heavy metal-containing waste slag.

[0018] Step 2, data preprocessing, converts the composition, process parameters and performance indicators of the solid waste material into standardized text descriptions as input to the model; the process parameters are calcination temperature or mixing ratio; the performance indicators include at least one of compressive strength, acid resistance, durability and frost resistance.

[0019] Step 3, structural similarity retrieval, calculates the similarity between materials based on the material composition fingerprint and microstructure information, and selects samples with high similarity as minority sample examples.

[0020] The material composition fingerprint includes at least one of element ratio, chemical bond type, and mineral phase composition, and is input into the model through standardized natural language text (i.e., converting material composition and parameters into a structured description with semantic readability).

[0021] The few-sample examples are sorted based on at least one of material composition similarity, process parameter similarity, or microstructure similarity, with high-similarity samples being input first.

[0022] Step 4: Build a hybrid instruction set, including zero-shot instructions and few-shot instructions, with zero-shot instructions accounting for 30%-50% and few-shot instructions accounting for 50%-70%; the zero-shot instructions contain task descriptions, and the few-shot instructions combine similar material examples with target material task instructions.

[0023] Step 5: Model fine-tuning. Using the QLoRA instruction-based fine-tuning technique, we adapt the base model to LLaMA2 or a similar open-source large language model. This freezes most of the model parameters and fine-tunes only the low-rank adapter to reduce GPU memory usage, enabling it to achieve target performance inference in few-shot or zero-shot scenarios. LLaMA2 or similar open-source models support the generation of long text inputs and numerical regression tasks.

[0024] The inference phase determines the optimal number of examples or batch size, such as 2-shot to 4-shot, through sensitivity analysis.

[0025] Step 6, inference optimization, optimizes the accuracy and stability of the model output through structure-aware example sorting and post-processing modules.

[0026] The present invention supports multi-task unified modeling, and a single model can simultaneously predict at least two performance indicators among compressive strength, acid resistance, durability and frost resistance.

[0027] To quickly predict the compressive strength of an alkali-activated slag-bentonite system under different microwave conditions, the following intelligent prediction example based on a command-based fine-tuning large language model was constructed to provide an example of microwave-cured strength prediction for a slag-bentonite pre-driven geopolymer.

[0028] (1) Original data and standardized input construction: The structural parameters obtained from the experiment are as follows: Material mass ratio: bentonite: slag = 1:2, bentonite calcined at 750℃; Activator: NaOH, concentration 8wt%; Maintenance method: Microwave treatment, 900W, 8 minutes; Performance indicators: 28-day compressive strength is 56MPa; The parameters are converted into standardized natural language input as follows: "A geopolymer is a mixture of bentonite calcined at 750°C and slag in a ratio of 1:2. After adding 8% NaOH as an activator, it is microwaved at 900W for 8 minutes. Please predict its 28-day compressive strength." (2) Example configuration for a few sample instructions: Example 1: "The composition is bentonite calcined at 700°C and slag in a ratio of 1:2, with an 8% NaOH addition. Under the conditions of microwave curing at 900W for 5 minutes, the compressive strength is 51.3MPa." Example 2: "Bentonite is calcined at 850°C, mixed with slag, the activator is 6% NaOH, and the curing method is room temperature for 28 days. The ultimate compressive strength is 43.2 MPa." (3) Model configuration and output: Use QLoRA to fine-tune the LLaMA2 model, train for 3 epochs, with a learning rate of 2e-5 and a batch size of 16; Instruction set ratio: zero-sample instructions 30%, few-sample instructions 70%; The number of samples was set to 3-shot, with priority sorting based on material composition and calcination temperature; The model inference output is: "The predicted compressive strength of the sample after 28 days is 56.1 ± 1.0 MPa. The microwave power and bentonite activity enhance the initial reaction rate and improve the density of the CASH gel, which is significantly better than the room temperature curing strength of approximately 43.9 MPa." (4) Performance verification and comparison: After fine-tuning, the LLM model prediction RMSE is ±1.2 MPa, which is better than the traditional gradient boosting tree model (RMSE is about 2.5 MPa).

[0029] Using structure sorting + Shapley factor interpretation, it is clear that the contribution rates of "bentonite calcination temperature" and "microwave power" are both over 30%.

[0030] The error between the model prediction results and the actual experiments is less than ±2%, which supports engineering application scenarios under small sample conditions.

[0031] (5) Ablation experiment verification: In order to verify the contribution of each submodule of the present invention to the model performance, the following ablation experiment is constructed: The experimental setup is as follows: Model A (full): using structural similarity ranking with a mixed instruction set, RMSE = 1.2 MPa, R² = 0.92; Model B: removes the structural ordering and only randomly selects samples, RMSE=2.0MPa, R²=0.85; Model C: Remove mixed instructions and use only zero samples, RMSE=3.1MPa, R²=0.71; Model D: Remove all enhancement modules and use only zero-sample task descriptions, RMSE=3.8MPa, R²=0.64.

[0032] The analysis results show that both structural sorting and hybrid instruction sets have a significant effect on improving model performance, and their combined use can obtain the optimal prediction results, verifying the effectiveness of this invention in data-scarce scenarios.

Claims

1. A method for predicting the properties of solid waste cementitious materials based on a large language model fine-tuned by instructions, characterized in that: The following steps are involved: Step 1, data collection, collecting solid waste material training data; Step 2: Data preprocessing: converting the composition, process parameters, and performance indicators of solid waste materials into standardized text descriptions as input to the model; Step 3, structural similarity retrieval, calculates the similarity between materials based on material composition fingerprints and microstructure information, and selects samples with high similarity as minority sample examples; Step 4: Construct a hybrid instruction set, including zero-shot instructions and few-shot instructions, where the zero-shot instructions contain task descriptions, and the few-shot instructions combine similar material examples with target material task instructions; Step 5: Model fine-tuning: Use instruction fine-tuning technology to adapt the pre-trained large language model so that it can infer the target performance prediction results in few-shot or zero-shot scenarios; Step 6, inference optimization, optimizes the accuracy and stability of the model output through structure-aware example sorting and post-processing modules.

2. The method for predicting properties of solid waste cementitious materials based on instruction fine-tuning large language model according to claim 1 is characterized in that: The training data in step 1 is derived from experimental data, literature data or a public material database, and covers at least one type of solid waste of fly ash, slag and heavy metal-containing waste residue.

3. The method for predicting properties of solid waste cementitious materials based on instruction fine-tuning large language model according to claim 1 is characterized in that: The process parameters in step 2 include at least one of calcination temperature, mixing ratio, water-binder ratio, alkali activator concentration, curing temperature and curing time; the performance indicators include at least one of compressive strength, acid resistance, durability and frost resistance.

4. The method for predicting properties of solid waste cementitious materials based on instruction fine-tuning large language model according to claim 1 is characterized in that: The small number of sample examples in step 3 are sorted based on at least one of material composition similarity, process parameter similarity, or microstructure similarity, and samples with high similarity are input first.

5. The method for predicting properties of solid waste cementitious materials based on instruction fine-tuning large language model according to claim 1 is characterized in that: The material composition fingerprint in step 3 includes at least one of element ratio, chemical bond type, and mineral phase composition, and is input into the model through standardized natural language text.

6. The method for predicting properties of solid waste cementitious materials based on instruction fine-tuning large language model according to claim 1 is characterized in that: The proportion of the mixed instruction set in step 4 is: zero-sample instructions account for 30%-50%, and few-sample instructions account for 50%-70%.

7. The method for predicting properties of solid waste cementitious materials based on instruction fine-tuning large language model according to claim 1 is characterized in that: The instruction fine-tuning in step 5 uses QLoRA technology to freeze most of the model parameters and only fine-tune the low-rank adapter.

8. The method for predicting properties of solid waste cementitious materials based on instruction fine-tuning large language model according to claim 1 is characterized in that: The base model of the large language model in step 5 is LLaMA2 or its variants, which supports the generation of long text input and numerical regression tasks.

9. The method for predicting properties of solid waste cementitious materials based on instruction fine-tuning large language model according to claim 1, characterized in that: The method supports multi-task unified modeling, and a single model can simultaneously predict at least two performance indicators among compressive strength, acid resistance, durability and frost resistance.

10. The method for predicting properties of solid waste cementitious materials based on instruction fine-tuning large language model according to claim 1, characterized in that: In the inference phase of step 5, the optimal number of examples is determined through sensitivity analysis, ranging from 2-shot to 4-shot.

Images