A method for predicting amorphous forming ability based on machine learning

Abstract

The application discloses an amorphous forming ability prediction method based on machine learning, comprising the following steps: acquiring an amorphous characteristic data set and preprocessing, dividing sample data in the amorphous characteristic data set into a training set and a test set according to a set proportion; acquiring a machine learning model, wherein the machine learning model comprises three machine learning models of KNN, LightGBM and RandomForest; adopting a Stacking ensemble learning algorithm to fuse the three acquired base machine learning models, obtaining a machine learning model after Stacking integration; and using a SLSQP algorithm to weight and average the KNN model, the LightGBM model, the RandomForest model and the model after Stacking fusion, obtaining a machine learning model for predicting the amorphous forming ability; and the technical problems that the amorphous forming ability representing an amorphous alloy only aims at specific alloy components and has low prediction accuracy are solved.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of amorphous forming ability prediction, and particularly relates to an amorphous forming ability prediction method based on machine learning. BACKGROUND

[0002] Amorphous alloys have good plasticity, corrosion resistance, and firmness, and are widely used in biomedical, military aerospace, high-tech and other fields. At present, the most important bottleneck of amorphous alloy application is that its forming ability is very limited, and improving the amorphous forming ability is the primary problem to be solved in amorphous preparation. Therefore, solving the problem of amorphous forming ability in alloys can not only promote the application of amorphous alloys, but also promote the development of condensed matter physics

[0003] Amorphous forming ability is used to characterize the difficulty of alloy forming amorphous, which can be evaluated by critical cooling rate or critical casting diameter. Although the critical cooling rate is a more suitable parameter, it is difficult to obtain through experiments and thermal analysis calculation, so a large number of scholars predict the critical casting diameter of metallic glass to characterize the amorphous forming ability by proposing some standards. These standards such as Parameter, glass transition temperature T rg , supercooled liquid phase region ΔT x and the like. However, these standards are only for specific alloy compositions and have low prediction accuracy. SUMMARY

[0004] The technical problem to be solved by the application is to provide an amorphous forming ability prediction method based on machine learning, so as to solve the technical problems of existing technology that only for specific alloy compositions and have low prediction accuracy in characterizing the amorphous forming ability of amorphous alloys.

[0005] The technical scheme of the application is as follows:

[0006] An amorphous forming ability prediction method based on machine learning, the method comprising:

[0007] Step 1, obtaining an amorphous feature data set and preprocessing, the data of the data set comprising glass transition temperature T g , crystallization transition temperature T x , melting end temperature T l and critical casting diameter D max ;

[0008] Step 2, the sample data in the amorphous feature data set is divided into a training set and a test set according to a set proportion;

[0009] Step 3, obtaining a machine learning model, the machine learning model comprising three machine learning models of KNN, LightGBM and RandomForest;

[0010] Step 4: Use the Stacking ensemble learning algorithm to fuse the three base machine learning models obtained to obtain the Stacking ensemble machine learning model;

[0011] Step 5: Use the SLSQP algorithm to weighted average the KNN model, LightGBM model, RandomForest model and Stacking model to obtain a machine learning model that predicts amorphous formation capability.

[0012] The preprocessing method is to subtract the minimum value of the attribute set from the difference between the maximum and minimum values ​​of the attribute set for each data point, so that the value of each attribute is between 0 and 1.

[0013] The method for dividing the data into training and test sets according to a set ratio is as follows: the sample data in the amorphous alloy dataset is divided into training and test sets in a 7:3 ratio.

[0014] Step 4 describes the method of fusing the three base machine learning models obtained using the Stacking ensemble learning algorithm to obtain the Stacking ensemble machine learning model, which includes:

[0015] The Stacking ensemble learning framework was used to fuse the three fitted machine learning models.

[0016] Stacking ensemble learning integrates the predictions from base learners as training data for a meta-learner to improve the model's generalization ability. Stacking uses the predictions from the first-level model as input to the second level. In the first-level model training, a 5-fold cross-validation method is used to train the base model. RandomForest is used as the meta-learner, and a random forest model is trained using the prediction matrix from the first layer, mapping 9-dimensional data to 3-dimensional data. Finally, the trained random forest model is used to train the dataset. Make a prediction, and the result is the final prediction of the Stacking method on the test set.

[0017] The method for obtaining the machine learning model for predicting amorphous formation capability is as follows: a weighted average of four learners is calculated using the SLSQP algorithm. The specific process of the SLSQP algorithm is as follows:

[0018]

[0019] For predicted values, This is the actual measured value. The weights for each model are... This is the sum of squares of the differences between the measured and predicted values.

[0020] The method further includes:

[0021] Step 5: Model evaluation and testing. The model evaluation method is as follows: The model is evaluated by calculating the Pearson correlation coefficient R, root mean square error (MSE), and mean absolute error (MAE). Since there is no negative correlation between predicted and measured values, R is between [0,1], with values ​​closer to 1 indicating better model performance. MSE and MAE are also between [0,1], with values ​​closer to 0 indicating better model performance. The calculation formula is:

[0022]

[0023]

[0024]

[0025] The model testing method is as follows: the three base learners obtained after training and the model after stacking are used to make predictions on the test set respectively. The prediction results of the four models are multiplied by their corresponding weights and then summed to obtain the model's result on the test set.

[0026] The beneficial effects of this invention are:

[0027] This invention selects different types of strong machine learning models in the first layer of Stacking. Compared with traditional ensemble learning, the combination of strong heterogeneous models is beneficial to enhancing the generalization ability of the model.

[0028] Stacking uses cross-validation in the first layer, and the training data for the second layer comes from the test prediction data of the first layer instead of the training prediction data, making the model closer to the performance on the test set and thus enhancing the model's generalization ability.

[0029] The acquisition of training data in Stacking is equivalent to the process of feature extraction. The automatic feature extraction capability of the algorithm has better stability compared to manual feature extraction.

[0030] Stacking allows the meta-learner and base learner to harmonize with each other, thus achieving a balanced state for the overall model.

[0031] By employing Blending ensemble learning modeling based on the SLSQP weight optimization algorithm, the performance and generalization ability of the model are improved.

[0032] This invention features fast computation speed, high prediction accuracy, low cost, and short cycle time. The dataset built by this invention covers a wide range of elements, and the model performs excellently on the dataset, indicating that this invention has universality and is applicable to various amorphous alloys.

[0033] This solves the technical problems of existing technologies for characterizing the amorphous forming ability of amorphous alloys, such as being limited to specific alloy compositions and having low prediction accuracy. Attached Figure Description

[0034] Figure 1 A flowchart illustrating the method for establishing a machine learning model for predicting amorphous formation capability provided by the present invention;

[0035] Figure 2 The overall framework diagram of the machine learning model for predicting amorphous formation capability provided by the present invention;

[0036] Figure 3 This is a framework diagram of the Stacking integration model provided in an embodiment of the present invention;

[0037] Figure 4 a, b, and c are line graphs comparing five models provided in the embodiments of the present invention;

[0038] Figure 5 a and b are performance verification diagrams of the Blending model provided in the embodiments of the present invention. Detailed Implementation

[0039] The purpose of this invention is to provide a machine learning-based method for predicting amorphous formation capability, in order to solve the problems of low prediction rate, weak generalization ability, high cost, long cycle, and only applicable to specific systems in the existing technology by constructing the model.

[0040] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0041] like Figure 1 As shown, the present invention provides a machine learning-based method for predicting amorphous forming ability, comprising:

[0042] Step 100: Obtain the amorphous alloy dataset. The amorphous alloy dataset includes sample data of amorphous alloy materials. The sample data includes independent variables and response variables. In this invention, the sample data is selected to be 874. Due to the critical casting diameter D... max The amorphous forming ability of an alloy can be represented by its corresponding glass transition temperature T during amorphous experiments. g Crystallization transformation temperature T x Melting end temperature T l Therefore, each sample includes 4 attributes, of which 3 are independent variables (glass transition temperature T). g Crystallization transformation temperature T x Melting end temperature T lOne response variable (critical casting diameter D) max In order to find a universal technical solution, the acquired dataset covers 42 common elements in amorphous alloys.

[0043] Step 101: Divide the sample data in the amorphous alloy dataset into a training set and a test set according to a set ratio. Specifically, divide the sample data in the amorphous alloy dataset into a training set and a test set in a 7:3 ratio.

[0044] Step 102: Obtain three basic machine learning models. The three basic machine learning models include the KNN model, the LightGBM model, and the RandomForest model. These three machine learning models were selected because they demonstrated the best predictive performance in the experiments designed in this invention.

[0045] Step 103: Use the Stacking ensemble learning algorithm to fuse the three base machine learning models obtained to obtain the Stacking ensemble machine learning model, thereby improving the model's prediction accuracy and generalization ability.

[0046] Step 104: Obtain the blended model. Specifically, the obtained KNN model, LightGBM model, RandomForest model, and Stacking model are fused together and then weighted and averaged using the SLSQP algorithm. The weighted average model is the blended model, which is also the machine learning model for predicting amorphous formation capability. Ultimately, this model exhibits excellent prediction accuracy and generalization ability, with its prediction accuracy improved to 0.9040, and the root mean square error and mean absolute error reduced to 0.0090 and 0.0437, respectively.

[0047] To improve the prediction accuracy of the constructed model, this invention also uses a test set to test the machine learning model that predicts amorphous formation capability.

[0048] The following uses KNN, LightGBM, and RandomForest models as examples to illustrate a machine learning-based method for predicting the amorphous forming ability of the above-mentioned materials provided by this invention. In practical applications, other models can also be used as the base machine learning model.

[0049] Step 1: Obtain the dataset and perform data preprocessing:

[0050] First, import the amorphous alloy dataset from the database. This dataset contains 874 samples, each with four attributes. One of these attributes is the critical casting diameter D. max It quantitatively describes the amorphous forming ability of amorphous alloys; the other three properties are the glass transition temperature T.g Crystallization transformation temperature T x Melting end temperature T l This invention primarily uses a blending ensemble learning model to predict the critical casting diameter D based on the aforementioned three attributes. max .

[0051] Secondly, the amorphous alloy dataset is divided into two parts: a training set (70% of the total samples) and a test set (30%). Since there is a significant magnitude relationship among the three characteristic temperatures, attributes with larger values ​​will have a greater impact on the model than those with smaller values; therefore, data standardization is necessary. This invention selects the 0-1 standardization method to ensure that the feature variables have the same scale. For each attribute, the standardization formula is:

[0052]

[0053] in For standardized data, This is the original data. and These are the maximum and minimum values ​​in the original data.

[0054] Step 2: Establish three base models: KNN, LightGBM, and RandomForest. In this embodiment, the main idea behind establishing these three models is to build a regression model using a training set.

[0055] Step 3: Use the Stacking ensemble learning algorithm to fuse the three base models obtained.

[0056] Stacking, an ensemble learning algorithm, integrates the predictions of base learners as training data for meta-learners to improve the model's generalization ability. Compared to other ensemble learning algorithms, Stacking's process of using the first-level model's predictions as input to the second level is equivalent to an automatic feature extraction process. This reduces the uncertainty of manual feature extraction and ensures the original dataset is adequately trained, reducing bias and improving the model's generalization ability and prediction accuracy. Below is a brief description of the Stacking ensemble learning algorithm's principles:

[0057] Given a dataset of size 1000 and 3 features, the dataset is first divided into a training set and a test set in a 7:3 ratio. Figure 2 The leftmost columns are Training Data and Test Data.

[0058] In the first layer of model training, a 5-fold cross-validation method is used to train the base model. The training data is divided into five equal parts, denoted as [A1, A2, A3, A4, A5]. For each model, such as KNN, four parts are used as training data and the remaining part as test data during each training iteration. This generates five different models [KNN1, KNN2…KNN5]. The prediction results of these five different models are summed to obtain the prediction result matrix K of the KNN model, which is a 700×3 matrix. The same operation is repeated for the other two models to obtain the prediction result matrices R and L of Rf and LightGBM, respectively. These three matrices are combined to obtain a total prediction result matrix D of size 700×9, which is used as the training data for the next layer model. The purpose of this approach is to effectively utilize the data and make the results as close as possible to the model's performance on the test set rather than the training set, thereby improving the model's generalization ability.

[0059] Based on the above process, RandomForest, as a meta-learner, uses the prediction result matrix D of the first layer to train a random forest model, mapping the 9-dimensional data to 3-dimensional (the original data consists of three features).

[0060] The method for acquiring test data for the meta-learner differs slightly from that for training data, requiring additional averaging of the prediction results. Taking KNN as an example, the five KNN models trained in the first layer predict the initial meta-test set (i.e., 300 data points), resulting in five 300×3 prediction matrices. These five different results are then averaged to obtain the KNN test prediction result matrix. The other two models repeat the above operation, ultimately yielding a 300×9 prediction result matrix. .

[0061] Based on the above process, the trained random forest model is used to analyze the dataset. Make a prediction, and the result is the final prediction result of the Stacking method on the test set. To verify whether the Stacking method is effective, simply check whether the test result is better than the three base learners.

[0062] Step 4: The three base learners and the stacked ensemble model are further fused using the Blending ensemble algorithm. The ensemble process involves weighted averaging of the four learners using the SLSQP algorithm. The specific process of the SLSQP algorithm is as follows:

[0063]

[0064] in For predicted values, This is the actual measured value. The weights for each model are... Let R be the sum of squares of the differences between the measured and predicted values. The goal of the optimization algorithm is to minimize the sum of squares of the differences between the measured and predicted values, under the constraint that the sum of weights is 1 and the weights range from 0 to 1. Since the predicted and measured values ​​are not correlated, R takes values ​​between [0,1] in this experiment, with values ​​closer to 1 indicating better model performance. MSE and MAE also take values ​​between [0,1], with values ​​closer to 0 indicating better model performance.

[0065] Step 5: Model Evaluation

[0066] In this invention, model evaluation is based on its performance on the test set. Specifically, this is achieved by calculating the Pearson correlation coefficient R, root mean square error (MSE), and mean absolute error (MAE). Since predicted and measured values ​​are not negatively correlated, R is between 0 and 1, with values ​​closer to 1 indicating better model performance. Similarly, MSE and MAE are also between 0 and 1, with values ​​closer to 0 indicating better model performance. The calculation formulas are as follows:

[0067]

[0068]

[0069]

[0070] in For predicted values, This is the actual measured value. Represents the dependent variable. Represents the independent variable. This represents the expected value, or average, of the product of the independent and dependent variables. This represents the expected value of the square of the dependent variable. Represents the square of the expected value of the dependent variable. Represents the square of the expected value of the independent variable. This represents the expected value of the square of the independent variable.

[0071] Step 6: Apply the model to the test set:

[0072] The three base learners obtained after training on the training set in the above embodiments, as well as the model after stacking integration, are used to predict the test set. The prediction results of the four models are multiplied by their corresponding weights and then summed to obtain the model's result on the test set.

[0073] The following examples illustrate the effects of the five models constructed using the embodiments provided above.

[0074] Figure 4 a, b, and c are line graphs comparing the performance of the five models used in this specific example. The red line in the line graph represents the model's performance on the training set, and the gray line represents the model's performance on the test set. Points where the red and gray lines differ significantly indicate that the corresponding model has a severe overfitting problem. Figure 4 Observation shows that the model corresponding to the point where the red line and the gray line differ the least is the model after blending. That is, the robustness of the model is improved after blending, which is in line with the expectations of this study.

[0075] The reason why the performance and robustness of the model are improved after blending is mainly because the prediction result of blending is a weighted average of the prediction results of the other four models. The algorithm gives greater weight to the model with good prediction performance, that is, it enhances the effect of the strong model and weakens the effect of the weak model, thereby achieving the best overall performance.

[0076] Since the model performed exceptionally well after blending, in order to rule out the possibility that the outstanding performance was due to accidental factors, this example was repeated multiple times to observe the fluctuations in the model's performance.

[0077] Retrieving duplicate instances, for example Figure 5 As shown. Figure 5 Figure a shows the change in prediction accuracy of the model with the number of iterations across 100 repeated instances. The vertical axis represents the Pearson correlation coefficient, and the horizontal axis represents the number of repeated experiments. The figure shows that the model's score varied between 0.87 and 0.98 across these 100 repeated instances, with the model score mostly fluctuating around the average of 0.93. To make the effect of Figure a more obvious and intuitive, this example also includes... Figure 5 The residual plot of figure a is... Figure 5 Image b. In Figure 5 In Figure b, the horizontal axis represents the number of iterations, and the vertical axis represents the difference between the prediction accuracy of the model in 100 repeated instances and the average prediction accuracy. The figure shows that the model's prediction accuracy fluctuated around an average of 0.93 during these 100 repeated instances, with a maximum error range of approximately [-0.05, 0.05], and no outliers with significant deviations were observed. The repeated instance results fully demonstrate the model's stability; even at its worst, it still achieved a prediction accuracy of approximately 87%, representing a significant improvement compared to traditional criteria and traditional machine learning algorithms.

Claims

1. A machine learning-based method for predicting amorphous forming ability, characterized in that: The method includes: Step 1: Obtain and preprocess the amorphous feature dataset. The dataset includes the glass transition temperature T. g Crystallization transformation temperature T x Melting end temperature T l and critical casting diameter D max ; Step 2: Divide the sample data in the amorphous feature dataset into a training set and a test set according to a set ratio; Step 3: Obtain the machine learning model, which includes three machine learning models: KNN, LightGBM, and RandomForest. Step 4: Use the Stacking ensemble learning algorithm to fuse the three base machine learning models obtained to obtain the Stacking ensemble machine learning model; Step 5: Use the SLSQP algorithm to weighted average the KNN model, LightGBM model, RandomForest model and Stacking model to obtain a machine learning model that predicts amorphous formation capability. The method for obtaining the machine learning model for predicting amorphous formation capability is as follows: a weighted average of four learners is calculated using the SLSQP algorithm. The specific process of the SLSQP algorithm is as follows: （1）； （2）； （3）； For predicted values, This is the actual measured value. The weights for each model are... This is the sum of squares of the differences between the measured and predicted values.

2. The amorphous forming ability prediction method based on machine learning according to claim 1, characterized in that: The preprocessing method is to subtract the minimum value of the attribute set from the difference between the maximum and minimum values ​​of the attribute set for each data point, so that the value of each attribute is between 0 and 1.

3. The amorphous forming ability prediction method based on machine learning according to claim 1, characterized in that: The method for dividing the data into training and test sets according to a set ratio is as follows: the sample data in the amorphous alloy dataset is divided into training and test sets in a 7:3 ratio.

4. The amorphous forming ability prediction method based on machine learning according to claim 1, characterized in that: Step 4 describes the method of fusing the three base machine learning models obtained using the Stacking ensemble learning algorithm to obtain the Stacking ensemble machine learning model, which includes: The Stacking ensemble learning framework was used to fuse the three fitted machine learning models. Stacking ensemble learning integrates the predictions from base learners as training data for a meta-learner to improve the model's generalization ability. Stacking uses the predictions from the first-level model as input to the second level. In the first-level model training, a 5-fold cross-validation method is used to train the base model. RandomForest is used as the meta-learner, and a random forest model is trained using the prediction matrix from the first layer, mapping 9-dimensional data to 3-dimensional data. Finally, the trained random forest model is used to train the dataset. Make a prediction, and the result is the final prediction of the Stacking method on the test set.

5. The amorphous forming ability prediction method based on machine learning according to claim 1, characterized in that: The method further includes: Step 5: Model evaluation and testing. The model evaluation method is as follows: The model is evaluated by calculating the Pearson correlation coefficient R, root mean square error (MSE), and mean absolute error (MAE). Since there is no negative correlation between predicted and measured values, R is between [0,1], with values ​​closer to 1 indicating better model performance. MSE and MAE are also between [0,1], with values ​​closer to 0 indicating better model performance. The calculation formula is: （4）； （5）； （6）； The model testing method is as follows: the three base learners obtained after training and the model after stacking are used to make predictions on the test set respectively. The prediction results of the four models are multiplied by their corresponding weights and then summed to obtain the model's result on the test set.

Images