C-Mn Steel Mechanical Property Prediction Method Based on Convolutional Feature Fusion of Multimodal Data

Abstract

The present invention discloses a method for predicting the mechanical properties of C-Mn steel based on the convolutional feature fusion of multi-modal data, belonging to the technical field of steel property prediction, and comprising the following steps: constructing an initial multi-modal data set; constructing an actual multi-modal data set; dividing the actual multi-modal data set into a training set and a test set; using a convolutional neural network model with convolutional feature fusion of multi-modal data to calculate the training data, optimizing the model parameters, obtaining a prediction model for the mechanical properties of C-Mn steel to be predicted, calculating the test data, and evaluating the generalization performance of the model, so as to obtain a prediction model for the mechanical properties of C-Mn steel based on the convolutional feature fusion of multi-modal data to be applied. The present invention adopts the above-mentioned method for predicting the mechanical properties of C-Mn steel based on the convolutional feature fusion of multi-modal data, effectively couples the component process and the microstructure image data information through convolutional neural network models of different dimensions, and improves the prediction accuracy of the mechanical properties of C-Mn steel.

Description

Technical Field

[0001] The present invention relates to the technical field of steel property prediction, and particularly to a method for predicting the mechanical properties of C-Mn steel based on convolutional feature fusion of multi-modal data. Background Art

[0002] During the steel manufacturing process, due to the influence of different chemical composition contents and processing technologies, the material will undergo continuous and complex chemical and physical changes, and there is a coupling effect among different chemical compositions, production processes, and microstructural organizations. This coupling effect directly affects the mechanical properties of the steel. The mechanical properties of steel mainly include yield strength (YS), tensile strength (TS), and elongation (EL). Therefore, establishing an accurate and quantitative relationship model between chemical composition, processing technology, and microstructure and mechanical properties is crucial for steel design and process optimization.

[0003] Currently, some researchers have numericalized the microstructural features to establish an intelligent prediction of mechanical properties based on composition, process, and numerically characterized microstructural feature data. For example, Chinese Patent No. CN117219206A discloses a method for predicting the mechanical properties of metal materials based on multi-model fusion. This method performs weighted fusion on the mechanical property prediction output values of a neural network and the mechanical property prediction output values screened based on data significance features to improve the prediction accuracy of mechanical properties. However, this mechanical property prediction model based on structured data-driven is difficult to capture the abstract and implicit cross-information in the data, resulting in poor model prediction performance. In addition, with the development of convolutional neural networks, in 2022, Wang et al. and in 2023, Ren et al. used fully connected layers to splice the composition-process and microstructural image convolutional features to establish a mechanical property prediction model based on multi-modal data of composition-process and microstructural images. However, this modeling method mainly simply splices the microstructural convolutional features extracted by the convolutional neural network and the composition-process information in the fully connected layer, which is similar to the current structured data-driven modeling method and does not fully consider the microstructural morphology and structured tissue features, resulting in low model prediction accuracy. Summary of the Invention

[0004] The object of the present invention is to provide a method for predicting the mechanical properties of C-Mn steel based on convolutional feature fusion of multi-modal data, which effectively couples the composition process and microstructural image data information through convolutional neural network models of different dimensions, and improves the prediction accuracy of the mechanical properties of C-Mn steel.

[0005] To achieve the above object, the present invention provides a method for predicting the mechanical properties of C-Mn steel based on convolutional feature fusion of multi-modal data, including the following steps:

[0006] S1. Collect specimens prepared from C-Mn steel of different grades, obtain metallographic images of the specimens prepared from C-Mn steel using a metallographic microscope, and form an initial multi-modal dataset D by combining the corresponding composition data, process data, and mechanical property data collected. p ;

[0007] S2. Preprocess the composition data, process data, and metallographic images, and form an actual multi-modal dataset D by combining the preprocessed data with the mechanical property data. T ;

[0008] S3. Divide the actual multi-modal dataset D in step S2 T into a training set and a test set.

[0009] S4. Use a convolutional neural network model with multi-modal data convolution feature fusion to calculate the training data in the training set, optimize the model parameters, and thus obtain a prediction model for the mechanical properties of the C-Mn steel to be predicted.

[0010] S5. Use the prediction model for the mechanical properties of the C-Mn steel to be predicted to calculate the test data in the test set, and evaluate the generalization performance of the model through evaluation metrics, and finally obtain a prediction model for the mechanical properties of C-Mn steel based on multi-modal data convolution feature fusion to be applied.

[0011] Preferably, in step S1, the composition data includes C, Si, Mn, P, S, Al, Nb, Ti, Cr, Mo, B, Cu, Ni, N, Sb; the process data includes tapping temperature, rough rolling exit temperature, finish rolling inlet temperature, finish rolling exit temperature, coiling temperature; the mechanical property data includes yield strength, tensile strength, and elongation;

[0012] The initial multi-modal dataset D p is obtained by corresponding the metallographic images with the composition data, process data, and mechanical property data one by one.

[0013] Preferably, in step S2, preprocess the composition data, process data, and metallographic images, and form an actual multi-modal dataset D by combining the preprocessed data with the mechanical property data T , and the specific operation is as follows:

[0014] S21. Calculate the linear correlations between the composition data, process data, and mechanical property data respectively using the Pearson correlation coefficient, rank the importance of the composition processes, and select the top N composition data and process data as the preprocessed composition data and process data;

[0015] S22. For the initial multi-modal dataset D pCrop the metallographic image in it to obtain a metallographic image that meets the requirements of the model input matrix size, which is the preprocessed metallographic image;

[0016] S23. Correspond the preprocessed metallographic image with the preprocessed composition data, preprocessed process data, and mechanical property data one by one to obtain the actual multi-modal dataset D T .

[0017] Preferably, in step S4, the specific operation is as follows:

[0018] S41. Construct a one-dimensional convolutional neural network model A to extract features from the preprocessed composition data and preprocessed process data to obtain composition and process convolutional features;

[0019] S42. Construct a two-dimensional convolutional neural network model B to extract features from the preprocessed metallographic image to obtain metallographic image convolutional features;

[0020] S43. Perform a dot product on the composition and process convolutional features obtained in step S41 and the metallographic image convolutional features obtained in step S42 to achieve the fusion of multi-modal data convolutional features and obtain the fused multi-modal data convolutional features;

[0021] S44. Use a two-dimensional convolutional neural network model D to further perform feature fusion and extraction on the fused multi-modal data convolutional features in step S43;

[0022] S45. Use the MSELoss loss function to calculate the loss between the predicted value and the true value of the mechanical properties, and continuously iterate to optimize the model parameters to obtain the final model parameter combination.

[0023] Preferably, the one-dimensional convolutional neural network model A includes four one-dimensional convolutional modules. Among them, the first three one-dimensional convolutional modules are each composed of a Conv1d, BatchNorm1d, ReLU, and MaxPool1d. The number of convolutional kernels is 8, 16, and 32 in sequence. The size of the convolutional kernels is 3×3, the stride is 1, and the padding is 1; the size of the MaxPool1d kernel is 2×2, and the stride is 2;

[0024] The fourth one-dimensional convolutional module is composed of a Conv1d, BatchNorm1d, ReLU, and AdaptiveAvgPool1d. Among them, the number of convolutional kernels is 128, the size of the convolutional kernels is 3×3, the stride is 1, and the padding is 1; the size of the AdaptiveAvgPool1d kernel is 1×1.

[0025] Preferably, the two-dimensional convolutional neural network model B includes 3 two-dimensional convolutional modules, and each of the 3 two-dimensional convolutional modules is respectively composed of a Conv2d, BatchNorm2d, ReLU, and MaxPool2d. Among them, the numbers of convolutional kernels are 16, 32, and 64 respectively, the size of the convolutional kernels is 3×3, the stride is 1, the padding is 1, and the kernel size of MaxPool2d is 2×2, and the stride is 2.

[0026] Preferably, the two-dimensional convolutional neural network model D includes 3 two-dimensional convolutional modules and 1 AdaptiveAvgPool2d layer. The first two two-dimensional convolutional modules are respectively composed of a Conv2d, BatchNorm2d, ReLU, and MaxPool2d. Among them, the numbers of convolutional kernels are 128 and 256 respectively, the sizes of the convolutional kernels are both 3×3, the strides are both 1, and the paddings are both 1; the kernel size of MaxPool2d is 2×2, and the stride is 2;

[0027] The AdaptiveAvgPool2d layer has a kernel size of 1×1;

[0028] The third two-dimensional convolutional module is a Conv2d layer, the size of its convolutional kernel is 1×1, the stride is 1, the number of convolutional kernels is 1, and the padding is 1.

[0029] Preferably, in step S5, the evaluation indexes include root mean square error RMSE and determination coefficient R 2 .

[0030] Therefore, the present invention adopts the above-mentioned method for predicting the mechanical properties of C-Mn steel based on convolutional feature fusion of multi-modal data, and its technical effects are as follows:

[0031] (1) The present invention breaks the traditional modeling method of numericalizing the microstructure image to establish the relationship between composition-process and microstructure image features and mechanical properties, extracts convolutional features from the morphological distribution and tissue structure of the microstructure image, and improves the prediction accuracy of the mechanical properties of C-Mn steel.

[0032] (2) Compared with the current method of convolving the composition process and microstructure image features in the fully connected layer, the present invention uses a one-dimensional convolutional neural network model to extract convolutional features from the more abstract cross-information of the composition process, effectively improving the prediction accuracy of the mechanical properties of C-Mn steel.

[0033] The technical solution of the present invention will be further described in detail below with reference to the drawings and embodiments. Description of the Drawings

[0034] Figure 1Schematic diagram of the process of a method for predicting the mechanical properties of C-Mn steel based on convolutional feature fusion of multimodal data according to the present invention;

[0035] Figure 2 It is a bar chart of RMSE of the method proposed by the present invention and the RF and FCLDS models; among them, Figure 2 (a) in is the RMSE comparison result of the yield strength (YS) and the tensile strength (TS); Figure 2 (b) in is the RMSE comparison result of the elongation (EL);

[0036] Figure 3 It is the R 2 bar chart of the method proposed by the present invention and the RF and FCLDS models. Detailed implementation manners

[0037] The technical solutions of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0038] Unless otherwise defined, the technical terms or scientific terms used in the present invention shall have the ordinary meanings understood by those of ordinary skill in the art to which the present invention belongs.

[0039] Embodiment 1

[0040] As Figure 1 shown, it is a schematic diagram of the process of a method for predicting the mechanical properties of C-Mn steel based on convolutional feature fusion of multimodal data according to the present invention, including the following steps:

[0041] S1. Data acquisition

[0042] Select C-Mn steel samples of different grades from a steel plant, obtain the metallographic images (microstructure images) of the samples with a metallographic microscope, and form an initial multimodal dataset D p with the collected composition data, process data and mechanical property data, and the data volume N = 100.

[0043] The composition data parameters include C, Si, Mn, P, S, Al, Nb, Ti, Cr, Mo, B, Cu, Ni, N, Sb;

[0044] The process parameters are specifically the tapping temperature (SRT), rough rolling exit temperature (RDT), finish rolling entry temperature (FET), finish rolling exit temperature (FDT), and coiling temperature (CT);

[0045] The metallographic images are obtained by observing and photographing the samples after grinding, polishing and etching with a metallographic microscope to obtain RGB three-channel images with a size of 5472×3648;

[0046] The output mechanical property parameters include yield strength (YS), tensile strength (TS), and elongation (EL).

[0047] The metallographic images are corresponded one by one with the composition data, process data, and mechanical property data to obtain the initial multi-modal dataset D p .

[0048] S2. Data preprocessing

[0049] S21. Calculate the linear correlation between the composition process and mechanical properties using the Pearson correlation coefficient, rank the importance of the composition process, and screen out the composition and process parameter values with a correlation greater than 0.5, specifically C, Si, Mn, P, S, Al, RDT, FDT, and CT. The calculation formula of the Pearson correlation coefficient is:

[0050]

[0051] Among them, n represents the number of samples; i represents the serial number of the sample; x i represents the data of the i-th sample x; y i represents the data of the i-th sample y; is the expectation of sample x; is the expectation of sample y;

[0052] S22. Crop the metallographic images, crop the metallographic images with a size of 5472×3648 into 1024×1024 to obtain the actual multi-modal dataset D T .

[0053] S3. Data division; Divide the actual multi-modal dataset D T randomly into a training set and a test set, and the ratio is 80%:20%, that is, the data volume of the training set is 80, and the data volume of the test set is 20.

[0054] S4. Establish a multi-modal data convolution feature fusion model

[0055] Figure 1 Shows the flow of the mechanical property prediction method of C-Mn steel based on multi-modal data convolution feature fusion. Among them, L represents the length of a convolution feature, C represents the number of convolution features, A represents a one-dimensional convolutional neural network model, B and D represent two-dimensional convolutional neural network models, and W and H represent the width and height of the two-dimensional convolution module respectively.

[0056] S41: Construct a one-dimensional convolutional neural network model A to extract features from the preprocessed composition data and preprocessed process data to obtain composition and process convolution features;

[0057] The one-dimensional convolutional neural network model includes four one-dimensional convolutional blocks. Among them, the one-dimensional convolution module A1 -A3 is respectively composed of a Conv1d, a BatchNorm1d, a ReLU, and a MaxPool1d. Among them, the number of convolutional kernels is 8, 16, and 32 respectively, the size of the convolutional kernel is 3, the stride is 1, and the padding is 1; the size of the kernel of MaxPool1d is 2×2, and the stride is 2. The one-dimensional convolutional module A4 consists of a Conv1d, a Batch Norm1d, a ReLU, and an AdaptiveAvgPool1d. Among them, the number of convolutional kernels is 128, the size of the convolutional kernel is 3×3, the stride is 1, and the padding is 1. The size of the kernel of AdaptiveAvgPool1d is 1×1. Finally, a component process data with a length of 9 becomes (128,1) after feature extraction by the one-dimensional convolutional neural network model.

[0058] S42. Construct a two-dimensional convolutional neural network model B to extract features from the preprocessed metallographic image and obtain the convolutional features of the metallographic image;

[0059] The two-dimensional convolutional neural network model B includes 3 two-dimensional convolutional blocks. Among them, the two-dimensional convolutional module B 1 -B 3 is respectively composed of convolutional blocks each consisting of a Conv2d, a BatchNorm2d, a ReLU, and a MaxPool2d. Among them, the number of convolutional kernels is 16, 32, and 64 respectively, the size of the convolutional kernel is 3×3, the stride is 1, the padding is 1, and the size of the kernel of MaxPool2d is 2×2, and the stride is 2.

[0060] Therefore, a three-channel metallographic image with a size of 1024×1024 becomes a convolutional feature matrix with a size of (64, 128, 128) after feature extraction by the two-dimensional convolutional neural network.

[0061] S43. Perform dot multiplication on the convolutional feature matrix with a size of (128,1) obtained in step S41 and the convolutional feature matrix with a size of (64, 128, 128) obtained in step S42 to achieve the fusion of multi-modal data convolutional features. The size of the fused matrix is (64, 128, 128).

[0062] S44. Use the two-dimensional convolutional neural network model D to further perform feature fusion and extraction on the multi-modal data convolutional features fused in step S43;

[0063] The two-dimensional convolutional neural network model D includes 3 two-dimensional convolutional blocks and 1 AdaptiveAvgPool2d layer. Among them, the two-dimensional convolutional module D 1 and D 2It is composed of convolutional blocks each consisting of a Conv2d, a BatchNorm2d, a ReLU, and a MaxPool2d. Among them, the number of convolutional kernels is 128 and 256 respectively, the size of the convolutional kernel is 3×3, the stride is 1, and the padding is 1; the kernel size of the MaxPool2d is 2×2 and the stride is 2. D 3 is an AdaptiveAvgPool2d layer with a kernel size of 1×1. The two-dimensional convolutional module D 4 is a Conv2d layer with a convolutional kernel size of 1×1, a stride of 1, the number of convolutional kernels is 1, and the padding is 1. The matrix size of the predicted output value of the model's mechanical properties is (1,1,1).

[0064] S45. Use the MSELoss loss function to calculate the loss between the predicted mechanical property value and the true value, and continuously iterate and optimize the model parameters to obtain a combination of model parameters with high training accuracy. The MSELoss loss function is:

[0065] l(x,y) = {l 1 ,l 2 ,...,l N} T ,l n = (x n - y n ) 2

[0066] Among them, l N represents the data sample l of the Nth batch; x n represents the predicted value; y n represents the true value; l n represents the squared value of the difference between the nth sample x and y.

[0067] S5. Use the mechanical property prediction model obtained in step S4 to calculate the test data in the test set, and use the Root Mean Square Error (RMSE) and the Coefficient of determination (R 2 ) to evaluate the generalization performance of the model.

[0068] The calculation formulas of RMSE and R 2 are as follows:[[]]

[0069]

[0070] Among them, n is the number of samples; i represents the sample number; x i represents the data of the ith sample x; f(x i ) and y iThey respectively represent the predicted value and the true value of the i-th sample.

[0071]

[0072] The above dataset was tested using the Random Forest (RF) and the Full Connection Layer Data Splice (FCLDS) model. The test results are as Figure 2 shown. Figure 2 (a) in Figure 2 shows the RMSE of the yield strength (YS) and the tensile strength (TS) of the three models on the test set. It can be seen that the RMSE values of YS and TS of the method proposed in the present invention are the lowest; Figure 3 (b) in 2 shows the RMSE of the elongation (EL) of the three models on the test set. Compared with the other two models, the RMSE value of the method proposed in the present invention is the lowest and the accuracy is the highest. In addition, Figure 3 (c) 2 shows the R 2 statistical results of the three models on the test set. It can be seen that the R 2 value of the method proposed in the present invention is the highest, indicating that the model has strong generalization ability.

[0073] Therefore, the present invention adopts the above-mentioned method for predicting the mechanical properties of C-Mn steel based on multi-modal data convolution feature fusion, effectively couples the composition process and the microstructure image data information through convolutional neural network models of different dimensions, and improves the prediction accuracy of the mechanical properties of C-Mn steel.

[0074] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that they can still modify or equivalently replace the technical solutions of the present invention, and these modifications or equivalent replacements cannot make the modified technical solutions deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. A method for predicting mechanical properties of C-Mn steel based on multimodal data convolution feature fusion, characterized in that: The following steps are involved: S1. Collect samples of C-Mn steel of different grades, obtain metallographic images of the samples using a metallographic microscope, and combine them with the collected corresponding composition data, process data and mechanical property data to form an initial multimodal data set D p ; S2. Preprocess the composition data, process data and metallographic images, and combine the preprocessed data with the mechanical properties data to form an actual multimodal data set D T ; S3: The actual multimodal dataset D in step S2 T Divide into training set and test set; S4. Calculate the training data in the training set using a convolutional neural network model that fuses multimodal data convolution features, optimize model parameters, and thereby obtain a prediction model for mechanical properties of the C-Mn steel to be predicted; S5. Calculate the test data in the test set using the mechanical property prediction model of the C-Mn steel to be predicted, and evaluate the generalization performance of the model through evaluation indicators, and finally obtain the mechanical property prediction model of the C-Mn steel based on multi-modal data convolution feature fusion to be applied; In step S4, the specific operations are: S41, constructing a one-dimensional convolutional neural network model A, performing feature extraction on the preprocessed component data and the preprocessed process data, and obtaining component and process convolution features; S42, constructing a two-dimensional convolutional neural network model B, performing feature extraction on the preprocessed metallographic image, and obtaining convolution features of the metallographic image; S43, performing a dot multiplication on the composition and process convolution features obtained in step S41 and the metallographic image convolution features obtained in step S42 to achieve multimodal data convolution feature fusion and obtain a fused multimodal data convolution feature; S44, using a two-dimensional convolutional neural network model D to further perform feature fusion and extraction on the multimodal data convolution features fused in step S43; S45. Use the MSELoss loss function to calculate the loss between the predicted value and the true value of the mechanical properties, continuously iterate and optimize the model parameters to obtain the final model parameter combination.

2. According to claim 1, a method for predicting mechanical properties of C-Mn steel based on multimodal data convolution feature fusion is characterized in that: In step S1, the composition data includes C, Si, Mn, P, S, Al, Nb, Ti, Cr, Mo, B, Cu, Ni, N, and Sb; the process data includes furnace temperature, rough rolling outlet temperature, finishing rolling inlet temperature, finishing rolling outlet temperature, and coiling temperature; the mechanical property data includes yield strength, tensile strength, and elongation; Initial multimodal dataset D p It is obtained by one-to-one correspondence between the metallographic image and the composition data, process data and mechanical property data.

3. The method for predicting mechanical properties of C-Mn steel based on multimodal data convolution feature fusion according to claim 2 is characterized in that: In step S2, the composition data, process data and metallographic images are preprocessed, and the preprocessed data and mechanical property data are combined into an actual multimodal data set D T , the specific operations are: S21, using the Pearson correlation coefficient to calculate the linear correlations between the component data, process data and mechanical property data, sorting the importance of the components and processes, and selecting the top N component data and process data as the preprocessed component data and process data; S22, for the initial multimodal dataset D p The metallographic image in is cropped to obtain a metallographic image that meets the model input matrix size requirement, which is the preprocessed metallographic image; S23, the preprocessed metallographic image is matched with the preprocessed composition data, preprocessed process data and mechanical property data one by one to obtain the actual multimodal data set D T .

4. The method for predicting mechanical properties of C-Mn steel based on multimodal data convolution feature fusion according to claim 3 is characterized in that: The one-dimensional convolutional neural network model A includes four one-dimensional convolutional modules, among which the first three one-dimensional convolutional modules are composed of a Conv1d, BatchNorm1d, ReLU and MaxPool1d respectively. The number of convolutional kernels is 8, 16 and 32 respectively. The size of the convolutional kernel is 3×3, the step size is 1 and the padding is 1. The size of the MaxPool1d kernel is 2×2 and the step size is 2. The fourth one-dimensional convolution module consists of a Conv1d, BatchNorm1d, ReLU and AdaptiveAvgPool1d. The number of convolution kernels is 128, the size of the convolution kernel is 3×3, the step size is 1, and the padding is 1; the size of the AdaptiveAvgPool1d kernel is 1×1.

5. The method for predicting mechanical properties of C-Mn steel based on multimodal data convolution feature fusion according to claim 4, characterized in that: The two-dimensional convolutional neural network model B includes three two-dimensional convolutional modules, each of which is composed of a Conv2d, BatchNorm2d, ReLU and MaxPool2d. The number of convolution kernels is 16, 32 and 64 respectively, the size of the convolution kernel is 3×3, the step size is 1, the padding is 1, and the kernel size of MaxPool2d is 2×2, with a step size of 2.

6. The method for predicting mechanical properties of C-Mn steel based on multimodal data convolution feature fusion according to claim 5, characterized in that: The 2D convolutional neural network model D includes three 2D convolutional modules and one adaptive average pooling layer AdaptiveAvgPool2d. The first two 2D convolutional modules are composed of a Conv2d, BatchNorm2d, ReLU and MaxPool2d respectively. The number of convolution kernels is 128 and 256 respectively, the size of the convolution kernel is 3×3, the step size is 1, and the padding is 1; the kernel size of MaxPool2d is 2×2, and the step size is 2; Adaptive average pooling layer AdaptiveAvgPool2d, whose kernel size is 1×1; The third two-dimensional convolution module is the convolution layer Conv2d, whose convolution kernel size is 1×1, the stride is 1, the number of convolution kernels is 1, and the padding is 1.

7. The method for predicting mechanical properties of C-Mn steel based on multimodal data convolution feature fusion according to claim 6, characterized in that: In step S5, the evaluation indicators include root mean square error RMSE and determination coefficient R 2 .

Images