A simulation data generation method of metal material multi-modal fusion

Abstract

The application belongs to the field of material data, and discloses a simulation data generation method of metal material multi-modal fusion. The method comprises the following steps: collecting data, extracting features to obtain a feature vector, fusing the feature vector through a multi-modal method to obtain a metal material multi-modal feature training data set, constructing a metal material simulation data diffusion model and a diffusion model loss function meeting physical condition constraints, training the diffusion model, and outputting high-quality simulation metal material data meeting the basic physical law of the material. The simulation data generation method of metal material multi-modal fusion provided by the application introduces physical constraints, so that the generated simulation data meets experimental data from a statistical perspective and meets the basic physical law. The method can effectively generate simulation data of metal materials under the condition of limited data, significantly reduces the experimental and calculation costs, and simultaneously fuses numerical and image modal data, which is more in line with the organization-property evolution relationship of the material.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of material data, and particularly relates to a simulation data generation method for metal material multi-modal fusion. BACKGROUND

[0002] Metal materials are the most important structural materials for human social development, and have an irreplaceable role in the industries of aviation, aerospace, navigation and automobile. Achieving high strength and high plasticity of metal materials is of great significance to the overall performance, service safety and service life of parts.

[0003] Additive Manufacturing (AM) technology has been widely used in the fields of aerospace, automobile manufacturing, biomedicine, etc. in recent years, and its significant feature is that it can manufacture parts with complex geometric structures and material properties. However, the process involves multi-scale and multi-physical field coupling problems of materials, such as microstructure evolution, stress deformation, defect formation, etc. In addition, the microstructure of the material, such as grain size, porosity, grain boundary distribution, etc., directly affects its mechanical properties, which greatly affects the mechanical properties and service life of the final product. Traditional material performance testing methods are usually time-consuming and costly. Through data generation, the dependence on a large number of experiments can be reduced, thereby saving time and resources, which is of great significance for the development of new materials and the improvement of existing materials. In order to improve the prediction ability of metal material behavior, simulation technology has gradually become a key tool.

[0004] Traditional simulation methods mainly rely on experimental and numerical simulation methods such as finite element analysis (FEA) and molecular dynamics (MD). However, these methods often require a large amount of experimental data for verification when dealing with complex, multi-defect, and multi-working condition metal materials, and the calculation cost is high. In addition, some parameters in the physical model, such as the evolution mechanism of micro-defects and organizational structure, are difficult to obtain directly through experimental data, resulting in insufficient simulation accuracy.

[0005] In recent years, with the development of machine learning and deep learning technology, data-driven methods have gradually been applied in the field of materials. By using the microstructure images of materials, process parameters and material performance data to construct a data generation model, high-quality material performance data can be generated, and the generated data can support further research and analysis by simulation methods or machine learning methods. However, these generation models have a series of problems in practical application. The microstructure images of materials, process parameters and material performance data belong to different modal data, and effectively fusing image data features and numerical data features is a challenge. Existing generation models are mostly trained based on the statistical characteristics of data, and the generated data lacks constraints on physical laws. Although the generated data may be close to experimental data in a statistical sense, it cannot guarantee its rationality in physical behavior.

[0006] Therefore, how to generate high-precision simulation data conforming to physical laws and capable of fusing multi-modal features under the condition of limited data has become a problem to be solved. SUMMARY

[0007] In view of the above defects or improvement needs of the prior art, the present application provides a simulation data generation method of metal material multi-modal fusion, which fuses metal material microstructure image data, material manufacturing process parameters and material performance data to construct a multi-modal data generation framework conforming to physical laws. The framework combines a diffusion model and a deep learning model driven by physical constraints, introduces a physical condition constraint loss function in the training process, so as to generate high-quality simulation data consistent with physical laws.

[0008] To achieve the above-mentioned purpose, according to one aspect of the present application, a simulation data generation method of metal material multi-modal fusion is provided, comprising the following steps:

[0009] S1: collecting original metal material process parameters, mechanical performance data and microstructure image data of metal material;

[0010] S2: extracting features of the metal material microstructure image collected in step S1 through a convolutional neural network to obtain a feature vector of the metal material microstructure image;

[0011] S3: preprocessing the original metal material process parameters and mechanical performance data collected in step S1 to obtain a metal material process parameter and mechanical performance numerical feature vector;

[0012] S4: fusing the metal material microstructure image feature vector obtained in step S2 and the process parameter and performance numerical feature vector obtained in step S3 through a multi-modal method to obtain a metal material multi-modal feature training data set;

[0013] S5: constructing a metal material simulation data diffusion model based on the metal material multi-modal feature training data set obtained in step S4;

[0014] S6: constructing a diffusion model loss function conforming to physical condition constraints based on the initial metal material simulation data diffusion model obtained in step S5;

[0015] S7: training the diffusion model based on the metal material multi-modal feature training data set, the initial diffusion model and the loss function thereof obtained in steps S4-S6, and outputting high-quality simulation metal material data conforming to the basic physical laws of the material.

[0016] As a preferred, step S2 specifically comprises the following steps:

[0017] S2.1: Constructing a convolutional neural network model M img , the model contains convolutional layers, pooling layers and activation functions, removes the last fully connected layer and retains the feature extraction part function;

[0018] S2.2: Preprocessing the image data to adapt to the input requirements of the convolutional neural network, obtaining the preprocessed image data set I';

[0019] S2.3: Input the preprocessed image set I' into the model M img , extract image features from low level to high level layer by layer through multi-layer convolutional layer, then improve nonlinearity through activation function, and finally reduce the dimension and calculation amount of feature map through pooling operation while retaining important image features, complete the extraction of feature vector F img of metal material microstructure image.

[0020] As a preferred, the preprocessing of the original metal material process parameters and mechanical property data in step S3 adopts normalization processing, and the method of the normalization processing is:

[0021]

[0022] Wherein, X represents the original metal material process parameters and mechanical property data, X data represents the numerical training data sample of metal material.

[0023] As a preferred, the fusion method in step S4 is: by introducing a gating mechanism to add dynamic weights to different modal data to adapt to multi-modal feature input, retain important features and control selective expression of features, while introducing a residual structure to improve the stability and robustness of the model, complete the fusion of multi-modal data features.

[0024] As a preferred, step S4 specifically includes the following steps:

[0025] S4.1: Adjust the numerical feature vector F data to the same dimension as the image feature vector F img ;

[0026] S4.2: Define a gating mechanism to dynamically adjust the fusion ratio of image features and numerical features, define a gating tensor Calculate the gating weight through a feedforward neural network as follows:

[0027] G=σ(W g ·F img +b g )

[0028] Wherein, Wg is the weight matrix, bg is the bias, σ is the activation function Sigmoid, which ensures that the value of G is between 0 and 1.

[0029] S4.3: Control the fusion weight of the image, numerical modal feature through the gating tensor G, form the gating fusion feature F gate :

[0030] F gate = G O F img + (1-G) O F data

[0031] S4.4: On the basis of the gating fusion feature, increase the residual feature connection F res , finally get the fusion feature F fusion :

[0032] F fusion = F gate (F img , F data ) + F res (F img , F data )

[0033] Wherein, the residual connection F res (F img , F data ) = F img + F data .

[0034] As preferred, the diffusion model in step S5 includes a forward diffusion process and a reverse diffusion process, the forward diffusion process gradually adds noise to the data to convert it into pure Gaussian noise by mapping the input metal material multi-modal training data sample to the Gaussian noise space; the reverse diffusion module gradually denoises from the noise to reconstruct the data to generate new metal material simulation data.

[0035] As preferred, step S5 specifically includes the following steps:

[0036] S5.1: In the forward process of the diffusion model, gradually add noise to the real data through the time step t to construct the Gaussian noise distribution, so that the training data gradually changes from the real data sample to pure noise, and the diffusion step is defined as follows:

[0037]

[0038] Wherein, x t is the data after adding noise, x0 is the original real metal material data, α t is a hyperparameter for controlling the noise level based on the time step t, and ∈ ~ N (0, I) is a random noise sampled from the standard normal distribution;

[0039] S5.2: In the process of backward diffusion, the model gradually restores from Gaussian noise to simulation data conforming to the characteristics of metal materials. For each time step t, the model defines the conditional probability p θ (x t-1 | x t ), representing the denoising process from time step t to t-1; by learning the mean function μ θ (x t , t) and variance parameter σ θ (t), step-by-step denoising is achieved to generate samples closer to real data, as follows:

[0040] x t-1 = μ θ (x, t) + σ θ (t) ∈

[0041] where μ θ (x t , t) predicts the data position at the current time step, σ θ (t) controls the randomness of denoising, and ∈ ~ N(0, I) is random noise sampled from the standard normal distribution.

[0042] As a preferred, the loss function Loss total in step S6 is constructed by combining data loss and physical consistency, as follows:

[0043] Loss total = Loss data + λ·Loss phys

[0044] where the data loss Loss data uses the evaluation index for regression tasks to measure the fitting loss between generated data and real data; the physical consistency loss Loss phys calculates the consistency loss between generated data and expected physical behavior according to the physical model and constraints; λ is a balance coefficient to adjust the direct weights of data loss and physical loss.

[0045] As a preferred, the loss function Loss total in step S6 is constructed by combining data loss and physical consistency, as follows:

[0046] S6.1: Calculate the data loss, the diffusion model involves modeling the noise distribution of randomly sampled variables, so the mean square error (MSE) evaluation index in the form of expectation is used to optimize the fitting degree of the model output and the real distribution:

[0047]

[0048] where, E(x0) represents the average error expectation value under the joint distribution of sample x0 and random sampling variable t; x0 represents the real distribution data, represents the model output data, represents the square Euclidean distance between the predicted value and the true value, and the model loss is calculated.

[0049] S6.2: Calculate the physical consistency loss, calculate the deviation of the generated data according to the physical model, force the generated data to meet the material characteristics, and ensure that the data meet the physical law:

[0050] Loss phys =λ JC Loss JC +λ vM Loss vM +λ HP Loss HP

[0051] Wherein, λ JC , λ vM , λ HP are weight coefficients for balancing the contribution of different physical loss terms in the total loss, Loss JC , Loss vM , Loss HP are loss functions of different physical condition constraints, and the size of the loss value is proportional to the gap between the generated data and the actual physical law;

[0052] Loss JC The loss function is defined as the absolute value of the difference between the generated stress σ gen and the predicted value of Johnson-Cook model, which is as follows:

[0053]

[0054] Wherein, σ gen is the generated stress, and the right side is the stress prediction value of Johnson-Cook model, A, B, C, m, n are material constants, T m is the melting point of the material, and T is set to room temperature 25℃. This loss function will guide the model to generate real material stress under different temperature, strain rate and strain conditions;

[0055] Loss vM The loss function is defined as the equivalent stress σ v calculated by the generated stress data should not exceed the yield strength σ yeild of the material, which is as follows:

[0056] Loss yield = max(0, σ v - σ y )

[0057] in, σ1, σ2, and σ3 are the maximum stresses generated by the model on the material in different directions; the equivalent stress σ v and yield strength σ y The difference is compared with 0. When the generated equivalent stress exceeds the yield strength, the loss function is positive. If the generated data meets the yield criterion, the loss value is 0.

[0058] Loss vM The loss function is defined as the yield strength σ generated by the stress data y The absolute value of the Hall-Petch relationship varies with grain size d, as shown in the following formula:

[0059] Loss HP =|σ y,gen -(σ0+k y d -1 / 2 )|

[0060] Among them, σ y，gen is the generated yield strength, σ0 and k y is the material constant for the strengthening effect at the grain boundary.

[0061] Preferably, the training of the diffusion model in step S7 includes initializing parameters and iterative training, specifically including the following steps:

[0062] S7.1: Initialize the parameters of the diffusion model and gradually adjust them during training based on the impact of physical consistency loss to balance the physical consistency of the generated data and the data fitting effect;

[0063] S7.2: Iteratively train the diffusion model. For each training sample, each training data sample is converted into noise x through the forward diffusion process. t ; Generate data x' through the back diffusion process and calculate the loss function Loss total ; Through the back-propagation algorithm, the model parameters are updated to minimize the loss until the loss function converges and no longer decreases; by repeating the above steps, the diffusion model is trained so that it gradually learns to recover high-quality simulation data that conforms to the physical properties of metal materials from the noisy data.

[0064] In general, the above technical solutions conceived by the present invention have the following beneficial effects compared with the prior art:

[0065] 1. The present invention adopts a multimodal data fusion method, which not only conforms to the numerical regression analysis results, but also conforms to the material's organization-performance evolution relationship after fusing the microstructure image of the metal material, significantly improving the quality of the generated simulation data.

[0066] 2、The present application introduces physical constraints, so that the generated simulation data not only meets the basic physical laws, but also meets the basic physical laws.

[0067] 3、The method of the present application has high generalization ability, can effectively generate simulation data of metal materials under the condition of limited data, and can significantly reduce the experimental and calculation cost. BRIEF DESCRIPTION OF DRAWINGS

[0068] Figure 1 The basic steps of the metal material multi-modal fusion simulation data generation method of the present application are shown in the schematic diagram.

[0069] Figure 2 A preferred implementation flowchart of the present application is shown in the schematic diagram. DETAILED DESCRIPTION

[0070] In order to make the purpose, technical scheme and advantages of the present application clearer and more understandable, the present application will be further described in detail below in combination with the drawings and examples. It should be understood that the specific examples described herein are only used to explain the present application and do not limit the present application. In addition, the technical features involved in each embodiment of the present application described below can be combined with each other as long as they do not conflict with each other.

[0071] Diffusion Model is a new deep generative model, its essence is similar to training neural network to learn picture denoising process to realize the gradual generation process from a known Gaussian distribution to target distribution, its core idea is to gradually add noise to real data through a forward diffusion process, and then train a reverse diffusion process to learn how to recover data from noise. The generation process of diffusion model can be described as gradually denoising in multiple time steps, and finally generating new data similar to the distribution of training data. Diffusion model can naturally describe the continuous dynamic process of data according to its Stochastic Differential Equation (SDE) form, which is suitable for processing the smoothness and continuity of data in regression problem, and gradually converts the distribution of target data from a simple Gaussian distribution to target distribution, thereby generating corresponding regression prediction value.

[0072] In combination with Figure 1 and Figure 2 , the present embodiment provides a metal material multi-modal fusion simulation data generation method, which comprises the following steps:

[0073] Step S1: Collecting original metal material process parameters, mechanical property numerical data and microstructure image data.

[0074] The specific method is as follows: using scanning electron microscopy (SEM), metallographic microscope and other technologies to collect microstructure images of metal material samples, the microstructure images need to include grain structure, phase interface, defects and other key information; recording the process parameters of the metal material sample in the process, including laser power, scanning rate, scanning spacing, layer thickness, and measuring the material performance data of the sample by mechanical testing means such as universal testing machine, including elastic modulus, yield strength, tensile strength, elongation, and collecting 108 metal material sample data containing microstructure images and manufacturing process parameters, mechanical properties.

[0075] Step S2: Based on the microstructure image data of the data acquisition module, the features of the metal material microstructure image are extracted through a convolutional neural network (CNN), and a metal material microstructure image feature vector is obtained.

[0076] The specific implementation method is as follows:

[0077] Step S2.1: Construct a convolutional neural network model. Select a ResNet-50 pre-trained convolutional neural network model as an image feature extractor, and set it as model M img , which contains convolutional layers, pooling layers and activation functions, removes the last fully connected layer and retains the feature extraction part.

[0078] Step S2.2: Preprocess the image data to adapt to the input requirements of the convolutional neural network. First, adjust the size of the metal material microstructure image data set to 224x224 and the number of channels to 3, then divide the pixel value of each pixel in each channel of the image by 255 to normalize the pixel value to the interval [0, 1], and obtain the preprocessed image data set I′:

[0079]

[0080] Step S2.3: input the preprocessed image set I′ into the model M img , and extract the microstructure image features.

[0081] Each convolution kernel slides across the entire image, extracts features by local region weighted sum, and identifies local features. Then by stacking 6 convolution layers, the information from low-level features (such as edges, corners) to high-level features (such as grain structure, phase interface, defects) is extracted layer by layer, and the image features are nonlinearly fused through the ReLU activation function after the last convolution layer, and then global average pooling (GAP) is performed to obtain a fixed-size image feature vector F img , and the feature extraction process is represented as:

[0082] F img = GAP(M(I'))

[0083] The model removes the full connection layer for classification, and the feature vector output for a single sample has a dimension of C x M x N; C is the number of channels, the depth of the feature map; M is the height of the feature map; N is the width of the feature map. Specifically, a feature tensor with a size of 2048 x 1 x 1 is extracted based on the ResNet-50 convolutional neural network.

[0084] Step S3: The original metal material process parameters and mechanical properties collected are processed using a normalization method to extract a scalar value data feature vector.

[0085] The numerical data of the metal material process parameters and mechanical properties are defined as X, and an example is shown in the following table.

[0086]

[0087] The feature extraction method of the metal material process parameters and mechanical property data adopts the maximum and minimum normalization method, and the training sample X of each metal material numerical data is extracted by the following method. data

[0088]

[0089] Among the 108 data, there are 19 kinds of materials. The 19 material names are converted into 22-dimensional binary numerical feature vectors through one-hot encoding, such as “17-4PH” converted into [1, 0, 0, …, 0], and “Ti6-Al4-V” converted into [0, 1, 0, …, 0];

[0090] X data is combined with the one-hot encoding features of the material name to obtain the numerical feature vector F data .

[0091] Specifically, after numerical feature extraction, the feature vector is obtained, as shown in the following table.

[0092]

[0093] Step S4: Fuse the metal material microstructure image feature vector extracted above with the numerical feature vector of the process parameters and mechanical properties to obtain the multi-modal metal material training data input.

[0094] The fusion method is to add dynamic weights to different modal data by introducing a gating mechanism to adapt to multi-modal feature input, retain important features and control the selective expression of features, while introducing a residual structure to improve the stability and robustness of the model, and complete the fusion of multi-modal data features.​

[0095] That is, the image feature dimension extracted by the pre-step is much higher than the numerical feature dimension, and direct fusion will cause the image feature to dominate, greatly weaken the influence of numerical features, and lead to the loss of important information. Therefore, a gating structure and a residual structure are designed here to introduce dynamic weights to adapt to the feature input of multiple modalities, retain important features and control the selective expression of features, while improving the stability and robustness of the model. Specifically, the feature vector extracted from the metal material microstructure image obtained from the above steps Process parameters, mechanical property extracted feature vector Further, the dimension of the feature vector is adjusted to make the shape of the two modal data the same; then the influence of different modal features is dynamically adjusted through the gating mechanism, and finally the feature fusion forms the multi-modal feature F fusion .

[0096] Step S4.1 adjusts the numerical feature F data to the same dimension as the image feature F img . The dimensions and shapes of the numerical features F img and F data are adjusted by repeating and expanding, as follows:

[0097]

[0098] Step S4.2 defines a gating mechanism to dynamically adjust the fusion ratio of image features and numerical features. Specifically, define the gating tensor G The gating weight is calculated by a feedforward neural network, as follows:

[0099] G = σ (W g · F img + b g )

[0100] where Wg is the weight matrix, bg is the bias, σ is the activation function Sigmoid, and G is ensured to be between 0 and 1.

[0101] Step S4.3 fuses image features and numerical features by weighting with the gating tensor. Specifically, the fusion weight of image and numerical modal features is controlled by the gating tensor G to form the gated fusion feature F gate , as follows:

[0102] F gate = G img ⊙ F data

[0103] Step S4.4 introduces a residual structure to form the final fusion feature. Specifically, based on the gated fusion feature, a residual feature connection F res, to obtain the fused feature F fusion , as follows:

[0104] F fusion = F gate (F img , F data ) = F res (F img , F data )

[0105] where the residual connection F res (F img , F data ) = F img + F data .

[0106] Step S5: Based on the metal material multi-modal feature training dataset, a metal material simulation data diffusion model is constructed.

[0107] The construction of the metal material simulation data diffusion model mainly includes a forward diffusion process and a reverse diffusion process. The forward diffusion process gradually adds noise to the data to convert it into pure Gaussian noise by mapping the input metal material multi-modal training data sample to the Gaussian noise space; the reverse diffusion module gradually denoises from the noise to reconstruct the data to generate new metal material simulation data.

[0108] Step S5.1 adds noise to the real data by training the diffusion model through the forward diffusion process. Specifically, in the forward process of the diffusion model, noise is gradually added to the real data through time step t to construct a Gaussian noise distribution, so that the training data gradually changes from real data samples to pure noise. The original real metal material data obtained by the preceding step is defined as x0, and the diffusion step is defined as follows:

[0109]

[0110] where x t is the data after adding noise, a t is a hyperparameter that controls the noise level based on time step t, and ∈ ~ N(0, I) is random noise sampled from a standard normal distribution.

[0111] Step S5.2 makes the model learn to reconstruct data from noise through the reverse diffusion process. In the reverse diffusion process, the model gradually restores from Gaussian noise to simulation data that conforms to the characteristics of metal materials. This process restores noise samples to samples that conform to the real distribution through the denoising step. Specifically, for each time step t, the model defines the conditional probability p θ (x t-1 |x t ), which represents the denoising process from time step t to t-1. By learning the mean function μθ (x t , t) and variance parameter σ θ (t), a step-by-step denoising is realized to generate samples closer to the real data, as follows:

[0112] x t-1 = μ θ (x, t) + σ θ (t) ∈

[0113] wherein μ θ (x t , t) predicts the data position at the current time step, σ θ (t) controls the randomness of denoising, and ∈ ~ N(0, I) is random noise sampled from the standard normal distribution.

[0114] Step S6 constructs a diffusion model loss function conforming to the physical condition constraint.

[0115] In the loss function of the diffusion model designed by the application, a physical consistency loss term is introduced to ensure that the generated data conforms to the basic physical laws of metal materials. The total loss function is defined to include a data loss part and a physical consistency loss part, and a comprehensive loss function is designed, as follows:

[0116] Loss total = Loss data + λ·Loss phys

[0117] wherein Loss data is the data loss, Loss phys is the physical consistency loss, and λ is a balance coefficient for adjusting the direct weight of the data loss and the physical loss.

[0118] Step S6.1 calculates the data loss to measure the difference between the generated data and the real data. Specifically, the diffusion model involves modeling the noise distribution of the random sampling variable, so the mean square error (MSE) evaluation index in the form of expectation is used to optimize the fitting degree of the model output and the real distribution, and the Euclidean distance between the feature vectors of the metal material multi-modal real training data samples and the predicted data is calculated, and the expectation value after squaring is taken to constitute the data loss function:

[0119]

[0120] wherein represents the average error expectation value under the joint distribution of the sample x0 and the random sampling variable t; x0 represents the real distribution data, represents the model output data, represents the squared Euclidean distance between the predicted value and the real value.

[0121] Step S6.2 calculates the physical consistency loss to ensure the physical rationality of the generated data. The deviation of the generated data is calculated according to the physical model, and the generated data is forced to meet the material properties, ensuring that the data meets the physical laws, including the Johnson-Cook model, the von Mises yield criterion, and the Hall-Petch relationship, as follows:

[0122] Loss phys = λ JC Loss JC + λ vM Loss vM + λ HP Loss HP

[0123] wherein λ JC , λ vM , λ HP are weight coefficients for balancing the contributions of different physical loss terms in the total loss, Loss JC , Loss vM , Loss HP are loss functions of different physical condition constraints, and the size of the loss value is proportional to the gap between the generated data and the actual physical law.

[0124] Specifically, the Loss JC loss function is defined as the absolute value of the difference between the generated stress σ gen and the predicted value of the Johnson-Cook model. The generated data includes stress and strain, and the Johnson-Cook model describes the stress response of the material under different strain rates, strains, and temperatures, as follows:

[0125]

[0126] wherein σ gen is the generated stress, and the right side is the stress prediction value of the Johnson-Cook model, A, B, C, m, and n are material constants, T m is the melting point of the material, and T is set to room temperature 25°C. This loss function will guide the model to generate real material stress under different temperature, strain rate, and strain conditions.

[0127] Loss vM is defined as the equivalent stress σ v calculated from the generated stress data, which should not exceed the yield strength σ yeild of the material. The von Mises criterion is used to judge the yield condition under multi-axial stress state, as follows:

[0128] Loss yield= max(0, σ v - σ y )

[0129] wherein, σ1, σ2, σ3 are the maximum stresses in different directions that the material is subjected to. The above equation compares the von Mises equivalent stress σ v with the yield strength σ y , and the loss function is positive when the generated equivalent stress exceeds the yield strength, and is zero if the generated data satisfies the yield criterion. This loss constrains the generated data to be in the yield state under multiaxial stress, ensuring that the generated data conforms to the yield behavior of the material.

[0130] Loss vM The loss function is defined as the absolute value of the difference between the generated stress data σ y and the Hall-Petch relationship, which should change with the grain size d. The Hall-Petch relationship describes the effect of grain size on the yield strength of the material, and is as follows:

[0131] Loss HP = |σ y,gen - (σ0+ k y d -1 / 2 )|

[0132] wherein σ y，gen is the generated yield strength, and σ0and k y are material constants for the strengthening effect of the grain boundary. This loss term ensures that the yield strength of the generated data conforms to the physical law when the grain size changes.

[0133] Step S7 trains the diffusion model to output high-quality simulated metal material data that conforms to the basic physical laws of the material.

[0134] Model training includes initialization of parameters and iterative training. Initialization of parameters is to initialize and adjust the key hyperparameters of the diffusion model at the beginning of training, to ensure effective training and convergence of the model. The iterative training process includes forward diffusion of damaged data, backward diffusion of generated data, and backward propagation of optimized model. Through the forward diffusion process, each training data sample is transformed into noise x t ; through the backward diffusion process, data x` is generated, and the loss function Loss total is calculated; through the backpropagation algorithm, the model parameters are updated to minimize the loss until the loss function converges and no longer decreases. By repeatedly training the diffusion model as described above, it gradually learns to recover high-quality simulation data that conforms to the physical properties of metal materials from noisy data.

[0135] Step S7.1: Initialize the parameters of the diffusion model. Initialize and adjust the key hyperparameters, set the number of diffusion steps T to 200 steps, set the noise intensity parameter α t =α0(1-decay_rate) t , where α0 is set to 0.95, decay_rate is set to 0.005, and noise is gradually added with time step t; the learning rate η is set to 0.0001; the constraint weight of the physical loss is initially set to 1, and is gradually adjusted during the training process according to the influence of the physical consistency loss to balance the physical consistency of the generated data and the data fitting effect.

[0136] Step S7.2 iteratively trains the diffusion model. Specifically, for each training sample, each training data sample is converted into noise x through the forward diffusion process. t ; Generate data x' through the back diffusion process and calculate the loss function Loss total Using the backpropagation algorithm, the model parameters are updated to minimize the loss until the loss function converges and no longer decreases. By repeating the above steps, the diffusion model is trained to gradually recover high-quality simulation data that conforms to the physical properties of metal materials from noisy data.

[0137] It will be easily understood by those skilled in the art that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for generating simulation data for multimodal fusion of metal materials, characterized in that: The following steps are involved: S1: Collect the process parameters, mechanical properties data and microstructure image data of the original metal material; S2: extracting features from the metal material microstructure image collected in step S1 through a convolutional neural network to obtain a feature vector of the metal material microstructure image; S3: Preprocessing the original metal material process parameters and mechanical property data collected in step S1 to obtain numerical feature vectors of the metal material process parameters and mechanical properties; S4: fusing the metal material microstructure image feature vector obtained in step S2 with the process parameter and performance numerical feature vector obtained in step S3 through a multimodal method to obtain a metal material multimodal feature training dataset; S5: Based on the metal material multimodal feature training data set obtained in step S4, a metal material simulation data diffusion model is constructed; S6: Based on the initial metal material simulation data diffusion model obtained in step S5, a diffusion model loss function that meets the physical condition constraints is constructed; the loss function The construction is achieved by combining data loss and physical consistency as follows: Among them, data loss Loss data The evaluation index used for regression tasks is used to measure the fitting loss between generated data and real data; physical consistency loss Loss phys The consistency loss between generated data and expected physical behavior is calculated based on the physical model and constraints; λ is the balance coefficient, which adjusts the direct weight of data loss and physical loss; Loss Function The construction method specifically includes the following steps: S6.1: Calculate data loss. The diffusion model involves modeling the noise distribution of randomly sampled variables, so the expected form of the mean squared error evaluation indicator is used to optimize the fit between the model output and the true distribution: in, Represents a sample and randomly sampled variables t The expected value of the average error under the joint distribution of represents the real distribution data, Represents the model output data, Represents the squared Euclidean distance between the predicted value and the true value, and calculates the model loss; S6.2: Calculate the physical consistency loss, calculate the deviation of the generated data according to the physical model, force the generated data to meet the material properties, and ensure that the data meets the physical laws: Among them, λ JC ,λ vM ,λ HP Is the weight coefficient used to balance the contribution of different physical loss items in the total loss, Loss JC 、Loss vM 、Loss HP It is a loss function constrained by different physical conditions. The size of the loss value is proportional to the gap between the generated data and the actual physical laws. Loss JC The loss function is defined as the generated stress σ gen The absolute value of the difference from the Johnson-Cook model prediction is as follows: in, is the generated stress, and the right side is the stress prediction value of the Johnson-Cook model. A, B, C, m, and n are material constants. T m is the melting point of the material, and T is set to room temperature 25°C; this loss function will guide the model to generate realistic material stresses under different temperatures, strain rates, and strain conditions; Loss vM The loss function is defined as the equivalent stress σ calculated from the generated stress data v Should not exceed the yield strength σ of the material yeild , specifically as follows: in, , σ1, σ2, σ3 are the maximum stresses generated by the model on the material in different directions; the equivalent stress σ v and yield strength σ y The difference is compared with 0. When the generated equivalent stress exceeds the yield strength, the loss function is positive. If the generated data meets the yield criterion, the loss value is 0. Loss vM The loss function is defined as the yield strength σ generated by the stress data y The absolute value of the Hall-Petch relationship varies with grain size d, as shown in the following formula: in, is the generated yield strength, σ0 and k y is the material constant of the strengthening effect of grain boundaries; S7: Based on the metal material multimodal feature training data set, the initial diffusion model and its loss function obtained in steps S4-S6, the diffusion model is trained to output high-quality simulated metal material data that conforms to the basic physical laws of the material.

2. The method for generating simulation data of multimodal fusion of metal materials according to claim 1, characterized in that: Step S2 specifically includes the following steps: S2.1: Building a Convolutional Neural Network Model M img ,The model contains convolutional layers, pooling layers and activation functions,,removes its last fully connected layer, and retains some feature extraction,functions; S2.2: Preprocess the image data to meet the input requirements of the convolutional neural network and obtain the preprocessed image dataset ; S2.3: The preprocessed image set Input to model M img In the process, the image features are extracted from low level to high level layer by layer through multiple convolution layers, and the nonlinearity is enhanced through activation function. Finally, the dimension and calculation amount of the feature map are reduced through pooling operation, while retaining the important features of the image, and the feature vector F of the metal material microstructure image is extracted. img .

3. The method for generating simulation data of multimodal fusion of metal materials according to claim 1, characterized in that: In step S3, the preprocessing of the process parameters and mechanical property data of the original metal material is performed by normalization processing, and the normalization processing method is as follows: Among them, X represents the process parameters and mechanical properties data of the original metal material, X data Represents the numerical training data samples of metal materials.

4. The method for generating simulation data of multimodal fusion of metal materials according to claim 1, characterized in that: The fusion method in step S4 is: by introducing a gating mechanism to add dynamic weights to different modal data to adapt to multimodal feature input, retain important features and control the selective expression of features, while introducing a residual structure to improve the stability and robustness of the model, thereby completing the fusion of multimodal data features.

5. The method for generating simulation data of multimodal fusion of metal materials according to claim 4, characterized in that: Step S4 specifically includes the following steps: S4.1: Adjusting Numerical Feature Vectors by Repetition and Expansion F data and image feature vector F img to the same dimension; S4.2: Define a gating mechanism to dynamically adjust the fusion ratio of image features and numerical features, and define the gating tensor , the gate weights are calculated through a feedforward neural network as follows: in, is the weight matrix, is the bias, Is the activation function Sigmoid, ensuring The value of is between 0 and 1; S4.3: Control the fusion weights of image and numerical modal features through the gated tensor G to form gated fusion features F gate : S4.4: Add residual feature connections based on gated fusion features F res , and finally get the fusion feature F fusion : Among them, the residual connection .

6. The method for generating simulation data of multimodal fusion of metal materials according to claim 1, characterized in that: The diffusion model in step S5 includes a forward diffusion process and a backward diffusion process. The forward diffusion process maps the input metal material multimodal training data samples to the Gaussian noise space, gradually adding noise to the data to convert it into pure Gaussian noise; the backward diffusion module gradually removes noise from the noise, reconstructs the data and generates new metal material simulation data.

7. The method for generating simulation data of multimodal fusion of metal materials according to claim 6, characterized in that: Step S5 specifically includes the following steps: S5.1: In the forward process of the diffusion model, noise is gradually added to the real data at time step t to construct a Gaussian noise distribution, so that the training data gradually transforms from real data samples to pure noise. The diffusion step is defined as follows: Among them, x t is the data after adding noise, x0 is the original real metal material data, α t is a hyperparameter that controls the noise level based on the time step t, ϵ~N(0,I) is random noise sampled from a standard normal distribution; S5.2: During the back diffusion process, the model gradually restores the Gaussian noise to the simulation data that conforms to the characteristics of the metal material. For each time step t, the model defines the conditional probability p θ (x t-1 |x t ), represents the denoising process from time step t to t-1; by learning the mean function μ θ (x t , t) and variance parameter σ θ (t), realize gradual denoising and generate samples closer to real data, as shown in the following formula: Among them, μ θ (x t , t) predict the data position at the current time step, σ θ (t) controls the randomness of denoising, ϵ~N(0,I) is the random noise sampled from the standard normal distribution.

8. The method for generating simulation data of multimodal fusion of metal materials according to claim 1, characterized in that: Training the diffusion model in step S7 includes initializing parameters and iterative training, specifically including the following steps: S7.1: Initialize the parameters of the diffusion model and gradually adjust them during training based on the impact of physical consistency loss to balance the physical consistency of the generated data and the data fitting effect; S7.2: Iteratively train the diffusion model. For each training sample, each training data sample is converted into noise x through the forward diffusion process. t ; Generate data x' through the back diffusion process and calculate the loss function Loss total ; Through the back-propagation algorithm, the model parameters are updated to minimize the loss until the loss function converges and no longer decreases; by repeating the above steps, the diffusion model is trained so that it gradually learns to recover high-quality simulation data that conforms to the physical properties of metal materials from the noisy data.

Images