A method for generating titanium alloy microstructure images based on a deep learning model

Abstract

The present invention discloses a method for generating titanium alloy microstructure images based on a deep learning model, belonging to the field of image generation. The method includes the following steps: obtaining an original titanium alloy microstructure image, performing forward noise addition processing on the original titanium alloy microstructure image to obtain a pure noise image; constructing a deep learning model, and inputting the pure noise image into the deep learning model for reverse denoising to generate a predicted titanium alloy microstructure image. The present invention not only improves the efficiency of image generation and reduces costs, but also can generate different types of titanium alloy microstructure images by adjusting model parameters, providing an efficient tool for materials science research.

Description

Technical Field

[0001] The present invention belongs to the technical field of image generation, and particularly relates to a method for generating titanium alloy microstructure images based on a deep learning model. Background Art

[0002] Titanium alloys are widely used in the fields of aviation, medical treatment, and high-performance engineering due to their excellent mechanical properties and corrosion resistance. The microstructure images of titanium alloys are of great significance for materials science research and application development. However, the traditional methods for obtaining titanium alloy microstructure images have problems such as low efficiency, high difficulty in recognition, difficulty in feature extraction, and high cost. In recent years, the application of deep learning models in the field of image generation has provided a new technical path to solve this problem. Therefore, the present invention discloses a method for generating titanium alloy microstructure images based on a deep learning model. Summary of the Invention

[0003] To solve the above technical problems, the present invention proposes a method for generating titanium alloy microstructure images based on a deep learning model to solve the problems existing in the above prior art.

[0004] To achieve the above object, the present invention provides a method for generating titanium alloy microstructure images based on a deep learning model, including:

[0005] Obtaining an original titanium alloy microstructure image, and performing forward noise addition processing on the original titanium alloy microstructure image to obtain a pure noise image;

[0006] Constructing a deep learning model, and inputting the pure noise image into the deep learning model for reverse denoising to generate a predicted titanium alloy microstructure image.

[0007] Optionally, the expression for forward noise addition is:

[0008]

[0009] In the formula, p(x t |x t-1 ) represents forward noise addition, β t is a preset noise weight, I is an identity matrix, and x t represents the pure noise image.

[0010] Optionally, the expression for the pure noise image is:

[0011]

[0012] In the formula, is the cumulative product of denoising coefficients, ∈ is Gaussian noise in the forward noise addition process, and x t represents the pure noise image.

[0013] Optionally, the process of the deep learning model includes predicting the noise at each step; wherein, the expression for predicting the noise is:

[0014]

[0015] In the formula, μ θ (x t ,t) and represent the mean and variance predicted by the network, and p θ (x t-1 |x t ) represents the predicted noise.

[0016] Optionally, the expression for reverse denoising is:

[0017]

[0018] In the formula, ∈′ is the Gaussian noise in the reverse denoising process, and x t-1 represents the denoised image at the (t - 1)-th step.

[0019] Optionally, the deep learning model further includes a number of embedding layers, and the embedding layers are used to embed performance parameters into each upsampling layer and downsampling layer; the performance parameters include: temperature, cooling time, forging temperature, heat treatment, image magnification factor.

[0020] Optionally, before generating the predicted titanium alloy microstructure image based on the deep learning model, there is also a model training stage, and the model training stage optimizes the model based on minimizing the loss value;

[0021] wherein, the expression of the loss function is:

[0022]

[0023] In the formula, represents the loss value.

[0024] Compared with the prior art, the present invention has the following advantages and technical effects:

[0025] The method for generating a titanium alloy microstructure image of the present invention not only improves the efficiency of image generation and reduces the cost, but also can generate different types of titanium alloy microstructure images by adjusting the model parameters, providing a feasible tool for materials science research. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The drawings constituting a part of this application are used to provide a further understanding of this application. The schematic embodiments of this application and their descriptions are used to explain this application and do not constitute an improper limitation to this application. In the drawings:

[0027] Figure 1It is the flowchart of the method according to the embodiment of the present invention;

[0028] Figure 2 It is the schematic diagram of the forward diffusion process and the reverse denoising process according to the embodiment of the present invention;

[0029] Figure 3 It is the schematic diagram of the intuitive comparison between the alloy microstructure diagram generated by the model according to the embodiment of the present invention and the real alloy microstructure diagram;

[0030] Figure 4 It is the schematic diagram of the quantitative fitting between the alloy microstructure reconstructed by the model and the real alloy microstructure according to the embodiment of the present invention. Specific embodiments

[0031] It should be noted that, without conflict, the embodiments in the present application and the features in the embodiments may be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in conjunction with the embodiments.

[0032] It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than here.

[0033] Embodiment 1

[0034] The present invention provides a method for generating titanium alloy microstructure images based on the Denoising Diffusion Probabilistic Models (DDPM). This method gradually adds noise to the original titanium alloy microstructure image to a pure noise image through the forward diffusion process, and then gradually removes the noise through the reverse denoising process until the initial image is restored. A deep learning model with a UNet architecture is used for training to accurately predict the image generation during the denoising process. This method can efficiently generate high-quality titanium alloy microstructure images, not only improving the efficiency of image generation, reducing costs, but also generating different types of titanium alloy microstructure images by adjusting the model parameters, providing an efficient tool for materials science research.

[0035] As Figure 1 shown, this embodiment provides a method for generating titanium alloy microstructure images based on a deep learning model, including: obtaining the original titanium alloy microstructure image, performing forward noise addition processing on the original titanium alloy microstructure image to obtain a pure noise image; constructing a deep learning model, and inputting the pure noise image into the deep learning model for reverse denoising to generate a predicted titanium alloy microstructure image.

[0036] Before training the model, the data is preprocessed first. In this study, 1157 SEM images of the microstructure of high-strength and tough titanium alloys with different STA processes and different magnification factors are obtained through experiments. The original images of 1536×1103 are cropped into images of 512×512.

[0037] The magnification factors of the collected SEM images of the titanium alloy microstructure range from 5000 times to 100,000 times, a total of 23 types of magnification factors. Combining with the STA heat treatment process data, the preprocessed data is finally classified into different categories.

[0038] In this embodiment, 87% of the data is used for model training, and 13% of the data is reserved. The data that the model has not seen is extracted from here for detailed quantitative comparison analysis to test the generalization ability of the model on unseen data.

[0039] The DDPM model is implemented based on the Pytorch deep learning platform, trained for 300 epochs. The step size, i.e., the learning rate, used when updating the weights is 0.0003. The size of the input to the model is 512×512, the dimension of the embedding layer is 100, and according to the division of the training set and the test set, the number of classes (num_classes) is set to 28.

[0040] In the image evaluation stage, a model for segmenting the SEM images of the titanium alloy microstructure is trained based on the deep learning module of MIPAR. First, appropriate recipes can be set through operations such as improving contrast and binarization to segment different phases in the tissue images.

[0041] Then, manually fine-tune the effect of the segmented image. The labeled image obtained in this step will be used as the training data for the deep learning module to improve the effect of image segmentation.

[0042] In this study, image segmentation training is carried out for different phases respectively, and finally, a good overall image segmentation effect is achieved by superimposing and using.

[0043] This model is trained on a device equipped with an Intel Xeon 8352Y processor and an NVIDIA RTX 4090 graphics card.

[0044] The MSE loss function is used to minimize the gap between the target data and the predicted data.

[0045] A specific implementation process of a method for generating titanium alloy tissue images based on the denoising diffusion probability model includes:

[0046] Step 1, as Figure 2As shown, forward diffusion is performed: by gradually adding noise, the original titanium alloy tissue image is denoised to a pure noise image. The specific operation is as follows: in the forward diffusion process: by gradually adding noise, the original titanium alloy tissue image is denoised to a pure noise image, and x0 represents the original titanium alloy tissue image, X t It represents the image after t steps of noise addition. The noise addition process is shown in the following formula:

[0047]

[0048] Among them, β t is the noise coefficient at time step t, and I is the identity matrix.

[0049] Step 2: Starting from the original image x0, gradually add noise to generate the noisy image x t :

[0050]

[0051] in is the cumulative product of the denoising coefficients, is Gaussian noise.

[0052] Step 3: reverse denoising: starting from the pure noise image, gradually remove the noise until the original titanium alloy tissue image is restored. t To begin with, the image is usually generated from a pure noise sample.

[0053] Step 4, define the denoising neural network (U-Net)∈ θ (x t ,t), where θ is the model parameter, the goal is to predict the noise at each step:

[0054] Among them, μ θ (x t ,t) and are the mean and variance of the predictions made by the network, p θ (x t-1 |x t ) represents the predicted noise, that is, when x is known t In the case of x t-1 Probability of occurrence.

[0055] The denoising process of step t is expressed as follows:

[0056]

[0057] Among them, μ θ and Σ θ Predicted by the deep learning model, θ is the model parameter. θ (x t, t) represents a complete covariance matrix that can capture and model the dependencies between different features.

[0058] The noise prediction module consists of a Unet network with conditional inputs, aiming to predict the noise term ∈(x t , t, c), where x t represents the noisy image, t is the time step, and c is the given condition.

[0059] Step 5, as Figures 3 - 4 shown, gradually perform reverse denoising starting from the noisy image x t until the original image x0 is restored,

[0060]

[0061] where, is the Gaussian noise in the reverse denoising process, and x t-1 represents the denoised image at the (t - 1)-th step.

[0062] Step 6, the control of model training is achieved by minimizing the loss function,

[0063]

[0064] The loss function calculates the difference between the predicted noise and the true noise of the denoising network.

[0065] Step 7 In the U-Net (Step 4) part of the model, the embedding of titanium alloy heat treatment conditions is implemented. The embedding vectors related to performance parameters are incorporated into each upsampling and downsampling layer. These embedding vectors are first linearly mapped to tensors consistent with the layer spatial dimensions and then fused with the feature maps, thereby imposing specific process parameter constraints on the model. For example, in classes Down and Up, for each condition parameter such as temperature, cooling time, etc., there is a dedicated embedding layer (nn.Embedding) to convert these conditions into features that the model can use. The embedding results of each condition are integrated into the downsampling (Down) and upsampling (Up) paths of the model.

[0066] To achieve unified representation of conditions, the embedding layer is used to map each condition to the same dimension so that the model can more conveniently process and learn different input conditions.

[0067] The Unet network introduces a self-attention mechanism, including six groups of symmetric downsampling and upsampling modules. Four convolutional layers, group normalization layers, and Gaussian error linear unit (GELU) activation functions are connected after each module.

[0068] Attach a self-attention mechanism with four heads after each downsampling layer to help the model handle long-range and wide-ranging dependencies between image regions.

[0069] The self-attention mechanism helps the model dynamically adjust the feature weights according to the task requirements, strengthen important features and ignore non-critical information, and improve the interpretability of the model, because the output attention map can show which regions of the image the model pays more attention to when making decisions.

[0070] To alleviate the problem of gradient vanishing caused by the increase in network depth, this model applies skip connection between each group of downsampling and upsampling, effectively alleviating this problem.

[0071] Step 8, the specific parameters include temperature (two stages), cooling time (two stages), forging temperature, heat treatment, image magnification, etc. These parameters are converted through embedding vectors and combined with image data during model training and generation, affecting the finally generated image.

[0072] Step 9, in the forward propagation function forward of the model, the features generated by these embedding layers are merged with the image data and participate in subsequent network calculations together. By adjusting the values of these parameters, the specific conditions of the generated image can be controlled, such as changing the temperature or cooling time to observe the changes in the microstructure of the titanium alloy.

[0073] Step 10, in the prediction functions predict and predictByPara, generate images dynamically according to the provided parameters, such as finding the closest condition configuration according to the actual physical properties (such as tensile strength), and then generating the corresponding image.

[0074] During the model training phase, to improve the convergence stability of the model, reduce the oscillation during training, and improve the prediction performance of the model, an Exponential Moving Average (EMA) strategy is adopted to update the network parameters. EMA achieves this by calculating the sliding average of the model parameters, thus reducing the volatility of the model weights during training.

[0075] Image generation: Randomly generate a Gaussian noise image x t , input it into the trained deep learning model, and predict the mean μ θ (x t , t) and variance Σ θ (x t , t), and sample according to the sampling formula from a Gaussian distribution with mean μ θ (x t , t) and variance Σ θ (x t , t) to obtain xt1 , loop T times until x0 is obtained, which is the generated titanium alloy microstructure image.

[0076] The above is only a preferred specific embodiment of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by those skilled in the art within the technical scope disclosed in the present application should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method for generating titanium alloy microstructure images based on a deep learning model, characterized in that Including the following steps: Obtain the original titanium alloy microstructure image, and perform positive noise addition processing on the original titanium alloy microstructure image to obtain a pure noise image; Construct a deep learning model, and input the pure noise image into the deep learning model for reverse denoising to generate a predicted titanium alloy microstructure image; The deep learning model further includes a number of embedding layers, and the embedding layers are used to embed performance parameters into each upsampling layer and downsampling layer; the performance parameters include: temperature, cooling time, forging temperature, heat treatment, image magnification; The process of the deep learning model includes predicting the noise at each step. Define the U-Net denoising neural network to predict the noise at each step. The U-Net denoising neural network integrates the embedding vectors related to the performance parameters into each upsampling and downsampling layer. These embedding vectors are first linearly mapped into tensors that match the layer spatial dimensions, and then fused with the feature maps, so as to impose specific process parameter constraints on the model; The Unet network introduces a self-attention mechanism, including six groups of symmetric downsampling and upsampling modules. Four convolutional layers, a group normalization layer, and a Gaussian error linear unit activation function are connected after each module; Attach a self-attention mechanism with four heads after each downsampling layer to help the model handle long-distance and wide-ranging dependencies between image regions; The deep learning model applies a skip connection between each group of downsampling and upsampling to alleviate the problem of gradient disappearance caused by the increase in network depth; Before generating the predicted titanium alloy microstructure image based on the deep learning model, there is also a model training stage, and the model training stage optimizes the model based on the minimum loss value; Among them, the expression of the loss function is: In the formula, represents the loss value.

2. The method for generating a titanium alloy microstructure image based on a deep learning model according to claim 1, characterized in that, The expression of positive noise addition is: where p(x t |x t-1 ) represents forward noise addition, β t is a preset noise weight, I is the identity matrix, and x t represents the pure noise image.

3. The method for generating a titanium alloy microstructure image based on a deep learning model according to claim 2, wherein The expression of the pure noise image is: Wherein, is the cumulative product of denoising coefficients, ∈ is the Gaussian noise in the forward noise addition process, and x t represents the pure noise image.

4. The method for generating a titanium alloy microstructure image based on a deep learning model according to claim 3, wherein The process of the deep learning model includes predicting the noise at each step; among them, the expression of predicting the noise is: where μ θ (x t , t) and represent the mean and variance predicted by the network, and p θ (x t-1 |x t ) represents the predicted noise.

5. The method for generating a titanium alloy microstructure image based on a deep learning model according to claim 4, wherein The expression of reverse denoising is: In the formula, ∈′ is the Gaussian noise in the reverse denoising process, and x t-1 represents the image denoised at the (t - 1)-th step.

Images