Multiphase heterogeneous material digital twinning modeling method based on deep convolutional neural network
By inserting small sample features into the VGG16-UNet model to enhance the attention module and optimize the training parameters, the recognition accuracy of small samples of defects and damage in X-CT images of multiphase heterogeneous materials is improved, and the problem of low accuracy of existing models is solved. , realizes high-precision digital twin modeling of multiphase heterogeneous materials.
Patent Information
- Application Number
- CN202411225253.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-03
- Publication Date
- 2024-12-13
AI Technical Summary
The existing semantic segmentation model has low accuracy in identifying defects and damage samples in X-CT images of multiphase heterogeneous materials, making it difficult to effectively distinguish pores and cracks inside the material.
Using a multiphase heterogeneous material digital twin modeling method based on deep convolutional neural network, the improved VGG16-UNet semantic segmentation model is used, and small sample features are inserted at each layer of the network to enhance the attention module, through Focal loss and TverskyLoss loss functions , Adam optimization algorithm and specific training hyperparameters are iteratively trained to improve the recognition accuracy of the model.
The segmentation accuracy of each phase and the recognition accuracy of small samples of multiphase heterogeneous materials were improved, with an average accuracy of 83.18%. The recognition accuracy of initial pore defects and extended crack damage was increased by 2.29% and 1.16% respectively, significantly improving the modeling accuracy.
Smart Images
Figure CN119131487A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of multi-scale material modeling and computer vision technology, and specifically to a multi-phase heterogeneous material digital twin modeling method based on a deep convolutional neural network. Background Art
[0002] Multiphase heterogeneous materials, such as concrete, rock and biological tissue, are usually composed of different components and structural phases. The complexity of these materials makes it a challenge to accurately predict their mechanical, physical and chemical properties. Understanding and predicting the behavior of materials at different scales (from micro to macro) has attracted more and more attention. Scale modeling is one of the key technologies to address this challenge. This method reveals how microstructure affects macroscopic properties, thus providing a theoretical basis and data support for material design and optimization.
[0003] With the development of contemporary manufacturing, X-ray computed tomography (X-CT) has been used for nondestructive testing of multiphase composite materials. Reconstructing the digital twin model of the material using the tomographic image can provide a detailed understanding of the influence of the microstructures such as the initial pore defects, initial interface cracks and crack extension inside the material on the macroscopic mechanical behavior of the material. The image segmentation methods currently used for digital twin reconstruction of multiphase heterogeneous materials have the following limitations: (1) the temporary adjustment of a large number of parameters is time-consuming and labor-intensive; (2) the long-tail effect of the CT values of each phase affects the accuracy of the reconstructed model; (3) the CT values of the pores and cracks inside the material are similar and difficult to distinguish effectively.
[0004] In recent years, with the breakthroughs made in artificial intelligence and big data technologies, the combination of computer vision technology and X-CT images has provided a good method for the field of multi-scale modeling of materials. Limited by the small number and small area of defect and damage samples in X-CT images of multi-phase heterogeneous materials, the current semantic segmentation model has low recognition accuracy for small samples. Summary of the invention
[0005] The present invention provides a digital twin modeling method for multi-phase heterogeneous materials based on deep convolutional neural networks, which mainly solves the problem that the existing semantic segmentation model has low accuracy in identifying small sample targets of defects and damages in X-CT images of multi-phase heterogeneous materials.
[0006] In order to achieve the above purpose, the technical solutions provided are as follows:
[0007] A multi-phase heterogeneous material digital twin modeling method based on a deep convolutional neural network is characterized by comprising the following steps:
[0008] Step 1: Prepare a material sample; and perform X-CT scanning on the prepared material sample to obtain tomographic images of each material sample;
[0009] Step 2: semantically annotate each tomographic image of the material sample obtained in step 1, and establish a proprietary image dataset of multi-phase heterogeneous materials based on X-CT; and divide the dataset into training set, validation set and test set according to a certain ratio;
[0010] Step 3: Establish a network architecture of the model, where the model is based on an improved VGG16-UNet semantic segmentation model, and insert a module dedicated to small sample feature enhancement attention in each layer of the network;
[0011] Step 4: Determine the training loss function, optimization algorithm and training hyperparameters of the model established in step 3 through cross-validation experiments;
[0012] Step 5: Use the training set to iteratively train the model until the loss function converges to obtain the final weight model;
[0013] Step 6: Load the final weight model to perform semantic segmentation on the X-CT images in the validation set and the test set, and evaluate the performance of the model;
[0014] Step 7: Determine whether the weight model is the optimal model. If so, proceed to step 8. If not, return to step 5.
[0015] Step 8: Use the optimal model to segment each tomographic image of the material sample; and use the segmentation result image to reconstruct a three-dimensional digital twin model of the material sample.
[0016] Furthermore, in step one, each material phase in the multi-phase heterogeneous material has a different density, and the initial pore damage defect and the crack extension damage have different morphological characteristics.
[0017] Furthermore, in step 2, the multi-phase heterogeneous material proprietary image dataset is a pixel-level semantic label produced according to the CT value and grayscale value difference results corresponding to the density values of each phase material, and the pixel-level semantic label content includes: pixel-level sample labels of each material phase and sample labels of initial defects and later damage extension phases; and then the training set, validation set and test set are divided according to appropriate proportions.
[0018] Furthermore, in step three, the improved VGG16-UNet semantic segmentation model consists of a backbone feature extraction part, a feature fusion part and a decoding part.
[0019] Furthermore, the backbone feature extraction part adopts VGG16, the feature fusion part inserts a small sample feature enhanced attention module, and the decoding part adopts the decoding part of UNet.
[0020] Furthermore, in step 4, the model training loss function is: Focal loss and TverskyLoss.
[0021] Furthermore, in step 4, the model optimization algorithm is: Adam (β1=0.90; β2=0.999).
[0022] Furthermore, in step 4, the model training hyperparameters include: training batch parameters, initial learning rate, learning rate reduction strategy, number of learning rate reduction rounds, weight attenuation coefficient, total number of training rounds, total number of iteration rounds is 100 rounds, and whether to use pre-trained weights.
[0023] Furthermore, in step five, the convergence of the loss function indicates that after a sufficient number of iterations, the loss function value tends to a stable minimum value.
[0024] Furthermore, in step six, the model performance evaluation index is: intersection over union ratio, that is, the ratio of the overlapping area of the label mask and the prediction mask to the sum of their respective areas.
[0025] Furthermore, in step seven, the optimal model is the model with the highest intersection-over-union ratio among all training models.
[0026] Furthermore, in step eight, the material digital twin model includes a three-dimensional model of each material phase, initial pore defects, and extended crack damage.
[0027] The beneficial effects of the present invention are as follows: the semantic segmentation model of multiphase heterogeneous materials provided by the present invention is suitable for the segmentation of phases with different densities, with an average accuracy of up to 83.18%; by inserting the designed small sample feature enhanced attention module into the benchmark model, the recognition accuracy of small samples such as initial pore defects and extended crack damage can be improved, and the accuracy improvement for the two types of small samples can reach 2.29% and 1.16% respectively; further, the improvement of the segmentation accuracy of each phase in the tomographic image of multiphase heterogeneous materials significantly improves the modeling accuracy of each phase. BRIEF DESCRIPTION OF THE DRAWINGS
[0028] Figure 1 is a flow chart of the present invention;
[0029] Figure 2 It is a tomographic image of the material sample in the present invention;
[0030] Figure 3 It is the annotation mask of the tomographic image of the material sample in the present invention;
[0031] Figure 4 This is a diagram of the improved VGG16-UNet network model architecture in the present invention;
[0032] Figure 5 It is a relationship diagram of the training loss changing with the number of training rounds in the present invention;
[0033] Figure 6 It is the mask diagram of each phase segmentation of the sample fault in the present invention;
[0034] Figure 7 This is the three-dimensional reconstruction result of the single building in the present invention. DETAILED DESCRIPTION
[0035] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0036] Example
[0037] A digital twin modeling method for multiphase heterogeneous materials based on deep convolutional neural network, such as Figure 1 The process diagram shown in FIG. 1 includes the following specific steps:
[0038] Step 1: For the preparation of multiphase heterogeneous material samples, silicate composite multiphase materials are used, and the sample size is 100mm×100mm×100mm; for the acquisition of each tomographic image of the sample, the fourth generation (G4) plus medical spiral X-ray CT scanner is used. The scanning system of this equipment consists of a rotating radiation source and a ring detector, with a scanning resolution of 0.1mm×0.1mm×0.1mm and a density resolution of 3‰ of the water absorption coefficient. The obtained tomographic image is as follows: Figure 2 shown.
[0039] Step 2: For the X-CT images of the multiphase heterogeneous material samples obtained in step 1, the deep learning semantic segmentation and annotation tool labelme is used to annotate the data according to the gray value range of each phase. The label content includes the pixel-level classification information of each phase sample of the material, where the material phase categories include: cement-based phase and aggregate-doped phase, and the damage categories include: initial pore defect phase and extended crack damage phase; the annotation mask is as follows: Figure 3 As shown in the figure. After the labeling is completed, there are 600 1024×1024 images in total. The images are then cropped into 512×512 sub-images with a step length of 256, so the number of sub-images is 4500. Finally, the dataset is divided into training set, validation set and test set according to 7:2:1.
[0040] Step 3: Establish the network architecture of the model. The network is based on the improved VGG16-UNet semantic segmentation model, and inserts the designed small sample feature enhancement attention module at each level of the network. Among them, VGG16 is the backbone feature extraction part, the small sample feature enhancement attention module is the feature enhancement part, and the UNet decoder part is the feature decoding part. The network architecture of the specific model is as follows: Figure 4 shown.
[0041] Step 4: Adjust and set the model training loss function, optimization algorithm, and training hyperparameters. The specific setting parameters are shown in Table 1.
[0042] Table 1 Model training parameter settings
[0043]
[0044] Step 5: Use the training set to iteratively train the model until the loss function converges and obtains the final weight model; the relationship between training loss and the number of training rounds is shown in the figure below: Figure 5 shown.
[0045] Step 6: Load the final weighted model to perform semantic segmentation on the X-CT images in the validation set and the test set, and evaluate the performance of the model; the evaluation indicator is the intersection of union (IoU).
[0046] Step 7: Evaluate the performance of each weight model and select the optimal model. The obtained performance indicators are shown in Table 2. The IoU of the improved VGG16-UNet for the cement-based phase, aggregate-doped phase, initial pore defect phase, and extended crack damage phase can reach 70.76%, 77.17%, 91.39%, and 93.41%, respectively, and the overall index is as high as 83.18%, which is 1.25% higher than the baseline model.
[0047] Table 2 Performance index comparison
[0048]
[0049] Step 8: Use the improved optimal model of VGG16-UNet to segment the tomographic images of the material sample, such as Figure 6 As shown, Figure 6 (a) is the segmentation mask image of the doped aggregate phase. Figure 6 (b) is the 3D digital twin model of the initial pore defect. Figure 6 (c) is a three-dimensional digital twin model of the extended crack damage. Furthermore, the segmentation mask image is used to further reconstruct the three-dimensional digital twin model of each phase of the material sample, such as Figure 7 shown. Figure 7 (a) is the 3D digital twin model of the doped aggregate phase. Figure 7 (b) is the 3D digital twin model of the initial pore defect. Figure 7 (c) 3D digital twin model of extended crack damage.
[0050] It can be found that the accuracy of the tomographic image recognition results is extremely high, achieving high-precision segmentation of each phase of multi-phase heterogeneous materials and accurate reconstruction of the three-dimensional digital twin model.
[0051] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.
Claims
1. A digital twin modeling method for multiphase heterogeneous materials based on deep convolutional neural network, characterized in that: The steps include: Step 1: Prepare a material sample; and perform X-CT scanning on the prepared material sample to obtain tomographic images of each material sample; Step 2: semantically annotate each tomographic image of the material sample obtained in step 1, and establish a proprietary image dataset of multi-phase heterogeneous materials based on X-CT; and divide the dataset into training set, validation set and test set according to a certain ratio; Step 3: Establish a network architecture of the model, where the model is based on an improved VGG16-UNet semantic segmentation model, and insert a module dedicated to small sample feature enhancement attention in each layer of the network; Step 4: Determine the training loss function, optimization algorithm and training hyperparameters of the model established in step 3 through cross-validation experiments; Step 5: Use the training set to iteratively train the model until the loss function converges to obtain the final weight model; Step 6: Load the final weight model to perform semantic segmentation on the X-CT images in the validation set and the test set, and evaluate the performance of the model; Step 7: Determine whether the weight model is the optimal model. If so, proceed to step 8. If not, return to step 5. Step 8: Use the optimal model to segment each tomographic image of the material sample, and use the segmentation result image to reconstruct the three-dimensional digital twin model of the material sample.
2. According to claim 1, a multi-phase heterogeneous material digital twin modeling method based on deep convolutional neural network is characterized in that: In step one, each material phase in the multi-phase heterogeneous material has a different density, and the initial pore damage defect and the crack extension damage have different morphological characteristics.
3. According to claim 1, a multi-phase heterogeneous material digital twin modeling method based on deep convolutional neural network is characterized in that: In step 2, the multi-phase heterogeneous material proprietary image dataset is a pixel-level semantic label produced according to the CT value and grayscale value difference results corresponding to the density values of each phase material, and the pixel-level semantic label content includes: pixel-level sample labels of each material phase and sample labels of initial defects and later damage extension phases; and then the training set, validation set and test set are divided according to appropriate proportions.
4. The method for digital twin modeling of multiphase heterogeneous materials based on deep convolutional neural network according to claim 1 is characterized in that: In step three, the improved VGG16-UNet semantic segmentation model consists of a backbone feature extraction part, a feature fusion part and a decoding part; the backbone feature extraction part adopts VGG16, the feature fusion part inserts a small sample feature enhanced attention module, and the decoding part adopts the decoding part of UNet.
5. The method for digital twin modeling of multiphase heterogeneous materials based on deep convolutional neural network according to claim 1, characterized in that: In step 4, the model training loss function is: Focal loss and Tversky Loss; the model optimization algorithm is: Adam (β1=0.90; β2=0.999); The model training hyperparameters include: training batch parameters, initial learning rate, learning rate reduction strategy, number of learning rate reduction rounds, weight decay coefficient, total number of training rounds, total number of iteration rounds is 100 rounds, and whether to use pre-training weights.
6. The method for digital twin modeling of multiphase heterogeneous materials based on deep convolutional neural network according to claim 1, characterized in that: In step five, the convergence of the loss function indicates that after a sufficient number of iterations, the loss function value tends to a stable minimum value.
7. The method for digital twin modeling of multiphase heterogeneous materials based on deep convolutional neural network according to claim 1, characterized in that: In step six, the model performance evaluation index is: intersection over union ratio, that is, the ratio of the overlapping area of the label mask and the prediction mask to the sum of their respective areas.
8. The method for digital twin modeling of multiphase heterogeneous materials based on deep convolutional neural network according to claim 1, characterized in that: In step seven, the optimal model is the model with the highest intersection-over-union ratio among all training models.
9. The method for digital twin modeling of multi-phase heterogeneous materials based on deep convolutional neural network according to claim 1, characterized in that: In step eight, the material digital twin model includes a three-dimensional model of each material phase, initial pore defects, and extended crack damage.