A material crack extraction method and system based on convolutional neural network and transformer

Abstract

The present application belongs to the technical field of cross of material damage evolution research and deep learning, aiming at the problem that the crack is difficult to extract and the boundary positioning is not accurate due to noise interference in the existing CT image, a material crack extraction method and system based on convolutional neural network and Transformer are proposed. The method firstly carries out in-situ tensile experiment and CT scanning on the sample, and constructs the training data set through image preprocessing; then a segmentation model is constructed: the encoder extracts features through five-level convolution downsampling, the decoder fuses multi-scale features through transposed convolution upsampling combined with jump connection; the model is trained by adopting five-fold cross-validation strategy, the loss function is the weighted combination of pixel overlap loss and classification loss; finally, the crack segmentation result is output and the segmentation accuracy is calculated, and the damage evolution analysis is realized through three-dimensional reconstruction. The present application can extract the crack with high precision when the CT image noise is serious, and provide high-precision data support for material damage evolution research.

Description

Technical Field

[0001] This invention patent belongs to the field of deep learning image segmentation, and in particular relates to a method and system for extracting material cracks based on convolutional neural networks and Transformers. Background Technology

[0002] Studying the damage evolution mechanisms of materials, especially the initiation, propagation, and connectivity of cracks, is of great significance for material design, performance optimization, and reliability assessment. Currently, material damage research relies on CT imaging technology, which can capture the internal evolution process, thus allowing for the extraction of crack information from CT images. However, in some complex situations, poor CT image quality and artifacts make crack extraction challenging.

[0003] Traditional methods based on edge detection and region growing are susceptible to noise interference, which can easily lead to incomplete crack extraction or high false detection rates. Some existing convolutional neural networks, such as U-Net, have shown good performance in some semantic segmentation tasks, but in complex environments, the crack morphology in material CT images becomes blurred, the contrast is low, and it is difficult to distinguish from background noise. The generalization ability and segmentation accuracy of existing models are still not ideal.

[0004] To address the above problems, this invention proposes a material crack extraction method based on convolutional neural networks and Transformer. The method uses convolutional neural networks to automatically configure the network structure and extract local features, while the attention mechanism of Transformer integrates global information, significantly improving the crack extraction accuracy. Summary of the Invention

[0005] To address the challenge of accurately extracting cracks from material CT images under various complex environments using existing deep learning methods, this invention provides a material crack extraction method and system based on convolutional neural networks and Transformers. This method can extract cracks from material CT images with relatively high accuracy, providing methodological support for subsequent material damage evolution analysis.

[0006] To achieve the above objectives, the present invention provides the following solution:

[0007] A method for extracting material cracks based on convolutional neural networks and Transformers includes:

[0008] Step 1, CT Scan: The sample is placed in a CT imaging device and stretched. After each period of force is applied, a CT scan is performed to obtain all CT images of the sample from the start to the final fracture.

[0009] Step 2, Data Preprocessing: After obtaining the CT image data of the sample from start to breakage, the CT slice images are leveled, corrected, and normalized to 0-1.

[0010] Step 3: Dataset Construction: Randomly select a portion of CT slice images, manually annotate the cracks, and perform cropping and data augmentation to construct the dataset.

[0011] Step 4, Model Building: After the image is input into the network, the model automatically configures the network structure, selects the optimal architecture, and builds a crack extraction model based on convolutional neural networks and Transformers, which includes an encoder structure, a decoder structure, and a feature fusion module.

[0012] Step 5, Model Training and Validation: The model is trained using a five-fold cross-validation method, and the highest accuracy set is selected as the final accuracy of the model.

[0013] Step 5, Model Training and Validation: The model is trained using a five-fold cross-validation method, and the highest accuracy set is selected as the final accuracy of the model.

[0014] Step 6, Model Testing: After the model training is completed, calculate the Dice coefficients of the model-predicted image and the manually labeled image to obtain the final crack extraction accuracy of the model.

[0015] Step 7, Other Data Processing: After the model training and testing are completed, all other CT images of the sample are input into the model for prediction, and finally the crack evolution of the sample under different stress conditions is obtained.

[0016] Optionally, the dataset construction includes: manually annotating the layered cracks in the XZ-direction slices of the samples, setting the pixel values ​​of the layered crack portions to 1, and setting the pixel values ​​of other background portions to 0. To ensure improved model training speed, the selected CT images are cropped to obtain 256*256 images, and the dataset is augmented through random rotation, flipping, and cropping to improve the model's generalization and robustness. All datasets are divided into training set:test set = 9:1 as the model's dataset; the training set is used for model training and validation, and the test set is used for the final accuracy test of the model.

[0017] Optionally, the encoder structure includes: the encoder consists of six network layers, namely E0, E1, E2, E3, E4, and E5 layers.

[0018] The input to layer E0 is a preprocessed single-channel CT image. After passing through a 3*3 convolution, instance normalization, and LeakyReLU activation function, it outputs an initial feature map with 32 channels.

[0019] The input to layer E1 is the output of layer E0. First, it is downsampled by 2*2, and then the result is convolved to output a feature image with 64 channels.

[0020] The input to layer E2 is the output of layer E1. It is first downsampled by 2*2, and then the result is convolved to output a feature image with 128 channels.

[0021] The input to layer E3 is the output of layer E2. First, it undergoes Patch Embedding, where the input feature map is divided into 4x4 patches and projected onto the Transformer dimension. Through 4x4 convolutions, the output is a sequence of length 256, with an embedding length of 256. Then, positional encoding is added, and the sequence is fed into two Transformer modules for processing, resulting in a feature image with 256 channels. E3 introduces the global modeling capability of the Transformer to capture global semantic features.

[0022] One Transformer module includes a Layer Norm, a multi-head self-attention mechanism, a Layer Norm, and an MLP layer. The Layer Norm is used to normalize the features; the multi-head self-attention mechanism captures diverse dependencies of the input sequence from different subspaces by computing multiple sets of attention weights in parallel; the MLP layer consists of two fully connected network layers and activation functions, which are used to supplement the linearity of the multi-head self-attention mechanism.

[0023] The input to layer E4 is the output of layer E3. First, it undergoes a 2x2 downsampling convolution to increase the number of channels to 512. Then, it passes through a 4x4 convolution to obtain the patch embedding. Positional encoding is then added, and the input is fed into two Transformer modules for processing. The final output is a feature image with 512 channels. The Transformer modules used in E4 have the same structure as those used in E3. E4 extracts global semantic information from the low-resolution feature map, enhancing the features of the material crack region.

[0024] The input to layer E5 is the output of layer E4. A 2x2 convolution operation is performed on the input to obtain patch embedding, and then positional encoding is added. This is then fed into two Transformer modules for processing, and the final output is a feature image with 512 channels. Layer E5 refines global features and enhances the ability to identify crack boundaries and micro-crack structures.

[0025] Optionally, the decoder structure includes a five-layer network consisting of layers D4, D3, D2, D1, and D0.

[0026] The input to layer D4 is the output of encoder layer E5. First, it is upsampled through two transposed convolutions. Then, the feature map from encoder layer E4 is concatenated with the upsampled feature map via skip connections. This convolution is then processed through two consecutive convolutional blocks, each consisting of two 3x3 convolutional layers. Each convolutional layer is followed by instance normalization and a LeakyReLU activation function. The final output is a feature map with 256 channels.

[0027] The input to layer D3 is the output of layer D4. First, the layers undergo two upsampling operations. Then, the feature maps from encoder layer E3 are concatenated, followed by two convolutional operations, ultimately outputting a feature map with 128 channels. The structure of the convolutional layers is identical to that of layer D4.

[0028] The input to layer D2 is the output of layer D3. First, the feature map is upsampled twice. Then, the feature map from encoder E2 is concatenated with the upsampled feature map via skip connections. Finally, two convolutional processes are performed, resulting in a feature map with 64 channels. The structure of the convolutional layers is consistent with that of layer D4.

[0029] The input to layer D1 is the output of layer D3. First, the input is upsampled twice. Then, the feature maps from the corresponding encoder layers are concatenated, followed by two convolutional processes. This yields the output feature map. The final output is a feature map with 32 channels. The structure of the convolutional layers is consistent with that of layer D4.

[0030] The input to the D0 layer network is the output of the D2 layer network, which is also the final output of the network. The number of channels is mapped to the number of judgment categories through 1*1 convolution, and then Sigmoid (binary) activation is used to finally obtain the crack extraction result.

[0031] Optionally, the feature fusion module includes:

[0032] An attention mechanism is added after each layer of the encoder network structure, performing attention in both spatial and channel dimensions, thus integrating the CBAM structure into the encoder structure design. The CBAM structure is added after layers E0, E1, and E2, and after layers E3, E4, and E5. The output format is then converted to the Transformer encoding format.

[0033] The CBAM structure consists of a channel attention mechanism and a spatial attention mechanism connected in series, dynamically weighting the channels and spatial dimensions of the input feature map. The channel attention mechanism passes the input feature map through two parallel max-pooling and average-pooling layers, transforming it into a C*1*1 map. Then, it passes through an MLP layer to generate channel weights. Finally, the original feature map is multiplied by the weighted channels to obtain the output of the channel attention mechanism. The spatial attention mechanism obtains two 1*H*W feature maps through max-pooling and average-pooling layers. These two feature maps are then concatenated, and spatial information is fused using a 7*7 convolution to generate a spatial weight map. Finally, the original feature map is multiplied positionally by the weight map to obtain the output of the spatial attention mechanism. These two modules are then combined sequentially to form the CBAM structure.

[0034] Optionally, the damage function of the model is:

[0035] A loss function weighted by Dice and cross-entropy is used. Dice and cross-entropy are two commonly used loss functions that are often used together in image segmentation tasks. The Dice coefficient is used to evaluate the degree of overlap between the segments; a higher Dice coefficient indicates a better overlap between the predicted and ground truth values. The cross-entropy loss function measures the distributional difference between the predicted and ground truth values, and its expression is as follows:

[0036]

[0037] Where P represents the number of pixels predicted as positive and actually positive, FP represents the number of pixels predicted as positive but actually negative, and FN represents the number of pixels predicted as negative but actually positive. N represents the predicted class, y i Labels that indicate authenticity This represents the predicted probability of the model.

[0038] The model damage function is expressed as:

[0039] Loss = Loss Dice +Loss CrossEntroy .

[0040] Optionally, the model employs a five-fold cross-validation training strategy:

[0041] The training set was divided into 5 parts, with each part used as the validation set and the remaining parts used as the training set for model training. The set with the highest accuracy was then selected as the final accuracy of the model.

[0042] The Adam optimizer is used during training, with an initial learning rate of 0.01. The learning rate is dynamically adjusted during training using a polynomial decay mechanism, which helps to accelerate training and model convergence. The formula is as follows:

[0043]

[0044] Where baselr represents the initial learning rate, iters represents the current iteration number, max_iters represents the maximum number of iterations, and power represents the learning rate decay exponent, which is set to 0.9. Each fold trains for 200 epochs.

[0045] This invention also provides a material crack extraction system based on convolutional neural networks and Transformers. The system applies the above-described method and is characterized by comprising:

[0046] Data acquisition module: configured to perform CT scan and preprocessing operations in steps 1-3;

[0047] Model building module: configured to implement the encoder structure, decoder structure, and feature fusion module in step 4;

[0048] Training optimization module: Configured to perform five-fold cross-validation and dynamic learning rate adjustment in step 5;

[0049] Crack Analysis Module: Configured to generate crack segmentation results after running the training model and calculate the Dice accuracy index.

[0050] One embodiment of this invention patent has the following beneficial effects: The material crack extraction method and system based on convolutional neural networks and Transformers proposed in this invention patent can accurately identify cracks in material CT images under complex environments, reducing artifacts and noise interference, and providing a foundation for subsequent damage evolution analysis of materials. This method uses nnUNet as its basic architecture, transferring methods from the field of medical image segmentation to the field of composite materials, and combining it with the multi-head attention mechanism of Transformers further improves the robustness and generalization of the method. Attached Figure Description

[0051] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0052] Figure 1 This is a flowchart of a material crack extraction method based on convolutional neural networks and Transformers according to an embodiment of the present invention;

[0053] Figure 2 This is a schematic diagram of the network architecture of a material crack extraction method based on convolutional neural network and Transformer according to an embodiment of the present invention;

[0054] Figure 3 This is a structural diagram of the Transformer module in an embodiment of the present invention for a material crack extraction method based on convolutional neural networks and Transformer. Detailed Implementation

[0055] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0056] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0057] Example 1

[0058] A method for extracting material cracks based on convolutional neural networks and Transformers includes:

[0059] This embodiment uses an in-situ CT experiment of C / SiC ceramic matrix composite material under high temperature environment as an example to illustrate the implementation of the method of the present invention. Figure 1 For the specific implementation process.

[0060] Step 1, CT scan: The sample is placed in a high-temperature device and a CT imaging device, and the sample is stretched. After each load is applied, a CT scan is performed. After the sample breaks, a CT scan is performed again to obtain all CT images of the sample from the beginning to the final breakage.

[0061] Step 2, Data Preprocessing: After obtaining the CT image data of the sample from start to breakage, the CT slice images are leveled and corrected. Then, the CT slice images of the sample are normalized to map the pixel values ​​of the CT images to between 0 and 1.

[0062] Step 3: Dataset Construction: Randomly select a portion of CT slice images, manually annotate the layered cracks, and perform cropping and data augmentation to construct the dataset.

[0063] Step 4, Model Construction: After inputting the image into the network, the model can automatically configure the network structure based on factors such as the image size, modality, and number of channels in the dataset, selecting the optimal architecture and dynamically adjusting key parameters such as patch size and damage function. A crack extraction model is constructed based on convolutional neural networks and Transformers, including an encoder structure, a decoder structure, and a feature fusion module. A schematic diagram of the model network architecture is shown below. Figure 2As shown.

[0064] Step 5, Model Training and Validation: The model is trained using a five-fold cross-validation method, and the highest accuracy set is selected as the final accuracy of the model.

[0065] Step 6, Model Testing: After model training is complete, input the test set constructed in Step 3 into the model, obtain the model processing results, calculate the Dice coefficients of the model-predicted images and manually labeled images, and average the Dice coefficients of all test set images to obtain the final crack extraction accuracy of the model. The formula for calculating the Dice coefficient is:

[0066]

[0067] Where TP represents the number of pixels that are predicted to be positive and are actually positive, FP represents the number of pixels that are predicted to be positive but are actually negative, and FN represents the number of pixels that are predicted to be negative but are actually positive.

[0068] Step 7, Other Data Processing: After the model training and testing are completed, the model prediction results are manually evaluated to see if they meet the expectations. If they meet the standards, all other CT images of the sample are input into the model for prediction. Finally, the crack evolution of the sample under different stress conditions is obtained, which provides support for subsequent damage evolution analysis.

[0069] Furthermore, the dataset construction includes: manually annotating the layered cracks in the XZ-direction slices of the samples, setting the pixel values ​​of the layered crack portions to 1, and setting the pixel values ​​of other background portions to 0. To ensure improved model training speed, the selected CT images are cropped to obtain 256*256 images, and the dataset is augmented through random rotation, flipping, and cropping to improve the model's generalization and robustness. All datasets are divided into training and testing sets in a 9:1 ratio, serving as the model's dataset. The training set is used for model training and validation, while the testing set is used for final accuracy testing.

[0070] Specifically:

[0071] C / SiC ceramic matrix composite samples were placed in a high-temperature device and a CT imaging device, and the samples were stretched until they were pulled to break under initial pre-tight conditions.

[0072] CT scans were performed on the samples without high temperature, acquiring 2235 projection images. The temperature was then increased to 1200℃, with gradually increasing force. CT scans were performed at 500N, 800N, 1100N, and after fracture, with 2235 projection images acquired in each scan. The six sets of CT projection images were reconstructed using an analytical reconstruction algorithm to obtain six sets of CT slice images.

[0073] After acquiring CT image data of the sample from its inception to fracture, the CT slices were leveled and calibrated to vertically center the sample, ensuring clear observation of the internal structure in the XZ direction slices. Based on this, the CT slices were normalized, mapping the pixel values ​​of the CT images to a range of 0-1.

[0074] From a series of CT images under different external force conditions, a portion was randomly selected as the dataset for model training. Layered cracks in the XZ-direction slices of the samples were manually labeled, with the pixel values ​​of the layered crack portions set to 1, and the pixel values ​​of other background portions set to 0.

[0075] To ensure faster model training, the selected CT images were cropped to 256*256 pixels. Data augmentation was then performed on the dataset using random rotation, flipping, and cropping to improve the model's generalization and robustness. All datasets were divided into a 9:1 ratio (training:test) as the model's dataset. The training set was used for model training and validation, while the test set was used for final accuracy testing.

[0076] Furthermore, the encoder structure comprises a six-layer network consisting of E0, E1, E2, E3, E4, and E5 layers.

[0077] The input to layer E0 is a preprocessed single-channel CT image. After passing through a 3*3 convolution, instance normalization, and LeakyReLU activation function, it outputs an initial feature map with 32 channels.

[0078] The input to layer E1 is the output of layer E0. First, it is downsampled by 2*2, and then the result is convolved to output a feature image with 64 channels.

[0079] The input to layer E2 is the output of layer E1. It is first downsampled by 2*2, and then the result is convolved to output a feature image with 128 channels.

[0080] The input to layer E3 is the output of layer E2. First, it undergoes Patch Embedding, where the input feature map is divided into 4x4 patches and projected onto the Transformer dimension. Through 4x4 convolutions, the output is a sequence of length 256, with an embedding length of 256. Then, positional encoding is added, and the sequence is fed into two Transformer modules for processing, resulting in a feature image with 256 channels. E3 introduces the global modeling capability of the Transformer to capture global semantic features.

[0081] One Transformer module includes a Layer Norm, a multi-head self-attention mechanism, a Layer Norm, and an MLP layer. The Layer Norm normalizes the features; the multi-head self-attention mechanism captures diverse dependencies of the input sequence from different subspaces by computing multiple sets of attention weights in parallel; the MLP layer consists of two fully connected layers and an activation function, used to compensate for the linearity limitations of the multi-head self-attention mechanism. The Transformer module structure is as follows: Figure 3 As shown.

[0082] The input to layer E4 is the output of layer E3. First, it undergoes a 2x2 downsampling convolution to increase the number of channels to 512. Then, it passes through a 4x4 convolution to obtain the patch embedding. Positional encoding is then added, and the input is fed into two Transformer modules for processing. The final output is a feature image with 512 channels. The Transformer modules used in E4 have the same structure as those used in E3. E4 extracts global semantic information from the low-resolution feature map, enhancing the features of the material crack region.

[0083] The input to layer E5 is the output of layer E4. A 2x2 convolution operation is performed on the input to obtain patch embedding, and then positional encoding is added. This is then fed into two Transformer modules for processing, and the final output is a feature image with 512 channels. Layer E5 refines global features and enhances the ability to identify crack boundaries and micro-crack structures.

[0084] Furthermore, the decoder structure consists of five network layers: D4, D3, D2, D1, and D0.

[0085] The input to layer D4 is the output of encoder layer E5. First, it is upsampled through two transposed convolutions. Then, the feature map from encoder layer E4 is concatenated with the upsampled feature map via skip connections. This convolution is then processed through two consecutive convolutional blocks, each consisting of two 3x3 convolutional layers. Each convolutional layer is followed by instance normalization and a LeakyReLU activation function. The final output is a feature map with 256 channels.

[0086] The input to layer D3 is the output of layer D4. First, the layers undergo two upsampling operations. Then, the feature maps from encoder layer E3 are concatenated, followed by two convolutional operations, ultimately outputting a feature map with 128 channels. The structure of the convolutional layers is identical to that of layer D4.

[0087] The input to layer D2 is the output of layer D3. First, the feature map is upsampled twice. Then, the feature map from encoder E2 is concatenated with the upsampled feature map via skip connections. Finally, two convolutional processes are performed, resulting in a feature map with 64 channels. The structure of the convolutional layers is consistent with that of layer D4.

[0088] The input to layer D1 is the output of layer D3. First, the input is upsampled twice. Then, the feature maps from the corresponding encoder layers are concatenated, followed by two convolutional processes. This yields the output feature map. The final output is a feature map with 32 channels. The structure of the convolutional layers is consistent with that of layer D4.

[0089] The input to the D0 layer network is the output of the D2 layer network, which is also the final output of the network. The number of channels is mapped to the number of judgment categories through 1*1 convolution, and then Sigmoid (binary) activation is used to finally obtain the crack extraction result.

[0090] Furthermore, the feature fusion module comprises:

[0091] An attention mechanism is added after each layer of the encoder network structure, performing attention in both spatial and channel dimensions, thus integrating the CBAM structure into the encoder structure design. The CBAM structure is added after layers E0, E1, and E2, and after layers E3, E4, and E5. The output format is then converted to the Transformer encoding format.

[0092] The CBAM structure consists of a channel attention mechanism and a spatial attention mechanism connected in series, dynamically weighting the channels and spatial dimensions of the input feature map. The channel attention mechanism passes the input feature map through two parallel max-pooling and average-pooling layers, transforming it into a C*1*1 map. Then, it passes through an MLP layer to generate channel weights. Finally, the original feature map is multiplied by the weighted channels to obtain the output of the channel attention mechanism. The spatial attention mechanism obtains two 1*H*W feature maps through max-pooling and average-pooling layers. These two feature maps are then concatenated, and spatial information is fused using a 7*7 convolution to generate a spatial weight map. Finally, the original feature map is multiplied positionally by the weight map to obtain the output of the spatial attention mechanism. These two modules are then combined sequentially to form the CBAM structure.

[0093] Furthermore, the damage function of the model is:

[0094] A loss function weighted by Dice and cross-entropy is used. Dice and cross-entropy are two commonly used loss functions that are often used together in image segmentation tasks. The Dice coefficient is used to evaluate the degree of overlap between the segments; a higher Dice coefficient indicates a better overlap between the predicted and ground truth values. The cross-entropy loss function measures the distributional difference between the predicted and ground truth values, and its expression is as follows:

[0095]

[0096] Where P represents the number of pixels predicted as positive and actually positive, FP represents the number of pixels predicted as positive but actually negative, and FN represents the number of pixels predicted as negative but actually positive. N represents the predicted class, y i Labels that indicate authenticity This represents the predicted probability of the model.

[0097] The model damage function in this embodiment is expressed as follows:

[0098] Loss = Loss Dice +Loss CrossEntroy

[0099] Furthermore, the model employs a five-fold cross-validation training strategy:

[0100] The training set was divided into 5 parts, with each part used as the validation set and the remaining parts used as the training set for model training. The set with the highest accuracy was then selected as the final accuracy of the model.

[0101] The Adam optimizer is used during training, with an initial learning rate of 0.01. The learning rate is dynamically adjusted during training using a polynomial decay mechanism, which helps to accelerate training and model convergence. The formula is as follows:

[0102]

[0103] Where baselr represents the initial learning rate, iters represents the current iteration number, max_iters represents the maximum number of iterations, and power represents the learning rate decay exponent, which is set to 0.9. Each fold trains for 200 epochs.

[0104] Furthermore, the model training proposed in this embodiment employs a five-fold cross-validation training method. The training set is divided into five parts, with each part serving as the validation set and the remaining parts serving as the training set for model training. Finally, the set with the highest accuracy is selected as the final accuracy of the model.

[0105] After model training is complete, the test set constructed in step 3 is input into the model to obtain the model processing results. The Dice coefficients of the model-predicted images and manually labeled images are calculated. The average of the Dice coefficients of all test set images is then calculated to obtain the final crack extraction accuracy of the model. The Dice coefficient measures the spatial overlap between the predicted segmentation result and the ground truth labeled image. Its range is between (0, 1). The closer its value is to 1, the higher the degree of overlap with the ground truth labeled image. The formula for calculating the Dice coefficient is:

[0106]

[0107] Where TP represents the number of pixels that are predicted to be positive and are actually positive, FP represents the number of pixels that are predicted to be positive but are actually negative, and FN represents the number of pixels that are predicted to be negative but are actually positive.

[0108] Furthermore, after the model training and testing are completed, the model prediction results are manually evaluated to see if they meet the expectations. If they meet the standards, all other CT images of the sample are input into the model for prediction to obtain crack-annotated images of CT slices of the sample under different stress conditions.

[0109] Furthermore, after processing the six sets of CT slice data, the continuous slices with annotated cracks were imported into the three-dimensional visualization software. This allowed observation of the evolution of the crack from the initial crack to the final fracture, providing support for subsequent damage evolution analysis.

[0110] Example 2

[0111] To more clearly demonstrate the technical solution of this invention, the following description uses a ceramic matrix composite material sample as an example:

[0112] This embodiment discloses a material crack extraction method using convolutional neural networks and Transformers, including:

[0113] 1. CT scan: The C / SiC ceramic matrix composite sample is placed in a high-temperature device and CT imaging equipment, and the sample is stretched in an initial pre-tightened state until the sample is stretched to the point of fracture.

[0114] 1.1 CT Image Acquisition: The sample was CT scanned without high temperature, and 2235 projection images were acquired. Then the temperature was increased to 1200℃, and the force was gradually increased. CT scans were performed at 500N, 800N, 1100N and after fracture, with 2235 projection images acquired for each CT scan.

[0115] 1.2 CT Image Reconstruction: Six sets of CT experimental projection images were reconstructed using an analytical reconstruction algorithm to obtain six sets of CT slice images.

[0116] 2. Data Preprocessing: After acquiring the CT image data of the sample from start to fracture, the CT slices are leveled and calibrated to vertically center the sample, ensuring clear observation of the internal structure in the XZ direction slices. Based on this, the CT slices are normalized, mapping the pixel values ​​of the CT images to a range of 0-1.

[0117] 3. Dataset Construction: A portion of CT images under a series of different external force conditions is randomly selected as the dataset for model training.

[0118] 3.1 Data annotation: The layered cracks in the XZ direction slices of the sample were manually annotated, with the pixel values ​​of the layered cracks set to 1 and the pixel values ​​of the other background parts set to 0.

[0119] 3.2 Data Augmentation: To ensure faster model training, the selected CT images were cropped to 256*256 pixels. Data augmentation was then performed on the dataset through random rotation, flipping, and cropping to improve the model's generalization and robustness. All datasets were divided into a 9:1 ratio (training set:test set) as the model's dataset. The training set was used for model training and validation, while the test set was used for final accuracy testing.

[0120] 4. Model Building:

[0121] 4.1 Automated configuration: After the image is input into the network, the model can automatically configure the network structure based on the size of the dataset images, the modality of the images, the number of channels, etc., select the optimal architecture, and dynamically adjust key parameters such as patch size and damage function.

[0122] 4.2 Model Construction: The network architecture proposed in this embodiment is based on the nnUNet model, which includes an encoder structure, a decoder structure, and a feature fusion module.

[0123] The encoder structure primarily extracts semantic features through downsampling. The encoder consists of five network layers: E0, E1, E2, E3, E4, and E5.

[0124] The input to layer E0 is a preprocessed single-channel CT image. After passing through a 3*3 convolution, instance normalization, and LeakyReLU activation function, it outputs an initial feature map with 32 channels.

[0125] The input to layer E1 is the output of layer E0. First, it is downsampled by 2*2, and then the result is convolved to output a feature image with 64 channels.

[0126] The input to layer E2 is the output of layer E1. It is first downsampled by 2*2, and then the result is convolved to output a feature image with 128 channels.

[0127] The input to layer E3 is the output of layer E2. First, it undergoes Patch Embedding, where the input feature map is divided into 4x4 patches and projected onto the Transformer dimension. Through 4x4 convolutions, the output is a sequence of length 256, with an embedding length of 256. Then, positional encoding is added, and the sequence is fed into two Transformer modules for processing, resulting in a feature image with 256 channels. E3 introduces the global modeling capability of the Transformer to capture global semantic features.

[0128] One Transformer module includes a Layer Norm, a multi-head self-attention mechanism, a Layer Norm, and an MLP layer. The Layer Norm is used to normalize the features; the multi-head self-attention mechanism captures diverse dependencies of the input sequence from different subspaces by computing multiple sets of attention weights in parallel; the MLP layer consists of two fully connected network layers and activation functions, which are used to supplement the linearity of the multi-head self-attention mechanism.

[0129] The input to layer E4 is the output of layer E3. First, it undergoes a 2x2 downsampling convolution to increase the number of channels to 512. Then, it passes through a 4x4 convolution to obtain the patch embedding. Positional encoding is then added, and the resulting data is fed into two Transformer modules for processing. The final output is a feature image with 512 channels. The Transformer modules have the same structure as those used in layer E3. E4 extracts global semantic information from the low-resolution feature map, enhancing the features of the material crack region.

[0130] The input to layer E5 is the output of layer E4. A 2x2 convolution operation is performed on the input to obtain patch embedding, and then positional encoding is added. This is then fed into two Transformer modules for processing, and the final output is a feature image with 512 channels. Layer E5 refines global features and enhances the ability to identify crack boundaries and micro-crack structures.

[0131] Decoder Structure: The decoder structure progressively upsamples the low-resolution features extracted by the encoder and restores spatial details, ultimately generating an accurate segmentation result. The decoder consists of five layers: D4, D3, D2, D1, and D0.

[0132] The input to layer D4 is the output of encoder layer E5. First, it is upsampled through two transposed convolutions. Then, the feature map from encoder layer E4 is concatenated with the upsampled feature map via skip connections. This convolution is then processed through two consecutive convolutional blocks, each consisting of two 3x3 convolutional layers. Each convolutional layer is followed by instance normalization and a LeakyReLU activation function. The final output is a feature map with 256 channels.

[0133] The input to layer D3 is the output of layer D4. First, the layers undergo two upsampling operations. Then, the feature maps from encoder layer E3 are concatenated, followed by two convolutional operations, ultimately outputting a feature map with 128 channels. The structure of the convolutional layers is identical to that of layer D4.

[0134] The input to layer D2 is the output of layer D3. First, the feature map is upsampled twice. Then, the feature map from encoder E2 is concatenated with the upsampled feature map via skip connections. Finally, two convolutional processes are performed, resulting in a feature map with 64 channels. The structure of the convolutional layers is consistent with that of layer D4.

[0135] The input to layer D1 is the output of layer D3. First, the input is upsampled twice. Then, the feature maps from the corresponding encoder layers are concatenated, followed by two convolutional processes. This yields the output feature map. The final output is a feature map with 32 channels. The structure of the convolutional layers is consistent with that of layer D4.

[0136] The input to the D0 layer network is the output of the D2 layer network, which is also the final output of the network. The number of channels is mapped to the number of judgment categories through 1*1 convolution, and then Sigmoid (binary) activation is used to finally obtain the crack extraction result.

[0137] Feature fusion module: In this embodiment, an attention mechanism is added after each layer of the encoder network structure to perform attention in the spatial and channel dimensions, thus integrating the CBAM structure into the encoder structure design. The CBAM structure is added after layers E0, E1, and E2, and after layers E3, E4, and E5. The output format is then converted to Transformer format.

[0138] The CBAM structure consists of a channel attention mechanism and a spatial attention mechanism connected in series, dynamically weighting the channels and spatial dimensions of the input feature map. The channel attention mechanism passes the input feature map through two parallel max-pooling and average-pooling layers, transforming it into a C*1*1 map. Then, it passes through an MLP layer to generate channel weights. Finally, the original feature map is multiplied by the weighted channels to obtain the output of the channel attention mechanism. The spatial attention mechanism obtains two 1*H*W feature maps through max-pooling and average-pooling layers. These two feature maps are then concatenated, and spatial information is fused using a 7*7 convolution to generate a spatial weight map. Finally, the original feature map is multiplied positionally by the weight map to obtain the output of the spatial attention mechanism. These two modules are then combined sequentially to form the CBAM structure.

[0139] After the image is input into the network, the loss is calculated based on the loss function. A loss function weighted by Dice and cross-entropy is used, which are two commonly used loss functions often used together in image segmentation tasks. The Dice coefficient is used to evaluate the degree of overlap between the segments; a higher Dice coefficient indicates a better overlap between the predicted and ground truth values. The cross-entropy loss function measures the distribution difference between the predicted and ground truth values, and its expression is as follows:

[0140]

[0141] Where P represents the number of pixels predicted as positive and actually positive, FP represents the number of pixels predicted as positive but actually negative, and FN represents the number of pixels predicted as negative but actually positive. N represents the predicted class, y i Labels that indicate authenticity This represents the predicted probability of the model.

[0142] The model damage function in this embodiment is expressed as follows:

[0143] Loss = Loss Dice +Loss CrossEntroy

[0144] 5. Model Training and Validation: The model training proposed in this embodiment adopts a five-fold cross-validation method. The training set is divided into five parts, each of which is used as the validation set, and the remaining parts are used as the training set for model training. Finally, the set with the highest accuracy is selected as the final accuracy of the model.

[0145] The Adam optimizer is used during training, with an initial learning rate of 0.01. The learning rate is dynamically adjusted during training using a polynomial decay mechanism, which helps to accelerate training and model convergence. The formula is as follows:

[0146]

[0147] Where baselr represents the initial learning rate, iters represents the current iteration number, max_iters represents the maximum number of iterations, and power represents the learning rate decay exponent, which is set to 0.9. Each fold trains for 200 epochs.

[0148] 6. Model Testing: After model training, input the test set constructed in step 3 into the model, obtain the model processing results, calculate the Dice coefficients of the model-predicted images and manually labeled images, and average the Dice coefficients of all test set images to obtain the final crack extraction accuracy of the model. The Dice coefficient measures the spatial overlap between the predicted segmentation result and the ground truth labeled image. Its range is between (0, 1). The closer its value is to 1, the higher the overlap with the ground truth labeled image. The formula for calculating the Dice coefficient is:

[0149]

[0150] Where TP represents the number of pixels that are predicted to be positive and are actually positive, FP represents the number of pixels that are predicted to be positive but are actually negative, and FN represents the number of pixels that are predicted to be negative but are actually positive.

[0151] 7. Other data processing: After the model training and testing are completed, the model prediction results are manually evaluated to see if they meet the expectations. If they meet the standards, all other CT images of this sample are input into the model for prediction to obtain crack annotation images of CT slices of the sample under different stress conditions.

[0152] 8. Data Visualization: After processing the 6 sets of CT slice data, the continuous slices with annotated cracks are imported into the 3D visualization software. The evolution process of the crack from the initial crack to the final fracture can be observed, providing support for subsequent damage evolution analysis.

[0153] Example 3

[0154] This embodiment provides a material crack extraction system based on convolutional neural networks and Transformers. This system achieves high-precision dynamic monitoring of material cracks by integrating multimodal feature fusion and adaptive learning mechanisms. The system architecture is as follows: Figure 1 As shown, it includes the following core modules:

[0155] Data acquisition module:

[0156] Perform a CT scan, apply progressive tensile loading to the sample and simultaneously acquire CT image sequences, covering the entire lifecycle from the initial state to fracture.

[0157] The original CT images are leveled and normalized to 0-1 to eliminate equipment noise and grayscale distribution deviation.

[0158] Construct a labeled dataset: Manually label the layered crack regions of the XZ direction slices (pixel value 1 represents cracks, 0 represents background), randomly crop to a size of 256×256, and perform data augmentation by rotation and reversal, and divide the training set and test set in a 9:1 ratio.

[0159] Model building module:

[0160] Encoder structure: Employs a six-layer cascaded network (E0-E5), wherein:

[0161] Layers E0-E2 extract local features through convolutional downsampling, and each layer is followed by a CBAM attention module (containing a channel and spatial attention cascaded structure) to dynamically enhance the response in the crack region.

[0162] Layers E3-E5 introduce a Transformer module that divides the feature map into 4×4 patches and embeds positional encoding, capturing global crack topology through a multi-head self-attention mechanism.

[0163] Decoder structure: A five-layer upsampling network (D4-D0) is used. Each layer restores the resolution through transposed convolution and is connected to the corresponding layer features of the encoder by skip connection. The crack boundary is refined by combining double convolutional blocks.

[0164] Feature fusion: CBAM structure is embedded at the output of each layer of the encoder. Its channel attention generates weights through MLP, and spatial attention generates spatial weights through 7×7 convolution, thereby achieving cross-dimensional feature optimization.

[0165] Training optimization module:

[0166] A five-fold cross-validation strategy was adopted, with each fold training for 200 epochs. The Adam optimizer's initial learning rate was set to 0.01 and dynamically adjusted using polynomial decay (decay exponent 0.9).

[0167] The loss function is a weighted combination of the Dice coefficient and the cross-entropy:

[0168]

[0169] Where P represents the number of pixels predicted as positive and actually positive, FP represents the number of pixels predicted as positive but actually negative, and FN represents the number of pixels predicted as negative but actually positive. N represents the predicted class, y i Labels that indicate authenticity This represents the predicted probability of the model.

[0170] The model damage function in this embodiment is expressed as follows:

[0171] Loss = Loss Dice +LossCrossEntroy

[0172] Crack Analysis Module:

[0173] Load the optimal model parameters, output a binary crack segmentation map for the test set, and calculate the Dice coefficient as an accuracy indicator. Visualize the crack evolution process at different stress stages and generate a three-dimensional dynamic crack propagation map.

Claims

1. A method for extracting material cracks based on convolutional neural networks and Transformers, characterized in that, The method includes the following steps: Step 1, CT scan: Place the sample in a CT imaging device, stretch the sample, and perform a CT scan after each force is applied to obtain all CT images of the sample from the beginning to the final fracture. Step 2, Data Preprocessing: After obtaining the CT image data of the sample from start to breakage, the CT slice images are leveled, corrected, and normalized to 0-1. Step 3: Dataset Construction: Randomly select a portion of CT slice images, manually annotate the layered cracks, and perform cropping and data augmentation to construct the dataset; Step 4, Model Building: After the image is input into the network, the model automatically configures the network structure, selects the optimal architecture, and builds a crack extraction model based on convolutional neural networks and Transformers, which includes an encoder structure, a decoder structure, and a feature fusion module. In step 4, the encoder structure includes: a network consisting of six layers, namely E0, E1, E2, E3, E4, and E5 layers; The input to layer E0 is a preprocessed single-channel CT image, which is then processed through a 3-channel... The convolution with 3, instance normalization, and LeakyReLU activation function output an initial feature map with 32 channels; The input to layer E1 is the output of layer E0, and it first passes through 2...

2. Downsample, then perform convolution operation on the result to output a feature image with 64 channels; The input to layer E2 is the output of layer E1, and it first passes through layer 2...

2. Downsample, then perform convolution operation on the result to output a feature image with 128 channels; The input to layer E3 is the output of layer E2. First, it undergoes Patch Embedding, which divides the input feature map into 4... A 4-patch is applied and projected onto the Transformer dimension; through 4 The convolution of 4 outputs a sequence with a length of 256 and an embedding length of 256. Then, positional encoding is added, and the sequence is fed into two Transformer modules for processing to obtain a feature image with 256 channels. E3 introduces the global modeling capability of Transformer to capture global semantic features. One Transformer module includes a Layer Norm, a multi-head self-attention mechanism, a Layer Norm, and an MLP layer. The Layer Norm is used to normalize the features. The multi-head self-attention mechanism captures the diverse dependencies of the input sequence from different subspaces by computing multiple sets of attention weights in parallel. The MLP layer consists of two fully connected network layers and activation functions to supplement the linearity of the multi-head self-attention mechanism. The input to layer E4 is the output of layer E3, and it first passes through 2...

2. Downsampling convolution increases the number of channels to 512, then passes through 4... Four convolutions are performed to obtain Patch Embedding, then positional encoding is added, and the input is fed into two Transformer modules for processing. The final output is a feature image with 512 channels. The Transformer module has the same structure as the Transformer module used in E3. E4 extracts global semantic information from the low-resolution feature map to enhance the features of the material crack region. The input to layer E5 is the output of layer E4. The input is then subjected to 2... Two convolutional operations are performed to obtain Patch Embedding, then positional encoding is added, and the input is fed into two Transformer modules for processing. The final output is a feature image with 512 channels. The E5 layer refines the global features and enhances the ability to identify crack boundaries and micro-crack structures. Step 5, Model Training and Validation: The model is trained using a five-fold cross-validation method, and the highest accuracy set is selected as the final accuracy of the model. Step 6, Model Testing: After the model training is completed, calculate the Dice coefficients of the model-predicted image and the manually labeled image to obtain the final crack extraction accuracy of the model. Step 7, Other Data Processing: After the model training and testing are completed, all other CT images of the sample are input into the model for prediction, and finally the crack evolution of the sample under different stress conditions is obtained.

2. The material crack extraction method based on convolutional neural networks and Transformers according to claim 1, characterized in that, In step 3, the dataset construction includes: manually annotating the layered cracks in the XZ direction slices of the samples, setting the pixel values ​​of the layered crack portions to 1, and setting the pixel values ​​of other background portions to 0; cropping the selected CT images to obtain 256 The dataset contains 256 images, which are augmented through random rotation, flipping, and cropping. All datasets are divided into training and test sets in a 9:1 ratio, serving as the model's dataset. The training set is used for model training and validation, while the test set is used for the final accuracy test of the model.

3. The material crack extraction method based on convolutional neural networks and Transformers according to claim 1, characterized in that, In step 4, the decoder structure includes a five-layer network consisting of layers D4, D3, D2, D1, and D0. The input to layer D4 is the output of layer E5 of the encoder network. First, it is upsampled through two transposed convolutions. Then, the feature map from encoder layer E4 is concatenated with the upsampled feature map via skip connections. Finally, it is processed through two consecutive convolutional blocks, each consisting of two 3x3 convolutional layers. The convolutional layers have 3 layers, each followed by instance normalization and the LeakyReLU activation function; the final output is a feature map with 256 channels. The input to the D3 layer network is the output of the D4 layer network. First, it is upsampled twice, then the feature maps of the encoder E3 layer are concatenated, and then two convolution processes are performed to finally output a feature map with 128 channels. The structure of the convolutional layer is the same as that of the D4 layer. The input to the D2 layer network is the output of the D3 layer network. First, it is upsampled twice. Then, the feature map of encoder E2 is concatenated with the upsampled feature map through skip connections. Then, it is convolved twice to finally output a feature map with 64 channels. The structure of the convolutional layer is the same as that of the D4 layer. The input to the D1 layer network is the output of the D3 layer network. First, it is upsampled twice, then the feature maps of the corresponding layers of the encoder are concatenated, and then two convolution processes are performed to obtain the output feature map; the final output is a feature map with 32 channels; the structure of the convolutional layer is the same as that of the D4 layer. The input to layer D0 is the output of layer D2, which is also the final output of the network. This is achieved through 1... A convolution of 1 maps the number of channels to the number of categories, and then a sigmoid activation is applied to finally obtain the crack extraction result.

4. The material crack extraction method based on convolutional neural networks and Transformers according to claim 1, characterized in that, In step 4, the feature fusion module comprises: An attention mechanism is added after each network layer of the encoder structure to perform attention in the spatial and channel dimensions, and the CBAM structure is integrated into the design of the encoder structure. The CBAM structure is added after the E0, E1, and E2 network layers, and after the E3, E4, and E5 network layers. Then the output format is converted into the Transformer encoding format. The CBAM structure consists of a channel attention mechanism and a spatial attention mechanism connected in series, dynamically weighting the channel and spatial dimensions of the input feature map. The channel attention mechanism passes the input feature map through two parallel max pooling layers and average pooling layers, transforming the feature map into a C1... 1 1. Then, the channel weights are generated through an MLP layer. Finally, the original feature map is multiplied by the weighted channels to obtain the output of the channel attention mechanism. The spatial attention mechanism obtains two values ​​through max pooling and average pooling layers. H The feature map of W is then used, and the two feature maps are concatenated using a 7-bit algorithm. The convolution of module 7 fuses spatial information to generate a spatial weight map. Finally, the original feature map and the weight map are multiplied position by position to obtain the output of the spatial attention mechanism. Finally, the two modules are combined in series to form the CBAM structure.

5. The material crack extraction method based on convolutional neural networks and Transformers according to claim 1, characterized in that, In step 4, the damage function of the model is: A loss function weighted by Dice and cross-entropy is used. The Dice coefficient is used to evaluate the degree of overlap in the segments; a higher Dice coefficient indicates a better overlap between the predicted and true values. The cross-entropy loss function is used to measure the distributional difference between the predicted and true values, and its expression is as follows: Where TP represents the number of pixels predicted as positive and actually positive, FP represents the number of pixels predicted as positive but actually negative, FN represents the number of pixels predicted as negative but actually positive, and N represents the predicted category. Labels that indicate authenticity This represents the predicted probability of the model; The model damage function is expressed as: 。 6. The material crack extraction method based on convolutional neural networks and Transformers according to claim 1, characterized in that... In step 5, the model's five-fold cross-validation training strategy is as follows: The training set is divided into 5 parts, each of which is used as the validation set, and the remaining parts are used as the training set for model training. Finally, the set with the highest accuracy is selected as the final accuracy of the model. The Adam optimizer was used during training, with an initial learning rate of 0.

01. The learning rate was dynamically adjusted during training using a polynomial decay formula, as follows: Where baselr represents the initial learning rate, iters represents the current iteration number, max_iters represents the maximum number of iterations, and power represents the learning rate decay exponent, which is set to 0.

9. Each fold trains for 200 epochs.

7. A material crack extraction system based on convolutional neural networks and Transformers, wherein the system applies the method described in any one of claims 1-6, characterized in that, include: Data acquisition module: configured to perform CT scan and preprocessing operations and construct data in steps 1-3; Model building module: configured to implement the encoder structure, decoder structure and feature fusion module in step 4; Training optimization module: Configured to perform five-fold cross-validation and dynamic learning rate adjustment in step 5; Crack Analysis Module: Configured to generate crack segmentation results after running the training model and calculate the Dice accuracy index.

Images