A deep learning segmentation method for carbon fiber composite material dataset

Abstract

The application provides a kind of deep learning segmentation method of carbon fiber composite material data set, obtains the image of relevant carbon fiber composite material by XCT tomography technique, constructs the original data set of model, then generates two-dimensional random synthetic carbon fiber image by parameterized modeling, and generates the same virtual data set as the real structure by the adversarial learning style transfer network Pix2PixHD, the original data set and the virtual data set are input as a hybrid data set into the semantic segmentation network based on Swin-Tranformer, and finally form a semantic segmentation network capable of accurately segmenting carbon fiber composite material image. The method solves the problem of insufficient original data set and difficult manual annotation, can realize intelligent and accurate segmentation of composite material image, and saves a lot of labor cost.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and more specifically to a deep learning segmentation method for carbon fiber composite material datasets. Background Technology

[0002] The properties of carbon fiber reinforced polymer (CFRP) composites are closely related not only to the properties of the constituent materials but also to their morphology. Geometric defects in CFRP can induce substantial perturbations to the ideal structure, triggering failure mechanisms and impairing strength and lifespan. A precise three-dimensional representation of the CFRP structure, where different material phases are segmented and labeled, facilitates the rational selection of materials, manufacturing processes, and operating parameters. XCT (extracorporeal computed tomography) technology has been applied to identify the properties of CFRP, analyzing its internal structure and describing its morphological composition in a non-destructive manner. The application of XCT primarily involves image data acquisition and data analysis. To extract useful information from the images, it is necessary to analyze them after acquisition. The most crucial step in this detailed analysis is determining the boundaries of the yarns in the two-dimensional image.

[0003] Image quality is equally important for quantitative analysis. Image quality limits the measurable types of irregularities in CFRP, as well as the extent to which the spatial distribution of the matrix material and matrix porosity can be measured. Often, it is very difficult to automatically extract individual warp or weft yarns when the yarns are in contact with each other, especially in fully compacted CFRP, primarily due to low contrast between key components, noisy data, and the low attenuation rate of carbon-based materials. Due to high texture and noise, image intensity cannot provide sufficient information for accurate segmentation of CFRP, especially for yarns in different directions (weft, warp, adhesive, etc.).

[0004] This difficulty in automatic segmentation can now be addressed using neural networks. Many artificial intelligence image processing algorithms are used for detecting image information after composite material imaging, such as Artificial Neural Networks (ANN), Multilayer Perceptrons (MLP), Recurrent Neural Networks (RNN), and Support Vector Machines (SVM). However, these algorithms often require a large amount of data samples, but the difficulty and time-consuming nature of segmenting these datasets often result in insufficient data to achieve adequate segmentation accuracy. Summary of the Invention

[0005] To address the aforementioned issues, this invention proposes a deep learning segmentation method for carbon fiber composite material datasets. This method primarily involves expanding the dataset of carbon fiber reinforced composite material images obtained through XCT tomography and constructing a semantic segmentation neural network model with good performance using the synthetic dataset.

[0006] To achieve the above objectives, this invention provides a deep learning segmentation method for carbon fiber composite material datasets, comprising the following steps:

[0007] Step 1: Manually segment randomly selected composite material XCT inspection images using manual annotation methods to construct the original dataset;

[0008] Step 2: Perform parametric statistical analysis based on the original dataset, and perform parametric random modeling to expand the dataset and construct a hybrid dataset;

[0009] Step 3: Based on the hybrid dataset, construct the Swin-Tranformer semantic segmentation network;

[0010] Step 4: Based on the Swin-Tranformer semantic segmentation network, generate a complete CFRP semantic segmentation image as the final result.

[0011] Preferably, step 1 specifically includes:

[0012] Step 1.1: Obtain XCT images of the composite material using XCT tomography.

[0013] Step 1.2: Remove the air portion from the outer layer of the composite material XCT image, and extract one image every 10 frames;

[0014] Step 1.3: Manually filter and label all extracted images to obtain the original dataset.

[0015] Preferably, the original dataset mainly includes four categories: warp yarns, weft yarns, resin matrix, and pores.

[0016] Preferably, step 2 specifically includes:

[0017] Step 2.1: Perform parametric statistical analysis based on the microstructural features of the original dataset;

[0018] Step 2.2: Based on the results of parametric statistical analysis, the dataset is expanded using parametric stochastic modeling to generate a synthetic dataset;

[0019] Step 2.3: Combine the synthetic dataset with the original dataset to form a hybrid dataset.

[0020] Preferably, step 2.1 specifically includes:

[0021] Step 2.1.1: Collect shape parameters;

[0022] Step 2.1.2: Fit the distribution of the collected shape parameters;

[0023] Step 2.1.3: Perform statistical analysis on the shape parameters of the fitted distribution;

[0024] The statistical analysis includes: statistical analysis of the geometric parameters of warp yarns with a rectangular profile and statistical analysis of the geometric parameters of weft yarns with an elliptical profile.

[0025] Preferably, step 2.2 specifically includes:

[0026] Step 2.2.1: Generate a warp random model based on warp parameterized statistical analysis;

[0027] Step 2.2.2: Generate a weft yarn stochastic model based on the parametric statistical analysis of the weft yarn;

[0028] Step 2.2.3: Based on the warp random model and the weft random model, generate artificially synthesized label images;

[0029] Step 2.2.4: Based on the artificially synthesized label image, generate a carbon fiber composite material image with the same microstructure as the real one, and use the carbon fiber composite material image with the same microstructure as the real one as the synthetic dataset.

[0030] Preferably, step 2.2.4 specifically includes:

[0031] Step 2.2.4.1: Modify the generation part of the Pix2PixHD network so that the generation part trains two generator models at different scales simultaneously, and fuses the features of the two generator models at different scales.

[0032] Step 2.2.4.2: Based on the generator model after feature fusion, generate an image of carbon fiber composite material with the same microstructure as the real one.

[0033] Preferably, step 3 specifically includes:

[0034] Step 3.1 The hybrid dataset is processed through a Swin-Tranformer network to obtain feature vector maps at four scales;

[0035] Step 3.2 Input the feature vector maps of the four scales into the UperNet network for processing to obtain the class prediction vector. Based on the class prediction vector, construct the Swin-Tranformer semantic segmentation network.

[0036] Preferably, step 3.1 specifically includes:

[0037] Step 3.1.1 Input the training images from the mixed dataset into the network. After processing by the block partitioning module, the image is obtained. The image is divided into different small blocks according to pixels. The pixels of the small blocks in all channels are stretched into one-dimensional feature vectors. All one-dimensional feature vectors are combined into a feature vector map.

[0038] Step 3.1.2 processes the feature vector map through four stages to obtain four feature vector maps at different scales. Each stage consists of block merging and a Swin-Transformer Block.

[0039] Preferably, step 3.2 specifically includes:

[0040] Step 3.2.1 The bottom feature vector map is processed by the pyramid pooling module to obtain the first multi-scale composite feature map;

[0041] Step 3.2.2 Perform multi-scale feature fusion processing on the four feature vector maps to obtain a second composite feature map with multiple scales;

[0042] Step 3.2.3 Normalize the two composite feature maps using the softmax function to obtain the class prediction vector.

[0043] Compared with the prior art, the present invention has the following advantages and technical effects:

[0044] This invention discloses a deep learning segmentation method for carbon fiber composite material datasets. Images of relevant carbon fiber composite materials are acquired using XCT (extracorporeal computed tomography) technology to construct the original dataset for the model. Then, two-dimensional synthetic images are generated through parametric modeling and combined with a Pix2PixHD style transfer network to generate a synthetic dataset. The original and synthetic datasets are then input as a hybrid dataset into a Swin-Tranformer-based semantic segmentation network, ultimately forming a semantic segmentation network capable of completely segmenting XCT images. This model can achieve pixel-level segmentation of composite material images, solving the problems of insufficient data volume due to the difficulty and time-consuming manual annotation of the original dataset, resulting in more accurate segmentation results and saving significant manual costs. Attached Figure Description

[0045] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0046] Figure 1 This is a schematic diagram of the deep learning segmentation method for a carbon fiber composite material dataset according to the present invention.

[0047] Figure 2This is a schematic diagram of the process steps for generating a synthetic dataset according to the present invention;

[0048] Figure 3 This is a schematic diagram of the present invention for the parametric analysis of the microstructure of real datasets;

[0049] Figure 4 The following is an example of the artificially synthesized data results of the present invention, wherein (a) is the original dataset image, (b) is the original dataset label image, (c) is the synthesized dataset image, and (b) is the synthesized dataset label image. Detailed Implementation

[0050] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0051] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0052] Example 1:

[0053] like Figures 1-4 As shown, this invention provides a deep learning segmentation method for carbon fiber composite material datasets, comprising the following steps:

[0054] Step 1: Manually segment randomly selected composite material XCT inspection images using manual annotation methods to construct the original dataset;

[0055] Step 2: Perform parametric statistical analysis based on the original dataset, and perform parametric stochastic modeling to expand the dataset and construct a hybrid dataset;

[0056] Step 3: Construct the Swin-Tranformer semantic segmentation network based on the hybrid dataset;

[0057] Step 4: Based on the Swin-Tranformer semantic segmentation network, generate a complete CFRP semantic segmentation image as the final result.

[0058] In this embodiment, step 1 specifically includes:

[0059] Step 1.1: Obtain XCT images of the composite material using XCT tomography. Assume a total of 1450 images are obtained.

[0060] Step 1.2: Remove the air portion of the outer layer of the composite material XCT image, extracting one image every 10 frames to ensure that target information is not lost, while reducing the workload of sample extraction;

[0061] Step 1.3: Manually filter and label all extracted images to obtain the original dataset of 139 images.

[0062] In this embodiment, the original dataset mainly includes four categories: warp yarn, weft yarn, resin matrix, and pores.

[0063] In this embodiment, step 2 specifically includes:

[0064] Step 2.1: Perform parametric statistical analysis based on the microstructural features of the original dataset;

[0065] In this embodiment, step 2.1 specifically includes:

[0066] Step 2.1.1: Collect shape parameters;

[0067] Step 2.1.2: Fit the distribution of the collected shape parameters;

[0068] Step 2.1.3: Perform statistical analysis on the shape parameters of the fitted distribution;

[0069] The statistical analysis includes: statistical analysis of the geometric parameters of warp yarns with a rectangular profile and statistical analysis of the geometric parameters of weft yarns with an elliptical profile.

[0070] The method for statistical analysis of warp geometric parameters is as follows: label the warp fibers in the image, and statistically analyze the average width, standard deviation of width, length and other information of the warp, calculate the centroid of each warp, and read the distance between the centroids of each warp to generate multi-strand warp.

[0071] The method for statistical analysis of weft yarn geometric parameters is as follows: An ellipse is fitted using the FitEllipse function in the OpenCV library. The weft yarn tilt angle is calculated. The farthest distance of each weft yarn in the image is manually labeled and used as the major axis. The two intersection points of the lines perpendicular to the major axis are calculated to determine the minor axis length, thus obtaining the ratio of the major axis to the minor axis. Each collected weft yarn data point is substituted into the ellipse representation equation with the same center point. The ratio of the actual radius to the ideal radius of each pixel at each angle is calculated to achieve ellipse irregularity.

[0072] Statistical analysis of warp and weft geometric parameters: The fitter library is used to automatically search for distributions supported in the scipy library and fit them. The mean width, standard deviation, and length of the warp yarns are automatically fitted, as well as the major diameter, minor diameter ratio, and tilt angle of the weft yarns.

[0073] Step 2.2: Based on the results of parametric statistical analysis, the dataset is expanded by parametric random modeling to prevent the semantic segmentation effect from falling short of expectations due to the small size of the original dataset, and a synthetic dataset is generated.

[0074] In this embodiment, step 2.2 specifically includes:

[0075] Step 2.2.1: Generate a warp random model based on warp parameterized statistical analysis;

[0076] Specifically, the process involves generating random warp threads by first creating a straight line as its centerline, and then filling the width of the centerline. The centerline is formed by selecting 4 to 6 nodes from a straight line, dividing it into two equal parts, including the upper and lower endpoints. The nodes in the middle part are randomly offset by coordinates, with the maximum offset determined empirically. The generated points are then fitted with a curve using spline interpolation, which is then used as the centerline.

[0077] Based on the generated centerline length, a corresponding number of widths are generated. One to three nodes are randomly selected, and the generated widths are sorted according to these nodes, using an alternating ascending and descending order to approximate the uneven width shape of real warp yarns. This method is also used to generate the required tip portions for other categories. The generated curved, irregular rectangles are then subjected to random elastic deformation. This random elastic deformation is achieved by randomly generating two matrices of the same size as the image, representing the horizontal and vertical offsets of each pixel's coordinates, respectively. Gaussian filtering is used to smooth these matrices, ensuring matrix continuity and forming coordinate offset matrices in two directions. The original image coordinates are then applied with these two offsets, and the grayscale values ​​of the offset pixels are interpolated to reconstruct the image, achieving the effect of random elastic deformation. Additionally, median filtering is used to remove burrs generated during the width sorting process. Different types of warp yarns are arranged at regular intervals.

[0078] Step 2.2.2: Generate a weft yarn stochastic model based on the parametric statistical analysis of the weft yarn;

[0079] Specifically, the process includes: generating a basic ellipse shape using the fitted minor axis length and major-minor axis ratio as shape parameters; setting nodes of equal length on the ellipse circumference; generating an offset for all nodes based on the fitted weft offset; applying this offset to the length of all nodes (720 nodes were collected during generation); then using a special smoothing method to take the midpoint of adjacent nodes of the ellipse; after multiple smoothing operations, median filtering is applied to the entire ellipse to eliminate burrs; and finally, the ellipse is tilted based on the fitted tilt angle.

[0080] Step 2.2.3: Generate artificially synthesized label images based on the warp and weft random models;

[0081] Specifically, the process includes: For generating the composite label image, the original image size is used as the generated image size. The upper and lower boundaries of the resin area are manually defined. Warp yarns are generated first, and their positions are arranged according to the generated spacing until the remaining width of the image cannot accommodate the next generated spacing and warp yarn width. Then, the two ends of the warp yarns are selected as the lateral boundaries of the resin area. Next, weft yarns are generated, and their placement is randomly selected. The placement area expands upwards and downwards, generating the shape of truncated weft yarns on the original image boundaries. The overlap rate with the already placed fibers is then calculated to determine whether to place the weft yarn. Finally, placement stops when a certain number of weft yarn placement failures occur or the total fiber area of ​​the image reaches a certain threshold.

[0082] Step 2.2.4: Based on the artificially synthesized label images, generate carbon fiber composite material images with the same microstructure as the real ones, and use the carbon fiber composite material images with the same microstructure as the real ones as the synthetic dataset.

[0083] In this embodiment, step 2.2.4 specifically includes:

[0084] Step 2.2.4.1: Modify the generation part of the Pix2PixHD network so that the generation part trains generator models at two different scales simultaneously and fuses the features of the generator models at the two different scales.

[0085] Step 2.2.4.2: Based on the generator model after feature fusion, generate an image of carbon fiber composite material with the same microstructure as the real one.

[0086] The Pix2PixHD network is a conditional generative adversarial network (cGAN) that uses supervised learning to make the generated image more accurately similar to the input target image in terms of grayscale distribution. Its main purpose is to train a generator G, which takes the original image x and passes it through the generator network G to obtain a fake image G(x). Simultaneously, a discriminator D is trained, which receives both the generated fake image G(x) and the real image. Through training, the features of the original image become similar to those of the generated image, making it impossible for the discriminator D to distinguish whether the image is fake.

[0087] Step 2.3: Combine the synthetic dataset with the original dataset to form a hybrid dataset.

[0088] In this embodiment, 120 images with good synthesis results were manually selected from the 200 synthetic images to form a synthetic dataset, which, together with the original dataset of 139 images, constituted the final dataset. We randomly selected 15 images from the 139 original images as the validation set, and combined the remaining 124 original images with the 120 synthetic images to form the training set.

[0089] In this embodiment, the experimental data preprocessing is also included, specifically:

[0090] Randomly crop the image with a size of 1119×1431 to 448×448;

[0091] Data augmentation of images by horizontal flipping, vertical flipping, and rotation;

[0092] Random image cropping is used because the original images are large; this reduces computational cost, adapts to the SwinTransformer, and speeds up training. Effective data augmentation enhances model robustness and prevents overfitting. Finally, the resulting images are fed into the network for training.

[0093] In this embodiment, step 3 specifically includes:

[0094] Step 3.1 Process the mixed dataset through the Swin-Tranformer network to obtain feature vector maps at four scales;

[0095] In this embodiment, step 3.1 specifically includes:

[0096] Step 3.1.1 Input the training images from the mixed dataset into the network. After processing by the block partitioning module, the images are obtained. The images are divided into different small blocks according to pixels. The pixels of the small blocks in all channels are stretched into one-dimensional feature vectors. All one-dimensional feature vectors are combined into a feature vector map.

[0097] Step 3.1.2 processes the feature vector map through four stages to obtain four feature vector maps at different scales. Each stage consists of block merging and a Swin-Transformer Block.

[0098] The Swin-TransformerBlock in step 3.1.2 includes a layer normalization layer, a window multi-head self-attention layer, and a multi-layer perception layer.

[0099] The training process of the Swin-Tranformer semantic segmentation network is represented as follows:

[0100] The Swin-Transformer network consists of four stages: a block partitioning module. Each stage comprises block merging and a Swin-TransformerBlock. The main function of block merging is to reduce the resolution of the feature map, except in the first stage. After the small blocks are combined and linearly encoded, they are input into the Swin-TransformerBlock.

[0101] The block partitioning module transforms the smallest unit of an image from pixels into blocks. Each block consists of 4×4 pixels; that is, the pixel matrix is ​​divided using blocks containing 4×4 pixels, and the pixel values ​​in all channels of each block are stretched into a one-dimensional vector. The specific details of the block partitioning module are as follows:

[0102] The first layer is the input layer, and the input image sample has 3 channels and a size of 448*448.

[0103] The second layer is a convolutional layer with 3 input channels, 96 output channels, a kernel size of 4*4, and a kernel stride of 4*4.

[0104] The third layer is a layer normalization layer with 96 input channels.

[0105] The feature vector map with 96 channels and a size of 112*112 obtained from the block partitioning module is input into Swin-TransformerBlock. The specific details of Swin-TransformerBlock are as follows:

[0106] The first layer is a layer normalization layer with 96 input channels;

[0107] The second layer is a fully connected layer with 96 input channels and 288 output channels;

[0108] The third layer is a window self-attention module with 96 input channels and 96 output channels;

[0109] The fourth layer is a layer normalization layer with 96 input channels;

[0110] The fifth layer is a multi-layer sensing layer with 96 input channels and 96 output channels;

[0111] The 6th layer is a layer normalization layer with 96 input channels;

[0112] The 7th layer is a fully connected layer with 96 input channels and 288 output channels;

[0113] The 8th layer is a moving window self-attention module with 96 input channels and 96 output channels;

[0114] The 9th layer is a layer normalization layer with 96 input channels;

[0115] The 10th layer is a multi-layer sensing layer with 96 input channels and 96 output channels.

[0116] In this process, after layers 1 and 6, the input feature vector map is divided into 7×7 blocks. Each block's feature vector map is flattened into 2080 feature vectors with 96 channels and a length of 49. The window self-attention module divides the feature matrix into three equal parts based on the number of channels, obtaining three features: Q, K, and V. Each feature has 96 channels and a length of 49, resulting in 2080 features. Then, through matrix transposition and copying, three independent weight matrices for each window are obtained. The calculation formula for the weight matrix is ​​as follows:

[0117]

[0118] Among them, Q, K, and V represent three features respectively; B represents the relative positional deviation; and dk represents the variance of feature K, which is a constant.

[0119] The specific details of the multi-layer perception layer are as follows:

[0120] The first layer is a fully connected layer with 96 input channels and 384 output channels;

[0121] The second layer is the GeLU excitation layer;

[0122] The third layer is a fully connected layer with 384 input channels and 96 output channels.

[0123] The moving window self-attention module moves all windows by 3 window sizes, moving the left and top 3 windows in the feature map to the right and bottom, resulting in a new feature matrix. The self-attention of the new feature matrix is ​​calculated in the same way as the window self-attention module, which will not be elaborated here.

[0124] The feature vector with 96 channels and a length of 12544, obtained through the Swin-TransformerBlock, is input and restored to a feature matrix of size 112*112. This matrix is ​​then input into the block merging module to reduce the resolution and adjust the number of channels to achieve a hierarchical design, thereby increasing the receptive field. The specific details of the block merging module are as follows:

[0125] The first layer is an expanded layer with 96 input channels, an input size of 112*112, 384 output channels, an output size of 3136, a kernel size of 2*2, and a kernel stride of 2*2.

[0126] The second layer is a layer normalization layer with 96 input channels;

[0127] The third layer is a fully connected layer with 384 input channels and 192 output channels.

[0128] The subsequent three stages are the same as described above, so they will not be repeated here. Each stage yields feature vector maps of different sizes: the first stage has a size of 112*112 and 96 channels; the second stage has a size of 56*56 and 192 channels; the third stage has a size of 28*28 and 384 channels; and the fourth stage has a size of 14*14 and 768 channels.

[0129] Step 3.2 Input the feature vector maps of the four scales into the UperNet network for processing to obtain the class prediction vector. Based on the class prediction vector, construct the Swin-Tranformer semantic segmentation network.

[0130] In this embodiment, step 3.2 specifically includes:

[0131] Step 3.2.1 The bottom feature vector map is processed by the pyramid pooling module to obtain the first multi-scale composite feature map;

[0132] Step 3.2.2 Perform multi-scale feature fusion processing on the four feature vector maps to obtain a second composite feature map with multiple scales;

[0133] Step 3.2.3 Normalize the two composite feature maps using the softmax function to obtain the class prediction vector.

[0134] The pyramid pooling module aggregates contextual information from different regions to improve the network's ability to acquire global information. The pyramid pooling module consists of four parallel convolutional operations. The first part is detailed below:

[0135] The first layer is an adaptive average pooling layer with an output size of 1*1;

[0136] The second layer is a convolutional layer with 768 input channels, 512 output channels, and a kernel size of 1*1.

[0137] The third layer is a batch normalization layer with 512 input channels;

[0138] The fourth layer is the ReLU excitation layer;

[0139] The 5th layer is an upsampling layer with an input size of 1*1 and an output size of 14*14.

[0140] The details of Part Two are as follows:

[0141] The first layer is an adaptive average pooling layer with an output size of 2*2;

[0142] The second layer is a convolutional layer with 768 input channels, 512 output channels, and a kernel size of 1*1.

[0143] The third layer is a batch normalization layer with 512 input channels;

[0144] The fourth layer is the ReLU excitation layer;

[0145] The 5th layer is an upsampling layer with an input size of 2*2 and an output size of 14*14;

[0146] The details of Part Three are as follows:

[0147] The first layer is an adaptive average pooling layer with an output size of 3*3;

[0148] The second layer is a convolutional layer with 768 input channels, 512 output channels, and a kernel size of 1*1.

[0149] The third layer is a batch normalization layer with 512 input channels;

[0150] The fourth layer is the ReLU excitation layer;

[0151] The 5th layer is an upsampling layer with an input size of 3*3 and an output size of 14*14;

[0152] The details of Part Four are as follows:

[0153] The first layer is an adaptive average pooling layer with an output size of 6*6.

[0154] The second layer is a convolutional layer with 768 input channels, 512 output channels, and a kernel size of 1*1.

[0155] The third layer is a batch normalization layer with 512 input channels;

[0156] The fourth layer is the ReLU excitation layer;

[0157] The 5th layer is an upsampling layer with an input size of 6*6 and an output size of 14*14.

[0158] The inputs to the four parts mentioned above are all low-level feature maps with a size of 14*14 and 768 channels. The four parts are operated in parallel, and the output feature maps are all 14*14 in size and 512 in channel. They are then concatenated with the low-level feature maps to obtain a size of 14*14 and 2816 channels.

[0159] The number of input channels for multi-scale feature fusion varies, therefore it consists of four parallel convolutional operations. Except for the last layer, which is the output of the pyramid pooling module, the other inputs are the outputs of each stage of the Swin-Transformer. The specific details of the first part are as follows:

[0160] The first layer is a convolutional layer with 96 input channels, 512 output channels, and a kernel size of 1*1.

[0161] The second layer is a batch normalization layer with 512 input channels;

[0162] The third layer is the ReLU excitation layer;

[0163] The fourth layer is a convolutional layer with 512 input channels, 512 output channels, and a 3*3 kernel size.

[0164] The fifth layer is a batch normalization layer with 512 input channels;

[0165] Layer 6 is the ReLU excitation layer;

[0166] The 7th layer is an upsampling layer with an input size of 112*112 and an output size of 112*112.

[0167] The details of Part Two are as follows:

[0168] The first layer is a convolutional layer with 192 input channels and 512 output channels, and the kernel size is 1*1.

[0169] The second layer is a batch normalization layer with 512 input channels;

[0170] The third layer is the ReLU excitation layer;

[0171] The fourth layer is a convolutional layer with 512 input channels, 512 output channels, and a 3*3 kernel size.

[0172] The fifth layer is a batch normalization layer with 512 input channels;

[0173] Layer 6 is the ReLU excitation layer;

[0174] The 7th layer is an upsampling layer with an input size of 56*56 and an output size of 112*112.

[0175] The details of Part Three are as follows:

[0176] The first layer is a convolutional layer with 384 input channels, 512 output channels, and a kernel size of 1*1.

[0177] The second layer is a batch normalization layer with 512 input channels;

[0178] The third layer is the ReLU excitation layer;

[0179] The fourth layer is a convolutional layer with 512 input channels, 512 output channels, and a 3*3 kernel size.

[0180] The fifth layer is a batch normalization layer with 512 input channels;

[0181] Layer 6 is the ReLU excitation layer;

[0182] The 7th layer is an upsampling layer with an input size of 28*28 and an output size of 112*112.

[0183] The details of Part Four are as follows:

[0184] The first layer is an upsampling layer with an input size of 14*14 and an output size of 112*112.

[0185] The second layer is a convolutional layer with 512 input channels, 512 output channels, and a kernel size of 3*3.

[0186] The third layer is a batch normalization layer with 512 input channels;

[0187] The fourth layer is the ReLU excitation layer.

[0188] As mentioned above, the feature vector maps output from the four parts all have 512 channels and a size of 112*112. After concatenation, a composite vector map with 2048 channels is obtained. The class prediction vector is then obtained through convolution and the softmax function, as detailed below:

[0189] The first layer is a convolutional layer with 2048 input channels, 512 output channels, and a kernel size of 3*3.

[0190] The second layer is a batch normalization layer with 512 input channels;

[0191] The third layer is the ReLU excitation layer;

[0192] The fourth layer is a convolutional layer with 512 input channels, 4 output channels, and a kernel size of 1*1.

[0193] The 5th layer is an upsampling layer with an input size of 112*112 and an output size of 448*448.

[0194] The category prediction vector obtained above has a size of 448*448 for the input image and 4 channels. After passing through the softmax function, a normalized category prediction probability of length 2 is obtained. The 4 channels correspond to pores, resin matrix, weft yarn and warp yarn respectively. The channel with the highest prediction probability is assigned to the corresponding category, thus obtaining the semantic segmentation result map of the entire input image.

[0195] The above are merely preferred embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A deep learning segmentation method of a carbon fiber composite material dataset, characterized by, The method comprises the following steps: Step 1: manually segmenting randomly selected composite XCT detection images by using a manual labeling method to construct an original data set; Step 2: performing parameterized statistical analysis based on the original data set and parameterized random modeling to expand the data set and construct a mixed data set; Step 3: constructing a Swin-Tranformer semantic segmentation network based on the mixed data set; Step 4: generating a CFRP complete semantic segmentation image as the final result based on the Swin-Tranformer semantic segmentation network; The step 2 specifically comprises: Step 2.1: performing parameterized statistical analysis based on the microstructure features of the original data set; Step 2.2: expanding the data set by parameterized random modeling based on the results of the parameterized statistical analysis to generate a synthetic data set; Step 2.3: combining the synthetic data set with the original data set to form a mixed data set; The step 2.1 specifically comprises: Step 2.1.1: collecting shape parameters; Step 2.1.2: fitting the distribution of the collected shape parameters; Step 2.1.3: performing statistical analysis on the distribution-fitted shape parameters; The statistical analysis includes warp yarn geometric parameter statistical analysis and weft yarn geometric parameter statistical analysis with contours similar to rectangles and ellipses, respectively; The step 2.2 specifically comprises: Step 2.2.1: generating a warp yarn random model according to the warp yarn parameterized statistical analysis; Step 2.2.2: generating a weft yarn random model according to the weft yarn parameterized statistical analysis; Step 2.2.3: generating an artificial synthetic label image based on the warp yarn random model and the weft yarn random model; Step 2.2.4: generating a carbon fiber composite material image with the same real microstructure based on the artificial synthetic label image, and taking the carbon fiber composite material image with the same real microstructure as the synthetic data set; The step 2.2.4 specifically comprises: Step 2.2.4.1: modifying the generation part of the Pix2PixHD network to simultaneously train two generator models at different scales, and fusing the features of the two generator models at different scales; Step 2.2.4.2: generating a carbon fiber composite material image with the same real microstructure based on the feature-fused generator model.

2. The method of claim 1, wherein the method further comprises: The step 1 specifically comprises: Step 1.1: obtaining a composite XCT image through XCT tomography; Step 1.2: cropping to remove the air portion outside the composite XCT image, and taking out one picture every 10 frames; Step 1.3: manually screening and labeling all the taken-out pictures to obtain an original data set.

3. The method of claim 2, wherein the method further comprises: The original data set mainly includes four categories, namely warp yarn, weft yarn, resin base, and pore.

4. The method of claim 1, wherein, The step 3 specifically comprises: Step 3.1: processing the mixed data set through the Swin-Tranformer network to obtain four-scale feature vector graphs; Step 3.2, the feature vector maps of four scales are input into the UPerNet network for processing to obtain a category prediction vector, and based on the category prediction vector, a Swin-Transformer semantic segmentation network is constructed.

5. The method of claim 4, wherein the method further comprises: The step 3.1 specifically comprises: Step 3.1.1, the training images in the mixed dataset are input into the network, processed by a block division module to obtain pictures, the pictures are divided into different small blocks according to pixels, the pixels of the small blocks in all channels are stretched into one-dimensional feature vectors, and all one-dimensional feature vectors are combined into a feature vector map; Step 3.1.2, the feature vector map is processed through four stages to obtain four feature vector maps at different scales, wherein each stage is composed of a block merging and a Swin-Transformer Block.

6. The method of claim 4, wherein the method further comprises: The step 3.2 specifically comprises: Step 3.2.1, the bottom feature vector map is processed through a pyramid pooling module to obtain a first composite feature map of multiple scales; Step 3.2.2, the four feature vector maps are processed through multi-scale feature fusion to obtain a second composite feature map of multiple scales; Step 3.2.3, the two composite feature maps are normalized through a softmax function to obtain a category prediction vector.

Images