A deep learning-based method for identifying pore defects in resin-based composite materials

Abstract

This invention discloses a deep learning-based method for identifying pore defects in resin-based composite materials, relating to the field of intelligent image detection. The method comprises: obtaining cross-sectional optical microscopic images of the resin-based composite laminate to be identified; preprocessing each cross-sectional optical microscopic image; constructing a pretrained convolutional neural network model; and using the preprocessed cross-sectional optical microscopic images as input to the pretrained convolutional neural network model to perform pore segmentation and content calculation. By introducing advanced deep learning techniques and combining them with targeted data processing and training strategies, the invention successfully overcomes many limitations of traditional optical microscopic image pore assessment methods, providing a more accurate, robust, and automated solution for composite material pore content assessment that does not require parameter calibration.

Description

Technical Field

[0001] The present invention belongs to the field of intelligent image detection, and in particular relates to a method for identifying pore defects in resin-based composite materials based on deep learning. Background Art

[0002] Composite laminates are widely used in aerospace, automotive, energy, and other fields due to their excellent properties, such as high specific strength and high specific modulus. However, defects such as pores are inevitably generated during the manufacturing process of composite materials. The presence of pores, even at low levels, can significantly reduce the mechanical properties of the composite material, particularly matrix-dominated properties such as interlaminar shear strength, compressive strength, and fatigue life. In severe cases, it can lead to premature structural failure, affecting its safety and reliability. Therefore, accurately assessing the pore content in composite materials is crucial for monitoring manufacturing quality and ensuring structural integrity.

[0003] Currently, there are multiple methods for assessing the porosity of composite materials, including density methods (such as acid digestion and matrix burnout), X-ray computed tomography, ultrasonic testing, thermal imaging, and optical microscopy. To improve analysis efficiency and avoid time-consuming manual interpretation, automated image segmentation techniques are often used to identify pores in microscopic images. Pixel intensity-based threshold segmentation is currently the most commonly used automated analysis method. This method exploits the fact that pore regions typically exhibit different grayscale values ​​than the composite matrix and fibers in an image. Pores are separated from the background by setting one or more grayscale thresholds.

[0004] However, the traditional threshold-based pore segmentation method has the following significant defects:

[0005] Sensitive to imaging conditions: The accuracy of the threshold method is highly dependent on the acquisition conditions of the microscopic image, such as illumination uniformity and contrast. Slight changes in imaging conditions may cause the preset threshold to fail, resulting in erroneous segmentation results.

[0006] Sensitive to material type: Different types of composite materials (such as different fibers, matrices, colors, etc.) present different grayscale characteristics in microscopic images, making it difficult to directly apply the threshold parameters optimized for one material to another.

[0007] Inadequate recognition of complex pores: When pores are large or irregular in shape, reflections from the microscope's light source may create brighter areas within the pores. The pixel intensity in these areas may be similar to or even higher than that of the matrix. Traditional thresholding methods struggle to accurately identify the complete boundaries of such pores, often misidentifying pore areas as matrix and underestimating the porosity. Furthermore, scratches, contaminants, or fiber ends in the image may sometimes be misidentified as pores.

[0008] Although some studies have attempted to apply machine learning algorithms to pore detection in X-ray tomography images, thermal images, or ultrasonic signals, and have conducted some explorations in pore segmentation in optical microscopy images, the advantages of machine learning-based methods over traditional thresholding methods when processing complex microscopic images with significant pixel intensity scattering have not been fully verified, or the model robustness and calibration-free characteristics need to be further improved.

[0009] Patent CN 111507966 B discloses a composite material pore detection method based on a UNET deep network. The method ultimately obtains the minimum circumscribed rectangle of each pore block through means such as M-adjacency. However, the patent does not consider the pixel intensity characteristics of pores in microscope images, does not construct a data set containing microscope images of multiple types of composite laminates, and does not automatically measure and statistically analyze the pore data of laminates with different manufacturing processes, nor does it use this data to enhance the network's adaptability to different samples. The patent is lacking in the network's adaptability to microscope samples with different backgrounds and the reliability of the inference results, making it difficult to obtain more accurate pore characteristics. Summary of the Invention

[0010] In response to the above-mentioned deficiencies in the prior art, the present invention provides a method for identifying pore defects in resin-based composite materials based on deep learning, which solves the problems of unstable identification results, low degree of automation, and high manual dependence in the prior art.

[0011] In order to achieve the above-mentioned purpose of the invention, the technical solution adopted by the present invention is: a method for identifying pore defects in resin-based composite materials based on deep learning, comprising:

[0012] Acquire a cross-sectional optical microscopic image of the resin-based composite laminate to be identified;

[0013] Preprocess each cross-sectional optical microscopy image;

[0014] Build a pre-trained convolutional neural network model;

[0015] The preprocessed cross-sectional optical microscopy images are used as input to the pre-trained convolutional neural network model for pore segmentation and content calculation.

[0016] Furthermore, the specific method for preprocessing each cross-sectional optical microscopy image includes:

[0017] A true value mask is created for each cross-sectional optical microscopic image to obtain a microscopic image with the true value mask;

[0018] Dataset partitioning of microscopic images with ground-truth masks;

[0019] All microscopic images after data set division were subjected to pixel intensity inversion and image segmentation.

[0020] Furthermore, a true value mask is produced for each cross-sectional optical microscopy image by: generating a preliminary true value mask through automatic threshold segmentation; and producing a pixel-level binary true value mask image based on the preliminary true value mask by manual fine correction to obtain a microscopy image with a true value mask.

[0021] Furthermore, the pixel intensity inversion is specifically: inverting the pores that appear as dark areas in the microscopic image into bright areas, and inverting the pores that appear as bright areas in the microscopic image into dark areas.

[0022] Furthermore, the image segmentation specifically includes segmenting the microscopic image and its corresponding true value mask into a number of image blocks of fixed sizes.

[0023] Furthermore, the convolutional neural network model includes an encoder and a decoder;

[0024] The encoder includes four downsampling modules connected in sequence; the decoder includes four upsampling modules connected in sequence;

[0025] The downsampling module consists of two consecutive 3×3 convolutional layers and a 2×2 max pooling layer. Each 3×3 convolutional layer is followed by a batch normalization layer and a Leaky ReLU activation function layer. The input of the 2×2 max pooling layer is connected to the output of the last Leaky ReLU activation function layer.

[0026] The upsampling module consists of a 2×2 deconvolution layer and two consecutive 3×3 convolution layers. The output of the 2×2 deconvolution layer is connected to the input of the first 3×3 convolution layer. Each 3×3 convolution layer is sequentially connected to a batch normalization layer and a Leaky ReLU activation function layer.

[0027] Furthermore, the specific method for pore segmentation and content calculation is as follows:

[0028] The preprocessed cross-sectional optical microscopy images are used as input to a pre-trained convolutional neural network model to obtain a pore probability map indicating that each pixel belongs to a pore.

[0029] Apply a preset probability threshold to the pore probability map; classify pixels with values ​​greater than the preset threshold as pores, otherwise classify them as non-pores, and generate a binary pore segmentation image;

[0030] According to the binary pore segmentation image, the ratio of the total number of pixels in the pore area to the total number of pixels in the image is counted to obtain the porosity of the material, which is calculated as follows:

[0031] .

[0032] Furthermore, in the process of training the convolutional neural network model, the binary cross entropy loss function is used as the optimization target, and the Adam optimization algorithm is adopted to iteratively update the model weights. The initial learning rate is set to 0.001, the batch size is set to 40 image blocks, and the total number of training rounds is set to 400 rounds.

[0033] Secondly, the present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of a method for identifying pore defects in resin-based composite materials based on deep learning.

[0034] Finally, the present invention also provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of a method for identifying pore defects in resin-based composite materials based on deep learning.

[0035] The beneficial effects of the present invention are:

[0036] 1. Porosity assessment using a pre-trained convolutional neural network model eliminates the need for time-consuming and experience-dependent manual parameter calibration (e.g., thresholds) for different images, material types, or imaging conditions, as is required with traditional thresholding methods. During training, the convolutional neural network model learns the mapping from image features to pore identification, demonstrating excellent generalization capabilities. The model's adaptability is further enhanced by constructing a training dataset containing a variety of composite materials and performing standardized preprocessing, such as pixel intensity inversion. This overcomes the significant drawbacks of existing technologies, which are highly sensitive to imaging conditions and material types and require frequent parameter adjustments, achieving truly automated and standardized porosity assessment, improving analysis efficiency, objective consistency of results, and demonstrating enhanced robustness.

[0037] 2. By employing an improved U-Net-based convolutional neural network (CNN) architecture and combining it with a microscopic image dataset with accurate ground-truth masks for end-to-end training, the model is able to learn the complex visual features and contextual information of pores. Unlike existing techniques that rely primarily on simple pixel intensity thresholds for segmentation, the convolutional neural network model of the present invention can recognize and distinguish complex pixel patterns caused by uneven lighting, differences in material optical properties, internal pore reflections, or blurred edges, significantly improving the accuracy of pore segmentation and the ability to identify complex pore morphologies. For example, for high-brightness areas or irregular edges within large pores that are difficult to handle using traditional thresholding methods, the present invention enables more precise boundary delineation and area calculation, resulting in more reliable pore content data. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 A schematic flow chart of a method for identifying pore defects in resin-based composite materials based on deep learning provided in an embodiment;

[0039] Figure 2 A schematic diagram of the convolutional neural network model structure provided in the embodiment;

[0040] Figure 3 This is a comparison chart of the average porosity error between the traditional algorithm and the method proposed in this invention. DETAILED DESCRIPTION

[0041] The specific embodiments of the present invention are described below to facilitate understanding of the present invention by those skilled in the art. However, it should be clear that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the appended claims, these changes are obvious, and all inventions and creations utilizing the concepts of the present invention are protected.

[0042] like Figure 1 As shown, in one embodiment of the present invention, a method for identifying pore defects in resin-based composite materials based on deep learning includes the following steps:

[0043] S1. Obtaining a cross-sectional optical microscopic image of the resin-based composite laminate to be identified;

[0044] The resin-based composite laminates collected in this example include three different types:

[0045] Type A: Fiberglass / epoxy laminate, manufactured by vacuum infusion process;

[0046] Type B: carbon fiber / epoxy resin laminate, manufactured by resin transfer molding (RTM) process;

[0047] Type C: Carbon fiber / epoxy resin laminate, manufactured by vacuum infusion process.

[0048] Specific sample polishing and image acquisition conditions are shown in Table 1. This example constructs a dataset containing microscope images of three different types of composite laminates, with corresponding ground truth masks generated for each image. By automatically measuring data such as pore size and frequency and performing statistical analysis, significant differences in pore size and frequency were found in laminates manufactured using different manufacturing processes. Using this data, a convolutional neural network model was trained and validated, enhancing the network's adaptability to microscope samples with varying backgrounds and improving the reliability of the network's inference results.

[0049] Table 1

[0050] Laminate Type Sandpaper grit (final polish) optical microscope A 2000 Olympus PMG3 (with CCD camera) B 1000 Olympus PMG3 (with CCD camera) C 1000 Olympus PMG3 (with CCD camera)

[0051] S2. Preprocess each cross-sectional optical microscopy image;

[0052] Specific methods include:

[0053] A ground-truth mask was created for each cross-sectional optical microscopy image. A preliminary ground-truth mask was generated through automated threshold segmentation (using in-house threshold-based software). Based on this preliminary ground-truth mask, a pixel-level binary ground-truth mask image was created using the open-source image processing software GIMP and fine-tuned manually. This resulted in a microscopic image with a ground-truth mask. In these mask images, pixels representing pores have a value of 255, while pixels representing matrix or fibers have a value of 0.

[0054] Dataset partitioning of microscopic images with ground-truth masks;

[0055] In order to adapt to the characteristics of the maximum pooling layer in the subsequent convolutional neural network (CNN), the pixel intensity of all microscopic images after the dataset is divided is inverted (the pores that appear as dark areas in the microscopic image are inverted to bright areas, and the pores that appear as bright areas in the microscopic image are inverted to dark areas). Through this operation, the pore areas that are usually darker in the image are inverted to brighter areas, while the matrix / fiber areas that are usually brighter are inverted to darker areas, which helps the network better capture the pore characteristics when downsampling.

[0056] To efficiently utilize GPU memory and increase the diversity of image sources in training batches, each high-resolution training image (and its corresponding ground-truth mask) is split into 20 smaller 256x256 pixel grayscale image patches. This allows the number of network filters to be increased without overloading GPU memory; it also allows each training batch to include images of all types of laminates. Gradient optimization-based training can better estimate gradients, resulting in more efficient training.

[0057] S3. Build a pre-trained convolutional neural network model. During the training of the convolutional neural network model, use the binary cross entropy loss function as the optimization target, adopt the Adam optimization algorithm to iteratively update the model weights, set the initial learning rate to 0.001, the batch size to 40 image blocks, and the total number of training rounds to 400 rounds.

[0058] On the training and validation datasets, we evaluated model performance using pixel-level segmentation metrics such as accuracy, precision, recall, and intersection over union (IoU). Specific performance data is shown in Table 2.

[0059] Table 2

[0060] index training set Validation set Accuracy 0.9870 0.9946 Accuracy 0.9471 0.9269 Recall 0.9927 0.9124 Intersection and Union 0.9640 0.9231

[0061] like Figure 2 As shown, the convolutional neural network model includes an encoder and a decoder; wherein the encoder includes four downsampling modules connected in sequence; the decoder includes four upsampling modules connected in sequence;

[0062] The downsampling module consists of two consecutive 3×3 convolutional layers and a 2×2 max pooling layer (to halve the feature map size and double the number of channels). Each 3×3 convolutional layer is followed by a batch normalization layer (to improve training stability and speed) and a Leaky ReLU activation function layer. The input of the 2×2 max pooling layer is connected to the output of the last Leaky ReLU activation function layer.

[0063] The upsampling module consists of a 2×2 deconvolution layer and two consecutive 3×3 convolution layers. The output of the 2×2 deconvolution layer is connected to the input of the first 3×3 convolution layer. Each 3×3 convolution layer is sequentially connected to a batch normalization layer and a Leaky ReLU activation function layer.

[0064] The output of the decoder is connected to a 1×1 convolutional layer to map the feature vector to the required number of categories (in this example, 1, indicating the probability of pores), and then connected to a sigmoid activation function to output the probability value of each pixel belonging to the pore category (ranging between 0 and 1).

[0065] S4. Use the pre-processed cross-sectional optical microscopy image as the input of the pre-trained convolutional neural network model to perform pore segmentation and content calculation:

[0066] The preprocessed cross-sectional optical microscopy images are used as input to a pre-trained convolutional neural network model to obtain a pore probability map indicating that each pixel belongs to a pore.

[0067] Apply a preset probability threshold to the pore probability map. In this embodiment, the probability threshold is set to 0.35. Pixels with values ​​greater than the preset threshold are classified as pores, and pixels otherwise are classified as non-pores, generating a binary pore segmentation image.

[0068] According to the binary pore segmentation image, the ratio of the total number of pixels in the pore area to the total number of pixels in the image is counted to obtain the porosity of the material, which is calculated as follows:

[0069] .

[0070] As shown in Table 3, the average absolute errors of the porosity predictions for the three laminates made of different materials in this embodiment are all less than 1%.

[0071] Table 3

[0072] Average porosity error Error standard deviation A 0.62% 0.26% B 0.32% 0.24% C 0.47% 0.18%

[0073] like Figure 3 As shown in Figure 2, the pore content estimation error generated by the proposed method is significantly lower than that of a manually optimized traditional threshold segmentation algorithm. While the model still has limitations when processing extremely small pores or pores with blurred edges, its overall segmentation performance and pore content assessment accuracy are excellent.

[0074] This example demonstrates in detail how to construct and train a convolutional neural network based on an improved U-Net architecture to achieve automatic and accurate segmentation and content assessment of pores in optical microscopic images of composite laminates, without the need for parameter calibration for different images, verifying the feasibility and superiority of the technical solution of the present invention.

[0075] In summary, the present invention utilizes an improved U-Net-based convolutional neural network (CNN) architecture and combines it with a microscopic image dataset with accurate ground-truth masks for end-to-end training, enabling the model to learn the complex visual features and contextual information of pores. Unlike existing techniques that primarily rely on simple pixel intensity thresholding for segmentation, the present CNN can recognize and distinguish complex pixel patterns caused by uneven illumination, differences in material optical properties, internal pore reflections, or blurred edges, significantly improving the accuracy of pore segmentation and the ability to identify complex pore morphologies. For example, for large pores with internal highlights or irregular edges that are difficult to handle using traditional thresholding methods, the present invention enables more precise boundary delineation and area calculation, resulting in more reliable pore content data. This method successfully overcomes many limitations of traditional optical microscopy pore assessment methods, providing a more accurate, robust, automated, and calibration-free solution for composite material pore content assessment, with significant practical application value and technological advancement significance.

Claims

1. A method for identifying pore defects in resin-based composite materials based on deep learning, characterized in that: include: Acquire a cross-sectional optical microscopic image of the resin-based composite laminate to be identified; Preprocess each cross-sectional optical microscopy image; Specific preprocessing methods include: A preliminary ground-truth mask is generated through automatic threshold segmentation; a pixel-level binary ground-truth mask image is produced by manual fine correction based on the preliminary ground-truth mask, and a microscopic image with a ground-truth mask is obtained; Dataset partitioning of microscopic images with ground-truth masks; Perform pixel intensity inversion and image blocking on all microscopic images after data set division; Build a pre-trained convolutional neural network model; The convolutional neural network model includes an encoder and a decoder; The encoder includes four downsampling modules connected in sequence; the decoder includes four upsampling modules connected in sequence; The downsampling module consists of two consecutive 3×3 convolutional layers and a 2×2 max pooling layer. Each 3×3 convolutional layer is followed by a batch normalization layer and a Leaky ReLU activation function layer. The input of the 2×2 max pooling layer is connected to the output of the last Leaky ReLU activation function layer. The upsampling module consists of a 2×2 deconvolution layer and two consecutive 3×3 convolution layers. The output of the 2×2 deconvolution layer is connected to the input of the first 3×3 convolution layer. Each 3×3 convolution layer is followed by a batch normalization layer and a Leaky ReLU activation function layer. During the training of the convolutional neural network model, the binary cross entropy loss function was used as the optimization objective, and the Adam optimization algorithm was used to iteratively update the model weights. The initial learning rate was set to 0.001, the batch size was set to 40 image blocks, and the total number of training rounds was set to 400 rounds. The preprocessed cross-sectional optical microscopy images were used as input to a pre-trained convolutional neural network model for pore segmentation and content calculation. The specific method for pore segmentation and content calculation is: The preprocessed cross-sectional optical microscopy images are used as input to a pre-trained convolutional neural network model to obtain a pore probability map indicating that each pixel belongs to a pore. Apply a preset probability threshold to the pore probability map; classify pixels with values ​​greater than the preset threshold as pores, otherwise classify them as non-pores, and generate a binary pore segmentation image; According to the binary pore segmentation image, the ratio of the total number of pixels in the pore area to the total number of pixels in the image is counted to obtain the porosity of the material, which is calculated as follows: 。 2. The method for identifying pore defects in resin-based composite materials based on deep learning according to claim 1, characterized in that: Specifically, the pixel intensity inversion is as follows: the pores that appear as dark areas in the microscopic image are inverted into bright areas, and the pores that appear as bright areas in the microscopic image are inverted into dark areas.

3. The method for identifying pore defects in resin-based composite materials based on deep learning according to claim 1, characterized in that: Image segmentation is specifically to segment the microscopic image and its corresponding true value mask into several image blocks of fixed size.

4. A computer-readable storage medium storing a computer program, characterized in that: When the program is executed by a processor, the method described in any one of claims 1 to 3 is implemented.

5. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instruction is executed by a processor, the method according to any one of claims 1 to 3 is implemented.

Images