A defect detection method of a 3D printing dot array structure

Abstract

The present application relates to a kind of 3D printing dot structure defect detection method, belong to 3D printing defect detection technical field, solve the problem that prior art cannot describe the specific topographic details features of defect in dot structure body.The method of the present application comprises: obtaining the two-dimensional tomographic image data of 3D printing dot structure sample piece to construct training sample set;Improved Mask RCNN model that fusion attention mechanism and BiFPN network are constructed;The improved Mask RCNN model is trained by the training sample set;The two-dimensional tomographic image data of 3D printing dot structure to be detected is obtained to construct the data set to be detected;The data set to be detected is input into the improved Mask RCNN model trained, and obtains defect detection result.In the present application, Mask RCNN algorithm is applied to the defect detection of 3D printing dot structure, and through improving Mask RCNN model, the detection, classification and segmentation of defect can be realized simultaneously, and more accurate and comprehensive defect analysis is carried out.

Description

Technical Field

[0001] This invention relates to the field of 3D printing defect detection technology, and in particular to a defect detection method for 3D printed lattice structures. Background Technology

[0002] 3D-printed multifunctional lattice structures are formed by the expansion of nodes and connecting rods between nodes according to a certain spatial periodic law. They possess excellent mechanical properties such as ultra-lightweight, high porosity, high specific strength, high specific stiffness, high strength and toughness, and high energy absorption, as well as sound absorption, vibration reduction, heat dissipation, electromagnetic shielding, and excellent permeability. Combining structural and functional benefits, they are a high-performance multifunctional engineering structural material that can be used to make filters, energy absorption devices, silencers, heat exchange flame retardants, shock absorbers, and biomimetic materials. They have broad development prospects and significant demand in fields such as aerospace vehicle structures, modern ship structures, high-speed rail structures, and modern protective structures.

[0003] However, inherent defects inevitably arise during the fabrication process of 3D-printed lattice structures, severely impacting their lifespan and reliability. Therefore, understanding the shape characteristics and distribution of these defects is crucial for guiding the manufacturing and health assessment of lattice structures. 3D-printed lattice structures primarily rely on industrial CT technology and 3D reconstruction to acquire tomographic images for internal defect detection. Defects in complex metal 3D-printed lattice structures exhibit randomness and uncertainty, manifesting as indeterminate locations and gradual feature variations in CT slices. The randomness of these defects, meaning their location is not fixed, significantly increases the workload. Manual inspection or low-automation inspection of numerous lattice structure CT images results in low efficiency and high false negative and false positive rates. With the rapid development of deep learning, existing technologies utilize deep learning models such as YOLOv3 and Faster R-CNN for defect detection in lattice structure CT images. However, when using existing deep learning models for defect detection, only rectangular bounding boxes are used to pinpoint the defect's location, failing to accurately represent its shape, i.e., the specific morphological details of the defect within the lattice structure remain unknown. Summary of the Invention

[0004] Based on the above analysis, the present invention aims to provide a defect detection method for 3D printed lattice structures to solve the problem that existing defect detection methods cannot describe the specific morphological details of defects within the lattice structure.

[0005] This invention provides a defect detection method for 3D printed lattice structures, the method comprising:

[0006] Two-dimensional tomographic image data of 3D printed lattice structure sample parts were acquired and preprocessed to construct a training sample set;

[0007] An improved Mask RCNN model that integrates an attention mechanism and a BiFPN network is constructed.

[0008] The improved Mask R-CNN model is trained using the training sample set to obtain the trained improved Mask R-CNN model.

[0009] Two-dimensional tomographic image data of the 3D printed lattice structure to be detected are acquired and preprocessed to construct the detection dataset;

[0010] The dataset to be detected is input into the trained improved Mask RCNN model to obtain the defect detection results of the 3D printed dot matrix structure to be detected.

[0011] Based on a further improvement of the above method, the improved Mask R-CNN model includes:

[0012] A residual network is used to extract data features and output feature maps. The residual network includes attention residual blocks, which include convolutional layers and effective channel attention modules.

[0013] BiFPN network is used to perform feature fusion on the feature map output by the residual network and output the feature fused feature map.

[0014] A region proposal network is used to generate candidate bounding boxes on the feature map of the feature fusion.

[0015] A region of interest alignment layer is used to align candidate target boxes on the feature map fused with the features and generate a standard feature map of a preset size.

[0016] The object detection network is used to classify and regress candidate object boxes on the standard feature map to obtain predicted object boxes, and output the category and location information of the predicted object boxes.

[0017] The Mask subnetwork is used to perform semantic segmentation on the standard feature map to generate a high-precision predicted segmentation mask.

[0018] Based on a further improvement of the above method, the residual network includes a convolutional pooling layer, a first residual block, a second residual block, a third residual block, and a fourth residual block connected in sequence.

[0019] The convolutional pooling layer includes a convolutional layer and a pooling layer.

[0020] The first residual block, the second residual block, and the third residual block all include attention residual units, each of which comprises two convolutional layers and an effective channel attention module.

[0021] The fourth residual block includes a conventional residual unit, which includes two convolutional layers.

[0022] Based on a further improvement of the above method, the effective channel attention module includes:

[0023] The global average pooling layer is used to perform global average pooling on the input feature map of size H×W×C to obtain a feature vector of size 1×1×C; where H, W, and C represent the height, width, and number of feature channels, respectively.

[0024] One-dimensional convolutional layers are used to perform one-dimensional convolution calculations on the feature vectors to capture cross-channel interaction information; and

[0025] The simoid activation function is used to calculate the weights for each feature channel.

[0026] Based on the further improvement of the above method, the formula for calculating the weight of each feature channel is:

[0027] w = σ(f k (g(x)));

[0028] σ=1 / (1+e -x );;

[0029] In the formula, f k This represents a convolution operation, where w is the weight, g(x) represents the input feature map, σ represents the sigmoid activation function, and x is the input value.

[0030] Based on a further improvement to the above method, the formula for calculating the kernel size of the one-dimensional convolutional layer is:

[0031]

[0032] In the formula, k is the kernel size, |t| odd y is the odd number closest to t, C is the number of feature channels, and γ and b are fixed values.

[0033] Based on further improvements to the above method, the BiFPN network includes: a first upsampling channel module, a second upsampling channel module, a third upsampling channel module, a first downsampling channel module, a second downsampling module, a third downsampling channel module, a first feature fusion module, a second feature fusion module, a third feature fusion module, a fourth feature fusion module, a fifth feature fusion module, and a sixth feature fusion module.

[0034] Specifically, the first residual module outputs feature map C2, the second residual module outputs feature map C3, the third residual module outputs feature map C4, and the fourth residual module outputs feature map C5.

[0035] The first upsampling channel module upsamples the convolutional feature map C5 to obtain an upsampled image of C5. The first feature fusion module then fuses the upsampled image of C5 with the convolutional feature map of C4 to obtain feature map F4. The second upsampling channel module upsamples feature map F4 to obtain an upsampled image of F4. The second feature fusion module then fuses the upsampled image of F4 with the convolutional feature map of C3 to obtain feature map F3. The third upsampling channel module upsamples feature map F3 to obtain an upsampled image of F3. The third feature fusion module then fuses the upsampled image of F3 with the convolutional feature map of C2, and then performs convolution to obtain feature map P2. The first downsampling channel module upsamples the feature map... Figure P2 is downsampled to obtain a downsampled image of P2. The fourth feature fusion module fuses the downsampled image, feature map C3, and feature map F3, and then performs convolution to obtain feature map P3. The second downsampling channel module downsamples feature map P3 to obtain a downsampled image of P3. The fifth feature fusion module fuses the downsampled image of P3, feature map C4, and feature map F4, and then performs convolution to obtain feature map P4. The third downsampling channel module downsamples feature map P4 to obtain a downsampled image of P4. The sixth feature fusion module fuses the downsampled image of P4 and the feature map obtained by convolution with C5, and then performs convolution to obtain feature map P5. Feature map P5 is output as the feature map after feature fusion.

[0036] Based on a further improvement of the above method, the Mask sub-network includes: a fully convolutional network and a Sobel filter.

[0037] Specifically, a high-precision predicted segmentation mask is generated through the fully convolutional network, and the predicted segmentation mask and the real segmentation mask are input into the Sobel filter and convolved with its Sobel operator to obtain the segmentation edge detail information of the target.

[0038] The Sobel operator includes the two-dimensional operator S. x and S y The calculation formula is as follows:

[0039]

[0040]

[0041] Based on a further improvement of the above method, the target detection network includes:

[0042] Fully connected layers are used for classifying candidate bounding boxes and performing boundary regression to obtain predicted bounding boxes; and

[0043] The softmax activation function is used to output the probability of each category and obtain the category information of the predicted target box.

[0044] Based on a further improvement of the above method, the step of acquiring the two-dimensional tomographic image data of the 3D printed lattice structure to be detected and preprocessing it includes:

[0045] Collect CT scan data of the 3D printed lattice structure to be inspected and reconstruct it to obtain a three-dimensional view;

[0046] The three-dimensional view is cut along the xy, xz and xz directions respectively to obtain two-dimensional tomographic images in the three directions respectively;

[0047] By comparing the clarity of the internal structural information of the two-dimensional tomographic images in three directions, the two-dimensional tomographic image data with the highest clarity is selected as the two-dimensional tomographic image data to be used in practice.

[0048] The two-dimensional tomographic image data of the selected direction with the highest clarity is augmented, and the augmented two-dimensional tomographic image data is normalized.

[0049] Compared with the prior art, the present invention can achieve at least one of the following beneficial effects:

[0050] 1. In this invention, the Mask R-CNN algorithm is applied to defect detection of 3D printed dot matrix structures. By improving the Mask R-CNN model, it is possible to simultaneously detect, classify, and segment defects. Thus, the trained and improved Mask R-CNN model can perform more accurate and comprehensive defect analysis on 3D printed dot matrix structures.

[0051] 2. In this invention, an Efficient Channel Attention (ECA) mechanism is added to the residual network of the Mask RCNN model, which significantly improves the feature extraction capability of the residual network. Under the condition of high gray value in dot matrix CT images, it can better capture subtle differences in image features, thereby obtaining more accurate and reliable defect detection results.

[0052] 3. In this invention, the BiFPN network (Bidirectional Feature Pyramid Network) is used as the backbone of the Mask RCNN model. The BiFPN network combines inverse and lateral connections, realizing effective feature fusion at different scales, enriching the semantic information in the feature map, thereby improving the detection accuracy, and is particularly suitable for detecting weak defects.

[0053] 4. In this invention, a Sobel filter is added to the Mask sub-network (mask branch) of the Mask RCNN model to generate a high-precision prediction segmentation mask for the target, thereby increasing the detailed feature information of the defect image segmentation edge of the 3D printed dot matrix structure.

[0054] In this invention, the above-described technical solutions can be combined with each other to achieve more preferred combinations. Other features and advantages of this invention will be set forth in the following description, and some advantages may become apparent from the description or be learned by practicing the invention. The objects and other advantages of this invention can be realized and obtained from what is particularly pointed out in the description and drawings. Attached Figure Description

[0055] The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Throughout the drawings, the same reference numerals denote the same parts.

[0056] Figure 1 This is a flowchart of a defect detection method for 3D printed lattice structures according to an embodiment of the present invention;

[0057] Figure 2 This is a schematic diagram of the structure of the improved Mask RCNN model according to an embodiment of the present invention;

[0058] Figure 3 This is a schematic diagram illustrating the principle of the effective channel attention module in an embodiment of the present invention.

[0059] Figure 4 This is a schematic diagram of the Resnet50-CEA network according to an embodiment of the present invention;

[0060] Figure 5 This is a schematic diagram of the BiFPN network according to an embodiment of the present invention. Detailed Implementation

[0061] Preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings, which form part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not intended to limit the scope of the present invention.

[0062] A specific embodiment of the present invention discloses a training method for a defect detection model of a 3D printed lattice structure, such as... Figure 1 As shown. The method includes:

[0063] Step 1: Obtain two-dimensional tomographic image data of 3D printed lattice structure sample parts and preprocess them to construct a training sample set;

[0064] Step 2: Construct an improved Mask R-CNN model that integrates the attention mechanism and the BiFPN network;

[0065] Step 3: Train the improved Mask R-CNN model using the training sample set to obtain the trained improved Mask R-CNN model;

[0066] Step 4: Obtain two-dimensional tomographic image data of the 3D printed lattice structure to be detected and preprocess it to construct the dataset to be detected;

[0067] Step 5: Input the dataset to be detected into the trained improved Mask RCNN model to obtain the defect detection results of the 3D printed dot matrix structure to be detected.

[0068] Compared with the prior art, in this embodiment of the invention, the Mask RCNN algorithm is applied to the defect detection of 3D printed dot matrix structures. By improving the Mask RCNN model, the detection, classification and segmentation of defects can be achieved simultaneously, thereby enabling a more accurate and comprehensive defect analysis of 3D printed dot matrix structures.

[0069] Specifically, step 1 involves acquiring two-dimensional tomographic image data of 3D printed lattice structure sample parts and preprocessing them to construct a training sample set, including the following steps:

[0070] Step 11: Collect CT scan data of the 3D printed lattice structure sample and reconstruct it to obtain a three-dimensional view.

[0071] Step 12: Cut the three-dimensional view along a preset cutting direction to obtain two-dimensional tomographic image data. The preset cutting direction includes the xy direction, the xz direction, or the xz direction. For example, the xy plane is selected as the actual cutting direction, the spacing between the faults is 0.1 mm, the pixel size is 1474*1513, and a total of 119 two-dimensional tomographic images are extracted.

[0072] Step 13: Augmentation processing is performed on the two-dimensional tomographic image data. Specifically, the dataset is expanded by adjusting the brightness, contrast, hue, and saturation of the image, and by flipping the image horizontally or vertically.

[0073] Step 14: Normalize the augmented 2D tomographic image data. Specifically, normalize the image to 512×512 pixels, padding any insufficient pixels with zeros.

[0074] Step 15: Label the defects on the normalized two-dimensional tomographic image data.

[0075] Preferably, in step 15, a graphical annotation tool, such as Labelme, is used to annotate the defects on the normalized two-dimensional tomographic image data with polygonal boxes, thereby annotating the actual shape of the defects, obtaining the true target box, and generating and segmenting masks based on the true target box.

[0076] It should be noted that using polygonal box annotation means annotating according to the edge shape of the defect.

[0077] In this invention, when annotating defects in two-dimensional tomographic image data, polygonal boxes are used for annotation, which can annotate the actual shape of the defects and obtain the true segmentation mask information, thereby ensuring the accuracy of model training.

[0078] In step 2, the improved Mask R-CNN model, which integrates the attention mechanism and the BiFPN network, includes:

[0079] A residual network is used to extract data features and output feature maps. The residual network includes attention residual blocks, which include convolutional layers and effective channel attention modules.

[0080] BiFPN network is used to perform feature fusion on the feature map output by the residual network and output the feature fused feature map.

[0081] A region proposal network is used to generate candidate bounding boxes on the feature map of the feature fusion.

[0082] A region of interest alignment layer is used to align candidate target boxes on the feature map fused with the features and generate a standard feature map of a preset size.

[0083] The object detection network is used to classify and regress candidate object boxes on the standard feature map to obtain predicted object boxes, and output the category and location information of the predicted object boxes.

[0084] The Mask subnetwork is used to perform semantic segmentation on the standard feature map to generate a high-precision predicted segmentation mask.

[0085] In this invention, an Efficient Channel Attention (ECA) mechanism is added to the residual network of the Mask RCNN model, which significantly improves the feature extraction capability of the residual network. Under the condition of high gray value in dot matrix CT images, it can better capture subtle differences in image features, thereby obtaining more accurate and reliable defect detection results.

[0086] Specifically, to balance accuracy and efficiency, the residual network in this embodiment of the invention employs a ResNet50 that incorporates an Effective Channel Attention (ECA) module. ResNet50 learns image data through residual blocks, and the residual network effectively alleviates the gradient vanishing and training degradation problems of deep networks without increasing model parameters, thereby improving network convergence performance. Simultaneously, considering the high similarity of grayscale values ​​in dot matrix CT images, segmentation regions are easily affected, leading to false positives and false negatives. Therefore, to compensate for these shortcomings, this embodiment of the invention introduces an attention mechanism into the residual network. Attention-based methods allow the model to focus on and enhance effective feature information while ignoring some useless feature information, thereby improving the model's robustness. Given the corrective effect of the ECA module on channel features, a ResNet50 with ECA is used to enhance the ability to extract vital signs while increasing model complexity by almost no factor.

[0087] Specifically, the Effective Channel Attention (ECA) module includes a Global Average Pooling (GAP) layer, a one-dimensional convolutional layer with a kernel size of k, and a simoid activation function. The GAP layer performs global average pooling on the input feature map of size H×W×C to obtain a feature vector of size 1×1×C; the one-dimensional convolutional layer performs one-dimensional convolution on the feature vector to capture cross-channel interaction information; and the simoid activation function calculates the weight w for each feature channel.

[0088] ECA is a lightweight channel attention mechanism that effectively captures local cross-channel interaction information. It is an enhanced and improved version of the SE module, and its principle is as follows: Figure 3As shown, the main idea is to propose a strategy for local cross-channel interactions without dimensionality reduction. This strategy captures local cross-channel interaction information by considering each channel and its k nearest neighbors after global average pooling (GAP). The ECA module first uses GPA to compute an input feature map of size H×W×C (H, W, and C represent the height, width, and number of feature channels, respectively) to obtain a feature vector of size 1×1×C with a global receptive field. Secondly, it captures cross-channel interaction information through a one-dimensional convolution with a kernel size of k. The kernel size k is related to the number of input channels and is adaptively chosen to determine the coverage of local cross-channel interactions. Then, it computes the weight w for each feature channel using a sigmoid activation function and outputs a weight vector. Finally, it multiplies the output weight vector with the original input feature map to complete the original feature labeling in the channel dimension, making the extracted features more directional, suppressing ineffective features, and thus enhancing the extraction of effective features. The ECA module avoids dimensionality reduction, enabling the model to learn more effective channel attention. Furthermore, the module has very few parameters, determined only by its kernel size k (other parameters are almost negligible).

[0089] The effective channel attention module calculates the weights of each feature channel using the sigmoid activation function using the following formula:

[0090] w = σf k (g(x)));

[0091] In the formula, f k This represents a convolution operation, where w is the weight, g(x) represents the input feature map, and σ represents the sigmoid activation function.

[0092] Specifically, the sigmoid activation function is: σ = 1 / (1 + e^(-1 / 2)) -x ), where x is the input value.

[0093] The formula for calculating the kernel size of the effective channel attention module is:

[0094]

[0095] In the formula, k is the kernel size, |t| odd y is the odd number closest to t, C is the number of feature channels, and γ and b are fixed values.

[0096] In one embodiment, the residual network includes a convolutional pooling layer, a first residual block, a second residual block, a third residual block, and a fourth residual block connected in sequence. The convolutional pooling layer includes convolutional layers and pooling layers. The first, second, and third residual blocks each include an attention residual unit, which includes two convolutional layers (weight layers) and an effective channel attention module. The fourth residual block includes a regular residual unit, which includes two convolutional layers (weight layers).

[0097] Specifically, the first residual block includes 3 attention residual units, the second residual block includes 4 attention residual units, the third residual block includes 6 attention residual units, each attention residual unit includes two convolutional layers and one effective channel attention module, and there are skip connections between the two convolutional layers; the fourth residual block includes 3 regular residual units, each regular residual unit includes two convolutional layers (weight layers), and there are skip connections between the two convolutional layers.

[0098] Compared with the existing ResNet50 residual network, in this embodiment of the invention, an effective channel attention module (ECA module) is added to the first residual block, the second residual block, and the third residual block, but not to the convolutional pooling layer and the fourth residual block. This increases the expressive power and classification accuracy of the residual network while avoiding the problem of overfitting and reducing computational complexity and memory consumption.

[0099] like Figure 4As shown, after the image is input into the residual network, it undergoes processing in five different stages: convolutional pooling layers, the first residual block, the second residual block, the third residual block, and the fourth residual block, outputting feature maps C1, C2, C3, C4, and C5, respectively. C1 is the feature map obtained by the first convolution and pooling operation on the input image through the convolutional pooling layers; it is the smallest in scale. C2, C3, C4, and C5 are the feature maps output at different stages in the residual network, respectively. Specifically, C2 is the output of the first residual block, C3 is the output of the second residual block, C4 is the output of the third residual block, and C5 is the output of the fourth residual block, i.e., the last residual block. After the image is input into the residual network, it first undergoes the first convolution and pooling operation to process the original features of the input image. Then, it undergoes three convolution operations to extract lower-level and intermediate-level features, and finally, it undergoes one more convolution operation to extract higher-level semantic information. Low-level and intermediate-level features have a strong ability to perceive local details and contextual information of the target. Therefore, adding an attention mechanism to the residual blocks (first residual block, second residual block, and third residual block) used to extract features at this level can enhance the importance and expressive power of these features, thereby improving the performance of the residual network. Furthermore, an attention mechanism is applied to the reference block used to extract intermediate-level features to further enhance the importance and expressive power of these features. However, no attention mechanism is added when processing the original features of the input image and extracting higher-level semantic information, which helps reduce computational complexity and memory consumption. Therefore, in this embodiment of the invention, adding an ECA module only to the first three residual blocks in the residual network can reduce computational complexity and memory consumption while ensuring good network model performance.

[0100] In this invention, the BiFPN network (Bidirectional Feature Pyramid Network) is used as the backbone of the Mask RCNN model. The BiFPN network combines inverse and lateral connections, realizing effective feature fusion at different scales, enriching the semantic information in the feature map, thereby improving the detection accuracy, and is particularly suitable for detecting weak defects.

[0101] In one embodiment, the BiFPN network includes: a first upsampling channel module, a second upsampling channel module, a third upsampling channel module, a first downsampling channel module, a second downsampling module, a third downsampling channel module, a first feature fusion module, a second feature fusion module, a third feature fusion module, a fourth feature fusion module, a fifth feature fusion module, and a sixth feature fusion module.

[0102] Among them, the first residual module outputs feature map C2, the second residual module outputs feature map C3, the third residual module outputs feature map C4, and the fourth residual module outputs feature map C5.

[0103] The first upsampling channel module upsamples the convolutional feature map C5 to obtain an upsampled image of C5. The first feature fusion module then fuses the upsampled image of C5 with the convolutional feature map of C4 to obtain feature map F4. The second upsampling channel module upsamples feature map F4 to obtain an upsampled image of F4. The second feature fusion module then fuses the upsampled image of F4 with the convolutional feature map of C3 to obtain feature map F3. The third upsampling channel module upsamples feature map F3 to obtain an upsampled image of F3. The third feature fusion module then fuses the upsampled image of F3 with the convolutional feature map of C2, and then performs convolution to obtain feature map P2. The first downsampling channel module upsamples the feature map... Figure P2 is downsampled to obtain a downsampled image of P2. The fourth feature fusion module fuses the downsampled image, feature map C3, and feature map F3, and then performs convolution to obtain feature map P3. The second downsampling channel module downsamples feature map P3 to obtain a downsampled image of P3. The fifth feature fusion module fuses the downsampled image of P3, feature map C4, and feature map F4, and then performs convolution to obtain feature map P4. The third downsampling channel module downsamples feature map P4 to obtain a downsampled image of P4. The sixth feature fusion module fuses the downsampled image of P4 and the feature map obtained by convolution with C5, and then performs convolution to obtain feature map P5. Feature map P5 is output as the feature map after feature fusion.

[0104] In this embodiment of the invention, a BiFPN network is introduced as the feature fusion network for the Mask R-CNN algorithm. The BiFPN network structure incorporates inverse and lateral connections, allowing information to flow freely between different resolution levels. Therefore, the BiFPN network can efficiently utilize information from the feature pyramid and perform feature fusion in a balanced manner, thereby improving the accuracy and efficiency of target detection. Simultaneously, the BiFPN network progressively refines, aggregates, and enhances features through multiple repeated BiFPN blocks, making them more expressive and discriminative, thus improving the accuracy of target detection. Furthermore, the BiFPN network can handle targets of different sizes, proportions, and orientations, making it more robust. This improves the performance of the defect detection model for detecting small targets.

[0105] The principle of BiFPN is as follows: Figure 5As shown, BiFPN adds a shallow-to-deep path to the FPN (One-Way Feature Pyramid Network), fusing features from the shallow feature map with the previous deep feature map through max pooling. BiFPN allows for a thorough fusion of deep and shallow features, resulting in feature maps rich in semantic and positional information. Furthermore, BiFPN assigns trainable weights to each input feature map in each feature fusion operation, enabling the model to learn the importance of different features, improving feature fusion quality and increasing the model's detection accuracy.

[0106] In one embodiment, the Region Proposal Network (RPN) takes a feature map from the BiFPN network as input and generates a series of candidate bounding boxes on the feature map using a sliding window.

[0107] In one embodiment, the Region of Interest Alignment (RoIAlign) layer aligns candidate bounding boxes of different sizes into a fixed-size feature map for subsequent object classification and segmentation.

[0108] like Figure 2 As shown, the object detection network is used to classify and regress candidate object boxes on the standard feature map to obtain predicted object boxes, and outputs the category information and location information of the predicted object boxes.

[0109] In one embodiment, the object detection network includes: the fully connected layer and a softmax activation function. The fully connected layer is used for classifying candidate object boxes and performing boundary regression to obtain predicted object boxes; the softmax activation function is used to output the probability of each category to obtain the category information of the predicted object boxes.

[0110] In another embodiment, the object detection network includes convolutional layers and a softmax activation function. The convolutional layers are used for classifying candidate bounding boxes and performing boundary regression to obtain predicted bounding boxes; the softmax activation function is used to output the probability of each category to obtain the category information of the predicted bounding boxes.

[0111] In the target detection network of this invention, the candidate target boxes are classified and the boundary regression is performed through fully connected layers or convolutional layers to obtain the predicted target boxes. The probability of each category is output through the softmax activation function to obtain the category information of the predicted target boxes.

[0112] In the bounding box regression process, the candidate target boxes are subjected to bounding box regression to calculate the offset between the candidate target boxes and the real target boxes. By comparing the offsets, the candidate target box that is closest to the (annotated) real target box is selected as the predicted target box output, thereby enabling accurate acquisition of the target (defect) location information.

[0113] In this embodiment of the invention, the Mask sub-network is used to generate a high-precision predictive segmentation mask. It should be noted that masking technology is used to accurately separate objects in an image from the background. It achieves fine-grained segmentation of image regions by classifying and labeling each pixel. Each pixel is assigned a label, and this label information forms a two-dimensional matrix, i.e., a mask, which can accurately describe the position and boundaries of different objects in the image.

[0114] Specifically, the Mask sub-network includes a fully convolutional network and a Sobel filter. The fully convolutional network generates a high-precision predicted segmentation mask, which, along with the true segmentation mask, is then input into the Sobel filter and convolved with its Sobel operator to obtain detailed segmentation edge information of the target.

[0115] Specifically, the Sobel operator of the Sobel filter includes a two-dimensional operator S. x and S y The calculation formula is as follows:

[0116]

[0117]

[0118] The Sobel filter is a two-dimensional filter used for edge detection. The Sobel operator describes the horizontal and vertical gradients respectively. During gradient descent training, it can utilize not only the edge strength along the x and y axes but also the edge direction to minimize the total loss. This additional information can speed up training and reduce training time significantly.

[0119] In this invention, a Sobel filter is added to the Mask sub-network (mask branch) of the Mask RCNN model, which can calculate the loss between the edges in the predicted segmentation mask and the real segmentation mask, thereby increasing the detailed feature information of the defect image segmentation edges of the 3D printed dot matrix structure.

[0120] Step 3: Train the improved Mask R-CNN model using the training sample set to obtain the trained improved Mask R-CNN model.

[0121] The Mask R-CNN model is trained using a large amount of image data to learn and understand the features of various objects. During the computation process, convolution and pooling operations are performed on the input image to gradually extract its feature information. Ultimately, the output segmentation mask can accurately describe the location and boundaries of different objects in the image.

[0122] During implementation, the initial learning rate of the model was 0.01, the momentum factor was 0.9, the weight decay coefficient was 0.0001, and the model was trained for a total of 90 epochs.

[0123] Specifically, when training the improved Mask R-CNN model using the training sample set, a total loss function is calculated, and the improved Mask R-CNN model is optimized by minimizing the total loss function.

[0124] During training, the improved Mask R-CNN model adjusts model parameters (such as weights) based on the value of the loss function, making the predictions closer to the true labels.

[0125] The formula for calculating the total loss function is:

[0126] L total =L cls +L box +L mask +L edgel

[0127] In the formula, L total For total loss, L cls For classification error, L box For regression error, L mask For segmentation error, L edgel This refers to edge error.

[0128] The formula for calculating the edge error is as follows:

[0129]

[0130] In the formula, y represents the target edge being marked. The predicted edges are labeled.

[0131] The formula for calculating the classification error is:

[0132]

[0133] In the formula, z represents the true class and p1 represents the probability of the predicted class.

[0134] The formula for calculating the regression error is:

[0135] L box =∑[R(bbox_pred-bbox_target)];

[0136]

[0137] In the formula, bbox_pred is the position of the predicted target box, and bbox_target is the position of the actual target box.

[0138] The formula for calculating the segmentation error is:

[0139]

[0140] In the formula, m is the binarized representation of the true segmentation mask, and p2 is the probability of the predicted segmentation mask.

[0141] The total loss function of the improved Mask R-CNN model in this embodiment of the invention includes the classification error L. cls Regression error L box Segmentation error L mask and edge loss L edgel The four-part method, by introducing edge error into the total loss function, calculates the edge loss between the predicted segmentation mask and the real segmentation mask, which helps to improve the accuracy of model training.

[0142] Step 4: Acquire two-dimensional tomographic image data of the 3D printed lattice structure to be inspected and preprocess it, including the following steps:

[0143] Step 41: Collect CT scan data of the 3D printed lattice structure to be inspected and reconstruct it to obtain a three-dimensional view;

[0144] Step 42: Cut the three-dimensional view along the xy, xz and xz directions respectively to obtain two-dimensional tomographic images in the three directions respectively;

[0145] Step 43: Compare the clarity of the internal structure information of the two-dimensional tomographic images in the three directions, and select the two-dimensional tomographic image data with the highest clarity as the two-dimensional tomographic image data to be used in practice.

[0146] Step 44: Augment the two-dimensional tomographic image data in the direction with the highest selected clarity, and then normalize the augmented two-dimensional tomographic image data.

[0147] Specifically, during augmentation, the image is expanded by adjusting its brightness, contrast, hue, and saturation, and by flipping it horizontally or vertically. During normalization, the image is normalized to 512×512 pixels, with any insufficient pixels padded with zeros.

[0148] Step 5: Input the dataset to be detected into the trained improved Mask RCNN model to obtain the defect detection results of the 3D printed dot matrix structure to be detected.

[0149] Specifically, the defect detection results output by the embodiments of the present invention include defect location information, category information, and segmentation mask information, wherein the segmentation mask information can reflect the detailed feature information of the defect graphic segmentation edge.

[0150] In this invention, the Mask R-CNN algorithm is applied to defect detection of 3D printed dot matrix structures. By improving the Mask R-CNN model, it is possible to simultaneously detect, classify, and segment defects. Thus, the trained and improved Mask R-CNN model can perform more accurate and comprehensive defect analysis on 3D printed dot matrix structures.

[0151] Those skilled in the art will understand that all or part of the processes of the methods described in the above embodiments can be implemented by a computer program instructing related hardware, and the program can be stored in a computer-readable storage medium. The computer-readable storage medium may be a disk, optical disk, read-only memory, or random access memory, etc.

[0152] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A defect detection method for 3D printed lattice structures, characterized in that, The method includes: Two-dimensional tomographic image data of 3D printed lattice structure sample parts were acquired and preprocessed to construct a training sample set; An improved Mask RCNN model that integrates an attention mechanism and a BiFPN network is constructed. The improved Mask R-CNN model is trained using the training sample set to obtain the trained improved Mask R-CNN model; Two-dimensional tomographic image data of the 3D printed lattice structure to be detected are acquired and preprocessed to construct the detection dataset; The dataset to be detected is input into the trained improved Mask RCNN model to obtain the defect detection results of the 3D printed dot matrix structure to be detected; The improved Mask R-CNN model includes: A residual network is used to extract data features and output feature maps. The residual network includes attention residual blocks, which include convolutional layers and effective channel attention modules. BiFPN network is used to perform feature fusion on the feature map output by the residual network and output the feature fused feature map. A region proposal network is used to generate candidate bounding boxes on the feature map of the feature fusion. A region of interest alignment layer is used to align candidate target boxes on the feature map fused with the features and generate a standard feature map of a preset size. The object detection network is used to classify and regress candidate object boxes on the standard feature map to obtain predicted object boxes, and output the category and location information of the predicted object boxes. The Mask sub-network is used to perform semantic segmentation on the standard feature map to generate a high-precision predicted segmentation mask; The BiFPN network includes: a first upsampling channel module, a second upsampling channel module, a third upsampling channel module, a first downsampling channel module, a second downsampling module, a third downsampling channel module, a first feature fusion module, a second feature fusion module, a third feature fusion module, a fourth feature fusion module, a fifth feature fusion module, and a sixth feature fusion module. Specifically, the first residual module outputs feature map C2, the second residual module outputs feature map C3, the third residual module outputs feature map C4, and the fourth residual module outputs feature map C5. The first upsampling channel module upsamples the convolutional feature map C5 to obtain an upsampled image of C5. The first feature fusion module then fuses the upsampled image of C5 with the convolutional feature map of C4 to obtain feature map F4. The second upsampling channel module upsamples feature map F4 to obtain an upsampled image of F4. The second feature fusion module then fuses the upsampled image of F4 with the convolutional feature map of C3 to obtain feature map F3. The third upsampling channel module upsamples feature map F3 to obtain an upsampled image of F3. The third feature fusion module then fuses the upsampled image of F3 with the convolutional feature map of C2, and then performs convolution to obtain feature map P2. The first downsampling channel module upsamples the feature map... The P2 downsampled map is obtained by downsampling P2. The fourth feature fusion module fuses the downsampled P2 map, feature map C3, and feature map F3, and then performs convolution to obtain feature map P3. The second downsampling channel module downsamples feature map P3 to obtain a downsampled P3 map. The fifth feature fusion module fuses the downsampled P3 map, feature map C4, and feature map F4, and then performs convolution to obtain feature map P4. The third downsampling channel module downsamples feature map P4 to obtain a downsampled P4 map. The sixth feature fusion module fuses the downsampled P4 map and the feature map obtained by convolution with C5, and then performs convolution to obtain feature map P5. Feature map P5 is output as the feature map after feature fusion.

2. The method according to claim 1, characterized in that, The residual network consists of a convolutional pooling layer, a first residual block, a second residual block, a third residual block, and a fourth residual block connected in sequence. The convolutional pooling layer includes a convolutional layer and a pooling layer. The first residual block, the second residual block, and the third residual block all include attention residual units, each of which comprises two convolutional layers and an effective channel attention module. The fourth residual block includes a conventional residual unit, which includes two convolutional layers.

3. The method according to claim 1 or 2, characterized in that, The effective channel attention module includes: The global average pooling layer is used to perform global average pooling on the input feature map of size H×W×C to obtain a feature vector of size 1×1×C; where H, W, and C represent the height, width, and number of feature channels, respectively. One-dimensional convolutional layers are used to perform one-dimensional convolution calculations on the feature vectors to capture cross-channel interaction information; and The sigmoid activation function is used to calculate the weights for each feature channel.

4. The method according to claim 3, characterized in that, The formula for calculating the weight of each feature channel is: ； ； In the formula, This is represented as a convolution operation. As weight, Indicates the input feature map, This represents the sigmoid activation function. For input values.

5. The method according to claim 3, characterized in that, The formula for calculating the kernel size of the one-dimensional convolutional layer is: ； In the formula, The kernel size is [size]. Let C be the odd number closest to t, and C be the number of feature channels. and It is a fixed value.

6. The method according to claim 1, characterized in that, The Mask sub-network includes: a fully convolutional network and a Sobel filter. In this process, a high-precision predicted segmentation mask is generated through the fully convolutional network. The predicted segmentation mask and the real segmentation mask are then input into the Sobel filter and convolved with its Sobel operator to obtain the segmentation edge detail information of the target. The Sobel operator includes two-dimensional operators. and The calculation formula is as follows: ； 。 7. The method according to claim 1 or 2, characterized in that, The target detection network includes: The fully connected layer is used for classifying candidate bounding boxes and performing boundary regression to obtain predicted bounding boxes; and the softmax activation function is used to output the probability of each category to obtain the category information of the predicted bounding boxes.

8. The method according to any one of claims 1-2, characterized in that, The process of acquiring and preprocessing the two-dimensional tomographic image data of the 3D printed lattice structure to be detected includes: Collect CT scan data of the 3D printed lattice structure to be inspected and reconstruct it to obtain a three-dimensional view; The three-dimensional view is cut along the xy, xz and xz directions respectively to obtain two-dimensional tomographic images in the three directions respectively; By comparing the clarity of the internal structural information of the two-dimensional tomographic images in three directions, the two-dimensional tomographic image data with the highest clarity is selected as the two-dimensional tomographic image data to be used in practice. The two-dimensional tomographic image data of the selected direction with the highest clarity is augmented, and the augmented two-dimensional tomographic image data is normalized.

Images