A method for detecting internal defects of a 3D printing dot matrix structure

Abstract

The present application relates to a kind of 3D printing dot array structure defect detection model training method, belong to 3D printing defect detection technical field, solve the problem that prior art cannot describe the specific topographic details features of defect in dot array structure body.The method of the present application comprises: obtaining the two-dimensional tomographic image data of 3D printing dot array structure;After the two-dimensional tomographic image data is preprocessed, it is divided into training set and test set;Improved Mask RCNN model of fusion attention mechanism and BiFPN network is constructed;From training set, the training sample of batch size size is obtained to train improved Mask RCNN model;The improved Mask RCNN model trained is tested by test set to detect model precision, when model precision meets the requirement, improved Mask RCNN model is obtained as defect detection model as required.The improved Mask RCNN model is applied to the defect detection of 3D printing dot array structure in the present application, and the model precision is guaranteed by model training, so that accurate and comprehensive defect analysis is realized.

Description

Technical Field

[0001] This invention relates to the field of 3D printing defect detection technology, and in particular to a method for detecting internal defects in 3D printed lattice structures. Background Technology

[0002] 3D-printed multifunctional lattice structures are formed by the expansion of nodes and connecting rods between nodes according to a certain spatial periodic law. They possess excellent mechanical properties such as ultra-lightweight, high porosity, high specific strength, high specific stiffness, high strength and toughness, and high energy absorption, as well as sound absorption, vibration reduction, heat dissipation, electromagnetic shielding, and excellent permeability. Combining structural and functional benefits, they are a high-performance multifunctional engineering structural material that can be used to make filters, energy absorption devices, silencers, heat exchange flame retardants, shock absorbers, and biomimetic materials. They have broad development prospects and significant demand in fields such as aerospace vehicle structures, modern ship structures, high-speed rail structures, and modern protective structures.

[0003] However, inherent defects inevitably arise during the fabrication process of 3D-printed lattice structures, severely impacting their lifespan and reliability. Therefore, understanding the shape characteristics and distribution of these defects is crucial for guiding the manufacturing and health assessment of lattice structures. 3D-printed lattice structures primarily rely on industrial CT technology and 3D reconstruction to acquire tomographic images for internal defect detection. Defects in complex metal 3D-printed lattice structures exhibit randomness and uncertainty, manifesting as indeterminate locations and gradual feature variations in CT slices. The randomness of these defects, meaning their location is not fixed, significantly increases the workload. Manual inspection or low-automation inspection of numerous lattice structure CT images results in low efficiency and high false negative and false positive rates. With the rapid development of deep learning, existing technologies utilize deep learning models such as YOLOv3 and Faster R-CNN for defect detection in lattice structure CT images. However, the detection results from these existing deep learning models only indicate the defect location using rectangular bounding boxes, failing to accurately represent the defect's shape, i.e., they cannot reveal the specific morphological details of the defect within the lattice structure. Summary of the Invention

[0004] Based on the above analysis, the present invention aims to provide a training method for a defect detection model of 3D printed lattice structures, in order to solve the problem that the detection results output by existing defect detection models cannot describe the specific morphological details of defects within the lattice structure.

[0005] This invention provides a training method for a defect detection model of a 3D printed lattice structure, the method comprising:

[0006] Acquire two-dimensional tomographic image data of 3D printed lattice structures;

[0007] The two-dimensional tomographic image data is preprocessed, and the preprocessed data is divided into a training set and a test set;

[0008] An improved Mask R-CNN model that integrates an attention mechanism and a BiFPN network is constructed.

[0009] The improved Mask R-CNN model is trained using training samples of batch size obtained from the training set.

[0010] The trained improved Mask R-CNN model is tested using the test set to check the model accuracy. When the model accuracy meets the requirements, the improved Mask R-CNN model is obtained as the defect detection model. Otherwise, training samples of batch size are obtained from the training set again to train the improved Mask R-CNN model until the model accuracy meets the requirements.

[0011] Based on the further improvement of the above method, when training the improved Mask R-CNN model with the training set, the total loss function is calculated, and the improved Mask R-CNN model is optimized by minimizing the total loss function;

[0012] The formula for calculating the total loss function is:

[0013] L total =L cls +L box +L mask +L edgel

[0014] In the formula, L total For total loss, L cls For classification error, L box For regression error, L mask For segmentation error, L edgel This refers to edge error.

[0015] Based on a further improvement of the above method, the formula for calculating the edge error is:

[0016]

[0017] In the formula, L edge Let y represent the edge error, and y represent the marked target edge. The predicted edges are labeled.

[0018] Based on a further improvement of the above method, the formula for calculating the classification error is:

[0019]

[0020] In the formula, z is the true class and p1 is the probability of the predicted class;

[0021] The formula for calculating the regression error is:

[0022] L box =∑[R(bbox_pred-bbox_target)];

[0023]

[0024] The formula for calculating the segmentation error is:

[0025]

[0026] In the formula, m is the binarized representation of the true segmentation mask, and p2 is the probability of the predicted segmentation mask.

[0027] Based on a further improvement of the above method, the improved Mask R-CNN model includes:

[0028] A residual network is used to extract data features and output feature maps. The residual network includes attention residual blocks, which include convolutional layers and effective channel attention modules.

[0029] BiFPN network is used to perform feature fusion on the feature map output by the residual network and output the feature fused feature map.

[0030] A region proposal network is used to generate candidate bounding boxes on the feature map of the feature fusion.

[0031] A region of interest alignment layer is used to align candidate target boxes on the feature map fused with the features and generate a standard feature map of a preset size.

[0032] The object detection network is used to classify and regress candidate object boxes on the standard feature map to obtain predicted object boxes, and output the category and location information of the predicted object boxes.

[0033] The Mask subnetwork is used to perform semantic segmentation on the standard feature map to generate a high-precision predicted segmentation mask.

[0034] Based on a further improvement of the above method, the effective channel attention module includes:

[0035] The global average pooling layer is used to perform global average pooling on the input feature map of size H×W×C to obtain a feature vector of size 1×1×C; where H, W, and C represent the height, width, and number of feature channels, respectively.

[0036] One-dimensional convolutional layers are used to perform one-dimensional convolution calculations on the feature vectors to capture cross-channel interaction information; and

[0037] The simoid activation function is used to calculate the weights for each feature channel.

[0038] Based on a further improvement to the above method, the formula for calculating the kernel size of the one-dimensional convolutional layer is:

[0039]

[0040] In the formula, k is the kernel size, |t| odd y is the odd number closest to t, C is the number of feature channels, and γ and b are fixed values.

[0041] Based on a further improvement of the above method, the Mask sub-network includes: a fully convolutional network and a Sobel filter.

[0042] Specifically, a high-precision predicted segmentation mask is generated through the fully convolutional network. The predicted segmentation mask and the real segmentation mask are then input into the Sobel filter and convolved with its Sobel operator to obtain the segmentation edge detail information of the target.

[0043] Based on a further improvement of the above method, the preprocessing of the two-dimensional tomographic image data includes:

[0044] The two-dimensional tomographic image data is augmented.

[0045] The augmented two-dimensional tomographic image data is then normalized.

[0046] Defects in the normalized two-dimensional tomographic image data are labeled.

[0047] Based on a further improvement to the above method, the annotation of defects on the normalized two-dimensional tomographic image data includes:

[0048] Using graphic annotation tools, polygonal boxes are used to annotate defects on normalized two-dimensional tomographic image data.

[0049] Compared with the prior art, the present invention can achieve at least one of the following beneficial effects:

[0050] 1. In this invention, the Mask R-CNN algorithm is applied to defect detection of 3D printed dot matrix structures. By improving the Mask R-CNN model, it is possible to simultaneously detect, classify and segment defects. The model accuracy is ensured through model training. Thus, the trained and improved Mask R-CNN model can perform more accurate and comprehensive defect analysis on 3D printed dot matrix structures.

[0051] 2. In this invention, edge error is introduced into the total loss function to calculate the edge loss between the predicted segmentation mask and the real segmentation mask, which helps to improve the accuracy of model training.

[0052] 3. In this invention, when annotating defects in two-dimensional tomographic image data, polygonal boxes are used for annotation, which can annotate the actual shape of the defects and obtain the true segmentation mask information, thereby ensuring the accuracy of model training.

[0053] 4. In this invention, an effective channel attention (ECA) mechanism is added to the residual network of the Mask RCNN model, which significantly improves the feature extraction capability of the residual network. Under the condition of high gray value in dot matrix CT images, it can better capture subtle differences in image features, thereby obtaining more accurate and reliable defect detection results.

[0054] 5. In this invention, the BiFPN network (Bidirectional Feature Pyramid Network) is used as the backbone of the Mask RCNN model. The BiFPN network combines inverse and lateral connections, realizing effective feature fusion at different scales, enriching the semantic information in the feature map, thereby improving the detection accuracy, and is particularly suitable for detecting weak defects.

[0055] 6. In this invention, a Sobel filter is added to the Mask sub-network (mask branch) of the Mask RCNN model to generate a high-precision prediction segmentation mask for the target, thereby increasing the detailed feature information of the defect image segmentation edge of the 3D printed dot matrix structure.

[0056] In this invention, the above-described technical solutions can be combined with each other to achieve more preferred combinations. Other features and advantages of this invention will be set forth in the following description, and some advantages may become apparent from the description or be learned by practicing the invention. The objects and other advantages of this invention can be realized and obtained from what is particularly pointed out in the description and drawings. Attached Figure Description

[0057] The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Throughout the drawings, the same reference numerals denote the same parts.

[0058] Figure 1 This is a flowchart of a training method for a 3D printed lattice structure defect detection model according to an embodiment of the present invention;

[0059] Figure 2 This is a schematic diagram of the structure of the improved Mask RCNN model according to an embodiment of the present invention;

[0060] Figure 3 This is a schematic diagram illustrating the principle of the effective channel attention module in an embodiment of the present invention.

[0061] Figure 4 This is a schematic diagram of the Resnet50-CEA network according to an embodiment of the present invention;

[0062] Figure 5 This is a schematic diagram of the BiFPN network in an embodiment of the present invention. Detailed Implementation

[0063] Preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings, which form part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not intended to limit the scope of the present invention.

[0064] A specific embodiment of the present invention discloses a training method for a defect detection model of a 3D printed lattice structure, such as... Figure 1 As shown. The method includes:

[0065] Step 1: Obtain two-dimensional tomographic image data of the 3D printed lattice structure;

[0066] Step 2: Preprocess the two-dimensional tomographic image data and divide the preprocessed data into a training set and a test set;

[0067] Step 3: Construct an improved Mask R-CNN model that integrates the attention mechanism and the BiFPN network;

[0068] Step 4: Obtain training samples of batch size from the training set to train the improved Mask RCNN model;

[0069] The trained improved Mask R-CNN model is tested using the test set to check the model accuracy. When the model accuracy meets the requirements, the improved Mask R-CNN model is obtained as the defect detection model. Otherwise, training samples of batch size are obtained from the training set again to train the improved Mask R-CNN model until the model accuracy meets the requirements.

[0070] In this embodiment of the invention, the Mask R-CNN algorithm is applied to defect detection of 3D printed dot matrix structures. By improving the Mask R-CNN model, it is possible to simultaneously detect, classify, and segment defects. Furthermore, the model accuracy is ensured through model training. Thus, the trained and improved Mask R-CNN model can perform more accurate and comprehensive defect analysis on 3D printed dot matrix structures.

[0071] Specifically, step 1, acquiring two-dimensional tomographic image data of the 3D printed lattice structure, includes the following steps:

[0072] Step 11: Collect CT scan data of the 3D printed lattice structure and reconstruct it to obtain a three-dimensional view;

[0073] Step 12: Cut the three-dimensional view along the preset cutting direction to obtain two-dimensional tomographic image data.

[0074] Step 1 is the data acquisition step. Specifically, the preset cutting directions include the xy direction, the xz direction, and the xz direction.

[0075] Preferably, the three-dimensional view is cut along three directions: xy, xz, and xy-z, respectively, to obtain two-dimensional tomographic images in each direction. The clarity of the internal structural information in the three tomographic images is compared, and the two-dimensional tomographic image data with the highest clarity is selected as the actual two-dimensional tomographic image data used. For example, the xy plane is selected as the actual cutting direction, the spacing between the sections is 0.1 mm, the pixel size is 1474*1513, and a total of 119 two-dimensional tomographic images are obtained.

[0076] Step 2, the preprocessing of the two-dimensional tomographic image data, includes the following steps:

[0077] Step 21: Augmentation processing is performed on the two-dimensional tomographic image data. Specifically, the dataset is expanded by adjusting the brightness, contrast, hue, and saturation of the image, and by flipping the image horizontally or vertically.

[0078] Step 22: Normalize the augmented 2D tomographic image data. Specifically, normalize the image to 512×512 pixels, padding any insufficient pixels with zeros.

[0079] Step 23: Label the defects on the normalized two-dimensional tomographic image data.

[0080] Preferably, in step 23, a graphical annotation tool, such as Labelme, is used to annotate the defects on the normalized two-dimensional tomographic image data with polygonal boxes, thereby annotating the actual shape of the defects, obtaining the true target box, and generating a true segmentation mask based on the true target box.

[0081] It should be noted that using polygonal box annotation means annotating according to the edge shape of the defect.

[0082] In this invention, when annotating defects in two-dimensional tomographic image data, polygonal boxes are used for annotation, which can annotate the actual shape of the defects and obtain the true segmentation mask information, thereby ensuring the accuracy of model training.

[0083] In step 2, after preprocessing the data, the preprocessed data is divided into training and testing sets. For example, the acquired data can be divided into training and testing sets in a 4:1 ratio.

[0084] The improved Mask R-CNN model constructed in step 3, which integrates the attention mechanism and the BiFPN network, includes:

[0085] A residual network is used to extract data features and output feature maps. The residual network includes attention residual blocks, which include convolutional layers and effective channel attention modules.

[0086] BiFPN network is used to perform feature fusion on the feature map output by the residual network and output the feature fused feature map.

[0087] A region proposal network is used to generate candidate bounding boxes on the feature map of the feature fusion.

[0088] A region of interest alignment layer is used to align candidate target boxes on the feature map fused with the features and generate a standard feature map of a preset size.

[0089] The object detection network is used to classify and regress candidate object boxes on the standard feature map to obtain predicted object boxes, and output the category and location information of the predicted object boxes.

[0090] The Mask subnetwork is used to perform semantic segmentation on the standard feature map to generate a high-precision predicted segmentation mask.

[0091] In this invention, an Efficient Channel Attention (ECA) mechanism is added to the residual network of the Mask RCNN model, which significantly improves the feature extraction capability of the residual network. Under the condition of high gray value in dot matrix CT images, it can better capture subtle differences in image features, thereby obtaining more accurate and reliable defect detection results.

[0092] Specifically, to balance accuracy and efficiency, the residual network in this embodiment of the invention employs a ResNet50 that incorporates an Effective Channel Attention (ECA) module. ResNet50 learns image data through residual blocks, and the residual network effectively alleviates the gradient vanishing and training degradation problems of deep networks without increasing model parameters, thereby improving network convergence performance. Simultaneously, considering the high similarity of grayscale values ​​in dot matrix CT images, segmentation regions are easily affected, leading to false positives and false negatives. Therefore, to compensate for these shortcomings, this embodiment of the invention introduces an attention mechanism into the residual network. Attention-based methods allow the model to focus on and enhance effective feature information while ignoring some useless feature information, thereby improving the model's robustness. Given the corrective effect of the ECA module on channel features, a ResNet50 with ECA is used to enhance the ability to extract vital signs while increasing model complexity by almost no factor.

[0093] Specifically, the Effective Channel Attention (ECA) module includes a Global Average Pooling (GAP) layer, a one-dimensional convolutional layer with a kernel size of k, and a simoid activation function. The GAP layer performs global average pooling on the input feature map of size H×W×C to obtain a feature vector of size 1×1×C; the one-dimensional convolutional layer performs one-dimensional convolution on the feature vector to capture cross-channel interaction information; and the simoid activation function calculates the weight w for each feature channel.

[0094] ECA is a lightweight channel attention mechanism that effectively captures local cross-channel interaction information. It is an enhanced and improved version of the SE module, and its principle is as follows: Figure 3As shown, the main idea is to propose a strategy for local cross-channel interactions without dimensionality reduction. This strategy captures local cross-channel interaction information by considering each channel and its k nearest neighbors after global average pooling (GAP). The ECA module first uses GPA to compute an input feature map of size H×W×C (H, W, and C represent the height, width, and number of feature channels, respectively) to obtain a feature vector of size 1×1×C with a global receptive field. Secondly, it captures cross-channel interaction information through a one-dimensional convolution with a kernel size of k. The kernel size k is related to the number of input channels and is adaptively chosen to determine the coverage of local cross-channel interactions. Then, it computes the weight w for each feature channel using a sigmoid activation function and outputs a weight vector. Finally, it multiplies the output weight vector with the original input feature map to complete the original feature labeling in the channel dimension, making the extracted features more directional, suppressing ineffective features, and thus enhancing the extraction of effective features. The ECA module avoids dimensionality reduction, enabling the model to learn more effective channel attention. Furthermore, the module has very few parameters, determined only by its kernel size k (other parameters are almost negligible).

[0095] The effective channel attention module calculates the weights of each feature channel using the sigmoid activation function using the following formula:

[0096] w = σ(f k (g(x)));

[0097] In the formula, f k This represents a convolution operation, where w is the weight, g(x) represents the input feature map, and σ represents the sigmoid activation function.

[0098] Specifically, the sigmoid activation function is: σ = 1 / (1 + e^(-1 / 2)) -x ), where x is the input value.

[0099] The formula for calculating the kernel size of the effective channel attention module is:

[0100]

[0101] In the formula, k is the kernel size, |t| odd y is the odd number closest to t, C is the number of feature channels, and γ and b are fixed values.

[0102] In one embodiment, the residual network includes a convolutional pooling layer, a first residual block, a second residual block, a third residual block, and a fourth residual block connected in sequence. The convolutional pooling layer includes convolutional layers and pooling layers. The first, second, and third residual blocks each include an attention residual unit, which includes two convolutional layers (weight layers) and an effective channel attention module. The fourth residual block includes a regular residual unit, which includes two convolutional layers (weight layers).

[0103] Specifically, the first residual block includes 3 attention residual units, the second residual block includes 4 attention residual units, the third residual block includes 6 attention residual units, each attention residual unit includes two convolutional layers and one effective channel attention module, and there are skip connections between the two convolutional layers; the fourth residual block includes 3 regular residual units, each regular residual unit includes two convolutional layers (weight layers), and there are skip connections between the two convolutional layers.

[0104] Compared with the existing ResNet50 residual network, in this embodiment of the invention, an effective channel attention module (ECA module) is added to the first residual block, the second residual block, and the third residual block, but not to the convolutional pooling layer and the fourth residual block. This increases the expressive power and classification accuracy of the residual network while avoiding the problem of overfitting and reducing computational complexity and memory consumption.

[0105] like Figure 4As shown, after the image is input into the residual network, it undergoes processing in five different stages: convolutional pooling layers, the first residual block, the second residual block, the third residual block, and the fourth residual block, outputting feature maps C1, C2, C3, C4, and C5, respectively. C1 is the feature map obtained by the first convolution and pooling operation on the input image through the convolutional pooling layers; it is the smallest in scale. C2, C3, C4, and C5 are the feature maps output at different stages in the residual network, respectively. Specifically, C2 is the output of the first residual block, C3 is the output of the second residual block, C4 is the output of the third residual block, and C5 is the output of the fourth residual block, i.e., the last residual block. After the image is input into the residual network, it first undergoes the first convolution and pooling operation to process the original features of the input image. Then, it undergoes three convolution operations to extract lower-level and intermediate-level features, and finally, it undergoes one more convolution operation to extract higher-level semantic information. Low-level and intermediate-level features have a strong ability to perceive local details and contextual information of the target. Therefore, adding an attention mechanism to the residual blocks (first residual block, second residual block, and third residual block) used to extract features at this level can enhance the importance and expressive power of these features, thereby improving the performance of the residual network. Furthermore, an attention mechanism is applied to the reference block used to extract intermediate-level features to further enhance the importance and expressive power of these features. However, no attention mechanism is added when processing the original features of the input image and extracting higher-level semantic information, which helps reduce computational complexity and memory consumption. Therefore, in this embodiment of the invention, adding an ECA module only to the first three residual blocks in the residual network can reduce computational complexity and memory consumption while ensuring good network model performance.

[0106] In this invention, the BiFPN network (Bidirectional Feature Pyramid Network) is used as the backbone of the Mask RCNN model. The BiFPN network combines inverse and lateral connections, realizing effective feature fusion at different scales, enriching the semantic information in the feature map, thereby improving the detection accuracy, and is particularly suitable for detecting weak defects.

[0107] In one embodiment, the BiFPN network includes: a first upsampling channel module, a second upsampling channel module, a third upsampling channel module, a first downsampling channel module, a second downsampling module, a third downsampling channel module, a first feature fusion module, a second feature fusion module, a third feature fusion module, a fourth feature fusion module, a fifth feature fusion module, and a sixth feature fusion module.

[0108] Among them, the first residual module outputs feature map C2, the second residual module outputs feature map C3, the third residual module outputs feature map C4, and the fourth residual module outputs feature map C5.

[0109] The first upsampling channel module upsamples the convolutional feature map C5 to obtain an upsampled image of C5. The first feature fusion module then fuses the upsampled image of C5 with the convolutional feature map of C4 to obtain feature map F4. The second upsampling channel module upsamples feature map F4 to obtain an upsampled image of F4. The second feature fusion module then fuses the upsampled image of F4 with the convolutional feature map of C3 to obtain feature map F3. The third upsampling channel module upsamples feature map F3 to obtain an upsampled image of F3. The third feature fusion module then fuses the upsampled image of F3 with the convolutional feature map of C2, and then performs convolution to obtain feature map P2. The first downsampling channel module upsamples the feature map... Figure P2 is downsampled to obtain a downsampled image of P2. The fourth feature fusion module fuses the downsampled image, feature map C3, and feature map F3, and then performs convolution to obtain feature map P3. The second downsampling channel module downsamples feature map P3 to obtain a downsampled image of P3. The fifth feature fusion module fuses the downsampled image of P3, feature map C4, and feature map F4, and then performs convolution to obtain feature map P4. The third downsampling channel module downsamples feature map P4 to obtain a downsampled image of P4. The sixth feature fusion module fuses the downsampled image of P4 and the feature map obtained by convolution with C5, and then performs convolution to obtain feature map P5. Feature map P5 is output as the feature map after feature fusion.

[0110] In this embodiment of the invention, a BiFPN network is introduced as the feature fusion network for the Mask R-CNN algorithm. The BiFPN network structure incorporates inverse and lateral connections, allowing information to flow freely between different resolution levels. Therefore, the BiFPN network can efficiently utilize information from the feature pyramid and perform feature fusion in a balanced manner, thereby improving the accuracy and efficiency of target detection. Simultaneously, the BiFPN network progressively refines, aggregates, and enhances features through multiple repeated BiFPN blocks, making them more expressive and discriminative, thus improving the accuracy of target detection. Furthermore, the BiFPN network can handle targets of different sizes, proportions, and orientations, making it more robust. This improves the performance of defect detection models for detecting small targets.

[0111] The principle of BiFPN is as follows: Figure 5As shown, BiFPN adds a shallow-to-deep path to the FPN (One-Way Feature Pyramid Network), fusing features from the shallow feature map with the previous deep feature map through max pooling. BiFPN allows for a thorough fusion of deep and shallow features, resulting in feature maps rich in semantic and positional information. Furthermore, BiFPN assigns trainable weights to each input feature map in each feature fusion operation, enabling the model to learn the importance of different features, improving feature fusion quality and increasing the model's detection accuracy.

[0112] In one embodiment, the Region Proposal Network (RPN) takes a feature map from the BiFPN network as input and generates a series of candidate bounding boxes on the feature map using a sliding window.

[0113] In one embodiment, the Region of Interest Alignment (RoIAlign) layer aligns candidate bounding boxes of different sizes into a fixed-size feature map for subsequent object classification and segmentation.

[0114] like Figure 2 As shown, the object detection network is used to classify and regress candidate object boxes on the standard feature map to obtain predicted object boxes, and outputs the category information and location information of the predicted object boxes.

[0115] In one embodiment, the object detection network includes: the fully connected layer and a softmax activation function. The fully connected layer is used for classifying candidate object boxes and performing boundary regression to obtain predicted object boxes; the softmax activation function is used to output the probability of each category to obtain the category information of the predicted object boxes.

[0116] In another embodiment, the object detection network includes convolutional layers and a softmax activation function. The convolutional layers are used for classifying candidate bounding boxes and performing boundary regression to obtain predicted bounding boxes; the softmax activation function is used to output the probability of each category to obtain the category information of the predicted bounding boxes.

[0117] In the target detection network of this invention, the candidate target boxes are classified and the boundary regression is performed through fully connected layers or convolutional layers to obtain the predicted target boxes. The probability of each category is output through the softmax activation function to obtain the category information of the predicted target boxes.

[0118] In the bounding box regression process, the candidate target boxes are subjected to bounding box regression to calculate the offset between the candidate target boxes and the real target boxes. By comparing the offsets, the candidate target box that is closest to the (annotated) real target box is selected as the predicted target box output, thereby enabling accurate acquisition of the target (defect) location information.

[0119] In this embodiment of the invention, the Mask sub-network is used to generate a high-precision predictive segmentation mask. It should be noted that masking technology is used to accurately separate objects in an image from the background. It achieves fine-grained segmentation of image regions by classifying and labeling each pixel. Each pixel is assigned a label, and this label information forms a two-dimensional matrix, i.e., a mask, which can accurately describe the position and boundaries of different objects in the image.

[0120] Specifically, the Mask sub-network includes a fully convolutional network and a Sobel filter. The fully convolutional network generates a high-precision predicted segmentation mask, which, along with the true segmentation mask, is then input into the Sobel filter and convolved with its Sobel operator to obtain detailed segmentation edge information of the target.

[0121] Specifically, the Sobel operator of the Sobel filter includes a two-dimensional operator S. x and S y The calculation formula is as follows:

[0122]

[0123]

[0124] The Sobel filter is a two-dimensional filter used for edge detection. The Sobel operator describes the horizontal and vertical gradients respectively. During gradient descent training, it can utilize not only the edge strength along the x and y axes but also the edge direction to minimize the total loss. This additional information can speed up training and reduce training time significantly.

[0125] In this invention, a Sobel filter is added to the Mask sub-network (mask branch) of the Mask RCNN model, which can calculate the loss between the edges in the predicted segmentation mask and the real segmentation mask, thereby increasing the detailed feature information of the defect image segmentation edges of the 3D printed dot matrix structure.

[0126] Step 4: Obtain training samples of batch size from the training set to train the improved Mask R-CNN model; test the trained improved Mask R-CNN model using the test set to check the model accuracy. If the model accuracy meets the requirements, the improved Mask R-CNN model is obtained as the defect detection model; otherwise, obtain training samples of batch size from the training set again to train the improved Mask R-CNN model until the model accuracy meets the requirements.

[0127] The Mask R-CNN model was trained using a large amount of image data to learn and understand the features of various objects. During the computation process, convolution and pooling operations were performed on the input images to gradually extract feature information. Ultimately, the output predicted segmentation mask accurately describes the location and boundaries of different objects in the image. In practice, the model was trained with an initial learning rate of 0.01, a momentum factor of 0.9, and a weight decay coefficient of 0.0001 for a total of 90 epochs.

[0128] Specifically, in step 4, when training the improved Mask R-CNN model using the training set, the total loss function is calculated, and the improved Mask R-CNN model is optimized by minimizing the total loss function.

[0129] During training, the improved Mask R-CNN model adjusts model parameters (such as weights) based on the value of the loss function, making the predictions closer to the true labels.

[0130] The formula for calculating the total loss function is:

[0131] L total =L cls +L box +L mask +L edgel

[0132] In the formula, L total For total loss, L cls For classification error, L box For regression error, L mask For segmentation error, L edgel This refers to edge error.

[0133] The formula for calculating the edge error is as follows:

[0134]

[0135] In the formula, y represents the target edge being marked. The predicted edges are labeled.

[0136] The formula for calculating the classification error is:

[0137]

[0138] In the formula, z represents the true class and p1 represents the probability of the predicted class.

[0139] The formula for calculating the regression error is:

[0140] L box =∑[R(bbox_pred-bbox_target)];

[0141]

[0142] In the formula, bbox_pred is the position of the predicted target box, and bbox_target is the position of the actual target box.

[0143] The formula for calculating the segmentation error is:

[0144]

[0145] In the formula, m is the binarized representation of the true segmentation mask, and p2 is the probability of the predicted segmentation mask.

[0146] The total loss function of the improved Mask R-CNN model in this embodiment of the invention includes the classification error L. cls Regression error L box Segmentation error L mask and edge loss L edgel The model consists of four parts. By introducing edge error into the total loss function, the edge loss between the predicted segmentation mask and the real segmentation mask is calculated, which helps to improve the accuracy of model training.

[0147] When the total loss of the test results meets the accuracy requirements, it indicates that the training of the improved Mask R-CNN model of this invention is complete. The two-dimensional tomographic image data of the 3D printed dot matrix structure to be detected is input into the trained improved Mask R-CNN model of this embodiment to obtain the defect detection results.

[0148] Those skilled in the art will understand that all or part of the processes of the methods described in the above embodiments can be implemented by a computer program instructing related hardware, and the program can be stored in a computer-readable storage medium. The computer-readable storage medium may be a disk, optical disk, read-only memory, or random access memory, etc.

[0149] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A training method for a defect detection model of a 3D printed lattice structure, characterized in that, The method includes: Acquire two-dimensional tomographic image data of 3D printed lattice structures; The two-dimensional tomographic image data is preprocessed, and the preprocessed data is divided into a training set and a test set; An improved Mask R-CNN model that integrates an attention mechanism and a BiFPN network is constructed. The improved Mask R-CNN model is trained using training samples of batch size obtained from the training set. The trained improved Mask R-CNN model is tested using the test set to check the model accuracy. When the model accuracy meets the requirements, the improved Mask R-CNN model is obtained as the defect detection model. Otherwise, training samples of batch size are obtained from the training set again to train the improved Mask R-CNN model until the model accuracy meets the requirements. The improved Mask R-CNN model includes: A residual network is used to extract data features and output feature maps. The residual network includes attention residual blocks, which include convolutional layers and effective channel attention modules. BiFPN network is used to perform feature fusion on the feature map output by the residual network and output the feature fused feature map. A region proposal network is used to generate candidate bounding boxes on the feature map of the feature fusion. A region of interest alignment layer is used to align candidate target boxes on the feature map fused with the features and generate a standard feature map of a preset size. The object detection network is used to classify and regress candidate object boxes on the standard feature map to obtain predicted object boxes, and output the category and location information of the predicted object boxes. The Mask sub-network is used to perform semantic segmentation on the standard feature map to generate a high-precision predicted segmentation mask; The effective channel attention module includes: The global average pooling layer is used to perform global average pooling on the input feature map of size H×W×C to obtain a feature vector of size 1×1×C; where H, W, and C represent the height, width, and number of feature channels, respectively. A one-dimensional convolutional layer is used to perform one-dimensional convolution calculations on the feature vector to capture cross-channel interaction information; and a sigmoid activation function is used to calculate the weights for each feature channel.

2. The method according to claim 1, characterized in that, When training the improved Mask R-CNN model using the training set, the total loss function is calculated, and the improved Mask R-CNN model is optimized by minimizing the total loss function. The formula for calculating the total loss function is: In the formula, For total losses, For classification error, For regression error, For segmentation error, This refers to edge error.

3. The method according to claim 2, characterized in that, The formula for calculating the edge error is: ； In the formula, For edge error, For the marked target edge, The predicted edges are labeled.

4. The method according to claim 2, characterized in that, The formula for calculating the classification error is: ； In the formula, z represents the true category. To predict the probability of the category; The formula for calculating the regression error is: ； ； In the formula, To predict the location of the target box, The location of the actual target bounding box; The formula for calculating the segmentation error is: ； In the formula, m is the binarized representation of the actual segmentation mask. To predict the probability of segmentation mask.

5. The method according to claim 1, characterized in that, The formula for calculating the kernel size of the one-dimensional convolutional layer is: ； In the formula, The kernel size is [size]. Let C be the odd number closest to t, and C be the number of feature channels. and It is a fixed value.

6. The method according to claim 1, characterized in that, The Mask sub-network includes: a fully convolutional network and a Sobel filter. Specifically, a high-precision predicted segmentation mask is generated through the fully convolutional network. The predicted segmentation mask and the real segmentation mask are then input into the Sobel filter and convolved with its Sobel operator to obtain the segmentation edge detail information of the target.

7. The method according to any one of claims 1-4, characterized in that, The preprocessing of the two-dimensional tomographic image data includes: The two-dimensional tomographic image data is augmented. The augmented two-dimensional tomographic image data is then normalized. Defects in the normalized two-dimensional tomographic image data are labeled.

8. The method according to claim 7, characterized in that, The annotation of defects on the normalized two-dimensional tomographic image data includes: Using graphic annotation tools, polygonal boxes are used to annotate defects on normalized two-dimensional tomographic image data.

Images