3D printing lattice structure CT image defect detection method based on deep learning
By building a dual attention mechanism image defect detection module based on deep learning, the problems of low damage detection efficiency and poor accuracy of 3D printed dot matrix structures in the prior art are solved, and accurate and efficient detection of internal defects are achieved, which improves detection efficiency and accuracy, and reduces detection costs.
Patent Information
- Application Number
- CN202210325459.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-03-29
- Publication Date
- 2025-06-06
- Estimated Expiration
- 2042-03-29
AI Technical Summary
The existing 3D printed dot matrix structure damage detection methods are inefficient and have poor accuracy, and manual detection is sensitive to image gradient, resulting in high detection costs and large errors.
Using a deep learning method, a deep learning model is constructed by building a dual attention mechanism image defect detection module, identifying image data, and achieving damage-free detection of internal defects of 3D printed dot matrix structure.
It realizes accurate and efficient detection of internal defects of 3D printed dot matrix structure, improves detection efficiency and accuracy, reduces detection costs, and has the advantages of high detection accuracy, good real-time performance and high degree of automation.
Smart Images
Figure CN114638819B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of non-destructive detection of 3D printed lattice structures, and specifically to a 3D printed lattice structure CT image defect detection method based on deep learning. Background Art
[0002] 3D printed lattice structures have the advantages of low volume density, high specific mechanical properties, excellent shock absorption, and ideal vibration absorption, and are widely used in aerospace, machinery, submarines and other fields. Printing lattice structures through selective laser melting (SLM) technology not only has a fast printing speed, but also allows the design of complex internal structures to meet industrial needs and create great economic benefits. However, due to the melting, solidification and defects of the powder itself, the printed product may have many defects, reducing or even destroying the mechanical properties of the lattice structure, so it is particularly important to perform non-destructive testing on it.
[0003] So far, the most commonly used non-destructive detection methods include X-ray detection, eddy current detection, ultrasonic detection, etc. X-ray detection is the most common. The 3D printed dot matrix structure is scanned by industrial CT to obtain a tomographic image, and then manually inspected. The detection efficiency is low and the error is large. Due to the gradual change of the image, it causes great harm to human vision. Therefore, it is necessary to perform intelligent detection of defective images, which can not only improve the detection efficiency and accuracy, but also reduce the detection cost. Summary of the invention
[0004] In order to overcome the shortcomings of the prior art, the present invention constructs a deep learning model to recognize image data through an image defect detection module based on a dual attention mechanism, thereby realizing non-destructive detection of internal defects of 3D printed dot matrix structures and accurately and efficiently extracting image defect information.
[0005] To achieve the above purpose, the solution adopted by the present invention is:
[0006] A 3D printing lattice structure CT image defect detection method based on deep learning, comprising the following steps:
[0007] Step 1: Obtain defect image data through industrial CT;
[0008] Use industrial CT to scan 3D printed lattice structural parts, obtain internal defect image data, and save the defect image data;
[0009] Step 2: Gaussian filter processing defect image data;
[0010] All image data must be subjected to Gaussian filtering, and the defective image data is expanded and enhanced by mirroring, rotating, translating, distorting, adjusting contrast, and repeating pixel value operations; the expression of Gaussian filtering is as follows:
[0011]
[0012] Where: x represents the value of the pixel point in the horizontal direction of the input image; y represents the value of the pixel point in the vertical direction of the input image; G(x,y) represents the Gaussian function; σ represents the variance of the Gaussian filter function;
[0013] Step 3: Build a defect detection model based on deep learning network;
[0014] The image data with defect information is divided into a training set and a validation set. The image data is passed into the 3D printing lattice structure defect detection model based on deep learning for training. The image data passes through the Resizer Images module in turn and integrates the YOLOv4 defect detection module with a dual attention mechanism for defect detection.
[0015] Step 31: Build the Resizer Model module;
[0016] The Resizer Model module is mainly composed of a convolutional layer and a BatchNorm layer. The convolutional layer function is obtained as follows:
[0017] f x =ρ(w l x l +b l )
[0018] Where: f x represents the convolutional layer function; x l Represents the input image data matrix; w l and b l Respectively represent the weight and bias of the layer; ρ represents the activation function;
[0019] The BatchNorm layer function is obtained as follows:
[0020]
[0021] Where: y i represents the BatchNorm layer function; γ and β represent the first training parameter and the second training parameter respectively; Indicates the matrix normalization of the i-th input image data; i represents the number of the input image data;
[0022] Step 32: Build a YOLOv4 defect detection module based on dual attention mechanism;
[0023] The dual attention mechanism is composed of the SE attention mechanism and the CBAM attention mechanism;
[0024] The SE attention mechanism needs to be integrated into the feature extraction network to assign different weights to the feature extraction network channels; the CBAM attention mechanism is integrated into the feature extraction network to assign different weights to different positions of the image, so as to better highlight the defect position;
[0025] Step 33: Build the loss function of the deep learning neural network;
[0026] The loss function of the YOLOv4 deep learning neural network that integrates the Resizer Model module and the dual attention module is obtained as follows:
[0027]
[0028] Where: L CIOU represents the loss function of the YOLOv4 deep learning neural network; IOU represents the intersection-over-union ratio between the detection box and the true box; A represents the detection box; B represents the true box; d represents the Euclidean distance; A zx and B zx Represents the center coordinates of the detection box and the true box; c is the diagonal length of the minimum bounding box of A and B; ν and a represent the first correction coefficient and the second correction coefficient for the aspect ratio respectively;
[0029] Step 4: Determine whether the defect detection model training results meet the requirements;
[0030] When the mAP value of the validation set on the detection network is above 95%, the model training is completed; save the model parameters, which include: number of iterations, optimizer model and trained weights; the mAP value acquisition method is as follows:
[0031]
[0032] Where: mAP represents the mean average precision; k represents the kth calculation area; N represents the number of calculation areas; p(k) represents the precision; r(k) represents the recall rate;
[0033] Step 5: Use the trained model parameters for the defect detection model;
[0034] The trained model parameters are embedded in the defect detection model and applied to the internal defect detection of 3D printed lattice structural parts.
[0035] Preferably, in step 1, the 3D printed lattice structure is scanned by industrial CT, and the number of slice images is determined according to the length, width and height of the structure and the size of the defect to form the image data.
[0036] Preferably, the Resizer Model module in step 31 enables the network to automatically learn the size of the sliced image data information, train an image size that matches the detection network, and automatically adjust the size information input to the detection network, so as to achieve the best detection effect.
[0037] Preferably, the method for obtaining the input image data matrix standardization in step 31 is as follows:
[0038]
[0039] Where: ε represents the offset of the pixel value; represents the variance of pixel values; x i represents the image data of the i-th input; θ B Represents the mean value of pixels;
[0040] Said The method for obtaining the pixel value variance is as follows:
[0041]
[0042] Where m represents the number of pixels;
[0043] The θ B The method for obtaining the mean pixel value is as follows:
[0044]
[0045] Preferably, the YOLOv4 defect detection module in step 33 incorporates the Hilbert curve principle, and incorporates the Hilbert expansion layer into the last layer of the YOLOv4 backbone network, which can better retain the image data defect information.
[0046] Preferably, the first correction coefficient and the second correction coefficient in step 33 are specifically as follows:
[0047] The method for obtaining the first correction coefficient is as follows:
[0048]
[0049] Where: w gt and h gt Respectively represent the width and height of the true frame; w and h represent the width and height of the detection frame;
[0050] The method for obtaining the second correction coefficient is as follows:
[0051]
[0052] Preferably, the method for obtaining the precision p(k) and the recall r(k) in step 4 is as follows:
[0053] The method for obtaining the precision rate p(k) is as follows:
[0054]
[0055] Where: tp represents the number of correctly identified images; fp represents the number of correct images that are incorrectly identified as wrong images;
[0056] The method for obtaining the recall rate r(k) is as follows:
[0057]
[0058] Where: fn represents the number of error images mistakenly identified as correct images.
[0059] Compared with the prior art, the present invention has the following beneficial effects:
[0060] (1) The present invention constructs a neural network deep learning training model through an image defect detection module based on a dual attention mechanism, completes the judgment and identification of defects in input image data, and realizes non-destructive detection of internal defect structures of 3D printed dot matrix structural parts;
[0061] (2) The present invention accurately and efficiently extracts defect information and analyzes the influence of the defect on the mechanical properties of the lattice structure, and has the advantages of high detection accuracy, good real-time performance, and high degree of automation. BRIEF DESCRIPTION OF THE DRAWINGS
[0062] Figure 1 This is a control block diagram of a 3D printing lattice structure CT image defect detection method based on deep learning in an embodiment of the present invention;
[0063] Figure 2 is an overall flow chart of an embodiment of the present invention;
[0064] Figure 3 (a) and (b) are defect images before and after Gaussian filtering processing according to an embodiment of the present invention, respectively;
[0065] Figure 4 A neural network framework diagram of an embodiment of the present invention;
[0066] Figure 5 4 is a Hilbert principle diagram of an embodiment of the present invention. DETAILED DESCRIPTION
[0067] Hereinafter, embodiments of the present invention will be described with reference to the drawings.
[0068] The embodiment of the present invention provides a 3D printing lattice structure CT image defect detection method based on deep learning, which constructs a deep learning model to recognize image data through an image defect detection module based on a dual attention mechanism, realizes non-destructive detection of internal defects of 3D printing lattice structure, and accurately and efficiently extracts image defect information, such as Figure 1 The figure is a control block diagram of a 3D printing lattice structure CT image defect detection method based on deep learning according to an embodiment of the present invention; Figure 2 is an overall flow chart of an embodiment of the present invention;
[0069] In order to prove the applicability of the present invention, it is applied to an example, which specifically includes the following steps:
[0070] S1: Obtain defect image data through industrial CT;
[0071] Use industrial CT to scan 3D printed lattice structural parts, determine the number of slice images according to the length, width and height of the structural parts and the size of the defects, and form image data; obtain internal defect image data and save the defect image data.
[0072] S2: Gaussian filter processing defect image data;
[0073] All image data must be subjected to Gaussian filtering, and defect image data can be expanded and enhanced through operations such as mirroring, rotation, translation, distortion, contrast adjustment, and repeated pixel values. Figure 3 As shown, (a) is the image before processing, and (b) is the image after processing; the method for obtaining Gaussian filtering is as follows:
[0074]
[0075] Where: x represents the value of the pixel point in the horizontal direction of the input image; y represents the value of the pixel point in the vertical direction of the input image; G(x,y) represents the Gaussian function; σ represents the variance of the Gaussian filter function;
[0076] S3: Build a defect detection model based on deep learning network;
[0077] The image data with defect information is divided into a training set and a validation set. The image data is passed into the 3D printing lattice structure defect detection model based on deep learning for training. The image data passes through the Resizer Images module in turn and integrates the YOLOv4 defect detection module with a dual attention mechanism for defect detection.
[0078] S31: Build the Resizer Model module;
[0079] The Resizer Model module is mainly composed of convolutional layers and BatchNorm layers. The convolutional layer function is obtained as follows:
[0080] f x =ρ(w l x l +b l )
[0081] Where: f x represents the convolutional layer function; x l Represents the input image data matrix; w l and b l Respectively represent the weight and bias of the layer; ρ represents the activation function;
[0082] The Resizer Model module enables the network to automatically learn the size of sliced image data information, train the image size that matches the detection network, and automatically adjust the size information input to the detection network to achieve the best detection effect.
[0083] The BatchNorm layer function is obtained as follows:
[0084]
[0085] Where: y i represents the BatchNorm layer function; γ and β represent the first training parameter and the second training parameter respectively; Indicates the matrix normalization of the i-th input image data; i represents the number of the input image data;
[0086] The method for obtaining the normalized image data matrix is as follows:
[0087]
[0088] Where: ε represents the offset of the pixel value; represents the variance of pixel values; x i represents the image data of the i-th input; θ B Represents the mean value of pixels;
[0089] The method for obtaining the pixel value variance is as follows:
[0090]
[0091] Where m represents the number of pixels;
[0092] θ B The method for obtaining the mean pixel value is as follows:
[0093]
[0094] S32: Construct a YOLOv4 defect detection module based on dual attention mechanism;
[0095] The dual attention mechanism consists of the SE attention mechanism and the CBAM attention mechanism;
[0096] The SE attention mechanism needs to be integrated into the feature extraction network to assign different weights to the feature extraction network channels; the CBAM attention mechanism is integrated into the feature extraction network to assign different weights to different positions of the image and better highlight the defect location;
[0097] S33: Building a loss function for deep learning neural networks;
[0098] The YOLOv4 deep learning neural network that integrates the Resizer Model module and the dual attention module is as follows: Figure 4 As shown; is a neural network framework diagram of an embodiment of the present invention; the loss function acquisition method of the YOLOv4 deep learning neural network is as follows:
[0099]
[0100] Where: L CIOU represents the loss function of the YOLOv4 deep learning neural network; IOU represents the intersection-over-union ratio between the true box and the detection box; A represents the detection box; B represents the true box; d represents the Euclidean distance; A zx and B zx Represents the center coordinates of the detection box and the true box; c is the diagonal length of the minimum bounding box of A and B; ν and a represent the first correction coefficient and the second correction coefficient for the aspect ratio respectively;
[0101] The method for obtaining the first correction coefficient is as follows:
[0102]
[0103] Where: w gt and h gt Respectively represent the width and height of the true frame; w and h represent the width and height of the detection frame;
[0104] The method for obtaining the second correction coefficient is as follows:
[0105]
[0106] The YOLOv4 defect detection module incorporates the Hilbert curve principle. The Hilbert expansion layer is integrated into the last layer of the YOLOv4 backbone network, which can better retain the image data defect information. The Hilbert principle diagram is as follows Figure 5As shown: Expanding the input image pixels according to the Hilbert curve can better preserve the image spatial features and ensure the integrity of image defect information.
[0107] S4: Determine whether the defect detection model training results meet the requirements;
[0108] When the mAP value of the validation set on the detection network is above 95%, the model training is completed; save the model parameters, which include: number of iterations, optimizer model and trained weights; the mAP value acquisition method is as follows:
[0109]
[0110] Where: mAP represents the mean average precision; k represents the kth calculation area; N represents the number of calculation areas; p(k) represents the precision; r(k) represents the recall rate;
[0111] The method for obtaining the precision p(k) is as follows:
[0112]
[0113] Where: tp represents the number of correctly identified images; fp represents the number of correct images that are incorrectly identified as wrong images;
[0114] The method for obtaining the recall rate r(k) is as follows:
[0115]
[0116] Where: fn represents the number of error images mistakenly identified as correct images.
[0117] S5: Use the trained model parameters for the defect detection model;
[0118] The trained model parameters are embedded in the defect detection model and applied to the internal defect detection of 3D printed lattice structural parts.
[0119] As shown in Table 1, the calculation results of the IOU true box and the detection box when the intersection-over-union ratio is 0.5 and 0.75 respectively are listed; the calculation results of Efficient D1, YOLOV3, YOLOV4 and this method are listed. Through the comparison of the mAP average precision mean data, it can be clearly seen that the calculation results of this method are superior to those of the other three methods.
[0120] Table 1 Comparison of the calculation results of the present invention with other methods
[0121] mAP IOU=0.5 IOU=0.75 Efficient D1 89.75 20.01 YOLOV3 94.25 19.87 YOLOV4 96.75 20.31 Our Model 97.65 41.55
[0122] In summary, the detection results of this case prove that the 3D printing lattice structure CT image defect detection method based on deep learning has a good effect.
[0123] (1) The embodiment of the present invention constructs a deep learning model through an image defect detection module based on a dual attention mechanism, thereby realizing non-destructive detection of internal defects of 3D printed dot matrix structures, thereby solving the problem that internal defects of structural parts cannot be accurately identified;
[0124] (2) The embodiment of the present invention demonstrates the superiority of the method by comparing the calculation results. It can accurately and efficiently extract defect information and analyze the influence of the defect on the mechanical properties of the lattice structure. It has the advantages of high detection accuracy, good real-time performance, and high degree of automation.
[0125] The embodiments described above are only descriptions of the preferred implementation modes of the present invention, and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary technicians in this field should all fall within the protection scope determined by the claims of the present invention.
Claims
1. A 3D printing lattice structure CT image defect detection method based on deep learning, It is characterized in that It includes the following steps: Step 1: Obtain defect image data through industrial CT; Use industrial CT to scan 3D printed lattice structural parts, obtain internal defect image data, and save the defect image data; Step 2: Gaussian filter processing defect image data; All image data must be subjected to Gaussian filtering, and the defective image data is expanded and enhanced by mirroring, rotating, translating, distorting, adjusting contrast, and repeating pixel value operations; the expression of Gaussian filtering is as follows: Where: x represents the value of the pixel point in the horizontal direction of the input image; y represents the value of the pixel point in the vertical direction of the input image; G(x,y) represents the Gaussian function; σ represents the variance of the Gaussian filter function; Step 3: Build a defect detection model based on deep learning network; The image data with defect information is divided into a training set and a validation set. The image data is passed into the 3D printing lattice structure defect detection model based on deep learning for training. The image data passes through the Resizer Images module and the YOLOv4 defect detection module with dual attention mechanism for defect detection. Step 31: Build the Resizer Model module; The Resizer Model module includes a convolutional layer and a BatchNorm layer. The convolutional layer function is obtained as follows: f x =ρ(w l x l +b l ) Where: f x represents the convolutional layer function; x l Represents the input image data matrix; w l and b l Respectively represent the weight and bias of the layer; ρ represents the activation function; The BatchNorm layer function is obtained as follows: Where: y i represents the BatchNorm layer function; γ and β represent the first training parameter and the second training parameter respectively; Indicates the matrix normalization of the i-th input image data; i represents the number of the input image data; Step 32: Build a YOLOv4 defect detection module based on dual attention mechanism; The dual attention mechanism is composed of the SE attention mechanism and the CBAM attention mechanism; The SE attention mechanism needs to be integrated into the feature extraction network to assign different weights to the feature extraction network channels; the CBAM attention mechanism is integrated into the feature extraction network to assign different weights to different positions of the image, so as to better highlight the defect position; Step 33: Build the loss function of the deep learning neural network; The loss function of the YOLOv4 deep learning neural network that integrates the Resizer Model module and the dual attention module is obtained as follows: Where: L CIOU represents the loss function of the YOLOv4 deep learning neural network; IOU represents the intersection-over-union ratio between the true box and the detection box; A represents the detection box; B represents the true box; d represents the Euclidean distance; A zx and B zx Represents the center coordinates of the detection box and the real box; c is the diagonal length of the minimum bounding box of A and B; ν and a represent the first correction coefficient and the second correction coefficient for the aspect ratio respectively; The YOLOv4 defect detection module in step 33 is integrated with the Hilbert curve, and the Hilbert expansion layer is integrated into the last layer of the YOLOv4 backbone network, so as to better retain the image data defect information; Step 4: Determine whether the defect detection model training results meet the requirements; When the mAP value of the validation set on the detection network is above 95%, the model training is complete; Save the model parameters, which include: number of iterations, optimizer model and trained weights; the mAP value acquisition method is as follows: Where: mAP represents the mean average precision; k represents the kth calculation area; N represents the number of calculation areas; p(k) represents the precision; r(k) represents the recall rate; Step 5: Use the trained model parameters for the defect detection model; The trained model parameters are embedded in the defect detection model and applied to the internal defect detection of 3D printed lattice structural parts.
2. The method for detecting defects in 3D printed lattice structures based on deep learning according to claim 1, It is characterized in that In the step 1, the 3D printed lattice structure is scanned by industrial CT, and the number of slice images is determined according to the length, width and height of the structure and the size of the defect to form image data.
3. The 3D printing lattice structure CT image defect detection method based on deep learning according to claim 1, It is characterized in that The Resizer Model module in step 31 enables the network to automatically learn the size of the sliced image data information, train the image size that matches the detection network, and automatically adjust the size information input to the detection network.
4. The method for detecting defects in 3D printed lattice structures based on deep learning according to claim 1, It is characterized in that The method for obtaining the input image data matrix standardization in step 31 is as follows: Where: ε represents the offset of the pixel value; represents the variance of pixel values; x i represents the image data of the i-th input; θ B Represents the mean value of pixels; Said The method for obtaining the pixel value variance is as follows: Where m represents the number of pixels; The θ B The method for obtaining the mean pixel value is as follows:
5. The 3D printing lattice structure CT image defect detection method based on deep learning according to claim 1, It is characterized in that The first correction coefficient and the second correction coefficient in step 33 are specifically as follows: The method for obtaining the first correction coefficient is as follows: Where: w gt and h gt Respectively represent the width and height of the real frame; w and h represent the width and height of the detection frame; The method for obtaining the second correction coefficient is as follows:
6. The method for detecting defects in CT images of 3D printed lattice structures based on deep learning according to claim 1, It is characterized in that The method for obtaining the precision p(k) and recall r(k) in step 4 is as follows: The method for obtaining the precision rate p(k) is as follows: Where: tp represents the number of correctly identified images; fp represents the number of correct images that are incorrectly identified as wrong images; The method for obtaining the recall rate r(k) is as follows: Where: fn represents the number of error images mistakenly identified as correct images.