Small sample rocket bonding defect detection method based on deep learning

Abstract

This invention belongs to the field of computer vision and discloses a method for detecting rocket adhesion defects based on deep learning in a few samples. The method includes the following steps: collecting and preprocessing images of rocket adhesion defects; training a deep convolutional generative adversarial network (DCGAN) to further expand the defect dataset; using a publicly available image dataset, pre-training a YOLOv5s deep learning model and saving the parameters of the YOLOv5s deep learning model; modifying the structure of the trained YOLOv5s deep learning model by introducing a dense network into the C3 module, and training it again to obtain a defect detection model; deploying the defect detection model using multi-process and multi-threading techniques to perform rocket adhesion defect detection. This invention applies deep learning methods to rocket adhesion defect detection, achieving accurate detection of rocket adhesion defects in a few samples based on DCGAN and YOLOv5s, and utilizing multi-process technology to successfully ensure the accuracy of model detection while maintaining real-time detection.

Description

Technical Field

[0001] This invention belongs to the field of computer vision, specifically a method for detecting small-sample rocket adhesion defects based on deep learning. Background Technology

[0002] Weapons are key to warfare. Even minor flaws within weaponry on the battlefield can lead to the malfunction of advanced weapon systems or the overall functionality of a weapon system, potentially resulting in combat defeat and a passive, disadvantageous position. As science continues to advance and the equipment level and modernization of the armed forces improve, higher demands are placed on weaponry testing technologies.

[0003] During rocket production, various bonding defects, such as debonding, cracking, and delamination, can occur due to limitations in manufacturing processes and the complexity of the on-site environment. Currently, the identification of rocket bonding defects relies on workers' personal experience. Because the defects are diverse and some are small and difficult to detect, significant manpower is often required for product quality inspection, which is time-consuming, labor-intensive, and subjective, making it difficult to guarantee accuracy. Efficient and reliable automated quality inspection can significantly reduce labor costs and improve the accuracy of defect identification. Therefore, this invention proposes a small-sample rocket bonding defect detection method based on deep learning. Summary of the Invention

[0004] To address the problems of insufficient samples, time-consuming and labor-intensive testing, and inherent subjectivity in existing rocket bonding defect detection methods, which make it difficult to guarantee accuracy, this invention provides a small-sample rocket bonding defect detection method based on deep learning to meet current needs and further improve the detection efficiency and speed of rocket bonding defect detection.

[0005] To achieve the above objectives, the present invention employs the following technical solutions:

[0006] This invention provides a method for detecting small-sample adhesion defects in rocket projectiles based on deep learning, comprising the following steps:

[0007] S1. Collect and preprocess images of rocket adhesion defects to obtain a defect dataset;

[0008] S2. Based on the dataset in step S1, design a deep convolutional generative adversarial network to further expand the defect dataset;

[0009] S3. Build a target detector model based on the YOLOv5 architecture;

[0010] S4. Construct a defect detection model;

[0011] S5. Deploy the defect detection model using multi-process and multi-threading technologies to improve the detection speed of the defect detection model, and use the defect detection model to detect adhesive defects in rocket projectiles.

[0012] Furthermore, step S1 specifically includes:

[0013] S11. Collect images of defects in the adhesion of rocket projectiles through terminal equipment;

[0014] S12. Use the MAKE SENSE tool to annotate the rocket adhesion defect images collected in step S11, and record the annotated information in a txt file.

[0015] S13. Perform data augmentation on the dataset obtained in step S12 using a non-generative data augmentation method to expand the small sample defect images.

[0016] Furthermore, step S2 specifically includes:

[0017] S21. Generate an adversarial network using the following objective function:

[0018]

[0019] In the formula, D is the discriminant network, G is the generative adversarial network, pdta(x) represents the distribution of the real sample x, and p z (z) represents the distribution of random noise z, D(G(z)) represents the probability that the D network judges whether the image generated by G is real; E represents the batch input network;

[0020] S22. Train the generative adversarial network using a publicly available image dataset, fix the discriminant network D, optimize the parameters of the generative adversarial network G, and complete the expansion of the defect dataset.

[0021] Furthermore, step S3 specifically includes:

[0022] S31. Build a YOLOv5s deep learning model;

[0023] S32. Using the publicly available COCO image dataset, pre-train the YOLOv5s deep learning model and save the parameters of the YOLOv5s deep learning model.

[0024] S33. Modify the structure of the trained YOLOv5s deep learning model by introducing a dense module into the C3 module of the YOLOv5s deep learning model network. The dense module includes Conv(1×1)+BN+SiLU+Conv(3×3)+BN+SiLU, where SiLU serves as a non-linear activation function to achieve non-linearity of the network. The 1×1 convolution and 3×3 convolution can be considered as a combination; the 1×1 convolution reduces the number of input feature maps and improves computational efficiency, while the 3×3 convolution layer extracts spatial features. The output of the entire module is the stacking of the input and the output of the 3×3 convolution, followed by a transition layer with a 1×1 kernel to reduce the number of channels and decrease model complexity.

[0025] S34. In order to enhance the network's feature fusion capability and reduce computational cost, four max pooling layers with different kernel sizes are used to achieve feature fusion at different scales.

[0026] S35. Build the PANet module to achieve feature fusion through sampling operations, build the Output module to predict image features, generate bounding boxes and predict categories, and complete the construction of the object detector model based on the YOLOv5 architecture.

[0027] Furthermore, step S4 specifically involves: in the target detector model based on the YOLOv5 architecture built in step S35, first loading the model parameters obtained in step S32, and then importing the dataset obtained in step S2 into the model for training to obtain the defect detection model.

[0028] Furthermore, step S5 includes the following steps:

[0029] S51. When deploying the defect detection model obtained in step S4, multiple processes are started on the server based on GPU and memory, and the same defect detection model is started in each process.

[0030] S52. Establish a pipe on the server receiving end for inter-process communication, and number each received image. Distribute the received images equally among multiple processes through the pipe to train the model for detecting rocket adhesion defects.

[0031] S53. Establish a pipeline in the server return section to collect the results from step S52, sort the results according to their numbers, and then return the results.

[0032] Furthermore, the terminal device in step S11 is a shooting device; the non-generative data augmentation method in step S3 includes geometric transformation, color jitter, and random noise.

[0033] Furthermore, in step S32, the initial learning rate is set to 0.1, and the learning rate decay is set with a fixed step size decay strategy, where the learning rate is halved every two rounds, and the momentum range is 0.9; the following loss function and Adam optimizer are used.

[0034]

[0035] In the formula: x represents the sample, y represents the label, a represents the predicted output, and n represents the total number of samples.

[0036] Furthermore, when training the model using the dataset obtained in step S2, the loss function consists of three parts: localization loss, confidence loss, and class loss. The overall loss function is as follows:

[0037]

[0038] Where K represents the feature map output by the network being divided into K×K grids; k, s 2 B represents the output feature map cell and the number of anchors on each cell, respectively; α * The weight of the corresponding item, with a default value of α. box =0.05, α cls =0.3,α obj =0.7; This indicates whether the i-th cell and j-th anchorbox of the k-th output feature map are positive samples; 1 indicates a positive sample, and 0 indicates a negative sample. p ,t gt These represent the prediction vector and the ground-truth vector, respectively. This represents the weights used to balance the output feature maps at each scale;

[0039] The positioning loss is:

[0040] L box =(xx*) 2 +(yy*) 2 +(ww*) 2 +(hh*) 2

[0041] Where x, y, w, h represent the coordinates of the top left corner of the predicted bounding box and the length and width of the box, respectively; and x*, y*, w*, h* represent the coordinates of the top left corner of the ground truth bounding box and the length and width of the box, respectively.

[0042] Confidence loss function:

[0043]

[0044] An anchor box indicating a target; An anchor box without a target; λ noobj This represents the confidence loss weighting coefficient for cases without target anchor boxes. This represents the classification probability of the predicted bounding box. This represents the classification probability of the ground truth bounding box.

[0045] The classification loss function is:

[0046]

[0047]

[0048] x i The current category prediction value is represented by k, where k indicates that the feature map output by the network is divided into K×K grids. This represents the probability of the current class after the activation function. This is the true value (0 or 1) for the current category.

[0049] Compared with the prior art, the present invention has the following advantages:

[0050] 1. The rocket projectile bonding defect detection method based on few-sample deep learning of the present invention obtains a defect feature detection model by learning from a set of labeled defect samples, and identifies defects through the defect feature detection model. Then, incremental learning is performed using the image to be detected to adjust the model parameters, which effectively improves the accuracy of intelligent detection of rocket projectile bonding defects.

[0051] 2. When deploying the model, this invention adopts a multi-process approach, which greatly releases the performance of server multi-core and memory, improves the prediction speed of the model, and can basically meet the requirements of real-time detection.

[0052] 3. This invention optimizes the C3 module in the YOLOv5s network by designing the C3 module as a structure containing dense blocks and transition layers. The dense blocks enhance the feature extraction capability of the network, while the transition layers reduce the number of output channels. Then, the optimized YOLOv5 algorithm is used to identify defect images, which greatly improves the detection speed and accuracy of rocket bonding defects.

[0053] 4. The rocket bonding defect detection method provided by this invention, based on non-generative data augmentation methods such as brightness transformation, contrast transformation, and color transformation of rocket X-ray defect images, designs a deep convolutional generative adversarial network (DCGAN) to expand the defect dataset, which solves the problems of few samples and uneven distribution. It can effectively solve the problems of scarce defect samples and imbalanced defect samples in rocket bonding defect detection, and greatly improve the detection speed and accuracy of rocket bonding defects. Attached Figure Description

[0054] Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation

[0055] The present invention will be further described in detail below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0056] A method for detecting adhesion defects in rocket projectiles using a few samples based on deep learning, such as... Figure 1 As shown, it includes the following steps:

[0057] S1. Acquire images of the rocket's adhesion defects and annotate the images, including the following steps:

[0058] S1.1 Collect images of rocket adhesion defects through a terminal device, wherein the terminal device is a shooting device, including but not limited to a mobile phone or a camera.

[0059] S1.2 Use the MAKE SENSE tool to annotate the images of the rocket's bonding defects in sequence, and record the annotated information in a txt file.

[0060] S2. Organize the labeled data into a dataset, and perform data augmentation on the dataset to complete the first construction of the dataset, including the following steps;

[0061] S2.1. The dataset obtained in step S1 is first augmented with non-generative data augmentation methods to augment small sample defect images. Non-generative data augmentation methods include geometric transformation, color jitter, and random noise.

[0062] S3. Using the dataset constructed in step S2, train a Deep Convolutional Generative Adversarial Network (DCGAN) dataset to expand the network, and further expand the defective dataset.

[0063] S3.1 In step S3.1, the following objective function is used to obtain the generative adversarial network:

[0064]

[0065] In the formula, D is the discriminant network, G is the generative adversarial network, pdta(x) represents the distribution of the real sample x, and p z (z) represents the distribution of random noise z, D(G(z)) represents the probability that the D network judges whether the image generated by G is real; E represents the batch input network;

[0066] S3.2 uses a publicly available image dataset to train a generative adversarial network (DCGAN), fixing the discriminant network D and optimizing the parameters of the generative adversarial network G.

[0067] In this embodiment, the DCGAN network is built using the PyTorch open-source library, based on relevant information. PyTorch is an open-source symbolic mathematics system based on dataflow programming, widely used in the programming implementation of various machine learning and deep learning algorithms.

[0068] S4. Build a target detector model based on the YOLOv5 architecture;

[0069] S4.1 Build a YOLOv5s deep learning model;

[0070] S4.2. Using the publicly available COCO image dataset, pre-train the YOLOv5s deep learning model and save the model's parameters. Set the initial learning rate to 0.1 and configure learning rate decay with a fixed step size, reducing the learning rate to half its original value every two epochs. Set the momentum to 0.9. Use the following loss function and Adam optimizer:

[0071]

[0072] In the formula: x represents the sample, y represents the label, a represents the predicted output, and n represents the total number of samples.

[0073] S4.3 The structure of the trained YOLOv5s deep learning model is modified by introducing a dense network into module C3 of the YOLOv5s deep learning model network. The dense module consists of Conv(1×1)+BN+SiLU+Conv(3×3)+BN+SiLU, where SiLU serves as a non-linear activation function to achieve non-linearity of the network. The 1×1 and 3×3 convolutions can be considered as a combination; the 1×1 convolution reduces the number of input feature maps and improves computational efficiency, while the 3×3 convolutional layer extracts spatial features. The output of the entire module is the stacking of the input and the output of the 3×3 convolutions. Then, a transition layer with a 1×1 kernel is used to reduce the number of channels and decrease model complexity.

[0074] S4.4 To enhance the network's feature fusion capability and reduce computational cost, four max pooling layers with different kernel sizes are used to achieve feature fusion at different scales.

[0075] S4.5 builds the PANet module to achieve feature fusion through sampling operations, and builds the Output module to predict image features, generate bounding boxes and predict categories, thus completing the construction of the object detector model based on the YOLOv5 architecture.

[0076] S5. In the target detector model based on the YOLOv5 architecture built in step S4.5, first load the model parameters obtained in step S32, and then import the dataset obtained in step S3 into the model for training to obtain the defect detection model.

[0077] S6. Deploy the defect detection model using multi-process and multi-threading techniques to improve its detection speed, and then use the defect detection model to detect adhesive defects in rocket projectiles. This includes the following steps:

[0078] S6.1 When deploying the defect detection model obtained in step S5, multiple processes are started on the server based on GPU and memory, and the same defect detection model is started for each process.

[0079] S6.2 Establish a pipe on the server receiving end for inter-process communication, and number each received image. Distribute the received images equally among multiple processes through the pipe to detect rocket adhesion defects.

[0080] S6.3. Establish a pipeline in the server return section to collect the results from step S6.2, sort the results according to their numbers, and then return the results.

[0081] If the model detection efficiency is too slow during implementation, appropriate frame dropping can be used to achieve real-time defect detection without affecting the visual effect.

[0082] S7. The terminal uploads the image to the server, the server performs the detection and sends the result back to the terminal for display.

[0083] To further illustrate the performance of the rocket adhesion defect detection method provided in this invention, the Deep Convolutional Generative Adversarial Network (DCGAN) and the improved YOLOv5 designed in this invention were trained and validated on a rocket adhesion defect dataset. The defect categories included Debonding, Cracking, and Delamination. The performance of the above models was evaluated using mAP (accuracy), precision, and recall as evaluation metrics, and the results are shown in Table 1.

[0084] Table 1. Test results of various performance indicators

[0085]

[0086] As shown in Table 1, the data augmentation using Generative Adversarial Networks (DCGANs) significantly improves the classification model, indicating that DCGANs can effectively address the problem of limited and imbalanced datasets. Furthermore, compared to YOLOv5 with a residual structure, YOLOv5 with a dense structure shows a certain degree of improvement in recognition performance, with mAP increasing by 2.9%, Precision by 1%, and Recall by 4.8%.

[0087] Because 1×1 convolutions can reduce the number of input feature maps and improve computational efficiency, they offer better performance in inference speed. Table 2 shows a comparison of the actual training time of different models on GPU and CPU platforms.

[0088] Table 2 Comparison of Time Consumption

[0089] Yolov5s 42.2 102.6 This invention 36.6 82.1

[0090] As shown in Table 2, the improved network algorithm provided by this invention has a significantly faster inference speed than other models. The inference time is 36.6 ms when using GPU acceleration and 82.1 ms when using only CPU, which is slightly faster than the original model. Combining Tables 1 and 2, it can be seen that the lightweight generative adversarial network proposed in this invention can improve inference speed while maintaining high detection accuracy.

Claims

1. A method for detecting adhesion defects in rocket projectiles using a small sample size based on deep learning, characterized in that, Includes the following steps: S1. Collect and preprocess images of rocket adhesion defects to obtain a defect dataset; S2. Based on the dataset in step S1, design a deep convolutional generative adversarial network to further expand the defect dataset; S3. Build a target detector model based on the YOLOv5 architecture; S4. Construct a defect detection model; S5. Deploy a defect detection model using multi-process and multi-threading techniques to detect rocket adhesion defects. Step S3 specifically involves: S31. Build a YOLOv5s deep learning model; S32. Using the publicly available COCO image dataset, pre-train the YOLOv5s deep learning model and save the parameters of the YOLOv5s deep learning model. S33. Modify the structure of the trained YOLOv5s deep learning model by introducing a dense module into the C3 module of the YOLOv5s deep learning model network. The dense module includes Conv1×1+BN+SiLU+Conv3×3+BN+SiLU, where SiLU is used as a non-linear activation function to achieve non-linearity of the network. The 1×1 convolution and 3×3 convolution can be regarded as a combination. The 1×1 convolution can reduce the number of input feature maps and improve computational efficiency, while the 3×3 convolution layer can extract spatial features. The output of the entire module is the stacking of the input and the output of the 3×3 convolution, and then the number of channels is reduced by a transition layer with a 1×1 convolution kernel, thereby reducing the model complexity. S34. In order to enhance the network's feature fusion capability and reduce computational cost, four max pooling layers with different kernel sizes are used to achieve feature fusion at different scales. S35. Build the PANet module to achieve feature fusion through sampling operations, build the Output module to predict image features, generate bounding boxes and predict categories, and complete the construction of the object detector model based on the YOLOv5 architecture.

2. The method for detecting small-sample rocket adhesion defects based on deep learning according to claim 1, characterized in that, Step S1 specifically involves: S11. Collect images of defects in the adhesion of rocket projectiles through terminal equipment; S12. Use the MAKE SENSE tool to annotate the rocket adhesion defect images collected in step S11, and record the annotated information in a txt file. S13. Perform data augmentation on the dataset obtained in step S12 using a non-generative data augmentation method to expand the small sample defect images.

3. The method for detecting small-sample rocket adhesion defects based on deep learning according to claim 1, characterized in that, Step S2 specifically involves: S21. Obtain the generative adversarial network using the following objective function: In the formula, D is the discriminant network, and G is the generative adversarial network. Represents the distribution of the true sample x. This represents the distribution of random noise z. This represents the probability that the D network determines whether the image generated by G is real; E represents the batch input to the network. S22. Train the generative adversarial network using a publicly available image dataset, fix the discriminant network D, optimize the parameters of the generative adversarial network G, and complete the expansion of the defect dataset.

4. The method for detecting small-sample rocket adhesion defects based on deep learning according to claim 1, characterized in that, Specifically, step S4 involves loading the model parameters obtained in step S32 into the YOLOv5 architecture-based target detector model built in step S35, and then importing the dataset obtained in step S2 into the model for training to obtain the defect detection model.

5. The method for detecting small-sample rocket adhesion defects based on deep learning according to claim 1, characterized in that, Step S5 includes the following steps: S51. When deploying the defect detection model obtained in step S4, multiple processes are started on the server based on GPU and memory, and the same defect detection model is started in each process. S52. Establish a pipe on the server receiving end for inter-process communication, and number each received image. Distribute the received images equally among multiple processes through the pipe to train the model for detecting rocket adhesion defects. S53. Establish a pipeline in the server return section to collect the results from step S52, sort the results according to their numbers, and then return the results.

6. The method for detecting small-sample rocket adhesion defects based on deep learning according to claim 2, characterized in that: The terminal device in step S11 is a shooting device; the non-generative data augmentation method in step S3 includes geometric transformation, color jitter and random noise.

7. The method for detecting small-sample rocket adhesion defects based on deep learning according to claim 1, characterized in that: In step S32, the initial learning rate is set to 0.1, and the learning rate decay is set with a fixed step size decay strategy. The learning rate is halved every two rounds, and the momentum range is 0.

9. The following loss function and Adam optimizer are used. In the formula: Indicates a sample, Indicates a label, This represents the predicted output. This indicates the total sample size.

8. The method for detecting small-sample rocket adhesion defects based on deep learning according to claim 4, characterized in that: When training the model using the dataset obtained in step S2, the loss function consists of three parts: localization loss, confidence loss, and class loss. The overall loss function is as follows: Where K represents the feature map output by the network, which is divided into K×K grids; , , These represent the output feature map, the cell, and the number of anchors on each cell, respectively. , , The weight of the corresponding item, the default value is =0.05, =0.3, =0.7; This indicates whether the k-th output feature map, the i-th cell, and the j-th anchor box are positive samples. If they are positive samples, the value is 1; otherwise, it is 0. These represent the prediction vector and the ground-truth vector, respectively. This represents the weights used to balance the output feature maps at each scale; The positioning loss is: Where x, y, w, h represent the coordinates of the top left corner of the predicted bounding box and the length and width of the box, respectively; and x*, y*, w*, h* represent the coordinates of the top left corner of the ground truth bounding box and the length and width of the box, respectively. Confidence loss function: An anchor box indicating a target; An anchor box indicating no target; This represents the confidence loss weighting coefficient for cases without target anchor boxes. This represents the classification probability of the predicted bounding box. Represents the classification probability of the ground truth bounding box; The classification loss function is: This is the predicted value for the current category. This represents the probability of the current category after the activation function is applied. This is the true value for the current category.

Images