Failure spacecraft part detection method based on improved YOLOv5s
By introducing CBAM and multi-stage image enhancement networks in YOLOv5s, the illuminance and motion state problems in the detection of key components of the failed spacecraft are solved, the detection accuracy and robustness are improved, and efficient detection in complex environments is achieved.
Patent Information
- Application Number
- CN202411139491.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-19
- Publication Date
- 2024-11-29
- Estimated Expiration
- 2044-08-19
AI Technical Summary
The prior art lacks research on key components in the local and global component detection of failed spacecraft, and the detection performance caused by illumination and motion state problems in optical images often leads to missed or missed detection.
CBAM is introduced into the neck network of YOLOv5s, combining data augmentation and multi-stage image enhancement networks to improve the network's attention to the target and image quality, and improve detection accuracy and robustness.
It realizes accurate detection of failed spacecraft components in complex environments, improves mAP and Recall rates, and is especially suitable for image processing of noise and jitter blur in space environments.
Smart Images

Figure CN119048869A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a target detection method, and in particular to a failed spacecraft component detection method based on improved YOLOv5s. Background Art
[0002] The existence of failed spacecraft in orbit not only wastes precious orbital resources, but also potentially threatens space safety. Failed spacecraft may lose control due to natural impacts, accidents, and fuel exhaustion, becoming uncooperative targets. These failed spacecraft may disintegrate at any time, generating debris, posing a safety hazard to space missions and other spacecraft in orbit. Therefore, it is necessary to effectively capture or maintain failed spacecraft.
[0003] The key to the capture mission is to identify the key components of the spacecraft, such as solar panels, radar antennas, etc., while the on-orbit maintenance mission requires the identification of large-scale global components of the spacecraft, such as the spacecraft body and docking surface. Therefore, the detection and identification of local and global components of a failed spacecraft is an important engineering problem.
[0004] Optical imaging technology is suitable for detecting failed spacecraft components during the capture phase due to its advantages such as intuitive target detection, high imaging resolution, and large amount of information. In optical images, the detection of failed spacecraft components belongs to the field of target detection in computer vision. At present, spacecraft component detection methods based on optical images are mainly divided into two categories: traditional target detection methods and target detection methods based on convolutional neural networks (CNNs).
[0005] Traditional target detection methods are mainly based on the fitting of point, line, and circle features. However, different lighting conditions or targets require different fitting parameters, poor adaptability, and complex image preprocessing operations. With the development of deep learning technology, CNN-based target detection methods have achieved good results in the detection of local parts, such as occluded target detection in self-driving cars and material surface defect detection.
[0006] At present, most of the research on spacecraft detection and identification focuses on the detection of the overall outline of the spacecraft, and there are few reports on the detection of local key components (such as solar panels, antennas and radars). In addition, the detection method of failed spacecraft components in optical images mainly faces two problems:
[0007] (1) Illumination problem: Images taken under dark background conditions have low brightness, poor contrast, and many noise points, which seriously affect the image quality and perception, causing the target information to be submerged in the noise and reducing the performance of the detector;
[0008] (2) Motion state problem: Images taken when the spacecraft is in motion are prone to shaking and blurring, which makes it difficult to identify key components and limits the detection of key components of the spacecraft.
[0009] These problems often lead to missed detection or misdetection of critical components of failed spacecraft. Therefore, it is of great significance to study efficient and robust detection methods for spacecraft components. Summary of the invention
[0010] The purpose of the present invention is to solve the problems that most of the current research on spacecraft detection and identification is focused on the detection of the overall contour of the spacecraft, and there is a lack of research on the detection of local key components. In addition, the detection method of failed spacecraft components in optical images often leads to missed detection or false detection of key components of failed spacecraft due to illumination problems and motion state problems. A failed spacecraft component detection method based on improved YOLOv5s is provided.
[0011] In order to solve the deficiencies of the above-mentioned prior art, the present invention provides the following technical solutions:
[0012] A method for detecting failed spacecraft components based on improved YOLOv5s is special in that it includes the following steps:
[0013] Step 1: In YOLOv5s, a CAM (Channel Attention Module) and a SAM (Spatial Attention Module) are connected in sequence between each C3 module and CBS in the neck network, so that the output end of the C3 module is connected to the input end of the CAM, and the output end of the SAM is connected to the input end of the CBS, so as to obtain an initial target detection network; the CAM and SAM connected in sequence constitute a CBAM (Convolutional Block Attention Module);
[0014] Step 2: Use the model of the failed spacecraft to be detected to collect images of the failed spacecraft in a darkroom environment of the laboratory, perform data augmentation on the collected failed spacecraft images, divide them into a training set and a validation set, and annotate the images of the training set and the validation set;
[0015] Step 3: Input the training set and verification set obtained in step 2 into the initial target detection network in step 1 for training to obtain a trained target detection network;
[0016] Step 4: Establish a multi-stage image enhancement network, and use it to enhance the image of the failed spacecraft to be detected in the data set to be detected, so as to obtain the enhanced image of the failed spacecraft to be detected, and constitute the enhanced data set to be detected;
[0017] The multi-stage image enhancement network includes a stage one module, a stage two module and a stage three module, wherein the stage one module includes a stage one Conv (convolution), a stage one CAB (Channel Attention Block), a stage one U-Net and a stage one supervisory attention module connected in sequence; the stage two module includes a stage two Conv, a stage two CAB, a stage two splicing module, a stage two U-Net and a stage two supervisory attention module connected in sequence; the stage three module includes a stage three Conv one, a stage three CAB, a stage three splicing module, a stage three ORSNet (Original Resolution Sensing Network) and a stage three Conv two connected in sequence;
[0018] The input of the stage one Conv is a plurality of sub-images decomposed from the image of the failed spacecraft to be detected, the first output end of the stage one U-Net is connected to the first input end of the stage one supervisory attention module, the second output end is connected to the second input end of the stage two U-Net, the second input end of the stage one supervisory attention module is used to input the ground-truth image, the first output end is used to output the stage one restored image, and the second output end is connected to the stage two splicing module;
[0019] The input of the stage two Conv is the restored image of the stage one; the stage two splicing module is used to splice the output of the stage two CAB with the output of the stage one supervisory attention module channel by channel; the first output end of the stage two U-Net is connected to the first input end of the stage two supervisory attention module, and the second output end is connected to the second input end of the stage three ORSNet. The first output end of the stage two supervisory attention module is used to output the restored image of the stage two, and the second output end is connected to the stage three splicing module;
[0020] The input of the stage three Conv1 is the restored image of the stage two; the stage three splicing module is used to splice the output of the stage three CAB with the output of the stage two supervisory attention module channel by channel, and the stage three ORSNet includes n ORBs (Original Resolution Blocks) connected in sequence, which are used to process the image to retain fine spatial details; the output of the stage three Conv2 is used to add the restored image of the stage two element by element to obtain the restored image of the stage three, which is used as the enhanced image of the failed spacecraft to be detected, constituting the enhanced data set to be detected;
[0021] Step 5: Input the enhanced data set to be detected obtained in step 4 into the target detection network trained in step 3 to obtain the classification and positioning results of the failed spacecraft components, and use mAP (average precision) and Recall (recall rate) for evaluation to complete the detection of failed spacecraft components.
[0022] Further, in step 1, the initial target detection network includes a backbone network (Backbone), a neck network (Neck) and a head network (Head); the neck network includes a first Upsample (upsampling), a first Concat (concatenation), a first C3_2x1, a first CBAM, a first CBS, a second Upsample, a second Concat, a second C3_2x1, a second CBAM, a second CBS, a third Concat, a third C3_2x1, a third CBAM, a third CBS, a fourth Concat, a fourth C3_2x1, a fourth CBAM, a fourth CBS, a fifth Concat, and a fifth C3_2x1 connected in sequence; the output end of the first CBAM is also connected to the input end of the fourth Concat in the neck network.
[0023] Furthermore, the step 2 is specifically as follows:
[0024] Step 2.1, taking the model of the failed spacecraft to be detected as the target, using an LED light source to simulate the incident sunlight, and using a moving vehicle set on a slide rail and an observation camera set at the front end of the moving vehicle to simulate the observation of the spacecraft;
[0025] Step 2.2, make the incident light of sunlight illuminate the target at an incident angle of 0° to 90°, make the observation spacecraft move toward the target at a constant speed within a preset range of the distance from the target in a straight line to simulate the approach phase, and collect a jittered blurred image of the target at the same time; make the incident light of sunlight illuminate the target at an incident angle of 0° to 90°, make the observation spacecraft at a fixed distance from the target to simulate the hovering phase, and collect a noise image of the target at the same time; after the collection is completed, use the jittered blurred image and the noise image as the failed spacecraft image;
[0026] Step 2.3: perform data augmentation on the failed spacecraft images collected in step 2.2 and divide them into a training set and a validation set; then annotate the training set and the validation set, and the annotated targets include the spacecraft solar panels and the spacecraft body.
[0027] Furthermore, in step 2.2, the preset range is 0.5m to 5m.
[0028] Furthermore, the step 4 is specifically as follows:
[0029] Step 4.1, establish a multi-stage image enhancement network;
[0030] Step 4.2, the sub-images obtained by decomposing the image of the failed spacecraft to be detected are sequentially subjected to preliminary convolution processing by the first stage Conv, enhanced feature expression by the first stage CAB, and further restored and enhanced image details by the first stage U-Net. Finally, the supervised learning is performed with the multiple sub-images obtained by decomposing the image of the failed spacecraft to be detected by the first stage supervised attention module to improve the restoration quality, and the restored image of the first stage is output;
[0031] Step 4.3, the restored image of stage 1 is processed by stage 2 Conv for preliminary convolution, stage 2 CAB for enhanced feature expression, stage 2 splicing module and output of stage 1 supervisory attention module are spliced channel by channel, stage 2 U-Net is used to further restore and enhance image details, and finally the image is restored by stage 2 supervisory attention module and output stage 2;
[0032] Step 4.4, the restored image of stage 2 is processed by preliminary convolution of stage 3 Conv 1, enhanced feature expression of stage 3 CAB, channel-by-channel splicing of the output of stage 3 stitching module and stage 2 supervised attention module, and then input into stage 3 ORSNet, and processed by n sequentially connected ORBs to retain and enhance the fine spatial details of the image, and finally convolved by stage 3 Conv 2, and then element-by-element added with the restored image of stage 2 to obtain the restored image of stage 3 as the enhanced failed spacecraft image to be detected, constituting the enhanced data set to be detected, where n is an integer greater than 1.
[0033] Furthermore, the step 3 is specifically as follows:
[0034] Step 3.1, set training hyperparameters, including training cycle (Epoch), learning rate (Learning rate), momentum (Momentum), batch size (Batch size), and training framework;
[0035] Step 3.2: Input the training set obtained in step 2 into the initial target detection network in step 1 to calculate the output prediction result; use the loss function to calculate the training loss between the prediction result and the true annotation; perform gradient backpropagation according to the training loss, and update the network weights and biases through the optimizer;
[0036] Step 3.3: Perform a forward propagation calculation on the validation set obtained in step 2, and calculate the average loss of all validation samples as the validation loss;
[0037] Step 3.4, record the training loss and validation loss, and evaluate the training effect of the model; the evaluation indicators include mAP, Recall and Precision (accuracy);
[0038] Step 3.5: Repeat steps 3.2 to 3.4 until the predetermined training cycle is reached or the training loss and validation loss reach the preset conditions.
[0039] Compared with the prior art, the present invention has the following beneficial effects:
[0040] (1) The present invention discloses a method for detecting failed spacecraft components based on improved YOLOv5s. Firstly, to solve the problems of image noise and motion blur in a complex background of a space environment, CBAM (CAM and SAM) is added to the neck network structure of YOLOv5s to strengthen the important features of the target from the two dimensions of channel and space, enhance the network's attention to the target, and improve the target detection accuracy. Secondly, to solve the problem of lack of unified data for detection in a complex background of a space environment, images of failed spacecraft models are collected in a darkroom environment of a laboratory to establish a training set and a verification set for training an initial target detection network. Finally, a multi-stage image enhancement network is used to process the failed spacecraft images to be detected, enhance the details and contrast of the images, improve the image quality, and improve the accuracy and robustness of target detection. The method is particularly suitable for accurate detection of failed spacecraft components in a complex environment.
[0041] (2) The present invention achieves mAP 77.32% and Recall 85.88%. Compared with the original YOLOv5s, the method proposed in the present invention improves mAP by an average of 4.9%. Multiple groups of experiments show that the present invention combines image restoration and improved YOLOv5s, and has a good detection effect on image noise and jitter blur in spatial environments.
[0042] (3) In the target detection network of the present invention, a CBAM is added between each C3 module and CBS connected in sequence in the neck network of YOLOv5s; that is, an attention mechanism is added between two feature fusions to improve the network's attention to the target; after adding the CBAM, the network first performs feature enhancement operation on the feature map before the next feature fusion, so that the network ignores the interference of irrelevant information, focuses on key features, and fuses relatively important features; this not only makes the fused feature map contain more effective information and improves the target positioning accuracy, but also can achieve the purpose of reducing the amount of calculation and improving the model speed. BRIEF DESCRIPTION OF THE DRAWINGS
[0043] Figure 1 It is a structural schematic diagram of the target detection network in step 1 of an embodiment of a failed spacecraft component detection method based on improved YOLOv5s of the present invention;
[0044] Figure 2 is the noise image 1 collected in step 2.2 of the embodiment of the present invention;
[0045] Figure 3 The second noise image collected in step 2.2 of the embodiment of the present invention;
[0046] Figure 4 The noise image 3 collected in step 2.2 of the embodiment of the present invention;
[0047] Figure 5 The jittered blurred image 1 collected in step 2.2 of the embodiment of the present invention;
[0048] Figure 6 The second jittered blurred image collected in step 2.2 of the embodiment of the present invention;
[0049] Figure 7 The third jittery blurred image collected in step 2.2 of the embodiment of the present invention;
[0050] Figure 8 The failed spacecraft image 1 obtained by data augmentation in step 2.3 of the embodiment of the present invention;
[0051] Fig. 9 The second image of the failed spacecraft obtained by data augmentation in step 2.3 of the embodiment of the present invention;
[0052] Fig.10 The third image of the failed spacecraft obtained by data augmentation in step 2.3 of the embodiment of the present invention;
[0053] Fig.11 Schematic diagram of the structure of the multi-stage image enhancement network in step 4.1 of an embodiment of the present invention.
[0054] The following are the descriptions of the reference numerals:
[0055] 000-input image, 001-recovered image at stage 1, 002-recovered image at stage 2, 003-recovered image at stage 3;
[0056] 101-stage 1 Conv, 102-stage 2 Conv, 1031-stage 3 Conv 1, 1032-stage 3 Conv 2;
[0057] 201-Stage 1 CAB, 202-Stage 2 CAB, 203-Stage 3 CAB;
[0058] 301-stage 1 U-Net, 302-stage 2 U-Net;
[0059] 401-stage one supervisory attention module, 402-stage two supervisory attention module;
[0060] 5-stage three ORSNet, 501-first ORB, 50n-nth ORB. DETAILED DESCRIPTION
[0061] The present invention will be further described below in conjunction with the accompanying drawings and exemplary embodiments.
[0062] A method for detecting failed spacecraft components based on improved YOLOv5s includes the following steps:
[0063] Step 1: In YOLOv5s, a CAM and a SAM are connected in sequence between each C3 module and CBS in the neck network, so that the output of the C3 module is connected to the input of the CAM, and the output of the SAM is connected to the input of the CBS, to obtain the initial target detection network, as shown in Figure 1 As shown;
[0064] YOLOv5s includes backbone network, neck network and head network;
[0065] The backbone network includes Focus, the first CBS, C3_1x1, the second CBS, the first C3_1x3, the third CBS, the second C3_1x3, the fourth CBS, SPP (Spatial Pyramid Pooling), C3_2x1 and the fifth CBS, which are connected in sequence. Focus is used to decompose the processed image into multiple parts by channel and splice them in the depth dimension; CBS contains Conv, Batch Normalization (BN) and activation function (SiLU) for feature extraction and normalization; C3_1x1 represents a C3 module with 1 Bottle neck network layer and repeated 1 time, C3_1x3 represents a C3 module with 1 Bottl e neck network layer and repeated 3 times, and C3_2x1 represents a C3 module with 2 Bottle neck network layers and repeated 1 time; C3 module is used to extract deep features; SPP is used to perform maximum pooling operations at different scales and then splice the results;
[0066] The neck network includes the first Upsample, the first Concat, the first C3_2x1, the first CBS, the second Upsample, the second Concat, the second C3_2x1, the second CBS, the third Concat, the third C3_2x1, the third CBS, the fourth Concat, the fourth C3_2x1, the fourth CBS, the fifth Concat, and the fifth C3_2x1, which are connected in sequence;
[0067] The output of C3_1x1 in the backbone network is also connected to the input of the third Concat in the neck network, the output of the first C3_1x3 in the backbone network is also connected to the input of the second Concat in the neck network, the output of the second C3_1x3 in the backbone network is also connected to the input of the first Concat in the neck network, and the output of the fifth CBS in the backbone network is respectively connected to the input of the first Upsample and the fifth Concat in the neck network; Upsample in the neck network is used to increase the low-resolution feature map to the same size as the higher-resolution feature map, Concat is used to splice feature maps from different scales in the channel dimension, C3_2x1 is used to further extract features from the spliced feature map, and CBS is used to perform convolution operations on the input feature map to extract features;
[0068] The head network includes three YOLO layers in parallel, which receive three feature maps of different resolutions from the neck network respectively. Each YOLO layer contains Conv, which is used to process the input feature map of the corresponding resolution and output the feature map of the corresponding resolution; the input end of the high-resolution YOLO layer is connected to the output end of the third C3_2x1 in the neck network, the input end of the medium-resolution YOLO layer is connected to the output end of the fourth C3_2x1 in the neck network, and the input end of the low-resolution YOLO layer is connected to the output end of the fifth C3_2x1 in the neck network;
[0069] Reference Figure 1 In this embodiment, in the neck network of YOLOv5s, each C3 module and CBS connected in sequence are provided, specifically, between the first C3_2x1 and the first CBS, the second C3_2x1 and the second CBS, the third C3_2x1 and the third CBS, and the fourth C3_2x1 and the fourth CBS, CAM and SAM connected in sequence are provided, and the CAM and SAM connected in sequence constitute CBAM, and the output end of the CBAM between the first C3_2x1 and the first CBS is also connected to the input end of the fourth Concat in the neck network;
[0070] CBAM is used to enhance important features by focusing on different channels of the input feature map. The specific steps are as follows:
[0071] Step A1, perform global average pooling and global maximum pooling on the input feature map respectively to obtain two pooled feature maps of size C×1×1;
[0072] Step A2: The two pooled feature maps are respectively passed through a two-layer fully connected network (MLP) with shared weights to obtain two feature maps of size C×1×1;
[0073] Step A3: Add the two feature maps element-wise to obtain a fused feature map of size C×1×1;
[0074] Step A4: Calculate the channel attention map based on the fused feature map using the Sigmoid function;
[0075] Step A5: multiply the channel attention map by the input feature map element-wise to obtain an enhanced feature map;
[0076] The SAM enhances important features by focusing on different spatial locations of the input feature map. The specific steps are as follows:
[0077] Step B1, performing global average pooling and global maximum pooling on the enhanced feature map obtained in step A5, respectively, to obtain two pooled feature maps of size 1×H×W;
[0078] Step B2: concatenate the two pooled feature maps in the channel dimension to obtain a concatenated feature map of size 2×H×W;
[0079] Step B3: Perform a convolution operation on the concatenated feature map to obtain a convolution feature map of size 1×H×W;
[0080] Step B4, calculate the spatial attention map according to the convolution feature map through the Sigmoid function;
[0081] Step B5: multiply the spatial attention map and the enhanced feature map element-wise to obtain the final enhanced feature map;
[0082] Step 2: Use the model of the failed spacecraft to be detected to collect images of the failed spacecraft in a darkroom environment of the laboratory, perform data augmentation on the collected failed spacecraft images, divide them into a training set and a validation set, and annotate the images of the training set and the validation set;
[0083] Step 2.1, taking the model of the failed spacecraft to be detected as the target, using an LED light source to simulate the incident sunlight, and using a moving vehicle set on a slide rail and an observation camera set at the front end of the moving vehicle to simulate the observation of the spacecraft;
[0084] Step 2.2: Let the incident sunlight illuminate the target at an incident angle of 0° to 90°, and let the observation spacecraft move toward the target at a constant speed (0.5 m / s) within a preset range (0.5 m to 5 m) from the target in a straight line to simulate the approach phase, and at the same time collect a jittery and blurred image of the target, such as Figures 5 to 7 As shown; the incident sunlight is made to illuminate the target at an incident angle of 0° to 90°, and the observation spacecraft is made to simulate the hovering stage at a fixed distance (1m) from the target, and the noise image of the target is collected at the same time, as shown Figures 2 to 4As shown; after the acquisition is completed, the jitter blur image and the noise image are used as the failed spacecraft image;
[0085] The brightness, radiation area and radiation beam size of the LED light source can be manually adjusted to better simulate the collimation, uniformity and spectral characteristics of solar radiation;
[0086] Step 2.3: Perform data augmentation on the failed spacecraft image collected in step 2.2 (e.g. Figures 8 to 10 As shown in the figure, the training set and the validation set are divided into a training set and a validation set. Then, the Label lmg software is used to annotate the training set and the validation set according to the COCO dataset format. The annotated targets include the solar panels of the spacecraft and the spacecraft body. The sample number ratio of the training set and the validation set is 9:1.
[0087] Step 3: Input the training set and verification set obtained in step 2 into the initial target detection network in step 1 for training to obtain a trained target detection network;
[0088] Step 3.1, set the training hyper parameters as follows:
[0089] The training epochs are set to 300, the learning rate is set to 0.01, the momentum is set to 0.937, and the batch size is set to 16; the basic training framework is Pytorch, and the hardware is NVIDIA GeForce GTX 1650Ti;
[0090] Step 3.2: Input the training set obtained in step 2 into the initial target detection network in step 1 to calculate the output prediction result; use the loss function to calculate the training loss between the prediction result and the true annotation; perform gradient backpropagation according to the training loss, and update the network weights and biases through the optimizer;
[0091] Step 3.3: Perform a forward propagation calculation on the validation set obtained in step 2, and calculate the average loss of all validation samples as the validation loss;
[0092] Step 3.4, record the training loss and validation loss, and evaluate the training effect of the model; the evaluation indicators include mAP, Recall and Precision;
[0093] Step 3.5: Repeat steps 3.2 to 3.4 until the predetermined training cycle is reached, or the verification loss and severe loss meet the preset conditions;
[0094] Step 4: Establish a multi-stage image enhancement network, and use it to enhance the image of the failed spacecraft to be detected in the data set to be detected, so as to obtain the enhanced image of the failed spacecraft to be detected, and constitute the enhanced data set to be detected;
[0095] Step 4.1, reference Fig.11 , establish a multi-stage image enhancement network, which includes a stage one module, a stage two module and a stage three module, the stage one module includes a stage one Conv 101, a stage one CAB 201, a stage one U-Net 301 and a stage one supervisory attention module 401 connected in sequence; the stage two module includes a stage two Conv 102, a stage two CAB 202, a stage two splicing module, a stage two U-Net 302 and a stage two supervisory attention module 402 connected in sequence; the stage three module includes a stage three Conv one 1031, a stage three CAB 203, a stage three splicing module, a stage three ORSNet5 and a stage three Conv two 1032 connected in sequence;
[0096] The input image 000 of the stage 1 Conv 101 is a plurality of sub-images decomposed from the image of the failed spacecraft to be detected, the first output end of the stage 1 U-Net 301 is connected to the first input end of the stage 1 supervisory attention module 401, the second output end is connected to the second input end of the stage 2 U-Net 302, the second input end of the stage 1 supervisory attention module 401 is used to input the ground-truth image (input image 000), the first output end is used to output the stage 1 restored image 001, and the second output end is connected to the stage 2 splicing module;
[0097] The input of the stage two Conv 102 is the stage one restored image 001; the stage two splicing module is used to splice the output of the stage two CAB 202 with the output of the stage one supervisory attention module 401 channel by channel; the first output end of the stage two U-Net 302 is connected to the first input end of the stage two supervisory attention module 402, and the second output end is connected to the second input end of the stage three ORSNet 5, the first output end of the stage two supervisory attention module 402 is used to output the stage two restored image 002, and the second output end is connected to the stage three splicing module;
[0098] The input of the stage three Conv1 1031 is the stage two restored image 002; the stage three splicing module is used to splice the output of the stage three CAB 203 with the output of the stage two supervisory attention module 402 channel by channel, the first input end of the stage three ORSNet5 is connected to the output end of the stage three splicing module, the stage three ORSNet5 includes n sequentially connected ORBs, namely the first ORB 501, ..., the nth ORB 50n, for processing images to retain fine spatial details; the output of the stage three Conv2 1032 is used to add element by element with the stage two restored image 002 to obtain the stage three restored image 003 as the enhanced failed spacecraft image to be detected, constituting the enhanced data set to be detected, and n is an integer greater than 1;
[0099] Step 4.2: Initial recovery and enhancement
[0100] A plurality of sub-images decomposed from the image of the failed spacecraft to be detected are used as the input image 000, and are sequentially subjected to preliminary convolution processing by the stage 1 Conv 101, feature expression is enhanced by the stage 1 CAB 201, and image details are further restored and enhanced by the stage 1 U-Net 301. Finally, supervised learning is performed by the stage 1 supervised attention module 401 and the plurality of sub-images decomposed from the image of the failed spacecraft to be detected, so as to improve the restoration quality, and the stage 1 restored image 001 is output;
[0101] Step 4.3: Further refine the recovery
[0102] The restored image 001 of stage one is sequentially processed by Conv 102 of stage two for preliminary convolution, CAB202 of stage two for enhanced feature expression, the output of the splicing module of stage two and the supervisory attention module 401 of stage one are spliced channel by channel, U-Net 302 of stage two is further restored and enhanced for image details, and finally the image 002 of stage two is restored by the supervisory attention module 402 of stage two and the output of stage two;
[0103] Step 4.4: Fine restoration and enhancement
[0104] The restored image 002 of stage 2 is sequentially processed by preliminary convolution through stage 3 Conv 1 1031, feature expression is enhanced through stage 3 CAB 203, and the output of stage 3 splicing module and stage 2 supervisory attention module 402 are spliced channel by channel, and then input into stage 3 ORSNet 5, and the image is processed through n sequentially connected ORBs to retain and enhance the fine spatial details of the image, and finally convolution operation is performed through stage 3 Conv 2 1032, and then element-by-element addition is made to the restored image 002 of stage 2 to obtain the restored image 003 of stage 3;
[0105] Step 5: Input the enhanced data set to be detected obtained in step 4 into the target detection network trained in step 3 to obtain the classification and positioning results of the failed spacecraft components, and use mAP and regression rate for evaluation to complete the detection of failed spacecraft components.
[0106] In order to evaluate the effect of the embodiments of the present invention, comparative examples 1 and 2 are set for comparison. Comparative example 1 does not enhance the image of the failed spacecraft to be detected, and uses the original YOLOv5s to detect failed spacecraft components. Comparative example 2 enhances the image of the failed spacecraft to be detected, and uses the original YOLOv5s to detect failed spacecraft components, as shown in Table 1.
[0107] Table 1
[0108]
[0109] Experimental results show that the embodiments of the present invention can effectively improve the detection accuracy, and the mAP of the target detection network is improved by 4.9% compared with the original YOLOv5s. The reason is that the addition of the attention mechanism can suppress the interference of irrelevant information, retain more key features of the failed spacecraft to be detected, and enable the network to focus on more targets, thereby improving the detection accuracy. Compared with no image enhancement, the mAP of image enhancement performed by the multi-stage image enhancement network is improved by 1.4%. The reason is that image enhancement significantly improves the feature expression of the data set to be detected, and solves the problem of difficult target recognition in space environment with image noise and jitter blur.
Claims
1. A method for detecting failed spacecraft components based on improved YOLOv5s, characterized in that: The steps include: Step 1, in the neck network of YOLOv5s, a CAM and a SAM connected in sequence are set between each C3 module connected in sequence and the CBS, so that the output end of the C3 module is connected to the input end of the CAM, and the output end of the SAM is connected to the input end of the CBS, so as to obtain an initial target detection network; the CAM and SAM connected in sequence constitute CBAM; Step 2: Use the model of the failed spacecraft to be detected to collect images of the failed spacecraft in a darkroom environment of the laboratory, perform data augmentation on the collected failed spacecraft images, divide them into a training set and a validation set, and annotate the images of the training set and the validation set; Step 3: Input the training set and verification set obtained in step 2 into the initial target detection network in step 1 for training to obtain a trained target detection network; Step 4: Establish a multi-stage image enhancement network, and use it to enhance the image of the failed spacecraft to be detected in the data set to be detected, so as to obtain the enhanced image of the failed spacecraft to be detected, and constitute the enhanced data set to be detected; The multi-stage image enhancement network comprises a stage one module, a stage two module and a stage three module, wherein the stage one module comprises a stage one Conv (101), a stage one CAB (201), a stage one U-Net (301) and a stage one supervisory attention module (401) connected in sequence; the stage two module comprises a stage two Conv (102), a stage two CAB (202), a stage two splicing module, a stage two U-Net (302) and a stage two supervisory attention module (402) connected in sequence; the stage three module comprises a stage three Conv one (1031), a stage three CAB (203), a stage three splicing module, a stage three ORSNet (5) and a stage three Conv two (1032) connected in sequence; The input of the stage one Conv (101) is a plurality of sub-images decomposed from the image of the failed spacecraft to be detected, the first output end of the stage one U-Net (301) is connected to the first input end of the stage one supervisory attention module (401), the second output end is connected to the second input end of the stage two U-Net (302), the second input end of the stage one supervisory attention module (401) is used to input a ground-truth image, the first output end is used to output a stage one restored image (001), and the second output end is connected to the stage two splicing module; The input of the stage two Conv (102) is the stage one restored image (001); the stage two splicing module is used to splice the output of the stage two CAB (202) and the output of the stage one supervisory attention module (401) channel by channel; the first output end of the stage two U-Net (302) is connected to the first input end of the stage two supervisory attention module (402), and the second output end is connected to the second input end of the stage three ORSNet (5); the first output end of the stage two supervisory attention module (402) is used to output the stage two restored image (002), and the second output end is connected to the stage three splicing module; The input of the stage three Conv one (1031) is the stage two restored image (002); the stage three splicing module is used to splice the output of the stage three CAB (203) with the output of the stage two supervisory attention module (402) channel by channel, and the stage three ORSNet (5) includes n sequentially connected ORBs, which are used to process the image to retain fine spatial details; the output of the stage three Conv two (1032) is used to add the stage two restored image (002) element by element to obtain the stage three restored image (003), which is used as the enhanced failed spacecraft image to be detected, forming the enhanced data set to be detected; Step 5: Input the enhanced data set to be detected obtained in step 4 into the target detection network trained in step 3 to obtain the classification and positioning results of the failed spacecraft components, and use mAP and regression rate for evaluation to complete the detection of failed spacecraft components.
2. The method for detecting failed spacecraft components based on improved YOLOv5s according to claim 1, characterized in that: In step 1, the initial target detection network includes a backbone network, a neck network and a head network; the neck network includes a first Upsample, a first Concat, a first C3_2x1, a first CBAM, a first CBS, a second Upsample, a second Concat, a second C3_2x1, a second CBAM, a second CBS, a third Concat, a third C3_2x1, a third CBAM, a third CBS, a fourth Concat, a fourth C3_2x1, a fourth CBAM, a fourth CBS, a fifth Concat, and a fifth C3_2x1 connected in sequence; the output end of the first CBAM is also connected to the input end of the fourth Concat in the neck network.
3. The method for detecting failed spacecraft components based on improved YOLOv5s according to claim 2, characterized in that: The step 2 is specifically as follows: Step 2.1, taking the model of the failed spacecraft to be detected as the target, using an LED light source to simulate the incident sunlight, and using a moving vehicle set on a slide rail and an observation camera set at the front end of the moving vehicle to simulate the observation of the spacecraft; Step 2.2, making the incident sunlight illuminate the target at an incident angle of 0° to 90°, making the observation spacecraft move toward the target at a constant speed along a straight line within a preset range of the distance to the target to simulate the approach phase, and at the same time collecting a jittered blurred image of the target; The incident sunlight is made to illuminate the target at an incident angle of 0° to 90°, and the observation spacecraft is made to simulate a hovering stage at a fixed distance from the target, while collecting a noise image of the target; After the acquisition is completed, the jittered blurred image and the noisy image are regarded as the failed spacecraft image; Step 2.3: perform data augmentation on the failed spacecraft images collected in step 2.2 and divide them into a training set and a validation set; then annotate the training set and the validation set, and the annotated targets include the spacecraft solar panels and the spacecraft body.
4. The method for detecting failed spacecraft components based on improved YOLOv5s according to claim 3, characterized in that: In step 2.2, the preset range is 0.5m to 5m.
5. A method for detecting failed spacecraft components based on improved YOLOv5s according to any one of claims 1 to 4, characterized in that: The step 4 is specifically as follows: Step 4.1, establish a multi-stage image enhancement network; Step 4.2, the sub-images obtained by decomposing the image of the failed spacecraft to be detected are sequentially subjected to preliminary convolution processing by the stage one Conv (101), feature expression enhancement by the stage one CAB (201), and further restoration and enhancement of image details by the stage one U-Net (301), and finally supervised learning is performed by the stage one supervised attention module (401) and the sub-images obtained by decomposing the image of the failed spacecraft to be detected to improve the restoration quality, and the stage one restoration image (001) is output; Step 4.3, the restored image (001) of stage one is sequentially processed by the stage two Conv (102) for preliminary convolution, the stage two CAB (202) for enhanced feature expression, the stage two splicing module and the output of the stage one supervisory attention module (401) are spliced channel by channel, the stage two U-Net (302) further restores and enhances image details, and finally the stage two supervisory attention module (402) and the output stage two restored image (002); Step 4.4, the restored image (002) of stage 2 is sequentially subjected to preliminary convolution processing by stage 3 Conv 1 (1031), feature expression enhancement by stage 3 CAB (203), channel-by-channel splicing of the outputs of the stage 3 splicing module and the stage 2 supervisory attention module (402), and then input into stage 3 ORSNet (5), and the image is processed by n sequentially connected ORBs to retain and enhance the fine spatial details of the image, and finally convolution operation is performed by stage 3 Conv 2 (1032), and then element-by-element addition is made to the restored image (002) of stage 2 to obtain the restored image (003) of stage 3 as the enhanced image of the failed spacecraft to be detected, constituting the enhanced data set to be detected, where n is an integer greater than 1.
6. The method for detecting failed spacecraft components based on improved YOLOv5s according to claim 5, characterized in that: The step 3 is specifically as follows: Step 3.1, set training hyperparameters, including training cycle, learning rate, momentum, batch size, and training framework; Step 3.2: Input the training set obtained in step 2 into the initial target detection network in step 1 to calculate the output prediction result; use the loss function to calculate the training loss between the prediction result and the true annotation; perform gradient backpropagation according to the training loss, and update the network weights and biases through the optimizer; Step 3.3: Perform a forward propagation calculation on the validation set obtained in step 2, and calculate the average loss of all validation samples as the validation loss; Step 3.4, record the training loss and validation loss, and evaluate the training effect of the model; the evaluation indicators include mAP, Recall and Precision; Step 3.5: Repeat steps 3.2 to 3.4 until the predetermined training cycle is reached or the training loss and validation loss reach the preset conditions.
Citation Information
Patent Citations
Power transmission line part identification method based on improved YOLOv5
CN114821278A
Small sample chip appearance defect detection method and detection system based on improved YOLOv5
CN116228740A
PCB defect detection method based on improved YOLO v5 algorithm
CN117351288A
X-ray security check image forbidden article detection method based on improved YOLOv7
CN118397303A
Cited By
Failure spacecraft component detection network and method based on YOLOv8 network
CN120070965A
A YOLOv8 network-based failed spacecraft component detection network and method
CN120070965B
