A method for detecting failed space vehicle components based on improved YOLOv5

Abstract

The application relates to a target detection method, in particular to a failed spacecraft component detection method based on an improved YOLOv5s, which is used to solve the problems that the current research on spacecraft detection and identification is mostly concentrated on the detection of the overall contour of the spacecraft, the detection of local key components is lacked, and the detection method of failed spacecraft components in optical images often leads to the missing detection or false detection of the key components of the failed spacecraft due to the illumination problem and the motion state problem. The failed spacecraft component detection method based on the improved YOLOv5s adds CBAM in the neck network of YOLOv5s; secondly, in the darkroom environment of the laboratory, the images of the failed spacecraft model are collected to establish a training set and a verification set for training the initial target detection network; finally, a multi-stage image enhancement network is used to process the failed spacecraft images to be detected, so that the details and contrast of the images are enhanced, and the quality of the images is improved.

Description

Technical Field

[0001] The present invention relates to a target detection method, and in particular to a failed spacecraft component detection method based on improved YOLOv5s. Background Art

[0002] The presence of disabled spacecraft in orbit not only wastes valuable orbital resources but also poses a potential threat to space safety. Disabled spacecraft may lose control due to factors such as natural impacts, accidents, and fuel depletion, becoming unacceptable targets. These disabled spacecraft could disintegrate at any time, creating debris that could pose a safety risk to space missions and other spacecraft in orbit. Therefore, effective capture or maintenance of disabled spacecraft is essential.

[0003] The key to capture missions lies in identifying key spacecraft components, such as solar panels and radar antennas, while on-orbit maintenance missions require identifying large-scale, global components, such as the spacecraft body and docking surface. Therefore, the detection and identification of both local and global components of a failed spacecraft is a significant engineering problem.

[0004] Optical imaging technology, due to its intuitive target detection, high imaging resolution, and extensive information collection, is well-suited for detecting failed spacecraft components during the capture phase. Detecting failed spacecraft components in optical images falls within the field of object detection within computer vision. Currently, methods for detecting spacecraft components based on optical images fall into two main categories: traditional object detection methods and those based on convolutional neural networks (CNNs).

[0005] Traditional object detection methods are primarily based on fitting point, line, and circle features. However, different lighting conditions and targets require different fitting parameters, resulting in poor adaptability and requiring complex image preprocessing. With the advancement of deep learning technology, CNN-based object detection methods have achieved promising results in detecting local components, such as occluded objects in autonomous vehicles and detecting surface defects in materials.

[0006] At present, most of the research on spacecraft detection and identification focuses on the detection of the overall spacecraft outline, and there are few reports on the detection of local key components (such as solar panels, antennas and radars). In addition, the detection method of failed spacecraft components in optical images mainly faces two problems:

[0007] (1) Illumination problem: Images captured under dark background conditions have low brightness, poor contrast, and high noise, which seriously affects image quality and perception, causing target information to be submerged in noise and reducing detector performance;

[0008] (2) Motion state problem: Images taken when the spacecraft is in motion are prone to shaking and blurring, which makes it difficult to identify key components and limits the detection of key components of the spacecraft.

[0009] These problems often lead to missed detection or false detection of critical components of failed spacecraft. Therefore, it is of great significance to study efficient and robust spacecraft component detection methods. Summary of the Invention

[0010] The purpose of the present invention is to solve the shortcomings that the current research on spacecraft detection and identification is mostly focused on the detection of the overall contour of the spacecraft, and there is a lack of research on the detection of local key components. In addition, the detection method of failed spacecraft components in optical images often leads to missed detection or false detection of failed spacecraft key components due to illumination problems and motion state problems. A failed spacecraft component detection method based on improved YOLOv5s is provided.

[0011] In order to solve the deficiencies of the above-mentioned prior art, the present invention provides the following technical solutions:

[0012] A method for detecting failed spacecraft components based on improved YOLOv5s is special in that it includes the following steps:

[0013] Step 1: In the YOLOv5s neck network, a CAM (Channel Attention Module) and a SAM (Spatial Attention Module) are connected in sequence between each C3 module and CBS. The output of the C3 module is connected to the input of the CAM, and the output of the SAM is connected to the input of the CBS to obtain an initial target detection network. The sequentially connected CAM and SAM constitute a CBAM (Convolutional Block Attention Module).

[0014] Step 2: Use the model of the failed spacecraft to be detected to collect images of the failed spacecraft in a darkroom environment in the laboratory, perform data augmentation on the collected failed spacecraft images, divide them into a training set and a validation set, and annotate the images in the training set and validation set;

[0015] Step 3: Input the training set and validation set obtained in step 2 into the initial target detection network in step 1 for training to obtain a trained target detection network.

[0016] Step 4: Establish a multi-stage image enhancement network, and use it to perform image enhancement on the failed spacecraft to be detected in the detection data set to obtain enhanced failed spacecraft to be detected images, which constitute the enhanced detection data set;

[0017] The multi-stage image enhancement network includes a stage one module, a stage two module and a stage three module, wherein the stage one module includes a sequentially connected stage one Conv (convolution), a stage one CAB (Channel Attention Block), a stage one U-Net and a stage one supervisory attention module; the stage two module includes a sequentially connected stage two Conv, a stage two CAB, a stage two splicing module, a stage two U-Net and a stage two supervisory attention module; the stage three module includes a sequentially connected stage three Conv one, a stage three CAB, a stage three splicing module, a stage three ORSNet (Original Resolution Sensing Network) and a stage three Conv two;

[0018] The input of the stage one Conv is a plurality of sub-images obtained by decomposing the image of the failed spacecraft to be detected. The first output end of the stage one U-Net is connected to the first input end of the stage one supervisory attention module, and the second output end is connected to the second input end of the stage two U-Net. The second input end of the stage one supervisory attention module is used to input the ground-truth image, the first output end is used to output the stage one restored image, and the second output end is connected to the stage two splicing module.

[0019] The input of the stage two Conv is the restored image of the stage one; the stage two splicing module is used to splice the output of the stage two CAB with the output of the stage one supervisory attention module channel by channel; the first output end of the stage two U-Net is connected to the first input end of the stage two supervisory attention module, and the second output end is connected to the second input end of the stage three ORSNet. The first output end of the stage two supervisory attention module is used to output the restored image of the stage two, and the second output end is connected to the stage three splicing module;

[0020] The input of the stage 3 Conv1 is the restored image of the stage 2; the stage 3 splicing module is used to splice the output of the stage 3 CAB with the output of the stage 2 supervised attention module channel by channel. The stage 3 ORSNet includes n sequentially connected ORBs (Original Resolution Blocks) for processing images to retain fine spatial details; the output of the stage 3 Conv2 is used to add element-by-element to the stage 2 restored image to obtain the stage 3 restored image, which is used as the enhanced image of the failed spacecraft to be detected, forming the enhanced dataset to be detected;

[0021] Step 5: Input the enhanced dataset to be detected obtained in step 4 into the target detection network trained in step 3 to obtain the classification and positioning results of the failed spacecraft components. Use mAP (average precision) and Recall (recall rate) for evaluation to complete the detection of failed spacecraft components.

[0022] Furthermore, in step 1, the initial target detection network includes a backbone network (Backbone), a neck network (Neck) and a head network (Head); the neck network includes the first Upsample (upsampling), the first Concat (splicing), the first C3_2x1, the first CBAM, the first CBS, the second Upsample, the second Concat, the second C3_2x1, the second CBAM, the second CBS, the third Concat, the third C3_2x1, the third CBAM, the third CBS, the fourth Concat, the fourth C3_2x1, the fourth CBAM, the fourth CBS, the fifth Concat, and the fifth C3_2x1; the output end of the first CBAM is also connected to the input end of the fourth Concat in the neck network.

[0023] Furthermore, the step 2 is specifically as follows:

[0024] Step 2.1: Use the model of the failed spacecraft to be inspected as the target, use an LED light source to simulate incident sunlight, and use a mobile vehicle set on a slide rail and an observation camera set at the front end of the mobile vehicle to simulate observing the spacecraft;

[0025] Step 2.2: Sunlight is incident on the target at an angle of incidence of 0° to 90°, and the observation spacecraft is moved toward the target at a constant speed along a straight line within a preset range of distance from the target to simulate an approach phase, while simultaneously collecting a jittered and blurred image of the target; Sunlight is incident on the target at an angle of incidence of 0° to 90°, and the observation spacecraft is positioned at a fixed distance from the target to simulate a hovering phase, while simultaneously collecting a noise image of the target; after the acquisition is completed, the jittered and blurred image and the noise image are used as the failed spacecraft image;

[0026] Step 2.3: Perform data augmentation on the failed spacecraft images collected in step 2.2 and divide them into a training set and a validation set. Then, perform image annotation on the training set and the validation set, with the annotated targets including the spacecraft solar panels and the spacecraft itself.

[0027] Furthermore, in step 2.2, the preset range is 0.5m to 5m.

[0028] Furthermore, the step 4 is specifically as follows:

[0029] Step 4.1: Establish a multi-stage image enhancement network;

[0030] Step 4.2: The sub-images obtained by decomposing the image of the failed spacecraft to be detected are sequentially processed through the first stage Conv for preliminary convolution, the first stage CAB for feature expression enhancement, and the first stage U-Net for further restoration and enhancement of image details. Finally, the supervised learning is performed on the sub-images obtained by decomposing the image of the failed spacecraft to be detected by the first stage through the first stage supervised attention module to improve the restoration quality, and the restored image of the first stage is output.

[0031] Step 4.3: The restored image in stage 1 is processed sequentially through the Conv in stage 2 for preliminary convolution, the CAB in stage 2 for feature enhancement, the output of the splicing module in stage 2 is spliced ​​channel by channel with the output of the supervised attention module in stage 1, the U-Net in stage 2 is further restored and enhanced, and finally the restored image is processed through the supervised attention module in stage 2 and the output of stage 2.

[0032] Step 4.4: The restored image in stage 2 is processed sequentially through the initial convolution operation of stage 3 Conv1, the feature expression is enhanced by CAB in stage 3, and the output of the splicing module in stage 3 is spliced ​​channel by channel with the output of the supervised attention module in stage 2. Then, the image is input into the ORSNet in stage 3. The image is processed through n sequentially connected ORBs to retain and enhance the fine spatial details of the image. Finally, the image is convolved through stage 3 Conv2 and then element-wise added to the restored image in stage 2 to obtain the restored image in stage 3. This image is used as the enhanced image of the failed spacecraft to be detected, forming the enhanced dataset to be detected, where n is an integer greater than 1.

[0033] Furthermore, the step 3 is specifically as follows:

[0034] Step 3.1. Set training hyperparameters, including epoch, learning rate, momentum, batch size, and training framework.

[0035] Step 3.2: Input the training set obtained in step 2 into the object detection network initialized in step 1 to calculate the output prediction results; use the loss function to calculate the training loss between the prediction results and the true annotations; perform gradient backpropagation based on the training loss, and update the network weights and biases through the optimizer;

[0036] Step 3.3: Perform a forward propagation calculation on the validation set obtained in step 2, and calculate the average loss of all validation samples as the validation loss;

[0037] Step 3.4: Record the training loss and validation loss to evaluate the training effect of the model; the evaluation indicators include mAP, Recall and Precision (accuracy);

[0038] Step 3.5: Repeat steps 3.2 to 3.4 until the predetermined training cycle is reached or the training loss and validation loss meet the preset conditions.

[0039] Compared with the prior art, the present invention has the following beneficial effects:

[0040] (1) The present invention provides a method for detecting failed spacecraft components based on improved YOLOv5s. First, to address the problems of image noise and motion blur in a complex space environment, CBAM (CAM and SAM) is added to the neck network structure of YOLOv5s to strengthen the important features of the target from the two dimensions of channel and space, enhance the network's attention to the target, and improve the target detection accuracy; secondly, to address the lack of unified data for detection in a complex space environment, images of failed spacecraft models are collected in a darkroom environment of the laboratory to establish a training set and a verification set for training the initial target detection network; finally, a multi-stage image enhancement network is used to process the failed spacecraft images to enhance the details and contrast of the images, improve the image quality, and improve the accuracy and robustness of target detection. The method is particularly suitable for the accurate detection of failed spacecraft components in complex environments.

[0041] (2) The present invention achieved mAP 77.32% and Recall 85.88%. Compared with the original YOLOv5s, the method proposed in the present invention improved mAP by an average of 4.9%. Multiple experiments showed that the present invention, which combines image restoration and improved YOLOv5s, has a good detection effect on image noise and jitter blur in spatial environments.

[0042] (3) The target detection network in the present invention adds a CBAM between each C3 module and CBS connected in sequence in the neck network of YOLOv5s; that is, an attention mechanism is added between two feature fusions to improve the network's attention to the target; after adding the CBAM, the network first performs a feature enhancement operation on the feature map before the next feature fusion, so that the network ignores the interference of irrelevant information, focuses on key features, and fuses relatively important features; this not only makes the fused feature map contain more effective information and improves the target positioning accuracy, but also can achieve the purpose of reducing the amount of calculation and improving the model speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 This is a schematic diagram of the structure of the target detection network in step 1 of an embodiment of a failed spacecraft component detection method based on improved YOLOv5s of the present invention;

[0044] Figure 2 The noise image 1 collected in step 2.2 of the embodiment of the present invention;

[0045] Figure 3 The second noise image collected in step 2.2 of the embodiment of the present invention;

[0046] Figure 4 The third noise image collected in step 2.2 of the embodiment of the present invention;

[0047] Figure 5 The first jittered blurred image collected in step 2.2 of the embodiment of the present invention;

[0048] Figure 6 The second jittered blurred image collected in step 2.2 of the embodiment of the present invention;

[0049] Figure 7 The third jittered blurred image collected in step 2.2 of the embodiment of the present invention;

[0050] Figure 8 The first image of the failed spacecraft obtained by data augmentation in step 2.3 of the embodiment of the present invention;

[0051] Figure 9 The second image of the failed spacecraft obtained by data augmentation in step 2.3 of the embodiment of the present invention;

[0052] Figure 10 The third image of the failed spacecraft obtained by data augmentation in step 2.3 of the embodiment of the present invention;

[0053] Figure 11 Schematic diagram of the structure of the multi-stage image enhancement network in step 4.1 of an embodiment of the present invention.

[0054] The following are the descriptions of the reference numerals:

[0055] 000-input image, 001-recovered image in stage 1, 002-recovered image in stage 2, 003-recovered image in stage 3;

[0056] 101-stage 1 Conv, 102-stage 2 Conv, 1031-stage 3 Conv 1, 1032-stage 3 Conv 2;

[0057] 201-Phase 1 CAB, 202-Phase 2 CAB, 203-Phase 3 CAB;

[0058] 301-stage 1 U-Net, 302-stage 2 U-Net;

[0059] 401-Stage 1 Supervisory Attention Module, 402-Stage 2 Supervisory Attention Module;

[0060] 5-stage three ORSNet, 501-first ORB, 50n-nth ORB. DETAILED DESCRIPTION

[0061] The present invention will be further described below with reference to the accompanying drawings and exemplary embodiments.

[0062] A method for detecting failed spacecraft components based on improved YOLOv5s includes the following steps:

[0063] Step 1: In the YOLOv5s neck network, CAM and SAM are connected in sequence between each C3 module and CBS. The output of the C3 module is connected to the input of the CAM, and the output of the SAM is connected to the input of the CBS to obtain the initial target detection network, as shown in the following figure: Figure 1 As shown;

[0064] YOLOv5s includes backbone network, neck network and head network;

[0065] The backbone network includes Focus, the first CBS, C3_1x1, the second CBS, the first C3_1x3, the third CBS, the second C3_1x3, the fourth CBS, SPP (Spatial Pyramid Pooling), C3_2x1 and the fifth CBS, which are connected in sequence. Focus is used to decompose the processed image into multiple parts by channel and splice them in the depth dimension; CBS contains Conv, Batch Normalization (BN) and activation function (SiLU) for feature extraction and normalization; C3_1x1 represents a C3 module with 1 Bottle neck network layer repeated once, C3_1x3 represents a C3 module with 1 Bottle neck network layer repeated 3 times, and C3_2x1 represents a C3 module with 2 Bottle neck network layers repeated once; the C3 module is used to extract deep features; SPP is used to perform maximum pooling operations at different scales and then splice the results;

[0066] The neck network includes the first Upsample, the first Concat, the first C3_2x1, the first CBS, the second Upsample, the second Concat, the second C3_2x1, the second CBS, the third Concat, the third C3_2x1, the third CBS, the fourth Concat, the fourth C3_2x1, the fourth CBS, the fifth Concat, and the fifth C3_2x1, which are connected in sequence;

[0067] The output of C3_1x1 in the backbone network is also connected to the input of the third Concat in the neck network, the output of the first C3_1x3 in the backbone network is also connected to the input of the second Concat in the neck network, the output of the second C3_1x3 in the backbone network is also connected to the input of the first Concat in the neck network, and the output of the fifth CBS in the backbone network is respectively connected to the input of the first Upsample and the input of the fifth Concat in the neck network; Upsample in the neck network is used to upgrade the low-resolution feature map to the same size as the higher-resolution feature map, Concat is used to splice feature maps from different scales in the channel dimension, C3_2x1 is used to further extract features from the spliced ​​feature map, and CBS is used to perform convolution operations on the input feature map to extract features;

[0068] The head network consists of three YOLO layers in parallel, which receive feature maps of three different resolutions from the neck network respectively. Each YOLO layer contains Conv, which is used to process the input feature map of the corresponding resolution and output the feature map of the corresponding resolution. The input of the high-resolution YOLO layer is connected to the output of the third C3_2x1 in the neck network, the input of the medium-resolution YOLO layer is connected to the output of the fourth C3_2x1 in the neck network, and the input of the low-resolution YOLO layer is connected to the output of the fifth C3_2x1 in the neck network.

[0069] Reference Figure 1 In this embodiment, a CAM and a SAM are sequentially connected between each C3 module and the CBS in the neck network of YOLOv5s. Specifically, a CAM and a SAM are sequentially connected between the first C3_2x1 and the first CBS, the second C3_2x1 and the second CBS, the third C3_2x1 and the third CBS, and the fourth C3_2x1 and the fourth CBS. The sequentially connected CAM and SAM constitute a CBAM, and the output end of the CBAM between the first C3_2x1 and the first CBS is also connected to the input end of the fourth Concat in the neck network.

[0070] CBAM is used to enhance important features by focusing on different channels of the input feature map. The specific steps are as follows:

[0071] Step A1: Perform global average pooling and global maximum pooling on the input feature map to obtain two pooled feature maps of size C×1×1;

[0072] Step A2: Pass the two pooled feature maps through a two-layer fully connected network (MLP) with shared weights to obtain two feature maps of size C×1×1;

[0073] Step A3: Add the two feature maps element-wise and fuse them to obtain a fused feature map of size C×1×1;

[0074] Step A4: Calculate the channel attention map based on the fused feature map using the Sigmoid function;

[0075] Step A5: Multiply the channel attention map and the input feature map element-wise to obtain an enhanced feature map;

[0076] The SAM enhances important features by focusing on different spatial locations of the input feature map. The specific steps are as follows:

[0077] Step B1: Perform global average pooling and global maximum pooling on the enhanced feature map obtained in step A5 to obtain two pooled feature maps of size 1×H×W;

[0078] Step B2: Concatenate the two pooled feature maps in the channel dimension to obtain a concatenated feature map of size 2×H×W;

[0079] Step B3: Perform a convolution operation on the concatenated feature map to obtain a convolved feature map of size 1×H×W;

[0080] Step B4: Calculate the spatial attention map based on the convolution feature map using the Sigmoid function;

[0081] Step B5: Multiply the spatial attention map and the enhanced feature map element-wise to obtain the final enhanced feature map;

[0082] Step 2: Use the model of the failed spacecraft to be detected to collect images of the failed spacecraft in a darkroom environment in the laboratory, perform data augmentation on the collected failed spacecraft images, divide them into a training set and a validation set, and annotate the images in the training set and validation set;

[0083] Step 2.1: Use the model of the failed spacecraft to be inspected as the target, use an LED light source to simulate incident sunlight, and use a mobile vehicle set on a slide rail and an observation camera set at the front end of the mobile vehicle to simulate observing the spacecraft;

[0084] Step 2.2: Let the incident sunlight illuminate the target at an incident angle of 0° to 90°, and move the observation spacecraft toward the target at a constant speed (0.5 m / s) within a preset range (0.5 m to 5 m) in a straight line to simulate the approach phase. At the same time, collect a jittered and blurred image of the target, such as Figures 5 to 7 As shown in the figure, the incident sunlight is made to illuminate the target at an incident angle of 0° to 90°, and the observation spacecraft is made to simulate the hovering stage at a fixed distance (1m) from the target, while collecting the noise image of the target, as shown in the figure. Figures 2 to 4As shown; after the acquisition is completed, the jitter blurred image and the noise image are used as the failed spacecraft image;

[0085] The brightness, radiation area and radiation beam size of the LED light source can be adjusted manually, which can better simulate the collimation, uniformity and spectral characteristics of solar radiation;

[0086] Step 2.3: Perform data augmentation on the failed spacecraft image collected in step 2.2 (e.g. Figures 8 to 10 The training set and validation set are divided into training set and validation set. Then, Label lmg software is used to annotate the training set and validation set images according to the COCO dataset format. The annotated targets include the spacecraft solar panels and the spacecraft body. The sample size ratio of the training set and validation set is 9:1.

[0087] Step 3: Input the training set and validation set obtained in step 2 into the initial target detection network in step 1 for training to obtain a trained target detection network.

[0088] Step 3.1. Set the training hyperparameters as follows:

[0089] The training epochs are set to 300, the learning rate is set to 0.01, the momentum is set to 0.937, and the batch size is set to 16. The basic training framework is Pytorch, and the hardware is NVIDIA GeForce GTX 1650Ti.

[0090] Step 3.2: Input the training set obtained in step 2 into the object detection network initialized in step 1 to calculate the output prediction results; use the loss function to calculate the training loss between the prediction results and the true annotations; perform gradient backpropagation based on the training loss, and update the network weights and biases through the optimizer;

[0091] Step 3.3: Perform a forward propagation calculation on the validation set obtained in step 2, and calculate the average loss of all validation samples as the validation loss;

[0092] Step 3.4: Record the training loss and validation loss to evaluate the training effect of the model; the evaluation indicators include mAP, Recall and Precision;

[0093] Step 3.5: Repeat steps 3.2 to 3.4 until the predetermined training cycle is reached, or the validation loss and severe loss meet the preset conditions.

[0094] Step 4: Establish a multi-stage image enhancement network, and use it to perform image enhancement on the failed spacecraft to be detected in the detection data set to obtain enhanced failed spacecraft to be detected images, which constitute the enhanced detection data set;

[0095] Step 4.1, reference Figure 11 , establish a multi-stage image enhancement network, which includes a stage one module, a stage two module and a stage three module, the stage one module includes the stage one Conv 101, the stage one CAB 201, the stage one U-Net 301 and the stage one supervisory attention module 401 connected in sequence; the stage two module includes the stage two Conv 102, the stage two CAB 202, the stage two splicing module, the stage two U-Net 302 and the stage two supervisory attention module 402 connected in sequence; the stage three module includes the stage three Conv one 1031, the stage three CAB 203, the stage three splicing module, the stage three ORSNet5 and the stage three Conv two 1032 connected in sequence;

[0096] The input image 000 of the stage 1 Conv 101 is a plurality of sub-images obtained by decomposing the image of the failed spacecraft to be detected. The first output end of the stage 1 U-Net 301 is connected to the first input end of the stage 1 supervisory attention module 401, and the second output end is connected to the second input end of the stage 2 U-Net 302. The second input end of the stage 1 supervisory attention module 401 is used to input the ground-truth image (input image 000), the first output end is used to output the stage 1 restored image 001, and the second output end is connected to the stage 2 splicing module.

[0097] The input of the stage two Conv 102 is the stage one restored image 001; the stage two splicing module is used to splice the output of the stage two CAB 202 with the output of the stage one supervisory attention module 401 channel by channel; the first output end of the stage two U-Net 302 is connected to the first input end of the stage two supervisory attention module 402, and the second output end is connected to the second input end of the stage three ORSNet 5. The first output end of the stage two supervisory attention module 402 is used to output the stage two restored image 002, and the second output end is connected to the stage three splicing module;

[0098] The input of the stage three Conv1 1031 is the stage two restored image 002; the stage three splicing module is used to splice the output of the stage three CAB 203 with the output of the stage two supervisory attention module 402 channel by channel, and the first input end of the stage three ORSNet5 is connected to the output end of the stage three splicing module. The stage three ORSNet5 includes n sequentially connected ORBs, namely the first ORB 501, ..., the nth ORB 50n, which are used to process the image to retain fine spatial details; the output of the stage three Conv2 1032 is used to add element-by-element to the stage two restored image 002 to obtain the stage three restored image 003, which is used as the enhanced image of the failed spacecraft to be detected, constituting the enhanced data set to be detected, where n is an integer greater than 1;

[0099] Step 4.2: Initial recovery and enhancement

[0100] Multiple sub-images decomposed from the image of the failed spacecraft to be detected are used as input image 000. Initial convolution processing is performed in stage 1 Conv 101, feature expression is enhanced in stage 1 CAB 201, and image details are further restored and enhanced in stage 1 U-Net 301. Finally, supervised learning is performed on the multiple sub-images decomposed from the image of the failed spacecraft to be detected by stage 1 supervised attention module 401 to improve the restoration quality, and the restored image 001 in stage 1 is output.

[0101] Step 4.3: Further refine the recovery

[0102] The restored image 001 of stage 1 is sequentially processed by Conv 102 of stage 2 for preliminary convolution, the feature expression is enhanced by CAB202 of stage 2, the output of the splicing module of stage 2 and the supervisory attention module 401 of stage 1 are spliced ​​channel by channel, the U-Net 302 of stage 2 further restores and enhances the image details, and finally the image 002 of stage 2 is restored by the supervisory attention module 402 of stage 2 and the output of stage 2.

[0103] Step 4.4: Fine restoration and enhancement

[0104] The restored image 002 of stage 2 is sequentially passed through stage 3 Conv1 1031 for preliminary convolution processing, stage 3 CAB203 for feature expression enhancement, and stage 3 splicing module and stage 2 supervisory attention module 402 output channel by channel. Then, it is input into stage 3 ORSNet 5, and the image is processed by n sequentially connected ORBs to retain and enhance the fine spatial details of the image. Finally, it is convolved through stage 3 Conv2 1032 and then element-wise added with the restored image 002 of stage 2 to obtain the restored image 003 of stage 3.

[0105] Step 5: Input the enhanced dataset to be detected obtained in step 4 into the target detection network trained in step 3 to obtain the classification and positioning results of the failed spacecraft components. The mAP and regression rate are used for evaluation to complete the detection of failed spacecraft components.

[0106] In order to evaluate the effect of the embodiments of the present invention, comparative examples 1 and 2 are set for comparison. Comparative example 1 does not enhance the image of the failed spacecraft to be detected, and uses the original YOLOv5s to detect failed spacecraft components. Comparative example 2 enhances the image of the failed spacecraft to be detected, and uses the original YOLOv5s to detect failed spacecraft components, as shown in Table 1.

[0107] Table 1

[0108]

[0109] Experimental results show that the embodiments of the present invention can effectively improve detection accuracy, and the target detection network has a mAP improved by 4.9% compared with the original YOLOv5s. This is because the addition of the attention mechanism can suppress the interference of irrelevant information, retain more key features of the failed spacecraft to be detected, enable the network to focus on more targets, and improve detection accuracy. Compared with no image enhancement, the mAP of image enhancement performed through the multi-stage image enhancement network is improved by 1.4%. This is because image enhancement significantly improves the feature expression of the data set to be detected, solving the problem of difficult target recognition in the presence of image noise and jitter blur in space environments.

Claims

1. A method for detecting failed spacecraft components based on improved YOLOv5s, characterized in that: The steps include: Step 1: In the neck network of YOLOv5s, a CAM and a SAM are connected in sequence between each C3 module and CBS, so that the output of the C3 module is connected to the input of the CAM, and the output of the SAM is connected to the input of the CBS, to obtain an initial target detection network; the CAM and SAM connected in sequence constitute a CBAM; Step 2: Use the model of the failed spacecraft to be detected to collect images of the failed spacecraft in a darkroom environment in the laboratory, perform data augmentation on the collected failed spacecraft images, divide them into a training set and a validation set, and annotate the images in the training set and validation set; Step 3: Input the training set and validation set obtained in step 2 into the initial target detection network in step 1 for training to obtain a trained target detection network. Step 4: Establish a multi-stage image enhancement network, and use it to perform image enhancement on the failed spacecraft to be detected in the detection data set to obtain enhanced failed spacecraft to be detected images, which constitute the enhanced detection data set; The multi-stage image enhancement network includes a stage one module, a stage two module and a stage three module, wherein the stage one module includes a stage one Conv (101), a stage one CAB (201), a stage one U-Net (301) and a stage one supervisory attention module (401) connected in sequence; the stage two module includes a stage two Conv (102), a stage two CAB (202), a stage two splicing module, a stage two U-Net (302) and a stage two supervisory attention module (402) connected in sequence; the stage three module includes a stage three Conv one (1031), a stage three CAB (203), a stage three splicing module, a stage three ORSNet (5) and a stage three Conv two (1032) connected in sequence; The input of the stage one Conv (101) is a plurality of sub-images obtained by decomposing the image of the failed spacecraft to be detected, the first output end of the stage one U-Net (301) is connected to the first input end of the stage one supervisory attention module (401), the second output end is connected to the second input end of the stage two U-Net (302), the second input end of the stage one supervisory attention module (401) is used to input the ground-truth image, the first output end is used to output the stage one restored image (001), and the second output end is connected to the stage two splicing module; The input of the stage two Conv (102) is the stage one restored image (001); the stage two splicing module is used to splice the output of the stage two CAB (202) and the output of the stage one supervisory attention module (401) channel by channel; the first output end of the stage two U-Net (302) is connected to the first input end of the stage two supervisory attention module (402), and the second output end is connected to the second input end of the stage three ORSNet (5); the first output end of the stage two supervisory attention module (402) is used to output the stage two restored image (002), and the second output end is connected to the stage three splicing module; The input of the stage three Conv1 (1031) is the stage two restored image (002); the stage three splicing module is used to splice the output of the stage three CAB (203) with the output of the stage two supervisory attention module (402) channel by channel, and the stage three ORSNet (5) includes n sequentially connected ORBs for processing the image to retain fine spatial details; the output of the stage three Conv2 (1032) is used to add the stage two restored image (002) element by element to obtain the stage three restored image (003), which is used as the enhanced image of the failed spacecraft to be detected, constituting the enhanced data set to be detected; Step 5: Input the enhanced dataset to be detected obtained in step 4 into the target detection network trained in step 3 to obtain the classification and positioning results of the failed spacecraft components. The mAP and regression rate are used for evaluation to complete the detection of failed spacecraft components.

2. The method for detecting failed spacecraft components based on improved YOLOv5s according to claim 1, characterized in that: In step 1, the initial target detection network includes a backbone network, a neck network and a head network; the neck network includes a first Upsample, a first Concat, a first C3_2x1, a first CBAM, a first CBS, a second Upsample, a second Concat, a second C3_2x1, a second CBAM, a second CBS, a third Concat, a third C3_2x1, a third CBAM, a third CBS, a fourth Concat, a fourth C3_2x1, a fourth CBAM, a fourth CBS, a fifth Concat, and a fifth C3_2x1 connected in sequence; the output end of the first CBAM is also connected to the input end of the fourth Concat in the neck network.

3. The method for detecting failed spacecraft components based on improved YOLOv5s according to claim 2, characterized in that: The step 2 is specifically as follows: Step 2.1: Use the model of the failed spacecraft to be inspected as the target, use an LED light source to simulate incident sunlight, and use a mobile vehicle set on a slide rail and an observation camera set at the front end of the mobile vehicle to simulate observing the spacecraft; Step 2.2: Sunlight is incident on the target at an angle of 0° to 90°, and the observation spacecraft is moved toward the target at a constant speed along a straight line within a preset range of distance to the target to simulate the approach phase, while simultaneously capturing a jittered and blurred image of the target; The incident sunlight is made to illuminate the target at an incident angle of 0° to 90°, and the observation spacecraft is made to be at a fixed distance from the target to simulate the hovering phase, while simultaneously collecting a noise image of the target; After the acquisition is completed, the jitter-blurred image and the noise image are regarded as the failed spacecraft image; Step 2.3: Perform data augmentation on the failed spacecraft images collected in step 2.2 and divide them into a training set and a validation set. Then, perform image annotation on the training set and the validation set, with the annotated targets including the spacecraft solar panels and the spacecraft itself.

4. The method for detecting failed spacecraft components based on improved YOLOv5s according to claim 3, characterized in that: In step 2.2, the preset range is 0.5m to 5m.

5. A method for detecting failed spacecraft components based on improved YOLOv5s according to any one of claims 1 to 4, characterized in that: The step 4 is specifically as follows: Step 4.1: Establish a multi-stage image enhancement network; Step 4.2, the sub-images obtained by decomposing the image of the failed spacecraft to be detected are sequentially subjected to preliminary convolution processing by the first stage Conv (101), feature expression enhancement by the first stage CAB (201), and further restoration and enhancement of image details by the first stage U-Net (301), and finally supervised learning is performed on the sub-images obtained by decomposing the image of the failed spacecraft to be detected by the first stage supervised attention module (401) to improve the restoration quality, and the restored image (001) of the first stage is output; Step 4.3, the restored image (001) of stage one is sequentially processed by the Conv (102) of stage two for preliminary convolution processing, the feature expression is enhanced by the CAB (202) of stage two, the output of the splicing module of stage two and the supervisory attention module (401) of stage one is spliced ​​channel by channel, the U-Net (302) of stage two is further restored and enhanced in image details, and finally the restored image (002) of stage two is output by the supervisory attention module (402) of stage two; Step 4.4: The restored image (002) of stage 2 is sequentially subjected to preliminary convolution processing by stage 3 Conv1 (1031), feature expression enhancement by stage 3 CAB (203), channel-by-channel splicing of the outputs of the stage 3 splicing module and the stage 2 supervisory attention module (402), and then input into stage 3 ORSNet (5). The image is processed by n sequentially connected ORBs to retain and enhance the fine spatial details of the image, and finally convolution operation is performed by stage 3 Conv2 (1032). The image is then element-wise added to the restored image (002) of stage 2 to obtain the restored image (003) of stage 3, which is used as the enhanced image of the failed spacecraft to be detected, constituting the enhanced data set to be detected, where n is an integer greater than 1.

6. The method for detecting failed spacecraft components based on improved YOLOv5s according to claim 5, characterized in that: The step 3 is specifically as follows: Step 3.

1. Set training hyperparameters, including training cycle, learning rate, momentum, batch size, and training framework. Step 3.2: Input the training set obtained in step 2 into the object detection network initialized in step 1 to calculate the output prediction results; use the loss function to calculate the training loss between the prediction results and the true annotations; perform gradient backpropagation based on the training loss, and update the network weights and biases through the optimizer; Step 3.3: Perform a forward propagation calculation on the validation set obtained in step 2, and calculate the average loss of all validation samples as the validation loss; Step 3.4: Record the training loss and validation loss to evaluate the training effect of the model; the evaluation indicators include mAP, Recall and Precision; Step 3.5: Repeat steps 3.2 to 3.4 until the predetermined training cycle is reached or the training loss and validation loss meet the preset conditions.

Images