Space debris detection method applying convolutional neural network

Abstract

The invention discloses a space debris detection method applying a convolutional neural network, and relates to the field of image processing, and the method comprises the steps: obtaining image data of a to-be-detected space; a convolutional neural network is used as a framework, and a lightweight cross-scale feature fusion module and a content awareness feature recombination module are fused to construct a fragment detection model; and inputting the image data of the to-be-detected space into the debris detection model to obtain a space debris detection result. According to the invention, the detection problem in a complex observation environment can be solved, and the space debris detection accuracy is improved.

Description

Technical Field

[0001] The present application relates to the field of image processing, and in particular to a method for detecting spatial debris using a convolutional neural network. Background Art

[0002] Space debris is defined as inoperative man-made objects, including fragments and parts. Frequent space activities have also brought a huge burden to space. A large number of space debris floating in space poses a serious threat to the safe operation of spacecraft and space activities. Space debris moves in space at a high speed. Even if it is small in size, it has a large kinetic energy. Once it collides with a spacecraft, it will cause very serious damage.

[0003] Known detection methods have been proven to be successful in detecting bright, long-striped space debris, but have low accuracy in detecting dim, short-striped space debris. Most of these methods are developed for low-noise environments with high signal-to-noise ratios and limited clutter, but they cannot be used in complex environments with a lot of noise and clutter during astronomical observations. Summary of the invention

[0004] The purpose of this application is to provide a space debris detection method using a convolutional neural network, which can compensate for detection problems arising in complex observation environments and improve the accuracy of space debris detection.

[0005] To achieve the above objectives, this application provides the following solutions:

[0006] The present application provides a spatial debris detection method using a convolutional neural network, comprising:

[0007] Acquire image data of the space to be detected;

[0008] Constructing a fragment detection model; the fragment detection model is a neural network model based on a convolutional neural network architecture, integrating a lightweight cross-scale feature fusion module and a content-aware feature reorganization module; the convolutional neural network includes an input end, a backbone network, a neck network, and a head network;

[0009] The image data of the space to be detected is input into the debris detection model to obtain a space debris detection result.

[0010] Optionally, a fragment detection model is constructed, specifically including:

[0011] In the architecture of the convolutional neural network, the input end is configured to adaptively set the input image; a backbone network is constructed based on the CSPDarknet framework, and a neck network is constructed based on the lightweight cross-scale feature fusion module and the content-aware feature reconstruction module.

[0012] Optionally, the backbone network includes dense blocks and transition layers; the dense blocks and the transition layers form feature channels; each dense block is composed of multiple dense layers; the output of the current dense layer is connected to its own input, and the connected result is used as the input of the next dense layer; in the backbone network, the feature map passing through each feature channel is separately integrated by gradient flow truncation.

[0013] Optionally, in the process of constructing a neck network based on the lightweight cross-scale feature fusion module and the content-aware feature reconstruction module, the fusion module in the lightweight cross-scale feature fusion module is replaced with a C2f module, and the upsampling in the lightweight cross-scale feature fusion module is replaced with the content-aware feature reconstruction module.

[0014] Optionally, the C2f module includes: two convolution kernels, a residual processing module, a plurality of Bottleneck modules and a connection module;

[0015] In the C2f module, the feature data first passes through a convolution kernel, then passes through a residual processing module and multiple Bottleneck modules for Concat splicing, and then passes through a convolution kernel to obtain output feature data.

[0016] Optionally, in the neck network, a feature map of size H×W×C is input into the lightweight cross-scale feature fusion module, and the upsampling ratio is set to U, and the lightweight cross-scale feature fusion module outputs a new feature map of size UH×UW×C;

[0017] The kernel prediction module of the lightweight cross-scale feature fusion module generates position-specific kernels according to the content of the input feature map, and the content-aware feature reconstruction module reconstructs features using these position-specific kernels.

[0018] Optionally, in the process of adaptively setting the input image at the input end, the input image is scaled to a set size based on the short side of the input image.

[0019] According to the specific embodiments provided in this application, this application discloses the following technical effects:

[0020] This application provides a spatial debris detection method using a convolutional neural network. By using a convolutional neural network as the architecture, a lightweight cross-scale feature fusion module and a content-aware feature reorganization module are integrated to construct a debris detection model, which can enhance the model's adaptability to scale changes and its ability to detect small-scale objects, and ensure higher reconstruction quality and more detail retention in an environment with complex backgrounds to compensate for detection problems in complex observation environments. By inputting the image data of the space to be detected into the debris detection model, accurate detection of spatial debris can be achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0022] Figure 1 A schematic diagram of a flow chart of a method for detecting space debris using a convolutional neural network provided in one embodiment of the present application;

[0023] Figure 2 A schematic diagram of the fragment detection model structure provided in another embodiment of the present application. DETAILED DESCRIPTION

[0024] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0025] In order to make the purpose, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0026] In an exemplary embodiment, Figure 1 As shown, a space debris detection method is provided, which is executed by a computer device, and can be executed by a computer device such as a terminal or a server alone, or by a terminal and a server together. In the embodiment of the present application, the method is applied to a server or a terminal as an example for description, and includes the following steps 100 to 102. Among them:

[0027] Step 100: Acquire image data of the space to be detected.

[0028] Step 101: construct a fragment detection model. The fragment detection model (hereinafter referred to as the model) is a neural network model based on a convolutional neural network architecture, which integrates a lightweight cross-scale feature fusion module and a content-aware feature reconstruction module. The convolutional neural network includes an input end, a backbone network, a neck network, and a head network.

[0029] Step 102: Input the image data of the space to be detected into the debris detection model to obtain the space debris detection result.

[0030] Implementing the above steps 100 to 102 can compensate for detection problems arising in complex observation environments and improve the accuracy of space debris detection.

[0031] In another exemplary embodiment of the present application, in order to make the detection method efficient and low in complexity, in this embodiment, the fragment detection model constructed in step 101 is improved based on a convolutional neural network. For example, it is improved based on the YOLOv8 model. The YOLOv8 model is mainly divided into several parts: input end, backbone network, neck network, and head network. The original image is adaptively set at the input end, and the image size of the data set is uniformly scaled to a set size based on the short side to ensure that the input data is consistent, such as 640×640. Subsequently, the image is convoluted, features of different scales are extracted, and data is compressed through the CSPDarknet (Cross Stage Partial Darknet) structure of the backbone network. Then, the multi-scale feature maps are fused in the neck network to ensure that the feature maps contain semantic information and position information. Finally, the head network predicts the processed image.

[0032] In another exemplary embodiment of the present application, in order to reduce the number of model parameters and improve the accuracy of detecting small targets, a lightweight cross-scale feature fusion module (Cross-Scale Feature Fusion Module, CCFM) and a method for enhancing the upsampling of convolutional neural network feature maps, Content-Aware ReAssembly of FEatures (CARAFE), are integrated into the original YOLOv8 model structure. Based on this, the implementation process of the above step 101 can be described as:

[0033] In the architecture of the convolutional neural network, the input end is set to adaptively set the input image. The backbone network is built based on the CSPDarknet framework, and the neck network is built based on the lightweight cross-scale feature fusion module and the content-aware feature reconstruction module.

[0034] In another exemplary embodiment of the present application, the CSPDarknet framework is composed of some dense blocks and some transition layers, and each dense block is composed of k dense layers. th The output of the dense layer will be th The input of the dense layer is connected, and the result after connection will become (i+1) th The input of the dense layer. In order to reduce repeated gradient learning, the feature map passing through each channel is integrated separately by truncating the gradient flow to avoid repeated learning gradient information, so as to maximize the difference of gradient union and effectively improve learning efficiency. Among them:

[0035] xk =w k * [x0,x1,...,x k-1 ]

[0036] x T =w T * [x0,x1,...,x k ].

[0037] x U =w U * [x0,x T ]

[0038] Where * represents the convolution operator. [x0, x1, …] represents the concatenated features x0, x1, …. w i and x i I th The weights and outputs of the dense layer. If the back-propagation algorithm is used to update the weights, the equation for weight update can be expressed as:

[0039] w′ k =f(w k ,g 0″ ,g1,g2,...,g k-1 )

[0040] w′ T =f(w T ,g 0″ ,g1,g2,...,g k ).

[0041] w′ U =f(w U ,g 0′ ,g T )

[0042] Among them, f( ) is the function of weight update, g i Indicates that it is propagated to i th Gradients of dense layers. x0 does not pass through dense layers and is integrated separately. For the gradient information of updating weights, neither side contains repeated gradient information belonging to other sides. CSPDenseNet retains the advantages of feature reuse while preventing excessive repeated gradient information by truncating the gradient flow. This idea is implemented by designing a hierarchical feature fusion strategy and is used in some transition layers.

[0043] In another exemplary embodiment of the present application, CCFM is used in the neck network part to integrate features of different scales through fusion operations, so as to enhance the model's adaptability to scale changes and its ability to detect small-scale objects. At the same time, the original fusion module Fusion in CCFM is replaced with a C2f (CSPLayer_2Conv) module with fewer parameters and better feature extraction capabilities. First, a 1×1 convolution kernel is used to reduce the number of input channels to 1 / 2 of the original to reduce the amount of calculation and memory consumption. Then, a Split module (the Split module and the subsequent Concat) is used to form a convolution kernel. Figure 2 After the residual processing module in the image and the number of Bottleneck modules n are processed, the results of the residual processing module and the backbone module (i.e. the number of Bottleneck modules n) are concat-joined and then processed by a 1×1 convolution kernel for output to restore the number of channels of the feature map. Using a smaller convolution kernel reduces the number of input channels, thereby reducing the amount of calculation and memory consumption.

[0044] The feature fusion process is as follows Figure 2 As shown in the middle neck network part, the outputs P3, P4 and P5 of the backbone network are input into CCFM to fuse feature maps of multiple scales. The output P5 is upsampled and fused with P4 to obtain feature F1. Feature F1 is fused with output P3 after passing through the C2f module, the convolution layer and one upsampling, and then passed through the C2f module to obtain feature T1. Feature T1 is fused with feature F1 after passing through a convolution layer to obtain feature F2. Feature F2 is passed through the C2f module once to obtain feature T2. Feature T2 is fused with output P5 after passing through a convolution layer once to obtain feature F3. Feature F3 is passed through the C2f module once to obtain feature T3. Finally, the obtained features T1, T2 and T3 are used as the output of CCFM. The above feature processing flow is a simplified description. For the specific feature processing process, please refer to Figure 2 .

[0045] In this embodiment, CCFM and C2f modules adopt a more lightweight structure, which reduces the number of parameters and computational complexity of the algorithm. At the same time, CCFM enhances the algorithm's adaptability to scale changes and its ability to detect small-scale objects through the horizontal fusion of feature maps of different scales. Based on this, the model constructed in this application can improve the performance of target detection while maintaining high computational efficiency.

[0046] Subsequently, an upsampling method for enhancing convolutional neural network feature maps, namely CARAFE, was incorporated into CCFM. The content-aware feature recombination module first predicts the recombination kernel for each target position and then reconstructs the features with the predicted kernel. Given a feature map of size H×W×C and an upsampling ratio U, CARAFE will produce a new feature map of size UH×UW×C. Secondly, the kernel prediction module of CARAFE generates position-specific kernels based on the content of the input features, and then CARAFE uses these kernels to reconstruct the features, based on which higher reconstruction quality and more detail retention are guaranteed in environments with complex backgrounds. CARAFE predicts the recombination kernel by leveraging the underlying content information (i.e., image feature information) at each position and reconstructs the features in a predefined nearby area. Due to the introduction of content information, CARAFE can use adaptive and optimized recombination kernels at different positions, thereby performing better than mainstream upsampling operators such as interpolation or deconvolution.

[0047] In another exemplary embodiment of the present application, in order to accurately and objectively evaluate the method provided by the present application, in this embodiment, multiple indicators are selected as evaluation criteria for the model. Among them, the average precision mAP is selected to evaluate the detection performance of the model, and mAP is the average of the average precision AP of multiple categories. The relevant calculation formula is as follows:

[0048]

[0049] Among them, precision represents the probability of actually being a positive sample among all samples predicted to be positive. Recall represents the probability of being predicted to be a positive sample among samples actually being positive. The relevant calculation formula is as follows:

[0050]

[0051] True positive TP indicates the number of positive samples correctly classified in the test results, false positive FP indicates the number of negative samples incorrectly classified as positive in the test results, and false negative FN indicates the number of positive samples incorrectly classified as negative in the test results. These indicators are related to the calculation of AP. The value of AP varies between 0 and 1, and can reflect the changes in accuracy and recall. mAP is the average evaluation result of multi-class detection results. mAP50 represents the average accuracy of detection when the AP threshold of IOU is set to 0.5 in the experiment, and mAP95 represents the average accuracy of detection when the AP threshold of IOU is set to 0.95 in the experiment. At the same time, the experiment uses frames per second FPS to evaluate the computing speed of the model, and uses floating-point operations per second GFLOPs and the number of parameters Parameters to evaluate the complexity of the model.

[0052] 1600 star images observed by a large-field astronomical telescope are used as experimental data for comparative discussion. Table 1 shows the ablation experiment of the same framework algorithm. By successively adding CCFM network structures of different modules, CARAFE and CCFM with the fusion module changed to the C2f module as improved modules, experiments are carried out to verify the improvement effect. Table 2 compares the method provided in this application with some mainstream space target detection methods, including single-stage detection algorithms and two-stage detection algorithms. Among them, the single-stage detection algorithms include YOLOv2, YOLOv3-tiny, YOLOv5, YOLOv8, SSD, etc. The two-stage detection algorithms include Faster-RCNN. The experimental results show that the method provided in this application reduces the computational complexity and parameter amount of the algorithm while achieving a higher accuracy rate.

[0053] Table 1 Ablation experiment results

[0054]

[0055] The original YOLOv8 algorithm experiment obtained an mAP value of 91.5%, and a GFLOPs value of 28.6. The use of CCFM in the neck network instead of the original network structure is a key improvement. The model's adaptability to scale changes and the ability to detect small-scale objects are enhanced, which increases the mAP value of the improved algorithm to 92.9%, an increase of 1.7% compared to the baseline. The subsequent addition of CARAFE achieves more efficient and accurate feature reconstruction, which increases the algorithm's mAP value by 0.5%. Finally, the C2f module is used to replace the original fusion module of CCFM to reduce the model complexity and computational complexity, while increasing the mAP value by 0.3%. When each improvement group is combined, the improved algorithm has a mAP value of 93.7% and a GFLOPs value of 23.4.

[0056] Table 2 Comparison results of different models with multiple indicators

[0057]

[0058] The experimental results show that the mAP50 and mAP50-95 of the method provided in this application reached 93.7% and 46.7% respectively, which is 2.2% and 1.7% higher than the original YOLOv8 algorithm, proving that the accuracy of the method in this application has been improved, and it also has certain advantages over the other algorithms mentioned above. GFLOPs and Parameters were reduced by 18.2% and 35.1% respectively, and the model complexity and algorithm calculation amount were significantly reduced. Compared with the original YOLOv8, the method in this application has achieved lightweight improvements. At the same time, the reasoning speed has been improved compared to the original algorithm.

[0059] In summary, the neck network uses CCFM to integrate features of different scales through fusion operations to enhance the model's adaptability to scale changes and its ability to detect small-scale objects.

[0060] The upsampling method uses CARAFE to predict the reconstruction kernel by leveraging the underlying content information at each location and reassemble the features in a predefined nearby area, thereby ensuring higher reconstruction quality and more detail preservation in environments with complex backgrounds.

[0061] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0062] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A space debris detection method using convolutional neural network, characterized in that: The space debris detection method using convolutional neural network includes: Acquire image data of the space to be detected; Constructing a fragment detection model; the fragment detection model is a neural network model based on a convolutional neural network architecture, integrating a lightweight cross-scale feature fusion module and a content-aware feature reorganization module; the convolutional neural network includes an input end, a backbone network, a neck network, and a head network; The image data of the space to be detected is input into the debris detection model to obtain a space debris detection result.

2. The method for detecting space debris using a convolutional neural network according to claim 1, characterized in that: Build a fragment detection model, including: In the architecture of the convolutional neural network, the input end is configured to adaptively set the input image; a backbone network is constructed based on the CSPDarknet framework, and a neck network is constructed based on the lightweight cross-scale feature fusion module and the content-aware feature reconstruction module.

3. The method for detecting space debris using a convolutional neural network according to claim 2, characterized in that: The backbone network includes dense blocks and transition layers; the dense blocks and transition layers form feature channels; each dense block is composed of multiple dense layers; the output of the current dense layer is connected with its own input, and the result of the connection is used as the input of the next dense layer; in the backbone network, the feature map passing through each feature channel is individually integrated by gradient flow truncation.

4. The method for detecting space debris using a convolutional neural network according to claim 2, characterized in that: In the process of constructing a neck network based on the lightweight cross-scale feature fusion module and the content-aware feature reconstruction module, the fusion module in the lightweight cross-scale feature fusion module is replaced by a C2f module, and the upsampling in the lightweight cross-scale feature fusion module is replaced by the content-aware feature reconstruction module.

5. The method for detecting space debris using a convolutional neural network according to claim 4, characterized in that: The C2f module includes: two convolution kernels, a residual processing module, multiple Bottleneck modules and a connection module; In the C2f module, the feature data first passes through a convolution kernel, then passes through a residual processing module and multiple Bottleneck modules for Concat splicing, and then passes through a convolution kernel to obtain output feature data.

6. The method for detecting space debris using a convolutional neural network according to claim 5, characterized in that: In the neck network, a feature map of size H×W×C is input into the lightweight cross-scale feature fusion module, and the upsampling ratio is set to U, and the lightweight cross-scale feature fusion module outputs a new feature map of size UH×UW×C; The kernel prediction module of the lightweight cross-scale feature fusion module generates position-specific kernels according to the content of the input feature map, and the content-aware feature reconstruction module reconstructs features using these position-specific kernels.

7. The method for detecting space debris using a convolutional neural network according to claim 2, characterized in that: In the process of adaptively setting the input image at the input end, the input image is scaled to a set size based on the short side of the input image.

Images