A lightweight non-cooperative target local feature recognition method

Abstract

The application discloses a kind of lightweight non-cooperative target local feature identification methods, belong to aerospace field.The application implementation method is: using darkroom environment simulation space environment.Keep adjusting the attitude of spacecraft model, construct the data set of space non-cooperative target;Using a variety of data enhancement methods to expand database.Construct the detection network of lightweight non-cooperative target local feature, using the lightweight module of superimposed ShuffleNetV2 as the backbone network of LLFDN;Construct feature pyramid network fusion multiscale feature information, use attention module before feature pyramid network, carry out multiscale fusion prediction to the target feature extracted;Using Soft-NMS as new non-maximum suppression method;Using model pruning method, reduce the parameter quantity of LLFDN model;The lightweight LLFDN model obtained is fine-tuned, and the optimal LLFDN model is obtained;Using optimal LLFDN model realizes real-time and high accuracy, strong anti-interference ability of space non-cooperative target local feature identification.

Description

Technical Field

[0001] This invention belongs to the field of aerospace and relates to a method for recognizing local features of lightweight non-cooperative targets. Background Technology

[0002] With the continuous development of aerospace technology, a large number of spacecraft are operating in orbit and performing various missions. However, due to potential unexpected malfunctions and fuel depletion, these spacecraft may become non-cooperative targets, posing a threat to other spacecraft. Therefore, the recovery or on-orbit servicing of these spacecraft is of great significance for maintaining space security and conserving resources. Non-cooperative targets lack instruments such as coordinate sensors for measurement, and their imaging is easily interfered with by space lighting and noise. Furthermore, differences in local target features can affect detection accuracy, posing significant challenges to spacecraft recovery and on-orbit servicing.

[0003] Traditional methods based on point-line, edge feature, and keypoint matching are used to identify specific local features of non-cooperative spacecraft. These methods require manually designed features and are difficult to meet the demands of various space environments. Furthermore, local features on non-cooperative spacecraft can become blurred, scale-changed, and shadowed due to occlusion, lighting, and pose changes, which traditional methods struggle to handle. With the continuous development of artificial intelligence, neural network-based local feature recognition of non-cooperative targets in space has become a research hotspot. Detection models such as RCNN and YOLO are used to detect the entire satellite and local features of non-cooperative targets, including antennas, solar panels, supports, and nozzles. However, existing neural network-based local feature recognition models for non-cooperative targets in space have long training times, large model sizes, and heavy computational burdens. Due to the limited computing power of onboard computers, large models with high computational requirements are difficult to run. Therefore, it is necessary to propose a lightweight local feature recognition method for non-cooperative targets with high real-time performance, high accuracy, and strong anti-interference capabilities to ensure both real-time performance and accuracy. Summary of the Invention

[0004] The main objective of this invention is to provide a lightweight method for identifying local features of non-cooperative targets in space. By constructing a lightweight detection model, it can improve the detection efficiency of local features of non-cooperative targets in space while maintaining detection accuracy under limited computing resources. This invention features a lightweight target detection model that ensures both high real-time performance and detection accuracy, and also has the advantages of strong anti-interference capability. It is beneficial for deployment on devices with limited computing power, such as spaceborne computers, thereby improving the real-time performance, accuracy, and robustness of non-cooperative target detection in space.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0006] This invention discloses a lightweight method for identifying local features of non-cooperative targets. It utilizes a ground-based darkroom environment, a spacecraft model, a movable six-DOF turntable, and a camera to construct a ground-based data acquisition platform for non-cooperative targets in space, simulating the space environment using the darkroom setup. The spacecraft model is placed on the movable six-DOF turntable, and its attitude is continuously adjusted. Data is collected via the camera to construct a dataset of non-cooperative targets in space. Various data augmentation methods are used to expand the database, enhancing the generalization performance and robustness of the Lightweight Local Feature Detection Network (LLFDN). The database is then labeled and divided into training, validation, and test sets. A lightweight detection network for non-cooperative target local features is constructed, using a lightweight module superimposed with ShuffleNetV2 as the backbone network of LLFDN. Depthwise separable convolutions in ShuffleNet are used to reduce the complexity of the LLFDN model, while channel shuffling is employed to improve image feature extraction. A feature pyramid network is constructed to fuse multi-scale feature information. An attention module is used before the feature pyramid network to perform multi-scale fusion prediction on the extracted target features, resulting in fused features at multiple scales. This improves the multi-scale feature fusion network's ability to extract features of spatially non-cooperative local targets, thereby enhancing the detection accuracy of the LLFDN model. Considering the existence of numerous local features of spatially non-cooperative targets and the potential for occlusion, Soft-NMS is used as a novel non-maximum suppression method. To further reduce the number of parameters and computational cost of the LLFDN model, a model pruning method is employed. Unimportant weight parameters in the LLFDN model are filtered and removed, reducing the number of parameters and computational cost of the LLFDN model while ensuring that the model's accuracy is not affected. After pruning, the obtained lightweight LLFDN model needs to be fine-tuned and then trained to improve the detection accuracy of LLFDN, obtaining the optimal LLFDN model. The image of the non-cooperative spatial target to be identified is input into the pre-trained optimal LLFDN model, and the optimal LLFDN model is used to achieve real-time, high accuracy, and strong anti-interference ability for the identification of local features of non-cooperative spatial targets. The lightweight LLFDN model constructed by this invention has the advantages of high real-time detection accuracy and strong anti-interference ability, which is beneficial for deployment on devices with limited computing power, such as spaceborne computers, and improves the real-time performance, accuracy, and robustness of local feature identification of non-cooperative spatial targets.

[0007] The present invention discloses a lightweight method for identifying local features of non-cooperative targets, comprising the following steps:

[0008] Step 1: Construct a ground-based data acquisition platform for non-cooperative space targets using a ground-based darkroom environment, a spacecraft model, a movable six-degree-of-freedom turntable, and cameras. The darkroom environment simulates the space environment. The spacecraft model is placed on the movable six-degree-of-freedom turntable, and its position and attitude are continuously adjusted to simulate the spacecraft's attitude trajectory. Multiple light sources are used to simulate the actual lighting conditions in space, and image data of the spacecraft is acquired through a fixed-position camera to construct a dataset of non-cooperative space targets.

[0009] Step Two: The database is expanded using various data augmentation methods. A ground-based simulated space environment background is used as the space background. To improve data realism, some image data is extracted and added to the Earth image as the spacecraft data image background. Considering the complex characteristics of deep space lighting, noise, and environment, two data augmentation methods, photometric distortion and geometric distortion, are used to expand the dataset. For photometric distortion, the brightness, saturation, and noise of the images are enhanced to adapt to the effects of lighting and noise in space. When handling geometric distortion, random scaling, cropping, translation, shearing, and rotation are added to enhance the generalization performance and robustness of the lightweight local feature recognition model LLFDN. The database is then labeled and divided into training, validation, and test sets.

[0010] Step 3: Construct a lightweight detection network for local features of non-cooperative targets. A lightweight module of ShuffleNetV2 is used as the backbone network of LLFDN. Depthwise separable convolutions from ShuffleNet are utilized to reduce model complexity, while channel shuffling is employed to improve image feature extraction. The LLFDN backbone network designed in this invention consists of n lightweight ShuffleNetV2 modules, totaling m layers, with a conv_bn_relu_maxpool layer added at the beginning. This not only reduces the number of parameters and computational cost of the LLFDN model but also gives the LLFDN backbone network strong feature extraction capabilities.

[0011] ShuffleNetV2 reduces the number of model parameters through four strategies: minimizing memory access, reducing the use of group convolutions, network ensembles, and reducing element-wise operations. It also uses depthwise separable convolutions to further reduce the number of parameters. The number of parameters for depthwise separable convolutions is:

[0012] D k *D k *M+M*N

[0013] The computational workload is:

[0014] D k *D k *M*Wout *H out +M*W out *H out *N

[0015] Compared with standard convolution, it has significant advantages in terms of parameter count and computational cost. Therefore, using the method of stacking ShuffleNetV2 to construct the backbone network of the model can greatly reduce the parameter count and computational cost of the LLFDN model.

[0016] A feature pyramid network is constructed to fuse multi-scale feature information. Features extracted from the backbone are upsampled and fused with information from the previous layer, integrating edge and positional information of local features with rich texture information from the previous layer to improve local feature extraction capabilities. An attention module is used before the feature pyramid network to perform multi-scale fusion prediction on the extracted target features, resulting in fused features at multiple scales. This improves the multi-scale feature fusion network's ability to extract spatially non-cooperative local target features, thereby enhancing the detection accuracy of the LLFDN model.

[0017] Considering the existence of numerous local features for non-cooperative targets in space, and the potential for occlusion, Soft-NMS is used as a novel non-maximum suppression method. Compared to traditional NMS, which retains the highest-scoring bounding box while deleting other boxes with an overlap area greater than a certain proportion, Soft-NMS directly reduces the confidence of these boxes, thus preserving more bounding boxes. This is represented as...

[0018]

[0019] By preserving more predicted bounding boxes, Soft-NMS can better capture local feature information and improve the ability to extract local features when performing spatial non-cooperative target local feature recognition, due to the large number of local features and the existence of occlusion problems.

[0020] Step 4: To further reduce the number of parameters and computational cost of the LLFDN model, a pruning method is used. During training, regularization is used to calculate the LP norm of each convolutional kernel, and the parameters of some convolutional kernels are set to 0. As the LLFDN model is continuously trained, the number of pruned convolutional kernels is continuously updated. Some unimportant weight parameters in the LLFDN model are filtered and removed. The convolutional kernels with parameters of 0 in the trained model are removed to obtain a lightweight LLFDN model, which reduces the number of parameters and computational cost of the LLFDN model while ensuring that the accuracy of the model is not affected.

[0021] A set of LP norms are defined as follows:

[0022]

[0023] Where: x i It is an n-dimensional vector, where p is the order of the norm.

[0024] Step 5: After pruning the LLFDN model, the parameters of the obtained LLFDN model need to be fine-tuned, and then the lightweight LLFDN model is trained to improve the detection accuracy of LLFDN, obtaining the optimal LLFDN model. The parameters of the pruned LLFDN model are then modulated again to improve the detection accuracy of LLFDN, obtaining the optimal LLFDN model. The image of the spatial non-cooperative target to be identified is input into the pre-trained optimal LLFDN model, and the optimal LLFDN model is used to perform local feature recognition of spatial non-cooperative targets, improving the real-time performance, accuracy, and anti-interference ability of spatial non-cooperative target local feature recognition.

[0025] Beneficial effects:

[0026] 1. The present invention discloses a lightweight non-cooperative target local feature recognition method, which uses a lightweight module superimposed with ShuffleNetV2 as the backbone network of LLFDN, reducing the computational load of the model. At the same time, the convolution kernel pruning method is used to jointly compress the lightweight backbone network, further reducing the computational load and parameter count of the model, enabling the model to run on devices with low hardware requirements, improving the detection efficiency of spatial non-cooperative target local features while ensuring detection accuracy.

[0027] 2. The present invention discloses a lightweight non-cooperative target local feature recognition method, which constructs a feature pyramid network to fuse multi-scale feature information and introduces an attention module to improve the multi-scale fusion capability of target features, thereby obtaining multi-scale fused features, improving the feature pyramid's ability to detect local features of spatial non-cooperative targets, and enhancing the detection accuracy and efficiency of local features of spatial non-cooperative targets.

[0028] 3. The present invention discloses a lightweight non-cooperative target local feature recognition method, which adopts a ground-based semi-physical simulation dataset construction method, uses a darkroom environment to simulate the space environment, and then places a spacecraft model on a movable six-degree-of-freedom turntable. The position and attitude of the spacecraft model are continuously adjusted to simulate the attitude trajectory motion of the spacecraft. At the same time, multiple light sources are used to simulate the actual lighting conditions in space, and image data of the spacecraft are collected through a camera at a fixed position, thus constructing a space non-cooperative target local feature recognition method that is closer to the real situation.

[0029] 4. The present invention discloses a lightweight non-cooperative target local feature recognition method, which constructs a lightweight LLFDN model and uses the LP norm to prune the LLFDN model, thereby reducing the number of parameters and computational cost of the LLFDN model while ensuring that the accuracy of the model is not affected. This is beneficial for deployment on devices with limited computing power, such as spaceborne computers, and improves the real-time performance, accuracy and robustness of space non-cooperative target local feature recognition. Attached Figure Description

[0030] Figure 1 This is a flowchart illustrating a lightweight non-cooperative target local feature recognition method according to the present invention;

[0031] Figure 2 This is a schematic diagram of the lightweight backbone network structure of ShuffleNetV2 superimposed in this invention;

[0032] Figure 3 This is a schematic diagram of the depth-separable convolution structure in this invention;

[0033] Figure 4 This is a schematic diagram of the attention module (CA) structure in this invention;

[0034] Figure 5 This is a partial test result diagram of the LLFDN object detection network model in this invention on the test set. Detailed Implementation

[0035] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings. However, those skilled in the art will understand that the following embodiments are for illustrative purposes only and should not be considered as limiting the scope of the invention.

[0036] Reference Figure 1 This embodiment discloses a lightweight method for identifying local features of non-cooperative targets, and the specific implementation steps are as follows:

[0037] Step 1: Construct a ground-based data acquisition platform for non-cooperative space targets using a ground-based darkroom environment, a spacecraft model, a movable six-degree-of-freedom turntable, and cameras. The darkroom environment simulates the space environment. The spacecraft model is placed on the movable six-degree-of-freedom turntable, and its position and attitude are continuously adjusted to simulate the spacecraft's attitude trajectory. Multiple light sources are used to simulate the actual lighting conditions in space, and image data of the spacecraft is acquired through cameras at fixed positions to construct a dataset of non-cooperative space targets.

[0038] Step Two: The database is expanded using various data augmentation methods. A ground-based simulated space environment background is used as the space background. To improve data realism, some image data is extracted and Earth images are added as backgrounds for the spacecraft data images. Considering the complex characteristics of deep space lighting, noise, and environment, the dataset is expanded using two data augmentation methods: photometric distortion and geometric distortion. For photometric distortion, the brightness, saturation, and noise of the images are enhanced to adapt to the effects of lighting and noise in space. When handling geometric distortion, random scaling, cropping, translation, shearing, and rotation are added to enhance the generalization performance and robustness of the lightweight local feature recognition model LLFDN. The database is then labeled and divided into training, validation, and test sets. The augmented images are then resized to 1080×1080 pixels and randomly named with numerical identifiers for easy subsequent use.

[0039] Using LabelImg as a labeling tool, images are imported into LabelImg, and the locations of unknown deep-space targets in the images are marked with boxes, along with their type names. This method is used to label all images, generating XML-formatted label files, which are then output to a specified folder to create a deep-space target dataset. Software is then used to annotate the database, dividing it into training, validation, and test sets.

[0040] Step 3: Construct a lightweight detection network for local features of non-cooperative targets. A lightweight module of ShuffleNetV2 is used as the backbone network of LLFDN. Depthwise separable convolutions from ShuffleNet are utilized to reduce model complexity, while channel shuffling is employed to improve image feature extraction. The LLFDN backbone network designed in this invention consists of n lightweight ShuffleNetV2 modules, totaling m layers, with a conv_bn_relu_maxpool layer added at the beginning. This not only reduces the number of parameters and computational cost of the LLFDN model but also gives the LLFDN backbone network strong feature extraction capabilities.

[0041] ShuffleNetV2 reduces the number of model parameters through four strategies: minimizing memory access, reducing the use of group convolutions, network ensembles, and reducing element-wise operations. It also uses depthwise separable convolutions to further reduce the number of parameters. The number of parameters for depthwise separable convolutions is:

[0042] Dk*Dk*M+M*N

[0043] The computational workload is:

[0044] Dk*Dk*M*Wout*Hout+M*Wout*Hout*N

[0045] Compared with standard convolution, it has significant advantages in terms of parameter count and computational cost. Therefore, using the method of stacking ShuffleNetV2 to construct the backbone network of the model can greatly reduce the parameter count and computational cost of the LLFDN model.

[0046] A feature pyramid network is constructed to integrate feature information at multiple scales. The features extracted from the backbone are upsampled by 2 times and fused with the information of the previous layer. The edge and position information of local features are fused with the rich texture information of the previous layer to improve the extraction capability of local features.

[0047] In this embodiment, the nearest neighbor interpolation method is used for upsampling. By mapping the pixel values ​​in the original image to adjacent positions in the magnified image, a method image with similar content to the original image but with increased resolution can be obtained.

[0048] The formula for pixel mapping is as follows:

[0049] src x =dst x / scale

[0050] src y =dst y / scale

[0051] src x src y dst represents the coordinates in the original image. x dst y This represents the coordinates in the enlarged image, and scale represents the scaling factor, which is selected as 2 in this embodiment.

[0052] An attention module is used before the feature pyramid network to perform multi-scale fusion prediction on the extracted target features, resulting in fused features at multiple scales. This improves the multi-scale feature fusion network's ability to extract spatially non-cooperative local target features, thereby enhancing the detection accuracy of LLFDN. This invention uses a Coordinate Attention (CA) mechanism, and to capture more accurate location information interactions, global average pooling is decomposed:

[0053]

[0054]

[0055] The input feature map of size C*H*W is pooled along the X and Y directions to generate feature maps of size C*H*1 and C*1*W, respectively. The generated C*1*W feature maps are then transformed and concatenated using the Concat operation.

[0056] f=δ(F1([z h ,z w ]))

[0057] Split along spatial dimension f into f h and f w Then, pointwise convolution is used to increase the dimensionality, and finally, an activation function is used to obtain the attention vector. The output of CA attention can be expressed as:

[0058]

[0059] g h =σ(F h (f h ))

[0060] g w =σ(F w (f w ))

[0061] Considering the existence of numerous local features for non-cooperative targets in space, and the potential for occlusion, Soft-NMS is used as a novel non-maximum suppression method. Compared to traditional NMS, which retains the highest-scoring bounding box while deleting other boxes with an overlap area greater than a certain proportion, Soft-NMS directly reduces the confidence of these boxes, thus preserving more bounding boxes. This can be represented as:

[0062]

[0063] Soft-NMS can retain more predicted bounding boxes. When performing local feature recognition of non-cooperative targets in space, due to the large number of local features and the existence of occlusion problems, Soft-NMS can better obtain local feature information and improve the ability to extract local features.

[0064] Step 4: To further reduce the number of model parameters and computational cost, we use model pruning. During training, regularization is used to set the parameters of some convolutional kernels to 0. The number of pruned convolutional kernels is continuously updated as the model trains, filtering and removing unimportant weight parameters. Finally, the convolutional kernels with parameters of 0 in the trained model are removed to obtain a lightweight LLFDN model. This reduces the number of model parameters and computational cost while ensuring that the model's accuracy is not affected. In this embodiment, convolutional kernel pruning is used with a double-loop structure. The first loop is the normal training loop, and pruning is performed after each training loop. The second loop traverses each layer of the network, calculating the L2 norm of each convolutional kernel. The L2 norm is one of the commonly used norms; the most frequently used Euclidean distance is an example of an L2 norm, defined as follows:

[0065]

[0066] This represents the square root of the sum of the squares of the vector elements.

[0067] After calculating the L2 norm, select N with the lowest L2 norm. i+1 *P i Prune the convolutional kernels by setting some kernel parameters to 0. After all the loops are finished, the model weights will contain some convolutional kernels with values ​​of 0. Removing these kernels will give you the pruned model weights.

[0068] Step 5: After pruning, the parameters of the obtained LLFDN model need to be fine-tuned, and the lightweight LLFDN model is then trained to improve the detection accuracy of LLFDN, obtaining the optimal LLFDN model. The parameters of the pruned LLFDN model are then modulated again to improve the detection accuracy of LLFDN, obtaining the optimal LLFDN model. The image of the spatial non-cooperative target to be identified is input into the pre-trained optimal LLFDN model, and the optimal LLFDN model is used to achieve real-time and high accuracy, and strong anti-interference ability of spatial non-cooperative target local feature recognition.

[0069] The above detailed description further illustrates the purpose, technical solution, and beneficial effects of the invention. It should be understood that the above description is only a specific embodiment of the present invention and is used to explain the present invention. It is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A lightweight method for identifying local features of non-cooperative targets, characterized in that: Includes the following steps, Step 1: Construct a ground-based data acquisition platform for non-cooperative space targets using a darkroom environment, spacecraft model, movable six-degree-of-freedom turntable, and cameras. Simulate the space environment using the darkroom environment, place the spacecraft model on the movable six-degree-of-freedom turntable, continuously adjust the position and attitude of the spacecraft model to simulate the spacecraft's attitude trajectory motion, use multiple light sources to simulate the actual lighting conditions in space, and collect image data of the spacecraft through cameras in fixed positions to construct a dataset of non-cooperative space targets. Step 2: Expand the database using data augmentation methods. Using a ground-based simulated space environment background as the space background, extract some image data and add Earth images as the background for the spacecraft data images. Considering the complex characteristics of deep space lighting, noise, and environment, the dataset is expanded using two data augmentation methods: photometric distortion and geometric distortion. For photometric distortion, the brightness, saturation, and noise of the images are enhanced to adapt to the effects of lighting and noise in space. When handling geometric distortion, random scaling, cropping, translation, shearing, and rotation are added to enhance the generalization performance and robustness of the lightweight local feature recognition model LLFDN. Then, the database is labeled and divided into training, validation, and test sets. Step 3: Construct a lightweight non-cooperative target local feature detection network. Use lightweight modules of ShuffleNetV2 stacked as the backbone network of LLFDN. Utilize depthwise separable convolutions in ShuffleNet to reduce model complexity, and use channel shuffling to improve image feature extraction. Construct the LLFDN backbone network by stacking n lightweight modules of ShuffleNetV2, for a total of m layers, and add a conv_bn_relu_maxpool layer at the beginning of the backbone network. Step 4: Use the LLFDN model pruning method. During training, use regularization to calculate the LP norm of each convolutional kernel, set the parameters of some convolutional kernels to 0, and continuously update the number of pruned convolutional kernels as the LLFDN model is trained. Select and remove unimportant weight parameters in the LLFDN model, and remove the convolutional kernels with parameters of 0 in the trained model to obtain a lightweight LLFDN model. This reduces the number of parameters and computation of the LLFDN model, while ensuring that the accuracy of the model is not affected. Step 5: After pruning the LLFDN model, the parameters of the obtained LLFDN model need to be fine-tuned, and then the lightweight LLFDN model is trained to improve the detection accuracy of LLFDN and obtain the optimal LLFDN model. The parameters of the pruned LLFDN model are then modulated to improve the detection accuracy of LLFDN and obtain the optimal LLFDN model. The image of the spatial non-cooperative target to be identified is input into the pre-trained optimal LLFDN model, and the optimal LLFDN model is used to identify the local features of the spatial non-cooperative target.

2. The lightweight non-cooperative target local feature recognition method as described in claim 1, characterized in that: In step three, ShuffleNetV2 reduces the number of model parameters through four strategies: minimizing memory access, reducing the use of group convolutions, network ensembles, and reducing element-wise operations. It also uses depthwise separable convolutions to further reduce the number of parameters. The number of parameters for depthwise separable convolutions is: D k *D k *M+M*N The computational workload is: D k *D k *M*W out *H out +M*W out *H out *N Using the method of stacking ShuffleNetV2 to build the backbone network of the model can significantly reduce the number of parameters and computational cost of the LLFDN model; A feature pyramid network is constructed to integrate multi-scale feature information. The features extracted from the backbone are upsampled and fused with the information of the previous layer. The edge and position information of local features are fused with the rich texture information of the previous layer to improve the extraction capability of local features. An attention module is used in front of the feature pyramid network to perform multi-scale fusion prediction on the extracted target features, resulting in fused features at multiple scales. This improves the ability of the multi-scale feature fusion network to extract features of non-cooperative local targets in space, thereby enhancing the detection accuracy of the LLFDN model. Using Soft-NMS as a novel nonmaximum suppression method, more bounding boxes are preserved, represented as By preserving more predicted bounding boxes, Soft-NMS can better capture local feature information and improve the ability to extract local features when performing spatial non-cooperative target local feature recognition, due to the large number of local features and the existence of occlusion problems.

3. The lightweight non-cooperative target local feature recognition method as described in claim 2, characterized in that: In step four, A set of LP norms are defined as follows: Where: x i It is an n-dimensional vector, where p is the order of the norm.

Images