A method for identifying components of a spatial non-cooperative target based on light weight and attention mechanism

Abstract

The application discloses a space non-cooperative target component recognition method based on light weight and attention mechanism, and mainly comprises the following steps: firstly, three-dimensional software is used to generate image data of a space target, part of which is directly labeled as a data set, and the other part is used as input of a style transfer algorithm together with a real space target image for data enhancement, and the enhanced image data is labeled again, and the generated image data and the image directly generated by the three-dimensional software are used as the final data set; secondly, a light weight network model is constructed, the model is optimized by using an attention mechanism, and the model is trained by using a training data set; finally, the trained light weight network model is used for space target component recognition, and a final recognition result is obtained. The method can solve the feature component recognition problem under the conditions of difficulty in feature extraction of multi-modal targets in a complex space environment, limited computing resources of a spaceborne platform and the like, has the characteristics of small memory occupation and high recognition precision, and can realize effective recognition of five typical components (a sailboard, a main body, a tripod, a docking ring and a nozzle).

Description

Technical Field

[0001] This invention relates to the field of space target recognition, specifically, to a method for recognizing non-cooperative space target components based on lightweight and attention mechanisms. Background Technology

[0002] The detection and identification of key components of non-cooperative spacecraft is one of the key technologies for on-orbit servicing. On-orbit servicing missions mainly include refueling, on-orbit assembly, and space debris removal. Before conducting repairs or cleanup using autonomous on-orbit servicing, the primary task is to observe the operational status of the satellite target, clarifying its component composition and distribution. The observation results determine the success of the final servicing operation; therefore, research on imaging target identification technology is of great significance. With the continuous development of space target approach technology, various information about the target can typically be obtained using optical or radar detection techniques. This information can then be used to infer satellite structure and motion parameters. Compared with radar detection technology, optical-based detection technology has advantages such as high spatial resolution, relatively mature technology, and easily understandable imaging results. Furthermore, the equipment has a simple structure, low power consumption, and strong stealth during detection.

[0003] Visual perception is a crucial component of optical detection. Before the rise of deep learning-based target recognition technologies, most methods for identifying cooperative spacecraft components relied on prior knowledge and manually designed features, primarily falling into two categories: template matching-based and traditional target recognition. Compared to cooperative spacecraft, non-cooperative spacecraft lack features such as cooperative markers for measurement or specific operational equipment, posing challenges to traditional satellite component identification based on prior information. Furthermore, the identification of critical non-cooperative spacecraft components based on visual measurement is affected not only by conventional conditions like target structure, size, and surface materials, but also by rapid spacecraft movement and attitude changes, as well as interference from factors like lighting intensity and space background in complex environments. Therefore, traditional methods are somewhat lacking in accuracy, robustness, and computational efficiency. Deep learning-based target feature detection methods have developed rapidly in recent years. Driven by massive amounts of data, these algorithms offer significant advantages in extracting global features and background information from images, as well as handling deformation, lighting changes, occlusion, and other complex conditions.

[0004] Patent CN 112580407 A proposes a spatial target component recognition method based on a lightweight network model. This patent uses 3D simulation software to build a self-constructed spatial image dataset and trains a lightweight network model to recognize three types of spatial target components. However, this patented method can only recognize a limited number of model component types. Furthermore, the data generated by the 3D software differs somewhat from image data in real-world applications, hindering model generalization. Additionally, this patent only considers lightweight improvements to the recognition algorithm without optimizing recognition accuracy, resulting in relatively low accuracy.

[0005] In addition to considering the recognition accuracy of the model, the spaceborne platform has limited storage resources and high requirements for processing efficiency. Some existing high-precision target recognition models have large weights and occupy a lot of storage and computing resources. Therefore, such models cannot be directly used for target feature component recognition applications in the space environment. The recognition model needs to be lightweighted and improved. Summary of the Invention

[0006] This invention provides a method for identifying non-cooperative space target components based on lightweight and attention mechanisms. Its purpose is to solve the above-mentioned problems and provide a lightweight and efficient identification network to identify five typical components of non-cooperative spacecraft (solar panels, body, tripod, docking ring, and nozzle).

[0007] The present invention discloses a method for identifying non-cooperative spatial target components based on lightweighting and attention mechanisms, the method comprising the following steps:

[0008] Step (1): Generate space target image data containing five typical components of a non-cooperative spacecraft, namely the solar panel, body, tripod, docking ring, and nozzle, through 3D software rendering simulation. Use annotation software to annotate the target categories and locations, and use this as part of the dataset for training and subsequent testing of the lightweight network model.

[0009] Step (2): Using the spatial target images generated in part of step (1) and the target images taken in the dark room as input, train the style transfer neural network. Use the trained style transfer network to convert the style of the images generated by this part of the 3D software into an image style that is closer to the real scene. Replace the corresponding images in the original (1) with these images, and use annotation software to annotate the target category and location as a dataset for training the lightweight network model and subsequent testing.

[0010] Step (3): Lightweight network model construction and optimization, weight initialization and hyperparameter setting; load training dataset to train the improved lightweight network model, and retain the weight with the highest mAP on the validation set as the model weight.

[0011] Step (4): Based on the weight results retained from the training in step (3), use them as model weights during testing, load the spatial target test dataset, perform component recognition, and obtain the recognition results of five typical components (sailboard, body, tripod, docking ring, and nozzle).

[0012] The spatial target image simulated by 3D software rendering in step (1) above mainly includes model import, model component separation, material addition, lighting condition setting, camera parameter setting, and orbit constraint setting. LabelImg image annotation software is used to label the target category and location in the image.

[0013] The construction of the lightweight network model in step (3) above mainly includes the following three aspects:

[0014] a. Based on the YOLOv5s object detection model framework, a channel sparse training method is adopted to reduce the overall weight of the model.

[0015] b. Improve the model by using a feature fusion layer and SK-Net to enhance feature attention at the channel level and convolutional kernel level, respectively.

[0016] c. Use the transformer module to improve the model, enhancing its self-attention and its potential for recognizing multimodal targets.

[0017] Compared with existing technologies, the significant advantages of this invention are: (1) It provides a high-precision, small-volume spatial non-cooperative target feature component recognition method based on deep learning. After generating the dataset using 3D software, a style transfer algorithm is used for data augmentation to approximate the distribution of training and test data, thereby improving the model's generalization performance. (2) The model proposed in this invention can accurately identify components of five types of spatial non-cooperative targets, increasing the number of recognizable components. (3) In terms of model optimization, this invention improves the model's lightweight design while also enhancing the model's recognition accuracy through an attention mechanism, resulting in better recognition performance. Attached Figure Description

[0018] Figure 1 This is a flowchart illustrating the overall process of the method of the present invention.

[0019] Figure 2 This forms the basic framework of the improved lightweight network model of the method of this invention.

[0020] Figure 3 This is a schematic diagram of the channel sparse training method used in this invention.

[0021] Figure 4 This is a schematic diagram of the feature fusion layer module used in this invention.

[0022] Figure 5 This is a schematic diagram of the SK-Net module used in this invention.

[0023] Figure 6 This is a schematic diagram of the transformer module used in this invention.

[0024] Figure 7 This is an example of the recognition performance of the present invention on a spatial non-cooperative target dataset.

[0025] Figure 8 This is an example of the recognition effect of the present invention on real-space images. Detailed Implementation

[0026] The technical solution of the present invention will be further described below with reference to the accompanying drawings and preferred embodiments.

[0027] See appendix Figure 1 The present invention provides a spatial component recognition method based on lightweight networks and attention mechanisms, the specific implementation steps of which are as follows:

[0028] Step (1): Generate spatial target image data through 3D software rendering simulation, and use annotation software to annotate the target category and location as part of the dataset for training and subsequent testing of the lightweight network model.

[0029] Step (2): Using the spatial target images generated in part of step (1) and the target images taken in the dark room as input, train the style transfer neural network. Use the trained style transfer network to convert the style of the images generated by this part of the 3D software into an image style that is closer to the real scene. Replace the corresponding images in the original (1) with these images, and use annotation software to annotate the target category and location as a dataset for training the lightweight network model and subsequent testing.

[0030] Step (3): Lightweight network model construction and optimization, weight initialization and hyperparameter setting; load training dataset to train the improved lightweight network model, and retain the weight with the highest mAP on the validation set as the model weight.

[0031] Step (4): Based on the weight results retained from the training in step (3), use them as model weights during testing, load the spatial target test dataset, perform component recognition, and obtain the recognition results of five typical components (sailboard, body, tripod, docking ring, and nozzle).

[0032] The basic framework of the improved lightweight network model constructed is as follows: Figure 2 As shown.

[0033] The designed network model is based on YOLOv5s. Feature fusion layer modules and SK-Net modules are used in the backbone and neck parts to enhance the model's ability to handle multimodal target recognition problems and improve detection accuracy. Furthermore, transformer modules are introduced after the three output channels in the neck to explore the model's recognition potential. The model architecture of this invention consists of four parts. The backbone mainly consists of CSP_BottelNeck, CBL layer, and SPP layer, extracting features from low, medium, and high scales, and then providing the obtained features to PANet for feature aggregation. The three outputs are then input into the corresponding transformer modules for further feature extraction and fusion. Finally, each output of the transformer module is passed to the YOLO detection head to perform target classification and localization.

[0034] To deploy the model on resource-constrained platforms, this invention employs a channel sparsity training method. The scaling factor of the BatchNorm layer in the feature extraction network is L1 regularized, and then channels are pruned based on the training results to reduce the model size. The principle diagram is shown below. Figure 3 As shown.

[0035] Batch normalization (BN) is an algorithm that improves the convergence speed and stability of deep neural networks. As a standard method, it can solve the gradient vanishing problem during training while achieving better generalization performance. BatchNorm normalizes the internal activation values ​​using mini-batch input data. The BN layer performs the following transformation:

[0036]

[0037] Z in and Z out This represents the input and output of the BacthNorm layer, where B represents the current mini-batch, and μ... B and σ B The mean and standard deviation of the input activations in the current batch are given. γ and β are learnable affine transformation parameters (scaling and translation) that allow the network to retain information about the original features. The underlying principle of channel sparse training is to jointly train the network weights and the scaling factor γ, the latter using L1 regularization. Finally, using the trained γ as a measure of channel importance, channels with γ less than a certain threshold are pruned. The objective function is as follows:

[0038]

[0039] (x,y) represents the input and output, W represents the trainable weights, the first summation term is the normal network training loss, and g(-) is the scaling factor penalty caused by sparsity. λ is the balancing coefficient, which balances the importance of the two terms. In the experiment, g(s) = |s| is chosen, which is L1 regularization.

[0040] The network can automatically identify unimportant channels during training. Channels with a small scaling factor (orange) will be pruned, which can be achieved by removing the input and output connections of all channels. Therefore, a simplified version of the network can be obtained by retaining the important feature channels and removing those that contribute little to the network.

[0041] To improve the model's ability to handle multimodal problems, this invention uses a feature fusion layer and SK-Net to help the model resist easily confused feature information and improve the model's recognition performance.

[0042] The main idea of ​​the feature fusion layer is to enhance the channel-level attention of the network, such as... Figure 4 As shown, a one-dimensional vector corresponding to the input feature map is generated through global pooling. The importance of each feature channel is adaptively learned through a fully connected network, generating a multi-scale fusion coefficient vector that matches the original feature map. Finally, to obtain the output of the feature fusion layer, the fusion coefficient vector and the corresponding input feature map are multiplied point-by-point, and then the feature maps obtained in the previous step are added point-by-point.

[0043] SK-Net is a simple yet powerful attention module. It is also a lightweight module that can be integrated into most common convolutional neural network architectures for end-to-end training. Figure 5 As shown, given a feature map, multiple branches with different convolutional kernel sizes are fused using softmax attention guided by information from these branches. The effective receptive field size of neurons in the fusion layer depends on the network's attention to these branches. SK-Net improves the network's attention to convolutional kernels to some extent.

[0044] Considering the potential of self-attention mechanisms in visual tasks, this invention attaches a transformer module to each of the three output channels of the network's Neck section, as shown in the following module structure. Figure 6As shown, each transformer module has two sub-layers. The first sub-layer is a multi-head attention layer, and the second sub-layer (MLP) is a fully connected layer. Additionally, residual connections are used to connect each sub-layer. Compared to traditional transformer modules, this invention removes the LayerNorm module to improve computational speed. Since the feature map generated at the neck end has low pixel counts after complex feature extraction and fusion processes, adding a transformer module here significantly reduces memory consumption compared to adding it to earlier parts of the network.

[0045] The improvements and optimizations to the network in this invention are mainly in the following aspects:

[0046] (1): Retain the basic framework structure of the model and use the channel sparse training method to reduce the model size so that it can be deployed on resource-constrained platforms.

[0047] (2): The feature fusion layer and SK-Net are used respectively to enhance the model’s attention at the channel level and convolution kernel level, thereby enhancing the model’s ability to process multimodal target recognition and improving the model’s recognition accuracy.

[0048] (3): The transformer module is used to enhance the model’s self-attention and improve the model’s recognition potential.

[0049] Example:

[0050] To demonstrate the effectiveness of the algorithm of this invention, the following experiment was conducted to identify spatial target components:

[0051] (1) Generation of spatial target image data

[0052] Due to various limitations, it is difficult to obtain a large number of real spatial target images required for training deep neural networks. Therefore, some spatial target images are simulated by rendering with 3D software. The specific process is as follows:

[0053] Step 1: Import CAD Models. Download some of the target model files (Chang'e-2 satellite model, TDRS satellite model) publicly available on the aerospace agency's website and import them into 3D software.

[0054] Step 2: Separate Model Components. Since the satellite CAD model is imported as a single structure, different material textures need to be applied to different parts when adding materials later. Therefore, it is necessary to first separate the various components of the imported model. Considering the specific focus of this study, the satellite CAD model components are divided into five categories: solar panels, body, tripod, docking ring, and nozzles.

[0055] Step 3: Adding Materials. Rendering the materials of the CAD model is achieved through texture mapping. Prior knowledge and literature review reveal that the materials, surface coverings, and colors of various satellite components typically differ. For example, aluminum alloy stiffened plates or sandwich panel structures were commonly used in early solar panels; later, aluminum honeycomb sandwich structures with graphite fiber composite panels were adopted. More advanced lightweight solar panels use graphite fiber composite materials as the frame, covered with a polyamide film. Solar wings with larger flexible areas are entirely made of thin-film materials. The satellite's main surface is often covered with a temperature-controlled coating, utilizing thermal radiation or absorption characteristics to regulate temperature. Taking all factors into consideration, materials were added to the various satellite model components in this study based on currently common satellite surface materials.

[0056] Step 4: Lighting Condition Settings. When acquiring images in a space environment, sunlight conditions are extremely important, as the imaging effects vary greatly under different lighting conditions. Therefore, in the rendering simulation, a parallel light source with an ultra-long distance is used to simulate sunlight.

[0057] Step 5: Camera parameter settings. The data required for this invention is image data captured by an optical camera. Therefore, in order to make the obtained simulated images closer to those of real space-based satellites, an optical camera is added during rendering, with an imaging resolution of 6600×4440.

[0058] Step 6: Orbit Constraint Setting. To improve the model's generalization performance, it is necessary to acquire images of the satellite model from multiple angles and in different attitudes. In addition to the satellite's own rotation, the camera's orbital flight provides multi-angle and multi-directional imaging effects of the satellite under the current attitude conditions. Therefore, the camera's orbital radius is set to 30km, and orbital flight and imaging are performed with the target satellite as the center.

[0059] (2) Dataset Augmentation

[0060] This invention uses satellite images taken in a darkroom as test images. To improve the generalization performance of the model, a style transfer-based data augmentation method is employed. A portion of the satellite images generated by 3D software rendering, along with the darkroom satellite images, are selected as input to the Cycle-GAN style transfer algorithm to train the model. The trained model is then used to convert the style of the simulation software-generated images to a style closer to real images. Finally, these images replace the original simulation images to form the final dataset for subsequent model training and validation testing.

[0061] (3) Model training and parameter setting

[0062] This study utilizes the PyTorch deep learning framework (version 1.9) and Python 3.7. The server configuration includes an Intel Xeon Silver 4210 CPU and an NVIDIA Quadro RTX 6000 GPU. Common machine learning and image processing libraries such as NumPy and OpenCV-Python are used to build an improved lightweight network model, which is trained on a self-built sample dataset. The optimization strategy used during training is Adam. The dataset contains 6360 images, divided in an 8:1:1 ratio: 5088 images for training, 636 images for validation, and 636 images for testing.

[0063] Training parameter settings: The number of iterations is set to 100, the input image resolution is 1280×1280, and the batch size is 16 images. The initial learning rate is set to 0.5 and 0.999, and cosine annealing decay is used.

[0064] (4) Analysis of experimental results

[0065] To verify the effectiveness of the method of this invention, Mean Average Precision (mAP) was used to quantitatively test the recognition accuracy of the constructed model. Precision (p) describes the ability of the classification model to return only relevant objects, and thus represents the proportion of correctly classified samples among all results. The formula for calculating precision is as follows:

[0066]

[0067] Recall (r) describes the ability of a classification model to identify all relevant targets, thus representing the proportion of all targets correctly predicted. The formula for recall is as follows:

[0068]

[0069] TP, FP, and FN represent the number of correctly classified positive examples, misclassified positive examples, and misclassified negative examples, respectively. The AP for each class is obtained by integrating the PR (precision-recall) curves, and the average AP across all classes is mAP.

[0070] The calculation formula is as follows:

[0071]

[0072] In the above formula, 'a' represents the number of categories in the sample.

[0073] The method of this invention was used to detect and identify non-cooperative targets in space on a test dataset. Experimental results are reported as mAP@.5:.95 (the average of all 10 IoU thresholds, ranging from [0.5 to 0.95]) and mAP50 (IoU = 0.5).

[0074] To demonstrate the advantages of the proposed model in terms of detection accuracy and model size, experiments were conducted under the same conditions using the standard YOLOv5s model and several mainstream target recognition algorithms. The comparative experimental results shown in Table 1 demonstrate that the improved lightweight network target recognition model YSFCR constructed by the method of this invention significantly reduces the model size while maintaining high detection accuracy, outperforming the other methods compared. Furthermore, to further verify the model's generalization performance, this study used real satellite images not present in the dataset to validate the model, such as… Figure 8 As shown, the results demonstrate that the model of this invention exhibits excellent generalization performance and possesses the capability for practical deployment.

[0075] Table 1 shows the relevant parameters of the embodiments of the present invention and the comparison results with other commonly used algorithms on spatial non-cooperative target datasets.

[0076] Table 1

[0077]

[0078] Finally, it should be noted that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for identifying components of a non-cooperative space target based on lightweight and attention mechanism, characterized in that, The method comprises the following steps: Step (1), generate space target picture data containing five typical components of space non-cooperative spacecraft, such as sail, body, tripod, docking ring and nozzle, through three-dimensional software rendering simulation, and use labeling software to label target categories and positions as part of the data set for training and subsequent testing of the lightweight network model; Step (2), use part of the space target pictures generated in step (1) and target images taken in a darkroom as input to train a style transfer neural network, use the trained style transfer network to convert the style of these three-dimensional software generated images to a style closer to the real scene, replace the corresponding images in step (1) with these images, and use the labeling software to label the target categories and positions as a data set for training and subsequent testing of the lightweight network model; Step (3), lightweight network model construction and optimization, add feature fusion layer module and SK-Net module to the Backbone part and Neck part of the Yolov5s network main part, and add transformer module after the three output channels of the main network; Weight initialization and hyperparameter setting are performed; Load the training data set to train the improved lightweight network model, and reserve the weight with the highest mAP of the validation set as the model weight; Step (4), according to the weight result reserved in step (3), use it as the model weight during testing, load the space target test data set, and perform component recognition to obtain the recognition results of the five typical components, such as sail, body, tripod, docking ring and nozzle.

2. The method of claim 1, wherein: The construction of the lightweight network model in step (3) comprises the following three aspects: a. Based on the target detection model YOLOv5s framework, the channel sparse training method is used to reduce the overall weight size of the model; b. Use feature fusion layer and SK-Net to improve the model, respectively improve the feature attention of the model at the channel level and the convolution kernel level; c. Use the transformer module to improve the model, improve the self-attention of the model and the recognition potential for multi-modal targets.

Images