Lightweight spacecraft pose estimation method based on high-resolution network

Abstract

The invention relates to a spacecraft pose estimation method, in particular to a lightweight spacecraft pose estimation method based on a high-resolution network, and solves the technical problem that in an existing spacecraft pose estimation method, key point regression is difficult to meet precision and efficiency requirements at the same time. According to the method, the key point regression model is based on a high-resolution network, depth separable convolution is adopted in the feature extraction process, the parameter quantity and the calculation quantity are greatly reduced, meanwhile, a mixed attention mechanism is combined, the high expression ability is reserved in feature extraction, the lightweight degree is further improved, and the method is suitable for large-scale popularization and application. The balance of precision and efficiency is realized; in the feature fusion process, cavity convolution is adopted to enlarge a receptive field, global information in an image can be captured, the pixel position of an occluded key point can be more accurately deduced, then key local and global features are focused in combination with a mixed attention mechanism, the effectiveness of feature representation is further improved, and the regression precision of the key point is improved.

Description

Technical Field

[0001] The invention relates to a spacecraft pose estimation method, and in particular to a lightweight spacecraft pose estimation method based on a high-resolution network. Background Art

[0002] Measuring the relative position and attitude of spacecraft is crucial for related space missions, such as space rendezvous, docking, defensive maneuvers, and on-orbit services. Vision-based spacecraft pose estimation methods are mainly divided into geometric feature-based methods and template matching-based methods according to the measurement method. The geometric feature-based method is a relatively mature solution in the current research on spacecraft pose estimation. It mainly determines the geometric features on the spacecraft first, and then extracts the target features from the image containing the spacecraft through feature extraction technology, and matches it with the existing three-dimensional features of the spacecraft, and transforms it into solving the PnP (Perspective-n-Point, perspective n-point solver) problem to obtain the relative pose. Therefore, feature extraction of spacecraft is a key step in this process.

[0003] When extracting spacecraft features, for spacecraft with obvious geometric features, strong geometric features such as rectangles and circles can be extracted. For spacecraft with unclear geometric features, classic SIFT (Scale Invariant Feature Transform), SURF (Speeded Robust Feature Method), ORB (Orirnted FAST and Rotated BRIEF, a method that combines FAST key point detection and BRIEF descriptor) and other methods can be used to complete feature extraction. Naasz et al. used Sobel filter to extract edges in the pose estimation system of the Hubble Space Telescope Servicing Mission 4 (HST-SM4) and verified the robustness of the algorithm under different lighting conditions. However, the above methods perform poorly in terms of stability and robustness when facing complex lighting environments, weak surface texture of the target spacecraft, or occlusion caused by posture changes.

[0004] Deep learning refers to learning more complex nonlinear mapping relationships in a supervised, semi-supervised, or unsupervised manner by increasing the number of neural network layers to extract more advanced features. Deep convolutional neural networks have developed rapidly in computer vision fields such as ground pose estimation due to their powerful feature extraction and expression capabilities, and have gradually attracted the attention of researchers in the field of spacecraft pose estimation.

[0005] At present, the research on spacecraft pose estimation methods based on deep convolutional neural networks mainly focuses on the regression of key points of spacecraft. There are two main methods: (1) using the fully connected layer to directly regress the key point coordinates; (2) using the method of predicting Gaussian heat maps, and then finding the index corresponding to the peak value as the key point coordinates. The method of directly regressing the key point coordinates using the fully connected layer depends on the distribution of the training data, which is easy to cause overfitting during training, thereby losing spatial generalization ability. In contrast, full convolution has better spatial generalization ability. The method of predicting Gaussian heat maps can not only realize full convolution, but also learn the correlation between key points and key points, and between key points and image background.

[0006] In order to promote the application of spacecraft pose estimation methods based on deep convolutional neural networks in actual space missions, Park and Sharma et al. decoupled the spacecraft pose estimation network structure into the spacecraft target detection network and the key point regression network to determine the bounding box coordinates of the spacecraft and the two-dimensional coordinates of the key points respectively. Huo et al. proposed a lightweight network framework based on tiny-YOLOv3 to predict the two-dimensional coordinates of the key points of the projected three-dimensional model of the constructed spacecraft, modified the output of tiny-YOLOv3, established a box reliability judgment mode for detecting the RoI area of ​​the spacecraft, and replaced the fully connected layer with a convolutional layer to output a Gaussian heat map to achieve the regression of the key points of the spacecraft. B. Chen et al. used Faster R-CNN based on a high-resolution network to detect the RoI area of ​​the spacecraft in the input image, then cropped and resized the RoI area, and then used a high-resolution network to regress the key points. However, the above methods did not consider the situation where the key points were occluded due to pose changes when detecting key points, and the problem of large parameters and memory usage caused by the complexity of the high-resolution network model, making it difficult to meet the requirements of accuracy and efficiency at the same time. Summary of the invention

[0007] The purpose of the present invention is to solve the technical problem that key point regression in existing spacecraft pose estimation methods is difficult to meet the requirements of accuracy and efficiency at the same time, and to provide a lightweight spacecraft pose estimation method based on a high-resolution network.

[0008] To achieve the above object, the technical solution adopted by the present invention is: A lightweight spacecraft pose estimation method based on a high-resolution network is special in that it includes the following steps: Step 1: Select a spacecraft dataset, and then preprocess it to obtain an initial dataset; the spacecraft dataset consists of spacecraft images with pose labels; Step 2: Select key points on the spacecraft, and then use the spacecraft images and their pose labels in the initial data set to perform multi-viewing of the spacecraft. Figure 3 Dimensional reconstruction is used to obtain the three-dimensional coordinates of the key points of the spacecraft, and then the two-dimensional coordinates of the key points of the spacecraft image are obtained by using the projection equation, and they are annotated as training labels in the spacecraft images of the initial data set to obtain the training data set; Step 3: Establish a key point regression initial model based on the high-resolution network, and input the training data set into the key point regression initial model; Step 4: Use the training data set to train the key point regression initial model to obtain the key point regression model; Step 5: Obtain the target spacecraft image and input it into the key point regression model. The key point regression model uses deep separable convolution combined with attention mechanism to extract the key point features of the target spacecraft image, and then uses void convolution combined with hybrid attention mechanism to fuse the key point features to obtain the predicted two-dimensional coordinates of the key points of the target spacecraft image. Step 6: Perform pose estimation based on the predicted two-dimensional coordinates of the key points of the target spacecraft image to obtain the predicted pose of the target spacecraft, thereby completing the pose estimation of the spacecraft.

[0009] Furthermore, in step 3, the key point regression initial model is established based on the high-resolution network, specifically: The Basicblock unit of the feature extraction module in each Stage structure of the high-resolution network is replaced with the ShuffleNet-v2-Block unit based on the SE attention mechanism, and the convolution unit of the feature fusion module in each Stage structure of the high-resolution network is replaced with a hollow convolution unit, and then the scSE module is added between the feature extraction module and the feature fusion module of each Stage structure of the high-resolution network; The ShuffleNet-v2-Block unit based on the SE attention mechanism includes an equal mapping channel and a depth convolution channel parallel to each other, and the depth convolution channel includes a first SE module, a 3×3 depth convolution module and a second SE module connected in sequence.

[0010] Furthermore, step 4 is specifically as follows: Step 4.1, extract features from a spacecraft image in the training data set to obtain an initial feature map; Step 4.2, convolve the initial feature map, add a scale branch, obtain the initial feature map of the two scale branches, and input it into the feature extraction module of the Stage structure; Step 4.3: The feature extraction module uses the ShuffleNet-v2-Block unit based on the SE attention mechanism to extract features from the initial feature map of each scale branch, obtains the deep feature maps of the two scale branches, and inputs them into the scSE module; Step 4.4, the scSE module performs spatial information calibration and channel information calibration on the depth feature maps of the two scale branches respectively to obtain a spatial information calibrated depth feature map and a channel information calibrated depth feature map, and then adds the two together to obtain a calibrated depth feature map of the two scale branches, and inputs it into the feature fusion module; Step 4.5, the hole convolution unit of the feature fusion module performs convolution processing on the calibrated depth feature maps of the two scale branches, extracts their global information, obtains all scale branches of the calibrated depth feature maps of the two scale branches, and then fuses the calibrated depth feature maps of the corresponding scale branches respectively to obtain the fused feature map of the two scale branches; Step 4.6: Use the fused feature map of the two scale branches obtained in step 4.5 as the initial feature map, and then return to step 4.2 until the fused feature maps of four different scale branches are obtained; Step 4.7, fusing the fused feature maps of the four different scale branches to obtain a feature map of the spacecraft image; Step 4.8, generate a key point prediction heat map based on the feature map of the spacecraft image, obtain the predicted two-dimensional coordinates of the key points of the spacecraft image, calculate the loss function value between the predicted two-dimensional coordinates of the key points of the spacecraft image and the corresponding training labels, and judge whether the loss function converges according to the loss function value. If the loss function converges, the key point regression model is obtained; otherwise, return to step 4.1 to process the next spacecraft image in the training data set.

[0011] Furthermore, step 4.3 is specifically as follows: 4.3.1. The ShuffleNet-v2-Block unit based on the SE attention mechanism performs channel separation on the initial feature maps of the two scale branches, and inputs the initial feature maps of each scale branch into the equal mapping channel and the deep convolution channel respectively. The equal mapping channel performs equal mapping on the initial feature maps of the two scale branches respectively, and obtains the mapping feature maps of the two scale branches; 4.3.2. The first SE module of the deep convolution channel performs adaptive average pooling on the initial feature maps of the two scale branches, scales them to the minimum scale, and then splices them. Then, 1×1 convolution, ReLU activation, and Sigmoid activation are performed in sequence to obtain the weights corresponding to the two scale branches. Then, they are upsampled to the corresponding scale branches, and the weight maps corresponding to the initial feature maps of the two scale branches are obtained. 4.3.3. According to the weight maps corresponding to the initial feature maps of the two scale branches, the initial weighted feature maps of the two scale branches are calculated by the following formula, and then sent to the 3×3 depth convolution module of the depth convolution channel:

[0012] in, represents the initial weighted feature map of the h-th scale branch, 1≤h≤2, Represents the weight map corresponding to the initial feature map of the h-th scale branch, represents the initial feature map of the h-th scale branch, It means element-wise multiplication; 4.3.4. The 3×3 deep convolution module of the deep convolution channel performs 3×3 deep convolution operations on the initial weighted feature maps of the two scale branches respectively to obtain the deep convolution feature maps of the two scale branches, and sends them to the second SE module of the deep convolution channel; 4.3.5. The second SE module of the deep convolution channel respectively processes the deep convolution feature maps of the two scale branches through global average pooling, fully connected layer, ReLU activation, fully connected layer, Sigmoid activation and Ws weight calculation in sequence to obtain the weight maps corresponding to the deep convolution feature maps of the two scale branches; 4.3.6. According to the weight maps corresponding to the deep convolution feature maps of the two scale branches, the weight maps corresponding to the deep convolution feature maps of the two scale branches are element-wise multiplied with the deep convolution feature maps of the two scale branches to obtain the deep convolution weighted feature maps of the two scale branches; 4.3.7. Concatenate the mapping feature maps of the two scale branches obtained in step 4.3.1 and the deep convolution weighted feature maps of the two scale branches obtained in step 4.3.6 respectively, and then perform channel mixing and channel shuffling in sequence to obtain the deep feature maps of the two scale branches, and input them into the scSE module.

[0013] Furthermore, step 2 is specifically as follows: Step 2.1, select M key points on the spacecraft, where M is an integer, M≥5; Step 2.2, select N spacecraft images in the initial data set, and respectively mark the two-dimensional coordinates of M key points of the N spacecraft images, and then use the particle swarm optimization algorithm to optimize the two-dimensional coordinate reprojection errors of the M key points of the N spacecraft images to obtain the three-dimensional coordinates of the M key points of the spacecraft; wherein N is an integer, N ≥ 10; Step 2.3: According to the three-dimensional coordinates of the M key points of the spacecraft and the pose label of each spacecraft image in the initial data set, the projection equation is used to calculate the two-dimensional coordinates of the M key points of each spacecraft image in the initial data set, and the two-dimensional coordinates are marked in the initial data set as training labels to obtain the training data set.

[0014] Furthermore, step 2.2 is specifically as follows: 2.2.1. Select N spacecraft images from the initial data set and mark the two-dimensional coordinates of M key points of the N spacecraft images respectively; 2.2.2. According to the two-dimensional coordinates of the M key points in the N spacecraft images, the projection equation is used to calculate the feasible solutions of the three-dimensional coordinates of the M key points of the spacecraft; 2.2.3. Using the particle swarm optimization algorithm, the feasible solutions of the three-dimensional coordinates of the M key points of the spacecraft are iterated respectively through the following formulas, and the global optimal solutions of the feasible solutions of the three-dimensional coordinates of the M key points of the spacecraft are searched respectively according to the reprojection errors of the two-dimensional coordinates of the corresponding key points:

[0015]

[0016]

[0017] in, t Indicates the current iteration number, represents the total number of iterations, k=1,2,3,…,K, K is the number of feasible solutions for the three-dimensional coordinates of the key points of the spacecraft, , Respectively expressed in t Second and t +1 iteration of the spacecraft key point k The update speed of feasible solutions of three-dimensional coordinates, , Respectively expressed in t Second and t +1 iteration of the spacecraft key point k The current position of the feasible solution in three dimensions, The key points of the spacecraft are represented by k The individual optimal position of feasible solutions in three-dimensional coordinates, is the optimal position of the population of feasible solutions for the three-dimensional coordinates of the key points of the spacecraft, It means to generate a random number between 0 and 1. refers to the individual learning factor, Refers to social learning factors; w represents the inertia weight, represents the upper bound of the inertia weight, Indicates the lower bound of the inertia weight; The two-dimensional coordinate reprojection error of the key point is calculated by the following formula:

[0018] in, Indicates i Spacecraft image j The 2D coordinate reprojection error of key points is 1≤ i ≤N,1≤ j ≤M, Indicates i Spacecraft image j The current position of the feasible solution of the three-dimensional coordinates of the key points, Indicates i Spacecraft image j The feasible solution of the three-dimensional coordinates of the key points is Indicates i Spacecraft image j The two-dimensional coordinates of the key points, proj () indicates that the three-dimensional coordinates are mapped to the imaging plane to obtain the two-dimensional coordinates. represents the square of the two-norm; 2.2.4. Compare the reprojection errors of the global optimal solutions of each feasible solution for the three-dimensional coordinates of the key points of the spacecraft, and take the global optimal solution of the feasible solution for the three-dimensional coordinates with the smallest reprojection error as the three-dimensional coordinates of the key points of the spacecraft, and obtain the three-dimensional coordinates of the M key points of the spacecraft respectively.

[0019] Furthermore, in step 3, the loss function of the key point regression initial model is:

[0020] in, is the loss function, Represents spacecraft image j The visibility of key points is provided by pose labels, v j =0 means the spacecraft image j Key points are outside the spacecraft image. v j =1 means the spacecraft image j Key points are blocked in the spacecraft image. v j =2 means the spacecraft image j The key points are visible. For spacecraft images jPrediction heatmap of key points, Represents spacecraft image j The predicted 2D coordinates of the key points, For spacecraft images j The real heat map of key points, Represents spacecraft image j The two-dimensional coordinates of the key points, h ( ) indicates solving the heat map.

[0021] Furthermore, in step 2.1, the key points are eight corner points and three antenna endpoints of the spacecraft.

[0022] Furthermore, in step 1, the preprocessing includes: adding random Gaussian white noise, adjusting brightness and contrast, and randomly flipping and rotating.

[0023] Compared with the prior art, the present invention has the following beneficial technical effects: 1. In the lightweight spacecraft pose estimation method based on high-resolution network provided by the present invention, deep separable convolution is used in the feature extraction process, which greatly reduces the number of parameters and calculations; at the same time, combined with the attention mechanism, it not only retains a high expression ability in feature extraction, but also further improves the degree of lightweight, thereby achieving a balance between accuracy and efficiency; 2. In the lightweight spacecraft pose estimation method based on high-resolution network provided by the present invention, the dilated convolution is used to expand the receptive field in the feature fusion process, which can capture the global information in the image, effectively utilize the geometric connection relationship between key points, and more accurately infer the pixel position of the occluded key points, which can improve the accuracy of key point regression in occluded scenes, thereby improving the accuracy of pose estimation; 3. In the lightweight spacecraft pose estimation method based on high-resolution network provided by the present invention, the hybrid attention mechanism is combined in the feature fusion process to complement the dilated convolution. The dilated convolution expands the receptive field to obtain global information, and the hybrid attention mechanism focuses on key local and global features, which can improve the effectiveness of feature representation and further improve the accuracy of key point regression, thereby further improving the accuracy of pose estimation; 4. In the lightweight spacecraft pose estimation method based on high-resolution network provided by the present invention, an scSE module is added between the feature extraction module and the feature fusion module, and a hybrid attention mechanism is introduced, which can simultaneously extract more useful spatial feature information and channel feature information, making the fusion of features of different scales more efficient; 5. In the lightweight spacecraft pose estimation method based on high-resolution network provided by the present invention, the ShuffleNet-v2-Block unit is used to connect two channels in series and then output the result, and then the channel mixing operation is used to ensure the effective association of the two channels. Compared with the pixel-by-pixel addition operation method, the application of element-level operations and group convolution can be reduced, and the computational efficiency of the key point regression model can be further improved; 6. In the lightweight spacecraft pose estimation method based on high-resolution network provided by the present invention, SE module is adopted, and the attention mechanism is introduced to perform channel weighting to replace the two 1×1 convolution operations in the ShuffleNet-v2-Block unit, so that the feature map is scaled to the minimum resolution, the number of parameters is reduced, and the computational complexity can be further reduced; 7. In the lightweight spacecraft pose estimation method based on high-resolution network provided by the present invention, a particle swarm optimization algorithm is used to solve the three-dimensional coordinates of the key points of the spacecraft, and a linearly decreasing inertia weight is used to balance the globality and convergence speed of the search, thus avoiding the problem that the traditional particle swarm optimization algorithm is prone to fall into the local optimum and converges too slowly in the later stage; 8. In the lightweight spacecraft pose estimation method based on high-resolution network provided by the present invention, image enhancement of spacecraft data set is performed by preprocessing means such as adding random Gaussian white noise, adjusting brightness and contrast, and random flipping and rotation, which can improve the sample type richness of the data set and enhance the generalization ability of the key point regression model. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 This is a schematic diagram of key points selected in step 2.1 of an embodiment of the present invention; Figure 2 This is a flow chart of the particle swarm optimization algorithm in step 2.2.3 of an embodiment of the present invention; Figure 3 The optimization curve diagram for solving the first key point in step 2.2.3 of the embodiment of the present invention; Figure 4 This is a schematic diagram of the three-dimensional coordinates of the key points obtained in step 2.3 of the embodiment of the present invention; Figure 5 This is the structure diagram of the existing spacecraft key point regression network based on high-resolution network; Figure 6 This is the structural diagram of the existing ShuffleNet-v2-Block unit; Figure 7 This is a schematic diagram of the operation of 3×3 depth convolution in the existing ShuffleNet-v2-Block unit; Figure 8This is a structural diagram of the ShuffleNet-v2-Block unit based on the SE attention mechanism used in step 3 of an embodiment of the present invention; Fig. 9 Schematic diagram of a dilated convolution kernel of a dilated convolution unit in step 3 of an embodiment of the present invention, wherein the expansion rate of (a) is 1, the expansion rate of (b) is 2, and the expansion rate of (c) is 4; Fig.10 This is a schematic diagram of the scSE module operation used in step 3 of an embodiment of the present invention; Fig.11 This is a training flow chart of the key point regression initial model in step 4 of the present invention; Fig.12 Schematic diagram of the first SE module performing channel weighting in step 4.3.2 of an embodiment of the present invention; Fig.13 The graphs are the change curves of the loss function and the key point regression accuracy during the training of the initial model of key point regression in the embodiment of the present invention, wherein (a) is the change curve of the loss function, and (b) is the change curve of the key point regression accuracy; Fig.14 The figures are schematic diagrams of the results of key point regression of target spacecraft images in different environments using an embodiment of the present invention, where (a) and (b) are dark environments, (c) and (d) are occluded, and (e) and (f) are complex backgrounds. DETAILED DESCRIPTION

[0025] The following is a further detailed description of a lightweight spacecraft pose estimation method based on a high-resolution network proposed by the present invention in conjunction with the accompanying drawings and specific embodiments. It should be understood by those skilled in the art that these embodiments are only used to explain the technical principles of the present invention and are not intended to limit the scope of protection of the present invention.

[0026] A lightweight spacecraft pose estimation method based on a high-resolution network comprises the following steps: Step 1: Select the spacecraft dataset SPEED+, which consists of spacecraft images with pose labels, and then preprocess it to obtain the initial dataset. The preprocessing includes adding random Gaussian white noise, adjusting brightness and contrast, and randomly flipping and rotating, which can improve the sample type richness of the dataset and enhance the generalization ability of the model.

[0027] The spacecraft dataset SPEED+ includes synthetic images, light box images, and sun lamp images. Synthetic images refer to spacecraft images synthesized by computers. Light box images refer to spacecraft images captured by simulating reflected lighting using a light box with a diffuser. Like synthetic images, the earth background is randomly added to the light box images. Sun lamp images refer to spacecraft images captured by directly simulating the high-intensity light of the sun. Compared with light box images, sun lamp images involve more challenging lighting effects, such as higher contrast and blurring of spacecraft boundaries and shapes caused by overexposure of the camera.

[0028] Step 2: Select key points on the spacecraft, and then use the spacecraft images and their pose labels in the initial data set to perform multi-viewing of the spacecraft. Figure 3 Dimensional reconstruction is performed to obtain the three-dimensional coordinates of the key points of the spacecraft, and then the two-dimensional coordinates of the key points of the spacecraft image are obtained by using the projection equation, and they are annotated as training labels in the spacecraft images of the initial data set to obtain the training data set. Specifically: Step 2.1, select eight corner points of the spacecraft and three antenna endpoints, a total of 11 points as key points, and the key points are numbered as follows: Figure 1 shown.

[0029] Step 2.2: Select 20 spacecraft images from the initial data set, and mark the two-dimensional coordinates of the 11 key points of the 20 spacecraft images respectively. Then, use the particle swarm optimization algorithm to optimize the two-dimensional coordinate reprojection error of the 11 key points of the 20 spacecraft images to obtain the three-dimensional coordinates of the 11 key points of the spacecraft. Step 2.2 is as follows: 2.2.1. Select 20 spacecraft images from the initial data set and mark the two-dimensional coordinates of 11 key points of the 20 spacecraft images.

[0030] 2.2.2. Based on the two-dimensional coordinates of the 11 key points in the 20 spacecraft images, the projection equation is used to calculate the feasible solutions of the three-dimensional coordinates of the 11 key points of the spacecraft.

[0031] 2.2.3. Using the particle swarm optimization algorithm, the feasible solutions of the three-dimensional coordinates of the 11 key points of the spacecraft are iterated respectively through the following formulas, and the global optimal solutions of the feasible solutions of the three-dimensional coordinates of the 11 key points of the spacecraft are searched according to the reprojection errors of the two-dimensional coordinates of the corresponding key points:

[0032]

[0033]

[0034] in, t Indicates the current iteration number, Represents the total number of iterations. In this embodiment, Take 200, k =1,2,3,…,K, K is the number of feasible solutions for the three-dimensional coordinates of the key points of the spacecraft, , Respectively expressed in t Second and t +1 iteration of the spacecraft key point k The update speed of feasible solutions of three-dimensional coordinates, , Respectively expressed in t Second and t +1 iteration of the spacecraft key point k The current position of the feasible solution in three dimensions, The key points of the spacecraft are represented by k The individual optimal position of feasible solutions in three-dimensional coordinates, is the optimal position of the population of feasible solutions for the three-dimensional coordinates of the key points of the spacecraft, It means to generate a random number between 0 and 1. refers to the individual learning factor, Refers to social learning factors; w Represents the inertia weight, which is used to describe the influence between the update speed of feasible solutions of three-dimensional coordinates of each generation, and the global optimization ability of particle swarm optimization algorithm and w Proportional, represents the upper bound of the inertia weight, represents the lower bound of the inertia weight; in this embodiment, Take 1.2, Take 1.2.

[0035] Among them, the two-dimensional coordinate reprojection error of the key point is calculated by the following formula:

[0036] in, Indicates i Spacecraft image j The 2D coordinate reprojection error of key points is 1≤ i ≤N,1≤ j ≤M, Indicates i Spacecraft image j The current position of the feasible solution of the three-dimensional coordinates of the key points, Indicates i Spacecraft image jThe feasible solution of the three-dimensional coordinates of the key points is Indicates i Spacecraft image j The two-dimensional coordinates of the key points, proj () indicates that the three-dimensional coordinates are mapped to the imaging plane to obtain the two-dimensional coordinates. represents the square of the 2-norm.

[0037] In this embodiment, the feasible solution of the three-dimensional coordinates of the key points of the spacecraft is used as the particle of the particle swarm optimization algorithm, and the two-dimensional coordinate reprojection error of the key points of the spacecraft image is used as the cost function of the particle swarm optimization algorithm. After initializing the particles, the cost function value of each particle is calculated, and the cost function value of each particle is calculated by comparing the first k The cost function value of a particle f k and the individual optimal cost function value and the global optimal cost function value The size of is used to determine whether to update the individual optimal solution and the global optimal solution of the population. The optimization process of the particle swarm optimization algorithm is as follows Figure 2 As shown in , there are two optimization processes in each iteration: the first optimization is to update the individual optimum, that is, to compare the cost function value of the particle in this iteration with the individual optimal cost function value. If the cost function value of this iteration is better, then the particle replaces the individual optimal particle. The second optimization is to update the global optimum, that is, to compare the individual optimal cost function value of the particle with the global optimal cost function value. If the individual optimal cost function value of the particle is better than the global optimal cost function value, then the individual optimal replaces the global optimal. The change of the cost function value in the key point solution process of the first spacecraft is shown in Figure 3 As shown, it can be seen that with the increase of the number of iterations, the cost function value of the feasible solution of the three-dimensional coordinates of the key points of the spacecraft gradually decreases, that is, the reprojection error of the two-dimensional coordinates of the key points in the corresponding spacecraft image gradually decreases, and the three-dimensional coordinates of the key points of the spacecraft are becoming more and more accurate.

[0038] 2.2.4. Compare the reprojection errors of the global optimal solutions of each feasible solution of the three-dimensional coordinates of the key points of the spacecraft, and take the global optimal solution of the feasible solution of the three-dimensional coordinates with the smallest reprojection error as the three-dimensional coordinates of the key points of the spacecraft, and obtain the three-dimensional coordinates of the 11 key points of the spacecraft. As shown in Table 1, the three-dimensional coordinates of the 11 key points of the spacecraft, X, Y, Z in the table represent the coordinates of the X-axis, Y-axis, and Z-axis of the space coordinate system, respectively.

[0039] Table 1

[0040] Step 2.3, according to the three-dimensional coordinates of the 11 key points of the spacecraft and the pose label of each spacecraft image in the initial data set, the projection equation is used to calculate the two-dimensional coordinates of the 11 key points of each spacecraft image in the initial data set, and it is annotated as a training label in the initial data set, such as Figure 4 As shown, the training data set is obtained.

[0041] Step 3: Establish a key point regression initial model based on the high-resolution network, and input the training data set into the key point regression initial model.

[0042] The high-resolution network includes four sub-networks layer1, layer2, layer3 and layer4 connected in sequence. Layer2, layer3 and layer4 include a Transition structure and a Stage structure respectively. The output of the Transition structure is connected to the input of the Stage structure. The Transition structure is used to adjust the resolution and number of channels of the feature map, thereby increasing and connecting scale branches. The Stage structure is used for feature extraction of branches of different scales, as well as feature fusion of different sub-networks and branches of different scales. The initial model of key point regression is established based on the high-resolution network, specifically: The Basicblock unit of the feature extraction module in each Stage structure of the high-resolution network is replaced with the ShuffleNet-v2-Block unit based on the SE (Squeeze-and-Excitation) attention mechanism, and the convolution unit of the feature fusion module in each Stage structure of the high-resolution network is replaced with a hole convolution unit, and then the scSE module (Concurrentspatial and channel Squeeze & Excitation, an attention module combining spatial and channel information) is added between the feature extraction module and the feature fusion module of each Stage structure of the high-resolution network. Among them, the ShuffleNet-v2-Block unit based on the SE attention mechanism includes an equal mapping channel and a depth convolution channel parallel to each other, and the depth convolution channel includes the first SE module, the 3×3 depth convolution module and the first SE module connected in sequence.

[0043] The loss function of the initial model of key point regression is:

[0044] in, is the loss function, Represents spacecraft image j The visibility of key points is provided by pose labels, v j=0 means the spacecraft image j Key points are outside the spacecraft image. v j =1 means the spacecraft image j Key points are blocked in the spacecraft image. v j =2 means the spacecraft image j The key points are visible. For spacecraft images j Prediction heatmap of key points, Represents spacecraft image j The predicted 2D coordinates of the key points, For spacecraft images j The real heat map of key points, Represents spacecraft image j The two-dimensional coordinates of the key points, h ( ) indicates solving the heat map.

[0045] In this embodiment, the initial model of key point regression in step 3 is based on high-resolution networks (HRNet). By adopting a feature fusion strategy of parallel subnets, the features extracted by the model are always kept at high resolution during the whole process. Multi-resolution feature fusion with a hybrid attention mechanism is added to make the fusion of features of different resolutions more efficient. At the same time, the model is designed to be lightweight to further improve the computational efficiency.

[0046] The HRNet network adopts a feature fusion strategy of parallel subnets to ensure that the features extracted by the network always maintain high resolution throughout the process. Figure 5 As shown in the figure, the HRNet network contains four sub-networks layer1, layer2, layer3 and layer4. Feature extraction is first performed through layer1, and then the Transition structure and Stage structure in layer2, layer3 and layer4 are used to complete the addition of sub-networks and the extraction and fusion of features.

[0047] Among them, the Transition structure adds a new scale branch to the network. Based on the output of layer1, Transition1 obtains two different scale branches by parallelizing two convolution layers with a convolution kernel size of 3×3, namely a scale branch with a downsampling of 4 times and a scale branch with a downsampling of 8 times. In Transition2, a scale branch with a downsampling of 16 times is created based on the original two scale branches. In Transition3, a scale branch with a downsampling of 32 times is created based on the three scale branches. The stage structure completes the feature extraction and fusion of different sub-network features. Each sub-network can receive information from other sub-networks through the stage structure. The feature maps of different sub-networks are upsampled or downsampled to fuse cross-scale information. Taking Stage3 as an example, for each scale branch, the network depth is first increased through 4 BasicBlock residual structures to enable it to extract deeper information. The output at each scale is obtained by fusion of the outputs at all scales. For example, for the output of the 4-fold downsampling scale branch, the output of the 4-fold downsampling scale branch and the output of the 8-fold downsampling scale branch are upsampled by 2 times through the Up×2 module, and the output of the 16-fold downsampling scale branch is upsampled by 4 times through the Up×4 module. The fusion output of the 4-fold downsampling scale branch is obtained by ReLU. stage3×4 means that the BasicBlock and feature fusion module need to be stacked 4 times. Among them, the Up×n module is composed of a convolution layer with a convolution kernel size of 1×1 and a normalization layer (BN layer), and finally upsampled by n times to obtain the upsampled result. Compared with the Up×n module, the Down×n module adds a convolution layer with a convolution kernel size of 3×3 and a stride of 2 for each 2-fold downsampling.

[0048] In this embodiment, the following methods are used to reduce the number of model parameters and the amount of calculation as much as possible while ensuring accuracy: (1) Replace the Basicblock unit of the feature extraction module in each Stage structure with the ShuffleNet-v2-Block unit based on the SE attention mechanism.

[0049] like Figure 6 As shown in Figure 1, the existing ShuffleNet-v2-Block unit includes a 1×1 convolution, a 3×3 depth convolution, and a 1×1 convolution connected in sequence. The 3×3 depth convolution uses a depthwise separable convolution (DSC) instead of a normal convolution, which can greatly reduce the number of parameters and the amount of calculation.

[0050] If the input is D F ×D F ×M, the output is D F×D F ×N, then N D K ×D K The amount of computation for a normal convolution with a ×M convolution kernel is D K ×D K ×M×D F ×D F × N. Figure 7 As shown in the figure, the 3×3 depth convolution in the ShuffleNet-v2-Block unit consists of M D K ×D K ×1 convolution kernel and point-by-point convolution with N 1×1×M convolution kernels, and its computational cost is D K ×D K ×M+D F ×D F + M×D F ×D F ×N. The ratio of the computational complexity of ordinary convolution to depth-separable convolution is:

[0051] It can be seen that using depth-wise separable convolution instead of ordinary convolution can greatly reduce the number of parameters and calculations.

[0052] If a 1×1 convolution with C channels has quadratic time complexity , 3×3 depthwise convolution has linear time complexity , then the complexity of two 1×1 convolutions is much higher than that of 3×3 depth convolution when the number of channels C>5 Therefore, the network parameters can be further reduced by replacing the 1×1 convolution in the ShuffleNet-v2-Block unit. For the ShuffleNet-v2-Block unit, the shuffle operation and the depth convolution do not exchange information across channels, and the 1×1 convolution plays a key role in exchanging information across channels. Since the role of the 1×1 convolution is to exchange information on the feature map in the channel dimension, the channel weighting method used by the SE attention mechanism is borrowed, such as Figure 8 As shown in the figure, the ShuffleNet-v2-Block unit based on the SE attention mechanism in this embodiment gives different weights to each channel in the feature map according to the feature information to highlight certain features and suppress other features, thereby replacing the role of 1×1 convolution. The computational complexity of channel weighting is linearly related to the number of channels, which is much lower than the complexity of 1×1 convolution.

[0053] (2) Replace the convolutional unit of the feature fusion module in each Stage structure with a dilated convolutional unit.

[0054] Compared with ordinary convolution, the dilated convolution defines the spacing between the values ​​during convolution kernel operation through the parameter of dilation rate, such as Fig. 9 As shown, the dilated convolution kernels with expansion rates of 1, 2, and 4 are respectively shown. It can be seen that for the same convolution kernel, the receptive field can be increased by increasing the expansion rate, so that the global information (such as the geometric connection relationship between key points) can be more effectively used to infer the pixel coordinates of the occluded key points. In this embodiment, the fusion of high-resolution features and low-resolution features is achieved by downsampling with a 3×3 convolution with a step size of 2. In order to increase the receptive field while downsampling the parallel subnet, this embodiment uses a dilated convolution with an expansion rate of 2 instead of a normal convolution in the downsampling process.

[0055] (3) Add the scSE module between the feature extraction module and the feature fusion module of each stage structure of the high-resolution network.

[0056] The scSE module refers to a module with space and channel squeeze excitation, such as Fig.10 As shown in the figure, the spatial squeezing excitation module sSE integrates the multi-channel features back into the original features after dimensionality reduction; the channel squeezing excitation module cSE uses global pooling to reduce dimensionality, and uses channel weighting to give each channel of the original features a weight. The results of the spatial squeezing excitation module are added to the results of the channel squeezing excitation module to obtain the output of the final feature map. The scSE attention mechanism is introduced between the feature extraction stage and the feature fusion stage, which can make the fusion of features of different resolutions more efficient.

[0057] Step 4: Use the training data set to train the initial key point regression model to obtain the key point regression model. Fig.11 As shown, specifically: Step 4.1: layer1 extracts features from a spacecraft image in the training data set, obtains an initial feature map, and sends it to layer2.

[0058] Step 4.2: The Transition structure of layer2 convolves the initial feature map, adds a scale branch, obtains the initial feature map of two scale branches, and inputs it into the feature extraction module of the Stage structure in layer2.

[0059] Step 4.3: The feature extraction module uses the ShuffleNet-v2-Block unit based on the SE attention mechanism to extract features from the initial feature map of each scale branch, obtain the deep feature maps of the two scale branches, and then input them into the scSE module.

[0060] The ShuffleNet-v2-Block unit based on the SE attention mechanism divides the channel into two channels of equal size at the input stage. Among them, the original logic in one channel is not changed, and the equal mapping method is adopted. Under the principle of equal size of input and output channels, the input features of the other channel are sequentially channel-weighted, convolved and channel-weighted again, thereby minimizing the amount of memory access. In this embodiment, the two channels are connected in series and then the results are output, and then the channel mixing operation is used to ensure the effective association of the two channels again, minimize the application of element-level operations and group convolution, and further improve the computational efficiency of the network.

[0061] like Figure 8 As shown, step 4.3 is specifically as follows: 4.3.1. The ShuffleNet-v2-Block unit based on the SE attention mechanism performs channel separation on the initial feature maps of the two scale branches, and inputs the initial feature maps of each scale branch into the equal mapping channel and the deep convolution channel respectively. The equal mapping channel performs equal mapping on the initial feature maps of the two scale branches respectively to obtain the mapping feature maps of the two scale branches.

[0062] 4.3.2, such as Fig.12 As shown in the figure, the first SE module of the deep convolution channel performs adaptive average pooling on the initial feature maps of the two scale branches, scales them to the minimum scale, and then splices them. It then performs 1×1 convolution, ReLU activation, and Sigmoid activation in sequence to obtain the weights corresponding to the two scale branches, and then upsamples them to the corresponding scale branches to obtain the weight maps corresponding to the initial feature maps of the two scale branches.

[0063] In this step, the weight of each channel comes from the initial feature maps of the two scale branches in the same pooling area, so it plays the role of 1×1 convolution in information exchange. On the other hand, this step scales the feature map to the minimum scale and calculates the small-scale feature map, and the computational complexity is much lower than directly performing 1×1 convolution.

[0064] 4.3.3. According to the weight maps corresponding to the initial feature maps of different scale branches, the initial weighted feature maps of the two scale branches are calculated by the following formula, and then sent to the 3×3 depth convolution module of the depth convolution channel:

[0065] in, represents the initial weighted feature map of the h-th scale branch, 1≤h≤2, Represents the weight map corresponding to the initial feature map of the h-th scale branch, represents the initial feature map of the h-th scale branch, Represents element-wise multiplication.

[0066] 4.3.4. The 3×3 deep convolution module of the deep convolution channel performs 3×3 deep convolution operations on the initial weighted feature maps of the two scale branches respectively to obtain the deep convolution feature maps of the two scale branches, and sends them to the second SE module of the deep convolution channel.

[0067] 4.3.5. The second SE module of the deep convolution channel respectively processes the deep convolution feature maps of the two scale branches through global average pooling, fully connected layer, ReLU activation, fully connected layer, Sigmoid activation and Ws weight calculation in sequence to obtain the weight maps corresponding to the deep convolution feature maps of the two scale branches.

[0068] 4.3.6. According to the weight maps corresponding to the deep convolution feature maps of the two scale branches, the weight maps corresponding to the deep convolution feature maps of the two scale branches are element-wise multiplied with the deep convolution feature maps of the two scale branches to obtain the deep convolution weighted feature maps of the two scale branches.

[0069] 4.3.7. Concatenate the mapping feature maps of the two scale branches obtained in step 4.3.1 and the deep convolution weighted feature maps of the two scale branches obtained in step 4.3.6 respectively, and then perform channel mixing and channel shuffling in sequence to obtain the deep feature maps of the two scale branches, and input them into the scSE module.

[0070] Step 4.4, the scSE module performs spatial information calibration and channel information calibration on the depth feature maps of the two scale branches respectively to obtain spatial information calibrated depth feature maps and channel information calibrated depth feature maps, and then adds the two together to obtain the calibrated depth feature maps of the two scale branches, and inputs them into the feature fusion module.

[0071] Step 4.5: The hole convolution unit of the feature fusion module performs convolution processing on the calibrated depth feature maps of the two scale branches, extracts their global information, obtains all scale branches of the calibrated depth feature map of each scale branch, and then fuses the calibrated depth feature maps of the corresponding scale branches respectively to obtain a fused feature map of the two scale branches. In this embodiment, the hole convolution unit performs a downsampling operation.

[0072] Step 4.6: The fused feature maps of the two scale branches obtained in step 4.5 are processed by layer3 and layer4 in turn according to the methods of steps 4.2-4.5 to obtain fused feature maps of four different scale branches.

[0073] In step 4.7, the Stage structure in layer4 fuses the fused feature maps of the four branches at different scales to obtain the feature map of the spacecraft image.

[0074] Step 4.8, generate a key point prediction heat map based on the feature map of the spacecraft image, obtain the predicted two-dimensional coordinates of the key points of the spacecraft image, calculate the loss function value between the predicted two-dimensional coordinates of the key points of the spacecraft image and the corresponding training labels, and judge whether the loss function converges according to the loss function value. If the loss function converges, the key point regression model is obtained; otherwise, return to step 4.1 to process the next spacecraft image in the training data set.

[0075] like Fig.13 As shown, this is the change curve of the loss function and the key point regression accuracy during the key point regression initial model training process of this embodiment. It can be seen that after the 120th training cycle, the loss function and the key point regression accuracy are basically stable, and the accuracy rate is about 97%.

[0076] Step 5: Obtain the target spacecraft image and input it into the key point regression model. The key point regression model uses deep separable convolution combined with attention mechanism to extract the key point features of the target spacecraft image, and then uses void convolution combined with hybrid attention mechanism to fuse the key point features to obtain the predicted two-dimensional coordinates of the key points of the target spacecraft image. Step 6: Perform pose estimation based on the predicted two-dimensional coordinates of the key points of the target spacecraft image to obtain the predicted pose of the target spacecraft, thereby completing the pose estimation of the spacecraft.

[0077] like Fig.14 As shown, a schematic diagram of the predicted two-dimensional coordinates of the key points obtained by using the key point regression model in this embodiment to perform key point regression on the target spacecraft image in a dark environment, occlusion and complex background. It can be seen that the key point regression model in this embodiment can accurately realize key point regression in a dark environment, occlusion and complex background.

[0078] On the other hand, the accuracy of key point regression in this embodiment is evaluated by using the average value of key point similarity. The key point similarity indicates the similarity between the predicted two-dimensional coordinates and the true two-dimensional coordinates of the key point. Its value range is between 0 and 1. The closer to 1, the higher the similarity. It is calculated by the following formula:

[0079]

[0080] Among them, AP represents the key point regression accuracy, taking the average similarity of the key points. OKS j Indicates j The similarity of key points; , which means that only the key points in the spacecraft image are calculated. dj Indicates j The Euclidean distance between the predicted 2D coordinates and the true 2D coordinates of the key points, s represents the target scale, k j Indicates j The weight of each key point.

[0081] As shown in Table 2, the comparison data of key point regression using the existing model and the key point regression model in this embodiment using the spacecraft dataset SPEED+ is shown.

[0082] The HRNet network and the KRN network (Keypoints Regression Network, from the article published by Tae Ha Park et al. at the 2019 AAS / AIAA Astrodynamics Expert Conference) were selected.<TowardsRobust Learning-Based Pose Estimation of Non-Cooperative Spacecraft> ) is used as a benchmark model to verify the key point regression model in this embodiment. First, the accuracy of the key point regression in this embodiment is verified. As shown in Table 2, the KRN network uses a lightweight MolilenetV2 as the backbone network for key point regression, with fewer parameters, but the accuracy of the key point regression is reduced by 4.62% compared with the HRNet network. First, the convolution unit of each Stage structure feature fusion module in the HRNet network is replaced with a hole convolution unit. The accuracy of the key point regression is 96.08%. Compared with the HNRe t network, the accuracy of the key point regression is improved by about 1.22%, indicating that changing ordinary convolution to hole convolution can effectively improve the receptive field, thereby enhancing the key point regression model's ability to infer occluded key points. On this basis, a scSE module is added between each Stage structure feature extraction module and feature fusion module of the HNRe t network. Compared with the HNRet network, the accuracy of key point regression is improved by 3.11%. It can be seen that the method of using dilated convolution when fusing features of different scales and the introduction of the scSE attention mechanism between the feature extraction stage and the feature fusion stage proposed in this embodiment can further improve the accuracy of key point regression compared with the HNRet network.

[0083] Table 2

[0084] Secondly, the lightweight degree of the key point regression model in this embodiment is verified. As shown in Table 3, AP is the accuracy of key point regression, Parameters is the parameter amount, and FLOPs is the network floating point operation amount. Although the accuracy of key point regression of the HRNet network is high, the parameter amount is as high as 28 million, and the network floating point operation amount is 8.35G. The present invention introduces the scSE attention mechanism to make the key point regression model more complex. Then, after replacing the Basicblock unit of the feature extraction module in each Stage structure of the HRNet network with the ShuffleNet-v2-Block unit, the parameter amount and the network floating point operation amount are greatly reduced compared with the HRNet network under the premise that the key point regression accuracy is almost unchanged. Finally, the weighted multiplication based on the SE attention mechanism is used to replace the 1×1 convolution operation in the ShuffleNet-v2-Block unit, and the parameter amount and the network floating point operation amount are further reduced. It can be seen that the key point regression model in this embodiment can greatly reduce the parameter amount and the network floating point operation amount while ensuring that the key point regression accuracy does not decrease, laying the foundation for its actual deployment in space.

[0085] Table 3

[0086] In the lightweight spacecraft pose estimation method based on a high-resolution network provided in this embodiment, the key point regression model is based on a high-resolution network as the backbone. In order to solve the problems of large memory usage and poor real-time performance, the ShuffleNet-v2-Block unit is used to lightweight the high-resolution network. The accuracy of the key point regression is further improved by adding a hybrid attention mechanism in the feature extraction process and using hollow convolution to expand the receptive field in the feature fusion process, which can achieve lightweight while ensuring accuracy.

[0087] It should be noted that, in this embodiment, SPEED (Spacecraft Pose Estimation Dataset) is the first public spacecraft pose estimation dataset released by Stanford University's Space Rendezvous and Docking Laboratory and ESA in 2019, and a new generation of spacecraft dataset SPEED+ was released in October 2021; the Transition structure is mainly used for information fusion and transition between sub-networks of different scales; the Stage structure is a stage of the HRNet network, and each Stage structure contains multiple parallel scale branches; layer refers to layer in deep neural networks; the Basicblock unit is a basic convolutional neural network building module; the ShuffleNet-v2-Block unit is a lightweight network that can maximize the calculation speed while ensuring accuracy; ReLU activation refers to Rectified Linear Unit, and Sigmoid activation is a common activation function in deep neural networks, and Ws refers to weight calculation.

Claims

1. A lightweight spacecraft pose estimation method based on a high-resolution network, characterized in that: The following steps are involved: Step 1: Select a spacecraft dataset, and then preprocess it to obtain an initial dataset; the spacecraft dataset consists of spacecraft images with pose labels; Step 2: Select key points on the spacecraft, and then use the spacecraft image and its pose label in the initial data set to perform multi-view 3D reconstruction on the spacecraft to obtain the 3D coordinates of the key points of the spacecraft, and then use the projection equation to obtain the 2D coordinates of the key points of the spacecraft image, and annotate them as training labels in the spacecraft image in the initial data set to obtain the training data set; Step 3: Establish a key point regression initial model based on the high-resolution network, and input the training data set into the key point regression initial model; Step 4: Use the training data set to train the key point regression initial model to obtain the key point regression model; Step 5: Obtain the target spacecraft image and input it into the key point regression model. The key point regression model uses deep separable convolution combined with attention mechanism to extract the key point features of the target spacecraft image, and then uses void convolution combined with hybrid attention mechanism to fuse the key point features to obtain the predicted two-dimensional coordinates of the key points of the target spacecraft image. Step 6: Perform pose estimation based on the predicted two-dimensional coordinates of the key points of the target spacecraft image to obtain the predicted pose of the target spacecraft, thereby completing the pose estimation of the spacecraft.

2. The method for lightweight spacecraft pose estimation based on high-resolution network according to claim 1 is characterized in that: In step 3, the initial model of key point regression is established based on the high-resolution network, specifically: The Basicblock unit of the feature extraction module in each Stage structure of the high-resolution network is replaced with the ShuffleNet-v2-Block unit based on the SE attention mechanism, and the convolution unit of the feature fusion module in each Stage structure of the high-resolution network is replaced with a hollow convolution unit, and then the scSE module is added between the feature extraction module and the feature fusion module of each Stage structure of the high-resolution network; The ShuffleNet-v2-Block unit based on the SE attention mechanism includes an equal mapping channel and a depth convolution channel parallel to each other, and the depth convolution channel includes a first SE module, a 3×3 depth convolution module and a second SE module connected in sequence.

3. The method for lightweight spacecraft pose estimation based on high-resolution network according to claim 2 is characterized in that: Step 4 is as follows: Step 4.1, extract features from a spacecraft image in the training data set to obtain an initial feature map; Step 4.2, convolve the initial feature map, add a scale branch, obtain the initial feature map of the two scale branches, and input it into the feature extraction module of the Stage structure; Step 4.3: The feature extraction module uses the ShuffleNet-v2-Block unit based on the SE attention mechanism to extract features from the initial feature map of each scale branch, obtains the deep feature maps of the two scale branches, and inputs them into the scSE module; Step 4.4, the scSE module performs spatial information calibration and channel information calibration on the depth feature maps of the two scale branches respectively to obtain a spatial information calibrated depth feature map and a channel information calibrated depth feature map, and then adds the two together to obtain a calibrated depth feature map of the two scale branches, and inputs it into the feature fusion module; Step 4.5, the hole convolution unit of the feature fusion module performs convolution processing on the calibrated depth feature maps of the two scale branches, extracts their global information, obtains all scale branches of the calibrated depth feature maps of the two scale branches, and then fuses the calibrated depth feature maps of the corresponding scale branches respectively to obtain the fused feature map of the two scale branches; Step 4.6: Use the fused feature map of the two scale branches obtained in step 4.5 as the initial feature map, and then return to step 4.2 until the fused feature maps of four different scale branches are obtained; Step 4.7, fusing the fused feature maps of the four different scale branches to obtain a feature map of the spacecraft image; Step 4.8, generate a key point prediction heat map based on the feature map of the spacecraft image, obtain the predicted two-dimensional coordinates of the key points of the spacecraft image, calculate the loss function value between the predicted two-dimensional coordinates of the key points of the spacecraft image and the corresponding training labels, and judge whether the loss function converges according to the loss function value. If the loss function converges, the key point regression model is obtained; otherwise, return to step 4.1 to process the next spacecraft image in the training data set.

4. According to the method for lightweight spacecraft pose estimation based on high-resolution network of claim 3, it is characterized in that step 4.3 is specifically: 4.3.

1. The ShuffleNet-v2-Block unit based on the SE attention mechanism performs channel separation on the initial feature maps of the two scale branches, and inputs the initial feature maps of each scale branch into the equal mapping channel and the deep convolution channel respectively. The equal mapping channel performs equal mapping on the initial feature maps of the two scale branches respectively, and obtains the mapping feature maps of the two scale branches; 4.3.

2. The first SE module of the deep convolution channel performs adaptive average pooling on the initial feature maps of the two scale branches, scales them to the minimum scale, and then splices them. Then, 1×1 convolution, ReLU activation, and Sigmoid activation are performed in sequence to obtain the weights corresponding to the two scale branches. Then, they are upsampled to the corresponding scale branches, and the weight maps corresponding to the initial feature maps of the two scale branches are obtained. 4.3.

3. According to the weight maps corresponding to the initial feature maps of the two scale branches, the initial weighted feature maps of the two scale branches are calculated by the following formula, and then sent to the 3×3 depth convolution module of the depth convolution channel: ； in, represents the initial weighted feature map of the h-th scale branch, 1≤h≤2, Represents the weight map corresponding to the initial feature map of the h-th scale branch, represents the initial feature map of the h-th scale branch, It means element-wise multiplication; 4.3.

4. The 3×3 deep convolution module of the deep convolution channel performs 3×3 deep convolution operations on the initial weighted feature maps of the two scale branches respectively to obtain the deep convolution feature maps of the two scale branches, and sends them to the second SE module of the deep convolution channel; 4.3.

5. The second SE module of the deep convolution channel respectively processes the deep convolution feature maps of the two scale branches through global average pooling, fully connected layer, ReLU activation, fully connected layer, Sigmoid activation and Ws weight calculation in sequence to obtain the weight maps corresponding to the deep convolution feature maps of the two scale branches; 4.3.

6. According to the weight maps corresponding to the deep convolution feature maps of the two scale branches, the weight maps corresponding to the deep convolution feature maps of the two scale branches are element-wise multiplied with the deep convolution feature maps of the two scale branches to obtain the deep convolution weighted feature maps of the two scale branches; 4.3.

7. Concatenate the mapping feature maps of the two scale branches obtained in step 4.3.1 and the deep convolution weighted feature maps of the two scale branches obtained in step 4.3.6 respectively, and then perform channel mixing and channel shuffling in sequence to obtain the deep feature maps of the two scale branches, and input them into the scSE module.

5. The method for lightweight spacecraft pose estimation based on high-resolution network according to any one of claims 1 to 4, characterized in that: Step 2 is as follows: Step 2.1, select M key points on the spacecraft, where M is an integer, M≥5; Step 2.2, select N spacecraft images in the initial data set, and respectively mark the two-dimensional coordinates of M key points of the N spacecraft images, and then use the particle swarm optimization algorithm to optimize the two-dimensional coordinate reprojection errors of the M key points of the N spacecraft images to obtain the three-dimensional coordinates of the M key points of the spacecraft; wherein N is an integer, N ≥ 10; Step 2.3: According to the three-dimensional coordinates of the M key points of the spacecraft and the pose label of each spacecraft image in the initial data set, the projection equation is used to calculate the two-dimensional coordinates of the M key points of each spacecraft image in the initial data set, and the two-dimensional coordinates are marked in the initial data set as training labels to obtain the training data set.

6. The method for lightweight spacecraft pose estimation based on high-resolution network according to claim 5 is characterized in that: Step 2.2 is as follows: 2.2.

1. Select N spacecraft images from the initial data set and mark the two-dimensional coordinates of M key points of the N spacecraft images respectively; 2.2.

2. According to the two-dimensional coordinates of the M key points in the N spacecraft images, the projection equation is used to calculate the feasible solutions of the three-dimensional coordinates of the M key points of the spacecraft; 2.2.

3. Using the particle swarm optimization algorithm, the feasible solutions of the three-dimensional coordinates of the M key points of the spacecraft are iterated respectively through the following formulas, and the global optimal solutions of the feasible solutions of the three-dimensional coordinates of the M key points of the spacecraft are searched respectively according to the reprojection errors of the two-dimensional coordinates of the corresponding key points: ； ； ； in, t Indicates the current iteration number, represents the total number of iterations, k=1,2,3,…,K, K is the number of feasible solutions for the three-dimensional coordinates of the key points of the spacecraft, , Respectively expressed in t Second and t +1 iteration of the spacecraft key point k The update speed of feasible solutions of three-dimensional coordinates, , Respectively expressed in t Second and t +1 iteration of the spacecraft key point k The current position of the feasible solution in three dimensions, The key points of the spacecraft are represented by k The individual optimal position of feasible solutions in three-dimensional coordinates, is the optimal position of the population of feasible solutions for the three-dimensional coordinates of the key points of the spacecraft, It means to generate a random number between 0 and 1. refers to the individual learning factor, Refers to social learning factors; w represents the inertia weight, represents the upper bound of the inertia weight, Indicates the lower bound of the inertia weight; The two-dimensional coordinate reprojection error of the key point is calculated by the following formula: ； in, Indicates i Spacecraft image j The 2D coordinate reprojection error of key points is 1≤ i ≤N,1≤ j ≤M, Indicates i Spacecraft image j The current position of the feasible solution of the three-dimensional coordinates of the key points, Indicates i Spacecraft image j The feasible solution of the three-dimensional coordinates of the key points is Indicates i Spacecraft image j The two-dimensional coordinates of the key points, proj () indicates that the three-dimensional coordinates are mapped to the imaging plane to obtain the two-dimensional coordinates. represents the square of the two-norm; 2.2.

4. Compare the reprojection errors of the global optimal solutions of each feasible solution for the three-dimensional coordinates of the key points of the spacecraft, and take the global optimal solution of the feasible solution for the three-dimensional coordinates with the smallest reprojection error as the three-dimensional coordinates of the key points of the spacecraft, and obtain the three-dimensional coordinates of the M key points of the spacecraft respectively.

7. The method for lightweight spacecraft pose estimation based on high-resolution network according to claim 6 is characterized in that: In step 3, the loss function of the key point regression initial model is: ； in, is the loss function, Represents spacecraft image j The visibility of key points is provided by pose labels, v j =0 means the spacecraft image j Key points are outside the spacecraft image. v j =1 means the spacecraft image j Key points are blocked in the spacecraft image. v j =2 means the spacecraft image j The key points are visible. For spacecraft images j Prediction heatmap of key points, Represents spacecraft image j The predicted 2D coordinates of the key points, For spacecraft images j The real heat map of key points, Represents spacecraft image j The two-dimensional coordinates of the key points, h ( ) indicates solving the heat map.

8. The method for lightweight spacecraft pose estimation based on high-resolution network according to claim 7 is characterized in that: In step 2.1, the key points are eight corner points and three antenna endpoints of the spacecraft.

9. The method for lightweight spacecraft pose estimation based on high-resolution network according to claim 8, characterized in that: In step 1, the preprocessing includes: adding random Gaussian white noise, adjusting brightness and contrast, and randomly flipping and rotating.

Images