Non-cooperative spacecraft monocular attitude estimation method and system

Abstract

The application discloses a non-cooperative spacecraft monocular attitude estimation method and system, wherein the method comprises the following steps: acquiring a shooting image of the non-cooperative spacecraft; inputting the shooting image into a pre-constructed target attitude estimation model to obtain attitude estimation of the non-cooperative spacecraft, including a position vector and a direction vector; wherein the training process of the model specifically comprises fusing scale weight and space weight, estimating a weighted fused feature map through a position estimation head to obtain an estimated position vector; estimating a weighted local feature map through a direction estimation head to obtain an estimated direction vector; dynamically generating a weight adjustment amount to optimize and update the fusion weight, and then re-performing attitude estimation; and iteratively optimizing the model to obtain an optimal target attitude estimation model. The application can accelerate the calculation speed and calculation frequency while maintaining a low attitude estimation error.

Description

TECHNICAL FIELD

[0001] The present application relates to the cross field of deep learning, image processing and spacecraft navigation, and in particular to a non-cooperative spacecraft monocular attitude estimation method and system combining scale-sensitive dynamic weight and local feature multi-scale extraction. BACKGROUND

[0002] Estimating the position and attitude of non-cooperative targets is one of the key technologies to achieve formation flight, space service, rendezvous and docking, etc.

[0003] The existing non-cooperative spacecraft attitude estimation system relies on multiple sensors, among which laser radar and stereo camera can provide high-precision 3D point cloud data and rich texture features, meeting the real-time requirements of some scenes, but laser radar and stereo camera are precision optical instruments, which are costly and have large data volume, requiring high computing power and energy consumption of on-board processing units, and are difficult to adapt to small spacecraft relying on solar power or resource-limited space mission scenarios. In contrast, monocular RGB / grayscale cameras have become the preferred solution for space resource-limited scenarios due to their low power consumption, lightweight, low cost, small size, etc., and have irreplaceable application potential in future on-orbit service missions. However, monocular camera-based attitude estimation faces two major challenges: first, images rely on optical imaging, which is easily affected by changes in light, camera perspective shift, complex background interference, resulting in poor stability of target feature extraction; second, the inherent scale ambiguity of monocular vision makes the position and attitude calculation ambiguous.

[0004] In recent years, deep learning methods have provided a new way to solve the above problems through a data-driven mode, but existing technologies still have bottlenecks: mainstream double-branch networks lack dynamic feature interaction mechanisms in position and direction estimation tasks, making it difficult to adapt to drastic changes in target scale; at the same time, the network model has too many parameters and high computational complexity, resulting in low measurement frequency and failing to meet the real-time requirements of on-board embedded devices. SUMMARY

[0005] The present application proposes a non-cooperative target monocular attitude estimation method and system combining scale-sensitive dynamic weight and local feature multi-scale extraction to address the problems of fixed feature fusion weight, large network parameter quantity, and difficulty in balancing estimation accuracy and real-time performance in existing monocular attitude estimation methods. The present application reduces network parameters, speeds up calculation, and maintains a low attitude estimation error.

[0006] The technical scheme adopted by the present application is:

[0007] A non-cooperative spacecraft monocular attitude estimation method combining scale-sensitive dynamic weight and local feature multi-scale extraction is provided, comprising the following steps:

[0008] obtaining a photographed image of the non-cooperative spacecraft;

[0009] inputting the photographed image into a pre-constructed target attitude estimation model to obtain an attitude estimation of the non-cooperative spacecraft, including a position vector and a direction vector;

[0010] The training process of the model specifically includes the following steps:

[0011] extracting global feature maps and local feature maps from the images photographed at different angles in the training set through the constructed initial target attitude estimation model; detecting the non-cooperative spacecraft target in the image to obtain the geometric features of the target and estimate the size of the target; generating spatial weights for the global feature maps and the local feature maps according to the pre-setting, and generating scale weights for the global feature maps and the local feature maps according to the size of the target, and fusing the two kinds of weights respectively to obtain the fusion weights of the two kinds of feature maps; weighting and fusing the two kinds of feature maps according to the fusion weights, estimating the estimated position vector by combining the geometric features of the target and the fused feature maps through the position estimation head, and estimating the estimated direction vector through the direction estimation head after weighting the local feature maps;

[0012] calculating the error between the estimated position information and the actual position information of the target, the estimated direction information and the actual direction information, generating a corresponding weight adjustment amount according to the error, optimizing and updating the fusion weight, and re-estimating the attitude; constantly updating the weight adjustment amount, iteratively optimizing the model, and verifying through the validation set to obtain the optimal target attitude estimation model.

[0013] The above technical solution further includes the step of: preprocessing the photographed image of the non-cooperative spacecraft, including rotation, brightness and contrast adjustment.

[0014] In the above technical solution, each image in the training set and the validation set includes the direction vector, the position vector, and the label of the target bounding box of the non-cooperative spacecraft.

[0015] In the above technical solution, the global feature maps and the local feature maps are extracted through a plurality of depth separable convolution layers in the target attitude estimation model, and the non-cooperative spacecraft target in the image is detected to obtain a bounding box, and the geometric features of the target are obtained according to the coordinates of the bounding box.

[0016] In the above technical solution, the position estimation head includes a global path and a local path, which specifically convolves and pools the global features and the local features weighted with the fusion weight to extract global position information and local structure information.

[0017] The output features of the global path and the local path are fused through splicing and convolution to form a fused position feature.

[0018] The fusion feature is fused with the position regression network to output a preliminary three-dimensional position of the target;

[0019] The geometric feature of the target is fused with the preliminary three-dimensional position, and the regression network is used to output the final three-dimensional position of the target.

[0020] According to the above technical solution, the fusion weight is specifically a weighted average of the spatial weight and the scale weight.

[0021] According to the above technical solution, when the fusion weight is updated by the weight adjustment amount, the weight adjustment amount weighted by the error learning rate is specifically added to the fusion weight.

[0022] The application also provides a non-cooperative spacecraft monocular attitude estimation system combining scale-sensitive dynamic weight and local feature multi-scale extraction, comprising:

[0023] An image acquisition module is configured to acquire a photographed image of the non-cooperative spacecraft.

[0024] An attitude estimation module is configured to input the photographed image into a pre-constructed target attitude estimation model to obtain an attitude estimation of the non-cooperative spacecraft, including a position vector and a direction vector.

[0025] A model training module is configured to train the model, specifically including: extracting global feature maps and local feature maps from images photographed at different angles in a training set by using the constructed initial target attitude estimation model; detecting the non-cooperative spacecraft target in the images to obtain the geometric feature of the target and estimate the size of the target; generating spatial weights of the global feature maps and the local feature maps according to a pre-setting, and generating scale weights of the global feature maps and the local feature maps according to the size of the target, and fusing the two kinds of weights to obtain fusion weights of the two kinds of feature maps; weighting and fusing the two kinds of feature maps according to the fusion weights, estimating the position vector by using a position estimation head in combination with the geometric feature of the target and the fused feature maps; and estimating the direction vector by using a direction estimation head in combination with the weighted local feature maps.

[0026] The error between the estimated position information and the actual position information, and the error between the estimated direction information and the actual direction information of the target are calculated, a corresponding weight adjustment amount is generated according to the error, the fusion weight is updated and optimized, and the attitude estimation is performed again; the weight adjustment amount is continuously updated, the model is iteratively optimized, and the verification set is verified to obtain an optimal target attitude estimation model.

[0027] According to the above technical solution, the system further comprises an image preprocessing module configured to preprocess the acquired photographed image of the non-cooperative spacecraft, including rotation, brightness and contrast adjustment.

[0028] The application further provides a computer storage medium, which stores a computer program executable by a processor, and the computer program performs the monocular attitude estimation method of a non-cooperative spacecraft combining scale-sensitive dynamic weight and local feature multi-scale extraction according to any one of the technical solutions.

[0029] The application has the following beneficial effects: the application realizes two-stage dynamic weight adjustment by designing scale perception and error feedback mechanism in the position estimation branch, and adopts local features in the direction estimation branch, thereby abandoning redundant global feature fusion, accelerating the calculation speed and calculation frequency, and maintaining a low attitude estimation error.

[0030] Further, the application adopts lightweight technologies such as depth separable convolution and adaptive pooling to compress network parameters, thereby maintaining network lightweight and improving the accuracy and robustness of non-cooperative spacecraft attitude estimation.

[0031] Of course, any product implementing the application does not necessarily need to achieve all the advantages described above. BRIEF DESCRIPTION OF DRAWINGS

[0032] In order to more clearly illustrate the technical solutions in the embodiments of the application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description are some embodiments of the application, and for those skilled in the art, other drawings can also be obtained without creative labor.

[0033] FIG. 1(a) is a flowchart of the monocular attitude estimation method of a non-cooperative target combining scale-sensitive dynamic weight and local feature multi-scale extraction according to an embodiment of the application;

[0034] FIG. 1(b) is a flowchart of the monocular attitude estimation method of a non-cooperative target combining scale-sensitive dynamic weight and local feature multi-scale extraction according to an embodiment of the application; Figure Two

[0035] FIG. 1(c) is a training and testing flowchart of the target attitude estimation model according to an embodiment of the application;

[0036] Figure 2 FIG. 2 is a result diagram of target area scale-based division and threshold determination of a training set target according to an embodiment of the application;

[0037] Figure 3 FIG. 3 is an effect diagram of data enhancement in the monocular attitude estimation method of a non-cooperative target combining scale-sensitive dynamic weight and local feature multi-scale extraction according to an embodiment of the application;

[0038] Figure 4 ​A network framework diagram of a non-cooperative target monocular pose estimation method combining scale-sensitive dynamic weights and local feature multi-scale extraction is provided for the embodiment of the application.

[0039] Fig. 5(a) is a total loss diagram of the embodiment of the application trained and verified using the SPEED dataset;

[0040] Fig. 5(b) is a bounding box loss diagram of the embodiment of the application trained and verified using the SPEED dataset;

[0041] Fig. 5(c) is a position loss diagram of the embodiment of the application trained and verified using the SPEED dataset;

[0042] Fig. 5(d) is a direction loss diagram of the embodiment of the application trained and verified using the SPEED dataset;

[0043] Fig. 5(e) is a score indicator loss diagram of the embodiment of the application trained and verified using the SPEED dataset;

[0044] Fig. 5(f) is a learning rate change diagram of the embodiment of the application trained and verified using the SPEED dataset;

[0045] Figure 6 Fig. 5(f) is a learning rate change diagram of the embodiment of the application trained and verified using the SPEED dataset; DETAILED DESCRIPTION

[0046] In order to make the purpose, technical solutions and advantages of the application clearer, the application will be further described in detail below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the application and do not limit the application.

[0047] It should be noted that the diagrams provided in the embodiments of the application only illustrate the basic concept of the application in a schematic manner, and therefore only the components related to the application are shown in the diagrams, not the number, shape and size of the components when actually implemented. The actual implementation of each component may be arbitrarily changed in terms of shape, number and proportion, and the layout pattern of the components may also be more complex.

[0048] In the application, it should also be noted that, if terms such as "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer" and the like appear, the indicated orientation or position relationship is based on the orientation or position relationship shown in the drawings, and is only for the convenience of describing the application and simplifying the description, and therefore cannot be understood as indicating or implying that the indicated device or element must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as limiting the application. In addition, if the terms "first" and "second" appear, they are only for description and distinction purposes and cannot be understood as indicating or implying relative importance.

[0049] Further, it should be noted that features of various embodiments of the present application can be combined or integrated, in whole or in part, and that as such, the various embodiments can be further modified than described and the applications developed according to such modifications, all of which are within the scope of the present application. Each of the various embodiments can be implemented independently, or in association with one another.

[0050] As shown in FIG. 1(a), the non-cooperative spacecraft monocular attitude estimation method of the embodiment of the present application combines scale-sensitive dynamic weight and local feature multi-scale extraction, mainly including the following steps:

[0051] S1, obtaining a photographed image of a non-cooperative spacecraft;

[0052] S3, inputting the photographed image into a pre-constructed target attitude estimation model to obtain an attitude estimation of the non-cooperative spacecraft, including a position vector and a direction vector;

[0053] The training process of the model specifically includes the following steps:

[0054] The global feature map and the local feature map of the image photographed from different angles in the training set are extracted through the constructed initial target attitude estimation model; the non-cooperative spacecraft target in the image is detected to obtain the geometric features of the target and estimate the size of the target; the spatial weight for generating the global feature map and the local feature map is generated according to the pre-setting, and the scale weight for generating the global feature map and the local feature map is generated according to the size of the target, and the fusion weight of the two kinds of feature maps is obtained by fusing the two kinds of weights respectively; the estimated position vector is obtained by estimating the position estimation head combined with the geometric features of the target and the fused feature maps; the estimated direction vector is obtained by estimating the weighted local feature map through the direction estimation head;

[0055] The error between the estimated position information and the actual position information of the target, and the estimated direction information and the actual direction information is calculated, and the corresponding weight adjustment amount is generated according to the error to optimize and update the fusion weight, and then the attitude estimation is performed again; the weight adjustment amount is updated continuously, the model is iteratively optimized, and the validation set is verified to obtain the optimal target attitude estimation model.

[0056] Further, as shown in FIG. 1(b), between steps S1 and S3, there is also a step S2 of pre-processing the obtained photographed image of the non-cooperative spacecraft, including rotation, brightness and contrast adjustment. Accordingly, step S3 specifically performs attitude estimation on the pre-processed photographed image.

[0057] In one specific embodiment of the present application, as shown in FIG. 1(c), the training of the model mainly includes the following steps:

[0058] S101, acquire a single-channel gray image data of a spaceborne scene, construct a data set containing a bounding box, a 3D position and a direction label, and complete data preprocessing;

[0059] S102, perform data augmentation on the data set based on a preset augmentation strategy, filter effective samples through sample verification and abnormality processing;

[0060] S103, construct a pose estimation network, configure network module parameters and complete parameter initialization;

[0061] S104, design a multi-task joint loss function and a training optimization strategy, train a network model and save an optimal model;

[0062] S105, perform inference testing on the trained model.

[0063] In this embodiment, the specific method of step S101 is: acquiring a single-channel gray image data of a spaceborne scene or a synthetic data set to construct a training, verification and test data set, and labeling the images in the training set, each image must contain three labels, i.e. a bounding box of a target, a position vector and a direction quaternion label, and completing data preprocessing of the training data.

[0064] Among them, the data set collection can be realized by using a method of synthesizing images by using a 3D model and labeling, in the present application, a published SPEED data set is used, which includes a training set, a test set, a real test set and a real set. Among them, 12000 images are generated by an optical simulator of SLAB using a high-fidelity texture model, and 300 images come from a TRON facility, which are taken by a physical model. The image resolution is 1920x1200, and the format is gray image. The distance to the camera is m, according to the distance, the target is divided into three categories of small, medium and large targets, and the selection of target area distribution and threshold is as shown in Figure 2 .

[0065] In terms of data set labels, the SPEED data set does not publish the target bounding box labels of the training set, so the labeling is performed first, and the original data set does not disclose the pose labels of the verification set, so the present application selects 12000 images of the training set to divide the training set subset and the verification set subset in a ratio of 8:2, sets the training set batch to 32, the verification set batch to 16, and the input size to 384x240.

[0066] In the implementation, the specific method of step S102 is: performing data augmentation on the data set based on a preset augmentation strategy, the augmentation result is as shown in Figure 3 , then filtering effective samples through sample verification and abnormality processing, and the augmentation and filtering methods are as follows:

[0067] In terms of data augmentation, the random rotation augmentation probability is set to 0.7, and the random angle is generated in [0, 1] and the random scaling coefficient is generated in [0.5, 2], and the image and the frame are randomly scaled at the same time.

[0068] The standard deviation is set to , and the blur kernel is set to , and the image is blurred with a probability of 50%. The brightness offset is set to , and the contrast scaling is set to .

[0069] To avoid the absence of training images or labels in the dataset, an image sample verification and exception handling mechanism is set up to skip abnormal samples and avoid unnecessary interruptions to the training process.

[0070] In implementation, the specific method of step S103 is to construct a pose estimation network model including a grayscale adaptation and initial convolution module, a multi-stage feature extraction module, a target detection module, a dynamic weight fusion module, and a position / direction estimation head, as shown in Figure 4 . The specific steps are as follows:

[0071] (1) Gray scale adaptation and initial convolution module

[0072] Step 1: Use a convolution layer to convert a single-channel grayscale image to a 3-channel feature, and initialize the convolution kernel parameters using He normal distribution;

[0073] Step 2: Use a convolution with a size of , a stride of 2, and a padding of 1, plus BN and ReLU6. The initial convolution layer is constructed to convert the input image into an input feature map that meets the multi-stage feature extraction module, with a size of [B, 32, 192, 1200];

[0074] (2) Multi-stage feature extraction module

[0075] Step 1: ds_conv1 deep convolution layer: receives the output of the initialization module, uses a deep convolution with a size of 3x3 to maintain the input channel number, and expands the channel number to 64 through a 1x1 convolution, and after batch normalization and ReLU6 activation, the output feature map size is [B, 64, 192, 120], which is rich in intermediate feature maps of basic texture and edge information, and inherits the bottom-level visual features of the initialization module output, and enhances the local spatial correlation, providing more rich feature expression for the subsequent depth separable convolution layer.

[0076] Step2: ds_conv2 depth convolution layer: receives the feature map output by ds_conv1, realizes spatial down-sampling through 3x3 depth convolution with a step of 2; then expands the channel number to 128 through 1x1 convolution, combines batch normalization, ReLU6 and SE attention mechanism with a compression rate of 1 / 16, and outputs local features , with a size of [B, 128, 96, 60], mainly containing local details such as target edge texture, structure, etc.

[0077] Step3: ds_conv3 depth convolution layer: receives the feature map output by ds_conv2, similar to the second layer, but the step is set to 1 to keep the spatial resolution unchanged; the output channel is maintained at 128 to obtain refined local features [B, 128, 96, 60], which focus more on the continuity of the target local outline (such as edge smoothness, structural integrity), reduce redundant information, and enhance the ability to depict key details of small targets.

[0078] Step4: ds_conv4 depth convolution layer: receives the feature map output by ds_conv3, realizes down-sampling through 3x3 depth convolution with a step of 2, and expands to 256 channels through 1x1 convolution, combines batch normalization, ReLU6 and SE module to enhance feature expression, and outputs middle-level features , with a size of [B, 256, 16, 10], which transitions from local details to structural features of the target, highlighting the distinction between the target and the background to provide structured information for subsequent features.

[0079] Step5: ds_conv5 depth convolution layer: continues the channel setting of the fourth layer (256 channels), further introduces a 3x3 local contrast normalization operator based on convolution and normalization to enhance the response to low-contrast areas, and outputs features with a size of [B, 256, 16, 10], which mainly include details of low-light and blurred areas, improve the robustness of features in complex space-borne scenes, and supplement the light adaptability expression of middle-level features.

[0080] Step6: ds_conv6 depth convolution layer: sets a 3x3 depth convolution with a step of 2 and combines a 1x1 convolution to increase the channel number to 512, while introducing a cross-layer connection mechanism to fuse the middle-level features of the fourth layer with the output of this layer, and obtains global features , with a size of [B, 512, 8, 5], which mainly contains the overall shape of the target, such as the spatial layout and outline of the spacecraft.

[0081] (3) Target detection module

[0082] Step1: edge enhancement, calculate The x / y direction Sobel gradient of the feature map, the formula is as follows:

[0083]

[0084] Wherein and are the Sobel gradient values of the first feature map in the horizontal direction and the vertical direction respectively, is a single-channel feature map in the horizontal direction Sobel operator, the value is 1 / 8[[-1,0,1],[-2,0,2],[-1,0,1]], is a vertical Sobel operator, the value is 1 / 8[[-1,-2,-1],[0,0,0],[1,2,1]], is a minimum constant to avoid numerical instability in gradient calculation, the value is ; is a fused gradient feature map, which is subsequently fused with the original feature map to highlight the boundary profile of the spacecraft.

[0085] Step2: Multi-scale fusion, upsample and to 8x5, concatenate with , and output the channel number as 1024;

[0086] Step3: bounding box prediction, use convolution to compress the output channel from 1024 to 256, and then use convolution to map to 4, output the coordinate of the feature map scale, and then map to the original image scale according to the down-sampling rate.

[0087] (4) Dynamic weight fusion module

[0088] Step1: Divide the target scale, according to the width height of the target original size bounding box output in the above steps and the threshold T1=45.2, T2=120.5, judge the target type. When , it is determined as a small target; when , it is determined as a large target; the rest is a medium target; and output the batch scale weight , which provides a scale basis for subsequent weight allocation;

[0089] Step2: Generate initial weight:

[0090] 1. The initial weight predictor receives the target bounding box coordinate parameters The system uses a 3-layer fully connected network with an input dimension of 4 and hidden layer dimensions of 64 / 128 / 64 respectively to map the input features. The features are then normalized using the Softmax function, and the output is a predicted weight vector. The dimension is [B, 2], and each row includes global feature weights and local feature weights.

[0091] 2. Secondly, scale weights are obtained by combining scale-aware rules. The dimension is [B,2]: where the weights for small targets are (0.4, 0.8), medium targets are (0.6, 0.6), and large targets are (0.8, 0.4).

[0092] 3. Finally, the prediction weights and scale weights are averaged element by element to obtain the initial weight vector:

[0093]

[0094] in, and These are the weights for predicting global features and the weights for predicting local features, respectively. and These are the scale-wide feature weights and the scale-local feature weights, respectively.

[0095] Step 3: Error Feedback Adjustment:

[0096] 4. During training, calculate the position error and orientation error based on the network prediction results and the true labels:

[0097]

[0098]

[0099] in, and The first The predicted and true location vectors of each sample. and These are the mean and standard deviation of the position vectors, respectively, obtained from the statistics of the real position vectors in the training set. This eliminates the absolute offset of the position data and the error calculation imbalance caused by the difference in the range of position values, i.e., the position value of nearby targets is smaller and the position value of distant targets is larger. and The first The predicted direction vector and the true direction vector of each sample.

[0100] 5. Input the error vector into the adjusted quantum network (channel structure 2→32→16→2), and output the weight correction amount. The initial weights are corrected according to the following formula:

[0101]

[0102] wherein, is the weight vector after error adjustment, is a trainable error learning rate, is a weight correction amount.

[0103] 6. The modified weight is interval constrained (limited in [0.1, 0.9]) and normalized to obtain the final weight vector .

[0104] (5) Position / direction estimation head

[0105] Step1: Position estimation head

[0106] 1. Global feature processing: Channel compression is performed on the weighted global feature map, which is mapped from 512 channels to 128 channels, and global average pooling operation is used to extract global spatial information;

[0107] 2. Local feature processing: Channel mapping is performed on the weighted local feature map, which is mapped from 128 channels to 128 channels, and average pooling is used to extract local structure information;

[0108] 3. Feature fusion: The features output by the global path and the local path are spliced, with a channel size of 256, and a 3x3 convolution is performed to compress the channel to 128 channels to enhance the cross-channel correlation, completing the feature fusion;

[0109] 4. Position regression: The fused features are input into a fully connected network (with scales of 128→64→3) to output a preliminary predicted position vector , which is adjusted by error feedback to obtain the final position prediction vector .

[0110] Step2: Direction estimation head

[0111] 1. Feature preprocessing unit: Channel compression (mapping from 128 channels to 64 channels) is performed on the weighted local features to enhance the target local structure information;

[0112] 2. Spatial pyramid pooling (SPP) enhancement: The preprocessed features are pooled at three different scales, i.e., 1x1, 2x2, and 4x4 pooling operations, respectively outputting 64, 256, and 1024-dimensional feature vectors;

[0113] 3. Feature fusion: The multi-scale pooled features are flattened and spliced to form a 1344-dimensional multi-scale fusion feature;

[0114] 4. Direction Regression: The fused features are processed through a fully connected network (size successively 1344→256→128→4) to output a four-dimensional direction vector. L2 normalization is then applied to ensure the magnitude of the direction vector is 1, thus obtaining a preliminary direction prediction vector. The final direction prediction vector adjusted with error feedback .

[0115] (6) Parameter initialization

[0116] Step 1: Depthwise separable convolutional layer, He normal initialization:

[0117]

[0118] in, When initializing a depthwise separable convolutional layer, the mean is set to 0 and the standard deviation is 0. The weight parameters are initialized using a normal distribution. The number of channels in the input feature map. The size of the kernel of the depth-separable convolutional layer.

[0119] Step 2: Fully connected layer, initialized with Xavier algorithm, mean 0, standard deviation . normal distribution ;

[0120] in, and These represent the number of input channels and the number of output channels for a fully connected layer, respectively.

[0121] Step 3: Batch Normalization (BN) layer, scaling factor =1, Offset coefficient =0, the moving mean is 0, and the moving variance is 1;

[0122] Step 4: Forward validation to ensure that the output dimensions meet expectations, namely, 4-dimensional bounding box, 3-dimensional position vector, and 4-dimensional orientation vector.

[0123] In implementation, step S104 involves designing a multi-task joint loss function and training optimization strategy, training the network model, and saving the optimal model. The specific process is as follows:

[0124] (1) Joint loss function for multiple tasks

[0125] The design includes bounding box detection loss, position estimation loss, and orientation estimation loss. The bounding box loss uses the generalized intersection-union (IUU) loss, calculated as follows:

[0126]

[0127] in, To perform an intersection-union comparison between the predicted bounding box and the ground truth bounding box, The minimum area of ​​the enclosed rectangle. The area of ​​the union;

[0128] The location estimation loss uses a normalized Euclidean distance metric to calculate the difference between the predicted location vector and the true location vector. The formula is as follows:

[0129]

[0130] in, and The first Each sample ultimately predicts its location vector and its corresponding true location vector. and These are the mean and standard deviation of the position vectors, respectively, obtained from statistics of the real position vectors in the training set.

[0131] The orientation loss is jointly optimized using normalized angle error and smoothed L1 loss, and the calculation formula is as follows:

[0132]

[0133] in, It is an inverse cosine function. To smooth the L1 loss function, and The first The final direction vector and the true direction vector of each sample.

[0134] The joint loss for multiple tasks is calibrated to the order of magnitude, with calibration coefficients of 1.0, 3.0, and 2.5. Then, the fusion weights are determined based on the actual task requirements of boundary detection, position estimation, and orientation estimation. Next, the individual losses are weighted and fused to obtain the total joint loss for multiple tasks, calculated using the following formula:

[0135]

[0136] in, , and , respectively, are the calibration coefficients for bounding box detection loss, position estimation loss, and orientation estimation loss, and 0.2 / 0.3 / 0.5 are the fusion weights for multi-task loss.

[0137] (2) Training optimization strategy

[0138] During training, the AdamW learning rate optimizer is set with an initial learning rate of [value missing]. The weight decays to Dynamic parameters are set to ;

[0139] In terms of learning rate scheduling, a standard cosine annealing strategy is used to gradually decrease the learning rate to [a certain value]. ;

[0140] In terms of gradient optimization, mixed precision computation is adopted, and the numerical precision is uniformly reduced for the forward and backward propagation of each batch of data; norm clipping is performed on the obtained gradient to prevent gradient explosion; training warm-up is achieved by accumulating the number of gradient steps in dynamic gradient accumulation - the above data is processed in 4 steps in the early stage of training, and then reduced to 1 step per epoch; at the same time, the loss is amplified by 128 times to avoid gradient underflow under low precision.

[0141] (3) Training and verification process

[0142] The training and verification results of this experiment are shown in Figures 5(a)-5(f). The specific operation steps are as follows:

[0143] 1. Put the network into training mode and load the training set data; calculate the joint loss through forward propagation. Backpropagation is performed to accumulate gradients; after the parameters are updated, the display cache is cleared to ensure training stability.

[0144] 2. During the validation phase, the network is placed in validation mode, gradient calculation is disabled, validation set data is loaded, and the relevant loss of the validation set is calculated. Simultaneously, the average intersection-over-union (AUC) ratio is used to calculate the loss. Location rating and direction rating The performance of the trained model is evaluated using the following formula:

[0145]

[0146]

[0147] in, and These are the predicted bounding box and the ground truth bounding box, respectively. and The first Each sample ultimately predicts its location vector and its corresponding true location vector; and The first The final direction vector and the true direction vector of each sample.

[0148] 3. In this invention, when... , or , and better than the historical best, save the best model parameters (best_model.pth), and save the optimizer and learning rate scheduler state; in the validation phase, if the loss of the validation set decreases by less than 25 epochs, terminate the training process and restore the best historical model to prevent overfitting.

[0149] In implementation, the specific method of step S105 is: after completing model training, the present application performs inference testing on the model to verify its real-time performance and stability, wherein one inference result is as shown in Figure 6 , including target detection and direction prediction.

[0150] The process of the present application is implemented based on a single training device, the GPU used is NVIDIA RTX A6000 (48 GB GDDR6, peak 309 TFLOPS FP16, driver 535.247, CUDA 12.2), the CPU is Intel Xeon Gold 6248R x2 (40 cores 80 threads), the training framework uses PyTorch 2.4.1 + cu11.8, the model weight is 24MB, and the peak occupancy of one forward propagation is 97MB, and the specific steps include:

[0151] (1) Read the best model file completed by training from the training device NVMe SSD storage path ( / data / model / ), which contains network parameters, optimizer and learning rate scheduler state. Load the parameters through the device graphics processor (GPU) and fix the mapping to the device number cuda:0 to avoid cross-device memory call errors. After loading, put the network in inference mode, turn off the Dropout random inactivation and batch normalization (BN) update functions in the training phase to ensure the stability of the inference calculation logic.

[0152] (2) Load the test data from the training device, with a resolution of 1920x1200, and perform consistent preprocessing procedures as in the training phase, including:

[0153] Step 1: Image scaling: use bilinear interpolation to unify to 384x240 pixels;

[0154] Step 2: Pixel normalization: linearly map pixel values from [0, 255] to [0, 1];

[0155] Step 3: Format conversion: convert to GPU tensor format, and disable data augmentation operations in the training phase.

[0156] (3) Set the inference parameter, set the batch size BATCH_SIZE to 16, and close the gradient calculation to reduce the memory occupation; set the GPU optimization parameter, and automatically select the optimal convolution algorithm;

[0157] (4) The specific batch inference iteration is as follows:

[0158] Step1: load 16 frames of test images into the memory through the CPU, and then transmit them to the GPU memory;

[0159] Step2: input into the pose estimation network, and perform feature extraction, dynamic weight fusion and spatial pyramid pooling operation layer by layer;

[0160] Step3: output the following prediction results: boundary box prediction value , position vector prediction value , and direction vector prediction value .

[0161] Step4: the final result is stored in the dictionary format to the NVMe SSD, and the current batch memory is released.

[0162] The above specific implementation steps completely cover the whole process of the non-cooperative target monocular pose estimation method from environment building to model landing. Through the precise selection of hardware configuration, the compatibility adaptation of software stack, and the fine design of five core steps of data processing, network construction, training optimization and inference test, the reproducibility and efficient landing of the technical scheme are realized. At the data level, based on the SPEED data set, key labels are supplemented and training / validation subsets are reasonably divided, and the sample diversity is improved combined with the targeted data enhancement strategy; at the network level, the multi-stage depth separable convolution, Sobel gradient edge detection and scale-sensitive dynamic weight fusion are used to balance the feature extraction accuracy and the calculation efficiency; at the training and inference level, the multi-task joint loss function is used to calibrate the task weight, the mixed precision calculation, the gradient clipping strategy is used to ensure the training stability, and the low memory occupation and low delay characteristics during inference are used to perfectly adapt to the limited computing resources of the satellite device. The whole implementation process does not need to be cooperated with multiple devices, the steps are closely connected, and the exception handling mechanism is perfect, which not only ensures the high accuracy of the model in the boundary box detection, 3D position estimation and direction prediction, but also meets the core needs of real-time and lightweight in the satellite scene, and provides a complete and feasible technical path for the engineering application of the non-cooperative target pose estimation.

[0163] The application also provides a non-cooperative spacecraft monocular pose estimation system combining scale-sensitive dynamic weight and local feature multi-scale extraction, which mainly comprises:

[0164] An image acquisition module is configured to acquire a shooting image of a non-cooperative spacecraft.

[0165] a pose estimation module, configured to input the photographed image into a pre-constructed target pose estimation model to obtain pose estimation of the non-cooperative spacecraft, including a position vector and a direction vector;

[0166] a model training module, configured to train the model, specifically including: extracting global feature maps and local feature maps from images photographed at different angles in a training set by using the constructed initial target pose estimation model; detecting the non-cooperative spacecraft target in the images to obtain geometric features of the target and estimate the size of the target; generating spatial weights of the global feature maps and the local feature maps, and generating scale weights of the global feature maps and the local feature maps according to the size of the target, and fusing the two kinds of weights respectively to obtain fusion weights of the two kinds of feature maps; respectively weighting and fusing the two kinds of feature maps according to the fusion weights, estimating the estimated position vector by combining the geometric features of the target and the fused feature maps through a position estimation head, and estimating the estimated direction vector by using the weighted local feature maps through a direction estimation head;

[0167] calculating the error between the estimated position information and the actual position information of the target and the error between the estimated direction information and the actual direction information, generating a corresponding weight adjustment amount according to the error, optimizing and updating the fusion weights, and re-performing pose estimation; continuously updating the weight adjustment amount, iteratively optimizing the model, and verifying the model through a validation set to obtain an optimal target pose estimation model.

[0168] Further, the system further includes an image preprocessing module configured to pre-process the photographed image of the non-cooperative spacecraft, including rotation, brightness and contrast adjustment.

[0169] The constructed target pose estimation model mainly includes a gray scale adaptation module, a multi-stage feature extraction module, a target detection module, a dynamic weight fusion module and a position and direction estimation module, and each module cooperates to realize target, position and direction estimation of a single-channel gray scale image.

[0170] The dynamic weight fusion module implements the following steps:

[0171] Step 3.1a: average the batch initial weights output by the initial weight predictor and the batch scale weights output by the scale perception sub-module according to the corresponding elements of the samples to obtain the batch fusion initial weights.

[0172] In the above steps, the initial weight predictor receives target bounding box coordinate parameters through a 3-layer fully connected network, the input dimension is 4, the hidden layer dimension is 64 / 128 / 64 in sequence, and the output is normalized batch initial weights , the dimension is [B, 2], each row includes global feature weight and local feature weight, realizing weight adaptive prediction based on target position information, i.e. explicitly indicating spatial attribute and scale attribute of the target, adapting the diversity of target scale and position, avoiding the limitation of using fixed weight, at the same time providing reasonable initial value for error feedback mechanism, ensuring stable convergence of training;

[0173] The scale perception sub-module judges the target type according to the preset threshold and , the width and height of the target boundary are input and , when , it is determined as a small target; when , it is determined as a large target; the rest is a medium target; and the batch scale weight is output, wherein: the global feature weight of the small target is 0.4, and the local feature weight is 0.8; the global / local feature weight of the medium target is 0.6; the global feature weight of the large target is 0.8, and the local feature weight is 0.4. Since the scale weight design is based on target scale preference, normalization is not required;

[0174] The batch initialization weight and the batch scale weight are arithmetically averaged according to the corresponding elements of the samples to obtain the batch fusion initialization weight:

[0175]

[0176] , wherein represents the th sample picture, is the fusion initial weight of the th sample, is the initialization weight of the th sample, is the scale weight of the th sample. and are the predicted global feature weight and the predicted local feature weight, respectively. and are the scale global feature weight and the scale local feature weight, respectively.

[0177] Step 3.2a: In the training stage, the batch position error and the batch direction error are calculated, and the batch error feature is constructed, the scale is [B, 2], B is the number of batch samples, and 2 corresponds to the normalized position error and direction error;

[0178] In the above steps, the batch position error is calculated. The preliminary 3D position coordinates output by the position estimation branch are obtained and the corresponding real 3D position coordinates , the batch position error is calculated, and the formula is as follows:

[0179]

[0180] wherein, and are the predicted position vector and the real position vector of the i-th sample, respectively. and are the mean and standard deviation of the position vector, respectively, which are obtained by statistical analysis of the real position vectors of the training set, eliminating the absolute deviation of the position data and the error calculation imbalance caused by the difference in the range of position values, i.e. the position value of the near target is smaller, and the value of the far target is larger;

[0181] Batch direction error calculation. Obtain the preliminary direction vector output by the direction estimation branch and the corresponding real direction vector , calculate the angle distance by vector dot product to obtain the batch direction error , and the formula is as follows:

[0182]

[0183] wherein, and are the predicted direction vector and the real direction vector of the i-th sample, respectively. L2 normalization is performed on the batch position error

[0184] and the direction error , respectively, to obtain the normalized position error and the direction error ; and the two are spliced into a two-dimensional error feature vector according to the samples.

[0185] Step 3.3a: input the batch error feature into the error feedback adjuster, and output the feature batch weight adjustment amount; combine the learnable error learning rate to update the fused initial weight:

[0186] In the above steps, the batch error feature is input into the error feedback adjuster, and through a 3-layer fully connected network, the input layer dimension is 2, the hidden layer dimension is 32 / 16 in turn, and the output layer uses the Tanh activation function. The position and direction error are mapped to the same dimension as the original weight adjustment amount, and the batch weight adjustment amount is output, realizing the adaptive prediction of the adjustment amount based on the error;

[0187] Combine the learnable rate to initialize the batch fused weight​ The update is made as follows:

[0188]

[0189] wherein, is the weight vector after error adjustment, is a trainable error learning rate, is the weight correction amount.

[0190] But to avoid extreme values of the weight, the updated weight is limited in the range [0.1, 0.9];

[0191] The limited weight is normalized by the Softmax function to ensure that the sum of the global feature weight and the local feature weight of each sample is 1, and the batch final adjustment weight is obtained to meet the numerical constraint of feature weighted fusion, and the calculation formula is as follows:

[0192]

[0193]

[0194] wherein, and are the global feature weight and the local feature weight of the i-th sample after error adjustment; and are the global feature weight and the local feature weight of the i-th sample after normalization. Step 3.4a: The normalized global weight is extended to a weight tensor matching the dimension of the global feature map, and the weighted global feature map is obtained by element-wise multiplication with the global feature map; similarly, the local weight and the local feature map are processed to obtain the weighted local feature map.

[0195] In the above steps, the batch final adjustment weight The global feature weight in the batch final adjustment weight

[0196] is extended to a four-dimensional weight tensor with a dimension of [B, 1, 1, 1] matching the dimension of the global feature map, and then multiplied element-wise with the global feature map to obtain the weighted global feature map, realizing dynamic weight distribution of the global feature. Similarly, the local feature weight in the batch final adjustment weight

[0197] is extended to a four-dimensional weight tensor with a dimension of [B, 1, 1, 1], and then multiplied element-wise with the local feature map to obtain the weighted local feature map, realizing dynamic weight distribution of the local feature.

[0198] ​Step 3.5a: Differentiated assignment of weighted feature maps according to the task characteristic difference between position estimation and direction estimation, ensuring that both branches can obtain adaptive feature input;

[0199] In the above step, the weighted global feature map obtained in step 3.4a is input into the position estimation branch together with the weighted local feature map. The global feature map provides spatial correlation information of the target and the scene, and the local feature map provides target contour detail information. Both of them realize information complementation through channel splicing and convolution fusion, and finally output accurate 3D position coordinates;

[0200] Similarly, the weighted local feature map obtained in step 3.4a is input into the direction estimation branch alone to maintain the exclusive dependence of direction estimation on target local geometric structure;

[0201] The multi-feature fusion design process of the position estimation head is as follows:

[0202] Step 3.1b: The position estimation head includes global path and local path, which respectively perform convolution and pooling processing on the globally fused global feature and the locally fused local feature to extract global position information and local structure information;

[0203] In the above step, the weighted global feature map output by step 3.4a is received. First, it is subjected to 1 layer of convolution for feature dimension reduction and semantic enhancement, with batch normalization (BN) and ReLU6 activation function to suppress gradient disappearance and improve feature nonlinear expression ability; then adaptive average pooling is used to uniformly compress the feature map spatial size to , obtaining the global path intermediate feature with size [B, 64, 4, 4].

[0204] The local path receives the weighted local feature map output by 3.4a, and through 1 layer of convolution, BN and ReLU6 activation function, it focuses on feature extraction of target local contour details; then adaptive maximum pooling is also used to compress the feature map spatial size to , obtaining the local path intermediate feature with size [B, 64, 4, 4].

[0205] Step 3.2b: The output features of the global path and the local path are spliced and convolved to form fused position features;

[0206] In the above step, the global path intermediate feature and the local path intermediate feature are spliced in the channel dimension to obtain a spliced feature with size [B, 128, 4, 4], which contains both global correlation and local constraint information at this time;

[0207] The spliced feature is subjected to 1 layer of Convolution, to realize the information complement of global and local features.

[0208] Adaptive average pooling is used for the feature map after convolution fusion, which is compressed into a global fusion position feature vector with a scale of [B, 64]. The vector not only retains the global spatial correlation but also contains local structural details, providing comprehensive feature support for position regression.

[0209] Step 3.3b: The preliminary three-dimensional position (x, y, z) of the target is output by the position regression network after the fusion features.

[0210] In the above steps, the global fusion position feature vector is input into a 3-layer fully connected network with hidden layer dimensions of 32 / 16, and ReLU6 is used as the activation function network for position regression. The output is a batch of preliminary 3D coordinate positions .

[0211] Step 3.4b: A bounding box feature processor is provided at the same time to convert the bounding box coordinates output by the target detection module into geometric features and fuse them with the preliminary position results. The final three-dimensional position is output by the regression network to realize the joint optimization of multiple features and geometric constraints.

[0212] In the above steps, the bounding box feature processor receives the target bounding coordinates output by the target detection module. First, the derived geometric features of the bounding box are calculated, i.e., the width and height of the target and the center coordinates, to obtain a 4-dimensional geometric feature vector ;

[0213] Then, a 1-layer fully connected network is used to map the 4-dimensional geometric feature vector to a 32-dimensional geometric feature, which is concatenated with the preliminary 3D position coordinates output by step 3.3b to obtain a 35-dimensional joint optimization feature.

[0214] The joint optimization feature is input into a final regression network composed of 2-layer fully connected layers to output the optimized final 3D coordinate position .

[0215] The spatial pyramid pooling enhancement design process of the direction estimation head is as follows:

[0216] Step 3.1c: The direction estimation head includes a feature preprocessing unit and a direction regression unit.

[0217] In the above steps, the feature preprocessing unit performs spatial information enhancement and noise suppression on the weighted local feature map output by the dynamic weight fusion module to improve the expression ability of the target geometric structure in the local feature. The direction regression unit covers the scale difference of different local structures of the target through multi-scale pooling, while avoiding the interference of scene redundant information introduced by global features.

[0218] Step 3.2c: The feature preprocessing unit convolves the weighted local features with a nonlinear activation function to enhance the expression of local spatial information.

[0219] The weighted local feature maps output by the dynamic weight fusion module are received, and a lightweight convolution layer is used to process them. The spatial receptive field of the small convolution kernel covers part of the information in the local neighborhood of the target, strengthening the geometric correlation between adjacent pixels. The lightweight convolution layer processes the weighted local feature maps, and the spatial receptive field of the small convolution kernel covers part of the information in the local neighborhood of the target, strengthening the geometric correlation between adjacent pixels.

[0220] After convolution processing, the batch normalization (BN) and ReLU6 activation function are applied in sequence, and finally the preprocessed feature map is output.

[0221] Step 3.3c: The direction regression unit uses a spatial pyramid pooling structure to pool and extract preprocessed features at multiple spatial scales, obtaining multi-scale feature representations.

[0222] In the above steps, the spatial pyramid pooling (SPP) structure is used to pool the preprocessed feature maps at multiple spatial scales. The step 3.2c is simultaneously input into the pooling branch of 、 、 The maximum pooling strategy is used to output [B, 32, 1, 1], [B, 32, 2, 2], and [B, 32, 4, 4] multi-scale pooling features.

[0223] Step 3.4c: The multi-scale pooling features are flattened and concatenated to form multi-scale fusion features, and the normalized four-dimensional direction vector is output through the fully connected regression network. This approach simultaneously utilizes local details and global spatial relationships in direction estimation, improving the accuracy of pose direction prediction.

[0224] In the above steps, the three groups of pooling features output by step 3.3c are flattened and converted into one-dimensional feature vectors with dimensions of 32, 128, and 512, respectively.

[0225] The three one-dimensional feature vectors are concatenated in the channel dimension to obtain multi-scale fusion features with a dimension of 672.

[0226] The multi-scale fusion features are input into the lightweight fully connected regression network, and finally a four-dimensional direction vector is output.

[0227] The output four-dimensional vector is subjected to L2 normalization, and finally a batch-normalized four-dimensional direction vector is output. The calculation formula is as follows:

[0228]

[0229] The function and flow design process of the target detection module are as follows:

[0230] Step 3.1d: The target detection module comprises an edge enhancement unit, a multi-scale feature fusion unit and a bounding box regression unit.

[0231] In the above step, the edge enhancement unit extracts gradient information of the feature map based on a fixed edge detection operator, generates an edge weight map, and fuses the original feature by weighting to highlight the target boundary profile.

[0232] The multi-scale feature fusion unit splices and fuses the weighted feature after edge enhancement and the deep feature output by the 6th layer of the depth separable convolution after spatial alignment, so as to cover the feature requirements of different scale targets and improve the universality of small target and large target bounding box detection.

[0233] The bounding box regression unit convolves and regresses the fused multi-scale feature to output the bounding box coordinates of the target in the original image scale, providing direct input for subsequent scale calculation.

[0234] Step 3.2d: The edge enhancement unit extracts gradient information of the feature map based on a fixed edge detection operator, generates an edge weight map, and fuses the original feature by weighting to highlight the target boundary profile.

[0235] In the above step, the output feature map of the edge enhancement unit in the 4th stage of depth separable convolution is received as the middle stage feature map .

[0236] The edge enhancement unit obtains a horizontal gradient map by convolution of the built-in Sobel-x operator ; obtains a vertical gradient map by convolution of the built-in Sobel-y operator .

[0237] The gradient amplitude is calculated to obtain an edge gradient map as an edge weight map, and the calculation process is as follows:

[0238]

[0239] The middle stage feature map is multiplied element by element to obtain a feature map after edge enhancement, and the fusion formula is:

[0240]

[0241] Step 3.3d: The multi-scale feature fusion unit splices and fuses the weighted feature and the deep feature in a higher stage after spatial alignment, so as to improve the accuracy of target bounding box detection under different scales.

[0242] In the above steps, the edge enhancement feature map output in step 3.2d will be used. Bilinear interpolation is used to downsample its spatial size to a deep feature map output by a depth-separable convolution in stage 6. Same size;

[0243] The spatially aligned weighted feature map is concatenated with the deep feature map along the channel dimension to obtain a multi-scale fused feature map. .

[0244] Step 3.4d: The bounding box regression unit performs convolutional regression on the fused features and outputs the bounding box coordinates of the target at the original image scale;

[0245] In the above steps, the bounding box regression unit adopts an anchor box matching strategy, combining the regression parameters output by the convolution with preset anchor boxes to calculate the bounding box coordinates of the target at the feature map scale. Then, by comparing the scale of the feature map with that of the original image, the bounding box coordinates at the scale of the original image are mapped back. The calculation formula is as follows:

[0246]

[0247] in, A fixed scaling factor is used between the size of the original image and the size of the feature map output by the boundary regression unit.

[0248] The final output is the bounding box coordinates of the original image in batches. .

[0249] Step 3.5d: The bounding box coordinates are not only used to calculate the scale information of the target and provide a scale division basis for the dynamic weight fusion module, but also to provide geometric features for position estimation that directly participate in 3D position estimation.

[0250] In the above steps, the coordinates of the original image scale bounding box output in step 3.4d are used. Calculate the width of the target With height ,Will Scale information as a target;

[0251] scale information The scale-aware submodule of the input dynamic weight fusion module is used to determine the scale based on a preset threshold. and Input the width of the target boundary. and height Determine the target type. When When, it is judged as a small goal; when At this time, it is determined that it is a large target; the rest is a medium target; the type of the target is determined to support the initial generation of the subsequent dynamic weight;

[0252] In addition, the output bounding box coordinates are also converted into multi-dimensional geometric features by the bounding box feature processor, including the aspect ratio, center coordinates and bounding box area of the bounding box, to form a geometric feature vector with a dimension of [B, 3]. The geometric feature vector is spliced with the preliminary 3D position vector output in step 3.3b to obtain a joint feature with a dimension of [B, 6], which is input into the position estimation branch to directly participate in 3D position estimation.

[0253] The design process of the model training method is as follows:

[0254] Step 1: data preparation and loading. Obtain a data set containing single-channel grayscale images and corresponding labels, perform enhancement processing such as rotation, blurring, brightness / contrast adjustment on the images, and divide the training set and the validation set according to the proportion;

[0255] In the above steps, a single-channel grayscale image data set of a non-cooperative target is obtained, and the image size is adjusted to Each image corresponds to 3 types of labels, i.e. the bounding box label , the real 3D position coordinates of the target , and the normalized real direction vector of the target ;

[0256] The single-channel grayscale image is subjected to data enhancement in the form of geometric transformation, noise and brightness / contrast adjustment;

[0257] Since the validation set of the data set used in the present application does not have corresponding labels, all the enhanced data sets are divided into a training set and a validation set according to a ratio of 8:2.

[0258] Step 2: network and optimization configuration. Load the network parameters to the computing device, and set the optimizer and the learning rate scheduling strategy; perform learning rate preheating in the early stage of training, and then use a periodic decay strategy to stabilize convergence;

[0259] In the above steps, the initialization parameters of the multi-stage feature extraction network composed of 6-stage depth separable convolution, the dynamic weight fusion module and the target detection module, i.e. the He normal initialization of the convolution kernel and the Xavier initialization of the fully connected layer, are loaded into the computing device;

[0260] The bounding box loss adopts an improved GIoU loss, which avoids training shock through gradient scaling and extreme value clamping, and finally normalizes the bounding box coordinates to the [0, 1] interval, and the calculation formula is as follows:

[0261]

[0262] in, To perform an intersection-union comparison between the predicted bounding box and the ground truth bounding box, The minimum area of ​​the enclosed rectangle. The area of ​​the union;

[0263] The location loss is calculated using the normalized Euclidean distance formula as follows:

[0264]

[0265] in, and The first Each sample ultimately predicts its location vector and its corresponding true location vector. and These are the mean and standard deviation of the position vectors, respectively, obtained statistically from the true position vectors in the training set; in this invention, they are statistically obtained as follows: , .

[0266] The orientation loss is calculated using the following formula, which combines normalized angle error and smoothed L1 loss:

[0267]

[0268] in, It is an inverse cosine function. To smooth the L1 loss function, and The first The final direction vector and the true direction vector of each sample.

[0269] The multi-task joint loss uses weighted fusion weights of boundary detection loss, position estimation loss, and orientation estimation loss, calibrated to an order of magnitude, with calibration coefficients of 1.0, 3.0, and 2.5, respectively. Combined with the training task priority weights, the final multi-task joint total loss is calculated as follows:

[0270]

[0271] in, , and , respectively, are the calibration coefficients for bounding box detection loss, position estimation loss, and orientation estimation loss, and 0.2 / 0.3 / 0.5 are the fusion weights for multi-task loss.

[0272] Configure the AdamW optimizer, with an initial learning rate set to... The weight decays to The momentum parameter is set to ; In terms of learning rate scheduling, the standard cosine annealing strategy is adopted, which makes the learning rate gently decrease to ; In terms of gradient optimization, gradient accumulation and gradient clipping are enabled to prevent gradient explosion.

[0273] Step 3: Forward propagation feature processing. The image generates feature maps through the gray scale adaptation module and the multi-stage feature extraction module. The dynamic weight fusion module generates fusion weights based on the bounding box, and the local and global features are weighted and input into the position estimation head and the direction estimation head respectively, and finally the preliminary results are output.

[0274] In the above steps, the gray scale image is converted into 3 channels after passing through the gray scale adaptation module, and then input into the initial convolutional layer composed of convolution, BN and ReLU6;

[0275] Then, the multi-stage feature extraction module composed of 6 layers of depth separable convolution is used. The feature map representing local features is output by the 2nd layer of depth separable convolution , and the global feature map representing deep features is output by the 6th layer of depth separable convolution ;

[0276] Then, the intermediate feature map for edge enhancement is output by the 4th layer of depth separable convolution ;

[0277] The target detection module performs edge enhancement and multi-scale fusion on , and outputs the bounding box coordinates of the original image scale ;

[0278] The dynamic weight fusion module predicts the initial weight based on the bounding box coordinates and the normalized initial weight , and fuses it with the scale-aware weight to generate the initial weight ;

[0279] According to the fused initial weight , the and are weighted. The weighted global feature map and the weighted local feature map are obtained.

[0280] The position estimation fusion head fuses and , and outputs the preliminary 3D position ;

[0281] The direction estimation head performs SPP pooling and channel full connection on , and outputs the preliminary normalized direction vector ;

[0282] Step 4: Weight adjustment and final output. Based on the error calculated from the preliminary prediction and the label, the fusion weight is adjusted through error feedback, the features are reweighted, and the final position and direction results are outputted;

[0283] In the above steps, according to the results of the preliminary prediction, the position error and the direction error are normalized and spliced into a 2-dimensional error feature vector ;

[0284] The is outputted through the error feedback adjuster, and the weight adjustment amount is updated by combining the learnable error learning rate with the generated in step 3, to obtain ;

[0285] The is numerically constrained and normalized: first clamped to the [0.1, 0.9] interval, and then the final adjustment weight is calculated by the Softmax function ;

[0286] Based on the , the global and local features are reweighted, i.e., the and the generated in step 3 are reused, and the logic of weight expansion and element-wise multiplication in step 3 is followed to obtain the optimized weighted global feature map and the weighted local feature map ;

[0287] The position estimation head fuses the , and the bounding box geometry features in step 3.4b to finally output the 3D position ; the direction estimation head receives the optimized weighted local feature map, and finally normalizes the output direction vector to output the final direction vector ;

[0288] Step 5: loss calculation and backpropagation. Calculate the bounding box accuracy, position score, and direction score on the validation set; save the complete model when any core indicator is better than the historical best value; use the early stopping mechanism to terminate training when there is no performance improvement for a long time, and keep the best model.

[0289] In the above steps, after the forward propagation is completed, the total loss , the bounding box loss , the position loss , and the direction loss are calculated and outputted, and the position score and the direction score are also outputted.

[0290] In back propagation, mixed precision training is adopted: the loss is amplified by 128 times before the gradient is calculated, when the gradient accumulates to a threshold, the optimizer updates the gradient driven by the gradient scaler (GradScaler), and the CUDA cache is cleaned up at the end of each batch;

[0291] After each training epoch, the evaluation indicators of bounding box accuracy, position score and direction score are calculated on the validation set , and , the calculation formula is as follows:

[0292]

[0293]

[0294] Among them, and are the predicted bounding box and the real bounding box respectively; and are the final predicted position vector of the th sample and its corresponding real position vector respectively; and are the final direction vector of the th sample and the real direction vector respectively.

[0295] Set the model saving rule, if the current position score or direction score is better than the historical best value, save the complete model, and set the early stopping mechanism to avoid overfitting, if there is no score improvement for 25 consecutive epochs, terminate the training;

[0296] Step 6: verification and saving. Evaluate the bounding box accuracy, position score and direction score on the validation set, save the model when the performance indicator is better than the historical best value, and terminate the training in advance when there is no performance improvement;

[0297] Step 7: inference stage optimization. Turn off data enhancement and error feedback mechanism during inference, directly calculate the fusion weight based on the target detection module and the initial weight prediction result; At the same time, the feature extraction module is lightened and quantized to meet the performance requirements of real-time inference.

[0298] In the above steps, all data enhancement and error feedback mechanisms of the dynamic weight fusion module are turned off in the inference stage, and is directly used as the final fusion weight, reducing the inference time;

[0299] Lightweight optimization is performed on the feature extraction module, which quantizes the 6-layer depth separable convolution layer to INT8 precision, and uses KL divergence to calibrate the quantity parameter; Replace adaptive pooling with fixed Max pooling reduces dynamic computation overhead; removes momentum updates of BN layers and Dropout layers dedicated to the training phase.

[0300] The embodiment has the advantages that: the prior art adopts fixed feature fusion weights or a single dimension rough division scale, and cannot adapt to the dynamic change of the scale of a non-cooperative target, the application divides the target into three scales of small, medium and large through a double-threshold scale perception module, and generates initial weights accordingly; and the feature fusion ratio is adjusted in real time through a dynamic weight fusion module, so that the cross-scale pose estimation accuracy fluctuation is controlled in a very small range, the small, medium and large targets can all obtain adaptive feature support, and accurate feature adaptation of cross-scale targets is realized.

[0301] In addition, the prior art relies on prior scale rules or single error feedback, and is prone to error accumulation in complex scenes, the application innovatively designs a double-error-driven dynamic weight adjustment mechanism: the position error and the direction error are spliced into 2-dimensional error features, and the error feedback adjuster outputs a weight adjustment amount in real time ; and a trainable error learning rate and weight constraint are introduced, which avoids one-sidedness of single error feedback and prevents extreme value shock of the weight;

[0302] Finally, in view of the contradiction between accuracy and real-time performance in the prior art, the application realizes balance through the cooperative design of a lightweight backbone and a dynamic mechanism: a 6-stage depth separable convolution is used to construct a backbone network, and SE attention modules and LCN normalization technology are used to retain key feature extraction capabilities while controlling the parameter size; and through multi-task optimization and gradient accumulation strategies, the engineering practical accuracy of position estimation and direction estimation is ensured, and finally the cooperative unity of "lightweight deployment" and "high-precision estimation" is realized.

[0303] The application further provides a computer readable storage medium, such as a flash memory, a hard disk, a multimedia card, a card-type memory (for example, an SD or DX memory, etc.), a random access memory (RAM), a static random access memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, an optical disk, a server, an App application store, etc., which stores a computer program, and the program is executed by a processor to realize corresponding functions. The computer readable storage medium of the embodiment is executed by the processor to realize the non-cooperative spacecraft monocular pose estimation method of the embodiment, which combines scale-sensitive dynamic weight and local feature multi-scale extraction.

[0304] It should be noted that, according to the needs of implementation, each step / component described in the present application can be split into more steps / components, or two or more steps / components or part operations of the steps / components can be combined into a new step / component, to achieve the purpose of the present application.

[0305] The size of the serial number of each step in the above embodiment does not mean the order of execution, and the execution order of each process should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0306] It should be understood that, for those skilled in the art, improvements or changes can be made according to the above description, and all these improvements and changes shall fall within the protection scope of the appended claims of the present application.

Claims

1. A method for monocular attitude estimation of non-cooperative spacecraft that combines scale-sensitive dynamic weights with multi-scale extraction of local features, characterized in that, Includes the following steps: Acquire images taken by non-cooperative spacecraft; The captured image is input into a pre-built target attitude estimation model to obtain the attitude estimate of the non-cooperative spacecraft, including the position vector and orientation vector; The training process of the model specifically includes the following steps: The initial target attitude estimation model is used to extract global and local feature maps from images taken from different perspectives in the training set. Non-cooperative spacecraft targets are detected in the images, their geometric features are obtained, and their size is estimated. Spatial weights for the global and local feature maps are generated based on pre-set values, and scale weights are generated based on the target size. These two weights are then fused to obtain a fusion weight for the two feature maps. The two feature maps are then weighted and fused again based on the fusion weight. A position estimation head is used to estimate the position vector by combining the target's geometric features with the fused feature map. Finally, a direction estimation head is used to estimate the direction vector by estimating the weighted local feature map. The error between the estimated position and orientation information of the target and the actual position and orientation information is calculated. Based on the error, a corresponding weight adjustment amount is generated, the fused weights are optimized and updated, and the attitude estimation is performed again. The weight adjustment amount is continuously updated to iteratively optimize the model, and the model is verified through a validation set to obtain the optimal target attitude estimation model.

2. The non-cooperative spacecraft monocular attitude estimation method combining scale-sensitive dynamic weights and local feature multi-scale extraction as described in claim 1, characterized in that, It also includes the step of preprocessing the acquired images taken by non-cooperative spacecraft, including rotation, brightness and contrast adjustments.

3. The non-cooperative spacecraft monocular attitude estimation method combining scale-sensitive dynamic weights and local feature multi-scale extraction as described in claim 1, is characterized in that... Each image in the training and validation sets includes the orientation vector, position vector, and target bounding box label of the non-cooperative spacecraft.

4. The non-cooperative spacecraft monocular attitude estimation method combining scale-sensitive dynamic weights and local feature multi-scale extraction as described in claim 1, characterized in that, The target attitude estimation model extracts global and local feature maps through multiple depthwise separable convolutional layers. At the same time, it detects non-cooperative spacecraft targets in the image to obtain bounding boxes, and obtains the geometric features of the target based on the coordinates of the bounding boxes.

5. The non-cooperative spacecraft monocular attitude estimation method combining scale-sensitive dynamic weights and local feature multi-scale extraction as described in claim 1, characterized in that, The location estimation head includes global and local paths. Specifically, it performs convolution and pooling on the global and local features with weighted fusion weights to extract global location information and local structural information. The output features of the global path and the local path are concatenated and fused through convolution to form a fused positional feature; The fused features are processed by a location regression network to output the preliminary three-dimensional position of the target; The target's geometric features are fused with its preliminary 3D position, and then a regression network is used to output the target's final 3D position.

6. The non-cooperative spacecraft monocular attitude estimation method combining scale-sensitive dynamic weights and local feature multi-scale extraction as described in claim 1, characterized in that, The fusion weight is specifically a weighted average of spatial weight and scale weight.

7. The non-cooperative spacecraft monocular attitude estimation method combining scale-sensitive dynamic weights and local feature multi-scale extraction as described in claim 1, characterized in that, When optimizing and updating the fusion weights using weight adjustment, specifically, an error learning rate-weighted weight adjustment is added to the fusion weights.

8. A non-cooperative spacecraft monocular attitude estimation system that combines scale-sensitive dynamic weights with multi-scale extraction of local features, characterized in that, include: The image acquisition module is used to acquire images taken by non-cooperative spacecraft. The attitude estimation module is used to input the captured image into a pre-built target attitude estimation model to obtain the attitude estimate of the non-cooperative spacecraft, including the position vector and orientation vector; The model training module, used for model training, specifically includes: extracting global and local feature maps from images taken from different perspectives in the training set using the constructed initial target attitude estimation model; detecting non-cooperative spacecraft targets in the images, obtaining the target's geometric features, and estimating the target size; generating spatial weights for the global and local feature maps, and generating scale weights for the global and local feature maps based on the target size; fusing the two weights separately to obtain a fused weight for the two feature maps; weighting and fusing the two feature maps separately according to the fused weight; estimating the estimated position vector by combining the target's geometric features and the fused feature map using a position estimation head; and estimating the estimated direction vector by estimating the weighted local feature map using a direction estimation head. The error between the estimated position and orientation information of the target and the actual position and orientation information is calculated. Based on the error, a corresponding weight adjustment amount is generated, the fused weights are optimized and updated, and the attitude estimation is performed again. The weight adjustment amount is continuously updated to iteratively optimize the model, and the model is verified through a validation set to obtain the optimal target attitude estimation model.

9. The non-cooperative spacecraft monocular attitude estimation system combining scale-sensitive dynamic weights and local feature multi-scale extraction as described in claim 8, characterized in that, The system also includes an image preprocessing module for preprocessing the acquired images taken by non-cooperative spacecraft, including rotation, brightness, and contrast adjustments.

10. A computer storage medium, characterized in that, It contains a computer program that can be executed by a processor, which performs the non-cooperative spacecraft monocular attitude estimation method that combines scale-sensitive dynamic weights and multi-scale extraction of local features as described in any one of claims 1-7.

Images