Monocular six-degree-of-freedom attitude estimation method and device for space target

Abstract

The invention provides a monocular six-degree-of-freedom attitude estimation method and device for a space target, and the method comprises the steps: inputting an image carrying edge information into a mask segmentation network model, obtaining binary mask information, and determining a current frame object image; inputting the current frame object image and the current frame image into a depth estimation network to obtain current frame object depth information; inputting an adjacent frame image of the current frame image and the current frame object depth information into a pre-trained space target six-degree-of-freedom attitude estimation network to obtain relative space target monocular six-degree-of-freedom attitude change information of the current frame and the adjacent frame; performing image reconstruction based on the relative space target monocular six-degree-of-freedom attitude change information and the current frame object depth information; and based on the relative space target monocular six-degree-of-freedom attitude change information and the absolute space target monocular six-degree-of-freedom attitude change information of the adjacent frame, determining the absolute space target monocular six-degree-of-freedom attitude change information of the current frame.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence and machine vision technology, and in particular to a monocular six-degree-of-freedom posture estimation method and device for a space target. Background Art

[0002] Monocular six-degree-of-freedom (6-DoF) attitude estimation of space targets is widely used in the aerospace field. Six-degree-of-freedom attitude estimation can accurately determine the position and direction of a spacecraft in space. It is the basis for real-time monitoring and attitude adjustment of spacecraft, and provides key data support for precise orbital control and navigation, ensuring smooth landing, navigation obstacle avoidance or precise docking.

[0003] At present, most methods for monocular 6-DoF attitude estimation of spacecraft mainly use single-frame images and prior information (3D detection box vertices, 3D models, radar point clouds, etc.) to supervise the spacecraft's attitude information. This limits their ability to fully utilize the temporal information in continuous images. Moreover, the above approach ignores the difficulty of obtaining the spacecraft's true attitude data and prior information. In addition, in the low-light and dim environment of deep space, the detailed information of the spacecraft is difficult to capture, which has a great impact on the final estimation accuracy.

[0004] It can be seen that the monocular six-degree-of-freedom posture estimation method of space targets in the related art has the technical problem of low estimation accuracy in low-light and dim environments. Summary of the invention

[0005] The present invention provides a monocular six-degree-of-freedom posture estimation method and device for a space target, so as to solve the defect that the monocular six-degree-of-freedom posture estimation method for a space target in the prior art has low estimation accuracy in a low-light dim environment, thereby improving the feature extraction capability of the network in a dim environment and alleviating the influence of the environment on the recognition accuracy.

[0006] The present invention provides a monocular six-degree-of-freedom posture estimation method for a space target, comprising the following steps. Preprocessing the input current frame image of the target spacecraft through an edge attention module to obtain an image carrying edge information; inputting the image carrying edge information into a pre-trained mask segmentation network model to obtain the binary mask information output by the mask segmentation network model, and determining the current frame object image based on the binary mask information; inputting the current frame object image and the current frame image into a depth estimation network to obtain the current frame object depth information output by the depth estimation network; inputting the adjacent frame images of the current frame image and the current frame object depth information into a pre-trained space target A six-degree-of-freedom attitude estimation network is used to obtain relative space target monocular six-degree-of-freedom attitude change information of the current frame and the adjacent frames output by the pre-trained space target six-degree-of-freedom attitude estimation network; image reconstruction is performed based on the relative space target monocular six-degree-of-freedom attitude change information and the object depth information of the current frame to obtain absolute space target monocular six-degree-of-freedom attitude change information of the adjacent frames; based on the relative space target monocular six-degree-of-freedom attitude change information and the absolute space target monocular six-degree-of-freedom attitude change information of the adjacent frames, the absolute space target monocular six-degree-of-freedom attitude change information of the current frame is determined.

[0007] According to a monocular six-degree-of-freedom posture estimation method for a space target provided by the present invention, before the adjacent frame images of the current frame image and the current frame object depth information are input into a pre-trained space target six-degree-of-freedom posture estimation network, the method further comprises: obtaining an initialized space target six-degree-of-freedom posture estimation network based on sequence images; inputting current frame image samples and adjacent frame image samples into the space target six-degree-of-freedom posture estimation network to obtain current frame mask samples, current frame depth map samples, and relative space target monocular six-degree-of-freedom posture change information samples between the current frame and the adjacent frames output by the space target six-degree-of-freedom posture estimation network; determining a loss function of the mask network based on the current frame mask samples and the true mask; and inputting the current frame depth map samples into the space target six-degree-of-freedom posture estimation network to obtain current frame mask samples, current frame depth map samples, and relative space target monocular six-degree-of-freedom posture change information samples between the current frame and the adjacent frames. The sample and the monocular six-degree-of-freedom posture change information sample relative to the spatial target are input into the image reconstruction network to obtain the reconstructed current frame image sample; based on the reconstructed current frame image sample and the current frame image sample, the structural similarity loss and the norm loss are determined as the reconstruction loss function; based on the reconstruction loss function and the edge-aware smoothing loss function, the loss function of the deep posture estimation network is determined; based on the loss function of the mask network and the loss function of the deep posture estimation network, the total loss function of the spatial target six-degree-of-freedom posture estimation network is obtained by weighting; when the deep posture estimation network reaches a preset number of training times or the loss function value of the total loss function is lower than a preset threshold, a pre-trained spatial target six-degree-of-freedom posture estimation network is obtained.

[0008] According to a monocular six-degree-of-freedom posture estimation method for a space target provided by the present invention, the current frame image of the input target spacecraft is preprocessed by an edge attention module to obtain an image carrying edge information, including: performing edge detection on the current frame image based on Sobel convolution by the edge attention module to obtain an edge amplitude map of the current frame image; performing Fourier transform on the edge amplitude map by the edge attention module to obtain a frequency domain feature map of the current frame image; performing discrete cosine transform based on the frequency domain feature map by the edge attention module to obtain an image carrying edge information.

[0009] According to a monocular six-degree-of-freedom posture estimation method for a space target provided by the present invention, the image carrying edge information is input into a pre-trained mask segmentation network model to obtain binary mask information output by the mask segmentation network model, including: extracting features of the image carrying edge information through a pre-trained mask segmentation network model to obtain features of different scales; inputting the features of different scales into an edge-based Fourier frequency domain attention module through the pre-trained mask segmentation network model to obtain a weighted frequency domain feature map output by the edge-based Fourier frequency domain attention module; decoding the weighted frequency domain feature map through the pre-trained mask segmentation network model to obtain the binary mask information of the current frame image.

[0010] According to a monocular six-degree-of-freedom posture estimation method for a space target provided by the present invention, the current frame object image and the current frame image are input into a depth estimation network to obtain the current frame object depth information output by the depth estimation network, including: encoding the current frame image through a depth estimation network to obtain a current frame multi-scale feature map; continuously convolving the current frame multi-scale feature map with the current frame object image through an attention gate feature selection module of the depth estimation network to obtain a current frame fusion feature; decoding the current frame fusion feature through a depth decoder of the depth estimation network to obtain a current frame depth map, wherein the current frame depth map is used to represent the current frame object depth information.

[0011] According to a monocular six-degree-of-freedom posture estimation method for a space target provided by the present invention, the adjacent frame images of the current frame image and the depth information of the current frame object are input into a pre-trained space target six-degree-of-freedom posture estimation network to obtain the relative monocular six-degree-of-freedom posture change information of the current frame and the adjacent frames output by the pre-trained space target six-degree-of-freedom posture estimation network, including: encoding the adjacent frame images of the current frame image through the pre-trained space target six-degree-of-freedom posture estimation network to obtain a multi-scale fusion image of adjacent frames; fusing the multi-scale fusion image of adjacent frames with the multi-scale feature map of the current frame through the attention gate feature selection module of the pre-trained space target six-degree-of-freedom posture estimation network to obtain the fusion features of the current frame and the adjacent frames; decoding the fusion features of the current frame and the adjacent frames through the posture decoder of the pre-trained space target six-degree-of-freedom posture estimation network to obtain the relative monocular six-degree-of-freedom posture change information of the current frame and the adjacent frames.

[0012] The present invention also provides a monocular six-degree-of-freedom posture estimation device for a space target, comprising the following modules: a preprocessing module, used to preprocess the current frame image of the input target spacecraft through an edge attention module to obtain an image carrying edge information; a first determination module, used to input the image carrying edge information into a pre-trained mask segmentation network model to obtain the binary mask information output by the mask segmentation network model, and determine the current frame object image based on the binary mask information; a first output module, used to input the current frame object image and the current frame image into a depth estimation network to obtain the current frame object depth information output by the depth estimation network; a second output module, used to output the adjacent image of the current frame image to the image of the current frame object; The frame image and the current frame object depth information are input into a pre-trained space target six-degree-of-freedom posture estimation network to obtain the relative space target monocular six-degree-of-freedom posture change information of the current frame and the adjacent frames output by the pre-trained space target six-degree-of-freedom posture estimation network; a reconstruction module is used to reconstruct the image based on the relative space target monocular six-degree-of-freedom posture change information and the current frame object depth information to obtain the absolute space target monocular six-degree-of-freedom posture change information of the adjacent frames; a second determination module is used to determine the absolute space target monocular six-degree-of-freedom posture change information of the current frame based on the relative space target monocular six-degree-of-freedom posture change information and the absolute space target monocular six-degree-of-freedom posture change information of the adjacent frames.

[0013] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, a monocular six-degree-of-freedom posture estimation method for a space target as described in any one of the above-mentioned methods is implemented.

[0014] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a monocular six-degree-of-freedom posture estimation method for a space target as described in any one of the above.

[0015] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the method for estimating the monocular six-degree-of-freedom posture of a space target as described in any one of the above is implemented.

[0016] The monocular six-degree-of-freedom posture estimation method and device of a space target provided by the present invention perform preprocessing through an edge attention module to obtain an image carrying edge information, call a mask segmentation network model based on the image carrying edge information, output binary mask information to determine the current frame object image; call a depth estimation network to perform depth estimation based on the current frame object image and the current frame image, and output the current frame object depth information; determine the relative space target monocular six-degree-of-freedom posture change information between the current frame and the adjacent frames based on the adjacent frame images and the current frame object depth information through a pre-trained space target six-degree-of-freedom posture estimation network; perform image reconstruction based on the relative space target monocular six-degree-of-freedom posture change information and the current frame object depth information to obtain the absolute space target of the adjacent frames. The monocular six-degree-of-freedom posture change information is combined with the depth information and the relative posture change information, so that it has strong robustness to common low-light and dim environments such as illumination changes and occlusions; based on the relative space target monocular six-degree-of-freedom posture change information and the absolute space target monocular six-degree-of-freedom posture change information of the adjacent frames, the absolute space target monocular six-degree-of-freedom posture change information of the current frame is determined. Therefore, by combining the relative posture change information and the absolute posture information of the adjacent frames, not only the relative changes between frames are considered, but also the known absolute posture is used as a reference, thereby reducing the cumulative error in posture estimation, and then solving the technical problem of low estimation accuracy in low-light and dim environments in the space target monocular six-degree-of-freedom posture estimation method in the related art. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced one by one below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 It is a flow chart of a monocular six-degree-of-freedom posture estimation method for a space target provided by the present invention.

[0019] Figure 2It is an overall flow chart of the monocular six-degree-of-freedom posture estimation method for a space target provided by the present invention.

[0020] Figure 3 The present invention provides a training flow chart of a spatial target six-degree-of-freedom posture estimation network based on sequence images.

[0021] Figure 4 It is the original RGB image provided by the present invention.

[0022] Figure 5 This is the image after high-frequency edge processing provided by the present invention.

[0023] Figure 6 It is the segmented mask image provided by the present invention.

[0024] Figure 7 It is an RGB image based on mask sampling provided by the present invention.

[0025] Figure 8 It is a posture estimation visualization image provided by the present invention.

[0026] Fig. 9 It is a structural flow chart of a monocular six-degree-of-freedom posture estimation device for a space target provided by the present invention.

[0027] Fig.10 It is a structural schematic diagram of a monocular six-degree-of-freedom posture estimation system for a space target provided by the present invention.

[0028] Fig.11 It is a schematic diagram of the physical structure of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0029] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0030] Monocular six-degree-of-freedom attitude estimation of space targets is widely used in the aerospace field. Six-degree-of-freedom attitude estimation can accurately determine the position and direction of a spacecraft in space. It is the basis for real-time monitoring and adjustment of the spacecraft's attitude, and provides key data support for precise orbital control and navigation, ensuring smooth landing, navigation obstacle avoidance or precise docking.

[0031] At present, most methods for monocular 6-DoF pose estimation of spacecraft mainly use single-frame images and prior information (3D detection box vertices, 3D models, radar point clouds, etc.) to supervise the pose information of spacecraft, which limits their ability to fully utilize the temporal information in continuous images. Moreover, the above approach ignores the difficulty of obtaining the real pose data and prior information of the spacecraft. In addition, in the low-light and dim environment of deep space, the detailed information of the spacecraft is difficult to capture, which has a great impact on the final estimation accuracy.

[0032] In the embodiment of the present invention, a sequence method is used to complete self-supervised training and recognition of posture, which greatly reduces the cost of obtaining prior information; for low-light and dim environments, a Fourier high-frequency attention mechanism is introduced to improve the network's feature extraction capability in dim environments and alleviate the impact of the environment on recognition accuracy.

[0033] A monocular six-DOF attitude estimation method for a space target provided by the present invention is mainly used to solve the problem of 6-DoF attitude estimation of a spacecraft under the influence of the environment and the difficulty in obtaining real data.

[0034] refer to Figure 1 , Figure 1 : is a flow chart of a monocular six-degree-of-freedom posture estimation method for a space target provided by the present invention, such as Figure 1 As shown, the method includes the following steps.

[0035] Step 101, preprocessing the current frame image of the input target spacecraft through the edge attention module to obtain an image carrying edge information.

[0036] In the embodiment of the present invention, the current frame image is the current frame RGB image, and the current frame may be a key frame (target frame) selected according to actual application conditions.

[0037] In an embodiment of the present invention, the edge attention module is used to solve the boundary fuzziness problem in the image segmentation task, that is, the problem that boundary pixels are difficult to accurately classify; by introducing the edge attention mechanism, the model can more accurately locate the edge areas in the image and pay more attention to these areas during the segmentation process, thereby improving the segmentation accuracy and edge clarity.

[0038] According to a monocular six-degree-of-freedom attitude estimation method for a space target provided by the present invention, the current frame image of the input target spacecraft is preprocessed by an edge attention module to obtain an image carrying edge information, which specifically includes the following steps.

[0039] Step 1011, performing edge detection on the current frame image based on Sobel convolution through an edge attention module to obtain an edge amplitude map of the current frame image.

[0040] In the embodiment of the present invention, a current frame (RGB) image is input, and a Sobel convolution operation is used to perform edge detection on the current frame image to obtain an edge amplitude map of the current frame image.

[0041] Step 1012, performing Fourier transform on the edge amplitude map through the edge attention module to obtain a frequency domain feature map of the current frame image; In the embodiment of the present invention, Fourier transform is used to transform the edge amplitude map into a frequency domain feature map.

[0042] Step 1013, performing discrete cosine transform based on the frequency domain feature map through the edge attention module to obtain an image carrying edge information.

[0043] In the embodiment of the present invention, a discrete cosine transform type II function is used to enhance high-frequency features such as edges, thereby improving subsequent image processing and analysis tasks. The discrete cosine transform type II formula can refer to the following formula (1): (1)

[0044] in, represents a sequence of input signals or data, represents the length of the input sequence, represents the coefficient after transformation, Representation sequence Length.

[0045] Through the embodiments of the present invention, high-frequency features such as edges can be enhanced, thereby improving subsequent image processing and analysis tasks.

[0046] Step 102: input the image carrying edge information into a pre-trained mask segmentation network model to obtain the binary mask information output by the mask segmentation network model, and determine the current frame object image based on the binary mask information.

[0047] In an embodiment of the present invention, the mask segmentation network model can use a neural network to automatically learn and extract image features, thereby achieving accurate image segmentation. A mask is a binary image in which pixel values ​​are usually 0 or 1 and are used to identify specific areas in an image. In the mask segmentation task, the goal of the mask segmentation network model is to generate a corresponding mask for each object in the image.

[0048] The current frame object is the target object located in the current frame image.

[0049] According to a monocular six-degree-of-freedom posture estimation method for a space target provided by the present invention, an image carrying edge information is input into a pre-trained mask segmentation network model to obtain binary mask information output by the mask segmentation network model, and based on the binary mask information, the method specifically includes the following steps: Step 1021, extracting features of the image carrying edge information through a pre-trained mask segmentation network model to obtain features of different scales.

[0050] In an embodiment of the present invention, a lightweight architecture for self-supervised monocular depth estimation (for example, a lightweight LiteMono model) is used to differentiate the RGB image of the input current frame, and after multiple maximum pooling, features of different scales (W / 2×H / 2, W / 4×H / 4, W / 8×H / 8, W / 16×H / 16) are obtained.

[0051] In the embodiment of the present invention, the difference may refer to the following formula (2): (2)

[0052] in, Representing images In position ( ) and the color channels are The pixel value of and Respectively represent images In position ( ) Location Direction and Directional gradient (a gradient is a measure of the rate of change of brightness or color intensity in an image); Representing images In the row direction there are pixels, Representing images In the column direction, pixels, Representing images There are three color channels (e.g., RGB images).

[0053] Step 1022, input features of different scales into the edge-based Fourier frequency domain attention module through the pre-trained mask segmentation network model, and obtain the weighted frequency domain feature map output by the edge-based Fourier frequency domain attention module.

[0054] In an embodiment of the present invention, features of different scales are input into an edge-based Fourier frequency domain attention module (DFDA, Diffence Frequency Domain DCT Attention) to obtain a weighted frequency domain feature map.

[0055] Step 1023, decoding the weighted frequency domain feature map through the pre-trained mask segmentation network model to obtain the binary mask information of the current frame image.

[0056] In the embodiment of the present invention, the weighted frequency domain feature map is decoded to obtain the mask (W×H, W / 2×H / 2, W / 4×H / 4) of the current frame.

[0057] After step 1023, the above-mentioned determining the current frame object image based on the binary mask information specifically includes: Step 1024, select a mask with the same size as the input current frame image, set the threshold to 0.5, perform pixel sampling on the key frame, and output the current frame object (RGB) image.

[0058] Through the embodiments of the present invention, the pre-trained mask segmentation network model can learn and recognize different objects and their boundaries in the image. By inputting an image carrying edge information, the model can more accurately capture the contour of the object, thereby generating a high-precision binary mask to clearly define the boundary between the object and the background.

[0059] Step 103: input the current frame object image and the current frame image into a depth estimation network to obtain the current frame object depth information output by the depth estimation network.

[0060] In an embodiment of the present invention, the depth estimation network is used to output the depth (ie, the distance from the camera) information of each pixel in the image.

[0061] According to a monocular six-degree-of-freedom posture estimation method for a space target provided by the present invention, a current frame object image and a current frame image are input into a depth estimation network to obtain the current frame object depth information output by the depth estimation network, which specifically includes the following steps: Step 1031, encode the current frame image through a depth estimation network to obtain a multi-scale feature map of the current frame.

[0062] In an embodiment of the present invention, an RGB image of a current frame is input and encoded using a lightweight architecture (e.g., LiteMono) for self-supervised monocular depth estimation to obtain features of different sizes (i.e., a multi-scale feature map of the current frame).

[0063] Among them, LiteMono is a lightweight convolutional neural network (CNN) and transformer hybrid architecture designed for self-supervised monocular depth estimation, including: continuous expansion convolution module and local-global feature interaction module.

[0064] Step 1032, through the attention gate feature selection module of the depth estimation network, the multi-scale feature map of the current frame is continuously convolved with the object image of the current frame to obtain the fusion feature of the current frame.

[0065] In an embodiment of the present invention, an attention gate feature selection module (ACSG, Attention CosineSimilarity Gate) is used to obtain a current frame fusion feature by performing continuous Conv3x3 convolution on the current frame multi-scale feature map and the current frame object image.

[0066] Specifically, the multi-scale feature map of the current frame and the object image of the current frame are channel-weighted, and the corresponding channel correlation is calculated using the cosine similarity function, where the correlation formula can refer to the following formula (3): (3)

[0067] in, represents correlation, A represents the first feature image (multi-scale feature map of the current frame), B represents the second feature image (object image of the current frame), and i represents the index of the pixel.

[0068] If the channels are related, the features of the two channels are added together. Otherwise, the two feature maps are connected in the channel dimension and the convolution function is used to compress the feature channels. For details, please refer to the following formula (4): (4)

[0069] in, represents the fusion feature (the fusion feature of the current frame), represents the first channel feature (i.e., the feature of the multi-scale feature map of the current frame), represents the second channel feature (i.e., the feature of the object image in the current frame), represents the convolution operation, represents feature concatenation, Indicates relevance.

[0070] Step 1033, decoding the current frame fusion features through the depth decoder of the depth estimation network to obtain a current frame depth map, wherein the current frame depth map is used to represent the depth information of the current frame object.

[0071] In the embodiment of the present invention, the current frame fusion feature is input into the depth decoder to obtain the current frame depth map to represent the current frame object depth information.

[0072] Step 104, input the adjacent frame images of the current frame image and the current frame object depth information into a pre-trained space target six-degree-of-freedom attitude estimation network, and obtain the relative space target monocular six-degree-of-freedom attitude change information between the current frame and the adjacent frames output by the pre-trained space target six-degree-of-freedom attitude estimation network.

[0073] In an embodiment of the present invention, the pre-trained space target six-degree-of-freedom posture estimation network is based on sequence images.

[0074] According to a monocular six-degree-of-freedom posture estimation method for a space target provided by the present invention, before the adjacent frame images of the current frame image and the current frame object depth information are input into a pre-trained six-degree-of-freedom posture estimation network for the space target, the method further includes the following steps.

[0075] Step B1, obtaining an initialized spatial target six-degree-of-freedom posture estimation network based on sequence images.

[0076] In the embodiment of the present invention, the weights of a space target six-degree-of-freedom attitude estimation network based on sequence images are initialized to obtain an initialized space target six-degree-of-freedom attitude estimation network based on sequence images.

[0077] Step B2, input the current frame image samples and the adjacent frame image samples into the space target six-degree-of-freedom posture estimation network, and obtain the current frame mask samples, the current frame depth map samples, and the relative space target monocular six-degree-of-freedom posture change information samples between the current frame and the adjacent frames output by the space target six-degree-of-freedom posture estimation network.

[0078] In an embodiment of the present invention, based on the network obtained in step B1, the current frame image (sample) and the adjacent frame image (sample) are input, and the current frame mask and the current frame depth map as well as the relative six-degree-of-freedom posture of the current frame and the adjacent frame are output.

[0079] Step B3, based on the current frame mask sample and the true mask, determine the loss function of the mask network.

[0080] In the embodiment of the present invention, the DiceLoss and BCELoss functions of the output mask (i.e., the current frame mask) and the true mask are calculated for the loss function of the mask network; Step B4, input the current frame depth map sample and the monocular six-degree-of-freedom posture change information sample relative to the spatial target into the image reconstruction network to obtain the reconstructed current frame image sample.

[0081] Step B5, based on the reconstructed current frame image sample and the current frame image sample, determine the structural similarity loss and the norm loss as the reconstruction loss function.

[0082] Step B6: determining the loss function of the deep pose estimation network based on the reconstruction loss function and the edge-aware smoothing loss function.

[0083] In an embodiment of the present invention, the current frame depth map (sample), the relative six-degree-of-freedom posture of the current frame and the adjacent frames, and the adjacent frame images (samples) are input into the image reconstruction network to obtain the reconstructed current frame image, and the structural similarity loss and 1-norm loss of the reconstructed current frame image and the input current frame image are calculated as the reconstruction loss function. The reconstruction loss function and the edge-aware smoothing loss function together constitute the depth posture estimation network loss function.

[0084] Step B7, weighting the loss function of the mask network and the loss function of the deep pose estimation network to obtain the total loss function of the six-degree-of-freedom pose estimation network for the space target.

[0085] Step B8, when the deep posture estimation network reaches a preset number of training times or the loss function value of the total loss function is lower than a preset threshold, a pre-trained spatial target six-degree-of-freedom posture estimation network is obtained.

[0086] In an embodiment of the present invention, the total loss function is a weighted sum of the loss function of the mask network and the loss function of the depth pose estimation network, and steps B2 to B6 are repeatedly performed until a preset number of training times is reached or the loss function value of the reconstruction error is lower than a set threshold.

[0087] According to a monocular six-degree-of-freedom attitude estimation method for a space target provided by the present invention, adjacent frame images of a current frame image and depth information of an object in the current frame are input into a pre-trained space target six-degree-of-freedom attitude estimation network, and relative space target monocular six-degree-of-freedom attitude change information of the current frame and the adjacent frames output by the pre-trained space target six-degree-of-freedom attitude estimation network is obtained, including the following specific steps: Step 1041, encoding adjacent frame images of the current frame image through a pre-trained space target six-degree-of-freedom posture estimation network to obtain a multi-scale fusion image of adjacent frames.

[0088] In an embodiment of the present invention, adjacent frame images (RGB images) of a current frame image are input and encoded using a lightweight architecture (eg, LiteMono) for self-supervised monocular depth estimation to obtain features of different scales.

[0089] Step 1042, through the attention gate feature selection module of the pre-trained spatial target six-degree-of-freedom posture estimation network, the adjacent frame multi-scale fusion map and the current frame multi-scale feature map are fused to obtain the fusion features of the current frame and the adjacent frames.

[0090] In the embodiment of the present invention, the multi-scale fusion map of adjacent frames is fused with the multi-scale feature map of the previous frame through an attention gate feature selection module (ACSG, Attention CosineSimilarity Gate).

[0091] Step 1043, decoding the fusion features of the current frame and the adjacent frames through the posture decoder of the pre-trained space target six-degree-of-freedom posture estimation network, to obtain the relative space target monocular six-degree-of-freedom posture change information between the current frame and the adjacent frames.

[0092] In the embodiment of the present invention, the fusion features of the current frame and the adjacent frames are input into the posture decoder to obtain the monocular six-degree-of-freedom posture change information of the relative space target of the current frame and the adjacent frames.

[0093] It should be noted that in the embodiment of the present invention, the depth estimation network and the attitude estimation network (space target six-degree-of-freedom attitude estimation network) together constitute the depth attitude estimation network. The feature extraction network of the depth estimation network and the attitude estimation network uses LiteMono and shares weights.

[0094] Step 105 , reconstructing an image based on the relative space target monocular six-degree-of-freedom posture change information and the object depth information of the current frame, to obtain the absolute space target monocular six-degree-of-freedom posture change information of adjacent frames.

[0095] The above-mentioned image reconstruction based on the relative space target monocular six-degree-of-freedom posture change information and the current frame object depth information is performed to obtain the absolute space target monocular six-degree-of-freedom posture change information of the adjacent frame, specifically including the following steps: Step 1051, obtaining the two-dimensional homogeneous matrix and camera intrinsic parameters corresponding to the current frame.

[0096] Step 1052, reconstruct the current frame according to the object depth information of the current frame and the monocular six-degree-of-freedom posture change information of the relative space target, and obtain the monocular six-degree-of-freedom posture change information of the absolute space target of the adjacent frame.

[0097] For details, please refer to the following formulas (5) to (6): (5)

[0098] (6)

[0099] in, is a two-dimensional homogeneous matrix, is the estimated 2D graph, are the camera parameters, is the attitude matrix, including the translation matrix and the rotation matrix, is the depth map of the current frame, is the RGB image of the current frame, is the estimated RGB image of the current frame.

[0100] Step 106, based on the relative space target monocular six-degree-of-freedom posture change information and the absolute space target monocular six-degree-of-freedom posture change information of the adjacent frames, determine the absolute space target monocular six-degree-of-freedom posture change information of the current frame.

[0101] Among them, the above-mentioned determination of the absolute space target monocular six-degree-of-freedom posture change information of the current frame based on the relative space target monocular six-degree-of-freedom posture change information and the absolute space target monocular six-degree-of-freedom posture change information of the adjacent frames specifically includes the following steps.

[0102] Step 1061, obtaining the absolute space target monocular six-degree-of-freedom posture change information and the relative space target monocular six-degree-of-freedom posture change information of adjacent frames.

[0103] Step 1062, determining the absolute space target monocular six-degree-of-freedom posture change information of the current frame.

[0104] For details, please refer to the following formulas (7) to (8): (7)

[0105] (8)

[0106] in, is the estimated object relative rotation change matrix, is the estimated object relative translation change matrix, is the true rotation matrix of the adjacent frames, is the real translation matrix of adjacent frames, is the estimated keyframe rotation matrix, is the estimated keyframe translation matrix.

[0107] Through the embodiments of the present invention, by combining relative posture change information and absolute posture information of adjacent frames, not only the relative change between frames is considered, but also the known absolute posture is used as a reference, thereby reducing the cumulative error in posture estimation.

[0108] Through the above embodiments provided by the present invention, the attitude information of the spacecraft can be obtained in a semi-supervised manner without obtaining the real attitude label and the depth label. The edge-based Fourier frequency domain attention module DFDA is designed to enhance the texture features of images in the dark environment of deep space, further improving the accuracy of mask segmentation. The attention gate feature selection module ACSG guides the network's focus to the spacecraft for feature extraction and fusion, effectively avoiding the influence of background changes on attitude estimation.

[0109] refer to Figure 2 , Figure 2 It is an overall flow chart of the monocular six-degree-of-freedom posture estimation method for a space target provided by the present invention.

[0110] The following describes a monocular six-degree-of-freedom posture estimation method for a space target proposed by the present invention, and an embodiment in a practical application scenario, which specifically includes the following steps.

[0111] Step S10, select an image of a spacecraft as a key frame image, input it into the edge attention module, and perform image preprocessing.

[0112] Specifically, an image of a spacecraft is selected as a key frame image and input into the edge attention module for image preprocessing.

[0113] First, the current frame is segmented as input, and the Sobel convolution operation is used to detect the edge of the entire image. After obtaining an edge amplitude map of an image, Fourier transform is performed to convert the edge amplitude map into a frequency domain feature map, and the discrete cosine transform type II function is used to enhance high-frequency features such as edges.

[0114] The edge information attention module uses the edge extraction algorithm to strengthen the image edge information as a weight coefficient and increase the texture information of the edge pixels in the original image.

[0115] Step S20, input the image with edge information obtained in the previous step into the trained mask segmentation network model to obtain its binary mask information and calculate the RGB image of the current frame object.

[0116] Specifically, the image with edge information obtained in the previous step is input into the trained mask segmentation network model to obtain its binary mask information and calculate the RGB image of the object in the current frame.

[0117] The mask segmentation network uses the lightweight LiteMono model as the feature extraction network, inputs the RGB image of the current frame, and performs differential, and after multiple maxpooling, obtains different scales, and inputs the edge-based Fourier frequency domain attention module DFDA to obtain the weighted frequency domain feature map. The weighted frequency domain feature map is decoded to obtain the mask of the current frame, selects the mask with the same size as the input current frame RGB image, sets the threshold to 0.5, performs pixel sampling on the key frame, and outputs the RGB image of the object.

[0118] The edge-based Fourier frequency domain attention module (DFDA, Diffence Frequency Domain DCTAttention) aims to enhance the texture features of images in deep space dim environments, further improving the accuracy of mask segmentation.

[0119] The mask segmentation network takes the current frame RGB image as input to obtain the binary mask information, and then samples the RGB image of the current frame target based on the mask information. Specifically: set the threshold to 0.5, when the binary mask information is greater than 0.5, the pixel value of the corresponding position of the current frame RGB image is retained, and when the binary mask information is less than 0.5, the pixel value of the corresponding position of the current frame RGB image is discarded.

[0120] The mask segmentation network is a supervised network, and the DiceLoss and BCELoss functions of the output mask and the true mask are used as the loss function of the mask network.

[0121] Here, Dice Loss is a loss function commonly used in image segmentation tasks, especially when dealing with imbalanced foreground and background pixels. BCELoss, or Binary Cross-Entropy Loss, is a loss function used for binary classification problems in machine learning and deep learning to measure the difference between the probability distribution predicted by the model and the true label.

[0122] The relevant formulas can refer to the following formulas (9)-(10): (9)

[0123] in, represents the Dice loss function, and Represent the set of non-zero (or foreground) pixels in the prediction results and the true label, respectively. is a small positive number used to prevent the denominator from being zero

[0124] Here, when the prediction is completely accurate (ie, X=Y), the Dice coefficient is 1 and the Dice Loss is 0. When the prediction does not overlap with the true label at all, the Dice coefficient is 0 and the Dice Loss is 1.

[0125] (10)

[0126] in, Represents the BCE loss function, when When it is equal to 1 (that is, the true label is positive), the loss function is simplified to If the model predicts The closer it is to 1, the The closer it is to 0, the smaller the loss; on the contrary, if The smaller it is (i.e. the greater the probability that the model predicts the negative class), the greater the loss.

[0127] when When it is equal to 0 (that is, the true label is a negative class), the loss function is simplified to If the model predicts The closer it is to 0, the The closer it is to 0, the smaller the loss; on the contrary, if The larger is (that is, the greater the probability that the model predicts the positive class), the greater the loss.

[0128] Step S30, inputting the RGB image of the current frame object obtained in the previous step and the current frame RGB image into a depth estimation network to obtain depth information of the object.

[0129] Specifically, the current frame RGB image is input and encoded by LiteMono to obtain features of different sizes. The multi-scale feature map of the key frame obtained by the attention gate feature selection module ACSG is fused with the features obtained by continuous Conv3x3 convolution of the object RGB image, and the fused features are input into the depth decoder to obtain the depth map of the current frame.

[0130] Step S40, select the adjacent frame image of the key frame image in the previous step, and input the features obtained by the depth estimation in the previous step into the posture estimation network to obtain the relative 6-DoF posture change information between the key frame and the adjacent frame.

[0131] Specifically, adjacent frame RGB images are input and encoded by LiteMono to obtain features of different sizes. The fused adjacent frame multi-scale feature maps are fused with the key frame multi-scale feature maps output in the depth estimation network through the attention gate feature selection module ACSG. The fused features are input into the posture decoder to obtain the relative 6-DoF posture information between the current frame and the adjacent frame.

[0132] Step S50, using the relative 6-DoF posture change information obtained in the previous step and the current frame depth information obtained in step S30 to perform image reconstruction, and reconstruct the adjacent frame RGB image into the current frame RGB image.

[0133] Specifically, step S30 to step S50 are deep pose estimation networks. The deep pose estimation network is a self-supervised network. During training, it is not necessary to obtain the real pose label and depth label. The network constrains the reconstruction loss and edge-aware smoothing loss of the current frame. The reconstruction loss includes the structural similarity loss (SSIM) and 1-norm (L1) loss of the current frame and the reconstructed current frame.

[0134] The reconstruction loss formula can refer to the following formula (11): (11)

[0135] in, Represents the predicted image With real image The loss between is a hyperparameter used to balance the weight between SSIM loss and L1 loss. The value range is usually between 0 and 1. It represents the structural similarity index, which is used to evaluate the structural similarity of two images. The closer the SSIM value is to 1, the more similar the two images are; the closer it is to 0, the greater the difference. Represents the L1 norm, also known as Manhattan distance or minimum absolute error (MAE), which is used to calculate the sum of the absolute values ​​of the pixel differences between the predicted image and the true image.

[0136] In an embodiment of the present invention, The default is 0.85.

[0137] The structural similarity loss formula can refer to the following formula (12): (12)

[0138] in, Represent two image windows (usually image blocks or the entire image), and Respectively The mean of and Respectively The variance of express The covariance of and are two constants used to avoid zero denominators and improve numerical stability.

[0139] Edge-aware smoothness loss is used to optimize the edge smoothness of the depth map. The edge-aware smoothness loss formula can refer to the following formula (13): (13)

[0140] in, represents the edge-aware smoothing loss function, Respectively represent images The width and height of represents the set of all pixels in the image, represents the displacement field ( )exist The absolute value of the gradient in the direction, and Representing images exist Exponential decay term in direction (adaptive weight).

[0141] Step S60, calculating the absolute posture information of the current frame based on the relative 6-DoF posture change information obtained in the previous step and the absolute 6-DoF posture information of the adjacent frames.

[0142] Specifically, the estimated rotation error and translation error of the posture can refer to the following formula (14): (14)

[0143] in, represents the total attitude error, represents the rotation error, represents the estimated or measured rotation matrix, represents the true rotation matrix, represents the translation error, where is the estimated or measured translation vector, is the real translation vector, Represents the translation vector The Euclidean norm (i.e. length) of .

[0144] The rotation error formula can refer to the following formula (15): (15)

[0145] in, represents the rotation error, represents the estimated or measured rotation matrix, represents the true rotation matrix, represents the transposed matrix, represents the trace of a matrix, which is the sum of the diagonal elements of the matrix. Represents the inverse cosine function.

[0146] The translation error formula can refer to the following formula (16): (16)

[0147] in, represents the translation error, The estimated translation vector is calculated and the true translation vector The difference between.

[0148] refer to Figure 3 , Figure 3 The present invention provides a training flow chart of a spatial target six-degree-of-freedom posture estimation network based on sequence images.

[0149] In an embodiment of the present invention, a training method of a six-degree-of-freedom posture estimation network for a space target based on sequence images can refer to the following steps: Step B10, initializing the weights of the spatial target six-degree-of-freedom posture estimation network based on the sequence images.

[0150] In an embodiment of the present invention, an initialized spatial target six-degree-of-freedom posture estimation network based on sequence images is obtained; Step B20, input the current frame image and the adjacent frame image, output the current frame mask and the current frame depth map and the relative six-degree-of-freedom posture of the current frame and the adjacent frame.

[0151] In the embodiment of the present invention, based on the network obtained in step B10, the current frame image and the adjacent frame image are input, the current frame mask and the current frame depth map and the relative six-degree-of-freedom posture of the current frame and the adjacent frame are output, and the DiceLoss and BCELoss functions of the output mask and the true mask are calculated for the loss function of the mask network; Step B30, input the current frame depth map and relative posture information obtained in the previous step and the adjacent frame images into the image reconstruction network to obtain a reconstructed current frame image.

[0152] In an embodiment of the present invention, the structural similarity loss and the 1-norm loss between the reconstructed current frame image and the input current frame image are calculated as a reconstruction loss function, and the reconstruction loss function and the edge-aware smoothing loss function together constitute a deep pose estimation network loss function; Step B40, calculate the loss function, the total loss function is the weighted loss function of the mask network and the loss function of the depth pose estimation network, and update the network parameters by gradient descent.

[0153] In an embodiment of the present invention, the total loss function is a weighted sum of the loss function of the mask network and the loss function of the depth pose estimation network, and steps B20 and B30 are repeatedly performed until a preset number of training times is reached or the loss function value of the reconstruction error is lower than a set threshold.

[0154] In the simulation experiment of the embodiment of the present invention, the software used is: Python 3.8, processor: E5, graphics card RTX3090, and operating system: Ubuntu.

[0155] In an embodiment of the present invention, NVIDIA RTX3090 is used in a Linux environment, and the Swisscube dataset is trained and evaluated based on the PyTorch platform with a data volume of 33,000 images, a sequence number of 350, and a batch size of 12, and the image size is 416x416. The feature extraction network of the deep network and the posture network is pre-trained on the KITTY dataset. The two networks use the same Adam optimizer with a weight decay of 1e−2 and an initial learning rate of 1e-4. A total of 30 rounds of training are performed, and after 20 rounds, the learning rate decays to 1e-5. The feature extraction network used by the mask segmentation network is ResNet18 pre-trained on ImageNet. The optimizer also uses Adam with a weight decay of 1e-2 and an initial learning rate of 1e-3. The learning rate decays to 1e-4 and 1e-5 at 10 and 20 rounds, respectively.

[0156] The key frame images were segmented by masks, sampled according to masks, and finally estimated relative changes. During verification, the real pose of the previous frame was used to obtain the predicted pose of the key frame according to the calculation method of rotation matrix and translation matrix, and finally the error was calculated with the real pose of the key frame. We selected a frame image from the final result for visualization and numerical display, where the angle error is 0.051 and the translation error is 0.016. For details, please refer to Figures 4 to 8 , Figures 4 to 8 To visualize the results: Figure 4 is the original RGB image, Figure 5 is the image after high-frequency edge processing, Figure 6 is the mask image after segmentation, Figure 7 is an RGB image based on mask sampling, Figure 8 Pose estimation visualization image.

[0157] The monocular six-degree-of-freedom attitude estimation device for a space target provided by the present invention is described below. The monocular six-degree-of-freedom attitude estimation device for a space target described below and the monocular six-degree-of-freedom attitude estimation method for a space target described above can be referenced to each other.

[0158] refer to Fig. 9 , Fig. 9 It is a structural flow chart of a monocular six-degree-of-freedom posture estimation device for a space target provided by the present invention, which includes: a preprocessing module 901, a first determination module 902, a first output module 903, a second output module 904, a reconstruction module 905, and a second determination module 906.

[0159] A preprocessing module 901 is used to preprocess the input current frame image of the target spacecraft through an edge attention module to obtain an image carrying edge information; A first determination module 902 is used to input the image carrying edge information into a pre-trained mask segmentation network model, obtain the binary mask information output by the mask segmentation network model, and determine the current frame object image based on the binary mask information; A first output module 903, configured to input the current frame object image and the current frame image into a depth estimation network to obtain the current frame object depth information output by the depth estimation network; The second output module 904 is used to input the adjacent frame images of the current frame image and the current frame object depth information into a pre-trained space target six-degree-of-freedom posture estimation network to obtain the relative space target monocular six-degree-of-freedom posture change information of the current frame and the adjacent frames output by the pre-trained space target six-degree-of-freedom posture estimation network; A reconstruction module 905 is used to reconstruct an image based on the relative space target monocular six-degree-of-freedom posture change information and the current frame object depth information to obtain the absolute space target monocular six-degree-of-freedom posture change information of adjacent frames; The second determination module 906 is used to determine the absolute space target monocular six-degree-of-freedom posture change information of the current frame based on the relative space target monocular six-degree-of-freedom posture change information and the absolute space target monocular six-degree-of-freedom posture change information of the adjacent frame.

[0160] Specifically, the monocular six-degree-of-freedom posture estimation device for the above-mentioned space target provided by the present invention can implement all the method steps implemented in the above-mentioned monocular six-degree-of-freedom posture estimation method embodiment for the space target, and can achieve the same technical effect. The parts and beneficial effects that are the same as those in the method embodiment will not be described in detail here.

[0161] refer to Fig.10 , Fig.10 : is a structural schematic diagram of a monocular six-degree-of-freedom attitude estimation system for a space target provided by the present invention, comprising: The mask segmentation module inputs the RGB image of the current frame, obtains the corresponding mask, and outputs the RGB image of the target in the current frame according to the set threshold; The depth estimation module inputs the RGB image of the current frame and the RGB image of the target in the current frame. The former is extracted by LiteMono, and the latter is extracted by continuous Conv3x3 convolution. The features extracted by the two are fused, and the fused features are decoded to obtain the depth map of the current frame; The posture estimation module inputs the RGB images of adjacent frames, extracts features through LiteMono, and fuses them with the fusion features obtained by the depth estimation network. After being decoded by the posture decoder, the relative posture information of the current frame and the adjacent frame is output; The image reconstruction module inputs the current frame depth map obtained by the depth estimation module, the adjacent frame RGB image and the relative posture information obtained by the posture estimation module, and outputs the reconstructed current frame RGB image.

[0162] Fig.11 is a schematic diagram of the physical structure of the electronic device provided by the present invention, such as Fig.11 As shown, the electronic device may include: a processor 1110, a communications interface 1120, a memory 1130 and a communication bus 1140, wherein the processor 1110, the communications interface 1120 and the memory 1130 communicate with each other through the communication bus 1140. The processor 1110 may call the logic instructions in the memory 1130 to execute a monocular six-degree-of-freedom attitude estimation method for a space target, the method comprising: preprocessing the input current frame image of the target spacecraft through an edge attention module to obtain an image carrying edge information; inputting the image carrying edge information into a pre-trained mask segmentation network model to obtain the binary mask information output by the mask segmentation network model, and determining the current frame object image based on the binary mask information; inputting the current frame object image and the current frame image into a depth estimation network to obtain the current frame object depth information output by the depth estimation network; and comparing the adjacent frames of the current frame image. The image and the object depth information of the current frame are input into a pre-trained spatial target six-degree-of-freedom attitude estimation network to obtain the relative spatial target monocular six-degree-of-freedom attitude change information of the current frame and the adjacent frames output by the pre-trained spatial target six-degree-of-freedom attitude estimation network; the image is reconstructed based on the relative spatial target monocular six-degree-of-freedom attitude change information and the object depth information of the current frame to obtain the absolute spatial target monocular six-degree-of-freedom attitude change information of the adjacent frames; the absolute spatial target monocular six-degree-of-freedom attitude change information of the current frame is determined based on the relative spatial target monocular six-degree-of-freedom attitude change information and the absolute spatial target monocular six-degree-of-freedom attitude change information of the adjacent frames.

[0163] In addition, the logic instructions in the above-mentioned memory 1130 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0164] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the monocular six-degree-of-freedom posture estimation method of the space target provided by the above-mentioned methods, and the method includes: preprocessing the current frame image of the input target spacecraft through an edge attention module to obtain an image carrying edge information; inputting the image carrying edge information into a pre-trained mask segmentation network model to obtain the binary mask information output by the mask segmentation network model, and determining the current frame object image based on the binary mask information; inputting the current frame object image and the current frame image into a depth estimation network; , obtain the current frame object depth information output by the depth estimation network; input the adjacent frame images of the current frame image and the current frame object depth information into the pre-trained spatial target six-degree-of-freedom attitude estimation network, and obtain the relative spatial target monocular six-degree-of-freedom attitude change information of the current frame and the adjacent frames output by the pre-trained spatial target six-degree-of-freedom attitude estimation network; reconstruct the image based on the relative spatial target monocular six-degree-of-freedom attitude change information and the current frame object depth information, and obtain the absolute spatial target monocular six-degree-of-freedom attitude change information of the adjacent frames; determine the absolute spatial target monocular six-degree-of-freedom attitude change information of the current frame based on the relative spatial target monocular six-degree-of-freedom attitude change information and the absolute spatial target monocular six-degree-of-freedom attitude change information of the adjacent frames.

[0165] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the monocular six-degree-of-freedom posture estimation method for a space target provided by the above-mentioned methods, the method comprising: preprocessing the current frame image of the input target spacecraft through an edge attention module to obtain an image carrying edge information; inputting the image carrying edge information into a pre-trained mask segmentation network model to obtain the binary mask information output by the mask segmentation network model, and determining the current frame object image based on the binary mask information; inputting the current frame object image and the current frame image into a depth estimation network to obtain the current frame image output by the depth estimation network. frame object depth information; input the adjacent frame images of the current frame image and the current frame object depth information into the pre-trained spatial target six-degree-of-freedom attitude estimation network, and obtain the relative spatial target monocular six-degree-of-freedom attitude change information of the current frame and the adjacent frames output by the pre-trained spatial target six-degree-of-freedom attitude estimation network; reconstruct the image based on the relative spatial target monocular six-degree-of-freedom attitude change information and the current frame object depth information, and obtain the absolute spatial target monocular six-degree-of-freedom attitude change information of the adjacent frames; determine the absolute spatial target monocular six-degree-of-freedom attitude change information of the current frame based on the relative spatial target monocular six-degree-of-freedom attitude change information and the absolute spatial target monocular six-degree-of-freedom attitude change information of the adjacent frames.

[0166] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0167] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0168] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A monocular six-degree-of-freedom attitude estimation method for a space target, characterized in that: include: The current frame image of the input target spacecraft is preprocessed through the edge attention module to obtain an image carrying edge information; Inputting the image carrying edge information into a pre-trained mask segmentation network model to obtain binary mask information output by the mask segmentation network model, and determining the current frame object image based on the binary mask information; Inputting the current frame object image and the current frame image into a depth estimation network to obtain the current frame object depth information output by the depth estimation network; Inputting adjacent frame images of the current frame image and the current frame object depth information into a pre-trained space target six-degree-of-freedom attitude estimation network to obtain relative space target monocular six-degree-of-freedom attitude change information of the current frame and the adjacent frames output by the pre-trained space target six-degree-of-freedom attitude estimation network; Based on the relative space target monocular six-degree-of-freedom posture change information and the current frame object depth information, image reconstruction is performed to obtain the absolute space target monocular six-degree-of-freedom posture change information of adjacent frames; Based on the relative space target monocular six-degree-of-freedom posture change information and the absolute space target monocular six-degree-of-freedom posture change information of the adjacent frame, the absolute space target monocular six-degree-of-freedom posture change information of the current frame is determined.

2. The monocular six-degree-of-freedom attitude estimation method for a space target according to claim 1, characterized in that: Before inputting the adjacent frame images of the current frame image and the current frame object depth information into a pre-trained spatial target six-degree-of-freedom posture estimation network, the method further includes: Obtain an initialized six-degree-of-freedom pose estimation network for spatial targets based on sequence images; Inputting the current frame image samples and the adjacent frame image samples into the space target six-degree-of-freedom attitude estimation network, obtaining the current frame mask samples, the current frame depth map samples, and the relative space target monocular six-degree-of-freedom attitude change information samples between the current frame and the adjacent frames output by the space target six-degree-of-freedom attitude estimation network; Determine a loss function of a mask network based on the current frame mask sample and the true mask; Inputting the current frame depth map sample and the relative space target monocular six-degree-of-freedom posture change information sample into an image reconstruction network to obtain a reconstructed current frame image sample; Based on the reconstructed current frame image sample and the current frame image sample, determining a structural similarity loss and a norm loss as a reconstruction loss function; Determining a loss function of a deep pose estimation network based on the reconstruction loss function and the edge-aware smoothing loss function; The total loss function of the spatial target six-degree-of-freedom posture estimation network is obtained by weighting the loss function of the mask network and the loss function of the deep posture estimation network; When the depth posture estimation network reaches a preset number of training times or the loss function value of the total loss function is lower than a preset threshold, a pre-trained space target six-degree-of-freedom posture estimation network is obtained.

3. The monocular six-degree-of-freedom attitude estimation method for a space target according to claim 1, characterized in that: The edge attention module is used to preprocess the current frame image of the input target spacecraft to obtain an image carrying edge information, including: Performing edge detection on the current frame image based on Sobel convolution through an edge attention module to obtain an edge amplitude map of the current frame image; Performing Fourier transform on the edge amplitude map through the edge attention module to obtain a frequency domain feature map of the current frame image; The edge attention module performs discrete cosine transform based on the frequency domain feature map to obtain an image carrying edge information.

4. The monocular six-degree-of-freedom attitude estimation method for a space target according to claim 1, characterized in that: The step of inputting the image carrying edge information into a pre-trained mask segmentation network model to obtain binary mask information output by the mask segmentation network model includes: Extracting features of the image carrying edge information through a pre-trained mask segmentation network model to obtain features of different scales; Inputting the features of different scales into an edge-based Fourier frequency domain attention module through the pre-trained mask segmentation network model to obtain a weighted frequency domain feature map output by the edge-based Fourier frequency domain attention module; The weighted frequency domain feature map is decoded by the pre-trained mask segmentation network model to obtain the binary mask information of the current frame image.

5. The monocular six-degree-of-freedom attitude estimation method for a space target according to claim 1, characterized in that: The step of inputting the current frame object image and the current frame image into a depth estimation network to obtain the current frame object depth information output by the depth estimation network includes: Encoding the current frame image through a depth estimation network to obtain a multi-scale feature map of the current frame; By using the attention gate feature selection module of the depth estimation network, continuously convolving the current frame multi-scale feature map with the current frame object image to obtain a current frame fusion feature; The current frame fusion feature is decoded by a depth decoder of the depth estimation network to obtain a current frame depth map, wherein the current frame depth map is used to represent the current frame object depth information.

6. The monocular six-degree-of-freedom attitude estimation method for a space target according to claim 5, characterized in that: The step of inputting adjacent frame images of the current frame image and the current frame object depth information into a pre-trained space target six-degree-of-freedom attitude estimation network to obtain relative space target monocular six-degree-of-freedom attitude change information of the current frame and the adjacent frames output by the pre-trained space target six-degree-of-freedom attitude estimation network includes: Encoding adjacent frame images of the current frame image through a pre-trained spatial target six-degree-of-freedom posture estimation network to obtain a multi-scale fusion image of adjacent frames; The adjacent frame multi-scale fusion map and the current frame multi-scale feature map are fused through the attention gate feature selection module of the pre-trained spatial target six-degree-of-freedom posture estimation network to obtain the fusion features of the current frame and the adjacent frame; The posture decoder of the pre-trained spatial target six-degree-of-freedom posture estimation network is used to decode the fusion features of the current frame and the adjacent frames to obtain the relative spatial target monocular six-degree-of-freedom posture change information of the current frame and the adjacent frames.

7. A monocular six-degree-of-freedom attitude estimation device for a space target, characterized in that: include: A preprocessing module is used to preprocess the current frame image of the input target spacecraft through the edge attention module to obtain an image carrying edge information; A first determination module is used to input the image carrying edge information into a pre-trained mask segmentation network model to obtain the binary mask information output by the mask segmentation network model, and determine the current frame object image based on the binary mask information; A first output module, configured to input the current frame object image and the current frame image into a depth estimation network to obtain the current frame object depth information output by the depth estimation network; A second output module is used to input the adjacent frame images of the current frame image and the depth information of the current frame object into a pre-trained space target six-degree-of-freedom posture estimation network to obtain the relative space target monocular six-degree-of-freedom posture change information of the current frame and the adjacent frames output by the pre-trained space target six-degree-of-freedom posture estimation network; A reconstruction module, used to reconstruct an image based on the relative space target monocular six-degree-of-freedom posture change information and the current frame object depth information to obtain the absolute space target monocular six-degree-of-freedom posture change information of the adjacent frame; The second determination module is used to determine the absolute space target monocular six-degree-of-freedom posture change information of the current frame based on the relative space target monocular six-degree-of-freedom posture change information and the absolute space target monocular six-degree-of-freedom posture change information of the adjacent frame.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the monocular six-degree-of-freedom posture estimation method for the space target as described in any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the monocular six-degree-of-freedom posture estimation method for a space target as described in any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the monocular six-degree-of-freedom posture estimation method for a space target as described in any one of claims 1 to 6 is implemented.

Images