A method and system for estimating artificial satellite posture

Abstract

The present invention discloses a method and system for estimating the pose of an artificial satellite, which relates to the field of artificial intelligence technology. The method comprises the following steps: randomly sampling multiple groups of artificial satellite pose sequences, pre-defining key points on an artificial satellite 3D model, and performing simulated rendering on the artificial satellite pose sequences to obtain a labeled data set; inputting the labeled data set into a feature extraction network model to obtain a multi-scale image feature map; performing semantic segmentation on the multi-scale image feature map to obtain a foreground mask image, performing key point offset vector prediction on each foreground pixel of the foreground mask image to obtain a key point offset vector field; processing the key point offset vector field using a RANSAC algorithm to find the best estimate of the 2D coordinates of the key points to obtain the pixel coordinates of the artificial satellite key points; and using a PnP algorithm to fit the pixel coordinates of the artificial satellite key points to the corresponding coordinates on the 3D model to obtain the pose of the target artificial satellite relative to the camera.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a method and system for estimating the position and posture of an artificial satellite. Background Art

[0002] Pose refers to the 3D rotation and 3D translation of an object in three-dimensional space. Accurately determining the pose of satellites and other spacecraft in a space environment is a crucial task in deep space exploration and aerospace engineering. It plays a crucial role in navigation and orbit control, docking and assembly operations, mission planning and execution, and is a key element in ensuring the smooth implementation of various space missions and promoting the advancement of aerospace science and technology. Traditional pose estimation techniques based on optical measurement and image processing are sensitive to image quality and lighting conditions and require extensive prior knowledge and complex image processing algorithms. High-precision pose recovery is particularly difficult in situations with drastic lighting variations or a lack of sufficient feature points. In recent years, pose estimation techniques based on deep learning have attracted widespread attention and research in fields such as computer vision, robotic navigation, autonomous driving, and aerospace. This is because deep learning, with its powerful representation learning and automatic feature extraction capabilities, can effectively address the complexity and uncertainty encountered by traditional pose estimation methods. While the application of deep learning in the field of pose estimation has made significant progress, significant research gaps remain in the specific area of satellite pose estimation, particularly in the absence of labeled satellite data. Summary of the Invention

[0003] In order to address the deficiencies mentioned in the above background technology, the purpose of the present invention is to provide a method and system for estimating the position and attitude of artificial satellites, which can provide technical support for accurately measuring the position and attitude of artificial satellites and other spacecraft in a space environment.

[0004] In a first aspect, the purpose of the present invention can be achieved by the following technical solutions: a method for estimating the position and posture of an artificial satellite, the method comprising the following steps:

[0005] Randomly sample multiple sets of satellite pose sequences, pre-define a set of key points on the satellite 3D model, and use the satellite 3D model to simulate and render the satellite pose sequences to obtain a labeled dataset;

[0006] Input the labeled dataset into the pre-established feature extraction network model to extract the multi-scale features of the image and output a multi-scale image feature map;

[0007] Perform semantic segmentation on the multi-scale image feature map to obtain a foreground mask image, and predict the key point offset vector for each foreground pixel in the foreground mask image to obtain a key point offset vector field;

[0008] The RANSAC algorithm is used to process the key point offset vector field to find the best estimate of the key point 2D coordinates, and the pixel coordinates of the satellite key points are obtained. The PnP algorithm is used to fit the pixel coordinates of the satellite key points and the corresponding coordinates on the 3D model to obtain the position of the target satellite relative to the camera.

[0009] In combination with the first aspect, in some implementations of the first aspect, the method further includes: the artificial satellite pose sequence is marked as {R i ,T i},in, represents the rotation matrix of the object at time i, Represents the translation vector of the object at time i.

[0010] In combination with the first aspect, in some implementations of the first aspect, the method further includes: the simulation rendering of the artificial satellite posture sequence is performed by rendering the RGB image sequence in a simulated space environment by 3dMax. i}, thereby obtaining a labeled dataset {I i ,R i ,T i}.

[0011] In combination with the first aspect, in certain implementations of the first aspect, the method further includes: the pre-established feature extraction network model adopts UNet++, and the formula of UNet++ is as follows:

[0012] embedding=encoder(I),f=multi-scale-decoder(embedding), where, is the input image, is the feature after encoding and downsampling in the network, It is the feature map obtained after the image passes through the multi-scale decoder and upsampling.

[0013] In conjunction with the first aspect, in certain implementations of the first aspect, the method further includes: inputting the labeled dataset into a pre-established feature extraction network model, extracting multi-scale features of the image, and outputting a multi-scale image feature map:

[0014] The multi-scale features of the image are extracted using the multi-scale pyramid structure, and residual connections are introduced. Dilated convolution is added to increase the receptive field by introducing holes between the convolution kernels without introducing additional parameters. This allows the network to better capture the features of satellites of different sizes. The final output multi-scale image feature map is:

[0015] In combination with the first aspect, in some implementations of the first aspect, the method further includes: performing semantic segmentation on the multi-scale image feature map and classifying it through a semantic segmentation head to obtain a foreground mask image M∈ For each foreground pixel [p x ,p y ], the key point offset vector field prediction head predicts a two-dimensional offset vector [Δ x ,Δ y ], which means that the 2D coordinates of the predefined i-th key point are [p x +Δ x ,p y +Δ y ], assuming that there are K key point coordinates defined in advance on the satellite model, K offset vector predictions will be generated for each pixel, and the final prediction output of all pixels is Serves as a keypoint offset vector field.

[0016] In combination with the first aspect, in certain implementations of the first aspect, the method also includes: the process of using the RANSAC algorithm to find the best estimate of the 2D coordinates of the key point in the key point offset vector field: randomly extracting a certain number of foreground pixels, and using the offset vectors of these pixel points to predict the 2D coordinates of the key points, generating a set of hypotheses, performing a consistency test on this set of hypotheses, calculating the proportion of the predictions of all foreground pixels that meet this set of hypotheses, repeating the above steps, each time selecting different random foreground pixels to solve the key point coordinates, and obtaining the pixel coordinates of the key points of the artificial satellite.

[0017] In conjunction with the first aspect, in certain implementations of the first aspect, the method further includes: using the PnP algorithm to solve the pixel coordinates of the key points of the artificial satellite to obtain the position and posture of the target artificial satellite relative to the camera:

[0018] After the 3D coordinates of the satellite key points are transformed into the camera coordinate system through the pose, they are compared with the 2D coordinates of the satellite key points estimated in the camera. The final 3D rotation R and 3D translation T are obtained by optimizing the following reprojection error:

[0019]

[0020] Among them, K is the camera internal parameter, and are the 3D coordinates and 2D pixel coordinates of the detected key points respectively. R, T are the rotation and translation parameters to be optimized, R * ,T * is the optimal estimate that minimizes the reprojection error.

[0021] In a second aspect, in order to achieve the above-mentioned object, the present invention discloses a satellite pose estimation system, comprising:

[0022] The data processing module is used to randomly sample multiple sets of satellite pose sequences, predefine a set of key points on the satellite 3D model, and simulate the satellite pose sequences using the satellite 3D model to obtain a labeled data set;

[0023] The feature extraction module is used to input the labeled data set into the pre-established feature extraction network model, extract the multi-scale features of the image, and output a multi-scale image feature map;

[0024] The semantic segmentation module is used to perform semantic segmentation on the multi-scale image feature map to obtain a foreground mask image, and predict the key point offset vector for each foreground pixel in the foreground mask image to obtain a key point offset vector field;

[0025] The pose solving module is used to process the key point offset vector field using the RANSAC algorithm to find the best estimate of the key point 2D coordinates, obtain the pixel coordinates of the satellite key points, and use the PnP algorithm to fit the pixel coordinates of the satellite key points and the corresponding coordinates on the 3D model to obtain the pose of the target satellite relative to the camera.

[0026] In conjunction with the second aspect, in some implementations of the second aspect, the system further includes: the artificial satellite posture sequence in the data processing module is marked as {R i ,T i},in, represents the rotation matrix of the object at time i, Represents the translation vector of the object at time i;

[0027] Or simulate the satellite pose sequence in the data processing module and render the RGB image sequence in the simulated space environment through 3dMax. i}, thereby obtaining a labeled dataset {I i ,R i ,T i};

[0028] Preferably, the feature extraction network model pre-established in the feature extraction module adopts UNet++, and the formula of UNet++ is as follows:

[0029] embedding=encoder(I),f=multi-scale-decoder(embedding),

[0030] in, is the input image, is the feature after encoding and downsampling in the network, It is the feature map obtained after the image passes through the multi-scale decoder and upsampling.

[0031] Preferably, the feature extraction module inputs the labeled data set into a pre-established feature extraction network model, extracts the multi-scale features of the image, and outputs a multi-scale image feature map:

[0032] The multi-scale features of the image are extracted using the multi-scale pyramid structure, and residual connections are introduced. Dilated convolution is added to increase the receptive field by introducing holes between the convolution kernels without introducing additional parameters. This allows the network to better capture the features of satellites of different sizes. The final output multi-scale image feature map is:

[0033] Preferably, the semantic segmentation module performs semantic segmentation on the multi-scale image feature map and classifies it through the semantic segmentation head to obtain a foreground mask image. For each foreground pixel [p x ,p y ], the key point offset vector field prediction head predicts a two-dimensional offset vector [Δ x ,Δ y ], which means that the 2D coordinates of the predefined i-th key point are [p x +Δ x ,p y +Δ y ], assuming that there are K key point coordinates defined in advance on the satellite model, K offset vector predictions will be generated for each pixel, and the final prediction output of all pixels is As a keypoint offset vector field;

[0034] Preferably, the pose solving module uses the RANSAC algorithm to find the best estimate of the 2D coordinates of the key points in the key point offset vector field: randomly extract a certain number of foreground pixels, and use the offset vectors of these pixels to predict the 2D coordinates of the key points, generate a set of hypotheses, perform consistency checks on the set of hypotheses, calculate the proportion of all foreground pixel predictions that meet the set of hypotheses, repeat the above steps, each time selecting different random foreground pixels to solve the key point coordinates, and obtain the pixel coordinates of the key points of the artificial satellite;

[0035] Preferably, the pose solving module uses the PnP algorithm to solve the pixel coordinates of the key points of the artificial satellite to obtain the pose of the target artificial satellite relative to the camera:

[0036] After the 3D coordinates of the satellite key points are transformed into the camera coordinate system through the pose, they are compared with the 2D coordinates of the satellite key points estimated in the camera. The final 3D rotation R and 3D translation T are obtained by optimizing the following reprojection error:

[0037]

[0038] Among them, K is the camera internal parameter, and are the 3D coordinates and 2D pixel coordinates of the detected key points respectively. R, T are the rotation and translation parameters to be optimized, R * ,T * is the optimal estimate that minimizes the reprojection error.

[0039] Beneficial effects of the present invention:

[0040] This paper addresses the difficulty of obtaining real-world pose-annotated satellite datasets. By using sequential pose sampling and 3D Max rendering to synthesize annotated data for network training, this method achieves high performance. This addresses the high labeling costs and sample acquisition challenges in this field, saving significant manpower and resources. The present invention detects the positions of key satellite points and, through post-processing using the RANSAC and PnP algorithms, further obtains accurate satellite pose estimates. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, those skilled in the art can derive other drawings based on these drawings without inventive effort.

[0042] Figure 1 It is a schematic flow chart of the method of the present invention;

[0043] Figure 2 This is a schematic diagram of a sample of a virtual labeled artificial satellite rendering data set of the present invention;

[0044] Figure 3 This is a schematic diagram of the virtual labeled satellite rendering process of the present invention;

[0045] Figure 4 Schematic diagram of a satellite pose estimation method based on key point detection according to the present invention;

[0046] Figure 5 1. It is a schematic diagram of an example of artificial satellite posture prediction result of the present invention;

[0047] Figure 6It is a schematic diagram of the system structure of the present invention. DETAILED DESCRIPTION

[0048] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.

[0049] Example 1:

[0050] The following is an introduction to the relevant terms involved in the embodiments of this application:

[0051] Pose Estimation: Pose estimation plays a crucial role in computer vision. It has numerous applications in robot control, robot navigation, augmented reality, and other areas using visual sensors to estimate robot pose. The foundation of pose estimation is finding corresponding points between the real world and the image projection. Based on the type of these point pairs (e.g., 2D-2D, 2D-3D, or 3D-3D), a corresponding pose estimation method is employed. The process of estimating pose from known point pairs is often referred to as solving PnP.

[0052] Multi-scale feature extraction: Multi-scale feature extraction is a commonly used technique in computer vision and image processing, which involves sampling an image at different granularities in order to capture both global information and local details.

[0053] Semantic segmentation: Semantic segmentation is an image segmentation technology in the field of computer vision. Its core goal is to assign each pixel in the image to a predefined category, thereby achieving a refined understanding of the image.

[0054] Compared to traditional image segmentation techniques, semantic segmentation focuses not only on dividing an image into regions but also on accurately classifying each pixel, assigning each pixel a semantic label indicating that it belongs to a specific category. This technology has broad applications in fields such as autonomous driving, medical image analysis, and robotic vision.

[0055] like Figure 1 As shown, a method for estimating the position and posture of an artificial satellite is characterized in that the method comprises the following steps:

[0056] Randomly sample multiple sets of satellite pose sequences, pre-define a set of key points on the satellite 3D model, and use the satellite 3D model to simulate and render the satellite pose sequences to obtain a labeled dataset;

[0057] Among them, the satellite pose sequence is marked as {Ri ,T i},in, represents the rotation matrix of the object at time i, Represents the translation vector of the object at time i; simulates the rendering of the satellite pose sequence by using 3dMax to render the RGB image sequence in a simulated space environment {I i}, thereby obtaining a labeled dataset {I i ,R i ,T i}.

[0058] Input the labeled dataset into the pre-established feature extraction network model to extract the multi-scale features of the image and output a multi-scale image feature map;

[0059] The pre-established feature extraction network model uses UNet++, and the formula of UNet++ is as follows:

[0060] embedding=encoder(I),f=multi-scale-decoder(embedding), where, is the input image, is the feature after encoding and downsampling in the network, It is the feature map obtained after the image passes through the multi-scale decoder and upsampling.

[0061] The labeled dataset is input into the pre-established feature extraction network model to extract the multi-scale features of the image and output the multi-scale image feature map: the multi-scale features of the image are extracted using the multi-scale pyramid structure, and residual connections are introduced to solve the gradient disappearance and gradient explosion problems in the deep network training process. In order to solve the problem of small convolution receptive field, we add dilated convolution to the network, increasing the receptive field by introducing holes between the convolution kernels without introducing additional parameters. This allows the network to better capture the features of satellites of different sizes, and the final output feature map is

[0062] Perform semantic segmentation on the multi-scale image feature map to obtain a foreground mask image, and predict the key point offset vector for each foreground pixel in the foreground mask image to obtain a key point offset vector field;

[0063] Semantic segmentation of multi-scale image feature maps: First, a semantic segmentation head is used to classify each pixel in the scene and obtain the foreground mask image of the target. Among them, 1 represents the foreground object and 0 represents the background. Then, for each foreground pixel [p x ,p y], the key point offset vector field prediction head predicts a two-dimensional offset vector [Δ x ,Δ y ], which means that the 2D coordinates of the predefined i-th key point are [p x +Δ x ,p y +Δ y ], assuming that there are K key point coordinates defined in advance on the satellite model, K offset vector predictions will be generated for each pixel, and the final prediction output of all pixels is It represents the offset vector field from each pixel to the predefined K key points.

[0064] The RANSAC algorithm is used to process the key point offset vector field to find the best estimate of the key point 2D coordinates, and the pixel coordinates of the satellite key points are obtained. The PnP algorithm is used to fit the pixel coordinates of the satellite key points and the corresponding coordinates on the 3D model to obtain the position of the target satellite relative to the camera.

[0065] The RANSAC algorithm is used to find the best estimate of the keypoint's 2D coordinates within the keypoint offset vector field: First, a certain number of foreground pixels are randomly selected and their offset vectors are used to predict the keypoint's 2D coordinates, generating a set of hypotheses. This set of hypotheses is then checked for consistency, calculating the proportion of foreground pixel predictions that meet the set of hypotheses. This step is repeated, each time selecting a different random foreground pixel to solve for the keypoint coordinates. All pixels are then voted on. Ultimately, the hypothesis with the most cumulative "inliers" support is considered the best estimate of the keypoint's 2D coordinates. This yields the pixel coordinates of the satellite's keypoints.

[0066] The process of using the PnP algorithm to solve the pixel coordinates of the satellite's key points to obtain the position of the target satellite relative to the camera:

[0067] Specifically, the present invention will be further described below through examples:

[0068] The pixel coordinates of the satellite key points in the image are detected through the key point offset vector field and RANSAC-based iterative estimation. Finally, we use the PnP (Perspective-n-Point) algorithm to solve the pose of the target satellite relative to the camera using a set of 2D image coordinates and the corresponding 3D model point cloud coordinates. Specifically, after transforming the 3D coordinates of the key points into the camera coordinate system through the pose, we compare them with the 2D coordinates of the key points estimated in the camera. We obtain the final 3D rotation R and 3D translation T by optimizing the following reprojection error:

[0069]

[0070] Among them, K is the camera internal parameter, and are the 3D coordinates and 2D pixel coordinates of the detected key points, respectively.

[0071] Example 2: The second aspect, as Figure 6 As shown, in order to achieve the above-mentioned purpose, the present invention discloses a satellite pose estimation system, comprising:

[0072] The data processing module is used to randomly sample multiple sets of satellite pose sequences, predefine a set of key points on the satellite 3D model, and simulate the satellite pose sequences using the satellite 3D model to obtain a labeled data set;

[0073] The feature extraction module is used to input the labeled data set into the pre-established feature extraction network model, extract the multi-scale features of the image, and output a multi-scale image feature map;

[0074] The semantic segmentation module is used to perform semantic segmentation on the multi-scale image feature map to obtain a foreground mask image, and predict the key point offset vector for each foreground pixel in the foreground mask image to obtain a key point offset vector field;

[0075] The pose solving module is used to process the key point offset vector field using the RANSAC algorithm to find the best estimate of the key point 2D coordinates, obtain the pixel coordinates of the satellite key points, and use the PnP algorithm to fit the pixel coordinates of the satellite key points and the corresponding coordinates on the 3D model to obtain the pose of the target satellite relative to the camera.

[0076] In conjunction with the second aspect, in some implementations of the second aspect, the system further includes: the artificial satellite posture sequence in the data processing module is marked as {R i ,T i},in, represents the rotation matrix of the object at time i, Represents the translation vector of the object at time i;

[0077] Or simulate the satellite pose sequence in the data processing module and render the RGB image sequence in the simulated space environment through 3dMax. i}, thereby obtaining a labeled dataset {I i ,R i ,T i};

[0078] Preferably, the feature extraction network model pre-established in the feature extraction module adopts UNet++, and the formula of UNet++ is as follows:

[0079] embedding=encoder(I),f=multi-scale-decoder(embedding), where, is the input image, is the feature after encoding and downsampling in the network, It is the feature map obtained after the image passes through the multi-scale decoder and upsampling.

[0080] Preferably, the feature extraction module inputs the labeled data set into a pre-established feature extraction network model, extracts the multi-scale features of the image, and outputs a multi-scale image feature map:

[0081] The multi-scale features of the image are extracted using the multi-scale pyramid structure, and residual connections are introduced. Dilated convolution is added to increase the receptive field by introducing holes between the convolution kernels without introducing additional parameters. This allows the network to better capture the features of satellites of different sizes. The final output multi-scale image feature map is:

[0082] Preferably, the semantic segmentation module performs semantic segmentation on the multi-scale image feature map and classifies it through the semantic segmentation head to obtain a foreground mask image. For each foreground pixel [p x ,p y ], the key point offset vector field prediction head predicts a two-dimensional offset vector [Δ x ,Δ y ], which means that the 2D coordinates of the predefined i-th key point are [p x +Δ x ,p y +Δ y ], assuming that there are K key point coordinates defined in advance on the satellite model, K offset vector predictions will be generated for each pixel, and the final prediction output of all pixels is As a keypoint offset vector field;

[0083] Preferably, the pose solving module uses the RANSAC algorithm to find the best estimate of the 2D coordinates of the key points in the key point offset vector field: randomly extract a certain number of foreground pixels, and use the offset vectors of these pixels to predict the 2D coordinates of the key points, generate a set of hypotheses, perform consistency checks on the set of hypotheses, calculate the proportion of all foreground pixel predictions that meet the set of hypotheses, repeat the above steps, each time selecting different random foreground pixels to solve the key point coordinates, and obtain the pixel coordinates of the key points of the artificial satellite;

[0084] Preferably, the pose solving module uses the PnP algorithm to solve the pixel coordinates of the key points of the artificial satellite to obtain the pose of the target artificial satellite relative to the camera:

[0085] After the 3D coordinates of the satellite key points are transformed into the camera coordinate system through the pose, they are compared with the 2D coordinates of the satellite key points estimated in the camera. The final 3D rotation R and 3D translation T are obtained by optimizing the following reprojection error:

[0086]

[0087] Among them, K is the camera internal parameter, and are the 3D coordinates and 2D pixel coordinates of the detected key points respectively. R, T are the rotation and translation parameters to be optimized, R * ,T * is the optimal estimate that minimizes the reprojection error.

[0088] This method can achieve an estimation accuracy of more than 85% when the rotation error is less than 10 degrees and the translation error is less than 10 units. For specific pose estimation results, please refer to Figure 5 .

[0089] Based on the same inventive concept, the present invention also provides a computer device, which includes: one or more processors and a memory for storing one or more computer programs; the program includes program instructions, and the processor is used to execute the program instructions stored in the memory. The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It is the computing core and control core of the terminal, which is used to implement one or more instructions, specifically for loading and executing one or more instructions in a computer storage medium to implement the above method.

[0090] It should be further explained that, based on the same inventive concept, the present invention also provides a computer storage medium having a computer program stored thereon, which executes the above method when executed by a processor. The storage medium can be any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electrical, magnetic, infrared, or semiconductor system, device or component, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device or component.

[0091] Throughout this specification, references to terms such as "one embodiment," "example," or "specific example" indicate that a specific feature, structure, material, or characteristic described in conjunction with that embodiment or example is included in at least one embodiment or example of the present disclosure. In this specification, schematic representations of these terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0092] The above shows and describes the basic principles, main features and advantages of the present disclosure. Those skilled in the art should understand that the present disclosure is not limited to the above embodiments. The above embodiments and descriptions are merely illustrative of the principles of the present disclosure. Various changes and improvements may be made to the present disclosure without departing from the spirit and scope of the present disclosure, and such changes and improvements shall fall within the scope of the present disclosure.

Claims

1. A method for estimating the position and attitude of an artificial satellite, characterized in that: The method comprises the following steps: Randomly sample multiple sets of satellite pose sequences, pre-define a set of key points on the satellite 3D model, and use the satellite 3D model to simulate and render the satellite pose sequences to obtain a labeled dataset; The artificial satellite posture sequence is simulated and rendered by 3dMax in a simulated space environment to render the RGB image sequence {I i }, thereby obtaining a labeled dataset {I i ,R i ,T i };in, represents the rotation matrix of the object at time i, Represents the translation vector of the object at time i; Input the labeled dataset into the pre-established feature extraction network model to extract the multi-scale features of the image and output a multi-scale image feature map; The pre-established feature extraction network model adopts UNet++, and the formula of UNet++ is as follows: embedding=encoder(I),f=multi-scale-decoder(embedding), in, is the input image, is the feature after encoding and downsampling in the network, It is the feature map obtained after the image passes through multi-scale-decoder and upsampling; The process of inputting the labeled dataset into the pre-established feature extraction network model, extracting the multi-scale features of the image, and outputting a multi-scale image feature map: The multi-scale features of the image are extracted using the multi-scale pyramid structure, and residual connections are introduced. Dilated convolution is added to increase the receptive field by introducing holes between the convolution kernels without introducing additional parameters. This allows the network to better capture the features of satellites of different sizes. The final output multi-scale image feature map is: Perform semantic segmentation on the multi-scale image feature map to obtain a foreground mask image, and predict the key point offset vector for each foreground pixel in the foreground mask image to obtain a key point offset vector field; The multi-scale image feature map is semantically segmented: the foreground mask image is obtained by classification through the semantic segmentation head For each foreground pixel [p x ,p y ], the key point offset vector field prediction head predicts a two-dimensional offset vector [Δ x ,Δ y ], which means that the 2D coordinates of the predefined i-th key point are [p x +Δ x ,p y +Δ y ], assuming that there are K key point coordinates defined in advance on the satellite model, K offset vector predictions will be generated for each foreground pixel, and the final prediction output of all foreground pixels is As a keypoint offset vector field; The RANSAC algorithm is used to process the key point offset vector field to find the best estimate of the key point's 2D coordinates, and the pixel coordinates of the satellite key points are obtained. The PnP algorithm is used to fit the pixel coordinates of the satellite key points to the corresponding coordinates on the 3D model to obtain the position of the target satellite relative to the camera. The process of using the PnP algorithm to solve the pixel coordinates of the satellite's key points to obtain the position of the target satellite relative to the camera: After the 3D coordinates of the satellite key points are transformed into the camera coordinate system through the pose, they are compared with the 2D coordinates of the satellite key points estimated in the camera. The final 3D rotation R and 3D translation T are obtained by optimizing the following reprojection error: Among them, k is the camera internal parameter, and are the 3D coordinates and 2D pixel coordinates of the detected key points, R, T are the rotation and translation parameters to be optimized, R * ,T * is the optimal estimate that minimizes the reprojection error.

2. A method for estimating the position and attitude of an artificial satellite according to claim 1, characterized in that: The satellite pose sequence is marked as {R i ,T i },in, represents the rotation matrix of the object at time i, Represents the translation vector of the object at time i.

3. The method for estimating the position and attitude of an artificial satellite according to claim 1, wherein: The process of using the RANSAC algorithm to find the best estimate of the 2D coordinates of the key point in the key point offset vector field is as follows: randomly extracting some foreground pixels and using the offset vectors of the pixel points of the foreground pixels to predict the 2D coordinates of the key point, generating a set of hypotheses, performing a consistency check on this set of hypotheses, calculating the proportion of all foreground pixel predictions that meet this set of hypotheses, repeating the above steps, each time selecting different random foreground pixels to solve the key point coordinates, and obtaining the pixel coordinates of the key point of the artificial satellite.

4. A satellite pose estimation system, characterized in that: include: The data processing module is used to randomly sample multiple sets of satellite pose sequences, predefine a set of key points on the satellite 3D model, and simulate the satellite pose sequences using the satellite 3D model to obtain a labeled data set; The data processing module simulates the rendering of the satellite pose sequence and uses 3dMax to render the RGB image sequence in the simulated space environment. i }, thereby obtaining a labeled dataset {I i ,R i ,T i };in, represents the rotation matrix of the object at time i, Represents the translation vector of the object at time i; Input the labeled dataset into the pre-established feature extraction network model to extract the multi-scale features of the image and output a multi-scale image feature map; The pre-established feature extraction network model adopts UNet++, and the formula of UNet++ is as follows: embedding=encoder(I),f=multi-scale-decoder(embedding), in, is the input image, is the feature after encoding and downsampling in the network, It is the feature map obtained after the image passes through multi-scale-decoder and upsampling; The feature extraction module inputs the labeled dataset into the pre-established feature extraction network model, extracts the multi-scale features of the image, and outputs the multi-scale image feature map: The multi-scale features of the image are extracted using the multi-scale pyramid structure, and residual connections are introduced. Dilated convolution is added to increase the receptive field by introducing holes between the convolution kernels without introducing additional parameters. This allows the network to better capture the features of satellites of different sizes. The final output multi-scale image feature map is: The semantic segmentation module is used to perform semantic segmentation on the multi-scale image feature map to obtain a foreground mask image, and predict the key point offset vector for each foreground pixel in the foreground mask image to obtain a key point offset vector field; The multi-scale image feature map is semantically segmented: the foreground mask image is obtained by classification through the semantic segmentation head For each foreground pixel [p x ,p y ], the key point offset vector field prediction head predicts a two-dimensional offset vector [Δ x ,Δ y ], which means that the 2D coordinates of the predefined i-th key point are [p x +Δ x ,p y +Δ y ], assuming that there are K key point coordinates defined in advance on the satellite model, K offset vector predictions will be generated for each foreground pixel, and the final prediction output of all foreground pixels is As a keypoint offset vector field; The pose solving module is used to process the key point offset vector field using the RANSAC algorithm to find the best estimate of the key point's 2D coordinates, obtain the pixel coordinates of the satellite's key points, and use the PnP algorithm to fit the pixel coordinates of the satellite's key points to the corresponding coordinates on the 3D model to obtain the pose of the target satellite relative to the camera; The pose solving module uses the PnP algorithm to solve the pixel coordinates of the satellite's key points to obtain the pose of the target satellite relative to the camera: After the 3D coordinates of the satellite key points are transformed into the camera coordinate system through the pose, they are compared with the 2D coordinates of the satellite key points estimated in the camera. The final 3D rotation R and 3D translation T are obtained by optimizing the following reprojection error: Among them, k is the camera internal parameter, and are the 3D coordinates and 2D pixel coordinates of the detected key points, R, T are the rotation and translation parameters to be optimized, R * ,T * is the optimal estimate that minimizes the reprojection error.

5. The artificial satellite pose estimation system according to claim 4, characterized in that: The satellite pose sequence in the data processing module is marked as {R i ,T i },in, represents the rotation matrix of the object at time i, Represents the translation vector of the object at time i; The pose solution module uses the RANSAC algorithm to find the best estimate of the 2D coordinates of the key points in the key point offset vector field: randomly extract some foreground pixels and use the offset vectors of the foreground pixels to predict the 2D coordinates of the key points. Then, a set of hypotheses is generated, and the consistency of this set of hypotheses is checked. The proportion of all foreground pixel predictions that meet this set of hypotheses is calculated. The above steps are repeated, each time a different random foreground pixel is selected to solve the key point coordinates, and the pixel coordinates of the satellite key points are obtained.

Images