Visual positioning method for satellite on-orbit fuel filling docking service

Abstract

The invention discloses a visual positioning method for satellite on-orbit fuel filling docking service, which comprises two stages of off-line training and on-line positioning, and comprises the following steps: firstly, acquiring image and pose information by using a camera, establishing a pre-matching database, carrying out feature matching, calculating the pose of the camera through structural motion, and generating sparse point cloud; carrying out dense reconstruction on the scene by adopting three-dimensional Gaussian sputtering, and generating a three-dimensional Gaussian sputtering model through training; after a camera at the tail end of the refueling satellite mechanical arm captures an original image in real time, a reference image most similar to the current image is recognized based on the pre-matching database, a corresponding initial pose is obtained, and the image is rendered to serve as a starting point of subsequent iterative optimization; and meanwhile, color migration correction is performed on the captured image by taking the rendered image as a reference, and inter-domain differences are eliminated to generate a target image in an optimization stage. According to the invention, a picture captured at an unknown view angle can be well positioned. And the calculation efficiency is improved while the accuracy is ensured as much as possible.

Description

Technical Field

[0001] The present invention belongs to the technical field of on-orbit fuel refueling, and specifically relates to a visual positioning method and system for satellite on-orbit fuel refueling docking service. Background Technique

[0002] With the rapid development of space technology, the extension of the on-orbit life of satellites has made space fuel replenishment a key challenge. How to accurately and efficiently complete fuel replenishment to ensure the continuous operation of satellites has become an important topic in the space field. In fuel replenishment, it is particularly crucial to use computer vision technology to solve the positioning problem of the docking hole and ensure the successful docking of the refueling satellite and the target satellite. This technology requires quickly identifying and positioning the position of the docking hole of the target satellite, so that the oil delivery device carried by the manipulator of the refueling satellite can successfully dock with the docking hole of the target satellite, complete the fuel replenishment task, and ensure the stable operation of the satellite in space.

[0003] The key to solving the visual positioning problem of satellite on-orbit docking service lies in the scene reconstruction of the complex space environment. In the space environment, due to extreme lighting conditions, limited field of view, and the fact that the satellite surface is usually covered with different materials, which may have strong reflectivity or light absorption, etc., it will lead to a decrease in image quality and difficulty in scene reconstruction. Taking ORB-SLAM3 as an example, in special scenes with complex shadows or local lighting, the local gray information near the feature points relied on by its BRIEF descriptor will show inconsistent distributions under different perspectives. For example, when the light source direction changes or the local part of the target object is covered by shadows, originally similar feature points may produce completely different descriptor representations, resulting in matching failures or incorrect matches. Summary of the Invention

[0004] To solve the above problems, the present invention discloses a visual positioning method for satellite on-orbit fuel refueling docking service. Based on three-dimensional Gaussian sputtering technology, combined with color transfer to process images, and using an optimization strategy to optimize and iterate the external parameters of the camera, so as to assist in accurately positioning the satellite docking hole and solve the problems proposed in the above background technique.

[0005] To achieve the above object, the technical solution of the present invention is as follows:

[0006] A visual positioning method for satellite on-orbit fuel refueling docking service includes two stages: offline training and online positioning. The specific steps are as follows:

[0007] In the offline training stage, first, a camera mounted on the end of the refueling satellite's robotic arm is used to collect detailed images of the target satellite, the docking hole, and the surrounding scene, obtaining the complete image and pose information of the scene; a pre-matching database is established based on the collected data for feature matching, and the camera pose is calculated through structure from motion and a sparse point cloud is generated; on this basis, three-dimensional Gaussian sputtering is used to perform dense reconstruction on the scene, and a three-dimensional Gaussian sputtering model with high fidelity is generated through training.

[0008] In the online positioning stage, after the camera at the end of the refueling satellite's robotic arm captures the original image in real time, based on the pre-matching database constructed in the offline training stage, the reference image most similar to the current image is identified, and the corresponding initial pose is obtained and the image is rendered as the starting point for subsequent iterative optimization; at the same time, taking the rendered image as the reference benchmark, color transfer correction is performed on the captured image to eliminate the inter-domain differences to generate the target image for the optimization stage.

[0009] For the obtained initial camera pose T0 and the target image I t , solving for the current camera pose in the three-dimensional Gaussian sputtering model coordinate system G is formulated as an optimization problem:

[0010] T i =argminL1(I i ,I t )

[0011] where I i represents the image rendered through the three-dimensional Gaussian sputtering differentiable renderer when the camera pose is T i , L1 is the L1 loss function, which is used to measure the pixel-level difference between the rendered image I i and the target image I t ; after describing the image difference, the three-dimensional pose is optimized accordingly. Since the rotation component exhibits strong nonlinearity, a gradient-free optimization algorithm is used to optimize it; for the translation component, the image difference is backpropagated to the camera pose parameters, and then the gradient descent method is used to optimize it; the advantages of combining the prior threshold and the early stopping method are used in the optimization process.

[0012] After solving for the optimal pose T t of the current camera in the three-dimensional Gaussian sputtering model coordinate system G through the pose optimization strategy, combined with the robotic arm servo system connected to the camera, its pose in the robotic arm base coordinate system B is further determined, and the transformation relationship between these two coordinate systems is solved:

[0013]

[0014] where, G T camRepresents the pose of the camera in coordinate system G, which is equal to the optimal pose T obtained above. t , B T cam Represents the pose of the camera in coordinate system B. Represents the homogeneous matrix for the transformation from the 3D Gaussian sputtering model coordinate system G to the robotic arm base coordinate system B.

[0015] After obtaining the above transformation relationship, the pose of the target satellite docking hole is pre-determined in the 3D Gaussian sputtering model G, and further mapped to the current robotic arm base coordinate system B:

[0016]

[0017] Among them, G T target Is the pose of the target point in coordinate system G. B T target Is the pose of the target point after being transformed to coordinate system B.

[0018] After determining the pose of the target docking hole in the current robotic arm coordinate system B, the robotic arm can be guided to accurately position to the target position, perform a compliant control insertion task, and complete the docking and fuel filling operations.

[0019] The beneficial effects of the present invention are:

[0020] (1) The present invention uses 3D Gaussian sputtering technology for satellite scene reconstruction. It represents the objects in the scene by fitting Gaussian distributions in 3D space, and the renderer can simulate the appearance changes of objects under different lighting conditions. This makes the method not rely on fixed grayscale patterns or local features and can effectively achieve visual positioning in complex environments.

[0021] (2) The present invention introduces color transfer to process the rendered images, avoiding significant color differences between the model-rendered images and the actual scenes caused by environmental factors such as lighting conditions and camera apertures. Such colors may cause the objective function to fall into a local optimum, affecting the optimization convergence speed and stability.

[0022] (3) The present invention uses a method combining a priori threshold and early stopping method in the pose optimization strategy. This strategy allows the error to fluctuate slightly near the target threshold and decides whether to continue optimization according to the error trend, significantly improving the optimization efficiency while maintaining high-quality results and meeting the real-time and accuracy requirements of satellite docking and fuel filling tasks in space. Description of the Drawings

[0023] Figure 1 Is the overall flowchart of the visual positioning method in the embodiment of the present invention.

[0024] Figure 2 It is a schematic diagram of camera pose optimization in an embodiment of the present invention.

[0025] Figure 3 It is a pseudo-code diagram of the camera pose optimization strategy in an embodiment of the present invention.

[0026] Figure 4 It is the configuration of the assembly system in an embodiment of the present invention.

[0027] Figure 5 It is the variation of L1 loss, position error, and angle error with the number of iterations during the experimental pose optimization in an embodiment of the present invention.

[0028] Figure 6 It is a schematic diagram of the result of the system experiment in an embodiment of the present invention. Detailed implementation manners

[0029] The present invention will be further clarified below in conjunction with the accompanying drawings and detailed implementation manners. It should be understood that the following detailed implementation manners are only used to illustrate the present invention and not to limit the scope of the present invention.

[0030] Embodiment

[0031] Figure 1 It is a schematic diagram of the overall process of the visual positioning method described in the present invention. The entire system method is divided into two stages: an offline training stage and an online positioning stage. In the offline training stage, based on the original training images and camera poses, a three-dimensional Gaussian sputtering technique is used to reconstruct the satellite and its surrounding scenes, generating a three-dimensional Gaussian sputtering model. In the online positioning stage, after the camera at the end of the refueling satellite manipulator captures the original image in real time, pre-matching processing needs to be performed first; based on the pre-matching database constructed in the offline training stage, the reference image most similar to the current image is identified, and the corresponding initial pose is obtained and the image is rendered as the starting point for subsequent iterative optimization. At the same time, taking the image rendered by the three-dimensional Gaussian sputtering model as a reference benchmark, the captured image is color-corrected to eliminate the inter-domain differences to generate the target image in the optimization stage; after image preprocessing and color transfer, the initial camera pose and the target image are obtained, the camera pose is optimized and solved, and after obtaining the optimal pose, the transformation relationship between the three-dimensional Gaussian sputtering model coordinate system G and the manipulator base coordinate system B can be obtained, so as to further map the pose of the docking hole of the target satellite in the coordinate system G to the current coordinate system B to guide the docking hole of the refueling satellite manipulator to complete the docking task.

[0032] Figure 2 It is a schematic diagram of camera pose optimization in an embodiment of the present invention. Solving the current camera pose on the three-dimensional Gaussian sputtering model coordinate system G can be modeled as an optimization problem: T i = argmin L1(I i , I t ), where Ii Indicates the image rendered by the 3D Gaussian sputtering differentiable renderer when the camera pose is T i ; L1 is the L1 loss function, which is used to measure the pixel-level difference between the rendered image and the target image. After describing the image difference, the 3D pose is optimized accordingly.

[0033] Figure 3 It is the pseudocode diagram of the camera pose optimization strategy in the embodiment of the present invention. The gradient-free optimization algorithm is adopted for the rotation component of the pose, and the gradient descent method is adopted for the translation component. During the optimization iteration process, the current pose estimate and its corresponding error are calculated each time, where calculating the error includes comparing the pixel-level difference between the rendered picture corresponding to the current pose and the target picture and obtaining the L1 loss. If the current error is less than the preset error threshold, further judgment is made: if the current error is less than the recorded minimum error, the patience counter is incremented by x and limited within the maximum y; otherwise, the patience counter is decremented by 1. If the current error is less than the currently recorded minimum error, the minimum error and the corresponding optimal pose are updated. When the number of iterations reaches N, or the current error is less than the threshold and the patience counter is decremented to 0, the optimization process stops, and the optimal pose corresponding to the recorded minimum error is returned as the final result.

[0034] Figure 4 It is the configuration diagram of the assembly system in the embodiment of the present invention. It is composed of a satellite model, a six-degree-of-freedom robotic arm of model JAKAZu12, and an Intel RealSense D435i depth camera mounted on the robotic arm. The system uses a computer as the main control host computer, uses an operation platform configured with AMD R 55600H and NVIDIA GeForce GTX 1650, and communicates with the lower computer development board in real time through a high-speed serial port, thereby controlling the operation of the robotic arm. The robotic arm is placed on an experimental platform; in order to simulate the weightless environment in space, four ropes are used to lift the satellite model so that it floats in the air and can move slightly within a certain range. There is a docking hole on the front of the satellite model, and the robotic arm needs to achieve precise positioning of the docking hole position.

[0035] Figure 5It shows the variation of L1 loss, position error, and angle error with the number of iterations during the camera pose optimization experiment in the embodiments of the present invention. There is an obvious correlation between the error function and the position distance and angle. As the number of iterations increases, the pose optimizer adjusts the current pose according to the metrics given by the error function, continuously reducing the error on the image, and gradually approaching the target pose in terms of position and angle. At the same time, the pose optimization strategy of the present invention quickly converges after several iterations in the experimentally set scenario, with a significant reduction in L1 loss, position, and angle errors, and can quickly capture the pose close to the final convergence result, showing a high optimization efficiency.

[0036] Figure 6 It is a schematic diagram of the results of the system experiment in the embodiments of the present invention. In the experiment, the method of the present invention was compared with the classic SLAM algorithm ORB - SLAM3. In terms of the average distance error, ORB - SLAM3 was 12.63 cm, and the method of the present invention was 3.96 cm; in terms of the average angle error, ORB - SLAM3 was 1.59°, and the method of the present invention was 1.44°. Compared with the traditional ORB - SLAM3 algorithm, the present invention performs better in controlling the distance and angle errors, and can significantly reduce its average error and median error. At the same time, the error distribution is highly concentrated and the tail is short, indicating its optimal accuracy and robustness.

[0037] The present invention is superior to the traditional visual positioning method in terms of position accuracy and angle accuracy, and can better locate the pictures captured from unknown perspectives. With the help of color transfer, the stability and positioning accuracy of the present invention are further improved; the proposed and applied pose optimization strategy improves the calculation efficiency while ensuring accuracy as much as possible.

[0038] It should be noted that the above content only illustrates the technical idea of the present invention and cannot be used to limit the protection scope of the present invention. For those of ordinary skill in the art of this technology, without departing from the principle of the present invention, several improvements and refinements can be made, and these improvements and refinements all fall within the protection scope of the claims of the present invention.

Claims

1. A visual positioning method for on-orbit fueling docking service of a satellite, characterized in that: It includes two stages: offline training and online positioning. The specific steps are as follows: In the offline training stage, first, a camera mounted on the end of the manipulator of the refueling satellite is used to collect detailed images of the target satellite, the docking hole, and the surrounding scene, obtaining the complete image and pose information of the scene. Based on the collected data, a pre-matching database is established for feature matching. The camera pose is calculated by structure from motion and a sparse point cloud is generated. On this basis, three-dimensional Gaussian sputtering is used to perform dense reconstruction of the scene, and a three-dimensional Gaussian sputtering model with high fidelity is generated through training. In the online positioning stage, after the camera at the end of the refueling satellite manipulator captures the original image in real time, based on the pre-matching database constructed in the offline training stage, the reference image most similar to the current image is identified, and the corresponding initial pose is obtained and the image is rendered as the starting point for subsequent iterative optimization. At the same time, taking the rendered image as a reference benchmark, color transfer correction is performed on the captured image to eliminate the inter-domain difference to generate the target image in the optimization stage.

2. The visual positioning method for in-orbit fuel filling docking service of a satellite according to claim 1, wherein: For the obtained initial camera pose \(T_0\) and the target image \(I\) t , solving for the current camera pose on the three-dimensional Gaussian sputtering model coordinate system \(G\) is modeled as an optimization problem: T i = argminL1(I i ,I t ) Among them, I i represents the image rendered by the three-dimensional Gaussian sputtering differentiable renderer when the camera pose is T i , L1 is the L1 loss function, which is used to measure the pixel-level difference between the rendered image I i and the target image I t ; after describing the image difference, the three-dimensional pose is optimized accordingly; since the rotation component exhibits strong nonlinearity, a gradient-free optimization algorithm is used to optimize it; for the translation component, the image difference is backpropagated to the camera pose parameters, and then the gradient descent method is used to optimize it.

3. The visual positioning method for on-orbit fuel filling and docking service of a satellite according to claim 2, characterized in that: The optimal pose T of the current camera in the 3D Gaussian sputtering model coordinate system G is solved through the pose optimization strategy t After that, combined with the robotic arm servo system connected to the camera, its pose in the robotic arm base coordinate system B is further determined, and the transformation relationship between these two coordinate systems is solved: Among them, G T cam represents the pose of the camera in the coordinate system G, which is equal to the optimal pose T obtained above t , B T cam represents the pose of the camera in the coordinate system B, represents the homogeneous matrix for the transformation from the three-dimensional Gaussian sputtering model coordinate system G to the robotic arm base coordinate system B.

4. The visual positioning method for on-orbit fuel filling and docking service of a satellite according to claim 3, characterized in that: After obtaining the transformation relationship between the two coordinate systems, the pose of the docking hole of the target satellite is predetermined in the three-dimensional Gaussian sputtering model G, and further mapped to the current manipulator base coordinate system B. Among them, G T target is the pose of the target point in coordinate system G, B T target is the pose of the target point transformed to coordinate system B; After determining the pose of the target docking hole in the current manipulator coordinate system B, the manipulator is guided to accurately position to the target position, and the compliant control insertion task is executed to complete the docking and fuel injection operations.

5. A vision positioning system for satellite in-orbit fueling docking service, characterized in that: It includes a satellite, a control module, a six-degree-of-freedom manipulator, and a depth camera mounted on the manipulator. There is a docking hole on the front of the satellite. The six-degree-of-freedom manipulator is mounted on the refueling satellite, and the depth camera is fixed at the end of the manipulator. The front end of the manipulator realizes accurate positioning with the docking hole under the control of the control module.

Images