Accurate model unknown non-cooperative spacecraft relative pose measurement method, medium and equipment
By using YOLOv10 and monocular relative attitude measurement network, combining spatial object detection and relative attitude datasets, the problem of relative attitude measurement of non-cooperative spacecraft with an accurate model is solved, and high-precision relative attitude measurement is achieved, which is suitable for a variety of spacecraft missions.
Patent Information
- Application Number
- CN202411591407.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-08
- Publication Date
- 2025-02-28
AI Technical Summary
The prior art is difficult to perform relative posture measurements on non-cooperative spacecraft with an accurate model unknown, especially in the case of only partial prior information, which cannot effectively provide accurate measurement results.
The spatial object detection network and monocular relative pose measurement network based on YOLOv10 are used to capture spacecraft images in the ground darkroom, construct spatial object detection data sets and relative pose data sets, and train the network model to extract the relative pose and position of the target.
Under the premise of accurate models that do not rely on the target, accurate relative posture measurement results can be provided, and are suitable for non-cooperative spacecraft with unknown accurate models, improving the accuracy and breadth of spacecraft measurements.
Smart Images
Figure CN119540349A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of space target posture measurement, and specifically relates to a relative posture measurement method, medium and equipment for non-cooperative spacecraft with unknown precise model. Background Art
[0002] In recent years, in order to solve the congestion problem in low-Earth orbit and extend the life of geosynchronous satellites, tasks such as space debris removal and on-orbit maintenance services have received increasing attention. These tasks include the debris removal mission of the Surrey Space Center, the e.Deorbit mission of the European Space Agency, the OAAN and Restore-L programs of NASA, etc. The relative position and attitude measurement of the target spacecraft is the key to completing these tasks. However, since the target object is usually a non-cooperative spacecraft such as a scrapped satellite or space debris, its status information cannot be effectively provided. Therefore, the relative position and attitude of the target spacecraft can only be measured by onboard sensors.
[0003] Satellite-borne sensors can be divided into two categories according to the measurement method: active and passive. Passive sensors are relatively more widely used due to their advantages such as simpler hardware structure and lower power consumption. Passive sensors mainly include binocular cameras and monocular cameras. Existing spacecraft, especially emerging small spacecraft such as cube satellites, are difficult to meet the baseline requirements of binocular cameras, and monocular cameras are more in line with the requirements of low power consumption and low quality. Therefore, monocular cameras have become a popular choice for satellite-borne attitude measurement sensors.
[0004] Early non-cooperative target pose estimation methods based on monocular cameras mainly rely on classical image processing algorithms to extract artificially designed relevant features from two-dimensional images, and then match these relevant features with the known accurate three-dimensional model of the target spacecraft to extract the relative position and pose information of the target. However, the features designed by these methods are difficult to apply to spacecraft with different structures and physical properties, and are easily affected by external environments such as extreme lighting environments and dynamic earth backgrounds.
[0005] In recent years, the great success of deep learning in image classification and target detection has led many researchers to try to use it to solve the problem of target pose measurement based on monocular cameras. These methods mostly rely on the powerful fitting ability of convolutional neural networks to learn the nonlinear mapping between input images and output labels, thereby calculating the relative pose information of the target. Although these methods have achieved certain results, their accuracy depends on key point regression and PnP algorithms, and they need to obtain the precise three-dimensional model of the target in advance, which makes it difficult to apply to non-cooperative spacecraft with unknown precise models. However, in actual tasks such as space debris removal and on-orbit maintenance services, the precise three-dimensional model of the target is often difficult to obtain, and only some prior information such as target type, approximate component composition, number of sails, etc. can be obtained, so these methods cannot be used. Summary of the invention
[0006] In view of the deficiencies in the prior art, the present invention provides a method, medium and device for measuring the relative posture of a non-cooperative spacecraft with an unknown precise model.
[0007] To achieve the above object, the present invention adopts the following technical solutions:
[0008] In a first aspect, the present invention provides a method for measuring relative position and attitude of a non-cooperative spacecraft with an unknown precise model, comprising the following steps:
[0009] Take spacecraft-related image data in a darkroom on the ground and construct a space target detection dataset through manual annotation;
[0010] The constructed space target detection dataset is used to train the space target detection network model based on YOLOv10;
[0011] The trained spatial target detection network model is used to extract the target area image from the captured image data, and a relative pose dataset is constructed in combination with the target relative pose label at the time of shooting.
[0012] A monocular relative pose measurement network model is built based on a convolutional neural network, and the constructed relative pose dataset is used to train the monocular relative pose measurement network model;
[0013] In actual application scenarios, a monocular camera is used to shoot the detection area and collect image data of the detection area;
[0014] Use the trained spatial target detection network model to detect the detection area image, obtain the target category and its position in the image, and extract the target image;
[0015] The extracted target image is measured using the trained monocular relative pose measurement network to obtain the relative pose and relative position of the target.
[0016] Optionally, the manual labeling method includes manually labeling space target category labels and space target bounding box labels.
[0017] Optionally, the space target detection network model can detect and identify target categories and common components, the target categories include satellites, rocket final stages and spacecraft, and the common components include the body, solar panels, tripods, docking rings, detection payloads and nozzles.
[0018] Optionally, the monocular relative pose measurement network model includes a backbone network, a relative attitude prediction subnetwork and a relative position prediction subnetwork, the output of the relative attitude prediction subnetwork is the relative attitude quaternion of the target, the target relative attitude quaternion is encoded using a discrete quaternion array through a soft classification method, and the output of the relative position prediction subnetwork is the three-axis relative position of the target.
[0019] Optionally, the relative posture prediction subnetwork includes a posture weight distribution branch and an error quaternion branch, the posture weight distribution branch is responsible for predicting the weight distribution of each quaternion in the discrete quaternion array, and the error quaternion branch is responsible for regressing the error quaternion. The results of the two branches are obtained by using a quaternion weighted average algorithm and quaternion multiplication to obtain the relative posture quaternion of the target.
[0020] Optionally, the relative posture prediction subnetwork includes 2 convolutional layers, 5 fully connected layers, 1 Softmax layer and 1 quaternion normalization layer; the relative position prediction subnetwork includes 1 convolutional layer and 6 fully connected layers.
[0021] In a second aspect, the present invention provides a computer-readable storage medium storing a computer program, wherein the computer program enables a computer to execute the relative posture measurement method of a non-cooperative spacecraft with an unknown precise model as described in the first aspect.
[0022] In a third aspect, the present invention provides an electronic device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method for measuring relative posture of a non-cooperative spacecraft with an unknown precise model as described in the first aspect is implemented.
[0023] The beneficial effects of the present invention are as follows: the present invention first uses a trained YOLOv10-based space target detection network to perform target detection on the detection area to obtain the type of target and extract the required target image; then uses a trained monocular relative pose measurement network to measure the extracted target image to obtain the relative pose of the target spacecraft. In the above manner, the present invention can perform relative pose measurement on non-cooperative spacecraft whose precise model is unknown with only partial prior information (such as target type, approximate component composition, number of sailboards, etc.), and can provide accurate measurement accuracy without relying on the precise model of the target, and has a wider range of applications in the field of spacecraft measurement. BRIEF DESCRIPTION OF THE DRAWINGS
[0024] Figure 1 The present invention is an overall flow chart of a method for measuring relative posture of non-cooperative spacecraft with unknown precise model.
[0025] Figure 2 It is a structural diagram of the monocular relative posture measurement network model of the present invention.
[0026] Figure 3 It is a schematic diagram of the space target detection effect on the test sample according to the embodiment of the present invention.
[0027] Figure 4 It is a schematic diagram of the relative posture measurement effect of an embodiment of the present invention on a test sample. DETAILED DESCRIPTION
[0028] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.
[0029] In one embodiment, the present invention proposes a method for measuring relative pose of non-cooperative spacecraft with unknown accurate model, such as Figure 1 As shown, it is mainly divided into two parts: offline training stage and online measurement stage. The specific implementation steps are as follows.
[0030] 1. Offline training phase.
[0031] The offline training phase mainly uses three datasets, namely SwissCube dataset, SPEED+ dataset and self-made dataset, to train the spatial target detection network model based on YOLOv10 and the monocular relative pose measurement network model. First, the spatial target detection network model based on YOLOv10 is trained using the images, spatial target category labels and spatial target bounding box labels of the three datasets; then the trained spatial target detection network model is used to detect the dataset images and extract the corresponding target images; then the extracted target images and the corresponding relative pose labels are used to train the monocular relative pose measurement network model; finally, the trained model is tested, and the training parameters are continuously adjusted according to the test results to optimize the network structure so that the final model meets the actual needs.
[0032] In this embodiment, a special structure of monocular relative pose measurement network model is designed. The specific structure of the network model is as follows: Figure 2 As shown in the figure, it mainly includes three parts: the backbone network, the relative posture prediction subnetwork and the relative position prediction subnetwork. The relative posture prediction subnetwork contains 2 convolutional layers, 5 fully connected layers, 1 Softmax layer and 1 quaternion normalization layer, and the relative position prediction subnetwork contains 1 convolutional layer and 6 fully connected layers. Experiments have found that this model has the best detection effect.
[0033] 2. Online measurement phase.
[0034] The online detection stage mainly applies the space target detection network model based on YOLOv10 and the monocular relative pose measurement network model trained in the offline part to the relative pose measurement of non-cooperative spacecraft. First, use a monocular camera to shoot the detection area to obtain the image of the detection area; secondly, use the trained space target detection network model based on YOLOv10 to detect the image of the detection area to obtain the category of the target and its position in the image; again, according to the detection results of the network model, combine the prior information to screen the target and extract the required target image; finally, use the trained monocular relative pose measurement network model to measure the extracted target image, and obtain the relative pose quaternion of the target and the three-axis relative position of the target through the relative pose prediction subnetwork and relative position prediction subnetwork respectively.
[0035] In this embodiment, the camera parameters of the monocular camera should be consistent with the camera parameters used in the offline training stage.
[0036] In this embodiment, the relative posture prediction subnetwork in the monocular relative posture measurement network model includes two branches: a posture weight distribution branch and an error quaternion branch. In the testing phase, the quaternion weighted average algorithm is first used to fit the preliminary posture quaternion by using the posture weight distribution predicted by the posture weight distribution branch and a pre-set discrete quaternion array, and then the predicted error quaternion and the preliminary posture quaternion are used to obtain the final target relative posture quaternion through quaternion multiplication.
[0037] Next, in order to illustrate the effectiveness of the method of this embodiment, a relative position and attitude measurement experiment of a non-cooperative spacecraft with an unknown precise model is completed as follows.
[0038] (1) Experimental setup.
[0039] Using the Tensorflow neural network learning framework and Anaconda4.7.10's Spyder (Python3.9) as the simulation platform, the computer is configured with an I9-12900H 2.5GHz CPU, 32G memory, and an NVIDIA RTX3060 GPU. Using Numpy, OpenCV-Python, Pytorch and other libraries, a YOLOv10-based spatial target detection network model and a self-designed monocular relative pose measurement network model were built.
[0040] Three datasets, including the SwissCube dataset, the SPEED+ dataset, and the self-made dataset, are used to perform offline training and online testing on the two network models of this embodiment. In the SwissCube dataset, the training set contains 30,356 images, the validation set contains 4,331 images, and the test set contains 8,522 images. In the SPEED+ dataset, the training set contains 47,966 synthetic images, the validation set contains 11,994 synthetic images, and the test set contains 6,740 real images under lightbox lighting conditions and 2,791 real images under sunlamp lighting conditions. In the self-made dataset, the training set contains 13,824 images, the validation set contains 2,744 images, and the test set contains 5,100 real images.
[0041] (2) Experimental results analysis
[0042] In order to verify the effectiveness of the method of this embodiment, mAP, axis angle error and Euclidean distance error are used as accuracy evaluation criteria to quantitatively analyze the results of space target detection, relative attitude measurement and relative position measurement. The calculation method of each evaluation criterion is as follows:
[0043] For the target detection evaluation standard mAP, its calculation formula is shown in formula 1-4:
[0044]
[0045] Among them, TP represents the number of correctly predicted positive samples, FP represents the number of incorrectly predicted negative samples, FN represents the number of incorrectly predicted positive samples, n represents the number of sampling points, and K represents the number of sample categories.
[0046] The evaluation standard for relative attitude is the axis angle error E a , and its calculation formula is as follows:
[0047]
[0048] in, represents the relative pose of the ith prediction, represents the actual relative posture of the ith object.
[0049] For the evaluation standard of relative position, the Euclidean distance error E p , and its calculation formula is as follows:
[0050]
[0051] in, represents the relative position of the i-th prediction, Indicates the actual relative position of the ith item.
[0052] The method of this embodiment is used to detect the test sets of three data sets. The effect of the spatial object detection network based on YOLOv10 is shown as follows: Figure 3 As shown in the figure, the colored labels in the figure represent the category names to which they belong, and the numbers after the labels represent the confidence of the network model in judging that it belongs to this category; the effect of the monocular relative pose measurement network is shown in Figure 4 As shown in the figure, the red arrow indicates the predicted three-axis attitude angle, and the green origin at the origin of the arrow indicates the predicted relative position. The quantitative results of the space target detection network based on YOLOv10 are shown in Table 1. It can be seen from the table that the detection accuracy of this method on the test sets of the three data sets is greater than 99%; the quantitative results of the monocular relative pose measurement network are shown in Table 2. It can be seen from the table that the relative attitude accuracy of this method on the test sets of the three data sets is better than 4°, and the relative position accuracy is better than 0.2 meters.
[0053] Table 1 Quantitative results of spatial object detection network based on YOLOv10
[0054] Dataset mAP(%) SwissCube 99.8 SPEED+ 99.5 Homemade dataset 99.4
[0055] Table 2 Quantitative results of monocular relative pose measurement network
[0056] Dataset Relative attitude error (°) Relative position error (m) SwissCube 2.8505 0.1451 SPEED+ 3.5261 0.1672 Homemade dataset 3.7797 0.1734
[0057] In another embodiment, the present invention proposes a computer-readable storage medium storing a computer program, wherein the computer program enables a computer to execute the relative posture measurement method of a non-cooperative spacecraft with an unknown precise model as described in the aforementioned embodiment.
[0058] In another embodiment, the present invention proposes an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method for measuring relative posture of a non-cooperative spacecraft with unknown precise model as described in the aforementioned embodiment is implemented.
[0059] In the embodiments disclosed in the present application, the computer storage medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. The computer storage medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the above. More specific examples of computer storage media can include electrical connections based on one or more lines, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above.
[0060] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed in this application can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.
[0061] The above are only preferred embodiments of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should be regarded as the protection scope of the present invention.
Claims
1. A method for measuring relative position and attitude of non-cooperative spacecraft with unknown precise model, characterized in that: The steps include: Take spacecraft-related image data in a darkroom on the ground and construct a space target detection dataset through manual annotation; The constructed space target detection dataset is used to train the space target detection network model based on YOLOv10; The trained spatial target detection network model is used to extract the target area image from the captured image data, and a relative pose dataset is constructed in combination with the target relative pose label at the time of shooting. A monocular relative pose measurement network model is built based on a convolutional neural network, and the constructed relative pose dataset is used to train the monocular relative pose measurement network model; In actual application scenarios, a monocular camera is used to shoot the detection area and collect image data of the detection area; Use the trained spatial target detection network model to detect the detection area image, obtain the target category and its position in the image, and extract the target image; The extracted target image is measured using the trained monocular relative pose measurement network to obtain the relative pose and relative position of the target.
2. A method for measuring relative position and attitude of non-cooperative spacecraft with unknown precise model as claimed in claim 1, characterized in that: The manual labeling method includes manually labeling space target category labels and space target bounding box labels.
3. The relative position and attitude measurement method of a non-cooperative spacecraft with unknown precise model as claimed in claim 1, characterized in that: The space target detection network model can detect and identify target categories and common components. The target categories include satellites, rocket final stages and spacecraft, and the common components include the body, solar panels, tripods, docking rings, detection payloads and nozzles.
4. The relative position and attitude measurement method of a non-cooperative spacecraft with an unknown accurate model as claimed in claim 1, characterized in that: The monocular relative posture measurement network model includes a backbone network, a relative posture prediction subnetwork and a relative position prediction subnetwork. The output of the relative posture prediction subnetwork is the relative posture quaternion of the target. The target relative posture quaternion is encoded using a discrete quaternion array through a soft classification method. The output of the relative position prediction subnetwork is the three-axis relative position of the target.
5. The method for measuring relative position and attitude of non-cooperative spacecraft with unknown precise model as claimed in claim 4, characterized in that: The relative posture prediction subnetwork includes a posture weight distribution branch and an error quaternion branch. The posture weight distribution branch is responsible for predicting the weight distribution of each quaternion in the discrete quaternion array, and the error quaternion branch is responsible for regressing the error quaternion. The results of the two branches are used to obtain the relative posture quaternion of the target by using the quaternion weighted average algorithm and quaternion multiplication.
6. A method for measuring relative position and attitude of non-cooperative spacecraft with unknown precise model as claimed in claim 4, characterized in that: The relative posture prediction subnetwork includes 2 convolutional layers, 5 fully connected layers, 1 Softmax layer and 1 quaternion normalization layer; the relative position prediction subnetwork includes 1 convolutional layer and 6 fully connected layers.
7. A computer-readable storage medium storing a computer program, characterized in that: The computer program enables a computer to execute the method for measuring relative position and attitude of a non-cooperative spacecraft with unknown precise model as described in any one of claims 1 to 6.
8. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the relative posture measurement method for non-cooperative spacecraft with unknown precise model as described in any one of claims 1 to 6 is implemented.