Spacecraft attitude estimation method based on prior model and ISAR image

Abstract

The invention discloses a spacecraft attitude estimation method based on a prior model and an ISAR image, and relates to the technical field of spacecraft in-orbit attitude estimation, and the method comprises the steps: S1, employing a target detection network YOLOv8 to extract key points from an ISAR (inverse synthetic aperture radar) image; s2, reading a three-dimensional key point on the priori three-dimensional model, and projecting the three-dimensional key point to a two-dimensional imaging plane in a posture adjustment mode to obtain a projection point; s3, calculating an error between the projection point and the key point so as to calculate a loss value through a weighted loss function; and S4, optimizing the function for solving the attitude angle of the spacecraft by adopting a particle swarm optimization (PSO) algorithm to obtain an attitude estimation result of the ISAR image. According to the spacecraft attitude estimation method based on the prior model and the ISAR image, the key points of the ISAR image and the key points of the prior model are matched, accurate attitude estimation can be achieved only through a small number of key points, and the anti-noise performance and estimation robustness are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the technical field of spacecraft on-orbit attitude estimation, and more specifically, to a spacecraft attitude estimation method based on a priori model and ISAR images. Background Art

[0002] Attitude estimation of spacecraft is an important part of ensuring its on-orbit operation safety and executing space missions. Current attitude estimation technology mainly relies on observations from Inverse Synthetic Aperture Radar (ISAR) images. ISAR technology has become the mainstream means of monitoring space targets due to its all-weather and all-time long-distance observation capabilities. Images obtained by ISAR can show the projection images of spacecraft at different angles. However, existing methods have some shortcomings in the attitude estimation process.

[0003] At present, there are two main types of posture estimation methods:

[0004] One of the methods is to estimate the attitude based on the contour features of the spacecraft in the ISAR image sequence. This method extracts the overall contour of the target and matches it with the image sequence sampled by the spacecraft model to determine the target attitude. However, when relying on the contour features of ISAR images for attitude estimation, the contour feature extraction is often not clear enough due to noise, poor image quality, and electromagnetic anisotropy, resulting in huge estimation errors and completely inconsistent with the actual angle, thus affecting the accuracy of attitude estimation. In addition, when using the model to obtain the image sequence, the number of images is huge and the matching time is long.

[0005] Another type of method is to achieve 3D reconstruction of the target based on multi-view image sequences. This method infers the pose of the target by reconstructing its 3D geometric structure. It requires multiple images from different perspectives and is easily affected by perspective changes and noise. That is, this method relies on multi-view image sequences for 3D reconstruction in pose estimation, which may lead to inaccurate estimation results due to the sensitivity of perspective changes and noise.

[0006] In addition, it should be noted that most of the methods in the existing technology have an estimation error of less than 5 degrees for images with clear image structure, which cannot meet the existing technical requirements. Therefore, the above limitations have also prompted people to explore more robust and efficient posture estimation solutions. Summary of the invention

[0007] An object of the present invention is to solve at least the above problems and / or disadvantages and to provide at least the advantages which will be described hereinafter.

[0008] In order to achieve these purposes and other advantages of the present invention, a method for estimating a spacecraft attitude based on a priori models and ISAR images is provided, using key points for matching, comprising:

[0009] S1. Extract key points from ISAR images using the target detection network YOLOv8 network;

[0010] S2. Read the 3D key points K on the prior 3D model of the spacecraft i , and project the three-dimensional key points onto the two-dimensional imaging plane by means of posture adjustment to obtain the projection points;

[0011] S3, the distance error D, slope error S and scale error P between the projection point and the key point are calculated to obtain the weighted loss function L(θ x ,θ y ,θ z )Calculate the loss value:

[0012] L(θ x ,θ y ,θ z )=w1D+w2S+w3P

[0013] In the above formula, w1, w2 and w3 are the weights corresponding to the distance error D, slope error S and proportional error P respectively, θ x ,θ y ,θ z They correspond to the rotation angles of the spacecraft around the x-axis, y-axis, and z-axis respectively;

[0014] S4. The particle swarm optimization (PSO) algorithm is used to optimize the function of solving the spacecraft attitude angle to obtain the attitude estimation result of the ISAR image.

[0015] Preferably, in S1, the prediction output results of the YOLOv8 network for each key point include: the pixel coordinates of the key point and the confidence of the key point, a judgment threshold is defined for the confidence, and when the confidence of the key point exceeds the judgment threshold, it is considered that it can be used for calculation in the distance error D, the slope error S and the proportional error P.

[0016] Preferably, in S2, the posture adjustment is performed by rotating the matrix R to K i The rotation operation is performed to obtain the rotated three-dimensional coordinates K' i , the K' i It is expressed by the following formula:

[0017] K' i =R·K i .

[0018] Preferably, in S3, the distance error D is obtained by the following formula:

[0019] D=∑ i c i ·||k' i -ki ||

[0020] In the above formula, c i is the confidence set of key points, k i is the key point extracted from the ISAR image, k' i For the three-dimensional key point K i The key point set after posture adjustment;

[0021] The slope error S is obtained by the following formula:

[0022]

[0023] In the above formula, t is the combination of the line connecting the edge of the solar panel on the aircraft and the line connecting the two points of the axis, c ij is the average confidence of the two points on the line, is the two-dimensional coordinate of key point i, is the two-dimensional coordinate of key point j;

[0024] The proportional error P is obtained by the following formula:

[0025]

[0026] In the above formula, c ijmn is the average confidence of the four key points i, j, m, and n involved.

[0027] Preferably, in S5, the PSO algorithm calculates the loss value of each particle after each iteration, and calculates the minimum matching loss value L of the current iteration according to the loss value. p Or the minimum matching loss value L in the global search history g Make updates;

[0028] Continue iterating the search until the stop condition is reached, and output the globally optimal pose estimation parameters through the following formula:

[0029]

[0030] The present invention has at least the following beneficial effects:

[0031] Firstly, by matching the key points of the ISAR image with the key points of the prior model, the present invention can achieve accurate attitude estimation relying on only a small number of key points, thus significantly improving the anti-noise performance and the robustness of the estimation;

[0032] Secondly, the present invention can achieve accurate posture estimation on a single image, thereby reducing the dependence on image sequences.

[0033] Other advantages, objectives and features of the present invention will be embodied in part through the following description, and in part will be understood by those skilled in the art through study and practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 It is the overall algorithm flow chart of the present invention;

[0035] Figure 2 This is a schematic diagram of the process of extracting key points of ISAR images using YOLOv8 in the present invention;

[0036] Figure 3 It is a schematic diagram after the ten key points of the prior three-dimensional model are proposed by the method of the present invention;

[0037] Figure 4 is the original ISAR image of the present invention;

[0038] Figure 5 In order to use the method of the present invention Figure 4 Schematic diagram of the image after key point extraction;

[0039] Figure 6 In order to use the method of the present invention Figure 4 A schematic diagram of the projection points connecting the pose estimation results;

[0040] Figure 7 For the contour matching method, Figure 4 Schematic diagram of the best estimation result obtained by matching the original ISAR image with the image sequence;

[0041] Figure 8 It is a schematic diagram of two groups of pictures with different picture contour clarity, respectively using the method of the present invention and the method based on contour matching, after performing posture angle estimation of target posture estimation;

[0042] Fig. 9 The figure is a line diagram showing the comparison of average errors of the method based on target contour matching in the prior art and the method of the present invention after performing attitude angle estimation of target attitude estimation for 10 ISAR images. DETAILED DESCRIPTION

[0043] The present invention is further described in detail below in conjunction with the accompanying drawings so that those skilled in the art can implement the invention with reference to the description.

[0044] The processing steps of the spacecraft attitude estimation method based on ISAR images and prior three-dimensional models in the present invention mainly include:

[0045] (1) Extracting key points of ISAR images. The present invention uses the YOLOv8 network to extract key points from the preprocessed ISAR images to obtain the coordinates and confidence of the key points. Specifically, an advanced target detection network YOLOv8 is used to extract the key points of the spacecraft from the ISAR images. Figure 2 As shown, the YOLOv8 network consists of the following parts:

[0046] Backbone network. Used for feature extraction, it adopts the Darknet53 structure, including basic convolutional units, spatial pyramid pooling module (SPPF) and C2f module.

[0047] Neck network: It adopts the PAN-FPN structure (path aggregation network-feature pyramid network) to achieve the fusion of multiple feature maps of different sizes to enhance the detection effect of multi-scale targets.

[0048] Head output head. The decoupled head structure is adopted to separate the classification and detection tasks, and the anchor-free mechanism is introduced to further improve the detection accuracy.

[0049] Furthermore, in order to extract all the linear structural information of the target, we take the vertices of the two solar panels and the center points of the two ends of the axis as key points. The distribution of the ten key points of the prior 3D model is as follows: Figure 3 shown.

[0050] The size of the input image data is 150×150 pixels, three channels, and the annotation data of the key points uses the YOLO format, which contains the pixel coordinates of the key points on the image. During the training process, the model learns the positional relationship between the target and the key points through a large amount of annotated data. During the prediction process, the model uses the learned knowledge to predict the position of the key points of the input image. The prediction output result of YOLOv8 for the key points contains two parts: (r, d) and c, where r and d represent the pixel coordinates of the key points, and c represents the confidence of the key point. The confidence value ranges from 0 to 1, which is used to measure the reliability of the prediction. The confidence mentioned in this method is the confidence of the corresponding key point in the YOLOv8 prediction result.

[0051] (2) Initialization. Input the key points of the prior 3D model of the initial attitude of the spacecraft and the key point positions of the ISAR image. Assume that the number of particles of the PSO algorithm is J = 100, and the maximum number of iterations is defined as 500. The particle positions are randomly initialized, the speed is initialized to 0, and other parameters are kept as default. The particle positions represent the attitude parameters, that is, the three angles θ x ,θ y ,θ z Define the lower bound of the search space as (0, 0, 0) and the upper bound as (360, 360, 360). Let Lp is the minimum matching loss value of the current iteration, and L g is the minimum matching loss value in the global search history. At the initial moment, L g Set to infinity.

[0052] (3) Posture adjustment. In the process of posture adjustment of the key points of the 3D model, the key points of the model are controlled by a rotation matrix R. Let the marked prior 3D model key points be K i =K1, K2, ..., K 10 , each key point K i Expressed as three-dimensional coordinates (x i ,y i , z i ), in this method, the value range of i is an integer from 1 to 10.

[0053] In order to adjust the posture of the key points of the model, it is necessary to apply the rotation matrix R to rotate the three-dimensional key points. The rotation matrix R consists of three posture angles θ x ,θ y ,θ z The θ that appears in this method x ,θ y ,θ z They correspond to the rotation angles of the spacecraft around the x-axis, y-axis, and z-axis respectively. The rotation matrix R can be expressed by the attitude angle as:

[0054] R=R z (θ z )R y (θ y )R x (θ x ) (1)

[0055] Where R z (θ z ), R y (θ y ), R x (θ x ) is expressed as

[0056]

[0057] The rotation matrix R acts on the three-dimensional key point K i The rotated three-dimensional coordinates K' are obtained. i , expressed as a three-dimensional coordinate (x' i , y' i , z' i ), expressed as:

[0058] K' i =R·K i (3)

[0059] Next, the rotated 3D key point K' i Projected onto the imaging RD plane. The two-dimensional key point k' after projection i The coordinates (x' i , y' i ) is obtained by ignoring the z-axis coordinate

[0060] k' i =(x' i , y' i ) (4)

[0061] Each particle corresponds to a candidate solution for the attitude. For the qth particle, any one of the set particles can adjust the attitude of the key points of the prior 3D model according to its position using formulas (2) and (3). Then, the projection coordinates of the key points under the qth attitude can be obtained according to formula (4). After obtaining the coordinates, they are matched with the key points of the ISAR image.

[0062] (4) Matching loss calculation. Since there is a lot of noise in ISAR images and the target structure is not clear, in this case, YOLOv8 is not absolutely accurate in extracting key points. Therefore, we propose a confidence adjustment strategy to improve the accuracy of pose estimation, as follows:

[0063] (a) We define a threshold threshold = 0.9 for the confidence of the key points of the ISAR image target predicted by YOLOv8. If the confidence of the key point exceeds this threshold, the point can be used for calculation.

[0064] (b) When selecting the optimal pose estimation result, we define the average of the confidences of all key points in the matching parameter as the confidence of this matching parameter.

[0065] According to the above strategy, using two sets of key points, three matching parameters are defined to calculate the matching degree of the two sets of key points, namely, distance error, slope error and scale error. The key points of ISAR image and the key points after attitude adjustment are denoted as k i ={k1, k2, ..., k 10} and k i '={k1',k2',…,k 10 '}, the confidence level is expressed as c i ={c1, c2, ..., c 10 Among them, the ISAR image key points with confidence lower than the threshold will not be included in the calculation.

[0066] The distance error is used to measure the Euclidean distance between the keypoints after posture adjustment and the target keypoints. It is defined as:

[0067] D=∑ i c i ·||k' i -k i || (5)

[0068] Where ||k' i -k i || represents the Euclidean distance between the i-th pair of key points.

[0069] The slope error is used to measure the difference between the slope of the key point connection line after posture adjustment and the slope of the target connection line. It is defined as:

[0070]

[0071] Where t is the combination of the line connecting the edge of the solar panel and the line connecting the two points of the axis. ij is the average confidence of the two points on the line, where j ranges from 1 to 10.

[0072] The ratio error is used to measure the difference between the length ratio of the key point connection after posture adjustment and the length ratio of the target connection. It is defined as:

[0073]

[0074] c ijmn is the average confidence of the four points involved, where j, m, and n range from 1 to 10.

[0075] The weighted sum of the above errors constitutes the matching loss function:

[0076] L(θ x ,θ y ,θ z )=w1D+w2S+w3P (8)

[0077] Where w1, w2 and w3 are the weights applied to the distance error, slope error and scale error respectively. According to the experiment, these weights are defined as 0.1, 5 and 1 respectively, because the slope error has a greater impact on the pose estimation result, while the distance error has a smaller impact.

[0078] This paper uses the particle swarm optimization (PSO) algorithm for parameter optimization. First, the particle swarm is initialized and the initial position is randomly generated for each particle, representing the possible attitude solution of the spacecraft. The particle position and velocity are updated, and the current position of the particle is adjusted according to the particle's historical optimal position and the global optimal position. The loss value is calculated. After each iteration, the loss value of each particle is calculated, and the optimal solution is updated according to the loss value. The search is iterated until the stop condition is reached, and the global optimal attitude parameters are output. The attitude estimation is obtained by solving the following minimization problem

[0079]

[0080] Using the projection coordinates of the key points of the qth particle and the pixel coordinates extracted from the ISAR image, the distance error, slope error, and scale error are calculated according to formulas (5), (6), and (7), and then summed up according to formula (9) to obtain the matching loss value of the current particle.

[0081] (5) Calculation of minimum matching loss value. The matching loss value of each particle is its fitness value. If the fitness value of the current particle is better than the minimum fitness value in the current iteration, then L is updated. p If the current loss value is better than the global historical minimum fitness value, then update L g Then, update the particle's velocity and position, and record the current particle position as the possible optimal solution.

[0082] (6) Termination judgment. Continue to iterate and update the optimal fitness value, global optimal fitness value and position of the particle. If the maximum number of iterations is reached, output the optimal attitude parameters and terminate the calculation. Otherwise, update the status of all particles and return to attitude adjustment to continue iteration. Finally, the attitude estimation result of the ISAR image is obtained.

[0083] Example 1

[0084] The spacecraft attitude estimation method based on ISAR images and prior three-dimensional models described in this embodiment is applied to actual spacecraft attitude estimation tasks. If there is an ISAR image data set of a spacecraft and a three-dimensional model B of the spacecraft, it is necessary to accurately measure the attitude of the spacecraft in the ISAR image, and the traditional attitude estimation method cannot meet the accuracy requirements due to noise interference and viewing angle limitations. Therefore, the method of the present invention is applied to achieve accurate estimation of the spacecraft attitude by matching the key points in the ISAR image with the key points of the prior three-dimensional model.

[0085] Specifically, Figure 4 The attitude angle parameter of the spacecraft in the ISAR image is (θ x ,θ y ,θ z )=(40.55°, -39.53°, 105.12°), the method of the present invention is used for Figure 4 The method for spacecraft attitude estimation shown is divided into the following steps:

[0086] Step 1: Use the YOLOv8 network to extract key points from the preprocessed ISAR image. The input data size is 150×150 pixels and three channels. The output data includes the coordinates of ten two-dimensional key points (10×2 dimensions) and their corresponding confidence levels (10×1 dimensions). The training set and validation set are divided in an 8:2 ratio. The total training cycle is 300. Other parameters are kept at the default settings. In this step, by preprocessing the ISAR image and then using YOLOv8 to extract key points, the following can be obtained: Figure 5 The ten extracted key point positions are shown in FIG.

[0087] Step 2: Read the coordinates of ten 3D key points of the prior 3D model B of the spacecraft. Apply the rotation matrix to adjust the attitude of the 3D key points of the model, and the adjusted 3D key points are projected onto the 2D imaging plane.

[0088] Step 4: Calculate the error between the projected point and the key points extracted from the ISAR image, including distance error, slope error, and scale error. These errors are combined into a weighted loss function.

[0089] Step 5: Use the particle swarm optimization (PSO) algorithm to iteratively optimize the posture parameters to minimize the loss function.

[0090] Step 6: Output the optimal attitude parameters and generate the attitude estimation result of spacecraft B. The estimation result of this method is (θ x ,θ y ,θ z )=(40.41°, -38.79°, 105.66°), and Figure 6 The middle line is the line connected according to the projection points of the 3D key points after rotating the 3D key points to obtain the minimum loss value.

[0091] In order to compare the effect of this example, Figure 4For a spacecraft in an ISAR image, the attitude estimation is performed using a contour matching-based method in the prior art. In the attitude estimation based on contour matching, the overall contour of the target is first extracted from the ISAR image. The accuracy of the contour directly determines the size of the subsequent attitude estimation error. The target contour in the ISAR image usually includes the boundary line of the object. Contour extraction usually uses edge detection algorithms, such as the Canny operator or the Sobel operator, which can identify obvious edges in the image and connect them to form a closed contour. Next, the viewpoint space of the target is discretized and sampled. The viewpoint space represents the set of perspectives from which the observer observes the target from different angles (i.e., different postures). In order to represent the attitude of the target as discrete sampling points, several key attitude angles are usually selected, and a sampling image sequence is generated based on these angles. Each image in these image sequences represents a two-dimensional projection of the target in a specific posture. Subsequently, the extracted target contour is matched with the image sequence corresponding to the discrete sampling points. A contour matching algorithm, such as the Hausdorff distance algorithm or the ICP (iterative closest point) algorithm, is used. These algorithms determine the similarity between two sets of points by calculating the geometric distance between them. During the matching process, the algorithm continuously adjusts the pose angle of the model to minimize the matching error (i.e., the distance between the two sets of points). Its goal is to find the pose angle that minimizes the matching error. When the algorithm converges to the minimum error, the corresponding pose angle is the estimated pose of the target. Figure 4 The estimated result is (θ x ,θ y ,θ z )=(41.47°, -35.94°, 108.34°), we get Figure 7 The best estimation result obtained by matching the ISAR image with the image sequence is shown. By comparing the difference between the estimation result of the present invention and the original image, it can be seen that the accuracy of the estimation result of the present invention is significantly better than that of the prior art method.

[0092] Example 2

[0093] like Figure 8 As shown, the method of the present invention and the method based on contour matching in the existing method are used to make an aircraft attitude comparison result to analyze the benefits of the existing technology and the present invention. Figure 8 Figures (a) and (d) are two original ISAR images, Figures (b) and (e) are schematic diagrams of the results of attitude estimation for Figures (a) and (d) using this method, and Figures (c) and (f) are schematic diagrams of the results of attitude estimation based on target contour matching using the existing technology. Figure 8 Take the first group of pictures in Figure (a) as an example. When the picture outline is clear enough, the estimation results of the two methods, Figure (b) and Figure (c), have small errors. Figure 8From the comparison of the estimation results in the second group of pictures, it can be seen that when the image structure is not clear enough, the method based on target contour matching will have estimation errors in (f), which is completely deviated from the actual angle, while the method in (e) can still provide a relatively accurate estimation result in this case.

[0094] It can be seen from this embodiment that compared with the method based on contour matching, the method of the present invention makes the spacecraft attitude estimation more accurate. By matching the key points of the ISAR image with the prior model, the dependence on complete contour extraction is reduced. Even when the image contour is not clear, high-precision attitude estimation can still be achieved.

[0095] Example 3

[0096] like Fig. 9 As shown in the line graph, the average error comparison of attitude angle estimation based on target contour matching and this method for target attitude estimation for 10 ISAR images is shown. Fig. 9 It can be seen that this method has y ,θ x and θ z The average estimation error of θ does not exceed 2 degrees, while the method based on target contour matching x and θ z The estimation error is relatively high, about 3 degrees. Compared with the most excellent and mature aircraft attitude estimation method currently, the present invention has better performance.

[0097] It can be seen from this embodiment that, compared with the traditional multi-view 3D reconstruction method, the present invention can achieve high-precision attitude estimation on a single ISAR image without the need for image sequences, and also obtain a smaller estimation error under the same conditions.

[0098] The above solution is only an illustration of a preferred embodiment, but is not limited thereto. When implementing the present invention, appropriate replacement and / or modification can be performed according to user needs.

[0099] Although the embodiments of the present invention have been disclosed above, they are not limited to the applications listed in the specification and the embodiments. It can be fully applied to various fields suitable for the present invention. For those familiar with the art, additional modifications can be easily realized. Therefore, without departing from the general concept defined by the claims and equivalent scope, the present invention is not limited to the specific details and the illustrations shown and described here.

Claims

1. A method for estimating spacecraft attitude based on a priori model and ISAR images, characterized in that: include: S1. Extract key points from ISAR images using the target detection network YOLOv8 network; S2. Read the 3D key points K on the prior 3D model of the spacecraft i , and project the three-dimensional key points onto the two-dimensional imaging plane by means of posture adjustment to obtain the projection points; S3, the distance error D, slope error S and scale error P between the projection point and the key point are calculated to obtain the weighted loss function L(θ x ,θ y ,θ z )Calculate the loss value: L(θ x ,the y ,the z )=w1D+w2S+w3P In the above formula, w1, w2 and w3 are the weights corresponding to the distance error D, slope error S and proportional error P respectively, θ x ,θ y ,θ z They correspond to the rotation angles of the spacecraft around the x-axis, y-axis, and z-axis respectively; S4. The particle swarm optimization (PSO) algorithm is used to optimize the function of solving the spacecraft attitude angle to obtain the attitude estimation result of the ISAR image.

2. The method for estimating a spacecraft attitude based on a priori model and ISAR images as claimed in claim 1, characterized in that: In S1, the prediction output results of the YOLOv8 network for each key point include: the pixel coordinates of the key point and the confidence of the key point. A judgment threshold is defined for the confidence. When the confidence of the key point exceeds the judgment threshold, it is considered that it can be used for calculation of the distance error D, slope error S and proportion error P.

3. The method for estimating a spacecraft attitude based on a priori model and ISAR images as claimed in claim 1, characterized in that: In S2, the posture adjustment is performed by rotating the matrix R to K i The rotation operation is performed to obtain the rotated three-dimensional coordinate K′ i , the K′ i It is expressed by the following formula: K′ i =R·K i 。 4. The method for estimating a spacecraft attitude based on a priori model and ISAR images as claimed in claim 1, characterized in that: In S3, the distance error D is obtained by the following formula: D=S i c i ·||k′ i -k i || In the above formula, c i is the confidence set of key points, k i is the key point extracted from the ISAR image, k′ i For the three-dimensional key point K i The key point set after posture adjustment; The slope error S is obtained by the following formula: In the above formula, t is the combination of the line connecting the edge of the solar panel on the aircraft and the line connecting the two points of the axis, c ij is the average confidence of the two points on the line, is the two-dimensional coordinate of key point i, is the two-dimensional coordinate of key point j; The proportional error P is obtained by the following formula: In the above formula, c ijmn is the average confidence of the four key points i, j, m, and n involved.

5. The method for estimating a spacecraft attitude based on a priori model and ISAR images as claimed in claim 1, characterized in that: In S5, the PSO algorithm calculates the loss value of each particle after each iteration, and calculates the minimum matching loss value L of the current iteration based on the loss value. p Or the minimum matching loss value L in the global search history g Make updates; Continue iterating the search until the stop condition is reached, and output the global optimal final posture estimation parameters through the following formula:

Images