Target pose estimation method based on spatial attention and intelligent particle swarm

Abstract

This invention provides a target attitude estimation method based on spatial attention and intelligent particle swarm optimization, comprising the following steps: acquiring a space-based optical image of a space target; using a spatial attention algorithm to perform feature extraction and keypoint detection to obtain the position information of the target's keypoints; combining the 3D model of the space target and the camera's intrinsic parameter matrix to establish a 2D-3D position correspondence between the target's keypoints; based on the 2D-3D position correspondence, using a PnP algorithm to calculate the initial pose of the space target; combining an optimized particle swarm optimization algorithm and an LM algorithm to obtain an attitude solution algorithm; using the attitude solution algorithm, optimizing the initial pose through iterative calculation to obtain the optimal pose. This invention can reduce keypoint detection errors caused by symmetrical features and further eliminate attitude estimation errors caused by outliers using the attitude solution algorithm, effectively improving the accuracy of space target attitude estimation.

Description

Technical Field

[0001] This invention relates to the field of spatial target attitude estimation technology, and in particular to a target attitude estimation method based on spatial attention and intelligent particle swarm optimization. Background Technology

[0002] In recent years, the number of space debris has increased dramatically, posing a significant threat to the space safety of near-Earth spacecraft. To accomplish the task of clearing active space debris, it is first necessary to capture spacecraft approaching the debris to be captured. After determining the motion state of the debris, a robotic arm or rope-net mechanism is used to capture it, and then the target is moved into a discard orbit or recovered. To accomplish this task, it is crucial to measure the motion parameters such as position and attitude of the space target, as well as the geometric parameters such as the morphology of its surface components. Accurate and robust estimation of the space target's attitude is an essential part of this process.

[0003] However, the characteristics of space targets observed by space-based observations are very easily affected by changes in target size and observation angle. Furthermore, some space targets have strong symmetry, meaning that when a space target rotates around a specific axis, its features on the image are almost identical. At the same time, the imaging quality of space-based observations is poor, the image resolution is low, and the surface texture information of space targets is weak. These conditions pose significant challenges to existing methods.

[0004] Currently, among tasks involving pose estimation of spatial targets based on visible light images, the most mature and generally best-performing methods are based on keypoint extraction. Traditional keypoint detection methods use manually designed feature descriptors such as SIFT, Hog, and SURF to accurately locate the semantic components of targets in images. However, these manually designed features are highly susceptible to changes in target scale, rotation, illumination, and viewing angle, resulting in poor robustness. The ambient lighting of spatial targets varies drastically, and the image quality is poor with low resolution. Furthermore, the surface texture information of spatial targets is weak, and target features are highly variable, thus severely limiting the performance of traditional methods.

[0005] Compared to traditional manual features, deep learning methods have achieved better results in recent years. This is because they can extract deep and robust semantically significant features of spatial targets through complex network structure design. The features extracted by deep networks can better adapt to complex imaging conditions and effectively improve the stability and accuracy of keypoint detection, thereby improving the results of spatial target pose estimation. However, the following problems exist in the current use of deep networks for spatial target pose estimation: 1) Spatial targets have strong symmetry, and certain specific pose pairs have almost identical features on the image plane, making it impossible for the network to distinguish the differences between different keypoints; 2) The surface of spatial targets often uses weak texture materials, with little surface texture information; 3) Space-based detection images are severely degraded due to defocus, platform jitter, motion blur, etc. These reasons can lead to keypoint detection errors, large errors, or even failure to converge in deep learning networks, ultimately seriously affecting the results of spatial target pose estimation. Summary of the Invention

[0006] To overcome the shortcomings of existing technologies, the purpose of this invention is to provide a target attitude estimation method based on spatial attention and intelligent particle swarm optimization. This method can reduce key point detection errors caused by symmetry features and further eliminate attitude estimation errors caused by outliers by using attitude solution algorithms, thereby effectively improving the accuracy of spatial target attitude estimation.

[0007] To achieve the above objectives, the present invention provides the following solution: a target pose estimation method based on spatial attention and intelligent particle swarm optimization, comprising the following steps:

[0008] Acquire space-based optical images of space targets, and use spatial attention algorithms to perform feature extraction and key point detection to obtain the location information of key points of the targets;

[0009] By combining the 3D model of the spatial target and the intrinsic parameter matrix of the camera, a 2D-3D positional correspondence relationship of the target key points is established. Based on the 2D-3D positional correspondence relationship, the initial pose of the spatial target is calculated using the PnP algorithm.

[0010] By combining the optimized particle swarm optimization algorithm and the LM algorithm, an attitude solution algorithm is obtained. Using the attitude solution algorithm, the initial pose is optimized through iterative calculation to obtain the optimal pose.

[0011] Optionally, acquire space-based optical images of the space target, and use a spatial attention algorithm to perform feature extraction and keypoint detection to obtain the location information of the target's key points, including:

[0012] Obtain the original feature map of a specific location in the space-based optical image, and flatten the feature dimensions of the original feature to obtain the flattened feature map;

[0013] The self-attention calculation result is obtained by using a spatial attention algorithm to perform self-attention calculation on the flattened feature map;

[0014] The self-attention calculation result is subjected to an inverse flattening operation to obtain an attention feature map. The attention feature map is then superimposed onto the original feature map pixel by pixel to obtain an enhanced feature map.

[0015] Optionally, a spatial attention algorithm is used to perform self-attention calculation on the flattened feature map to obtain the self-attention calculation result. This includes dividing the flattened feature map into different batches and different channels, and for each batch and each channel, performing query vector calculation, index vector calculation, and response vector calculation to obtain the query vector weight matrix, index vector weight matrix, and response vector weight matrix, thus completing the self-attention calculation. The self-attention calculation formula is:

[0016]

[0017] Where, d k Let Q be the size of the hidden layer dimension, Q, K, and V be the query input vector, index input vector, and response input vector, respectively, and T be the matrix transpose symbol.

[0018] Optionally, by combining the 3D model of the spatial target and the intrinsic parameter matrix of the camera, a 2D-3D positional correspondence relationship of the target's key points is established. Based on the 2D-3D positional correspondence relationship, the initial pose of the spatial target is calculated using the PnP algorithm, including:

[0019] Define the 3D point coordinates, 3D point homogeneous coordinates, projected point coordinates, projected point homogeneous coordinates, and camera intrinsic parameter matrix of the spatial target;

[0020] The perspective projection model of the camera's rotation and translation matrices is expanded into a system of equations and written in matrix form. The SVD algorithm is used to solve the matrix form to obtain the predicted values ​​of the rotation and translation matrices, thus completing the calculation of the initial pose. The rotation parameters of the rotation matrix are in quadruplet form.

[0021] Optionally, by combining the optimized particle swarm optimization algorithm and the LM algorithm, an attitude solution algorithm is obtained. Using this algorithm, the initial pose is optimized through iterative calculation to obtain the optimal pose, including:

[0022] Input the corresponding 2D-3D point group and initial attitude values, and define the inertia factor, maximum number of iterations, individual learning factor of rotation, social learning factor of rotation, individual learning factor of translation, social learning factor of translation, fitness function and population size;

[0023] Initialize the rotational particle swarm parameters and the translational particle swarm parameters, and input the initialized particle swarm parameters into the particle swarm. Randomly initialize the rotation-translation value and the rate of change of the rotation-translation value of each particle near the initial attitude value.

[0024] The optimal pose is obtained by iteratively calculating the pose solution algorithm.

[0025] Optionally, the iterative calculation process of the attitude solution algorithm includes:

[0026] Start the iteration. When the number of iterations is less than the preset maximum number of iterations, calculate the fitness value of the particle based on the current attitude value, and update the individual optimal position of the particle and the global optimal position of the population.

[0027] Based on the updated optimal position of the particle and the global optimal position of the population, update the rotation-translation velocity vector and position vector of the particle, and then perform the next iteration;

[0028] When the maximum number of iterations is reached, the iteration ends and the optimal pose is obtained.

[0029] This invention discloses the following technical effects by providing a target pose estimation method based on spatial attention and intelligent particle swarm optimization:

[0030] 1. This invention establishes spatial relationships between existing features through a spatial attention calculation module, solving the problem of attitude differentiation estimation at symmetrical angles, reducing the possibility of key point detection errors caused by symmetrical features, and improving the accuracy of attitude calculation.

[0031] 2. This invention designs an attitude optimization algorithm that combines the particle swarm optimization algorithm and the LM algorithm. This algorithm can effectively reduce the possibility of the attitude solution falling into a local optimum and is more inclined to find the global optimum. When the initial attitude value is estimated incorrectly, it can be optimized to the correct result, thus solving the problem of attitude estimation error and reducing the attitude estimation error.

[0032] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0033] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0034] Figure 1 This is a schematic diagram of the method flow provided in an embodiment of the present invention;

[0035] Figure 2 This is a schematic diagram of the spatial target attitude estimation process provided in an embodiment of the present invention;

[0036] Figure 3 A schematic diagram of the self-attention calculation process provided in an embodiment of the present invention;

[0037] Figure 4 This is a schematic diagram of the method flow for feature extraction and key point regression provided in an embodiment of the present invention. Detailed Implementation

[0038] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0039] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0040] like Figure 1-2 As shown, this invention provides a target pose estimation method based on spatial attention and intelligent particle swarm optimization, comprising the following steps:

[0041] 1. For example Figure 3-4 As shown, space-based optical images of space targets are acquired, and spatial attention algorithms are used for feature extraction and keypoint detection to obtain the location information of key points on the targets. This includes:

[0042] 1.1 Obtain the original feature map at a specific location in the space-based optical image, and flatten the feature dimensions B, C, H, W of the original feature map to obtain the flattened feature map. After flattening, the dimensions become B, C, hidden_dim, where hidden_dim is the dimension of the hidden layer.

[0043] 1.2 The flattened feature map is subjected to self-attention calculation using a spatial attention algorithm to obtain the self-attention calculation result. This includes:

[0044] The flattened feature map is divided into different batches and different channels. For each batch and each channel, the data is a 1*hidden_dim vector, defined as X. b,c .

[0045] Self-attention computation requires three input vectors (query vector, key vector, and value vector), all of which come from the input token X. b,c .

[0046] By performing query vector calculation, index vector calculation, and answer vector calculation, we obtain the query vector weight matrix, index vector weight matrix, and answer vector weight matrix, denoted as W, respectively. Q W K W V All are learnable parameters of dimension 1*hidden_dim; the formula for calculating self-attention is:

[0047]

[0048] Where, d k Let Q be the size of the hidden layer dimension. The calculated Attention(Q,K,V) dimension remains unchanged from the input, both being 1*hidden_dim. After calculating for each batch and each channel, attention vectors of dimensions B, C, and hidden_dim can be obtained. Q, K, and V are the query input vector, index input vector, and response input vector, respectively, and T is the matrix transpose.

[0049] 1.3 Perform an inverse flattening operation on the self-attention calculation result, reconstruct the dimensions to B, C, H, W to obtain an attention feature map, and superimpose the attention feature map onto the original feature map in a pixel-by-pixel manner to obtain an enhanced feature map.

[0050] In summary, this invention adds a spatial attention calculation module to two locations in the network, corresponding to the high-resolution feature map and the low-resolution feature map of the original network, respectively. The aim is to enhance spatial attention on both high- and low-resolution feature maps. The low-resolution feature map has stronger semantic features, which can better distinguish the features of different key points; the high-resolution feature map has stronger structural features, which can improve the regression accuracy of key point locations.

[0051] 2. For example Figure 4 As shown, by combining the 3D model of the spatial target and the camera's intrinsic parameter matrix, a 2D-3D positional correspondence relationship of the target's key points (9 pairs of key points) is established. Based on this 2D-3D positional correspondence relationship, the initial pose of the spatial target is calculated using the PnP algorithm. This includes:

[0052] Let the coordinates of a 3D point in space be [X w Y w Z w ] T The homogeneous coordinates of a 3D point are represented as [X... w Y w Zw 1] T The coordinates of the projection point are [uv]. T The homogeneous coordinates of the projection point are [uv 1]. T The camera intrinsic parameter matrix is ​​K.

[0053] The perspective projection model of the camera's rotation matrix R and translation matrix t is as follows:

[0054]

[0055] Expanding, we get:

[0056]

[0057] Write it in the form of a system of equations, and eliminate Z. c In matrix form:

[0058]

[0059] The above equation can be written in matrix form: AF = 0

[0060] When N≥6, the least squares solution under the constraint |F|=1 can be obtained. This can be solved using SVD; the last column of the V matrix is ​​the solution.

[0061] F = UDV T

[0062] because:

[0063] F = [KR Kt]

[0064] Therefore, the rotation matrix and translation matrix can be obtained as follows:

[0065]

[0066] The initial value of the spatial target's attitude, i.e., the predicted value of the rotation matrix, can be calculated through the above process. Predicted values ​​of translation matrix This is used as input for subsequent processes. Since the rotation matrix has nine variables, while the other representation of the pose, a quaternion, only has four, the rotation parameters are used as input in quaternion form to reduce the dimensionality of the input and improve the robustness of the algorithm. The formula for converting the rotation matrix to a quaternion is as follows:

[0067]

[0068] 3. Combining the optimized particle swarm optimization algorithm and the LM algorithm, a pose solution algorithm is obtained. Using this algorithm, the initial pose is optimized through iterative calculation to obtain the optimal pose. This includes:

[0069] 3.1 Input the corresponding 2D-3D point group and initial attitude values, and define the inertia factor, maximum number of iterations, individual learning factor of rotation, social learning factor of rotation, individual learning factor of translation, social learning factor of translation, fitness function and population size.

[0070] 3.2 Initialize the rotational particle swarm parameters and translational particle swarm parameters, and input the initialized particle swarm parameters into the particle swarm. Randomly initialize the rotation-translation value and the rate of change of the rotation-translation value of each particle near the initial attitude value.

[0071] 3.3 Perform iterative calculations of the attitude solution algorithm (as shown in Table 1 below) to obtain the optimal pose.

[0072] Table 1. Logical Rules of Attitude Solution Algorithm

[0073]

[0074]

[0075] As shown in Table 1 above, the iterative calculation process of the attitude solution algorithm is described in words:

[0076] Start the iteration. When the number of iterations is less than the preset maximum number of iterations, calculate the particle's fitness value (reprojection error value) based on the current attitude value, and update the individual optimal position of the particle and the global optimal position of the population.

[0077] Based on the updated optimal position of the particle and the global optimal position of the population, update the rotation-translation velocity vector and position vector of the particle, and then perform the next iteration;

[0078] When the maximum number of iterations is reached, the iteration ends and the optimal pose is obtained.

[0079] Therefore, by providing a target attitude estimation method based on spatial attention and intelligent particle swarm optimization, this invention can reduce key point detection errors caused by symmetry features and further eliminate attitude estimation errors caused by outliers using attitude solution algorithms, thus effectively improving the accuracy of spatial target attitude estimation.

[0080] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0081] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A target pose estimation method based on spatial attention and intelligent particle swarm optimization, characterized in that, Includes the following steps: Acquire space-based optical images of space targets, and use spatial attention algorithms to perform feature extraction and key point detection to obtain the location information of key points of the targets; By combining the 3D model of the spatial target and the intrinsic parameter matrix of the camera, a 2D-3D positional correspondence relationship of the target key points is established. Based on the 2D-3D positional correspondence relationship, the initial pose of the spatial target is calculated using the PnP algorithm. By combining the optimized particle swarm optimization algorithm and the LM algorithm, an attitude solution algorithm is obtained. Using this algorithm, the initial pose is optimized through iterative calculation to obtain the optimal pose; including: Input the corresponding 2D-3D point group and initial attitude values, and define the inertia factor, maximum number of iterations, individual learning factor of rotation, social learning factor of rotation, individual learning factor of translation, social learning factor of translation, fitness function and population size; Initialize the rotational particle swarm parameters and the translational particle swarm parameters, and input the initialized particle swarm parameters into the particle swarm. Randomly initialize the rotation-translation value and the rate of change of the rotation-translation value of each particle near the initial attitude value. The optimal pose is obtained by iterative calculation of the pose solution algorithm.

2. The target pose estimation method based on spatial attention and intelligent particle swarm optimization according to claim 1, characterized in that, Acquire space-based optical images of space targets, and use spatial attention algorithms to perform feature extraction and keypoint detection to obtain the location information of key points of the targets, including: Obtain the original feature map of a specific location in the space-based optical image, and flatten the feature dimensions of the original feature to obtain the flattened feature map; The self-attention calculation result is obtained by using a spatial attention algorithm to perform self-attention calculation on the flattened feature map; The self-attention calculation result is subjected to an inverse flattening operation to obtain an attention feature map. The attention feature map is then superimposed onto the original feature map pixel by pixel to obtain an enhanced feature map.

3. The target pose estimation method based on spatial attention and intelligent particle swarm optimization according to claim 2, characterized in that, The self-attention calculation of the flattened feature map is performed using a spatial attention algorithm to obtain the self-attention calculation result. This includes dividing the flattened feature map into different batches and different channels, and for each batch and each channel, calculating the query vector, index vector, and response vector to obtain the query vector weight matrix, index vector weight matrix, and response vector weight matrix, thus completing the self-attention calculation. The self-attention calculation formula is: in, The size of the hidden layer dimension. Q , K , V These are the query input vector, the index input vector, and the response input vector, respectively. T This is the matrix transpose symbol.

4. The target pose estimation method based on spatial attention and intelligent particle swarm optimization according to claim 3, characterized in that, By combining the 3D model of the spatial target and the intrinsic parameter matrix of the camera, a 2D-3D positional correspondence is established for the target's key points. Based on this 2D-3D positional correspondence, the initial pose of the spatial target is calculated using the PnP algorithm, including: Define the 3D point coordinates, 3D point homogeneous coordinates, projected point coordinates, projected point homogeneous coordinates, and camera intrinsic parameter matrix of the spatial target; The perspective projection model of the camera's rotation and translation matrices is expanded into a system of equations and written in matrix form. The SVD algorithm is used to solve the matrix form to obtain the predicted values ​​of the rotation and translation matrices, thus completing the calculation of the initial pose. The rotation parameters of the rotation matrix are in quadruplet form.

5. The target pose estimation method based on spatial attention and intelligent particle swarm optimization according to claim 4, characterized in that, The iterative calculation process of the attitude solution algorithm includes: Start the iteration. When the number of iterations is less than the preset maximum number of iterations, calculate the fitness value of the particle based on the current attitude value, and update the individual optimal position of the particle and the global optimal position of the population. Based on the updated optimal position of the particle and the global optimal position of the population, update the rotation-translation velocity vector and position vector of the particle, and then perform the next iteration; When the maximum number of iterations is reached, the iteration ends and the optimal pose is obtained.

Images