A space robot on-orbit capturing method based on analog binocular vision measurement

Abstract

The application discloses a kind of space robot in-orbit capture methods based on analog binocular vision measurement, in the application, the feature point of cubic target module is identified to analog binocular vision camera in simulation environment, the pose of target module is estimated based on fitting plane method, and the observation accuracy is improved;Then, the motion state of target module is corrected based on extended Kalman filtering method according to the observed target module pose under multiple frames of images, and then the motion trajectory of target module is fitted, the motion prediction of target is realized;Finally, the space robot trajectory planning method based on reinforcement learning is designed, to improve the authenticity of training result with accurate space robot model as training environment, the reward function that can guide the end point of robot to reach the desired capture point is designed, the training process is accelerated, and finally the capture of target module is realized.

Description

Technical Field

[0001] This invention relates to the field of on-orbit capture technology for space robots, and more particularly to an on-orbit capture method for space robots based on simulated binocular vision measurement. Background Technology

[0002] With the development of aerospace technology, space missions such as building large-aperture space telescopes and large space power stations have gradually gained attention from researchers both domestically and internationally. However, due to the limitations of current launch vehicle capabilities, it is still impossible to launch a single large structure. To address this issue, large space structures are developing towards modularization. After launching space modules with various independent functions into orbit, these modules are then assembled in orbit using space robots to form a large structure with specific functions. This modular assembly method also has other advantages, such as the ability to directly replace modules when they malfunction, improving the fault tolerance and system reliability of large space structures.

[0003] Currently, existing space robots have significant errors in their ability to capture targets in orbit, failing to achieve the desired target pose information measurement. Furthermore, their perception of the external environment is inadequate, making it impossible to correct the target's motion state, estimate its true motion state, or predict its future motion. Moreover, they cannot use reinforcement learning-based space robot trajectory planning methods with accurate space robot models as the training environment to improve the realism of the training results. Summary of the Invention

[0004] The purpose of this invention is to provide an on-orbit capture method for space robots based on simulated binocular vision measurement in order to solve the above-mentioned problems.

[0005] To achieve the above objectives, the present invention adopts the following technical solution:

[0006] A method for on-orbit capture of a space robot based on simulated binocular vision measurement includes the following steps:

[0007] S1. Based on a simulated binocular vision camera, the feature point information of the target module is obtained, the three-dimensional position markers of the feature points are obtained through a feature point matching algorithm, and then the position and orientation information of the target module are calculated according to the fitting plane method.

[0008] S2. Based on the target identification process in step S1, the target module is continuously observed and tracked. The motion state of the target is solved based on the EKF method to obtain the filtered estimation result of the target motion. Then, the motion trajectory of the target module is fitted to achieve the motion prediction of the target.

[0009] S3. A trajectory planning system for the space robot is trained based on reinforcement learning methods. The predicted target pose information is input into the trajectory planning system of the space robot to achieve rapid acquisition of the target module.

[0010] Preferably, the method for acquiring feature point information of the target module based on a simulated binocular vision camera in step S1 includes the following steps:

[0011] S11. Determine the size, color, and outward normal vector parameters of the cube target module in the simulation environment, as the observation object for the subsequent simulation of the binocular vision camera;

[0012] S12. Load the cube target module from step S11. Since the projection of the target in the camera is not a parallel projection, but converges at the camera's optical center, when simulating the image of the target, it is necessary to convert the coordinates of each vertex of the target in inertial space into pixel positions in the image. It is also necessary to judge and filter the planes that can be seen in the image, and finally retain the planes that can actually be seen in the image.

[0013] S13. The minimum eigenvalue algorithm is used to detect corner points in the image, which can detect pixels at edge and corner feature positions in the image;

[0014] S14. Use the pyramid optical flow method to match feature points in the left and right camera views, and solve for the three-dimensional spatial coordinates of all matched feature point pairs.

[0015] S15. For each successfully matched pair of points, color-mark the corresponding image. Centered on the feature point, select a certain pixel range and determine the color of the pixels near the feature point based on their RGB values. Mark the color of the feature point as the color of any pixels that have appeared near it. Based on the geometric features of the cube target, each feature point has at least one and at most three color marks.

[0016] S16. Based on the feature points with color labels, find the plane on which the target surface is located, and then perform three-dimensional state reconstruction of the cube target.

[0017] Preferably, the specific implementation method of step S12 includes the following steps:

[0018] S121. Solve for the coordinates of all vertices on the cube target in both the camera coordinate system and the pixel coordinate system, denoted as X. fc and X fuv ;

[0019] S122. Perform the first image visualization screening, calculate the outward-pointing normal vector of each surface of the target. The outward-pointing normal vector must point towards the camera side; that is, the angle between the outward-pointing normal vector and the Z-axis of the camera coordinate system must be greater than 90° for that plane to be visible in the camera image. In the case of parallel projection, all n out ·z c Planes greater than 0 can all be displayed in the camera image, where n out Let z be the outward normal vector of the plane. c The Z-axis of the camera coordinate system is used, but because the camera uses perspective imaging, the blind spot of the object becomes larger when it is projected. This strengthens the constraint of the plane that can be seen in the camera image, so a second screening is required.

[0020] S123. Perform a second image visualization filter to determine the planes that are actually visible in the camera image. Number the vertices of each plane in a certain order so that the vertices spiral outwards towards the outside of the target. Connect these vertices sequentially on the pixel plane. If the spirals are inwards, the plane can be seen in the camera image. If the spirals are outwards, the back of the plane is visible. For solid targets, the plane is not visible inside the object.

[0021] Preferably, the implementation method of step S16 specifically includes the following steps:

[0022] S161. Define the equation of a certain color plane as:

[0023] ax+by+cz+d=0 (1);

[0024] Extract points with the same color label, perform linear fitting on the plane containing these feature points, and the resulting plane function is:

[0025] f(z) = p 00 +p 10 x +p 01 y (2);

[0026] The relationship between the three coefficients and the four parameters of the plane equation is as follows:

[0027] [abcd]=[p 10 p 01 -1 p 00 (3);

[0028] S162. Select a feature point detected in the image that does not belong to the plane, and then randomly select one feature point from the feature points belonging to the fitting plane. Establish a vector n1 pointing from the plane to the feature points on other surfaces of the target. Let the outward normal vector of the fitted plane be n0, and the angle between the two vectors be a0. When the normal vector of the plane points out of the plane, a0 must be an obtuse angle. When the angle between the solved plane normal vector and the feature point vector is an obtuse angle, the plane parameter is the outward normal vector of the fitted plane. For cases where the angle is not obtuse, take the negative values ​​of the first three parameters of the plane equation.

[0029] S163. After obtaining the outward-pointing normal vectors of multiple planes, the centroid position and attitude angle of the target module are used as variables to design an index function and solve it using an optimization method. The index function is written in the following form:

[0030]

[0031] Where r center Represents the position vector of the centroid of an inertial space cube. Let represent the inertial space coordinates of the i-th feature point, n represent the total number of feature points, and L represent the side length of the target module. The normal vector of the plane passing through the centroid, after the plane of the fit is translated according to the side length of the cube. Let A represent the outward normal vector of the k-th fitted plane, m represent the number of fitted planes, and A Θ This represents the 3-1-2 coordinate transformation matrix between the inertial frame and the body frame at attitude angle Θ. This represents the outward normal vector of the color plane under zero orientation.

[0032] S164. Optimize the objective function to obtain the centroid position and three-axis attitude of the target module.

[0033] Preferably, the method for predicting the motion of the target in step S2 includes the following steps:

[0034] S21. Extended Kalman filtering expands the nonlinear functions f(*) and h(*) in a nonlinear system into a Taylor series, retains the first term, and neglects the other terms to obtain an approximate first-order linearized model. Then, it performs filtering estimation according to the standard Kalman filtering process. A general nonlinear system can be described by the following equations.

[0035]

[0036] Z(t)=h[X(t),t]+v(t); (5);

[0037] In the formula, W(t) and v(t) are both uncorrelated zero-mean white noise sequences, and they are also uncorrelated with the initial state X(0) or X0. That is, for t>t0, they have the following statistical properties:

[0038] E[w(t)]=0, E[w(t·w T [(τ)]=q(t)δ(t-τ)

[0039] E[v(t)]=0, E[v(t)·v T [(τ)]=r(t)δ(t-τ)

[0040] E[w(t)·v T (τ)]=0,E[X(0)·w T [(τ)]=0

[0041] E[X(0)·v T [(τ)]=0 (6);

[0042] After linearizing the nonlinear system, we obtain:

[0043]

[0044] δZ(t)=H(t)δX(t)+v(t) (7);

[0045] Discretize it further to obtain:

[0046]

[0047] δZ k =H k δX k +V k (8);

[0048] When T is a small quantity, we have:

[0049]

[0050]

[0051] Thus, the discrete nonlinear generalized Kalman filter equation is obtained as follows:

[0052]

[0053] For the observation Z, the calculation in the filtering system is as follows:

[0054]

[0055] Based on the above process, the position and attitude motion of the target module are observed. The target's position and motion satisfy Newton's second law, and the state matrix is ​​composed of position, velocity, and acceleration as state variables:

[0056] X R =[RV a] T (11);

[0057] Assuming the target is a constant-value maneuvering target during the observation process, i.e., its acceleration does not change, the corresponding state model can be written as:

[0058]

[0059] Where 03 represents a third-order all-zero matrix, E3 represents a third-order identity matrix, and the state matrix is:

[0060]

[0061] Visual measurement can only observe the target's location information; the observation matrix is ​​as follows:

[0062] H R =[E3 03 03] (14);

[0063] In the target's attitude motion, attitude angle, angular velocity, and angular acceleration are the state variables:

[0064] X A =[Θ ω α] T (15);

[0065] Unlike positional motion, in the attitude description of a rigid body, the first-order time derivative of the attitude angle is... They are not equal to angular velocity ω; the kinematic relationship between them is defined by Euler angles based on the 3-1-2 rotation method. The definition can be described as follows:

[0066]

[0067] Rewrite it as:

[0068]

[0069] Assuming the target angular acceleration remains unchanged during the observation process, the corresponding state model can be written as:

[0070]

[0071] The state matrix for attitude motion is:

[0072]

[0073] Visual measurements can only reveal the target's pose information; the observation matrix is ​​as follows:

[0074] H A =[E3 03 03] (20);

[0075] S22. When performing EKF filtering estimation, considering the many uncertainties in image processing, which may lead to the failure of centroid position and attitude angle observations, resulting in abrupt changes in the observation results, in order to avoid filtering bias caused by abrupt changes in the input observation results, the observation results are judged. If there is a large deviation between the current target pose observation result and the previous time, the confidence of the current observation result is considered to be low and needs to be processed. In the case where no effective pose information is obtained, that is, the observation result is set to zero, the observation result of the previous time is used as the current observation result and input into the filtering process. In the case where there is a large difference between the current position observation information and the previous time, the average value of the previous time and the current time observation results is taken to reduce the abrupt change of the current observation information.

[0076] S23. In order to describe the future motion state of the target, that is, to predict the target motion, it is necessary to add a time parameter to the motion information, and expand the target motion information from three-dimensional position coordinates to four-dimensional motion information of "position + time". After fitting the motion trajectory, it is possible to predict the position that the target can reach at a certain time in the future. Considering that the target's motion basically satisfies a quadratic function when it does not have autonomous maneuvering, the motion function fitting method is adopted. The fitting parameters of the target motion are continuously iterated in the simulation and optimized. Based on the derivation of the quadratic function formula, the motion trajectory can be fitted.

[0077] The quadratic function is written as:

[0078] x = At 2 / 2+Bt+C (21);

[0079] Substituting the position R, velocity V, and acceleration a obtained from the filtering estimate into the equation, we have:

[0080]

[0081] Preferably, the method for quickly capturing the target module in step S3 includes the following steps:

[0082] S31. Based on the dynamics model of the space robot, build a reinforcement learning training simulation environment for its trajectory planning system, determine its state variables and motion variables, and design a reward function;

[0083] The dynamic model of a space robot can be written in the following form:

[0084]

[0085] in These correspond to the control forces / torques of the space robot's base and each joint. These are the position / angle vectors corresponding to the space robot's base and each joint. Here, q represents the velocity and acceleration vectors corresponding to q, and M(q) is the mass matrix of the system. It is a nonlinear matrix related to velocity;

[0086] Based on the space robot dynamics model, select the system state variables. Where Δp e Let be the difference between the robot's end point and the desired position, and let flag be a marker indicating whether the robot's end point has reached the desired position. The system motion quantity is chosen as a = τ. During training, the target position is set as the endpoint. The success of robot motion planning depends on whether the robot's end point can reach the endpoint and remain stable. Therefore, the distance between the robot's end point and the target point is a crucial parameter for establishing the reward model. To enable the robot to quickly reach the target point, the single-step reward function during task planning is set as r = r1 + r2, where:

[0087] r1=-ω||Δp e || (24);

[0088] ω is a constant weight. Each time the robotic arm's end point reaches the target position, the robotic arm is given a constant reward of +1.

[0089]

[0090] S32. Design a training network based on the Deep Deterministic Policy Gradient (DDPG) algorithm. The DDPG algorithm adopts the Actor-Critic model, where the Actor makes decisions and the Critic makes evaluations: First, the state s is observed from the environment and passed to the Actor. Based on this state, the Actor makes a decision based on the existing policy and obtains the action a. After the action is applied to the environment, the environment will provide the reward feedback r for the current step and a new state. Based on the reward feedback r, the Critic will update the evaluation network for the Actor's behavior. The Actor then updates its own policy network along the direction suggested by the Critic. This completes one step of training, and then continues to loop until training is successful. Since neural networks require that the training data are independent, while the data obtained by the Actor through continuous interaction with the environment has a certain correlation, DDPG adopts an experience pool design. The data samples obtained from the interaction with the environment are stored in the experience pool, and then a set of data is randomly selected from it for training. This breaks the correlation between the data and achieves the independence of the samples.

[0091] S33. Design hyperparameters in the network such as learning rate, discount factor, experience pool size, training rounds, and maximum number of iterations;

[0092] S34. Complete the training of the space robot trajectory planning strategy based on the DDPG algorithm.

[0093] Preferably, the method for training the space robot trajectory planning strategy based on the DDPG algorithm in step S34 specifically includes the following steps:

[0094] S341. Actor, based on the current state s t Combine the current strategy μ to select action a t =μ(s) t |θ μ (and pass it to the environment to perform the action);

[0095] S342. Environment executes action a t Transition to the new state s t+1 And generate a reward value r after the action is performed. t Return it to the Actor;

[0096] S343.Actor combines (s t ,a t ,r t ,s t+1 The data is stored in the experience pool M as the dataset for training the network.

[0097] S344. Randomly select N sets of data from the experience pool and use them as mini-batch data for training the real policy network and the real Q network;

[0098] S345. The target policy network is based on (s i ,a i ,r i ,s i+1 The state s in ) i+1 Combined with network strategy θ μ′ Generate action μ′(s) i+1 The data is then fed into the target Q-network to calculate the target Q-value. The target Q-network calculates the target Q-value based on (s). i ,a i ,r i ,s i+1 ) and s i+1 The corresponding μ′(s) i+1 Calculate the target Q value:

[0099] y i =r i +γQ′(s i+1 ,μ′(s i+1 |θ μ′)|θ Q′ (26);

[0100] Simultaneously, the real-world strategy network is combined with (s) i ,a i ,r i ,s i+1 s in ) i Generate action a i The corresponding Q-value Q(s) is calculated by inputting it into the real Q-network. i ,a i |θ Q Therefore, the loss function of the Q-network is defined as:

[0101]

[0102] L can be obtained with respect to θ Q gradient

[0103] S346. Optimize and update the network parameters θ of the real-world Q-network. Q ;

[0104] S347. Calculate the gradient of the policy network.

[0105]

[0106] As can be seen from the above formula, the gradient update formula of the policy network consists of N sums of two-part multiplications. The first part is obtained by the Critic and is used to evaluate how the Actor should move to obtain a larger Q value. The second part is obtained by the Actor and is used to reflect how the Actor should update its own parameters so that the policy can take actions that can obtain a larger Q value. In general, it is to make the policy network learn to update its own parameters in the direction of obtaining a larger Q value.

[0107] S348. Optimize and update the network parameters θ of the real-world policy network. μ ;

[0108] S349. Analyze the network parameters θ of the target policy network and the target Q network respectively. μ′ and θ Q′ Perform a soft update:

[0109] θ Q′ ←ηθ Q +(1-η)θ Q′ (29);

[0110] θ μ′ ←ηθ μ +(1-η)θ μ′ (30);

[0111] When the above steps are completed, the training within one time step is completed. Restarting the loop from step S341 begins the training for the next time step. After all time steps of training are completed, a complete round of training is completed. Through the continuous iteration of rounds, the network parameters will be updated and developed in the direction of maximizing the Q value.

[0112] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are:

[0113] 1. In this application, the method for calculating the pose information of the target module based on a simulated binocular vision camera and the fitting plane method can simulate the imaging of the binocular vision camera in a simulation environment, and then calculate the position and orientation information of the target module according to the fitting plane method. This improves the problems that may occur when commonly used line endpoint detection of target vertices, such as too few image features and large errors in target pose information.

[0114] 2. In this application, the EKF-based estimation method can correct the target's motion state based on multiple consecutively observed target images, obtain the filtered estimation result of the target motion, and then fit the motion of the target without autonomous maneuvering based on a quadratic function.

[0115] 3. In this application, the space robot trajectory planning method based on reinforcement learning uses the dynamic model of the space robot as the training environment, which ensures the transferability of the training results to the real space robot system; the designed reward function is related to the distance between the end point and the desired position, which can guide the robot end point to move closer to the desired position as soon as possible, effectively improving the training effect of the space robot trajectory planning system. Attached Figure Description

[0116] Figure 1 This is a flowchart of the on-orbit capture method for space robots based on simulated binocular vision measurement proposed in this invention;

[0117] Figure 2 This is a schematic diagram of the camera view plane filtering method proposed in this invention;

[0118] Figure 3 This is a schematic diagram illustrating the vector relationship between the outward-pointing normal vector of the plane and the feature points proposed in this invention;

[0119] Figure 4 This is a flowchart of the target module motion information solving process proposed in this invention;

[0120] Figure 5 This is a schematic diagram of the cubic target module proposed in this invention;

[0121] Figure 6 The present invention provides a filter for estimating position and velocity errors;

[0122] Figure 7 The present invention provides a filter for estimating attitude angles and angular velocity errors.

[0123] Figure 8 This invention proposes a position motion prediction method based on quadratic function parameter fitting.

[0124] Figure 9 This invention presents the distribution of the total reward value and the number of steps used in the training process proposed in this invention.

[0125] Figure 10 This refers to the success rate during the training process proposed in this invention. Detailed Implementation

[0126] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0127] Please see Figure 1-10 The present invention provides a technical solution:

[0128] A method for on-orbit capture of a space robot based on simulated binocular vision measurement includes the following steps:

[0129] S1. Based on a simulated binocular vision camera, the feature point information of the target module is obtained, the three-dimensional position markers of the feature points are obtained through a feature point matching algorithm, and then the position and orientation information of the target module are calculated according to the fitting plane method.

[0130] The method for acquiring feature point information of the target module based on a simulated binocular vision camera in step S1 includes the following steps:

[0131] S11. Determine the size, color, and outward normal vector parameters of the cube target module in the simulation environment, such as... Figure 5 As shown, it serves as the observation object for subsequent simulation of a binocular vision camera;

[0132] S12. Load the cube target module from step S11. Since the projection of the target in the camera is not a parallel projection, but converges at the optical center of the camera, when simulating the image of the target, it is necessary to convert the coordinates of each vertex of the target in inertial space into the pixel position in the image. It is also necessary to judge and filter the planes that can be seen in the image, and finally retain the planes that can actually be seen in the image.

[0133] The specific implementation method of step S12 includes the following steps:

[0134] S121. Solve for the coordinates of all vertices on the cube target in both the camera coordinate system and the pixel coordinate system, denoted as X. fc and X fuv ;

[0135] S122. Perform the first image visualization screening, calculate the outward-pointing normal vector of each surface of the target. The outward-pointing normal vector must point towards the camera side; that is, the angle between the outward-pointing normal vector and the Z-axis of the camera coordinate system must be greater than 90° for that plane to be visible in the camera image. In the case of parallel projection, all n out ·z c Planes greater than 0 can all be displayed in the camera image, where n out Let z be the outward normal vector of the plane. c The Z-axis of the camera coordinate system is used, but because the camera uses perspective imaging, the blind spot of the object becomes larger when it is projected. This strengthens the constraint of the plane that can be seen in the camera image, so a second screening is required.

[0136] S123. Perform a second image visualization filter to determine the planes that are actually visible in the camera image. Number the vertices of each plane in a specific order, so that the vertices spiral outwards from the target. Connect these vertices sequentially on the pixel plane. If the spiral points inwards, the plane is visible in the camera image; if the spiral points outwards, the back of the plane is visible. For solid targets, this plane is not visible inside the object. Figure 2 As shown in (b).

[0137] S13. The minimum eigenvalue algorithm is used to detect corner points in the image. The theoretical principle of this method is the Harris corner detection algorithm, which can detect pixels at feature positions such as edges and corners in the image. Compared with Hough line detection, corner detection can obtain more feature points. The feature point recognition results are based on the minimum eigenvalue algorithm.

[0138] S14. Use the pyramid optical flow method to match feature points in the left and right camera views, and solve for the three-dimensional spatial coordinates of all matched feature point pairs.

[0139] S15. For each successfully matched point pair, color-mark it in its respective image. Centered on the feature point, select a certain pixel range, judge by the RGB values ​​of the pixels, mark the color of the pixels near the feature point, and mark the color that has appeared near the feature point as the color of the feature point. According to the geometric features of the cube target, a feature point has at least one and at most three color marks;

[0140] S16. Based on the feature points with color labels, find the plane on which the target surface is located, and then restore the three-dimensional state of the cube target;

[0141] The implementation method of step S16 specifically includes the following steps:

[0142] S161. Define the equation of a certain color plane as:

[0143] ax+by+cz+d=0 (1);

[0144] Extract points with the same color label, perform linear fitting on the plane containing these feature points, and the resulting plane function is:

[0145] f(z) = p 00 +p 10 x +p 01 y (2);

[0146] The relationship between the three coefficients and the four parameters of the plane equation is as follows:

[0147] [abcd]=[p 10 p 01 -1 p 00 (3);

[0148] S162. Select a feature point detected in the image that does not belong to the plane, and then randomly select one feature point belonging to the fitted plane. Establish a vector n1 pointing from the plane to the feature points on other surfaces of the target plane. Let the outward normal vector of the fitted plane be n0, and the angle between the two vectors be a0. When the normal vector of the plane points out of the plane, a0 must be an obtuse angle. Figure 3 As shown, when the angle between the plane normal vector and the feature point vector is obtuse, the plane parameter is the outward normal vector of the fitted plane. For cases where the angle is not obtuse, the first three parameters of the plane equation are negative.

[0149] S163. After obtaining the outward-pointing normal vectors of multiple planes, the centroid position and attitude angle of the target module are used as variables to design an index function and solve it using an optimization method. The index function is written in the following form:

[0150]

[0151] Where r center Represents the position vector of the centroid of an inertial space cube. Let represent the inertial space coordinates of the i-th feature point, n represent the total number of feature points, and L represent the side length of the target module. The normal vector of the plane passing through the centroid, after the plane of the fit is translated according to the side length of the cube. Let A represent the outward normal vector of the k-th fitted plane, m represent the number of fitted planes, and A ΘThis represents the 3-1-2 coordinate transformation matrix between the inertial frame and the body frame at attitude angle Θ. This represents the outward normal vector of the color plane under zero orientation;

[0152] The first two terms of the index function are related to the position of the centroid. The first term represents the sum of the distances from the centroid to the feature points. In a cube, the sum of the distances from the center of the cube to the points on each edge is the smallest. The second term is the sum of the distances from the centroid to the fitted plane passing through the centroid. The ideal value of this sum of distances is zero. The third term of the index function is related to the cube's attitude. The outward normal vector of the plane under zero attitude should coincide with the fitted outward normal vector of the plane after attitude coordinate transformation. Therefore, the sum of the errors between the plane normal vectors is used as the index. The smaller the error, the more accurate the attitude angle solution.

[0153] S164. Optimize and solve the objective function to obtain the centroid position and three-axis attitude of the target module;

[0154] S2. Following the target identification process in step S1, the target module is continuously observed and tracked. The motion state of the target is solved based on the EKF method to obtain the filtered estimation result of the target motion, such as... Figure 4 As shown, the motion trajectory of the target module is then fitted to achieve motion prediction of the target;

[0155] The method for predicting the motion of the target in step S2 includes the following steps:

[0156] S21. Extended Kalman filtering expands the nonlinear functions f(*) and h(*) in a nonlinear system into a Taylor series, retains the first term, and neglects the other terms to obtain an approximate first-order linearized model. Then, it performs filtering estimation according to the standard Kalman filtering process. A general nonlinear system can be described by the following equations.

[0157]

[0158] Z(t)=h[X(t),t]+v(t); (5);

[0159] In the formula, W(t) and v(t) are both uncorrelated zero-mean white noise sequences, and they are also uncorrelated with the initial state X(0) or X0. That is, for t>t0, they have the following statistical properties:

[0160] E[w(t)]=0, E[w(t·w T [(τ)]=q(t)δ(t-τ)

[0161] E[v(t)]=0, E[v(t)·v T [(τ)]=r(t)δ(t-τ)

[0162] E[w(t)·v T (τ)]=0,E[X(0)·w T [(τ)]=0

[0163] E[X(0)·v T [(τ)]=0 (6);

[0164] After linearizing the nonlinear system, we obtain:

[0165]

[0166] δZ(t)=H(t)δX(t)+v(t) (7);

[0167] Discretize it further to obtain:

[0168]

[0169] δZ k =H k δX k +V k (8);

[0170] When T is a small quantity, we have:

[0171]

[0172]

[0173] Thus, the discrete nonlinear generalized Kalman filter equation is obtained as follows:

[0174]

[0175] For the observation Z, the calculation in the filtering system is as follows:

[0176]

[0177] Based on the above process, the position and attitude motion of the target module are observed. The target's position and motion satisfy Newton's second law, and the state matrix is ​​composed of position, velocity, and acceleration as state variables:

[0178] X R =[RV a] T (11);

[0179] Assuming the target is a constant-value maneuvering target during the observation process, i.e., its acceleration does not change, the corresponding state model can be written as:

[0180]

[0181] Where 03 represents a third-order all-zero matrix, and E3 represents a third-order identity matrix. The state matrix is:

[0182]

[0183] Visual measurement can only observe the target's location information; the observation matrix is ​​as follows:

[0184] H R =[E3 03 03] (14);

[0185] In the target's attitude motion, attitude angle, angular velocity, and angular acceleration are the state variables:

[0186] X A =[Θ ω α] T (15);

[0187] Unlike positional motion, in the attitude description of a rigid body, the first-order time derivative of the attitude angle is... They are not equal to angular velocity ω; the kinematic relationship between them is defined by Euler angles based on the 3-1-2 rotation method. The definition can be described as follows:

[0188]

[0189] Rewrite it as:

[0190]

[0191] Assuming the target angular acceleration remains unchanged during the observation process, the corresponding state model can be written as:

[0192]

[0193] The state matrix for attitude motion is:

[0194]

[0195] Visual measurements can only reveal the target's pose information; the observation matrix is ​​as follows:

[0196] H A =[E3 03 03] (20);

[0197] S22. When performing EKF filtering estimation, considering the many uncertainties in image processing, which may lead to the failure of centroid position and attitude angle observations, resulting in abrupt changes in the observation results, in order to avoid filtering bias caused by abrupt changes in the input observation results, the observation results are judged. If there is a large deviation between the current target pose observation result and the previous time, the confidence of the current observation result is considered to be low and needs to be processed. In the case where no effective pose information is obtained, that is, the observation result is set to zero, the observation result of the previous time is used as the current observation result and input into the filtering process. In the case where there is a large difference between the current position observation information and the previous time, the average value of the previous time and the current time observation results is taken to reduce the abrupt change of the current observation information.

[0198] Based on the filtering estimation method described above, EKF parameters are designed, where the environmental noise is 5 × 10⁻⁶. -2 N, the estimated noise level for position observation is 0.1m, the estimated noise level for attitude observation is 0.1deg, and the external parameter matrix of the stereo camera is as follows:

[0199]

[0200] The intrinsic parameter matrices are as follows:

[0201]

[0202] The final filtered estimation of position and velocity errors for the cubic target is as follows: Figure 6 As shown, the attitude angle and angular velocity errors are as follows: Figure 7 As shown, after filtering estimation, the target position estimation error can converge to within 0.05m. Compared with the direct observation result, the noise is reduced and the error is reduced. The estimated result of the target motion velocity is closer to the real state. After about 5 seconds of observation, the motion velocity estimation error can converge to within ±0.02m / s, and the motion observation accuracy is high. The attitude angle estimation error can be further converged from within ±2deg of the observation result, and can reach ±1deg in most cases. The attitude angular velocity estimation error can be kept within ±2deg / s, which also has high estimation accuracy.

[0203] S23. In order to describe the future motion state of the target, that is, to predict the target motion, it is necessary to add a time parameter to the motion information, and expand the target motion information from three-dimensional position coordinates to four-dimensional motion information of "position + time". After fitting the motion trajectory, it is possible to predict the position that the target can reach at a certain time in the future. Considering that the target's motion basically satisfies a quadratic function when it does not have autonomous maneuvering, the motion function fitting method is adopted. The fitting parameters of the target motion are continuously iterated in the simulation and optimized. Based on the derivation of the quadratic function formula, the motion trajectory can be fitted.

[0204] The quadratic function is written as:

[0205] x = At 2 / 2+Bt+C (21);

[0206] Substituting the position R, velocity V, and acceleration a obtained from the filtering estimate into the equation, we have:

[0207]

[0208] A constant force of [-0.1 to 0.11] N and a constant torque of [0.01 to 0.02 to -0.01] Nm are applied to the target. The simulation step size is 0.05 s and the simulation duration is 5 s. Based on the target motion state obtained by filtering at each time step, the motion trajectory is iteratively optimized, and the target position and attitude at t = 5 s are estimated. The simulation results are as follows. Figure 8 As shown, as the simulation progresses, the parameters of the motion trajectory are iteratively fitted, resulting in improved fitting accuracy and consequently, improved motion prediction accuracy. Figure 8 (a) shows the prediction of the target's arrival position at time t=5s within each simulation step. Figure 8 (b) shows the prediction of the target's arrival position at the fifth second within each simulation step from the third second to the fifth second. As the simulation progresses, the prediction of the target's position gradually approaches the actual result.

[0209] S3. A trajectory planning system for a space robot is trained based on reinforcement learning methods. The predicted target pose information is input into the trajectory planning system of the space robot to achieve rapid acquisition of the target module.

[0210] The method for quickly capturing the target module in step S3 includes the following steps:

[0211] S31. Based on the dynamics model of the space robot, build a reinforcement learning training simulation environment for its trajectory planning system, determine its state variables and motion variables, and design a reward function;

[0212] The dynamic model of a space robot can be written in the following form:

[0213]

[0214] in These correspond to the control forces / torques of the space robot's base and each joint. These are the position / angle vectors corresponding to the space robot's base and each joint. Here, q represents the velocity and acceleration vectors corresponding to q, and M(q) is the mass matrix of the system. It is a nonlinear matrix related to velocity;

[0215] Based on the space robot dynamics model, select the system state variables. Where Δp e Let be the difference between the robot's end point and the desired position, and let flag be a marker indicating whether the robot's end point has reached the desired position. The system motion quantity is chosen as a = τ. During training, the target position is set as the endpoint. The success of robot motion planning depends on whether the robot's end point can reach the endpoint and remain stable. Therefore, the distance between the robot's end point and the target point is a crucial parameter for establishing the reward model. To enable the robot to quickly reach the target point, the single-step reward function during task planning is set as r = r1 + r2, where:

[0216] r1=-ω||Δp e || (24);

[0217] ω is a constant weight. Each time the robotic arm's end point reaches the target position, the robotic arm is given a constant reward of +1.

[0218]

[0219] S32. Design a training network based on the Deep Deterministic Policy Gradient (DDPG) algorithm. The DDPG algorithm adopts the Actor-Critic model, where the Actor makes decisions and the Critic makes evaluations: First, the state s is observed from the environment and passed to the Actor. Based on this state, the Actor makes a decision based on the existing policy and obtains the action a. After the action is applied to the environment, the environment will provide the reward feedback r for the current step and a new state. Based on the reward feedback r, the Critic will update the evaluation network for the Actor's behavior. The Actor then updates its own policy network along the direction suggested by the Critic. This completes one step of training, and then continues to loop until training is successful. Since neural networks require that the training data are independent, while the data obtained by the Actor through continuous interaction with the environment has a certain correlation, DDPG adopts an experience pool design. The data samples obtained from the interaction with the environment are stored in the experience pool, and then a set of data is randomly selected from it for training. This breaks the correlation between the data and achieves the independence of the samples.

[0220] S33. Design hyperparameters in the network such as learning rate, discount factor, experience pool size, training rounds, and maximum number of iterations;

[0221] S34. Complete the training of a space robot trajectory planning strategy based on the DDPG algorithm;

[0222] Step S34, which involves training a space robot trajectory planning strategy based on the DDPG algorithm, specifically includes the following steps:

[0223] S341. Actor, based on the current state s t Combine the current strategy μ to select action a t =μ(s) t |θ μ (and pass it to the environment to perform the action);

[0224] S342. Environment executes action a t Transition to the new state s t+1 And generate a reward value r after the action is performed. t Return it to the Actor;

[0225] S343.Actor combines (s t ,a t ,r t ,s t+1 The data is stored in the experience pool M as the dataset for training the network.

[0226] S344. Randomly select N sets of data from the experience pool and use them as mini-batch data for training the real policy network and the real Q network;

[0227] S345. The target policy network is based on (s i ,a i ,r i ,s i+1 The state s in ) i+1 Combined with network strategy θ μ′ Generate action μ′(s) i+1 The data is then fed into the target Q-network to calculate the target Q-value. The target Q-network calculates the target Q-value based on (s). i ,a i ,r i ,s i+1 ) and s i+1 The corresponding μ′(s) i+1 Calculate the target Q value:

[0228] y i =r i +γQ′(s i+1 ,μ′(s i+1 |θ μ′ )|θQ′ (26);

[0229] Simultaneously, the real-world strategy network is combined with (s) i ,a i ,r i ,s i+1 s in ) i Generate action a i The corresponding Q-value Q(s) is calculated by inputting it into the real Q-network. i ,a i |θ Q Therefore, the loss function of the Q-network is defined as:

[0230]

[0231] L can be obtained with respect to θ Q gradient

[0232] S346. Optimize and update the network parameters θ of the real-world Q network. Q ;

[0233] S347. Calculate the gradient of the policy network.

[0234]

[0235] As can be seen from the above formula, the gradient update formula of the policy network consists of N sums of two-part multiplications. The first part is obtained by the Critic and is used to evaluate how the Actor should move to obtain a larger Q value. The second part is obtained by the Actor and is used to reflect how the Actor should update its own parameters so that the policy can take actions that can obtain a larger Q value. In general, it is to make the policy network learn to update its own parameters in the direction of obtaining a larger Q value.

[0236] S348. Optimize and update the network parameters θ of the real-world policy network. μ ;

[0237] S349. Analyze the network parameters θ of the target policy network and the target Q network respectively. μ′ and θ Q′ Perform a soft update:

[0238] θ Q′ ←ηθ Q +(1-η)θ Q′ (29);

[0239] θ μ′ ←ηθ μ +(1-η)θ μ′ (30);

[0240] When the above steps are completed, the training within one time step is completed. Starting the loop again from step S341 will start the training for the next time step. When all time steps of training are completed, a complete round of training is completed. Through the continuous iteration of rounds, the network parameters will be updated and developed in the direction of maximizing the Q value.

[0241] Assuming the pose of the space robot's base is controllable, the space robot model is simplified to a planar three-DOF robotic arm. Based on the space robot's dynamics model, a reinforcement learning training simulation environment for its trajectory planning system is built to determine its state variables and motion variables, and a reward function is designed.

[0242] Design a training network based on the DDPG algorithm. Both the policy network and the Q network adopt a three-layer neural network structure. The input of the policy network is the system state s, and the output is the action a. The output layer of the policy network uses the tanh activation function to restrict the continuous output action to the range [-1,1]. Then, it is multiplied by the action amplitude parameter according to the requirements to obtain the desired action output. The remaining layers all use the ReLU activation function. The input of the Q network is the system state s, the action a, and a one-dimensional bias unit b. The output is the score of the state-action pair, i.e., the Q value. The Q network uses the ReLU activation function.

[0243] The hyperparameters of the network are designed, including learning rate, discount factor, experience pool size, training epochs, and maximum iteration steps. The learning rate of the policy network and the Q network are both set to 0.001, the discount factor is set to 0.9, and the size of its experience pool M is set to 10,000 sets of data. When the experience pool is full, 64 sets of data are randomly selected from it to train the policy network and the Q network. The maximum number of training epochs is set to 3,000. In order to avoid excessively long and meaningless training, the maximum number of iteration steps in each epoch is set to 200. If the target point is not reached after more than 200 iterations, the current epoch is forcibly terminated and the next epoch is started.

[0244] The trajectory planning strategy for a space robot was trained using the DDPG algorithm. After 3000 training iterations, the total reward value and the total number of iterations used in each round of the entire process were obtained, as shown below. Figure 9As shown in the figure, during the initial training phase (approximately rounds 1-250), the robot is in an exploration phase, and its scores in each round are not ideal. After about 50 rounds, although the total reward value obtained by the system is still at a low level, it shows some improvement. This is because the designed experience pool M is 10,000 in size. In the first 50 rounds, the experience pool is not yet full, and each round randomly selects actions for exploration. Once the experience pool is full, it begins to randomly select data from it to train the policy network and Q network. At this point, the robot's actions are generated by the policy network that has been trained to a certain extent, thus improving the training effect to some extent. Furthermore, the training in the first 250 rounds... The training was forcibly terminated after 200 iterations, and then the training process gradually converged. As training progressed, the robot was able to converge after 40-65 steps, and the total reward value of each round was mainly distributed around 29-30. This is related to the designed reward function and the round termination condition. In order to keep the robot end point in the desired position after it reaches it, the round ended after the robot end point kept in the target position for 30 steps. The robot end point received a reward value of +1 every time it passed the target position. Therefore, the total reward value is concentrated in the range of 29-30, which reflects that the trained robotic arm can quickly converge to the target position and keep the position basically unchanged.

[0245] Figure 9 This reflects the distribution of rewards and steps used during training, but the description of the changing trends in the training process is not obvious. Therefore, further data processing is needed. To avoid the influence of randomness to some extent, the average of the total reward value and the total number of steps used over several rounds is used as the average reward value and average number of steps for a stage. Here, we use the average situation every 10 rounds as an example to obtain the change curves of the average reward value and average number of steps. From this, we can see that the training process can be roughly divided into four stages: the first stage of training initially has a reward value of approximately -37 and 200 iterations, corresponding to the state where the robotic arm randomly selects actions to explore the environment; the second stage begins after the experience pool is full, that is, after training the policy network and Q network begins. The system immediately receives a significant increase in reward value, but the number of iterations required per round remains at 200. This corresponds to a state where the robotic arm's end effector can approach the target position but cannot reach or maintain it. The system then enters the third stage, where the trained policy network gradually provides an ideal trajectory, allowing the system to converge to the target position after several iterations. However, the required number of steps is still high, and the corresponding reward value is also relatively large. This indicates that the robotic arm's end effector will continuously pass the target position but cannot effectively maintain stability. Finally, in the fourth stage, the system tends to stabilize, typically achieving convergence after approximately 40-60 iterations per round, with a corresponding reward value of around 30. The training process is not static and can exhibit some fluctuations.

[0246] The outcome of each training round is recorded. If a round is forcibly terminated after 200 iterations, it is recorded as unsuccessful; if it is completed within 200 iterations, it is recorded as successful. The average success rate is calculated for every 100 rounds to obtain the success rate at different stages of the training process. Figure 10 As shown in the figure, the success rate of training increases rapidly at each stage as training progresses, and can maintain a success rate of over 95% thereafter.

[0247] The training results are saved. When planning a new task, the trajectory can be planned simply by following the parameters obtained from the training.

[0248] In summary, the on-orbit capture method for a space robot based on simulated binocular vision measurement provided in this embodiment simulates a binocular vision camera in a simulation environment, abstracts the target module as a cube, and simulates the human visual system by using a simulated binocular CCD camera. Simultaneously, left and right images of the target module are captured from different positions. A stereo matching algorithm is used to process the left and right image pairs to obtain the disparity map of the target module. The depth information of the object is obtained by combining optical principles such as projection geometry imaging and triangulation, achieving the effect of measuring the target's pose information. Then, by continuously observing and tracking the target, the true motion state of the target is estimated, and the motion of the target at a future moment is predicted. Based on the prediction results, a trajectory planning system for the space robot is designed to plan the robot's motion trajectory from the current state to the expected target position, ultimately achieving on-orbit capture of the target module.

[0249] The above description of the embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for on-orbit capture of a space robot based on simulated binocular vision measurement, characterized in that, Includes the following steps: S1. Based on a simulated binocular vision camera, the feature point information of the target module is obtained, the three-dimensional position markers of the feature points are obtained through a feature point matching algorithm, and then the position and orientation information of the target module are calculated according to the fitting plane method. S2. Based on the target identification process in step S1, the target module is continuously observed and tracked. The motion state of the target is solved based on the EKF method to obtain the filtered estimation result of the target motion. Then, the motion trajectory of the target module is fitted to achieve the motion prediction of the target. S3. A trajectory planning system for a space robot is trained based on reinforcement learning methods. The predicted target pose information is input into the trajectory planning system of the space robot to achieve rapid acquisition of the target module. The method for acquiring feature point information of the target module based on a simulated binocular vision camera in step S1 includes the following steps: S11. Determine the size, color, and outward normal vector parameters of the cube target module in the simulation environment, as the observation object for the subsequent simulation of the binocular vision camera; S12. Load the cube target module from step S11. Since the projection of the target in the camera is not a parallel projection, but converges at the optical center of the camera, when simulating the image of the target, it is necessary to convert the coordinates of each vertex of the target in inertial space into the pixel position in the image. It is also necessary to judge and filter the planes that can be seen in the image, and finally retain the planes that can actually be seen in the image. S13. The minimum eigenvalue algorithm is used to detect corner points in the image, which can detect pixels at edge and corner feature positions in the image; S14. Use the pyramid optical flow method to match feature points in the left and right camera views, and solve for the three-dimensional spatial coordinates of all matched feature point pairs. S15. For each successfully matched pair of points, color mark them in their respective images. Centered on the feature point, select a certain pixel range and judge by the RGB values ​​of the pixels. Mark the colors of the pixels near the feature point. Mark the colors that have appeared near the feature point as the color of the feature point. According to the geometric features of the cube target, a feature point has at least one and at most three color marks. S16. Based on the feature points with color labels, find the plane on which the target surface is located, and then perform three-dimensional state reconstruction of the cube target.

2. The on-orbit capture method for a space robot based on simulated binocular vision measurement according to claim 1, characterized in that, The specific implementation method of step S12 includes the following steps: S121. Solve for the coordinates of all vertices on the cube target in both the camera coordinate system and the pixel coordinate system, denoted as X. fc and X fuv ; S122. Perform the first image visualization screening, calculate the outward-pointing normal vector of each surface of the target. The outward-pointing normal vector must point towards the camera side; that is, the angle between the outward-pointing normal vector and the Z-axis of the camera coordinate system must be greater than 90° for that plane to be visible in the camera image. In the case of parallel projection, all n out ·z c Planes greater than 0 can all be displayed in the camera image, where n out Let z be the outward normal vector of the plane. c The Z-axis of the camera coordinate system is used, but because the camera uses perspective imaging, the blind spot of the object becomes larger when it is projected. This strengthens the constraint of the plane that can be seen in the camera image, so a second screening is required. S123. Perform a second image visualization filter to determine the planes that are actually visible in the camera image. Number the vertices of each plane in a certain order so that the vertices spiral outwards towards the outside of the target. Connect these vertices sequentially on the pixel plane. If the spirals are inwards, the plane can be seen in the camera image. If the spirals are outwards, the back of the plane is visible. For solid targets, the plane is not visible inside the object.

3. The on-orbit capture method for a space robot based on simulated binocular vision measurement according to claim 1, characterized in that, The implementation method of step S16 specifically includes the following steps: S161. Define the equation of a certain color plane as: ax+by+cz+d=0 (1); Extract points with the same color label, perform linear fitting on the plane containing these feature points, and the resulting plane function is: f(z)=p 00 +p 10 x +p 01 y (2); The relationship between the three coefficients and the four parameters of the plane equation is as follows: [a b c d]=[p 10 p 01 -1 p 00 ] (3); S162. Select a feature point detected in the image that does not belong to the plane. Select any feature point that belongs to the fitting plane and establish a vector n1 pointing from the plane to the feature point on other surfaces of the target. Let the outward normal vector of the fitted plane be n0, and the angle between the two vectors be a0. When the normal vector of the plane points out of the plane, a0 must be an obtuse angle. When the angle between the solved plane normal vector and the feature point vector is an obtuse angle, the plane parameter is the outward normal vector of the fitted plane. For cases where the angle is not obtuse, take the negative values ​​of the first three parameters of the plane equation. S163. After obtaining the outward normal vectors of multiple planes, the centroid position and attitude angle of the target module are used as variables. An index function is designed and solved using an optimization method. The index function is written in the following form: Where r center Represents the position vector of the centroid of an inertial space cube. Let represent the inertial space coordinates of the i-th feature point, n represent the total number of feature points, and L represent the side length of the target module. The normal vector of the plane passing through the centroid, after the plane of the fit is translated according to the side length of the cube. Let A represent the outward normal vector of the k-th fitted plane, m represent the number of fitted planes, and A Θ This represents the 3-1-2 coordinate transformation matrix between the inertial frame and the body frame at attitude angle Θ. This represents the outward normal vector of the color plane under zero orientation; S164. Optimize the objective function to obtain the centroid position and three-axis attitude of the target module.

4. The on-orbit capture method for a space robot based on simulated binocular vision measurement according to claim 1, characterized in that, The method for predicting the motion of the target in step S2 includes the following steps: S21. Extended Kalman filtering expands the nonlinear functions f(*) and h(*) in the nonlinear system into a Taylor series, retains the first term, and neglects the other terms to obtain an approximate first-order linearized model. Then, filtering estimation is performed according to the standard Kalman filtering process. The nonlinear system can be described by the following equations. Z(t) = h[X(t),t] + v(t) (5); In the formula, W(t) and v(t) are both uncorrelated zero-mean white noise sequences, and they are also uncorrelated with the initial state X(0) or X0. That is, for t>t0, they have the following statistical properties: E[w(t)]=0,E[w(t)·w T (τ)]=q(t)δ(t-τ) E[v(t)]=0,E[v(t)·v T (τ)]=r(t)δ(t-τ) E[w(t)·v T (τ)]=0,E[X(0)·w T (τ)]=0 E[X(0)·v T (τ)]=0 (6); After linearizing the nonlinear system, we obtain: δZ(t)=H(t)δX(t)+v(t) (7); Discretize it further to obtain: δZ k =H k δX k +V k (8); When T is a small quantity, we have: Thus, the discrete nonlinear generalized Kalman filter equation is obtained as follows: For the observation Z, the calculation in the filtering system is as follows: Based on the above process, the position and attitude motion of the target module are observed. The position and motion of the target satisfy Newton's second law. The state matrix is ​​composed of position, velocity, and acceleration as state variables: X R =[R V a] T (11); Assuming the target is a constant-value maneuvering target during the observation process, i.e., its acceleration does not change, the corresponding state model can be written as: Where 03 represents a third-order all-zero matrix, E3 represents a third-order identity matrix, and the state matrix is: Visual measurement can only observe the target's location information; the observation matrix is ​​as follows: H R =[E3 03 03] (14); In the target's attitude motion, attitude angle, angular velocity, and angular acceleration are the state variables: X A =[Θ ω a] T (15); Unlike positional motion, in the attitude description of a rigid body, the first-order time derivative of the attitude angle is... They are not equal to angular velocity ω; the kinematic relationship between them is defined by Euler angles based on the 3-1-2 rotation method. The definition can be described as follows: Rewrite it as: Assuming the target angular acceleration remains unchanged during the observation process, the corresponding state model can be written as: The state matrix for attitude motion is: Visual measurements can only reveal the target's pose information; the observation matrix is ​​as follows: H A =[E3 03 03] (20); S22. When performing EKF filtering estimation, considering the many uncertainties in image processing, which may lead to the failure of centroid position and attitude angle observations, resulting in abrupt changes in the observation results, in order to avoid filtering bias caused by abrupt changes in the input observation results, the observation results are judged. If there is a large deviation between the current target pose observation result and the previous time, the confidence of the current observation result is considered to be low and needs to be processed. In the case where no effective pose information is obtained, that is, the observation result is set to zero, the observation result of the previous time is used as the current observation result and input into the filtering process. In the case where there is a large difference between the current position observation information and the previous time, the average value of the previous time and the current time observation results is taken to reduce the abrupt change of the current observation information. S23. In order to describe the future motion state of the target, that is, to predict the target motion, it is necessary to add a time parameter to the motion information, and expand the target motion information from three-dimensional position coordinates to four-dimensional motion information of "position + time". After fitting the motion trajectory, it is possible to predict the position that the target can reach at a certain time in the future. Considering that the target's motion basically satisfies a quadratic function when it does not have autonomous maneuvering, the motion function fitting method is adopted. The fitting parameters of the target motion are continuously iterated in the simulation and optimized. Based on the derivation of the quadratic function formula, the motion trajectory can be fitted. The quadratic function is written as: x=At 2 / 2+Bt+C (21); Substituting the position R, velocity V, and acceleration a obtained from the filtering estimate into the equation, we have:

5. The on-orbit capture method for a space robot based on simulated binocular vision measurement according to claim 1, characterized in that, The method for quickly capturing the target module in step S3 includes the following steps: S31. Based on the dynamics model of the space robot, build a reinforcement learning training simulation environment for its trajectory planning system, determine its state variables and motion variables, and design a reward function; The dynamic model of a space robot can be written in the following form: in These correspond to the control forces / torques of the space robot's base and each joint. These are the position / angle vectors corresponding to the space robot's base and each joint. Here, q represents the velocity and acceleration vectors corresponding to q, and M(q) is the mass matrix of the system. It is a nonlinear matrix related to velocity; Based on the space robot dynamics model, select the system state variables. Where Δp e Let be the difference between the robot's end point and the desired position, and let flag be a marker indicating whether the robot's end point has reached the desired position. The system motion quantity is chosen as a = τ. During training, the target position is set as the endpoint. The success of robot motion planning depends on whether the robot's end point can reach the endpoint and remain stable. Therefore, the distance between the robot's end point and the target point is a crucial parameter for establishing the reward model. To enable the robot to quickly reach the target point, the single-step reward function during task planning is set as r = r1 + r2, where: r1=-ω||Δp e || (24); ω is a constant weight. Each time the robotic arm's end point reaches the target position, the robotic arm is given a constant reward of +1. S32. Design a training network based on the Deep Deterministic Policy Gradient (DDPG) algorithm. The DDPG algorithm adopts the Actor-Critic model, where the Actor makes decisions and the Critic makes evaluations: First, the state s is observed from the environment and passed to the Actor. Based on this state, the Actor makes a decision based on the existing policy and obtains the action a. After the action is applied to the environment, the environment will provide the reward feedback r for the current step and a new state. Based on the reward feedback r, the Critic will update the evaluation network for the Actor's behavior. The Actor then updates its own policy network along the direction suggested by the Critic. This completes one step of training, and then continues to loop until training is successful. Since neural networks require that the training data are independent, while the data obtained by the Actor through continuous interaction with the environment has a certain correlation, DDPG adopts an experience pool design. The data samples obtained from the interaction with the environment are stored in the experience pool, and then a set of data is randomly selected from it for training. This breaks the correlation between the data and achieves the independence of the samples. S33. Design the hyperparameters for learning rate, discount factor, experience pool size, training rounds, and maximum iteration steps in the network; S34. Complete the training of the space robot trajectory planning strategy based on the DDPG algorithm.

6. The on-orbit capture method for a space robot based on simulated binocular vision measurement according to claim 5, characterized in that, The method for training the spatial robot trajectory planning strategy based on the DDPG algorithm in step S34 specifically includes the following steps: S341. Actor, based on the current state s t Combine the current strategy μ to select action a t =μ(s) t |θ μ (and pass it to the environment to perform the action); S342. Environment executes action a t Transition to the new state s t+1 And generate a reward value r after the action is performed. t Return it to the Actor; S343.Actor combines (s t ,a t ,r t ,s t+1 The data is stored in the experience pool M as the dataset for training the network. S344. Randomly select N sets of data from the experience pool and use them as mini-batch data for training the real policy network and the real Q network; S345. The target policy network is based on (s i ,a i ,r i ,s i+1 The state s in ) i+1 Combined with network strategy θ μ′ Generate action μ′(s) i+1 The data is then fed into the target Q-network to calculate the target Q-value. The target Q-network calculates the target Q-value based on (s). i ,a i ,r i ,s i+1 ) and s i+1 The corresponding μ′(s) i+1 Calculate the target Q value: y i =r i +γQ′(s i+1 ,μ′(s i+1 |θ μ′ )|θ Q′ ) (26); Simultaneously, the real-world strategy network is combined with (s) i ,a i ,r i ,s i+1 s in ) i Generate action a i The corresponding Q-value Q(s) is calculated by inputting it into the real Q-network. i ,a i |θ Q Therefore, the loss function of the Q-network is defined as: L can be obtained with respect to θ Q gradient S346. Optimize and update the network parameters θ of the real-world Q network. Q ; S347. Calculate the gradient of the policy network. As can be seen from the above formula, the gradient update formula of the policy network consists of N sums of two-part multiplications. The first part is obtained by the Critic and is used to evaluate how the Actor should move to obtain a larger Q value. The second part is obtained by the Actor and is used to reflect how the Actor should update its own parameters so that the policy can take actions that can obtain a larger Q value. In general, it is to make the policy network learn to update its own parameters in the direction of obtaining a larger Q value. S348. Optimize and update the network parameters θ of the real-world policy network. μ ; S349. Analyze the network parameters θ of the target policy network and the target Q network respectively. μ′ and θ Q′ Perform a soft update: When the above steps are completed, the training within one time step is completed. Restarting the loop from step S341 begins the training for the next time step. After all time steps of training are completed, a complete round of training is completed. Through the continuous iteration of rounds, the network parameters will be updated and developed in the direction of maximizing the Q value.

Images