A method for grasping and positioning unmanned aerial vehicles on a mobile offshore platform

Abstract

This invention relates to a method for grasping and locating unmanned aerial vehicles (UAVs) on a maritime mobile platform. First, it establishes an imaging model and a time-reaction (RT) expression between control points in the UAV target coordinate system and imaging coordinates in the visual camera coordinate system. Multiple target circles are set on the UAV as markers. Then, n coplanar but non-collinear target circles are selected as control points, and the RT is solved using the established model. Simultaneously, the problem is transformed into a nonlinear optimization problem, and the LM algorithm is used to optimize and solve for the optimal iteration step size h to improve computational accuracy. When the UAV lands on the UAV platform, the visual camera acquires images of the target circles on the UAV, and the above steps can be used to solve for the UAV pose data to guide the grasping mechanism to complete the grasping process. This method has advantages such as high visual measurement accuracy and stable measurement results, and can meet the requirements for UAV grasping and recovery on maritime mobile platforms.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicles (UAVs), and more specifically to a method for capturing and locating UAVs on a mobile maritime platform. Background Technology

[0002] Unmanned aerial vehicles (UAVs) have a wide range of applications in the ocean, featuring remote monitoring and reconnaissance, rapid deployment and flexible maneuverability, all-weather operation, and reduced personnel risks. Current technologies primarily rely on experiments to test and optimize the performance of UAV systems under different sea conditions, ensuring efficient UAV recovery under various wave conditions, especially verifying the system's reliability and stability in simulated wave environments to guarantee the safety and accuracy of the recovery process.

[0003] In the process of drone capture and recovery, the traditional method relies on reference objects with known shapes and structures, such as control lines or control points, and uses mathematical transformation formulas and optimizations based on scene information to calibrate the camera. Then, the camera identifies cooperative markers on the drone to achieve spatial positioning and target tracking, finally guiding the capture device to adjust its posture to complete the drone capture and recovery operation. However, the above method has high requirements for scene information and camera movement. Offshore mobile platforms are usually in a state of turbulence caused by waves, and the scene information is even more complex when encountering severe weather or other emergencies. The above method often cannot meet the requirements of drone recovery.

[0004] A patent with authorization announcement number CN110044212B discloses a method for capturing and recovering a rotary-wing UAV based on visual measurement information. When the UAV enters the imaging range, a binocular vision system acquires image information of objects within the imaging area, processes and analyzes this information for judgment. When the UAV enters a clear imaging range, the binocular vision system uses a high-speed industrial camera to acquire and store image information of the UAV every Δt. After image processing, marker points on the UAV image are identified, and then the algorithm is used to obtain the UAV's pose vector at that moment, thereby achieving spatial positioning and target tracking of the UAV. Finally, during UAV capture, the corresponding pose of the capture device is adjusted through segmented trajectory fitting and target prediction. As described above, this method has high requirements for scene information and camera movement, and its control accuracy in complex sea conditions needs further improvement.

[0005] The patent with authorization announcement number CN114200948B discloses a vision-assisted autonomous landing method for UAVs. This method involves designing landing markers and using the UAV's own downward-facing camera to capture images of the landing markers and adjust its attitude to achieve autonomous landing. However, this method does not consider UAV capture and recovery. Summary of the Invention

[0006] The purpose of this invention is to provide a method for capturing and locating unmanned aerial vehicles (UAVs) on a maritime mobile platform. This method utilizes a visual camera to identify target circles marked on the UAV. Then, it obtains a translation matrix T and a rotation matrix R by constructing a model using the coordinates of points in the UAV target coordinate system and the imaging coordinates of corresponding points in the visual camera coordinate system. This allows for the calculation of the UAV's pose data to achieve UAV capture and recovery control. Furthermore, this invention uses the LM algorithm to calculate the optimal iteration step size, which has advantages such as high visual measurement accuracy and stable measurement results. This ensures the accuracy of UAV capture control and meets the UAV capture requirements of maritime mobile platforms.

[0007] The object of the present invention is achieved through the following technical solutions:

[0008] A method for capturing and locating unmanned aerial vehicles (UAVs) on a mobile maritime platform includes the following steps:

[0009] Step 1: Establish the measurement reference coordinate system, the UAV target coordinate system, and the visual camera coordinate system;

[0010] Step 2: Establish an imaging model and RT expression between a control point (X, Y, Z) in the UAV target coordinate system and the imaging coordinates (u, v) in the visual camera coordinate system, where R is the rotation matrix for the transformation between the measurement reference coordinate system and the visual camera coordinate system, and T is the translation matrix for the transformation between the measurement reference coordinate system and the visual camera coordinate system.

[0011] Step 3: The UAV is equipped with multiple target circles with marker points. Select n coplanar but non-collinear target circles as control points and solve RT in combination with the forming model, where n≥4;

[0012] Step 4: Transform the problem into a nonlinear optimization problem, and use the LM algorithm to optimize the solution for the optimal iteration step size h to improve computational accuracy;

[0013] Step 5: When the movement frequency of the drone platform reaches the set frequency, the drone lands on the drone platform. Then, the vision camera acquires images of the target circle marked on the drone and uses the above steps to solve the drone pose data to guide the grasping mechanism to complete the grasping.

[0014] In step two, the molding model is as shown in equation (1):

[0015]

[0016] In the above formula (1), fu and fv are the focal ratios of the visual camera, u0 and v0 are the principal point coordinates of the visual camera, fu, fv, u0 and v0 constitute the camera intrinsic parameter matrix, and s is the scaling factor.

[0017] RT is shown in equation (2) below:

[0018]

[0019] In equation (2) above, R includes r ij i = 0, 1, 2, j = 0, 1, 2, T includes tx, ty, tz;

[0020] In step three, for equation (1) above, setting the Z-direction component of the spatial coordinates on the right side of the equation to 0, we get:

[0021]

[0022] In equation (4) above, t0, t1, and t2 constitute the translation matrix T;

[0023] Let the product of the camera intrinsic parameter matrix containing fu, fv, u0, and v0 and the RT matrix be expressed as:

[0024]

[0025] Let h8 = 1.0 in equation (5) above, and the remaining elements in the H matrix be the unknowns to be solved;

[0026] Substituting equation (5) into equation (4) and rearranging, we get:

[0027]

[0028] Equation (6) above represents a control point in the UAV target coordinate system. At least four control points are required to complete the solution process.

[0029] Assuming the n control points are numbered 1, 2, 3…n, where n≥4, then equation (6) above is expanded to:

[0030]

[0031] make:

[0032]

[0033] Then the above equation (7) can be written as:

[0034] B·L=C (8);

[0035] Solving for equation (8) above, we get:

[0036] L(B T ·B) -1 ·B T ·C (9);

[0037] At this point, all elements of the H matrix have been solved, where (X1, Y1)...(X n Y nLet (u1, v1) represent the coordinates of each control point in the UAV target coordinate system, where (u1, v1)...(u...) n v n () represents the imaging coordinates of each control point in the visual camera coordinate system.

[0038] Moving the camera intrinsic parameter matrix on the right side of equation (5) to the left side of the equation, we get equation (10):

[0039]

[0040] Both sides of equation (10) above are 3×3 matrices with one-to-one correspondence of elements. Solving for the result, we get:

[0041]

[0042] r2 = r0 × r1

[0043]

[0044] In equation (11), the value of the proportional coefficient s is calculated by using the constraint that r0 and r1 satisfy the vector magnitude of 1.0, and then r0, r1, r2, and t are obtained, which is RT in equation (2).

[0045] In step three, the proportionality constant s is calculated as follows:

[0046] make:

[0047]

[0048] Then we have:

[0049] r0 = sA -1 H0

[0050] r1=sA -1 H1;

[0051] In the above formula, H0 is the matrix composed of h0, h3, and h6, H1 is the matrix composed of h1, h4, and h7, and r0 and r1 satisfy the constraint that the vector magnitude is 1.0. Then we have:

[0052] ||r0||=|||sA -1 H0||=1

[0053] ||r1||=l||sA -1 H1||=1;

[0054] Therefore, we get:

[0055]

[0056] Step four transforms the problem into a nonlinear optimization problem, specifically as follows:

[0057] The general form of the nonlinear least squares problem is shown in equation (12):

[0058]

[0059] x * =aegmin x {F(x)} (12);

[0060] In equation (12) above, f: R n →R m m→n, where n represents the number of independent variables and m represents the number of equations, x * This represents the optimal solution;

[0061] The control point reprojection error is used as f(x), as shown in equation (13):

[0062]

[0063] In the above formula (13), (u′) i ,v′ i (u) represents the imaging coordinates of the i-th control point calculated based on the updated value of the current optimization variable during each iteration. i ,v i ) represents the actual imaging coordinates of the i-th control point, and m represents the number of control points;

[0064] Step four involves finding the optimal iteration step size h as follows:

[0065] Expanding f in equation (12) above using Taylor series, we get:

[0066] f(x+h)=f(x)+J(x)h+O(||h|| 2 (14);

[0067] In equation (14) above, h represents the step size, O(||h|| 2 ) represents the Taylor expansion term, J∈R m×n It is a Jacobian matrix;

[0068] make Where “≡” indicates that it is always equal to, then we have:

[0069]

[0070] In equation (15) above, f = f(x), J = J(x), and from equation (15) above, it can be seen that the process of determining the iteration step size h is transformed into the process of searching for the minimum value of L(h), where:

[0071] L′(h)=J T f+J T Jh;

[0072] When L(h) reaches an extreme value, we have L ′ (h) = 0, and then the optimal step size h is obtained.

[0073] In step four, a damping term is introduced to limit the amplitude of h during the solution process for the step size h:

[0074]

[0075] In equation (16) above, h lm represents the step size after optimization by the LM algorithm, argmin represents the parameter value that minimizes or minimizes the function value, and μ represents the damping coefficient;

[0076] Differentiating equation (16) above, we get:

[0077]

[0078] Setting the derivative of equation (17) to 0, we get:

[0079] (J T J+μI)h lm =-J T f, μ≥0 (18);

[0080] In equation (18) above, I is the identity matrix. Then, the steepest descent method or the Gauss-Newton method is used to solve the problem based on the value of μ.

[0081] The advantages and positive effects of this invention are as follows:

[0082] 1. This invention utilizes a visual camera to identify target circles on a drone. Then, it obtains the translation matrix T and rotation matrix R by constructing a model using the coordinates of the point in the drone's target coordinate system and the imaging coordinates of the corresponding point in the visual camera's coordinate system. This allows the coordinates of the target circle in the measurement reference coordinate system to be obtained, which in turn obtains the drone's pose data to achieve drone capture and recovery control.

[0083] 2. This invention uses the reprojection error of the control point (marker target circle) as f(x) to transform the problem into a nonlinear optimization problem, and uses the LM algorithm to calculate the optimal iteration step size h to improve the calculation accuracy, thereby ensuring the accuracy of UAV grasping control. Attached Figure Description

[0084] Figure 1 A schematic flowchart of the method of the present invention.

[0085] Figure 2 This is a schematic diagram of a gripping mechanism using the method of the present invention.

[0086] Figure 3 for Figure 2Top view of the grabbing mechanism.

[0087] Figure 4 for Figure 3 A top-down view of the drone in the image.

[0088] Figure 5 for Figure 2 A schematic diagram showing the installation location of the vision camera on the grasping mechanism.

[0089] Figure 6 This is a comparison chart of the measurement accuracy of the method of the present invention and the existing Zhang Zhengyou method.

[0090] Among them, 1 is the drone, 101 is the target circle of the marker point, 2 is the grasping mechanism, 201 is the flange, 202 is the moving truss, 3 is the drone platform, and 4 is the vision camera. Detailed Implementation

[0091] The invention will now be described in further detail with reference to the accompanying drawings.

[0092] like Figure 1 As shown, the method of the present invention includes the following steps:

[0093] Step 1: Establish the measurement reference coordinate system, the UAV target coordinate system, and the visual camera coordinate system.

[0094] like Figures 2-5 As shown, the device of the present invention includes a drone 1, a grasping mechanism 2, and a drone platform 3, wherein the drone 1 is mounted on the drone platform 3, and as... Figure 5 As shown, the visual camera 4 is mounted on the gripping mechanism 2, which has three degrees of freedom control capability, enabling precise translational control in the X, Y, and Z directions. Through coordinated control of these three directions, the mechanical gripper of the gripping mechanism 2 can move flexibly in complex spatial environments. The UAV platform 3 can achieve roll angle changes of ±10° and pitch angle changes of ±5°, and the swing frequency is stably maintained at a level of not less than 0.1 Hz to coordinate with the turbulence of ocean waves. Both the gripping mechanism 2 and the UAV platform 3 are technologies known in the art and are commercially available products.

[0095] In this embodiment, the measurement reference coordinate system O B -X B Y B Z B The establishment of such as Figure 3 and Figure 5 As shown, it is located at the center of the flange 201 at the lower end of the moving truss 202 of the gripping mechanism 2, in coordinate system X. B The direction is consistent with the forward and backward movement direction of the truss, with the backward movement direction being positive. The coordinate system is Y. B The direction is consistent with the horizontal movement direction of the truss, and the leftward movement direction is positive, ZB Direction follows the right-hand rule.

[0096] In this embodiment, the UAV target coordinate system O A -X A Y A Z A The establishment of such as Figure 4 As shown, the UAV has multiple target circles 101 marked on it, and the origin of the UAV target coordinate system is O. A The distance between X and the nearest target circle 101 is L, X A and X B The direction is parallel and backward is positive, Y A and Y B The direction is parallel and to the left is positive, Z A The direction follows the right-hand rule. Additionally, when the center of the flange 201 at the lower end of the grabbing mechanism 2 truss is aligned with the UAV target coordinate system O... A When the gripper arms on both sides of the lower end of the grasping mechanism 2 are aligned and parallel to the central axis of the drone, the object being measured can be accurately grasped. For example... Figure 5 As shown, the length of the vision camera support arm is equal to L, and the design principle of each marker target circle 101 is: when the mechanical claw on the lower side of the gripping mechanism 2 accurately grips the object being measured, the marker target circle 101 must appear in the center of the field of view of the vision camera 4 without being obstructed.

[0097] Visual camera coordinate system O C -X C Y C Z C The establishment of such as Figure 5 As shown, the vision camera 4 is fixed on the moving truss 202 of the gripping mechanism 2, with the origin O. C Built on visual camera 4, X C and X B The direction is parallel and backward is positive, Y C and Y B The direction is parallel and to the left is positive, Z C Direction follows the right-hand rule.

[0098] The relationship between the coordinate systems is as follows: the measurement reference coordinate system is translated (translation matrix T) and rotated (rotation matrix R) to obtain the visual camera coordinate system. The imaging coordinate relationship between points in the UAV target coordinate system and corresponding points in the visual camera coordinate system is shown in the model forming process described below. The method of this invention sets N control points (i.e., target circles 101) in the UAV target coordinate system (i.e., the scene). After imaging by the visual camera 4, N corresponding feature points are formed. Then, the camera model intrinsic parameter matrix, the imaging coordinates of the N feature points, and the corresponding target model parameters (i.e., the three-dimensional coordinates of the N control points in the UAV target coordinate system) are used to solve RT. The coordinates of the feature points in the visual camera coordinate system in the measurement reference coordinate system are then obtained through RT and sent to the equipment control system to control the grasping mechanism 2 to move and grasp the UAV 1.

[0099] Step 2: Establish the imaging model and RT expression between a control point (X, Y, Z) in the UAV target coordinate system and the imaging coordinates (u, v) in the visual camera coordinate system.

[0100] The molding model is shown in equation (1) below:

[0101]

[0102] In equation (1) above, fu and fv are the focal ratios of the visual camera 4, which are the ratios of the camera's imaging focal length to the pixel size of the imaging target surface; u0 and v0 are the principal point coordinates of the visual camera 4. fu and fv, as well as u0 and v0, need to be obtained in advance through camera intrinsic parameter calibration and form the camera intrinsic parameter matrix in equation (1) above. This is a well-known technique in the field. In addition, in equation (1) above, s is the scaling factor.

[0103] The RT expression is shown in equation (2) below:

[0104]

[0105] In equation (2) above, R is the rotation matrix (r ij , i = 0, 1, 2, j = 0, 1, 2), which describes the rotation direction of the coordinate axes of the measurement reference coordinate system relative to the coordinate axes of the vision camera coordinate system. T is the translation matrix (tx, ty, tz), which describes the position of a point (such as the origin) in the measurement reference coordinate system in the vision camera coordinate system.

[0106] In addition, in equation (2) above, the R matrix contains 9 elements, but only three relatively independent variables (ax, ay, az), where ax, ay, and az represent the rotation angles around the x-axis, y-axis, and z-axis, respectively. The correspondence between ax, ay, az and R is as follows:

[0107] R = Rz·Ry·Rx

[0108]

[0109] Rx, Ry, and Rz represent the offset and rotation angles of any point in the UAV target coordinate system in the XYZ directions of the measurement reference coordinate system.

[0110] Step 3: Solve for RT.

[0111] exist Figure 4 From all points on the target circle 101 shown, select n coplanar but non-collinear marker points, n≥4. For the above equation (1), let the Z-direction component of the spatial coordinates on the right side of the equation be 0 (the three-dimensional coordinates of coplanar points are all 0 in the Z-direction), we can obtain:

[0112]

[0113] In equation (4) above, t0, t1, and t2 form a translation matrix T, which, like tx, ty, and tz, represents the position of a point in space in the visual camera coordinate system.

[0114] Let the product of the camera intrinsic parameter matrix containing fu, fv, u0, and v0 and the RT matrix be expressed as:

[0115]

[0116] Let h8 = 1.0 in equation (5) above, and the remaining elements in the H matrix be the unknowns to be solved. h8 can be set to any number without affecting the formula relationship. Setting it to 1.0 is only for convenience of calculation. If it is set to other numbers, the solution result will be scaled according to h8.

[0117] Substituting equation (5) into equation (4) and rearranging, we get:

[0118]

[0119] Equation (6) above represents a control point in the UAV target coordinate system. This invention requires at least four control points (four marker points, target circle 101) to complete the solution process, specifically:

[0120] Assuming the n marker points of the target circle 101 are numbered 1, 2, 3…n, where n≥4, then equation (6) above can be expanded to:

[0121]

[0122] make:

[0123]

[0124] Then the above equation (7) can be written as:

[0125] B·L=C (8);

[0126] Solving the system of least squares linear equations yields the following:

[0127] L = (B T ·B) -1 ·B T ·C (9);

[0128] At this point, all elements of matrix H have been solved (i.e., L), where the least squares linear equation system is a well-known technique in mathematics, (X1, Y1)...(X n Y n ) represents the coordinates of each control point (target circle 101) in the UAV target coordinate system, (u1, v1)...(u n v n ) represents the imaging coordinates of each control point in the vision camera coordinate system. All of these values ​​can be obtained through the vision camera system.

[0129] Moving the camera intrinsic parameter matrix on the right side of equation (4) to the left side of the equation, we get equation (10). Since h8 = 1.0 is used in the H matrix solution process, there will be a difference in the scaling factor s between the two sides of equation (10):

[0130]

[0131] Both sides of equation (10) above are 3×3 matrices with one-to-one correspondence of elements. Solving for the result, we can obtain:

[0132]

[0133] r2 = r0 × r1

[0134]

[0135] By utilizing the constraint that r0 and r1 satisfy the vector magnitude of 1.0, the value of the scaling factor s can be calculated, specifically as follows:

[0136] make:

[0137]

[0138] Then we have:

[0139] ||r0||=||sA -1 H0||=1

[0140] ||r1||=||sA -1 H1||=1;

[0141] In the above formula, H0 is the matrix composed of h0, h3, and h6, and H1 is the matrix composed of h1, h4, and h7, thus yielding:

[0142]

[0143] Then, according to the above formula (11), the values ​​of translation t and the third column vector r2 of the rotation matrix can be further obtained. r0, r1, r2, t, that is, RT.

[0144] Step 4: Transform the problem into a nonlinear optimization problem, and use the LM algorithm to optimize the solution for the optimal iteration step size h to improve computational accuracy.

[0145] Step 4.1: First, transform the problem into a nonlinear optimization problem, specifically:

[0146] The general form of the nonlinear least squares problem is shown in equation (12):

[0147]

[0148] x * =argmin x {F(x)} (12);

[0149] In equation (12) above, f: R n →R m m>n, where n represents the number of independent variables and m represents the number of equations, x * This represents the optimal solution.

[0150] The present invention uses the reprojection error of the control point (marker target circle 101) as f(x), as shown in the following equation (13):

[0151]

[0152] In the above formula (13), (u′) i ,v′ i ) represents the imaging coordinates of the i-th control point calculated based on the updated value of the current optimization variable during each iteration, which can be calculated using equations (2) and (3) above. i ,v i ) represents the actual imaging coordinates of the i-th control point, and m represents the number of control points.

[0153] Step 4.2: Use the LM algorithm to calculate the optimal iteration step size h to improve calculation accuracy.

[0154] This optimization step mainly involves determining the update step size (including direction and magnitude) of the optimization variables in each iteration, specifically:

[0155] A Taylor expansion of f in equation (12) yields:

[0156] f(x+h)=f(x)+J(x)h+O(‖h‖2 (14);

[0157] In the above formula (14), h represents the step size, O(‖h‖ 2 ) represents the Taylor expansion term, J∈R m×n The Jacobian matrix is:

[0158]

[0159] make Where “≡” indicates that it is always equal to, then we have:

[0160]

[0161] In equation (15) above, f = f(x), J = J(x), and from equation (15) above, it can be seen that the process of determining the iteration step size h is transformed into the process of searching for the minimum value of L(h):

[0162] L′(h)=J T f+J T Jh, L″(h) = J T J;

[0163] When L(h) reaches an extreme value, L′(h) = 0, and then the optimal step size h can be obtained.

[0164] In addition, the LM optimization algorithm of this invention introduces a damping term in the process of solving the step size h, as shown in equation (16) below, the purpose of which is to limit the amplitude of h:

[0165]

[0166] In equation (16) above, h lm represents the step size after optimization by the LM algorithm, argmin represents the parameter value that minimizes or minimizes the function value, and μ represents the damping coefficient.

[0167] Differentiating equation (16) above, we get:

[0168]

[0169] Setting the derivative of equation (17) to 0, we get:

[0170] (J T J+μI)h lm =-J T f, μ≥0 (18);

[0171] In equation (18) above, I represents the identity matrix;

[0172] For equation (18) above, when m>0, (J T J+μI) is a positive definite symmetric matrix, which guarantees hlm It moves in the direction of decreasing the objective function; when μ is very large, we have the following equation (19), and the steepest descent method is used to solve for the value. When μ is very small, we have the following equation (20), where h gn The original step size is used, and the solution is obtained using the Gauss-Newton method.

[0173]

[0174] The steepest descent method converges quickly when the initial value deviates significantly from the optimal solution. However, its convergence speed decreases considerably as the optimization variable approaches the optimal solution. In contrast, the Gauss-Newton method achieves good convergence at this point. This invention's LM optimization algorithm combines the steepest descent method and the Gauss-Newton method, leveraging their respective advantages to complement each other. The steepest descent method and the Gauss-Newton method are well-known techniques in this field and can be found in Chapter 5 of "Nonlinear Optimization Theory and Methods" (4th Edition), authored by Wang Yiju and Xiu Naihua.

[0175] Furthermore, in equation (20), the Taylor second-order expansion L(h) is used to approximate F(x+h). For the same size h, during the current iteration, when the optimization variable is far from the extreme point x... * When L(h) approximates F(x+h) less closely, it is desirable for the iterative process to more closely resemble the steepest descent method. Therefore, the value of μ can be appropriately increased. During the current iteration, when the optimization variable approaches the extreme point x... * When L(h) approximates F(x+h) to a high degree, it is desirable for the iterative process to be closer to the Gauss-Newton method, so the value of μ can be appropriately reduced.

[0176] The LM algorithm of this invention uses the following equation (21) to describe the approximation of ΔF and ΔL, where ρ represents the gain rate.

[0177]

[0178] △F=F(x)-F(x+h)

[0179] ΔL=L(0)-L(h) (21);

[0180] Step 5: When the motion frequency of the UAV platform 3 reaches the set frequency, the UAV 1 lands on the UAV platform 3. Then, the vision camera 4 acquires images of the target circle 101 marked on the UAV 1, and uses the UAV pose data obtained through the above steps to guide the grasping mechanism 2 to complete the grasping. Specifically:

[0181] The visual camera 4 acquires images of the target circle 101 on the UAV 1 and obtains the coordinates (X, Y, Z) of the target circle 101 in the UAV's own coordinate system and the imaging coordinates (u, v) in the visual camera coordinate system through the software system. Then, the H matrix is ​​obtained according to the above formula (9), and RT is obtained according to the above formula (11). Finally, the imaging coordinates (u, v) in the visual camera coordinate system are converted into coordinates in the measurement reference coordinate system through RT and sent to the equipment control system to control the gripping mechanism 2 to move and grip the UAV 1.

[0182] Furthermore, this invention uses the reprojection error of the control point (marker target circle 101) as f(x) to transform the problem into a nonlinear optimization problem, and utilizes the LM algorithm to calculate the optimal iteration step size h to improve computational accuracy, such as... Figure 6 As shown, the accuracy of the calibration method of the present invention is significantly better than that of the Zhang Zhengyou calibration method commonly used in the prior art.

Claims

1. A method for capturing and locating unmanned aerial vehicles (UAVs) on a maritime mobile platform, characterized in that: Includes the following steps: Step 1: Establish the measurement reference coordinate system, the UAV target coordinate system, and the visual camera coordinate system; The origin of the measurement reference coordinate system is established at the center of the flange (201) at the lower end of the moving truss (202) of the gripping mechanism (2), and the coordinate system X B The direction is consistent with the forward and backward movement direction of the truss, with the backward movement direction being positive. The coordinate system is Y. B The direction is consistent with the horizontal movement direction of the truss, and the leftward movement direction is positive, Z B Direction follows the right-hand rule; The UAV is equipped with multiple target circles (101) marked by markers. The distance between the origin of the UAV target coordinate system and the nearest target circle (101) is L, X A and X B The direction is parallel and backward is positive, Y A and Y B The direction is parallel and positive to the left, Z A The direction follows the right-hand rule; the design principle of each marker target circle (101) is: when the mechanical claw on the lower side of the gripping mechanism (2) accurately grips the object being measured, the marker target circle (101) can appear in the middle of the field of view of the vision camera (4) without being obstructed. The vision camera (4) is fixed on the moving truss (202) of the gripping mechanism (2), and the origin of the vision camera coordinate system is established on the vision camera (4). C and X B The direction is parallel and backward is positive, Y C and Y B The direction is parallel and positive to the left, Z C Direction follows the right-hand rule; The gripping mechanism (2) has three degrees of freedom control capability, and can perform precise translation control in the X, Y and Z directions respectively; Step 2: Establish an imaging model and RT expression between a control point (X, Y, Z) in the UAV target coordinate system and the imaging coordinates (u, v) in the visual camera coordinate system, where R is the rotation matrix for the transformation between the measurement reference coordinate system and the visual camera coordinate system, and T is the translation matrix for the transformation between the measurement reference coordinate system and the visual camera coordinate system. In this step, the molding model is as shown in equation (1): In the above formula (1), fu and fv are the focal ratios of the visual camera, u0 and v0 are the principal point coordinates of the visual camera, fu, fv, u0 and v0 constitute the camera intrinsic parameter matrix, and s is the scaling factor. RT is shown in equation (2) below: In equation (2) above, R includes r ij i = 0, 1, 2, j = 0, 1, 2, T includes tx, ty, tz; Step 3: Select n coplanar but non-collinear target circles (101) as control points and solve RT in combination with the forming model, where n≥4; In this step, for equation (1) above, setting the Z-direction component of the spatial coordinates on the right side of the equation to 0, we get: In equation (4) above, t0, t1, and t2 constitute the translation matrix T; Let the product of the camera intrinsic parameter matrix containing fu, fv, u0, and v0 and the RT matrix be expressed as: Let h8 = 1.0 in equation (5) above, and the remaining elements in the H matrix be the unknowns to be solved; Substituting equation (5) into equation (4) and rearranging, we get: Equation (6) above represents a control point in the UAV target coordinate system. At least four control points are required to complete the solution process. Assuming the n control points are numbered 1, 2, 3…n, where n≥4, then equation (6) above is expanded to: make: Then the above equation (7) can be written as: B·L=C (8); Solving for equation (8) above, we get: L=(B T ·B) -1 ·B T ·C (9); At this point, all elements of the H matrix have been solved, where (X1, Y1)...(X n , Y n Let (u1, v1) represent the coordinates of each control point in the UAV target coordinate system, where (u1, v1)...(u...) n , v n () represents the imaging coordinates of each control point in the visual camera coordinate system. Moving the camera intrinsic parameter matrix on the right side of equation (5) to the left side of the equation, we get equation (10): Both sides of equation (10) above are 3×3 matrices with one-to-one correspondence of elements. Solving for the elements, we get: r2＝r0×r1 In the above formula (11), the value of the proportional coefficient s is calculated by using the constraint condition that r0 and r1 satisfy the vector magnitude of 1.0, and then r0, r1, r2, t are obtained, which is RT in the above formula (2); Step 4: Transform the problem into a nonlinear optimization problem, and use the LM algorithm to optimize the solution for the optimal iteration step size h to improve computational accuracy; Step 5: When the movement frequency of the drone platform reaches the set frequency, the drone lands on the drone platform. Then, the vision camera acquires images of the target circle marked on the drone and uses the above steps to solve the drone pose data to guide the grasping mechanism to complete the grasping.

2. The method for capturing and locating unmanned aerial vehicles (UAVs) on a maritime mobile platform according to claim 1, characterized in that: In step three, the proportionality constant s is calculated as follows: make: Then we have: r0=sA -1 H0 r1=sA -1 H1; In the above formula, H0 is the matrix composed of h0, h3, and h6, H1 is the matrix composed of h1, h4, and h7, and r0 and r1 satisfy the constraint that the vector magnitude is 1.

0. Then we have: ||r0||=||sA -1 H0||=1 ||r1||=||sA -1 H1||=1; Therefore, we get:

3. The method for capturing and locating unmanned aerial vehicles (UAVs) on a maritime mobile platform according to claim 1, characterized in that: Step four transforms the problem into a nonlinear optimization problem, specifically as follows: The general form of the nonlinear least squares problem is shown in equation (12): x * =argmin x {F(x)} (12); In equation (12) above, f: R n →R m m > n, where n represents the number of independent variables and m represents the number of equations, x * This represents the optimal solution; The control point reprojection error is used as f(x), as shown in equation (13): In the above formula (13), (u′) i ,v′ i (u) represents the imaging coordinates of the i-th control point calculated based on the updated value of the current optimization variable during each iteration. i ,v i ) represents the actual imaging coordinates of the i-th control point, and m represents the number of control points.

4. The method for capturing and locating unmanned aerial vehicles (UAVs) on a maritime mobile platform according to claim 3, characterized in that: Step four involves finding the optimal iteration step size h as follows: Expanding f in equation (12) above using Taylor series, we get: f(x+h)=f(x)+J(x)h+O(||h|| 2 ) (14); In equation (14) above, h represents the step size, O(||h|| 2 ) represents the Taylor expansion term, J∈R m×n It is a Jacobian matrix; make Where "≡" represents an exact equality, then we have: In equation (15) above, f = f(x), J = J(x), and from equation (15) above, it can be seen that the process of determining the iteration step size h is transformed into the process of searching for the minimum value of L(h), where: L′(h)=J T f+J T Jh; When L(h) reaches an extreme value, L′(h) = 0, and then the optimal step size h can be obtained.

5. The method for capturing and locating unmanned aerial vehicles (UAVs) on a maritime mobile platform according to claim 4, characterized in that: In step four, a damping term is introduced to limit the amplitude of h during the solution process for the step size h: In equation (16) above, h lm represents the step size after optimization by the LM algorithm, argmin represents the parameter value that minimizes or minimizes the function value, and μ represents the damping coefficient; Differentiating equation (16) above, we get: Setting the derivative of equation (17) to 0, we get: (J T J+μI)h lm =-J T f,μ≥0 (18); In equation (18) above, I is the identity matrix. Then, the steepest descent method or the Gauss-Newton method is used to solve the problem based on the value of μ.

Images