On-orbit calibration method for space target pose measurement system based on vision and laser fusion

Abstract

The present invention provides an on-orbit calibration method for a space target posture measurement system that integrates vision and laser, which is used to solve the technical problems that the existing calibration methods based on special calibration objects such as chessboards have poor versatility in setting calibration objects for specific usage scenarios, or the existing calibration methods based on ordinary scenarios have large computational complexity and high requirements for computing resources. The on-orbit calibration method for a space target posture measurement system that integrates vision and laser provided by the present invention first establishes a joint calibration model of a visual sensor and a laser rangefinder, and on this basis, improves the traditional orthogonal iteration algorithm from two aspects: calculation of iterative initial values ​​and simplification of iterative algorithms, so as to solve and optimize the posture conversion relationship between the laser coordinate system and the visual coordinate system. It only uses a part of a non-cooperative target in space as a laser projection target, and can achieve on-orbit high-precision calibration of the joint posture measurement system of a visual sensor and a laser rangefinder.

Description

Technical Field

[0001] The present invention relates to the field of space non-cooperative target posture measurement, and in particular to an on-orbit calibration method for a space target posture measurement system that integrates vision and laser. Background Art

[0002] At present, non-cooperative targets in space usually use multi-sensor information fusion technology to achieve high-precision target pose measurement. For example, the joint measurement system that integrates a monocular vision sensor and a laser rangefinder can combine high lateral resolution vision and high-precision distance information to achieve high-precision pose estimation. In actual tasks, the joint measurement system of space target poses operates in a harsh space environment for a long time, and the relative pose relationship between the vision sensor and the laser rangefinder will change with different space tasks. For the joint measurement system of the vision sensor and the laser rangefinder, the pose conversion relationship between the coordinate system of the vision sensor and the coordinate system of the laser rangefinder can be obtained by high-precision calibration of the vision sensor and the laser rangefinder. Therefore, it is very necessary to perform on-orbit calibration of the relative pose relationship between the vision sensor and the laser rangefinder in the joint measurement system.

[0003] At present, according to the different types of calibration objects, on-orbit calibration methods can be roughly divided into the following three categories:

[0004] 1. The first category is the joint calibration method based on Zhang Zhengyou's chessboard. Zhang Q et al. proposed a joint calibration method for camera and 2D laser scanner based on chessboard. This method uses camera and laser scanner to measure the chessboard plane respectively, sets the distance from laser point to plane as constraint condition, and solves the required calibration parameters. However, the accuracy of this method is greatly affected by the angle between the plane where the laser point is located and the calibration plane. Unnikrishnan R et al. continued to study the joint calibration method of camera and 3D laser scanner on the basis of Zhang Q. Since the measurement system uses 3D laser scanner, multiple observations are required to obtain more data. Similar to the way Zhang Q et al. set constraint conditions, Vasconcelos F et al. obtained the calibration parameters by freely moving the chessboard and taking the depth at the coplanar line as constraint condition. On this basis, the solution of calibration parameters was reformulated as an n-point perspective (PnP) problem. However, this method has the problems of limited accuracy and numerical instability. Zhou Lipu et al. improved the above problem by estimating the pose parameters using the correspondence between three pairs of lines and planes as constraints, and obtained a more accurate solution. However, its robustness is still subject to certain limitations, and this method requires the use of a large chessboard.

[0005] 2. The second category is the calibration method based on special calibration objects. Tang et al. proposed a calibration method for the orientation parameters of laser ranging sensors based on spherical fitting. The calibration accuracy of this scheme is high, but the versatility is poor. Li G et al. used a right-angled triangular plate as a calibration object, and took the invisible intersection of the laser ranging sensor slice plane and the edge of the calibration plane as a constraint. The parameters were calibrated by minimizing the algebraic error between the measured intersection and its corresponding projection on the camera image plane. Rodriguez F et al. used a calibration plate with two concentric black rings and used the geometric properties of concentric circles as constraints for parameter calibration. Aliakbarpour H et al. used a freely moving high-brightness light source as a calibration object for matching, thereby achieving joint calibration of the camera and laser scanner. Zhou A et al. proposed a calibration method using conical and cylindrical surfaces as calibration objects, and used the geometric properties of the constant cone angle or cylinder diameter at multiple different positions and the constant position of the cone node at multiple different measurements as constraints to complete the parameter calibration. The accuracy of these algorithms has generally been improved to a certain extent compared to traditional checkerboard-based calibration methods, but in general, the calibration objects they require are set for specific usage scenarios, with strong customization and poor versatility.

[0006] 3. The third category is the calibration method based on common calibration objects. Wasielewski S et al. proposed a calibration method based on objects with V-shaped geometric features. The calibration parameters were obtained by calculating the intersection of the laser scanner scanning plane and the V-shaped calibration object and minimizing the distance from the projection point to the line segment detected in the image. This scheme requires a large number of observations, and the mismatch between the scanning point and the observation line will cause the error to increase. In order to improve the accuracy of the algorithm, Kwak K et al. reduced the impact of mismatch on the calibration results through the Huber weighting function. On this basis, Gomez-Ojeda R et al. proposed a calibration method based on objects with orthogonal trihedron geometric features. By setting two types of constraints, line to plane and point to plane, the problem of large errors caused by the single V-shaped geometric feature constraint was solved. Scaramuzza et al. proposed a joint calibration method for panoramic cameras and laser scanners. The calibration parameters were obtained by matching the feature points in the panoramic camera with the feature point distance information obtained by the laser scanner, and solving them using the PnP algorithm. This method has a large amount of calculation and requires high computing resources.

[0007] In summary, the existing research on calibration methods for joint measurement systems of visual sensors and laser rangefinders mainly focuses on calibration methods based on specific calibration objects such as chessboards. This type of method has a fast calibration speed and high accuracy, but it is more dependent on the choice of calibration objects. Calibration methods based on ordinary scenes often have large computational complexity and high requirements for computing resources, and cannot meet the requirements for high-precision and fast calibration of joint posture measurement systems under limited computing resources. Summary of the invention

[0008] The purpose of the present invention is to solve the technical problems that the existing calibration methods based on special calibration objects such as chessboards have poor versatility in setting the calibration objects for specific usage scenarios, or the existing calibration methods based on ordinary scenarios have large computational complexity and high requirements on computing resources, and to provide an on-orbit calibration method for a space target posture measurement system that integrates vision and laser.

[0009] In order to achieve the above purpose, the technical solution provided by the present invention is as follows:

[0010] An on-orbit calibration method for a space target posture measurement system integrating vision and laser is special in that it includes the following steps:

[0011] Step 1: Establish a joint calibration model of visual sensor and laser rangefinder;

[0012] Define the coordinate system of the laser rangefinder as the laser coordinate system , the coordinate system where the visual sensor is located is the visual coordinate system , taking the laser projection direction as the laser coordinate system The Z axis of the visual sensor is used as the visual coordinate system. Z axis; laser coordinate system and visual coordinate system There is a rotation matrix and translation vectors ;

[0013] Step 2, solving the spatial position of the laser spot;

[0014] According to the joint calibration model established in step 1, the laser coordinate system at the initial measurement position is solved respectively. The laser spot space point is in the visual coordinate system The corresponding points and the laser coordinate system The image point corresponding to the laser spot space point on the normalized imaging plane is obtained by synchronously changing the measurement positions of the laser rangefinder and the visual sensor for multiple times, solving and obtaining multiple groups of points corresponding to the laser spot space point in the corresponding visual coordinate system after changing the measurement position and the image point corresponding to the laser spot space point on the corresponding normalized imaging plane;

[0015] Step 3, solving and optimizing the posture transformation relationship between the laser coordinate system and the visual coordinate system;

[0016] The points corresponding to the laser spot space points in the corresponding visual coordinate system and the image points corresponding to the laser spot space points on the corresponding normalized imaging plane obtained in step 2 are used as constraint conditions, and the posture conversion relationship between the laser coordinate system and the visual coordinate system is solved and optimized through an improved orthogonal iterative algorithm; the improved orthogonal iterative algorithm includes:

[0017] a. Calculate the rotation matrix using the initial value calculation method based on the parallel perspective projection model The initial value of the iteration ;

[0018] b. Perform orthogonal iteration through the following simultaneous equations to obtain :

[0019]

[0020]

[0021]

[0022]

[0023] in, , , n is the number of iterations, is the pose matrix of the laser coordinate system at the k+1th position in the laser coordinate system at the kth position, For the The coordinates of the laser spot space points, I is the unit matrix, is the projection matrix of the image point, The initial value of the iteration is , is the projection point after the mth iteration, is the transpose of the projection point after the mth iteration, , ;

[0024] c. Calculate the final translation vector using the following formula:

[0025] .

[0026] Furthermore, in step 2, the laser coordinate system is solved by the following formula: The laser spot space point is in the visual coordinate system The corresponding point :

[0027]

[0028] in, is the spatial point of the laser spot at the initial measurement position.

[0029] Furthermore, in step 2, the laser coordinate system is solved by the following formula: The image point corresponding to the spatial point of the laser spot on the normalized imaging plane :

[0030]

[0031] in, K is the internal parameter matrix of the visual sensor, Z For image point scale information.

[0032] Furthermore, in step 3, the rotation matrix is ​​calculated using the initial value calculation method based on the parallel perspective projection model The initial value of the iteration for:

[0033]

[0034] in, is the depth information at the target center of mass, , , , are the x-direction and y-direction components of the image point coordinates corresponding to the target centroid coordinates of all laser spot space point sets, is the third-order identity matrix, , They are and The antisymmetric matrix of , , are the rotation components of the rotation matrix in the three degrees of freedom directions, is the translation amount of the translation vector in the z direction.

[0035] Compared with the prior art, the present invention has the following beneficial effects:

[0036] 1. The on-orbit calibration method of the space target posture measurement system with vision and laser fusion provided by the present invention first establishes a joint calibration model of the visual sensor and the laser rangefinder. On this basis, the traditional orthogonal iterative algorithm is improved from two aspects: the calculation of the initial value of the iteration and the simplification of the iterative algorithm, so as to solve and optimize the posture conversion relationship between the laser coordinate system and the visual coordinate system. The present invention only uses a part of the space non-cooperative target as the laser projection target to achieve on-orbit high-precision calibration of the joint posture measurement system of the visual sensor and the laser rangefinder.

[0037] 2. Compared with the on-orbit calibration method based on the traditional orthogonal iterative algorithm, the on-orbit calibration method of the space target posture measurement system integrating vision and laser provided by the present invention has been verified to have better performance in terms of accuracy and computational efficiency.

[0038] 3. The on-orbit calibration method of the space target posture measurement system integrating vision and laser provided by the present invention does not require specific markers and has the advantages of simple algorithm, strong real-time performance, good anti-noise interference performance and robustness. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 It is a schematic diagram of the joint calibration model established in step 1 of the embodiment of the on-orbit calibration method of the space target posture measurement system integrating vision and laser according to the present invention;

[0040] Figure 2 Schematic diagram of the principle of solving the posture transformation relationship between the laser coordinate system and the visual coordinate system based on orthogonal iteration in step 3 of the embodiment of the present invention;

[0041] Figure 3 The figure is a schematic diagram of the curves of attitude angle error changing with the number of data collection before and after the orthogonal iteration algorithm is improved, where the blue curve is the theta angle (azimuth angle) error before improvement, the orange curve is the psi angle (yaw angle) error before improvement, the pink curve is the phi angle (roll angle) error before improvement, the purple curve is the theta angle error after improvement, the green curve is the psi angle error after improvement, and the red curve is the phi angle error after improvement;

[0042] Figure 4 Figure 1 is a schematic diagram of the curve of the position error changing with the number of data acquisitions before and after the orthogonal iterative algorithm is improved, where the blue curve is the position error in the x-axis direction of the improved frontier visual coordinate system, the orange curve is the position error in the y-axis direction of the improved frontier visual coordinate system, the pink curve is the position error in the z-axis direction of the improved frontier visual coordinate system, the purple curve is the error in the x-axis direction of the visual coordinate system after improvement, the green curve is the error in the y-axis direction of the visual coordinate system after improvement, and the red curve is the error in the z-axis direction of the visual coordinate system after improvement;

[0043] Figure 5 It is a schematic diagram of the curve of the average number of iterations changing with the number of data collection before and after the improvement of the orthogonal iterative algorithm, where the blue curve is the curve of the average number of iterations changing with the number of data collection before the improvement, and the orange curve is the curve of the average number of iterations changing with the number of data collection after the improvement;

[0044] Figure 6The schematic diagram of the curve of the single operation time changing with the number of data collection before and after the improvement of the orthogonal iterative algorithm, where the blue curve is the curve of the single operation time changing with the number of data collection before the improvement, and the orange curve is the curve of the single operation time changing with the number of data collection after the improvement;

[0045] Figure 7 Schematic diagram of a curve showing the variation of position error with image point extraction accuracy in an embodiment of the present invention, wherein the blue curve is the position error along the x-axis direction of the visual coordinate system, the orange curve is the position error along the y-axis direction of the visual coordinate system, and the pink curve is the position error along the z-axis direction of the visual coordinate system;

[0046] Figure 8 Schematic diagram of a curve showing the change of attitude angle error with image point extraction accuracy in an embodiment of the present invention, wherein the blue curve is theta angle error, the orange curve is the psi angle error, and the pink curve is the phi angle error;

[0047] Fig. 9 Schematic diagram of a curve showing the change of position error with the spatial coordinate accuracy of a three-dimensional point in an embodiment of the present invention, wherein the blue curve is the position error along the x-axis direction of the visual coordinate system, the orange curve is the position error along the y-axis direction of the visual coordinate system, and the pink curve is the position error along the z-axis direction of the visual coordinate system;

[0048] Fig.10 Schematic diagram of a curve showing the change of attitude angle error with the spatial coordinate accuracy of a three-dimensional point in an embodiment of the present invention, wherein the blue curve is the theta angle error, the orange curve is the psi angle error, and the pink curve is the phi angle error;

[0049] Fig.11 In order to verify and analyze the accuracy of the on-orbit calibration method of the embodiment of the present invention, the image collected by the visual sensor, wherein (a) to (f) are images collected by the visual sensor when the measurement system is at different positions and postures;

[0050] Fig.12 In order to verify and analyze the accuracy of the on-orbit calibration method of an embodiment of the present invention, a schematic diagram is provided for comparing the laser spot after reprojection with the laser spot in the image collected by the visual sensor, wherein the red dot is the location of the laser spot after reprojection, the bright circular spot is the actual location of the laser spot, and the right half of the figure is a partial enlarged view of the left half. DETAILED DESCRIPTION

[0051] In order to make the advantages and features of the present invention more clear, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0052] An on-orbit calibration method for a space target posture measurement system integrating vision and laser, specifically comprising the following steps:

[0053] Step 1: Establish a joint calibration model of visual sensor and laser rangefinder.

[0054] In order to obtain the mapping relationship between the coordinate system of the visual sensor and the coordinate system of the laser rangefinder in space, it is necessary to establish a joint calibration model of the visual sensor and the laser rangefinder. The visual sensor in this embodiment is a monocular visual sensor, and the laser rangefinder is a one-dimensional laser rangefinder. The established joint calibration model is as follows: Figure 1 As shown, the coordinate system is the coordinate system where the laser rangefinder is located, hereinafter referred to as the laser coordinate system. The coordinate system of the visual sensor is referred to as the visual coordinate system. The laser projection direction is used as the laser coordinate system. The Z axis of the visual sensor is used as the visual coordinate system. The Z axis, the X axis and Y axis of the two coordinate systems are determined by the right-hand rule based on the determination of the Z axis.

[0055] Step 2: solve the spatial position of the laser spot.

[0056] According to the established joint calibration model, the The lower laser spot space point In the visual coordinate system The corresponding point , and its conversion relationship is:

[0057] (1)

[0058] in, is the rotation matrix between the laser coordinate system and the visual coordinate system, is the translation vector between the laser coordinate system and the visual coordinate system.

[0059] Under the premise of accurate calibration of the intrinsic parameters of the visual sensor, the intrinsic parameter matrix of the visual sensor can be obtained , then the laser coordinate system can be obtained The lower laser spot space point The corresponding image point on the normalized imaging plane , and its conversion relationship is:

[0060] (2)

[0061] in, For image point Since the laser projection direction is in the same direction as the Z axis of the laser coordinate system, the accurate distance information measured by the laser rangefinder can be used as the image point scale information.

[0062] Since there is a physical connection between the laser coordinate system and the visual coordinate system, it can be assumed that the parameters of the visual sensor (camera) will not change in a short period of time. Therefore, the measurement positions of the laser rangefinder and the visual sensor can be changed synchronously for multiple times. At this time, the laser spot will also change with the change of the projection direction. Then, according to the multiple data collected by the laser rangefinder and the visual sensor at different positions, the spatial point of the laser spot after the measurement position is changed in the laser coordinate system is calculated. The corresponding coordinates (i.e., the spatial position of the laser spot after the measurement position is changed) and the image point corresponding to the spatial point of the laser spot after the measurement position is changed on the normalized imaging plane.

[0063] Specifically, Figure 1 As shown, the laser coordinate system at the initial measurement position The laser spot space point under The corresponding point in the visual coordinate system and its corresponding image point on the normalized imaging plane After being solved, the measurement positions of the laser rangefinder and the visual sensor are changed synchronously. The visual coordinate system after the measurement position is changed becomes , the laser coordinate system after the position change becomes , at this time the laser coordinate system The lower laser spot space point In the laser coordinate system The corresponding point The coordinates are:

[0064] (3)

[0065] in, is the rotation matrix between the two laser coordinate systems before and after the position change, is the translation vector between the two laser coordinate systems before and after the position change.

[0066] At this time, the laser spot spatial point can be obtained The corresponding image point on the normalized imaging plane , and its conversion relationship is:

[0067] (4)

[0068] When the measurement positions of the laser rangefinder and the vision sensor change synchronously At this time, the spatial point of the laser spot can be obtained. The corresponding image point on the normalized imaging plane :

[0069] (5)

[0070] in, is the pose matrix of the visual coordinate system in the laser coordinate system, is the pose matrix of the laser coordinate system at the k+1th position in the laser coordinate system at the kth position.

[0071] By changing the measurement position multiple times in a short period of time, multiple sets of corresponding points of the laser spot space points in the corresponding laser coordinate system after the measurement position is changed and the corresponding image points of the laser spot space points on the corresponding normalized imaging plane after the measurement position is changed can be obtained. Taking the above correspondence as a constraint condition, when solving the posture transformation relationship between the laser coordinate system and the visual coordinate system, the purpose of completing calibration without using special calibration objects in the context of space tasks can be met.

[0072] Step 3: Solve and optimize the pose transformation relationship between the laser coordinate system and the visual coordinate system.

[0073] like Figure 2 As shown, assuming that in the laser coordinate system A total of n laser spots were collected. Laser spot The spatial position of is known, and the corresponding image point coordinates It is also known that , we can get the projection matrix of the image point for:

[0074] (6)

[0075] To facilitate subsequent calculations, normalization is performed to obtain:

[0076] (7)

[0077] Laser spot After the projection matrix transformation, the corresponding point on the line of sight can be obtained , ideally, the two points should completely coincide with each other, and the coincidence relationship can be described by the collinear equation:

[0078] (8)

[0079] At this time, the laser spot The corresponding collinearity error is:

[0080] (9)

[0081] Here, I represents the identity matrix, that is, a matrix whose diagonal elements are 1 and the rest of the elements are 0.

[0082] Will obtain By adding the collinear errors of the laser spots, the objective function of posture parameter solution can be obtained as:

[0083] (10)

[0084] in, Mainly contains rotation matrix and translation vector Two pieces of information, for a certain rotation matrix , we can find the translation vector The optimal solution is:

[0085] (11)

[0086] in, .

[0087] In getting the translation vector After that, the updated rotation matrix can be solved , the specific orthogonal iterative solution process is as follows, first define:

[0088] (12)

[0089] in, Represents the projection point of the laser spot on the line of sight ray, Represents the centroid coordinates of all projection points in the visual coordinate system, Represents the centroid coordinates of all laser spots in the laser coordinate system.

[0090] Assuming that the The calculation results of and According to the idea of ​​absolute orientation problem, we can know that Calculation results :

[0091] (13)

[0092] Where T represents the matrix transpose.

[0093] Use the singular value decomposition method to solve formula (13) and get Calculation results After multiple orthogonal iterations, when the function value of the objective function or the relative change of the function value of the objective function is less than a predetermined threshold, the orthogonal iteration is completed, and the final rotation matrix can be solved. and translation vector Since this method converges globally in theory, the initial value of the above orthogonal iterative solution can be selected arbitrarily.

[0094] The following improves the orthogonal iteration algorithm from two aspects: the calculation of the iteration initial value and the simplification of the iteration algorithm, so as to solve and optimize the posture relationship between the visual coordinate system and the laser coordinate system. The specific steps are as follows:

[0095] 1) Calculation of iterative initial value

[0096] The perspective projection model of the visual sensor is a nonlinear mapping relationship, which will increase the amount of calculation in the actual problem solving. Even if the perspective effect is not obvious enough, directly using the perspective projection model may cause the solution to become pathological. Therefore, when solving the initial value of the iteration under certain conditions, an approximate linear model can be used, such as the weak perspective projection model (weakperspective projection), the parallel perspective projection model (paraperspectiveprojection), etc.

[0097] Considering that the field of view of the visual sensor is relatively small in the on-orbit calibration task, and the depth change of the surface where the laser spot is located is small relative to the distance from the laser spot to the visual sensor, the coordinates of the imaging point are marked as , the 3D coordinates of the scene points are marked as , the three-dimensional coordinates of the target center of mass are , the depth information at the target center of mass is recorded as , then the weak perspective projection model is used to calculate the iterative initial value, and its expression is:

[0098] (14)

[0099] Here, f represents the focal length.

[0100] Since the weak perspective projection model assumes that the depth information of all points in the scene is consistent, it causes a certain error. Assume that the actual depth information value at a certain point is , is the depth change of the scene point. The projection at this point is calculated according to the perspective projection model. , according to the weak perspective projection model, the projection is calculated as , the difference between the two is Taylor expansion can be performed to obtain the projection error of the weak perspective projection model. for:

[0101] (15)

[0102] in, Represents the two-dimensional coordinates of the laser spot.

[0103] Formula (15) shows that the iterative initial value calculation error based on the weak perspective projection model is related to the depth change of the scene point and the 3D coordinates of the scene points When the laser spot is far away from the optical axis, the iterative initial value calculation method based on the weak perspective projection model will produce a large error.

[0104] For the parallel perspective projection model, the projection expression is:

[0105] (16)

[0106] Assume a point The actual three-dimensional coordinates of the scene point are , , , They are the three components of the difference between the three-dimensional coordinates of the target mass center under the parallel perspective projection model and the three-dimensional coordinates of the actual target mass center. The projection at this point is calculated according to the weak perspective projection model. , according to the parallel perspective projection model, the projection is calculated as , the difference between the two is Taylor expansion can be performed to obtain the projection error of the parallel perspective projection model. for:

[0107] (17)

[0108] in, , , They are the three components of the difference between the three-dimensional coordinates of the target center of mass under the weak perspective projection model and the three-dimensional coordinates of the actual target center of mass.

[0109] Comparing formula (15) and formula (17), it can be seen that the projection error based on the parallel perspective projection model is a second-order infinitesimal of the three-dimensional point, while the projection error based on the weak perspective projection model is a first-order infinitesimal of the three-dimensional point. This shows that when the three-dimensional position of the laser spot may be far away from the optical axis, the parallel perspective projection model is a more accurate approximate linear model than the weak perspective projection model. Therefore, the present invention adopts an initial value calculation method based on the parallel perspective projection model to achieve an improvement in iterative initial value calculation in joint calibration.

[0110] Assume that the spatial coordinates of the laser spot are , the corresponding image point coordinates , the centroid coordinates of all laser spot space points , the corresponding image point coordinates , for the convenience of calculation, the transformation matrix from the laser coordinate system to the visual coordinate system is Rewritten as:

[0111] (18)

[0112] in, , , are the rotation components of the rotation matrix in the three degrees of freedom directions, , , are the translation amounts of the translation vector in the x, y, and z directions respectively.

[0113] The projection equation of the parallel perspective projection model can be written as:

[0114] (19)

[0115] Combining formula (14) with formula (19), we can get:

[0116] (20)

[0117] In formula (20), , when the laser spot is far away from the visual sensor, The value of is much smaller than 1.

[0118] (twenty one)

[0119] From formula (21), we can know that:

[0120] (twenty two)

[0121] Since the vector , and are mutually orthogonal, so we can get:

[0122] (twenty three)

[0123] In formula (23) and means and The antisymmetric matrix of . Introducing homogeneous coordinates, let:

[0124] (twenty four)

[0125] The difference between the coordinates after imaging based on the perspective projection model and the coordinates after imaging based on the parallel perspective projection model is used as the objective function , we can get:

[0126] (25)

[0127] The optimization problem of formula (25) can be solved by setting the partial derivative of the objective function to 0, since The value of is much smaller than 1, so we can get:

[0128] (26)

[0129] In getting and After that, you can get and , then we can get the initial calculation result of the rotation matrix based on the parallel perspective projection model:

[0130] (27)

[0131] The result in formula (27) is the iterative initial value of the rotation matrix between the laser coordinate system and the visual coordinate system obtained based on the parallel perspective projection model.

[0132] 2) Simplification of orthogonal iterative algorithm

[0133] After the initial value of the rotation matrix is ​​given, the objective function of the pose estimation is minimized, and then the corresponding translation vector is solved. The projection point is updated with the translation vector and further calculated and decomposed by SVD to update the rotation matrix. In each iteration, the rotation matrix needs to be calculated separately. and translation vectors , actually calculates the translation vector It is only to update the projection point, and the calculation of the projection point is essentially to update the rotation matrix using the absolute orientation algorithm . But updating the rotation matrix It is not necessary to input the updated projection points before calculation. The present invention improves the orthogonal iterative algorithm and eliminates the steps of updating the projection points and updating the translation vector, thereby achieving the purpose of simplifying the iterative process, reducing the amount of calculation and improving the operation speed.

[0134] First, the update of the projection point is replaced. The projection point after the mth iteration for:

[0135] (28)

[0136] in:

[0137] (29)

[0138] , indicating that Matrices are stacked column by column into a column vector, represents the Kronecker product.

[0139] After replacing the update of the projection point, the iterative process needs to be replaced, so that:

[0140] (30)

[0141] in, is the transpose of the projection point after the mth iteration.

[0142] According to the method of finding the optimal solution based on absolute orientation, the singular value decomposition is: , U is an orthogonal matrix, then:

[0143] (31)

[0144] make , then:

[0145] (32)

[0146] According to formula (32), we only need to solve The matrix can be iterated. The matrix is ​​calculated as follows:

[0147] (33)

[0148] According to formula (33), we can find that The values ​​of the matrix do not change with iteration, so it can be calculated before iteration. Matrix, directly convert it during iteration Substituting the matrix into formula (33) for calculation, the projection point and translation vector are no longer calculated, which simplifies the iteration process and reduces the amount of calculation. Based on the above idea, the update of the objective function value also needs to be replaced, and we can get:

[0149] (34)

[0150] in:

[0151] (35)

[0152] From formula (35), we can see that the matrix The value of is a constant, so the calculation can be completed before the iteration starts, and the value can be directly used for subsequent calculations in the iteration.

[0153] After the iteration is completed, the final translation vector can be calculated as follows:

[0154] (36)

[0155] The improved iterative process only needs to calculate , and The iterative process can be completed with the values ​​of the three matrices, thus avoiding the tedious steps of calculating the translation vector and updating the projection point in each iteration of the orthogonal iterative algorithm, and reducing the computational complexity of the orthogonal iterative algorithm.

[0156] In summary, by improving the orthogonal iterative algorithm from two aspects, namely the calculation of the initial value of the iteration and the simplification of the iterative algorithm, the problem that the laser spot may be far away from the optical axis during joint calibration and the real-time performance of the calibration algorithm is not up to standard is solved, so as to complete the calibration task in the space background.

[0157] The on-orbit calibration method before the orthogonal iterative algorithm is improved and the on-orbit calibration method after the orthogonal iterative algorithm is improved according to the present invention are verified and analyzed by simulation experiments.

[0158] In this embodiment, the simulation experiment uses MATLAB r2020a, assuming that the image plane center coordinates of the visual sensor are , normalized focal length , the azimuth angle in the position conversion relationship between the visual coordinate system and the laser coordinate system , yaw angle , roll angle , the translation vector . Assuming that the laser rangefinder projects a laser point onto the solar panel, that is, the laser spot spatial point is located on the same plane in space, when generating three-dimensional spatial points, a series of feature points are randomly generated with the same plane as a constraint to replace the laser spot. Due to the randomness of the points, it is necessary to adopt a processing method of multiple simulations and taking the average value when solving the final solution to avoid the randomness of the final result. For calibration, the main factors affecting its accuracy are: the number of laser spots, the accuracy of laser spot extraction, the accuracy of the three-dimensional coordinates of the laser spot spatial point, etc. In the simulation process, the above influencing factors are mainly experimented, and the influence of each factor on the calibration method is quantitatively analyzed.

[0159] 1) The influence of the number of laser spots on calibration

[0160] The number of laser spots is expressed in calibration as the number of times the measurement system needs to change its posture, that is, a set of laser spot information can be obtained by changing the posture once. From a qualitative point of view, the more laser spots are obtained, the more points the orthogonal iterative algorithm uses during operation, the less affected by the points with poor properties, and the higher the final accuracy. On the other hand, the increase in the number of laser spots requires the measurement system to continuously adjust its posture to achieve it. Too many laser spots will also bring about a surge in the amount of calculation, which is unacceptable in space missions. Therefore, it is very necessary to reasonably simulate and estimate the number of laser spots required to meet the calculation accuracy. When performing measurement tasks, it is inevitable that there will be pixel errors in laser spot extraction, three-dimensional coordinate position errors of laser spot space points, etc., and these errors must be reflected in the simulation process. Therefore, when simulating the impact of the number of laser spots on measurement accuracy, it is assumed that there is a 0.1 pixel spot extraction pixel error and a 0.5 mm three-dimensional coordinate position error. Since too few laser spots may lead to problems such as non-convergence of the final result, in order to achieve a better convergence effect, the number of laser spots is calculated from a minimum of 4 and gradually increased to 30, that is, the measurement system performs 4 to 30 posture transformations and data collection. Under each data collection condition, 500 independent repeated experiments are performed. The simulation results are shown in the figure. Figure 3 and Figure 4 shown. Figure 3 This is a schematic diagram of the curve showing the change of attitude angle error with the number of data collection before and after the orthogonal iterative algorithm is improved. The blue curve is the theta angle (azimuth) error before improvement, the orange curve is the psi angle (yaw angle) error before improvement, the pink curve is the phi angle (roll angle) error before improvement, the purple curve is the theta angle error after improvement, the green curve is the psi angle error after improvement, and the red curve is the phi angle error after improvement. Figure 4 Figure 1 is a schematic diagram of the curve showing the change of position error with the number of data acquisition times before and after the orthogonal iterative algorithm is improved, where the blue curve is the position error in the x-axis direction of the improved frontier visual coordinate system, the orange curve is the position error in the y-axis direction of the improved frontier visual coordinate system, the pink curve is the position error in the z-axis direction of the improved frontier visual coordinate system, the purple curve is the error along the x-axis direction of the visual coordinate system after improvement, the green curve is the error along the y-axis direction of the visual coordinate system after improvement, and the red curve is the error along the z-axis direction of the visual coordinate system after improvement.

[0161] from Figure 3 and Figure 4It can be seen that when the number of data collection is small, the number of laser spots obtained is also small, and the attitude angle measurement error and position error of the measurement system are large. As the number of data collection increases, the number of laser spots increases, and the accuracy of the calibration method gradually improves. Among them, the error decreases fastest when the data is collected 4 to 8 times, and the error value basically tends to be stable after 10 data collections. Due to the randomness of the selection of laser spots, there are certain random errors in spot extraction and random errors in the coordinate position of three-dimensional points, and the number of independent repeated experiments is limited. Therefore, the error still has a small fluctuation in the stable stage, but its value can basically reflect the accuracy measurement results of the measurement system under the set error conditions.

[0162] During the first 10 data collections, the measurement accuracy of the on-orbit calibration method before the orthogonal iteration algorithm was improved was slightly lower than that of the on-orbit calibration method after the orthogonal iteration algorithm was improved. This is because after the orthogonal iteration algorithm was improved, the parallel perspective projection model was used in the calculation of the initial value of the iteration, while the weak perspective projection model was used before the orthogonal iteration algorithm was improved. When performing joint calibration in a spatial environment, it is impossible to determine the three-dimensional position of the laser spot in advance. The present invention adopts a random selection method within a certain feasible range in the selection of the laser spot, that is, the position of the laser spot has a certain degree of randomness, which may exist at a position close to the optical axis or at a position far from the optical axis. For the weak perspective projection model, when the laser spot is far from the optical axis, its posture estimation ability is poor, which results in a small number of laser spots and a laser spot appearing at a position far from the optical axis, which has a greater impact on the estimation of the initial value of the iteration, and this will have a greater impact on the convergence of the orthogonal iteration algorithm, which will lead to convergence to an erroneous result, and ultimately cause the problem of poor calibration accuracy.

[0163] Although the accuracy of the calibration method can be improved by increasing the number of laser spots, too many laser spots will reduce the efficiency of the algorithm and slow down the operation speed, which cannot meet the real-time requirements of space missions. Therefore, it is necessary to simulate the relationship between the number of data collection times of the measurement system and the algorithm operation efficiency under the same conditions. The simulation results are shown in Figure 2. Figure 5 and Figure 6 shown. Figure 5 It is a schematic diagram of the curve showing the average number of iterations changing with the number of data collection before and after the improvement of the orthogonal iterative algorithm. The blue curve is the curve showing the average number of iterations changing with the number of data collection before improvement, and the orange curve is the curve showing the average number of iterations changing with the number of data collection after improvement. Figure 6 The figure is a schematic diagram of the curve showing the change of single operation time with the number of data collection times before and after the improvement of the orthogonal iterative algorithm. The blue curve is the curve showing the change of single operation time with the number of data collection times before the improvement, and the orange curve is the curve showing the change of single operation time with the number of data collection times after the improvement.

[0164] from Figure 5 and Figure 6 It can be seen that with the increase in the number of data acquisition times, the number of laser spots obtained increases, the average number of iterations required in the operation gradually decreases, and the single operation time first decreases slightly, and then continues to grow. For the calibration algorithm, the more laser spots are obtained, the fewer iterations it takes for its objective function value to converge to a given threshold. When the number of laser spots is greater than 15, the average number of iterations tends to be stable. Since the two calibration methods use different iterative initial value calculation models, the accuracy of the iterative initial value estimation is different, and the number of iterations required for convergence is also different. Specifically, the on-orbit calibration method improved by the orthogonal iterative algorithm of the present invention is more accurate in iterative initial value estimation than before the improvement, and it can converge with fewer iterations.

[0165] The computational efficiency of the calibration method cannot be fully described by the number of iterations. When the number of acquired laser spots increases, even if the number of iterations decreases, the number of points that need to be calculated increases, and the total computation time increases accordingly. The number of iterations reflects the convergence of the algorithm, and the total computation time reflects the computation speed of the algorithm. Since the number of laser spots is small, the number of iterations required for the calibration operation is large, which will cause the calibration operation to take a long time. When the number of laser spots increases slightly, the number of iterations decreases significantly, and the total computation time decreases at this time. As the number of laser spots continues to increase, the number of iterations continues to decrease accordingly, but the number of points that need to be calculated increases, and the total computation time is in an upward state. However, for the same number of laser spots, the fewer the iterations, the less time it takes. Therefore, the on-orbit calibration method improved by the orthogonal iterative algorithm has greater advantages than before the improvement in terms of both the number of iterations and the total computation time.

[0166] By analyzing the influence of the number of laser spots on the calibration method, and comparing the attitude angle error, position error, average number of iterations and single operation time, it can be considered that when the number of laser spots is 8, the calibration error gradually begins to stabilize, and the average number of iterations decreases. Although the single operation time is not the shortest duration of the entire iterative optimization process, it is the optimal choice within the acceptable range of error and number of iterations. Therefore, after considering various influencing factors, this experiment can perform a more accurate calibration solution when 8 data acquisitions are performed, that is, when 8 laser spots are obtained. Therefore, this experiment uses 8 data acquisitions as the initial condition in subsequent simulations. Since the on-orbit calibration method after the orthogonal iterative algorithm has a greater advantage in terms of calibration accuracy, average number of iterations and single operation time when 8 laser spots are obtained, the algorithm is mainly simulated in subsequent experiments.

[0167] 2) The influence of laser spot image extraction accuracy on the calibration method

[0168] Under ideal conditions, the image point extraction accuracy of a specific mark can reach 0.01 pixels, and the image point extraction accuracy of a general laser spot can also reach 0.1 pixels. The specific process of the following simulation is: randomly select 8 coplanar laser spots in space, assume that the image point extraction error varies from 0.01 pixels to 1 pixel, and perform 500 independent repeated experiments at each level of error. The final result is the average of all experiments. The simulation results are shown in the figure below. Figure 7 and Figure 8 shown. Figure 7 Schematic diagram of a curve showing how the position error varies with the image point extraction accuracy in an embodiment of the present invention, wherein the blue curve is the position error along the x-axis direction of the visual coordinate system, the orange curve is the position error along the y-axis direction of the visual coordinate system, and the pink curve is the position error along the z-axis direction of the visual coordinate system. Figure 8 Schematic diagram of the curve showing the change of attitude angle error with image point extraction accuracy in an embodiment of the present invention, wherein the blue curve is the theta angle error, the orange curve is the psi angle error, and the pink curve is the phi angle error.

[0169] from Figure 7 , Figure 8 It can be seen that with the continuous increase of the laser spot image point extraction accuracy, the position error and attitude angle error of the calibration result are also increasing. The increase process has a certain degree of floating. This is mainly due to the fact that there is a certain degree of randomness in the setting of the image point extraction accuracy, and the number of independent repeated experiments is limited. Therefore, there is a certain deviation between the error curve and the actual error. When the image point extraction accuracy is 0.1 pixel, the calibration error is within the acceptable range. Therefore, the assumption that the image point extraction accuracy is set to 0.1 pixel in the previous article is reasonable.

[0170] 3) The influence of the three-dimensional coordinate accuracy of the laser spot on the calibration method

[0171] In the above simulation, the influence of the accuracy of the three-dimensional coordinates of the laser spot on the calibration method is not ignored, and its error is set to 0.5 mm in the simulation. For the vision and laser fusion measurement system, the three-dimensional coordinates of the laser spot are mainly measured by the laser rangefinder, and its measurement accuracy can reach the millimeter level. The specific process of the following simulation is: randomly select 8 coplanar laser spots in space, assuming that the accuracy of the three-dimensional coordinates of the laser spot varies from 0.01 mm to 1 mm, and perform 500 independent repeated experiments at each level of error. The final result is the average of all experiments. The simulation results are as follows: Fig. 9 and Fig.10 shown. Fig. 9Schematic diagram of the curve showing how the position error varies with the accuracy of the spatial coordinates of the three-dimensional point in an embodiment of the present invention, wherein the blue curve is the position error along the x-axis direction of the visual coordinate system, the orange curve is the position error along the y-axis direction of the visual coordinate system, and the pink curve is the position error along the z-axis direction of the visual coordinate system. Fig.10 Schematic diagram of the curve showing the change of attitude angle error with the accuracy of three-dimensional point space coordinates in an embodiment of the present invention, wherein the blue curve is the theta angle error, the orange curve is the psi angle error, and the pink curve is the phi angle error.

[0172] from Fig. 9 , Fig.10 It can be seen that as the accuracy of the three-dimensional spatial coordinates of the laser spot continues to increase, the position error and attitude angle error of the calibration result are also increasing. Compared with the image point extraction accuracy, the impact of the three-dimensional spatial coordinate accuracy on the calibration result is more obvious. When the accuracy of the three-dimensional spatial coordinates of the laser spot is 0.5 mm, the calibration error is within the acceptable range. Therefore, the assumption that the accuracy error of the three-dimensional spatial coordinates of the laser spot is set to 0.5 mm is reasonable.

[0173] In summary, the improvement of the orthogonal iterative algorithm by the present invention is reasonable and effective in terms of both accuracy and computational efficiency, and the on-orbit calibration method after the improvement of the orthogonal iterative algorithm has better performance.

[0174] In addition, the present invention also verifies and analyzes the accuracy of the on-orbit calibration method after the orthogonal iterative algorithm is improved through physical experiments.

[0175] The measurement system is fixed on a high-precision three-dimensional turntable, and the target to be measured (satellite) is placed on a translatable observation table. The measurement system projects a laser to the target to be measured, and the visual sensor collects images and records the current position and posture of the measurement system. By changing different positions and postures, images at multiple positions are obtained. According to the simulation results, after 8 data collections, the calibration task can be well achieved. Fig.11 As shown in the figure, the images collected by the visual sensor when the measurement system is in different positions and postures, the laser spot projected by the laser rangefinder can be clearly seen on the solar panel of the space target, and the centroid of the spot can be calculated to obtain the position information of the laser spot in the imaging plane. Combined with the distance information at the laser spot, the posture conversion relationship between the laser coordinate system and the visual coordinate system can be solved.

[0176] After calibration by the on-track calibration method, the rotation and translation matrix between the laser coordinate system and the visual coordinate system is obtained as follows:

[0177]

[0178] In order to verify the accuracy of the results, the three-dimensional coordinates of the laser spot are projected onto the imaging plane using the reprojection method and compared with the laser spot in the image. Fig.12 As shown, the red dot is the location of the laser spot after reprojection, and the bright circular spot is the actual location of the laser spot. It can be seen that there is a certain error between the actual position of the laser spot and the reprojection position, but basically the overlap effect is good. The average reprojection error of the images collected at different positions is 0.813 pixels.

[0179] Since there are various errors in the actual experiment, such as the measurement error of the laser rangefinder and the turntable accuracy error, there is a certain gap with the simulation results. However, since the on-orbit calibration method of the present invention has a relatively simple calibration process, a fast calculation rate, a relatively small amount of calculation and meets the accuracy requirements, it has the ability to realize on-orbit calibration tasks in a space environment and can realize the on-orbit calibration tasks of the vision and laser joint measurement system, providing theoretical support for related space technologies.

[0180] The above description is only used to illustrate the technical solution of the present invention rather than to limit it. For ordinary professional and technical personnel in the field, the specific technical solution recorded in the above embodiment can be modified, or some of the technical features therein can be replaced by equivalents, and these modifications or replacements do not make the essence of the corresponding technical solution deviate from the scope of the technical solution protected by the present invention.

Claims

1. An on-orbit calibration method for a space target posture measurement system integrating vision and laser, characterized in that: The following steps are involved: Step 1: Establish a joint calibration model of visual sensor and laser rangefinder; Define the coordinate system of the laser rangefinder as the laser coordinate system , the coordinate system where the visual sensor is located is the visual coordinate system , taking the laser projection direction as the laser coordinate system The Z axis of the visual sensor is used as the visual coordinate system. Z axis; laser coordinate system and visual coordinate system There is a rotation matrix and translation vectors ; Step 2, solving the spatial position of the laser spot; According to the joint calibration model established in step 1, the laser coordinate system at the initial measurement position is solved respectively. The laser spot space point is in the visual coordinate system The corresponding points and the laser coordinate system The image point corresponding to the laser spot space point on the normalized imaging plane is obtained by synchronously changing the measurement positions of the laser rangefinder and the visual sensor for multiple times, solving and obtaining multiple groups of points corresponding to the laser spot space point in the corresponding visual coordinate system after changing the measurement position and the image point corresponding to the laser spot space point on the corresponding normalized imaging plane; Step 3, solving and optimizing the posture transformation relationship between the laser coordinate system and the visual coordinate system; The points corresponding to the laser spot space points in the corresponding visual coordinate system and the image points corresponding to the laser spot space points on the corresponding normalized imaging plane obtained in step 2 are used as constraint conditions, and the posture transformation relationship between the laser coordinate system and the visual coordinate system is solved and optimized through an improved orthogonal iterative algorithm; The improved orthogonal iterative algorithm comprises: a. Calculate the rotation matrix using the initial value calculation method based on the parallel perspective projection model The initial value of the iteration ; b. Perform orthogonal iteration through the following simultaneous equations to obtain : ； ； ； ； in, , , n is the number of iterations, is the pose matrix of the laser coordinate system at the k+1th position in the laser coordinate system at the kth position, For the The coordinates of the laser spot space points, I is the unit matrix, is the projection matrix of the image point, The initial value of the iteration is , is the projection point after the mth iteration, is the transpose of the projection point after the mth iteration, , ; c. Calculate the final translation vector using the following formula: 。 2. The on-orbit calibration method for a space target posture measurement system integrating vision and laser according to claim 1 is characterized in that: In step 2, the laser coordinate system is solved by the following formula: The laser spot space point is in the visual coordinate system The corresponding point : ； in, is the spatial point of the laser spot at the initial measurement position.

3. The on-orbit calibration method for a space target posture measurement system integrating vision and laser according to claim 2 is characterized in that: In step 2, the laser coordinate system is solved by the following formula: The image point corresponding to the spatial point of the laser spot on the normalized imaging plane : ； in, K is the internal parameter matrix of the visual sensor, Z For image point scale information.

4. The on-orbit calibration method for a space target posture measurement system integrating vision and laser according to claim 3 is characterized in that: In step 3, the rotation matrix is ​​calculated using the initial value calculation method based on the parallel perspective projection model The initial value of the iteration for: ； in, is the depth information at the target center of mass, , , , are the x-direction and y-direction components of the image point coordinates corresponding to the target centroid coordinates of all laser spot space point sets, is the third-order identity matrix, , They are and The antisymmetric matrix of , , are the rotation components of the rotation matrix in the three degrees of freedom directions, is the translation amount of the translation vector in the z direction.

Images