A method and system for calculating external parameters of structured light iteratively

Abstract

The application provides a kind of structural light's external parameter iterative calculation method and system, the method is by establishing reference point, obtains the first parameter coordinate, second parameter coordinate and third parameter coordinate of reference point, and according to the first parameter coordinate, second parameter coordinate, third parameter coordinate and corresponding scaling factor to construct camera parameter matrix and projector parameter matrix, and it is obtained simultaneously to get structural light coordinate relationship parameter.Then establish parallel plane, the real distance of parallel plane is calculated by error function calculation parallel plane and the plane error between interval distance, finally the external parameter matrix, structural light coordinate relationship parameter and plane error are calculated simultaneously, and the external parameter calculation result is obtained.The application is by simultaneous matrix, and the error of external parameter is obtained by the way of establishing different parallel planes and using error function calculation, to calculate external parameter, improve the efficiency and stability of external parameter calculation, and reduce external parameter error by iterative calculation.

Description

Technical Field

[0001] This application relates to the field of iterative calculation of structured light extrinsic parameters, and in particular to a method and system for iterative calculation of structured light extrinsic parameters. Background Technology

[0002] Camera intrinsic parameters are parameters related to the camera's own characteristics, such as focal length and pixel size. Camera extrinsic parameters are parameters in the world coordinate system, such as the camera's position and rotation direction. Camera extrinsic parameters can be used to rotate and translate the coordinates of real-world points onto the camera's coordinate system.

[0003] The Levenberg-Marquardt (LM) algorithm is commonly used to calibrate and solve intrinsic and extrinsic parameters. The LM algorithm summarizes the errors of all samples before updating the weights. If the standard error surface has multiple local minima, the LM algorithm may sometimes get stuck in these local minima, resulting in low calibration accuracy.

[0004] Furthermore, stochastic gradient descent relies on a manually set learning rate. If the learning rate is inappropriate, the LM algorithm will not converge, or existing stochastic gradient descent methods with adaptive learning rates will converge slowly. During calibration, calibration methods typically use a calibration board (checkerboard, circle, or other graphic representation) to solve for the system's intrinsic and extrinsic parameters. For cases with low calibration accuracy, recalibration or error lookup tables are used for compensation. However, recalibration affects calibration efficiency, while using error lookup tables for compensation increases computational load and reduces system stability. Summary of the Invention

[0005] To improve the efficiency and stability of calibrating camera extrinsic parameters and reduce the error value of the extrinsic parameters, in a first aspect, some embodiments of this application provide a method for iterative calculation of extrinsic parameters of structured light, the method comprising:

[0006] Establish a reference point, and obtain the first parameter coordinates in the world coordinate system, the second parameter coordinates in the camera pixel coordinate system, and the third parameter coordinates in the projector pixel coordinate system based on the reference point;

[0007] A camera parameter matrix is ​​constructed based on the first parameter coordinates, the second parameter coordinates, and the camera scaling factor; and a projector parameter matrix is ​​constructed based on the first parameter coordinates, the third parameter coordinates, and the projector scaling factor.

[0008] By combining the camera parameter matrix and the projector parameter matrix, the structured light coordinate relationship parameters are obtained;

[0009] Establish a preset number of parallel planes; each parallel plane contains a set of plane points, the parallel planes are spaced at different distances and the straight lines formed by the corresponding sets of plane points are perpendicular to the parallel planes;

[0010] Obtain the true distance between the parallel planes, and calculate the plane error between the interval distance between the parallel planes and the true distance using the error function;

[0011] The extrinsic parameter matrix between the projector and the camera, the structured light coordinate relationship parameters, and the plane error are iteratively combined to calculate the extrinsic parameter calculation results.

[0012] In some embodiments, the method further includes combining the camera parameter matrix and the projector parameter matrix, and further comprising:

[0013] Calculate a first transformation matrix from the camera pixel coordinate system to the world coordinate system, and calculate a second transformation matrix from the projector pixel coordinate system to the world coordinate system;

[0014] Iterate through the first transformation parameter of the first transformation matrix and the second transformation parameter of the second transformation matrix;

[0015] The camera parameter matrix is ​​converted into a first simultaneous matrix in the world coordinate system according to the first transformation parameter, and the projector parameter matrix is ​​converted into a second simultaneous matrix in the world coordinate system according to the second transformation parameter.

[0016] The first and second simultaneous matrices are combined to calculate the structured light coordinate relationship parameters.

[0017] In some embodiments, the method further includes, in the step of calculating the first transformation matrix from the camera pixel coordinate system to the world coordinate system:

[0018] The camera intrinsic parameter matrix, camera rotation matrix, and camera translation matrix are obtained through calibration.

[0019] The camera intrinsic parameter matrix, the camera rotation matrix, and the camera translation matrix are input into a first preset formula to calculate the first transformation matrix.

[0020] In some embodiments, the method further includes, in the step of calculating the second transformation matrix from the projector pixel coordinate system to the world coordinate system:

[0021] Obtain the extrinsic rotation matrix between the projector and the camera; the extrinsic rotation matrix is ​​a matrix obtained by rotating the extrinsic parameters between the projector and the camera in coordinates.

[0022] Calculate the product of the extrinsic rotation matrix and the camera rotation matrix to obtain the projector rotation matrix; the projector rotation matrix is ​​the rotation matrix from the projector coordinate system to the world coordinate system;

[0023] The projection intrinsic parameter matrix and projection translation matrix are obtained through calibration.

[0024] The projection intrinsic parameter matrix, the projection translation matrix, and the projector rotation matrix are input into the second preset formula to calculate the second transformation matrix.

[0025] In some embodiments, the method further includes the step of obtaining the projection intrinsic parameter matrix and the projection translation matrix through calibration:

[0026] Obtain the extrinsic translation matrix between the projector and the camera; the extrinsic translation matrix is ​​a matrix obtained by performing coordinate translation on the extrinsic parameters between the projector and the camera.

[0027] Calculate the matrix product of the extrinsic rotation matrix and the camera translation matrix;

[0028] The product of the extrinsic translation matrix and the matrix is ​​summed to obtain the projection translation matrix.

[0029] In some embodiments, the camera intrinsic parameter matrix is:

[0030]

[0031] Among them, f cx f cy s is the ratio of the camera lens focal length to the physical size of each pixel in the x and y directions. cx s cy The coordinates of the lens center point in the camera pixel coordinate system;

[0032] The projection intrinsic parameter matrix is:

[0033]

[0034] Among them, f px f py s is the ratio of the projector lens focal length to the physical size of each pixel in the x and y directions. px s py The coordinates of the projector's center point in the projector's pixel coordinate system.

[0035] In some embodiments, in the step of calculating the plane error between the interval distance between the parallel planes and the true distance using the error function, the method further includes:

[0036] Obtain the interval distance between the parallel planes to be calculated;

[0037] The plane error is obtained by calculating the error between the true distance and the interval distance using the following error function;

[0038]

[0039] Where D is the true distance between parallel planes; d is the interval distance between parallel planes; α is the error function constant, α∈[0,1]; and N is the summation range of the error function.

[0040] In some embodiments, the method further includes the step of iteratively calculating the extrinsic parameter matrix between the projector and the camera, the structured light coordinate relationship parameters, and the plane error.

[0041] Compare the plane error with the distance error threshold;

[0042] If the plane error is greater than the distance error threshold, then the extrinsic parameter matrix between the projector and the camera, the structured light coordinate relationship parameters, and the plane error are iteratively combined for calculation.

[0043] If the plane error is less than the distance error threshold, then the plane error is output.

[0044] In some embodiments, the extrinsic parameter matrix includes an extrinsic rotation matrix and an extrinsic translation matrix, and the method further includes:

[0045] If the planar error is greater than the distance error threshold, the extrinsic rotation parameters of the extrinsic rotation matrix and the extrinsic translation parameters of the extrinsic translation matrix are input into the loss function as iterative parameters, and the structured light coordinate relationship parameters are combined to update the extrinsic rotation parameters and the extrinsic translation parameters.

[0046] Calculate the parameter errors of the external rotation parameters and the external translation parameters before and after the update;

[0047] The parameter error is then compared again with the error threshold as a plane error.

[0048] Secondly, some embodiments of this application provide a structured light extrinsic parameter iteration calculation system, the system including a projector, a camera, and a calculation module, the calculation module being configured to perform:

[0049] Establish a reference point, and obtain the first parameter coordinates in the world coordinate system, the second parameter coordinates in the camera pixel coordinate system, and the third parameter coordinates in the projector pixel coordinate system based on the reference point;

[0050] A camera parameter matrix is ​​constructed based on the first parameter coordinates, the second parameter coordinates, and the camera scaling factor; and a projector parameter matrix is ​​constructed based on the first parameter coordinates, the third parameter coordinates, and the projector scaling factor.

[0051] By combining the camera parameter matrix and the projector parameter matrix, the structured light coordinate relationship parameters are obtained;

[0052] Establish a preset number of parallel planes; each parallel plane contains a set of plane points, the parallel planes are spaced at the same distance from each other, and the straight line formed by the corresponding sets of plane points is perpendicular to the parallel plane;

[0053] Obtain the true distance between the parallel planes, and calculate the plane error between the interval between the parallel planes and the true distance using the error function;

[0054] The extrinsic parameters between the projector and the camera, the structured light coordinate relationship parameters, and the plane error are iteratively combined to calculate the extrinsic parameter calculation results.

[0055] As described above, this application provides a method and system for iterative calculation of extrinsic parameters in structured light. The method establishes a reference point and obtains its first parameter coordinates in the world coordinate system, second parameter coordinates in the camera pixel coordinate system, and third parameter coordinates in the projector pixel coordinate system. Based on these coordinates and corresponding scaling factors, camera parameter matrices and projector parameter matrices are constructed, and these matrices are combined to obtain the structured light coordinate relationship parameters. Then, parallel planes are established, and the plane error between the true distance to the parallel planes and the distance between them is calculated using an error function. Finally, the extrinsic parameter matrix, structured light coordinate relationship parameters, and plane error are combined to obtain the extrinsic parameter calculation result. This application improves the efficiency and stability of extrinsic parameter calculation by combining matrices, establishing different parallel planes, and using an error function to calculate the extrinsic parameter error. Furthermore, iterative calculation reduces the extrinsic parameter error. Attached Figure Description

[0056] To more clearly illustrate the technical solution of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0057] Figure 1 A simplified flowchart illustrating an iterative calculation method for extrinsic parameters of structured light, provided in an embodiment of this application;

[0058] Figure 2A flowchart illustrating an iterative calculation method for extrinsic parameters of structured light, provided as an embodiment of this application;

[0059] Figure 3 A schematic diagram illustrating the construction of a parallel plane for embodiments of this application;

[0060] Figure 4 This is a schematic diagram illustrating the output standard for determining plane error in an embodiment of this application. Detailed Implementation

[0061] To make the objectives and implementation methods of this application clearer, the exemplary implementation methods of this application will be clearly and completely described below with reference to the accompanying drawings of the exemplary embodiments of this application. Obviously, the exemplary embodiments described are only some embodiments of this application, and not all embodiments.

[0062] It should be noted that the brief descriptions of terms in this application are only for the convenience of understanding the embodiments described below, and are not intended to limit the embodiments of this application. Unless otherwise stated, these terms should be understood in their ordinary and common meaning.

[0063] The terms "first," "second," "third," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar or related objects or entities, and do not necessarily imply a specific order or sequence, unless otherwise specified. It should be understood that such terms are interchangeable where appropriate.

[0064] Because the LM algorithm has low accuracy in calibrating extrinsic parameters when the standard error surface has multiple local minima, compensation is made by recalibrating or using an error lookup table when the calibration accuracy is low, which increases the amount of computation and reduces the stability of the system.

[0065] To address the aforementioned problems, some embodiments of this application provide a method for iterative calculation of structured light extrinsic parameters. Figure 1 This is a simplified flowchart of the iterative calculation method for structured light extrinsic parameters. Figure 2 The flowchart illustrates the specific implementation of the iterative calculation method for structured light extrinsics, as follows: Figure 2 As shown, the method includes:

[0066] S100: Establish a reference point, and obtain the first parameter coordinates in the world coordinate system, the second parameter coordinates in the camera pixel coordinate system, and the third parameter coordinates in the projector pixel coordinate system based on the reference point.

[0067] In this embodiment, the reference point is a coordinate point used for subsequent coordinate transformations. For ease of description, the reference point is defined as point P. In the world coordinate system, the first parameter coordinate of point P is (X... w Y w Zw Among them, X w Y w Z w These are the coordinates of point P in the X, Y, and Z directions, respectively.

[0068] In the camera pixel coordinate system, the second parameter coordinates of point P are:

[0069] In the projector pixel coordinate system, the second parameter coordinates of point P are:

[0070] S200: Construct a camera parameter matrix based on the first parameter coordinates, the second parameter coordinates, and the camera scaling factor; and construct a projector parameter matrix based on the first parameter coordinates, the third parameter coordinates, and the projector scaling factor.

[0071] In this embodiment, camera parameter matrices and projector parameter matrices need to be constructed based on the camera scaling factor and projector scaling factor, respectively. These matrices can represent the imaging relationship of the optical system.

[0072] The camera parameter matrix is ​​as follows:

[0073]

[0074] Among them, s c It is the camera scaling factor, T c It is the first transformation matrix; T c11 -T c34 This is the first conversion parameter.

[0075] The projector parameter matrix is ​​as follows:

[0076]

[0077] Among them, s p It is the projector scaling factor, T p It is the second transformation matrix; T p11 -T p34 This is the second conversion parameter.

[0078] S300: Combine the camera parameter matrix and the projector parameter matrix to obtain the structured light coordinate relationship parameters.

[0079] By combining the camera parameter matrix and the projector parameter matrix, we can obtain the following matrix:

[0080]

[0081] After simplification, the above matrix can be represented as follows:

[0082]

[0083] In this embodiment, the optical system can perform parameter calibration based on the simplified matrix described above to obtain structured light coordinate relationship parameters. These structured light coordinate relationship parameters are the conversion parameters between camera pixel coordinates and projector pixel coordinates.

[0084] S400: Establish a preset number of parallel planes.

[0085] The parallel planes contain sets of planar points, and the parallel planes are spaced at different distances, with the lines formed by the corresponding sets of planar points perpendicular to the parallel planes. In this embodiment, multiple sets of parallel planes can be established, and the distance between the parallel planes is defined as d, such as... Figure 3 As shown in the example, this application uses two planes as an example. The distance between plane A and plane B is d, and the actual distance between plane A and plane B is D. The distance between planes refers to the distance between plane A and plane B established according to 3D reconstruction technology, and the actual distance is the distance between plane A and plane B in the real state.

[0086] It should be noted that the points at corresponding positions within a point set in a parallel plane can form several point set lines, all of which are perpendicular to the parallel plane. For example, plane A contains a point set P. i (X wi ,Y wi Z wi ), i∈(0,N), have a point set P in plane B. j (X wj ,Y wj Z wj ), j∈(0,N), each pair of P i P j The line set of points formed is perpendicular to both plane A and plane B.

[0087] S500: Obtain the true distance between the parallel planes, and calculate the plane error between the interval distance between the parallel planes and the true distance using the error function.

[0088] In this embodiment, the true distance between parallel planes can be calibrated using the ranging function to obtain the true distance between the parallel planes. After obtaining the true distance, the true distance and the interval distance are input into the error function, and the plane error between the interval distance and the true distance is obtained based on the result calculated by the error function.

[0089] S600: Iteratively combine the extrinsic parameter matrix between the projector and the camera with the structured light coordinate relationship parameters and the plane error to obtain the extrinsic parameter calculation result.

[0090] In this embodiment, after obtaining the structured light coordinate relationship parameters and the plane error, the extrinsic parameter matrix between the projector and the camera is simultaneously iteratively calculated with the structured light coordinate relationship parameters and the plane error to obtain the extrinsic parameter calculation result. The coordinate transformation of the extrinsic parameter matrix can be achieved through the light coordinate relationship parameters, and the plane error is used to verify the extrinsic parameter calculation result, thereby improving the efficiency of the extrinsic parameter calculation and reducing the extrinsic parameter error.

[0091] In some embodiments, to improve the efficiency of combining the camera parameter matrix and the projector parameter matrix, it is necessary to use the same coordinate format for the matrices. Therefore, during the combination process, a first transformation matrix from the camera pixel coordinate system to the world coordinate system and a second transformation matrix from the projector pixel coordinate system to the world coordinate system can also be calculated.

[0092] After obtaining the first and second transformation matrices, it is necessary to iterate through the first transformation parameters in the first transformation matrix and the second transformation parameters in the second transformation matrix, respectively. Then, based on the first and second transformation parameters, the camera parameter matrix is ​​converted into a first simultaneous matrix, and the projector parameter matrix is ​​converted into a second simultaneous matrix, respectively. Both the first and second simultaneous matrices are matrices in the world coordinate system after transformation. After unifying the coordinate format, the structured light coordinate relationship parameters are calculated by simultaneously forming the first and second simultaneous matrices.

[0093] In some embodiments, the first transformation matrix can be calculated by calibrating other camera parameters. In this embodiment, the camera intrinsic matrix, camera rotation matrix, and camera translation matrix can be obtained through calibration. Then, the first transformation matrix is ​​calculated using the camera intrinsic matrix, camera rotation matrix, and camera translation matrix.

[0094] In some embodiments, the camera intrinsic matrix, camera rotation matrix, and camera translation matrix can be calculated using preset formulas. For example, the camera intrinsic matrix, camera rotation matrix, and camera translation matrix can be input into a first preset formula to calculate a first transformation matrix.

[0095] In this embodiment, the camera intrinsic parameter matrix is:

[0096]

[0097] Among them, f cx f cy s is the ratio of the camera lens focal length to the physical size of each pixel (the size of the CMOS sensor) in the x and y directions. cx s cy The coordinates of the center point of the lens in the camera pixel coordinate system.

[0098] It should be noted that in some embodiments of this application, the x-direction can also be referred to as the width direction, and the y-direction can also be referred to as the height direction.

[0099] The calculation process for the first transformation matrix can be as follows:

[0100]

[0101] Where, r c For the camera rotation matrix; t c Let be the camera translation matrix.

[0102] The camera rotation matrix can be:

[0103]

[0104] Where, r c11 -r c33 These are the camera rotation parameters.

[0105] The camera translation matrix can be:

[0106]

[0107] Among them, t c11 -t c13 These are the camera translation parameters.

[0108] In some embodiments, the second transformation matrix can also be calculated by calibrating other parameters of the projector. However, before calibration, the projector rotation matrix needs to be calculated based on the matrices obtained from other calibrations. First, the extrinsic rotation matrix between the projector and the camera needs to be obtained. The extrinsic rotation matrix is ​​the matrix obtained by rotating the extrinsic parameters between the projector and the camera in coordinates.

[0109] After obtaining the calibrated extrinsic rotation matrix, the product of the extrinsic rotation matrix and the camera rotation matrix can be calculated to obtain the projector rotation matrix, which is the rotation matrix from the projector coordinate system to the world coordinate system. After obtaining the projector rotation matrix, the projection intrinsic parameter matrix and projection translation matrix are further calibrated to obtain them. Then, the projection intrinsic parameter matrix, projection translation matrix, and projector rotation matrix are input into the second preset formula to calculate the second transformation matrix.

[0110] In some embodiments, the extrinsic rotation matrix can be:

[0111]

[0112] Where, r s11 -r s33 These are external rotation parameters.

[0113] The calculation process for the projector rotation matrix is ​​as follows:

[0114]

[0115] In some embodiments, during the process of obtaining the projection translation matrix through calibration, the method can also obtain the extrinsic translation matrix between the projector and the camera, wherein the extrinsic translation matrix is ​​the matrix obtained by coordinate translation of the extrinsic parameters between the projector and the camera. Then, the matrix product of the extrinsic rotation matrix and the camera translation matrix is ​​calculated. After obtaining the matrix product, the extrinsic translation matrix and the matrix product are summed to obtain the projection translation matrix.

[0116] The calculation process for the projection translation matrix is ​​as follows:

[0117]

[0118] Among them, t s To obtain the extrinsic translation matrix between the projector and the camera:

[0119]

[0120] Among them, t c11 -t c13 These are the translation parameters of the extrinsic parameters;

[0121] The projection intrinsic parameter matrix is:

[0122]

[0123] Among them, f px f py s is the ratio of the projector lens focal length to the physical size of each pixel (the size of the CMOS sensor) in the x and y directions. px s py The coordinates of the projector's center point in the projector's pixel coordinate system.

[0124] The calculation process for the second transformation matrix is ​​as follows:

[0125]

[0126] After calculation, the second transformation parameter can be expressed as follows:

[0127] T p11 =f px ·(r s11 ·r c11 +r s12 ·r c21 +r s13 ·r c31 )+s px ·(r s31·r c11 +r s32 ·r c21 +r s33 ·r c31 );

[0128] T p12 =f px ·(r s11 ·r c12 +r s12 ·r c22 +r s13 ·r c32 )+s px ·(r s31 ·r c12 +r s32 ·r c22 +r s33 ·r c32 );

[0129] T p13 =f px ·(r s11 ·r c13 +r s12 ·r c23 +r s13 ·r c33 )+s px ·(r s31 ·r c13 +r s32 ·r c23 +r s33 ·r c33 );

[0130] T p14 =f px ·(t s11 +r s11 ·t c11 +r s12 ·t c21 +r s13 ·t c31 )+s px ·(t s31 +r s31 ·t c11 +r s32 ·t c21 +r s33 ·t c31 );

[0131] T p21 =f py ·(r s21 ·r c11 +r s22 ·r c21 +r s23·r c31 )+s py ·(r s31 ·r c11 +r s32 ·r c21 +r s33 ·r c31 );

[0132] T p22 =f py ·(r s21 ·r c12 +r s22 ·r c22 +r s23 ·r c32 )+s py ·(r s31 ·r c12 +r s32 ·r c22 +r s33 ·r c32 );

[0133] T p23 =f py ·(r s21 ·r c13 +r s22 ·r c23 +r s23 ·r c33 )+s py ·(r s31 ·r c13 +r s32 ·r c23 +r s33 ·r c33 );

[0134] T p24 =f py ·(t s21 +r s21 ·t c11 +r s22 ·t c21 +r s23 ·t c31 )+s py ·(t s31 +r s31 ·t c11 +r s32 ·t c21 +r s33 ·t c31 );

[0135] T p31 =r s31 ·r c11 +rs32 ·r c21 +r s33 ·r c31 ;

[0136] T p32 =r s31 ·r c12 +r s32 ·r c22 +r s33 ·r c32 ;

[0137] T p33 =r s31 ·r c13 +r s32 ·r c23 +r s33 ·r c33 ;

[0138] T p34 =t s31 +r s31 ·t c11 +r s32 ·t c21 +r s33 ·t c31 .

[0139] In some embodiments, during the calculation of plane error, if a large number of parallel planes are constructed, the interval distance between the parallel planes to be calculated can also be obtained. After obtaining the interval distance, the plane error can be obtained by calculating the error between the true distance and the interval distance using the following error function.

[0140]

[0141] Where D is the true distance between parallel planes; d is the interval distance between parallel planes; α is the error function constant, α∈[0,1]; N is the summation range of the error function.

[0142] like Figure 4 As shown, in some embodiments, the planar error can also be determined by setting a distance error threshold to determine whether it meets the standard. In this embodiment, the planar error can be compared with the distance error threshold. If the planar error is greater than the distance error threshold, it indicates that the planar error is large and does not meet the output standard. At this time, the extrinsic parameter matrix between the projector and the camera, the structured light coordinate relationship parameters, and the planar error are iteratively combined for calculation. During the iterative calculation process, the error function needs to be adjusted for r. s11 r s12 r s13 r s21 r s22 rs23 r s31 r s32 r s33 t s11 t s21 t s31 Several parameters are updated. If the plane error is less than the distance error threshold, it means that the plane error has reached the output standard, and the plane error is output at this time.

[0143] In some embodiments, the function for iterative computation may further include:

[0144]

[0145]

[0146]

[0147]

[0148]

[0149]

[0150]

[0151] t q1,k =t q1,k-1 -Δx q1，k

[0152] Among them, m=[1,2,3], n=[1,2,3], q=[1,2,3], λ∈(0,3), β=0.9, ε=10 -6 , k∈(0,10000], Δx represents the update amount.

[0153] During iterative calculations, the above formula is executed. r m n, k, t q1,k To update parameter r s11 r s12 r s13 r s21 r s22 r s23 r s31 r s32 r s33 , t s11 , t s21 , t s31 .

[0154] In some embodiments, the extrinsic parameter matrix includes an extrinsic rotation matrix and an extrinsic translation matrix. If the planar error is greater than the distance error threshold, the extrinsic rotation parameters of the extrinsic rotation matrix and the extrinsic translation parameters of the extrinsic translation matrix can be jointly input as iterative parameters into the loss function and combined with the structured light relationship parameters to update the extrinsic rotation and translation parameters. Then, the parameter errors of the extrinsic rotation and translation parameters before and after the update are calculated, and the parameter errors are compared again with the error threshold as planar errors to reduce the parameter errors of the extrinsic rotation and translation parameters and improve the calibration efficiency of the extrinsic parameters.

[0155] The above method can also be implemented through system hardware. In some embodiments of this application, a structured light extrinsic parameter iteration calculation system is also provided. The system includes a projector, a camera, and a calculation module, which is configured to perform:

[0156] S100: Establish a reference point, and obtain the first parameter coordinates in the world coordinate system, the second parameter coordinates in the camera pixel coordinate system, and the third parameter coordinates in the projector pixel coordinate system based on the reference point.

[0157] S200: Construct a camera parameter matrix based on the first parameter coordinates, the second parameter coordinates, and the camera scaling factor; and construct a projector parameter matrix based on the first parameter coordinates, the third parameter coordinates, and the projector scaling factor.

[0158] S300: Combine the camera parameter matrix and the projector parameter matrix to obtain the structured light coordinate relationship parameters.

[0159] S400: Establish a preset number of parallel planes.

[0160] The parallel planes contain a set of planar points, and the parallel planes are spaced at the same distance, with the straight line formed by the corresponding sets of planar points perpendicular to the parallel planes.

[0161] S500: Obtain the true distance between the parallel planes, and calculate the plane error between the interval between the parallel planes and the true distance using the error function.

[0162] S600: Iteratively combine the extrinsic parameters between the projector and the camera with the structured light coordinate relationship parameters and the plane error to obtain the extrinsic parameter calculation results.

[0163] As described above, this application provides a method and system for iterative calculation of extrinsic parameters in structured light. The method establishes a reference point and obtains its first parameter coordinates in the world coordinate system, second parameter coordinates in the camera pixel coordinate system, and third parameter coordinates in the projector pixel coordinate system. Based on these coordinates and corresponding scaling factors, camera parameter matrices and projector parameter matrices are constructed, and these matrices are combined to obtain the structured light coordinate relationship parameters. Then, parallel planes are established, and the plane error between the true distance to the parallel planes and the distance between them is calculated using an error function. Finally, the extrinsic parameter matrix, structured light coordinate relationship parameters, and plane error are combined to obtain the extrinsic parameter calculation result. This application improves the efficiency and stability of extrinsic parameter calculation by combining matrices, establishing different parallel planes, and using an error function to calculate the extrinsic parameter error. Furthermore, iterative calculation reduces the extrinsic parameter error.

[0164] Those skilled in the art will clearly understand that the techniques in the embodiments of the present invention can be implemented using software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solutions in the embodiments of the present invention, or the parts that contribute to the prior art, can be embodied in the form of a software product, which can be stored in a computer-readable storage medium.

[0165] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

[0166] For ease of explanation, the above description has been provided in conjunction with specific embodiments. However, the foregoing exemplary discussion is not intended to be exhaustive or to limit the embodiments to the specific forms disclosed above. Various modifications and variations can be made based on the foregoing teachings. The selection and description of the above embodiments are for the purpose of better explaining the contents of this disclosure, thereby enabling those skilled in the art to better utilize the described embodiments.

Claims

1. A method for iterative calculation of extrinsic parameters of structured light, characterized in that, The method includes: Establish a reference point, and obtain the first parameter coordinates in the world coordinate system, the second parameter coordinates in the camera pixel coordinate system, and the third parameter coordinates in the projector pixel coordinate system based on the reference point; A camera parameter matrix is ​​constructed based on the first parameter coordinates, the second parameter coordinates, and the camera scaling factor; and a projector parameter matrix is ​​constructed based on the first parameter coordinates, the third parameter coordinates, and the projector scaling factor. Calculate a first transformation matrix from the camera pixel coordinate system to the world coordinate system, and calculate a second transformation matrix from the projector pixel coordinate system to the world coordinate system; Iterate through the first transformation parameter of the first transformation matrix and the second transformation parameter of the second transformation matrix; The camera parameter matrix is ​​converted into a first simultaneous matrix in the world coordinate system according to the first transformation parameter, and the projector parameter matrix is ​​converted into a second simultaneous matrix in the world coordinate system according to the second transformation parameter. The first and second simultaneous matrices are combined to calculate the structured light coordinate relationship parameters, thus obtaining the structured light coordinate relationship parameters; Establish a preset number of parallel planes; each parallel plane contains a set of plane points, the parallel planes are spaced at different distances and the straight lines formed by the corresponding sets of plane points are perpendicular to the parallel planes; Obtain the true distance between the parallel planes, and calculate the plane error between the interval distance between the parallel planes and the true distance using an error function; The extrinsic parameter matrix between the projector and the camera, the structured light coordinate relationship parameters, and the plane error are iteratively combined to calculate the extrinsic parameter calculation results.

2. The method for iterative calculation of extrinsic parameters of structured light according to claim 1, characterized in that, In the step of calculating the first transformation matrix from the camera pixel coordinate system to the world coordinate system, the method further includes: The camera intrinsic parameter matrix, camera rotation matrix, and camera translation matrix are obtained through calibration. The camera intrinsic parameter matrix, the camera rotation matrix, and the camera translation matrix are input into a first preset formula to calculate the first transformation matrix.

3. The method for iterative calculation of extrinsic parameters of structured light according to claim 2, characterized in that, In the step of calculating the second transformation matrix from the projector pixel coordinate system to the world coordinate system, the method further includes: Obtain the extrinsic rotation matrix between the projector and the camera; the extrinsic rotation matrix is ​​a matrix obtained by rotating the extrinsic parameters between the projector and the camera in coordinates. Calculate the product of the extrinsic rotation matrix and the camera rotation matrix to obtain the projector rotation matrix; the projector rotation matrix is ​​the rotation matrix from the projector coordinate system to the world coordinate system; The projection intrinsic parameter matrix and projection translation matrix are obtained through calibration. The projection intrinsic parameter matrix, the projection translation matrix, and the projector rotation matrix are input into the second preset formula to calculate the second transformation matrix.

4. The method for iterative calculation of extrinsic parameters of structured light according to claim 3, characterized in that, The method further includes the following steps in obtaining the projection intrinsic parameter matrix and the projection translation matrix through calibration: Obtain the extrinsic translation matrix between the projector and the camera; the extrinsic translation matrix is ​​a matrix obtained by performing coordinate translation on the extrinsic parameters between the projector and the camera. Calculate the matrix product of the extrinsic rotation matrix and the camera translation matrix; The product of the extrinsic translation matrix and the matrix is ​​summed to obtain the projection translation matrix.

5. The method for iterative calculation of extrinsic parameters of structured light according to claim 3, characterized in that, The camera intrinsic parameter matrix is: in, , For camera lens focal length and x , y The ratio of the physical size of each pixel in the direction. , The coordinates of the lens center point in the camera pixel coordinate system; The projection intrinsic parameter matrix is: in, , For the projector lens focal length and x , y The ratio of the physical size of each pixel in the direction. , The coordinates of the projector's center point in the projector's pixel coordinate system.

6. The method for iterative calculation of extrinsic parameters of structured light according to claim 1, characterized in that, In the step of iteratively calculating the extrinsic parameter matrix between the projector and the camera, the structured light coordinate relationship parameters, and the plane error, the method further includes: Compare the plane error with the distance error threshold; If the planar error is greater than the distance error threshold, then the extrinsic parameter matrix between the projector and the camera, the structured light coordinate relationship parameters, and the planar error are iteratively combined for calculation. If the plane error is less than the distance error threshold, then the plane error is output.

7. The method for iterative calculation of extrinsic parameters of structured light according to claim 6, characterized in that, The extrinsic parameter matrix includes an extrinsic rotation matrix and an extrinsic translation matrix, and the method further includes: If the planar error is greater than the distance error threshold, the extrinsic rotation parameters of the extrinsic rotation matrix and the extrinsic translation parameters of the extrinsic translation matrix are input into the loss function as iterative parameters, and the structured light coordinate relationship parameters are combined to update the extrinsic rotation parameters and the extrinsic translation parameters. Calculate the parameter errors of the external parameter rotation and external parameter translation before and after the update; The parameter error is then compared again with the error threshold as a plane error.

8. A structured light extrinsic parameter iterative calculation system, characterized in that, The system includes a projector, a camera, and a computing module, the computing module being configured to perform: Establish a reference point, and obtain the first parameter coordinates in the world coordinate system, the second parameter coordinates in the camera pixel coordinate system, and the third parameter coordinates in the projector pixel coordinate system based on the reference point; A camera parameter matrix is ​​constructed based on the first parameter coordinates, the second parameter coordinates, and the camera scaling factor; and a projector parameter matrix is ​​constructed based on the first parameter coordinates, the third parameter coordinates, and the projector scaling factor. Calculate a first transformation matrix from the camera pixel coordinate system to the world coordinate system, and calculate a second transformation matrix from the projector pixel coordinate system to the world coordinate system; Iterate through the first transformation parameter of the first transformation matrix and the second transformation parameter of the second transformation matrix; The camera parameter matrix is ​​converted into a first simultaneous matrix in the world coordinate system according to the first transformation parameter, and the projector parameter matrix is ​​converted into a second simultaneous matrix in the world coordinate system according to the second transformation parameter. The first and second simultaneous matrices are combined to calculate the structured light coordinate relationship parameters, thus obtaining the structured light coordinate relationship parameters; Establish a preset number of parallel planes; each parallel plane contains a set of plane points, the parallel planes are spaced at the same distance from each other, and the straight line formed by the corresponding sets of plane points is perpendicular to the parallel plane; Obtain the true distance between the parallel planes, and calculate the plane error between the interval between the parallel planes and the true distance using an error function; The extrinsic parameters between the projector and the camera, the structured light coordinate relationship parameters, and the plane error are iteratively combined to calculate the extrinsic parameter calculation results.

Images