Parameter calibration method, three-dimensional reconstruction method, electronic device, and storage medium

Abstract

The application provides a parameter calibration method, a three-dimensional reconstruction method, an electronic device and a storage medium. The parameter calibration method is applied to a three-dimensional camera, and the three-dimensional camera comprises a laser, a galvanometer and at least one optical camera. The parameter calibration method comprises the following steps: obtaining initial calibration parameters of the three-dimensional camera; performing iterative calibration on the initial calibration parameters; obtaining target calibration parameters of the three-dimensional camera when a stop condition of the iterative calibration is reached, and completing calibration of the three-dimensional camera. The method provided by the application can solve the problem of how to improve the accuracy of the calibrated parameters in laser three-dimensional reconstruction and improve the effect of laser three-dimensional reconstruction.

Description

Technical Field

[0001] This application relates to three-dimensional reconstruction technology, and more particularly to a parameter calibration method, a three-dimensional reconstruction method, an electronic device, and a storage medium. Background Technology

[0002] With the rapid development of machine vision, intelligent manufacturing, security, and logistics, non-contact rapid 3D reconstruction technology has become a major research hotspot. Currently, the mainstream 3D reconstruction solutions mainly include binocular RGB solutions, structured light solutions, and Time-of-Flight (TOF) solutions.

[0003] Laser 3D reconstruction uses a laser and an industrial camera to perform 3D reconstruction. The laser scans an object and sends the reflected laser light to the industrial camera, which then captures the reflected light. In 3D reconstruction, the laser galvanometer system and the industrial camera must first be calibrated, and then 3D reconstruction is performed based on the calibration parameters. However, existing methods do not avoid the inherent errors in the calibration parameters themselves, leading to inaccurate calibration parameters and consequently affecting the 3D reconstruction results.

[0004] Improving the accuracy of calibration parameters in laser 3D reconstruction and enhancing the reconstruction effect remains a worthwhile consideration. Summary of the Invention

[0005] This application provides a parameter calibration method, a three-dimensional reconstruction method, an electronic device, and a storage medium to address how to improve the accuracy of calibrated parameters in laser three-dimensional reconstruction and enhance the laser three-dimensional reconstruction effect.

[0006] In a first aspect, this application provides a parameter calibration method applied to a 3D camera, the 3D camera including a laser, a galvanometer, and at least one optical camera, the method comprising:

[0007] Obtain the initial calibration parameters of the 3D camera;

[0008] The initial calibration parameters are iteratively calibrated until the stopping condition for iterative calibration is met, at which point the target calibration parameters of the 3D camera are obtained, thus completing the calibration of the 3D camera.

[0009] In one embodiment, the initial calibration parameters include at least: intrinsic parameters of the optical camera, distortion coefficients of the optical camera, linear parameters of the laser, and nonlinear parameters of the laser; wherein, the linear parameters of the laser include the intrinsic parameters of the laser, as well as a first rotation matrix and a first translation matrix between the laser and the optical camera;

[0010] The step of iteratively calibrating the initial calibration parameters until the stopping condition for iterative calibration is met to obtain the target calibration parameters of the 3D camera includes:

[0011] For the first calibration in the iterative calibration, the intrinsic parameters of the optical camera, the distortion coefficients, the linear parameters, and the nonlinear parameters in the initial calibration parameters are jointly updated to obtain the calibration parameters;

[0012] For each calibration after the first calibration, the intrinsic parameters of the optical camera, the distortion coefficient, the linear parameter, and the nonlinear parameter in the calibration parameters obtained from the previous calibration are jointly updated; when the stopping condition of iterative calibration is reached, the calibration parameters of the last calibration are obtained as the target calibration parameters.

[0013] In one embodiment, for each calibration in the iterative calibration, jointly updating the intrinsic parameters of the optical camera, the distortion coefficients, the linear parameters, and the nonlinear parameters includes:

[0014] Obtain the three-dimensional coordinates of each feature point on the calibration board in multiple calibration board coordinate systems;

[0015] Based on the three-dimensional coordinates of each feature point in multiple calibration plate coordinate systems, multiple second rotation matrices and multiple second translation matrices between the calibration plate and the optical camera are calibrated to obtain updated multiple second rotation matrices and updated multiple second translation matrices.

[0016] Based on the updated second rotation matrices and the updated second translation matrices, the pixel coordinates of each feature point in each calibration plate image are obtained. The number of calibration plate images is multiple, which are taken by the optical camera when the calibration plate and the optical camera are in different relative positions.

[0017] The intrinsic parameters of the optical camera and the distortion coefficients are re-determined based on the pixel coordinates of each feature point in each calibration plate image;

[0018] Based on the pixel coordinates of each feature point in each calibration board image, and the updated multiple second rotation matrices and updated multiple second translation matrices, the linear parameters and the nonlinear parameters are redefined;

[0019] The calibration parameters include the redefined intrinsic parameters of the optical camera, the distortion coefficients, the linear parameters, and the nonlinear parameters.

[0020] In one embodiment, obtaining the pixel coordinates of each feature point in each calibration board image based on the updated plurality of second rotation matrices and the updated plurality of second translation matrices includes:

[0021] Based on the three-dimensional coordinates of each feature point in the calibration board coordinate system, as well as the updated second rotation matrix and the updated second translation matrix, the pixel coordinates of each feature point in each calibration board image are obtained.

[0022] In one embodiment, after redetermining the linear parameters and the nonlinear parameters based on the pixel coordinates of each feature point in each calibration board image, and the updated plurality of second rotation matrices and updated plurality of second translation matrices, the method further includes:

[0023] Based on the three-dimensional coordinates of each feature point in the optical camera coordinate system, the updated multiple second rotation matrices and the updated multiple second translation matrices, as well as the redefined linear parameters and nonlinear parameters, are then jointly updated.

[0024] For each calibration in the iterative calibration, the calibration parameters include the redefined intrinsic parameters of the optical camera and the distortion coefficients, as well as the jointly updated linear parameters and nonlinear parameters.

[0025] In one embodiment, prior to the first calibration in the iterative calibration, the method further includes:

[0026] The intrinsic parameters of the laser, the nonlinear parameters, the first rotation matrix, and the first translation matrix in the initial calibration parameters are updated based on the three-dimensional coordinates of each feature point on the calibration board in the camera coordinate system when the calibration board and the optical camera are in different relative positions. Furthermore, the intrinsic parameters of the optical camera and the distortion coefficients are updated based on the pixel coordinates of each feature point in each calibration board image to obtain the parameters to be updated. The calibration board images are multiple, obtained by the optical camera when the calibration board and the optical camera are in different relative positions.

[0027] Based on the three-dimensional coordinates of each feature point in the camera coordinate system when the calibration plate and the optical camera are in different relative positions, the laser intrinsic parameters to be updated, the nonlinear parameters to be updated, the rotation matrix to be updated, and the translation matrix to be updated in the parameters to be updated are jointly updated. Then, combined with the optical camera intrinsic parameters to be updated and the distortion coefficients to be updated in the parameters to be updated, the initial update parameters are obtained.

[0028] The initial calibration parameters jointly updated during the first calibration are the initial update parameters.

[0029] In one embodiment, updating the intrinsic parameters of the laser, the nonlinear parameters, the first rotation matrix, and the first translation matrix in the initial calibration parameters based on the three-dimensional coordinates of each feature point on the calibration plate in the camera coordinate system when the calibration plate and the optical camera are in different relative positions includes:

[0030] Based on the three-dimensional coordinates of each feature point in the camera coordinate system, the intrinsic parameters of the laser are updated to obtain the laser intrinsic parameters to be updated.

[0031] The nonlinear parameters are updated based on the three-dimensional coordinates of each feature point in the camera coordinate system and the intrinsic parameters of the laser to be updated, to obtain the nonlinear parameters to be updated.

[0032] The first rotation matrix and the first translation matrix are updated based on the three-dimensional coordinates of each feature point in the camera coordinate system, the intrinsic parameters of the laser to be updated, and the nonlinear parameters to be updated, to obtain the rotation matrix to be updated and the translation matrix to be updated.

[0033] In one embodiment, obtaining the initial calibration parameters of the 3D camera includes:

[0034] Multiple calibration plate images are acquired when the calibration plate and the optical camera are in different relative positions, and each calibration plate has multiple feature points.

[0035] Based on the multiple calibration board images, obtain the pixel coordinates of each feature point in each calibration board image;

[0036] Obtain the rotation and translation matrices between the calibration board coordinate system and the camera coordinate system when the calibration board and the optical camera are in different relative positions;

[0037] The optical camera is calibrated based on the pixel coordinates of each feature point in each calibration plate image, the rotation matrix and translation matrix between the calibration plate coordinate system and the camera coordinate system, to obtain the intrinsic parameters of the optical camera and the distortion coefficients.

[0038] The initial value estimation yields the linear parameters and the nonlinear parameters.

[0039] In one embodiment, the stopping condition for the iterative calibration includes any one of the following:

[0040] The difference between any parameter obtained in the Nth calibration and any parameter obtained in the (N-1)th calibration is within a preset range;

[0041] The number of iterations for calibration is equal to the preset number;

[0042] The iteration calibration duration is greater than or equal to the preset duration.

[0043] In one embodiment, the initial calibration parameters are obtained based on a camera-like model calibration. The camera-like model includes a camera-like coordinate system. The Y-axis of the camera-like coordinate system is the direction of the rotation axis of the galvanometer, and the Z-axis is the depth-of-field direction of the incident light exiting the galvanometer at a preset angle.

[0044] On the other hand, this application provides a three-dimensional reconstruction method applied to a three-dimensional camera, the three-dimensional camera including a laser, a galvanometer, and at least one optical camera, the method comprising:

[0045] Obtain the target calibration parameters of the 3D camera obtained by the parameter calibration method described in the first aspect;

[0046] 3D reconstruction is performed based on the target calibration parameters of the 3D camera.

[0047] On the other hand, this application provides a parameter calibration device applied to a 3D camera, the 3D camera including a laser, a galvanometer, and at least one optical camera, the device comprising:

[0048] The acquisition module is used to acquire the initial calibration parameters of the 3D camera;

[0049] The calibration module is used to iteratively calibrate the initial calibration parameters. When the stopping condition of the iterative calibration is reached, the target calibration parameters of the 3D camera are obtained, and the calibration of the 3D camera is completed.

[0050] On the other hand, this application provides a three-dimensional reconstruction apparatus applied to a three-dimensional camera, the three-dimensional camera including a laser, a galvanometer, and at least one optical camera, the apparatus comprising:

[0051] The acquisition module is used to acquire the target calibration parameters of the three-dimensional camera obtained by the parameter calibration method described in the first aspect;

[0052] The processing module is used to perform 3D reconstruction based on the target calibration parameters of the 3D camera.

[0053] On the other hand, this application provides an electronic device, including: a processor, and a memory communicatively connected to the processor;

[0054] The memory stores computer-executed instructions;

[0055] The processor executes computer execution instructions stored in the memory to implement the parameter calibration method as described in the first aspect, or the three-dimensional reconstruction method as described in the second aspect.

[0056] On the other hand, this application provides a computer-readable storage medium storing computer-executable instructions, which, when executed, cause the computer to perform the parameter calibration method as described in the first aspect, or to perform the three-dimensional reconstruction method as described in the second aspect.

[0057] This application provides a parameter calibration method applied to a 3D camera, which includes a laser, a galvanometer, and at least one optical camera. The method involves acquiring initial calibration parameters of the 3D camera, iteratively calibrating these initial parameters, and obtaining target calibration parameters for the 3D camera when a stopping condition for the iterative calibration is met, thus completing the calibration of the 3D camera. The purpose of iterative calibration is to optimize the initial calibration parameters; during the iterative calibration process, errors in the initial calibration parameters are gradually corrected. Therefore, the parameter calibration method provided by this application can, to a certain extent, avoid errors inherent in the calibration parameters themselves, improve the accuracy of the calibrated parameters in laser 3D reconstruction, and thereby enhance the laser 3D reconstruction effect.

[0058] Experiments have shown that 3D reconstruction based on these target calibration parameters can further reduce the fitting error during the calibration process, improving accuracy by at least 5% on top of the original bat accuracy. Attached Figure Description

[0059] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0060] Figure 1 A schematic diagram illustrating an application scenario of the parameter calibration method provided in this application;

[0061] Figure 2 A flowchart illustrating a parameter calibration method provided in one embodiment of this application;

[0062] Figure 3 A schematic diagram illustrating the acquisition of a calibration plate image provided for one embodiment of this application;

[0063] Figure 4 A schematic diagram of a camera-like model provided for one embodiment of this application;

[0064] Figure 5 A schematic diagram of the process of any calibration in the iterative calibration provided for one embodiment of this application;

[0065] Figure 6 A flowchart illustrating any calibration step in an iterative calibration process provided for another embodiment of this application;

[0066] Figure 7 A schematic diagram illustrating the process of optimizing the initial calibration parameters before iterative calibration, provided for one embodiment of this application;

[0067] Figure 8 A flowchart illustrating a portion of the optimization process in one embodiment of this application;

[0068] Figure 9 A flowchart illustrating a three-dimensional reconstruction method provided in one embodiment of this application;

[0069] Figure 10 A schematic diagram of a parameter calibration device provided in one embodiment of this application;

[0070] Figure 11 A schematic diagram of a three-dimensional reconstruction apparatus provided in one embodiment of this application;

[0071] Figure 12 A schematic diagram of an electronic device provided for one embodiment of this application.

[0072] The accompanying drawings have illustrated specific embodiments of this disclosure, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concepts of this disclosure to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0073] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0074] In the description of this application, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0075] With the rapid development of machine vision, intelligent manufacturing, security, and logistics, non-contact rapid 3D reconstruction technology has become a major research hotspot. Currently, the mainstream 3D reconstruction solutions mainly include binocular RGB solutions, structured light solutions, and Time-of-Flight (TOF) solutions.

[0076] Laser 3D reconstruction uses a laser and an industrial camera to perform 3D reconstruction. The laser scans the object and sends the reflected laser light to the industrial camera, which then captures the reflected light. The 3D reconstruction process first requires calibrating both the laser's galvanometer system and the industrial camera, and then using the calibrated parameters to perform the 3D reconstruction.

[0077] However, existing methods do not avoid the inherent errors in the calibration parameters themselves during calibration, leading to inaccurate calibration parameters and consequently affecting the 3D reconstruction results. These inherent errors in the calibration parameters may be caused by improper calibration procedures or other reasons. Improving the accuracy of the calibration parameters in laser 3D reconstruction can enhance the overall reconstruction outcome.

[0078] Based on this, this application provides a parameter calibration method, a 3D reconstruction method, an electronic device, and a storage medium. The parameter calibration method is applied to a 3D camera, which includes a laser, a galvanometer, and at least one optical camera. The method includes obtaining initial calibration parameters of the 3D camera, iteratively calibrating these initial parameters, and obtaining target calibration parameters of the 3D camera when a stopping condition for iterative calibration is met, thus completing the calibration of the 3D camera. The purpose of iterative calibration is to optimize the initial calibration parameters; during the iterative calibration process, errors in the initial calibration parameters are gradually corrected. Therefore, the parameter calibration method provided by this application can, to a certain extent, avoid errors inherent in the calibration parameters themselves, improve the accuracy of the calibrated parameters in laser 3D reconstruction, and thereby enhance the laser 3D reconstruction effect.

[0079] The parameter calibration method provided in this application is applicable to electronic devices, such as computers, laboratory servers, and 3D cameras. Please refer to... Figure 1 The electronic device acquires the initial calibration parameters of the 3D camera, performs iterative calibration on the initial calibration parameters, and obtains the target calibration parameters of the 3D camera when the stopping condition of the iterative calibration is reached, thus completing the calibration of the 3D camera.

[0080] Please see Figure 2 One embodiment of this application provides a parameter calibration method applied to a 3D camera, which includes a laser, a galvanometer, and at least one optical camera.

[0081] The parameter calibration method includes:

[0082] S210, obtain the initial calibration parameters of the 3D camera.

[0083] The initial calibration parameters of the 3D camera can be obtained after initial calibration, or they can be acquired from an external device. The method for initial calibration of the 3D camera can be selected according to actual needs, and this embodiment does not limit it. Preferably, the initial calibration of the 3D camera can be performed using a binarized fringe pattern. That is, a binarized fringe pattern is projected onto a calibration board using a laser, and then multiple images of the calibration board (including the binarized fringe pattern) are taken using an optical camera. The initial calibration of the 3D camera is then performed based on these multiple calibration board images.

[0084] The initial calibration parameters include at least the intrinsic parameter K of the optical camera. cam The distortion coefficient D of an optical camera cam The linear parameters and nonlinear parameters of the laser are a0, a1, a2, a3, a4, and a5. The linear parameters include the intrinsic parameter K. laser and the first rotation matrix between the laser and the optical camera. and the first translation matrix

[0085] In an optional embodiment, the intrinsic parameters K of the optical camera are obtained during the initial calibration of the 3D camera. cam and the distortion coefficient D cam At that time, multiple calibration board images are acquired, and initial calibration is performed based on these multiple calibration board images. For example... Figure 3 As shown, these multiple images of the calibration plate are taken when the calibration plate and the optical camera are in different relative positions. Figure 3 Images acquired at only three exemplary positions are shown. Each calibration board has multiple feature points. During initial calibration based on these multiple calibration board images, the pixel coordinates of each feature point in each calibration board image are obtained. The rotation and translation matrices between the calibration board coordinate system and the camera coordinate system are obtained when the calibration board and the optical camera are in different relative positions. Based on the pixel coordinates of each feature point in each calibration board image, and the rotation and translation matrices between the calibration board coordinate system and the camera coordinate system, the optical camera is calibrated to obtain the intrinsic parameters and distortion coefficients of the optical camera. It should be noted that these multiple calibration board images are images acquired when the calibration board and the optical camera are in different relative positions. Correspondingly, there are multiple calibration board coordinate systems, multiple rotation matrices between the calibration board coordinate system and the camera coordinate system, and multiple translation matrices between the calibration board coordinate system and the camera coordinate system.

[0086] When calibrating the optical camera, based on the pixel coordinates of each feature point in each calibration plate image, and the rotation and translation matrices between the calibration plate coordinate system and the camera coordinate system, the calibration can be performed using the formula... Determine the intrinsic parameters and distortion coefficients of the optical camera. Where x... ab K represents the pixel coordinates of the feature point in the calibration board image. cam D represents the internal parameters of an optical camera. cam R represents the distortion coefficient, m is the number of feature points on each calibration plate, and n represents the number of calibration plate images taken by the optical camera. i This represents the rotation matrix between the calibration board coordinate system and the camera coordinate system, T. i This represents the translation matrix between the calibration board coordinate system and the camera coordinate system.

[0087] In an optional embodiment, when the initial calibration of the 3D camera is performed to obtain the linear parameters and nonlinear parameters a0, a1, a2, a3, a4, a5 of the laser, the initial value estimation is used to obtain the linear parameters and nonlinear parameters.

[0088] The initial calibration parameters obtained from the initial calibration may contain errors, so it is necessary to recalibrate to optimize the calibration parameters, and to iterate the calibration to continuously optimize the calibration parameters, thereby improving the accuracy of the calibration parameters.

[0089] Please see Figure 4 In one optional embodiment, the initial calibration parameters are obtained based on a camera-like model calibration. This camera-like model includes a camera-like coordinate system, where the Y-axis direction is the direction of the rotation axis of the galvanometer, and the Z-axis direction is the depth-of-field direction of the incident light exiting the galvanometer at a preset angle. During calibration based on the camera-like model, a coded pattern can be projected onto a calibration board using a laser. The calibration board image captured by the optical camera and the coded pattern on the calibration board are then used to calibrate the 3D camera.

[0090] Specifically, after taking multiple images of the calibration board (with the calibration board and camera in different relative positions), the encoded pattern on the calibration board is decoded to obtain the decoding result. Based on the decoding result, the decoding result of each feature point on the calibration board is determined. Based on the decoding result of each feature point, the number of feature points on the calibration board, and the distance between adjacent feature points, the laser and optical camera are calibrated to obtain the initial calibration parameters.

[0091] The initial calibration parameters are obtained based on the decoding results of each feature point and the feature point information of the calibration board, and are divided into the following steps 1) to 3):

[0092] 1) Obtain the intrinsic parameter K of the optical camera based on the information of the feature points in the calibration plate. cam The distortion coefficient D of an optical camera cam The rotation matrices R1, R2, ..., R between the calibration plate and the optical camera n The translation matrices T1, T2, ..., T between the calibration board and the optical camera n , where n equals the number of calibration board images.

[0093] The information about the feature points in the calibration board includes the identification information of the feature points in the calibration board, and the feature point information of the calibration board (including at least the number of feature points on the calibration board and the distance between adjacent feature points). The identification information of the feature points in the calibration board refers to the identified feature points.

[0094] Specifically, the information of the feature points in the calibration board, including the identification information of the feature points in the calibration board, the number of feature points on the calibration board, and the distance between adjacent feature points, is used as input parameters for the calibration of the optical camera to obtain the intrinsic parameter K of the optical camera. cam The distortion coefficient D of an optical camera cam The rotation matrices R1, R2, ..., R between the calibration plate and the optical camera n The translation matrices T1, T2, ..., T between the calibration board and the optical camera n .

[0095] 2) Based on the decoding results of each feature point, the rotation matrix between the calibration board and the optical camera, and the translation matrix between the calibration board and the optical camera, determine the linear parameters of the laser, the rotation matrix between the laser and the optical camera, and the translation matrix between the laser and the optical camera.

[0096] Specifically, based on the decoding results of each feature point, the rotation matrices R1, R2, ..., R1 between the calibration board and the optical camera are... n The translation matrices T1, T2, ..., T between the calibration board and the optical camera n The linear parameter K of the laser is obtained through linear initial value estimation. laser Rotation matrix between laser and optical camera Translation matrix between laser and optical camera

[0097] 3) Determine the nonlinear parameters of the laser based on the intrinsic parameters of the optical camera, the distortion coefficient of the optical camera, the rotation matrix between the calibration plate and the optical camera, the translation matrix between the calibration plate and the optical camera, the linear parameters of the laser, the rotation matrix between the laser and the optical camera, and the translation matrix between the laser and the optical camera.

[0098] Specifically, based on the intrinsic parameter K of the optical camera cam The distortion coefficient D of an optical camera cam The rotation matrices R1, R2, ..., R between the calibration plate and the optical camera n Translation matrices T1, T2, ..., T between the calibration plate and the optical camera n Laser linearity parameter K laser Rotation matrix between laser and optical camera Translation matrix between laser and optical camera The nonlinear parameter D of the laser is solved iteratively using a nonlinear optimization method. laser Nonlinear optimization methods include gradient descent, Gauss-Newton's method, and the LM algorithm.

[0099] In an optional embodiment, the calibration parameters obtained from the calibration can also be optimized using the bundle adjustment method to obtain more accurate initial calibration parameters.

[0100] S220, perform iterative calibration on the initial calibration parameters, and obtain the target calibration parameters of the 3D camera when the stopping condition of the iterative calibration is reached, thus completing the calibration of the 3D camera.

[0101] Iterative calibration refers to substituting the result of the previous calibration into the next calibration process, with the aim of continuously optimizing the calibration result. The stopping condition for iterative calibration includes any of the following: the difference between any parameter obtained in the Nth calibration and any parameter obtained in the (N-1)th calibration is within a preset range; the number of iterations is equal to the preset number; the duration of the iteration is greater than or equal to the preset duration.

[0102] In this embodiment, when the difference between any parameter obtained in the Nth calibration and any parameter obtained in the (N-1)th calibration falls within a preset range, it indicates that the result of the iterative calibration is approaching stability, and the iterative calibration can be terminated at this point. This preset range can be set according to actual needs, and is not limited in this embodiment.

[0103] In an optional embodiment, during iterative calibration of the initial calibration parameters, for the first calibration in the iterative calibration, the intrinsic parameters of the optical camera, the distortion coefficient, the linear parameter, and the nonlinear parameter in the initial calibration parameters are jointly updated to obtain the calibration parameters. For each subsequent calibration, the intrinsic parameters of the optical camera, the distortion coefficient, the linear parameter, and the nonlinear parameter in the calibration parameters obtained from the previous calibration are jointly updated. When the stopping condition of the iterative calibration is met, the calibration parameters obtained from the last calibration are taken as the target calibration parameters.

[0104] That is, the first calibration in the iterative calibration jointly updates the initial calibration parameters, and each subsequent calibration jointly updates the calibration parameters obtained from the previous calibration. The following describes one specific calibration method for any one of the iterative calibrations. It is important to note that when this "any one calibration" refers to the first calibration, the jointly updated parameter is the initial calibration parameter.

[0105] like Figure 5 The flowchart shown illustrates that, for any calibration in the iterative calibration (i.e., each calibration), the pose relationship between the optical camera and the calibration board is first optimized. Based on the result of the pose relationship optimization, the intrinsic parameters of the optical camera and the distortion coefficient are then optimized, as are the linear and nonlinear parameters.

[0106] When optimizing the pose relationship between the optical camera and the calibration board, the three-dimensional coordinates of each feature point on the calibration board in multiple calibration board coordinate systems are obtained. Based on the three-dimensional coordinates of each feature point in the calibration board coordinate system, multiple second rotation matrices R and multiple second translation matrices T between the calibration board and the optical camera are updated to obtain multiple second rotation matrices R and multiple second translation matrices T.

[0107] Based on the three-dimensional coordinates of each feature point in the calibration plate coordinate system, multiple second rotation matrices R and multiple second translation matrices T are used to obtain updated multiple second rotation matrices R and updated multiple second translation matrices T between the calibration plate and the optical camera. For example, according to the formula... This yields multiple updated second rotation matrices R and multiple updated second translation matrices T. Where x ij R represents the three-dimensional coordinates of the feature point in the calibration plate coordinate system. i T represents the rotation matrix between each calibration plate coordinate system and the camera coordinate system of the optical camera. i This represents the translation matrix between the coordinate system of each calibration board and the camera coordinate system of the optical camera. `m` represents the number of feature points on each calibration board, and `n` represents the number of calibration board images taken by the optical camera. It should be noted that the `n` calibration board images are taken when the optical camera and the calibration board are in different relative positions. After iterating through each feature point and each calibration board image, this formula... R in i That is, the updated second rotation matrix R,T i This is the updated second translation matrix T.

[0108] When further optimizing the intrinsic parameters and distortion coefficients of the optical camera based on the results of pose relationship optimization, the pixel coordinates x of each feature point in each calibration plate image are obtained based on the updated second rotation matrices R and the updated second translation matrices T. ab Based on the pixel coordinates x of each feature point in each calibration board image. ab The intrinsic parameters and distortion coefficients of the optical camera were redefined. It should be noted that multiple images of the calibration board were taken by the optical camera when the calibration board and the optical camera were in different relative positions.

[0109] Based on the updated second rotation matrices R and the updated second translation matrices T, the pixel coordinates x of each feature point in each calibration board image are obtained. ab Specifically, based on the three-dimensional coordinates of each feature point in the calibration board coordinate system, and the updated second rotation matrices R and T, the pixel coordinates x of each feature point in each calibration board image are obtained. ab .

[0110] Based on the pixel coordinates x of each feature point in each calibration board image a，b When redetermining the intrinsic parameters of the optical camera and the distortion coefficient, for example, according to the formula... The intrinsic parameters and distortion coefficients of the optical camera are redefined. Where x ab K represents the pixel coordinates of the feature point in the calibration board image. cam D represents the internal parameters of an optical camera. cam R represents the distortion coefficient, m is the number of feature points on each calibration plate, and n represents the number of calibration plate images taken by the optical camera. i That is, the updated second rotation matrix R,T i This is the updated second translation matrix T. After traversing each feature point and each calibration board image, K is determined. cam That is, the redefined intrinsic parameters of the optical camera, D cam That is, the redefined distortion coefficient.

[0111] When optimizing the linear and nonlinear parameters based on the results of pose relationship optimization, the second rotation matrix R and the second translation matrix T are updated, along with the pixel coordinates x of each feature point in each calibration board image. ab Then, the linear parameter and the nonlinear parameter are redefined.

[0112] When redetermining this linear parameter, for example, according to the formula Redetermine the linear parameter. Where x ab K represents the pixel coordinates of the feature point in the calibration board image. laser This represents the linear parameter. This represents the first rotation matrix. Let K represent the second rotation matrix, m be the number of feature points on each calibration board, and n be the number of calibration board images taken by the optical camera. After traversing each feature point and each calibration board image, K is determined. laser That is, the linear parameter that has been redefined.

[0113] When redetermining this nonlinear parameter, for example, according to the formula The nonlinear parameter is redefined. Here, a0, a1, a2, a3, a4, a5 represent the nonlinear parameter, and x... ab The pixel coordinates of the feature points in the calibration board image are represented by , m is the number of feature points on each calibration board, and n represents the number of calibration board images taken by the optical camera. After iterating through each feature point and each calibration board image, the determined a0, a1, a2, a3, a4, and a5 are the redefined nonlinear parameters.

[0114] Therefore, the calibration parameters obtained from any calibration include the redefined intrinsic parameters of the optical camera, the distortion coefficient, the linear parameter, and the nonlinear parameter.

[0115] like Figure 6 As shown, in an optional embodiment, after redetermining the linear parameter and the nonlinear parameter based on the pixel coordinates of each feature point in each calibration board image, as well as the updated multiple second rotation matrices and the updated multiple second translation matrices (i.e., after optimizing the linear parameter and the nonlinear parameter based on the pose relationship optimization results), the updated multiple second rotation matrices R and the updated multiple second translation matrices T, as well as the redetermined linear parameter and the nonlinear parameter, can be jointly updated again.

[0116] For example, according to the formula The updated second rotation matrices R and T, along with the redefined linear and nonlinear parameters, are then jointly updated. In the formula, X... ij K represents the three-dimensional coordinates of the feature point in the camera coordinate system. laser R represents the redefined linear parameter, and a0, a1, a2, a3, a4, a5 represent the redefined nonlinear parameter. i Represents the updated second rotation matrix R,T i This represents the updated second translation matrix T. The first rotation matrix... and the first translation matrix This is known. m is the number of feature points on each calibration board, and n represents the number of images of the calibration board taken by the optical camera. The 3D coordinates of each feature point in the camera coordinate system are traversed to obtain the jointly updated linear parameter K. laser The nonlinear parameters a0, a1, a2, a3, a4, a5, the second rotation matrix R, and the second translation matrix T.

[0117] For each calibration in the iterative calibration, the calibration parameters obtained in each calibration include the redefined intrinsic parameter K of the optical camera. cam and the distortion coefficient D cam It also includes the linear parameter K obtained from the joint update. laser And the nonlinear parameters a0, a1, a2, a3, a4, a5.

[0118] After iterative calibration, the obtained calibration parameters will be more accurate.

[0119] In summary, the above embodiments provide a parameter calibration method applied to a 3D camera, which includes a laser, a galvanometer, and at least one optical camera. The parameter calibration method includes obtaining initial calibration parameters of the 3D camera, iteratively calibrating these initial calibration parameters, and obtaining the target calibration parameters of the 3D camera when a stopping condition for iterative calibration is met, thus completing the calibration of the 3D camera. The purpose of iterative calibration is to optimize the initial calibration parameters; during the iterative calibration process, errors in the initial calibration parameters are gradually corrected. Therefore, the parameter calibration method provided in this application can, to a certain extent, avoid errors inherent in the calibration parameters themselves, improve the accuracy of the calibrated parameters in laser 3D reconstruction, and thereby enhance the laser 3D reconstruction effect.

[0120] Experiments have shown that 3D reconstruction based on these target calibration parameters can further reduce the fitting error during the calibration process, improving accuracy by at least 5% on top of the original bat accuracy.

[0121] In an optional embodiment, the initial calibration parameters can be optimized before performing the iterative calibration, that is, before the first calibration in the iterative calibration.

[0122] For details, please see Figure 7 Based on the 3D coordinates of each feature point on the calibration board in the camera coordinate system when the calibration board and the optical camera are in different relative positions, the intrinsic parameters of the laser, the nonlinear parameters, the first rotation matrix, and the first translation matrix in the initial calibration parameters are updated. Based on the pixel coordinates of each feature point in each calibration board image, the intrinsic parameters of the optical camera and the distortion coefficient are updated separately to obtain the parameters to be updated. Based on the 3D coordinates of each feature point in the camera coordinate system when the calibration board and the optical camera are in different relative positions, the intrinsic parameters of the laser to be updated, the nonlinear parameters to be updated, the rotation matrix to be updated, and the translation matrix to be updated in the parameters to be updated are jointly updated. Combined with the intrinsic parameters of the optical camera to be updated and the distortion coefficients to be updated in the parameters to be updated, the initial updated parameters are obtained. The initial calibration parameters jointly updated during the first calibration in the iterative calibration are the initial updated parameters.

[0123] Please see Figure 8The laser's intrinsic parameters, nonlinear parameters, first rotation matrix, and first translation matrix in the initial calibration parameters are updated in steps. First, the laser's intrinsic parameters are updated based on the 3D coordinates of each feature point in the camera coordinate system to obtain the laser's intrinsic parameters to be updated. Then, the nonlinear parameters are updated based on the 3D coordinates of each feature point in the camera coordinate system and the laser's intrinsic parameters to be updated to obtain the nonlinear parameters to be updated. Finally, the first rotation matrix and the first translation matrix are updated based on the 3D coordinates of each feature point in the camera coordinate system, the laser's intrinsic parameters to be updated, and the nonlinear parameters to be updated to obtain the rotation matrix and the translation matrix to be updated.

[0124] For example, according to the formula The intrinsic parameters, nonlinear parameters, first rotation matrix, and first translation matrix of the laser in the initial calibration parameters are updated step by step. Where X... ij K represents the three-dimensional coordinates of the feature point in the camera coordinate system. laser a0, a1, a2, a3, a4, and a5 represent the linear parameters, while a0, a1, a2, a3, a4, and a5 represent the nonlinear parameters. This represents the first rotation matrix. This represents the first translation matrix. m is the number of feature points on each calibration board, and n represents the number of images of the calibration board taken by the optical camera. After iterating through the three-dimensional coordinates of each feature point in the camera coordinate system, the intrinsic parameter K of the laser is obtained. laser The nonlinear parameters a0, a1, a2, a3, a4, a5, and the first rotation matrix. and the first translation matrix That is, the updated intrinsic parameter K of the laser laser The nonlinear parameters a0, a1, a2, a3, a4, a5, and the first rotation matrix. and the first translation matrix

[0125] When updating in steps based on the above formula, the parameters not to be updated are known, while the parameters to be updated are unknown. For example, first update the intrinsic parameter K of the laser. laser When updating, the formula in a0, a1, a2, a3, a4, and a5 are all parameters in the initial calibration parameters, and they are known.

[0126] The intrinsic parameters of the optical camera and the distortion coefficient are updated based on the pixel coordinates of each feature point in each calibration board image to obtain the parameters to be updated. The calibration board images are multiple, captured by the optical camera when the calibration board and the optical camera are in different relative positions.

[0127] For example, according to the formula The intrinsic parameters of the optical camera and the distortion coefficient are updated to obtain the parameters to be updated. Where x... ab K represents the pixel coordinates of the feature point in the calibration board image. cam The intrinsic parameters of the optical camera, k1, k2, k3, p1, p2 are D. cam The specific parameters in the table represent the distortion coefficient. m is the number of feature points on each calibration board, and n represents the number of calibration board images taken by the optical camera.

[0128] Then, based on the three-dimensional coordinates of each feature point in the camera coordinate system when the calibration plate and the optical camera are in different relative positions, the laser intrinsic parameters to be updated, the nonlinear parameters to be updated, the rotation matrix to be updated, and the translation matrix to be updated in the parameters to be updated are jointly updated. For example, according to the formula... The laser intrinsic parameters, nonlinear parameters, rotation matrix, and translation matrix to be updated are jointly updated. In the formula, K... laser a0, a1, a2, a3, a4, and a5 represent the laser intrinsic parameters to be updated, and a4, a5 represent the nonlinear parameters to be updated. Represents the rotation matrix to be updated. This represents the translation matrix to be updated.

[0129] After jointly updating the laser intrinsic parameters, nonlinear parameters, rotation matrix, and translation matrix in the parameters to be updated, and combining them with the intrinsic parameters of the optical camera and the distortion coefficients in the parameters to be updated, the initial updated parameters are obtained. As described above, the initial calibration parameters jointly updated during the first calibration are the initial updated parameters.

[0130] It should be noted that the initial optimization of the calibration parameters performed before this iterative calibration, that is, before the first calibration in the iterative calibration, does not need to participate in the iterative calibration. This initial optimization can greatly improve the accuracy of the initial calibration parameters, and subsequent iterative calibrations can further improve the accuracy of the calibration parameters, thereby improving the laser 3D reconstruction effect.

[0131] Please see Figure 9 One embodiment of this application also provides a three-dimensional reconstruction method applied to a three-dimensional camera, the three-dimensional camera including a laser, a galvanometer, and at least one optical camera. The three-dimensional reconstruction method includes:

[0132] S910, obtains the target calibration parameters of the 3D camera obtained by calibration through a preset calibration method.

[0133] The preset calibration method is the parameter calibration method provided in any of the above embodiments.

[0134] The S920 performs 3D reconstruction based on the target calibration parameters of the 3D camera.

[0135] During the parameter calibration process, the calibration is continuously iterated and updated to make the target calibration parameters more accurate. Therefore, the 3D reconstruction based on these target calibration parameters yields significantly improved reconstruction results. Experiments have shown that 3D reconstruction based on these target calibration parameters can improve the accuracy of the original bat by at least 5%.

[0136] Please see Figure 10 An embodiment of this application also provides a parameter calibration device 10, applied to a 3D camera, the 3D camera including a laser, a galvanometer, and at least one optical camera, the parameter calibration device 10 including:

[0137] The acquisition module 11 is used to acquire the initial calibration parameters of the 3D camera. These initial calibration parameters are obtained based on the calibration of a camera-like model, which includes a camera-like coordinate system. The Y-axis of this camera-like coordinate system is the direction of the rotation axis of the galvanometer, and the Z-axis is the depth direction of the incident light exiting the galvanometer at a preset angle.

[0138] The calibration module 12 is used to iteratively calibrate the initial calibration parameters. When the stopping condition of the iterative calibration is reached, the target calibration parameters of the 3D camera are obtained, thus completing the calibration of the 3D camera. The stopping condition of the iterative calibration includes any one of the following: the difference between any parameter obtained in the Nth calibration and any parameter obtained in the (N-1)th calibration is within a preset range; the number of iterations is equal to the preset number; the duration of the iteration is greater than or equal to the preset duration.

[0139] The initial calibration parameters include at least: the intrinsic parameters of the optical camera, the distortion coefficient of the optical camera, the linear parameters of the laser, and the nonlinear parameters of the laser; wherein, the linear parameters of the laser include the intrinsic parameters of the laser, as well as the first rotation matrix and the first translation matrix between the laser and the optical camera. Specifically, the calibration module 12 is used to jointly update the intrinsic parameters of the optical camera, the distortion coefficient, the linear parameter, and the nonlinear parameter in the initial calibration parameters for the first calibration in the iterative calibration, to obtain the calibration parameters; for each calibration after the first calibration, it jointly updates the intrinsic parameters of the optical camera, the distortion coefficient, the linear parameter, and the nonlinear parameter in the calibration parameters obtained from the previous calibration; when the stopping condition of the iterative calibration is reached, the calibration parameters of the last calibration are obtained as the target calibration parameters.

[0140] The calibration module 12 is specifically used to obtain the three-dimensional coordinates of each feature point on the calibration board in multiple calibration board coordinate systems; to calibrate multiple second rotation matrices and multiple second translation matrices between the calibration board and the optical camera based on the three-dimensional coordinates of each feature point in multiple calibration board coordinate systems, thereby obtaining updated multiple second rotation matrices and updated multiple second translation matrices; to obtain the pixel coordinates of each feature point in each calibration board image based on the updated multiple second rotation matrices and updated multiple second translation matrices, wherein there are multiple calibration board images, which are obtained by the optical camera when the calibration board and the optical camera are in different relative positions; to redetermine the intrinsic parameters and the distortion coefficient of the optical camera based on the pixel coordinates of each feature point in each calibration board image; and to redetermine the linear parameter and the nonlinear parameter based on the pixel coordinates of each feature point in each calibration board image, as well as the updated multiple second rotation matrices and updated multiple second translation matrices; the calibration parameters include the redetermined intrinsic parameters of the optical camera, the distortion coefficient, the linear parameter, and the nonlinear parameter.

[0141] The calibration module 12 is specifically used to obtain the pixel coordinates of each feature point in each calibration board image based on the three-dimensional coordinates of each feature point in the calibration board coordinate system, as well as the updated multiple second rotation matrices and the updated multiple second translation matrices.

[0142] The calibration module 12 is also used to jointly update the updated multiple second rotation matrices and multiple updated second translation matrices, as well as the redefined linear parameter and nonlinear parameter, based on the three-dimensional coordinates of each feature point in the optical camera coordinate system. For each calibration in the iterative calibration, the calibration parameters include the redefined intrinsic parameters of the optical camera and the distortion coefficient, as well as the jointly updated linear parameter and nonlinear parameter.

[0143] The calibration module 12 is further configured to update the intrinsic parameters of the laser, the nonlinear parameters, the first rotation matrix, and the first translation matrix in the initial calibration parameters based on the three-dimensional coordinates of each feature point on the calibration board in the camera coordinate system when the calibration board and the optical camera are in different relative positions. It also updates the intrinsic parameters of the optical camera and the distortion coefficient based on the pixel coordinates of each feature point in each calibration board image to obtain the parameters to be updated. The calibration board images are multiple, obtained by the optical camera when the calibration board and the optical camera are in different relative positions. Based on the three-dimensional coordinates of each feature point in the camera coordinate system when the calibration board and the optical camera are in different relative positions, the intrinsic parameters of the laser to be updated, the nonlinear parameters to be updated, the rotation matrix to be updated, and the translation matrix to be updated in the parameters to be updated are jointly updated. Then, combined with the intrinsic parameters of the optical camera to be updated and the distortion coefficient to be updated in the parameters to be updated, the initial updated parameters are obtained. The initial calibration parameters jointly updated during the first calibration are the initial updated parameters.

[0144] The calibration module 12 is also used to update the intrinsic parameters of the laser based on the three-dimensional coordinates of each feature point in the camera coordinate system to obtain the laser intrinsic parameters to be updated; to update the nonlinear parameter based on the three-dimensional coordinates of each feature point in the camera coordinate system and the laser intrinsic parameters to be updated to obtain the nonlinear parameter to be updated; and to update the first rotation matrix and the first translation matrix based on the three-dimensional coordinates of each feature point in the camera coordinate system, the laser intrinsic parameters to be updated, and the nonlinear parameter to be updated to obtain the rotation matrix and the translation matrix to be updated.

[0145] The acquisition module 11 is specifically used to acquire multiple calibration plate images, which are images acquired when the calibration plate and the optical camera are in different relative positions. Each calibration plate has multiple feature points. Based on the multiple calibration plate images, the pixel coordinates of each feature point in each calibration plate image are acquired. The rotation and translation matrices between the calibration plate coordinate system and the camera coordinate system are acquired when the calibration plate and the optical camera are in different relative positions. Based on the pixel coordinates of each feature point in each calibration plate image and the rotation and translation matrices between the calibration plate coordinate system and the camera coordinate system, the optical camera is calibrated to obtain the intrinsic parameters and distortion coefficients of the optical camera. Initial value estimation is used to obtain the linear parameters and nonlinear parameters.

[0146] Please see Figure 11 An embodiment of this application also provides a three-dimensional reconstruction device 20, applied to a three-dimensional camera, the three-dimensional camera including a laser, a galvanometer, and at least one optical camera, the three-dimensional reconstruction device 20 including:

[0147] The acquisition module 21 is used to acquire the target calibration parameters of the three-dimensional camera obtained by calibration through a preset calibration method, such as the parameter calibration method provided in any of the above embodiments.

[0148] Processing module 22 is used to perform 3D reconstruction based on the target calibration parameters of the 3D camera.

[0149] Please see Figure 12 One embodiment of this application also provides an electronic device 30, including: a processor 31, and a memory 32 communicatively connected to the processor 31. The memory 32 stores computer-executable instructions, and the processor 31 executes the computer-executable instructions stored in the memory 32 to implement the parameter calibration method provided in any of the preceding embodiments, or the three-dimensional reconstruction method provided in any of the preceding embodiments.

[0150] This application also provides a computer-readable storage medium storing computer-executable instructions that, when executed, cause a computer to perform a parameter calibration method as provided in any of the preceding embodiments, or a three-dimensional reconstruction method as provided in any of the preceding embodiments.

[0151] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the parameter calibration method as provided in any of the preceding embodiments, or the three-dimensional reconstruction method as provided in any of the preceding embodiments.

[0152] It should be noted that the aforementioned computer-readable storage media can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic random access memory (FRAM), flash memory, magnetic surface memory, optical disc, or compact disc read-only memory (CD-ROM), etc. It can also be various electronic devices that include one or any combination of the above-mentioned memories, such as mobile phones, computers, tablet devices, personal digital assistants, etc.

[0153] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0154] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0155] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0156] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0157] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0158] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0159] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A parameter calibration method, characterized in that, Applied to a 3D camera, the 3D camera comprising a laser, a galvanometer, and at least one optical camera, the method includes: Obtain the initial calibration parameters of the 3D camera; The initial calibration parameters are iteratively calibrated until the stopping condition of the iterative calibration is met, at which point the target calibration parameters of the 3D camera are obtained, and the calibration of the 3D camera is completed. The iterative calibration includes jointly updating the intrinsic parameters of the optical camera, the distortion coefficient of the optical camera, the linear parameters of the laser, and the nonlinear parameters of the laser in each calibration. For each calibration in the iterative calibration, the joint update of the intrinsic parameters of the optical camera, the distortion coefficients, the linear parameters, and the nonlinear parameters includes: Based on the three-dimensional coordinates of each feature point on the calibration board in multiple calibration board coordinate systems, multiple second rotation matrices and multiple second translation matrices between the calibration board and the optical camera are calibrated to obtain updated multiple second rotation matrices and updated multiple second translation matrices. Based on the updated second rotation matrices and the updated second translation matrices, the pixel coordinates of each feature point in each calibration plate image are obtained. The number of calibration plate images is multiple, which are taken by the optical camera when the calibration plate and the optical camera are in different relative positions. The intrinsic parameters of the optical camera and the distortion coefficients are re-determined based on the pixel coordinates of each feature point in each calibration plate image; Based on the pixel coordinates of each feature point in each calibration plate image, and the updated multiple second rotation matrices and updated multiple second translation matrices, the linear parameters and the nonlinear parameters are redefined; the calibration parameters include the redefined intrinsic parameters of the optical camera, the distortion coefficients, the linear parameters, and the nonlinear parameters.

2. The method according to claim 1, characterized in that, The initial calibration parameters include at least: the intrinsic parameters of the optical camera, the distortion coefficient of the optical camera, the linear parameters of the laser, and the nonlinear parameters of the laser; wherein, the linear parameters of the laser include the intrinsic parameters of the laser, as well as the first rotation matrix and the first translation matrix between the laser and the optical camera; The step of iteratively calibrating the initial calibration parameters until the stopping condition for iterative calibration is met yields the target calibration parameters for the 3D camera, including: For the first calibration in the iterative calibration, the intrinsic parameters of the optical camera, the distortion coefficients, the linear parameters, and the nonlinear parameters in the initial calibration parameters are jointly updated to obtain the calibration parameters; For each calibration after the first calibration, the intrinsic parameters of the optical camera, the distortion coefficient, the linear parameter, and the nonlinear parameter in the calibration parameters obtained from the previous calibration are jointly updated; when the stopping condition of iterative calibration is reached, the calibration parameters of the last calibration are obtained as the target calibration parameters.

3. The method according to claim 1, characterized in that, The pixel coordinates of each feature point in each calibration board image, obtained based on the updated multiple second rotation matrices and the updated multiple second translation matrices, include: Based on the three-dimensional coordinates of each feature point in the calibration board coordinate system, as well as the updated second rotation matrix and the updated second translation matrix, the pixel coordinates of each feature point in each calibration board image are obtained.

4. The method according to claim 1, characterized in that, After redetermining the linear parameters and the nonlinear parameters based on the pixel coordinates of each feature point in each calibration board image, and the updated plurality of second rotation matrices and updated plurality of second translation matrices, the method further includes: Based on the three-dimensional coordinates of each feature point in the optical camera coordinate system, the updated multiple second rotation matrices and the updated multiple second translation matrices, as well as the redefined linear parameters and nonlinear parameters, are then jointly updated. For each calibration in the iterative calibration, the calibration parameters include the redefined intrinsic parameters of the optical camera and the distortion coefficients, as well as the jointly updated linear parameters and nonlinear parameters.

5. The method according to claim 2, characterized in that, Before the first calibration in the iterative calibration, the method further includes: The intrinsic parameters of the laser, the nonlinear parameters, the first rotation matrix, and the first translation matrix in the initial calibration parameters are updated based on the three-dimensional coordinates of each feature point on the calibration board in the camera coordinate system when the calibration board and the optical camera are in different relative positions. Furthermore, the intrinsic parameters of the optical camera and the distortion coefficients are updated based on the pixel coordinates of each feature point in each calibration board image to obtain the parameters to be updated. The calibration board images are multiple, obtained by the optical camera when the calibration board and the optical camera are in different relative positions. Based on the three-dimensional coordinates of each feature point in the camera coordinate system when the calibration plate and the optical camera are in different relative positions, the laser intrinsic parameters to be updated, the nonlinear parameters to be updated, the rotation matrix to be updated, and the translation matrix to be updated in the parameters to be updated are jointly updated. Then, combined with the optical camera intrinsic parameters to be updated and the distortion coefficients to be updated in the parameters to be updated, the initial update parameters are obtained. The initial calibration parameters jointly updated during the first calibration are the initial update parameters.

6. The method according to claim 5, characterized in that, The step of updating the intrinsic parameters of the laser, the nonlinear parameters, the first rotation matrix, and the first translation matrix in the initial calibration parameters based on the three-dimensional coordinates of each feature point on the calibration board in the camera coordinate system when the calibration board and the optical camera are in different relative positions includes: Based on the three-dimensional coordinates of each feature point in the camera coordinate system, the intrinsic parameters of the laser are updated to obtain the laser intrinsic parameters to be updated. The nonlinear parameters are updated based on the three-dimensional coordinates of each feature point in the camera coordinate system and the intrinsic parameters of the laser to be updated, to obtain the nonlinear parameters to be updated. The first rotation matrix and the first translation matrix are updated based on the three-dimensional coordinates of each feature point in the camera coordinate system, the intrinsic parameters of the laser to be updated, and the nonlinear parameters to be updated, to obtain the rotation matrix to be updated and the translation matrix to be updated.

7. The method according to claim 2, characterized in that, The acquisition of the initial calibration parameters of the 3D camera includes: Multiple calibration plate images are acquired when the calibration plate and the optical camera are in different relative positions, and each calibration plate has multiple feature points. Based on the multiple calibration board images, obtain the pixel coordinates of each feature point in each calibration board image; Obtain the rotation and translation matrices between the calibration board coordinate system and the camera coordinate system when the calibration board and the optical camera are in different relative positions; The optical camera is calibrated based on the pixel coordinates of each feature point in each calibration plate image, the rotation matrix and translation matrix between the calibration plate coordinate system and the camera coordinate system, to obtain the intrinsic parameters of the optical camera and the distortion coefficients. The initial value estimation yields the linear parameters and the nonlinear parameters.

8. The method according to claim 1, characterized in that, The stopping condition for the iterative calibration includes any one of the following: The difference between any parameter obtained in the Nth calibration and any parameter obtained in the (N-1)th calibration is within a preset range; The number of iterations for calibration is equal to the preset number; The iteration calibration duration is greater than or equal to the preset duration.

9. The method according to claim 1, characterized in that, The initial calibration parameters are obtained based on a camera-like model calibration. The camera-like model includes a camera-like coordinate system. The Y-axis of the camera-like coordinate system is the direction of the rotation axis of the galvanometer, and the Z-axis is the depth-of-field direction of the incident light exiting the galvanometer at a preset angle.

10. A three-dimensional reconstruction method, characterized in that, Applied to a 3D camera, the 3D camera comprising a laser, a galvanometer, and at least one optical camera, the method includes: Obtain the target calibration parameters of the three-dimensional camera calibrated by the parameter calibration method according to any one of claims 1-9; 3D reconstruction is performed based on the target calibration parameters of the 3D camera.

11. A parameter calibration device, characterized in that, Applied to a 3D camera, the 3D camera comprising a laser, a galvanometer, and at least one optical camera, the apparatus is used to implement the parameter calibration method according to any one of claims 1-9, the apparatus comprising: The acquisition module is used to acquire the initial calibration parameters of the 3D camera; The calibration module is used to iteratively calibrate the initial calibration parameters. When the stopping condition of the iterative calibration is reached, the target calibration parameters of the 3D camera are obtained, and the calibration of the 3D camera is completed.

12. A three-dimensional reconstruction device, characterized in that, Applied to a 3D camera, the 3D camera including a laser, a galvanometer, and at least one optical camera, the device includes: The acquisition module is used to acquire the target calibration parameters of the three-dimensional camera obtained by the parameter calibration method according to any one of claims 1-9; The processing module is used to perform 3D reconstruction based on the target calibration parameters of the 3D camera.

13. An electronic device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the parameter calibration method as described in any one of claims 1 to 9, or the three-dimensional reconstruction method as described in claim 10.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed, cause the computer to perform the parameter calibration method as described in any one of claims 1 to 9, or to perform the three-dimensional reconstruction method as described in claim 10.

Images