Calibration method based on three-dimensional machine vision camera

Abstract

The invention provides a camera calibration method based on three-dimensional machine vision, which relates to the technical field of computer vision, and realizes accurate camera and hand-eye calibration by configuring a robot with a depth sensor and a camera and a fixed calibration plate with a calibration block and a calibration ball. Firstly, a calibration plate is fixed, a robot is controlled to shoot at different spatial positions so as to collect images and depth data, and a robot attitude matrix and a transformation matrix of the robot attitude matrix at each position are calculated; determining an extrinsic parameter matrix of the camera by using the obtained image data, and calculating a transformation matrix of the extrinsic parameter matrix; three-dimensional point cloud data obtained from the depth sensor is processed, internal parameters of the camera are estimated, and external parameters are calculated by solving the PnP problem. And finally, optimizing internal and external parameters by using a nonlinear optimization method so as to minimize a re-projection error. Calibration accuracy is verified through experiments, and obtained parameters are applied to subsequent image processing and computer vision tasks so as to improve overall system performance.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a method based on three-dimensional machine vision camera calibration. Background Art

[0002] Accurate camera calibration is crucial in fields such as industrial automation, robot navigation, and 3D reconstruction. Traditional camera calibration methods often rely on 2D image processing technology, which limits the accuracy and adaptability of calibration. With the development of 3D imaging technology, using 3D data for camera calibration can greatly improve the accuracy and stability of calibration.

[0003] Among them, camera calibration is a basic and critical step in the field of computer vision. Its purpose is to recover the geometric information of the real world from image data. Whether in application fields such as industrial automation, robot navigation, virtual reality, and 3D reconstruction, the accuracy of camera calibration directly affects the overall performance of the system. Through calibration, the camera's intrinsic parameters (such as focal length, principal point, distortion coefficient, etc.) and extrinsic parameters (the position and posture of the camera relative to a reference object) can be determined. However, traditional camera calibration has the following limitations: 1. 2D image dependency: Traditional calibration methods usually rely on 2D images to estimate camera parameters by identifying specific patterns in the image (such as a checkerboard). This method may not be accurate enough when dealing with depth information and complex scenes.

[0004] 2. Poor environmental adaptability: In a changing operating environment, such as changes in illumination and viewing angle, the robustness of traditional 2D image calibration methods is poor.

[0005] 3. Operational complexity: The relative position between the camera and the calibration plate needs to be adjusted multiple times to collect sufficient data, which is cumbersome and time-consuming.

[0006] With the advancement of technology, 3D machine vision technology has gradually emerged, providing a new direction for camera calibration. 3D machine vision technology uses depth cameras, laser scanning and other methods to obtain 3D point cloud data in space, which can capture the spatial information of the scene more comprehensively. Therefore, how to calibrate the camera based on 3D machine vision technology so as to integrate depth information, automate data collection and processing, significantly improve efficiency and reduce manual operation errors has become a technical problem that needs to be solved urgently. Summary of the invention

[0007] In view of this, in order to improve the accuracy and reliability of the machine vision system, the present invention proposes a method based on three-dimensional machine vision camera calibration, which aims to solve the limitations of traditional two-dimensional image calibration methods in accuracy, efficiency and environmental adaptability. By integrating robots and three-dimensional imaging technology, the calibration accuracy is improved, and the calibration process of three-dimensional cameras and robot systems is simplified. It can be widely used in industrial inspection, robot navigation, three-dimensional reconstruction and other fields.

[0008] The present invention is implemented by the following technical solutions: In a first aspect, the present invention provides a method for calibrating a three-dimensional machine vision camera, the method comprising the following steps: A robot with a depth sensor and a camera installed on its manipulator is configured, and a fixed calibration board is configured, on which a calibration block and a calibration ball are provided at a fixed spatial position for camera calibration and hand-eye calibration; Keeping the calibration plate fixed, the robot is controlled to shoot the calibration plate at at least three different spatial positions to collect image and depth data; The end position posture provided by the robot controller is transformed to obtain the robot posture matrix of each shooting position, and the robot transformation matrix from the initial position to other positions is calculated; Use camera calibration to determine the camera's extrinsic matrix from image data and calculate the camera's extrinsic transformation matrix from its initial position to other positions; The depth sensor is used to obtain 3D point cloud data corresponding to each shooting position, and the camera intrinsic parameter matrix is ​​estimated using camera calibration. The camera extrinsic parameters are calculated by solving PnP. The camera extrinsic parameters include rotation matrix and translation vector. Nonlinear optimization of internal and external parameters is used to minimize the reprojection error of feature points in all images. The accuracy of calibration is verified by calculating the reprojection error and the image quality after distortion correction. The fitted internal and external parameters are applied to image processing and computer vision tasks.

[0009] As a further solution of the present invention, the depth sensor is a height sensor arranged at the end of the manipulator, and the camera is a high-resolution camera fixedly installed at the end of the manipulator of the robot.

[0010] As a further solution of the present invention, the calibration plate carries an easily recognizable pattern, which is a checkerboard, for accurately capturing image data and spatial position information.

[0011] As a further solution of the present invention, the robot is controlled to photograph the calibration plate at at least three different spatial positions, including an initial position and at least two positions at different angles, and the photographed spatial positions are named: initial position, position 1, and position 2, respectively.

[0012] As a further solution of the present invention, when the robot is controlled to photograph the calibration plate at different spatial positions, the three-dimensional point cloud data corresponding to each photographing position is acquired through the depth sensor.

[0013] As a further solution of the present invention, when obtaining the robot posture matrix of each shooting position, the robot posture matrix H0, H1, H2 of each position is calculated using the end position posture data provided by the robot controller, and the transformation matrix A1, A2 corresponding to the robot from the initial position to the positions of two different angles is calculated based on the posture matrix; wherein:

[0014] .

[0015] As a further solution of the present invention, when the extrinsic parameter matrix is ​​obtained, the extrinsic parameter matrices M0, M1, and M2 corresponding to the initial position, position 1, and position 2 are determined by camera calibration from the image data obtained at different spatial positions; when the extrinsic parameter transformation matrix is ​​calculated, the transformation matrices B1 and B2 of the camera from the initial position to position 1 and position 2 are calculated based on the extrinsic parameter matrix; wherein:

[0016] .

[0017] As a further solution of the present invention, when the depth sensor acquires three-dimensional point cloud data corresponding to each shooting position and calibrates it, it includes: (1) Convert the rotation matrix into a rotation vector:

[0018] (2) Normalization of rotation vector:

[0019] (3) Calculate the modified Rodriguez vector:

[0020] (4) Calculate the initial rotation vector:

[0021] (5) Calculate the rotation vector:

[0022] (6) Calculate the rotation matrix:

[0023] (7) Calculate the translation matrix:

[0024] In the formula, and They are two rotation matrices respectively; and The rotation matrices are and The rotation vector obtained by the conversion; is the Rodriguez rotation formula, which is used to convert the rotation matrix into a rotation vector; and They are the rotation vectors and The corresponding rotation angle; and They are the rotation vectors and The module length of and They are the rotation vectors and Normalized unit vector; and are the corrected Rodriguez vectors, respectively; and The rotation angles are and half the sine value; : Function that converts a vector into an antisymmetric matrix; is the initial rotation vector; : is the sum of two modified Rodriguez vectors; is the final rotation vector; is the initial rotation vector; The rotation vector The calculated rotation matrix.

[0025] As a further solution of the present invention, camera calibration is used to estimate the camera's intrinsic parameter matrix. When the camera's extrinsic parameters are calculated by solving PnP, Zhang's correction method is used to estimate the camera's intrinsic parameters. PnP is used to estimate the camera's extrinsic parameters. Levenberg-Marquardt is used to fine-tune the intrinsic and extrinsic parameters to minimize the reprojection error.

[0026] Through the above steps, the method of the present invention can achieve high-precision camera calibration, improve the overall performance and reliability of the visual system, and is suitable for high-demand industrial applications and research fields.

[0027] The present invention also includes a computer device, comprising: at least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor executes the method based on three-dimensional machine vision camera calibration.

[0028] The present invention also includes a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the method based on three-dimensional machine vision camera calibration.

[0029] Compared with the prior art, the method based on three-dimensional machine vision camera calibration provided by the present invention has the following beneficial effects: 1. Improved calibration accuracy. By combining the data of the depth sensor and the high-resolution camera, the method of the present invention can more accurately acquire and process information in three-dimensional space, and the comprehensive use of image and depth data can significantly improve the accuracy of camera calibration, especially when processing three-dimensional objects and complex scenes.

[0030] 2. Enhanced system robustness. The method of the present invention not only performs well in an ideal laboratory environment, but also maintains high performance in a changing external environment (such as different lighting conditions and dynamic scenes). This is due to the use of a depth sensor, which reduces the sensitivity to changes in the external environment, thereby enhancing the robustness of the system.

[0031] 3. Simplified operation and automated calibration. By using a robot to automate the calibration process, the method of the present invention greatly simplifies the operation of data collection and processing, reducing labor costs and error rates. The robot can accurately control the camera to capture data from multiple angles and positions, ensuring the comprehensiveness and consistency of the data. Moreover, the automated process reduces the time and labor required for traditional manual calibration, achieving efficient data collection and processing, which is particularly important for industrial applications that require a large number of equipment calibrations in a short period of time, such as quality inspection equipment on a production line.

[0032] 4. It has good scalability and adaptability. The method of the present invention is applicable to various types of cameras and various application scenarios, including but not limited to industrial automation, robot navigation, virtual reality, augmented reality, etc. The versatility and adaptability of this method allow it to be easily integrated into different technologies and products. It can also optimize the performance of subsequent computer vision tasks. Accurate camera calibration is the basis for computer vision tasks such as high-quality image processing, three-dimensional reconstruction, and object recognition. The camera parameters optimized by the method of the present invention can improve the accuracy and efficiency of these tasks, thereby enhancing the functionality of the entire visual system.

[0033] In summary, the camera calibration method based on three-dimensional machine vision of the present invention not only improves the accuracy and efficiency of calibration, but also simplifies the operation and optimizes the cost by integrating robots and depth sensing.

[0034] These and other aspects of the present invention will be more clearly understood in the following description of the embodiments. It should be understood that the above general description and the following detailed description are only exemplary and explanatory and cannot limit the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or related technologies, the following briefly introduces the drawings required for use in the exemplary embodiments or related technical descriptions. The drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation to the present invention. In the drawings: Figure 1 This is a device structure diagram of a method for calibrating a three-dimensional machine vision camera according to an embodiment of the present invention.

[0036] Figure 2 The present invention is a flowchart of a method for calibrating a three-dimensional machine vision camera according to an embodiment of the present invention.

[0037] In the figure: 1-robot, 2-manipulator, 3-camera, 4-depth sensor, 5-calibration plate, 6-calibration ball, 7-calibration block. DETAILED DESCRIPTION

[0038] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0039] In some of the processes described in the specification and claims of the present invention and the above-mentioned figures, multiple operations that appear in a specific order are included, but it should be clearly understood that these operations may not be executed in the order in which they appear in this article or executed in parallel. The serial numbers of the operations, such as 101, 102, etc., are only used to distinguish different operations, and the serial numbers themselves do not represent any execution order. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel. It should be noted that the descriptions of "first", "second", etc. in this article are used to distinguish different messages, devices, modules, etc., do not represent the order of precedence, and do not limit the "first" and "second" to be different types.

[0040] The following will be combined with the drawings in the exemplary embodiments of the present invention to clearly and completely describe the technical solutions in the exemplary embodiments of the present invention. Obviously, the exemplary embodiments described are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0041] In order to improve the accuracy and reliability of machine vision systems, the present invention proposes a method based on three-dimensional machine vision camera calibration, aiming to solve the limitations of traditional two-dimensional image calibration methods in accuracy, efficiency and environmental adaptability. By integrating robots and three-dimensional imaging technology, the calibration accuracy is improved, and the calibration process of three-dimensional cameras and robot systems is simplified. It can be widely used in many fields such as industrial inspection, robot navigation and three-dimensional reconstruction.

[0042] The technical solution of the present invention is further described below in conjunction with specific embodiments: See also Figure 1 and Figure 2 As shown, a method based on three-dimensional machine vision camera calibration is provided in one embodiment of the present invention, and the method comprises the following steps: Step S10, configure a robot 1 with a depth sensor 4 and a camera 3 installed on a manipulator 2, and configure a fixed calibration board 5, on which a calibration block 7 and a calibration ball 6 are provided at a fixed spatial position for camera calibration and hand-eye calibration.

[0043] In this step, the depth sensor 4 is a height sensor disposed at the end of the manipulator 2, and the camera 3 is a high-resolution camera fixedly mounted at the end of the manipulator 2 of the robot 1. The calibration plate 5 has an easily recognizable pattern, which is a checkerboard, for accurately capturing image data and spatial position information.

[0044] Therefore, before executing the method based on three-dimensional machine vision camera calibration, first prepare a robot 1, a camera 3 mounted on a manipulator 2, a depth sensor 4, a calibration block 7 with a fixed spatial position for camera calibration and hand-eye calibration, and a calibration ball 6.

[0045] Step S20, keeping the calibration plate 5 fixed, and controlling the robot 1 to photograph the calibration plate 5 at at least three different spatial positions to collect image and depth data.

[0046] In this step, the robot 1 is controlled to photograph the calibration plate 5 at at least three different spatial positions, including an initial position and at least two positions at different angles. The photographed spatial positions are named as: initial position, position 1, and position 2.

[0047] When the robot 1 is controlled to shoot the calibration plate 5 at different spatial positions, the three-dimensional point cloud data corresponding to each shooting position is acquired through the depth sensor 4.

[0048] Step S30, the end position posture provided by the robot controller is transformed to obtain the robot posture matrix of each shooting position, and the robot transformation matrix from the initial position to other positions is calculated.

[0049] In this step, when obtaining the robot posture matrix of each shooting position, the robot posture matrix H0, H1, H2 of each position is calculated using the end position posture data provided by the robot controller, and the transformation matrix A1, A2 corresponding to the robot 1 from the initial position to the positions of two different angles is calculated based on the posture matrix; wherein:

[0050] .

[0051] In this step, the end position posture provided by the robot controller can be transformed to obtain the three posture matrices H0, H1, H2 of the robot 1, thereby obtaining the transformation matrices A1 and A2 corresponding to the two movements of the tool, where A1: initial position to position 1, A2: initial position to position 2.

[0052] Step S40: Determine the camera's extrinsic parameter matrix from the image data using camera calibration, and calculate the extrinsic parameter transformation matrix of camera 3 from the initial position to other positions.

[0053] In this step, when the extrinsic parameter matrix is ​​obtained, the extrinsic parameter matrices M0, M1, and M2 corresponding to the initial position, position 1, and position 2 are determined by camera calibration from the image data obtained at different spatial positions; when the extrinsic parameter transformation matrix is ​​calculated, the transformation matrices B1 and B2 of the camera 3 from the initial position to position 1 and position 2 are calculated based on the extrinsic parameter matrix; wherein:

[0054] .

[0055] Among them, the initial position, position 1, and position 2 obtained by camera calibration correspond to the external parameter matrices M0, M1, and M2 respectively; thereby, the transformation matrices B1 and B2 corresponding to the two movements of camera 3 can be obtained, B1: from the initial position to position 1, B2: from the initial position to position 2.

[0056] Step S50, calibrate the three-dimensional point cloud data corresponding to each shooting position obtained by the depth sensor 4, use the camera calibration to estimate the intrinsic parameter matrix of the camera 3, and calculate the external parameters of the camera 3 by solving PnP. The external parameters of the camera 3 include the rotation matrix and the translation vector.

[0057] In this step, when the depth sensor 4 obtains the three-dimensional point cloud data corresponding to each shooting position and calibrates it, it includes: (1) Convert the rotation matrix into a rotation vector:

[0058] (2) Normalization of rotation vector:

[0059] (3) Calculate the modified Rodriguez vector:

[0060] (4) Calculate the initial rotation vector:

[0061] (5) Calculate the rotation vector:

[0062] (6) Calculate the rotation matrix:

[0063] (7) Calculate the translation matrix:

[0064] In the formula, in the formula, and They are two rotation matrices respectively; and The rotation matrices are and The rotation vector obtained by the conversion; is the Rodriguez rotation formula, which is used to convert the rotation matrix into a rotation vector; and They are the rotation vectors and The corresponding rotation angle; and They are the rotation vectors and The module length of and They are the rotation vectors and Normalized unit vector; and are the corrected Rodriguez vectors, respectively; and The rotation angles are and half the sine value; : Function that converts a vector into an antisymmetric matrix; is the initial rotation vector; : is the sum of two modified Rodriguez vectors; is the final rotation vector; is the initial rotation vector; The rotation vector The calculated rotation matrix; is the identity matrix, used for multiplication operations.

[0065] The above parameter definitions are as follows: 1. Rotation Matrix (R): In three-dimensional space, a rotation matrix is ​​a 3x3 matrix that describes the rotation of an object around a point. It can be expressed as a rotation around the x-axis, y-axis, or z-axis, or obtained from a rotation vector using the Rodriguez formula.

[0066] 2. Rotation vector (r): A rotation vector is a three-dimensional vector that represents the direction of the rotation axis and the magnitude of the rotation. Its magnitude represents the angle of rotation, and its direction represents the rotation axis.

[0067] 3. Normalize: Normalization refers to the process of scaling the length of a vector to 1. In rotation vector normalization, the unit rotation vector is obtained by dividing by the length of the vector.

[0068] 4. Modified Rodriguez vector (pb): This is the rotation vector calculated by the Rodriguez formula and is used to describe the transformation of an object from one position to another.

[0069] 5. Initial rotation vector (pa): This is the rotation vector of the object in its initial position.

[0070] 6. Rotation matrices (Ra, Rb): These are rotation matrices used to calculate rotation vectors, which may represent different rotation operations or rotations in different coordinate systems.

[0071] 7. Translation Matrix (P): The translation matrix is ​​a 4x4 matrix used to describe the translation transformation of an object in space. It usually contains a 3x3 identity matrix and a translation vector.

[0072] 8. Translation vectors (px, py, pz): These are the translation components in the translation matrix and represent the displacement of the object in the x, y, and z axes.

[0073] 9. Normalized rotation vectors (ea, eb): These are the normalized rotation vectors, that is, the unit rotation vectors.

[0074] 10. Coefficients (1, 2, 4) in the Rodriguez transform formula: These coefficients are the parameters in the Rodriguez formula used to convert a rotation vector into a rotation matrix.

[0075] 11. I in matrix multiplication: Here I stands for the identity matrix, which is used for multiplication operations.

[0076] 12. Vector Cross Product (PxPI): This is the cross product of vector p and vector q.

[0077] 13. Vector dot product (V4): This is the dot product result of vector v, which is usually used to calculate scalars.

[0078] 14. Skew function: This is a function that converts a vector into a skew-symmetric matrix, which is often used to calculate the rotation matrix.

[0079] 15. Sin and cos functions: These are trigonometric functions used to calculate certain components of rotation and translation.

[0080] Step S60: Use nonlinear optimization of internal and external parameters to minimize the reprojection error of feature points in all images, verify the accuracy of calibration by calculating the reprojection error and the image quality after distortion correction, and apply the fitted internal and external parameters to image processing and computer vision tasks.

[0081] In this step, the camera calibration is used to estimate the intrinsic parameter matrix of camera 3. When the extrinsic parameters of camera 3 are calculated by solving PnP, Zhang's correction method is used to estimate the intrinsic parameters of camera 3, and PnP is used to estimate the extrinsic parameters of camera 3. Levenberg-Marquardt is used to fine-tune the intrinsic and extrinsic parameters to minimize the reprojection error.

[0082] Specifically, when estimating intrinsic parameters, a camera calibration algorithm (using Zhang's correction method) is used to estimate the intrinsic parameter matrix of camera 3. This usually involves mathematical methods such as least squares method and iterative solution to fit the intrinsic parameters by optimizing the objective function. When estimating extrinsic parameters, for each set of matched feature points, the extrinsic parameters of camera 3 (rotation matrix and translation vector) are calculated. This can be achieved by solving the PnP (Perspective-n-Point) problem. Commonly used algorithms include EPnP, P3P, etc. When optimizing, nonlinear optimization methods (such as Levenberg-Marquardt algorithm) are used to fine-tune the intrinsic and extrinsic parameters to minimize the reprojection error of feature points in all images. When verifying and evaluating, the accuracy of the calibration results is evaluated by calculating indicators such as reprojection error and image quality after distortion correction. When the results are applied, the fitted intrinsic and extrinsic parameters are applied to subsequent image processing and computer vision tasks, such as 3D reconstruction, object tracking, image stitching, etc.

[0083] It should be understood that, although described in a certain order, these steps are not necessarily performed in sequence in the above order. Unless there is clear explanation in this article, the execution of these steps does not have strict order restriction, and these steps can be performed in other orders. Moreover, a part of the steps of the present embodiment may include a plurality of steps or a plurality of stages, and these steps or stages are not necessarily performed at the same time, but can be performed at different times, and the execution order of these steps or stages is not necessarily performed in sequence, but can be performed in turn or alternately with at least a part of the steps or stages in other steps or other steps.

[0084] The method of calibrating a three-dimensional machine vision camera of the present invention can more accurately acquire and process information in three-dimensional space by combining the data of the depth sensor 4 and the high-resolution camera. The method of the present invention can make comprehensive use of the image and depth data, so that the accuracy of camera calibration is significantly improved, especially when processing three-dimensional objects and complex scenes. The advantages are more obvious. The method of the present invention not only performs well in an ideal laboratory environment, but also maintains high performance in a changing external environment (such as different lighting conditions and dynamic scenes). This is due to the use of the depth sensor 4, which reduces the sensitivity to changes in the external environment, thereby enhancing the robustness of the system. By using a robot to automate the calibration process, the method of the present invention greatly simplifies the operation of data acquisition and processing, and reduces labor costs and error rates. The robot 1 can accurately control the camera 3 to capture data from multiple angles and positions, ensuring the comprehensiveness and consistency of the data. Moreover, the automated process reduces the time and labor required for traditional manual calibration, and realizes efficient data acquisition and processing, which is particularly important for industrial applications that need to complete the calibration of a large number of equipment in a short time, such as quality inspection equipment on a production line. The method of the present invention is applicable to various types of cameras and various application scenarios, including but not limited to industrial automation, robot navigation, virtual reality, augmented reality, etc. The versatility and adaptability of this method make it easy to be integrated into different technologies and products. It can also optimize the performance of subsequent computer vision tasks. Accurate camera calibration is the basis for computer vision tasks such as high-quality image processing, 3D reconstruction and object recognition. The camera parameters optimized by the method of the present invention can improve the accuracy and efficiency of these tasks, thereby enhancing the functionality of the entire visual system.

[0085] In summary, the camera calibration method based on three-dimensional machine vision of the present invention not only improves the accuracy and efficiency of calibration, but also simplifies the operation and optimizes the cost by integrating robots and depth sensing.

[0086] In one embodiment, a computer device is also provided in an embodiment of the present invention, comprising at least one processor and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor performs the steps of the method based on three-dimensional machine vision camera calibration.

[0087] In one embodiment, the present invention further provides a computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable the computer to execute the steps of the method based on three-dimensional machine vision camera calibration.

[0088] A person skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be realized by instructing the relevant hardware through a computer program represented by computer instructions, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory.

[0089] Non-volatile memory may include read-only memory, magnetic tape, floppy disk, flash memory or optical storage, etc. Volatile memory may include random access memory or external cache memory. As an illustration and not limitation, RAM may be in various forms, such as static random access memory or dynamic random access memory, etc.

[0090] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for calibrating a three-dimensional machine vision camera, characterized in that: The method comprises the following steps: A robot with a depth sensor and a camera installed on its manipulator is configured, and a fixed calibration board is configured, on which a calibration block and a calibration ball are provided at a fixed spatial position for camera calibration and hand-eye calibration; Keeping the calibration plate fixed, the robot is controlled to shoot the calibration plate at at least three different spatial positions to collect image and depth data; The end position posture provided by the robot controller is transformed to obtain the robot posture matrix of each shooting position, and the robot transformation matrix from the initial position to other positions is calculated; Use camera calibration to determine the camera's extrinsic matrix from image data and calculate the camera's extrinsic transformation matrix from its initial position to other positions; The depth sensor is used to obtain 3D point cloud data corresponding to each shooting position, and the camera intrinsic parameter matrix is ​​estimated using camera calibration. The camera extrinsic parameters are calculated by solving PnP. The camera extrinsic parameters include rotation matrix and translation vector. Nonlinear optimization of internal and external parameters is used to minimize the reprojection error of feature points in all images. The accuracy of calibration is verified by calculating the reprojection error and the image quality after distortion correction. The fitted internal and external parameters are applied to image processing and computer vision tasks.

2. The method for calibrating a three-dimensional machine vision camera according to claim 1, wherein: The depth sensor is a height sensor arranged at the end of the manipulator, and the camera is a high-resolution camera fixedly installed at the end of the manipulator of the robot.

3. The method for calibrating a three-dimensional machine vision camera according to claim 2, wherein: The calibration plate has an easily recognizable pattern, which is a checkerboard pattern, and is used to accurately capture image data and spatial position information.

4. The method for calibrating a three-dimensional machine vision camera according to claim 1, wherein: When the robot is controlled to shoot the calibration plate at at least three different spatial positions, including an initial position and at least two positions at different angles, the captured spatial positions are named as: initial position, position 1, and position 2.

5. The method for calibrating a three-dimensional machine vision camera according to claim 4, wherein: When the robot is controlled to shoot the calibration plate at different spatial positions, the three-dimensional point cloud data corresponding to each shooting position is obtained through the depth sensor.

6. The method for calibrating a three-dimensional machine vision camera according to claim 5, wherein: When obtaining the robot posture matrix of each shooting position, the robot posture matrix H0, H1, H2 of each position is calculated using the end position posture data provided by the robot controller, and the transformation matrix A1, A2 corresponding to the robot from the initial position to the position of two different angles is calculated based on the posture matrix; where: 。 7. The method for calibrating a three-dimensional machine vision camera according to claim 6, wherein: When the external parameter matrix is ​​obtained, the external parameter matrices M0, M1, and M2 corresponding to the initial position, position 1, and position 2 are determined by using camera calibration from the image data obtained at different spatial positions. When calculating the extrinsic parameter transformation matrix, the transformation matrices B1 and B2 of the camera from the initial position to position 1 and position 2 are calculated based on the extrinsic parameter matrix; where: 。 8. The method for calibrating a three-dimensional machine vision camera according to claim 7, wherein: When the depth sensor is used to obtain the three-dimensional point cloud data corresponding to each shooting position, the calibration includes: 1) Convert the rotation matrix into a rotation vector: 2) Normalization of rotation vector: 3) Calculate the modified Rodriguez vector: 4) Calculate the initial rotation vector: 5) Calculate the rotation vector: 6) Calculate the rotation matrix: 7) Calculate the translation matrix: In the formula, and They are two rotation matrices respectively; and The rotation matrices are and The rotation vector obtained by the conversion; is the Rodriguez rotation formula, which is used to convert the rotation matrix into a rotation vector; and They are the rotation vectors and The corresponding rotation angle; and They are the rotation vectors and The module length; and They are the rotation vectors and Normalized unit vector; and are the corrected Rodriguez vectors, respectively; and The rotation angles are and half the sine value; : Function that converts a vector into an antisymmetric matrix; is the initial rotation vector; : is the sum of two modified Rodriguez vectors; is the final rotation vector; is the initial rotation vector; The rotation vector The calculated rotation matrix.

9. The method for calibrating a three-dimensional machine vision camera according to claim 1, wherein: The camera calibration is used to estimate the camera's intrinsic parameter matrix. When the camera's extrinsic parameters are calculated by solving PnP, Zhang's correction method is used to estimate the camera's intrinsic parameters. PnP is used to estimate the camera's extrinsic parameters. Levenberg-Marquardt is used to fine-tune the intrinsic and extrinsic parameters to minimize the reprojection error.

Images