Three-dimensional imaging automatic calibration system and method

Abstract

The invention discloses a three-dimensional imaging automatic calibration system and method, and the system comprises an infrared light source which is used for transmitting a light beam; the laser galvanometer can move in the preset direction, and the laser galvanometer integrates a laser source and a galvanometer and is used for emitting laser in the specific direction; the laser galvanometer is located between the infrared light source and the calibration plate, and the calibration plate is used for receiving a light beam emitted by the infrared light source or receiving laser emitted by the laser galvanometer; the camera is located between the infrared light source and the calibration plate, moves in the preset direction and is used for collecting images on the calibration plate. The three-dimensional imaging automatic calibration system and method provided by the invention aim to solve the calibration problem of the laser galvanometer and the camera at different relative positions and angles, and ensure that the camera and the laser galvanometer can be automatically calibrated stably, quickly and repeatedly.

Description

Technical Field

[0001] The present invention relates to the technical field of camera calibration, and in particular to a three-dimensional imaging automatic calibration system and method. Background Art

[0002] In the field of three-dimensional calibration technology, calibration methods usually rely on accurately arranging calibration plates in different positions in space. However, this method has poor robustness, and because the position of the calibration plate needs to be adjusted multiple times to obtain data, it takes a lot of time, resulting in low overall efficiency. In addition, the imaging method of the prior art has strict requirements on the placement of the camera during the calibration process. Different camera combination placement methods will lead to different degrees of measurement accuracy errors, poor consistency, and difficulty in repeated implementation, which is not conducive to efficient and accurate three-dimensional reconstruction.

[0003] Therefore, the prior art still needs to be improved and enhanced. Summary of the invention

[0004] The technical problem to be solved by the present invention is to provide a three-dimensional imaging automatic calibration system and method in view of the above-mentioned defects of the prior art, aiming to solve the problems of high cost, complicated operation and difficulty in achieving accurate calibration caused by the traditional calibration method when the relative positions and angles of the galvanometer and the camera are different.

[0005] In order to solve the above technical problems, the technical solution adopted by the present invention is as follows:

[0006] In a first aspect, the present invention provides a three-dimensional imaging automatic calibration system, wherein the system comprises:

[0007] an infrared light source for emitting a light beam;

[0008] A laser galvanometer, which can move along a preset direction and integrates a laser source and a galvanometer, and is used to emit laser light in a specific direction;

[0009] A calibration plate, wherein the laser galvanometer is located between the infrared light source and the calibration plate, and the calibration plate is used to receive the light beam emitted by the infrared light source or the laser emitted by the laser galvanometer;

[0010] A camera is located between the infrared light source and the calibration plate and moves along a preset direction to collect an image on the calibration plate.

[0011] In one implementation, the system further includes:

[0012] A motion platform, the motion platform is located between the infrared light source and the calibration plate, and the laser galvanometer and the camera are located on the motion platform, and the motion platform is used to control the movement of the laser galvanometer and the camera.

[0013] In one implementation, the system further includes:

[0014] A manipulator is connected to the calibration plate and is used to move the calibration plate.

[0015] In a second aspect, an embodiment of the present invention further provides a three-dimensional imaging automatic calibration method based on the three-dimensional imaging automatic calibration system, wherein the method comprises:

[0016] Acquire the distance between the camera and the calibration plate, and the relative position parameters between the laser galvanometer and the camera, and move the camera and the laser galvanometer to corresponding positions based on the distance and the relative position parameters;

[0017] When the camera and the laser galvanometer are located at corresponding positions, a preset first motion trajectory is obtained, the calibration plate is moved based on the first motion trajectory, a first trajectory image of the calibration plate is collected by the camera during the movement, and the internal parameters of the camera are calibrated based on the first trajectory image;

[0018] Acquire a preset second motion trajectory, move the calibration plate to a preset position based on the second motion trajectory, acquire posture information of the calibration plate relative to the camera at the preset position and a straight line image formed on the calibration plate by the laser lines generated by the laser galvanometer, and calibrate the camera and the laser galvanometer based on the posture information and the straight line image.

[0019] In one implementation, the calibrating the camera internal parameters based on the first trajectory image then includes:

[0020] Preprocessing the first trajectory image, identifying feature points on the first trajectory image, and obtaining image coordinates of the feature points;

[0021] Obtaining the world coordinates of the feature point, and converting the world coordinates into pixel coordinates in an image coordinate system;

[0022] Calculating a root mean square error based on the image coordinates and the pixel coordinates;

[0023] If the RMS error value is less than a preset threshold, it is determined that the calibration of the camera internal parameters is correct.

[0024] In one implementation, the acquiring the posture information of the calibration plate at the preset position relative to the camera and the straight line image formed on the calibration plate by the laser lines generated by the laser galvanometer includes:

[0025] Moving the calibration plate to a first position of the second motion trajectory, and calculating and recording a first posture of the calibration plate relative to the camera at the first position;

[0026] Controlling the laser galvanometer to emit a laser line at the first position and form a first straight line on the calibration plate;

[0027] Acquiring the first straight line based on the camera to obtain a first straight line image;

[0028] Moving the calibration plate to a second position of the second motion trajectory, and calculating and recording a second posture of the calibration plate relative to the camera at the second position;

[0029] Controlling the laser galvanometer to emit a laser line at the second position and form a second straight line on the calibration plate;

[0030] The second straight line is captured based on the camera to obtain a second straight line image.

[0031] In one implementation, the calibrating the camera and the laser galvanometer based on the posture information and the straight line image includes:

[0032] Acquire pixel centers of laser lines in the first straight line image and the second straight line image;

[0033] Based on the pixel centers, fitting the straight lines in the first straight line image and the second straight line image using a fitting algorithm to obtain a first fitting straight line and a second fitting straight line;

[0034] The second fitting straight line is converted into the coordinate system of the first posture to obtain a second reference straight line, or the first fitting straight line is converted into the coordinate system of the second posture to obtain a first reference straight line;

[0035] The camera and the laser galvanometer are calibrated based on the first fitting straight line and the second reference straight line, or based on the second fitting straight line and the first reference straight line.

[0036] In one implementation, the calibrating the camera and the laser galvanometer based on the first fitting straight line and the second reference straight line, or based on the second fitting straight line and the first reference straight line, includes:

[0037] Based on the first fitting straight line and the second reference straight line, or based on the second fitting straight line and the first reference straight line, obtaining an equation and a normal vector of the laser surface in a camera coordinate system;

[0038] Based on the equation and the normal vector, a rotation matrix and a translation vector of the laser surface relative to the camera coordinate system are obtained;

[0039] The camera and the laser galvanometer are calibrated based on the rotation matrix and the translation vector.

[0040] In one implementation, the calibrating the camera and the laser galvanometer further includes:

[0041] Based on the calibration of the camera and the laser galvanometer, a calibration relationship between the camera and the laser galvanometer is obtained, and the calibration plate is moved to a target position;

[0042] At the target position, controlling the laser galvanometer to emit laser light, and projecting the laser light onto the calibration plate to obtain a laser pattern;

[0043] Using the camera to capture the laser pattern, and extracting key feature points of the laser pattern through an image processing algorithm;

[0044] Based on the calibration relationship, the spatial coordinates of the key feature points are calculated to generate a corresponding point cloud image;

[0045] Based on the point cloud image, the calibration results of the camera and the laser galvanometer are verified.

[0046] In a third aspect, an embodiment of the present invention further provides a terminal, wherein the terminal includes a memory, a processor, and a three-dimensional imaging automatic calibration program stored in the memory and executable on the processor, and when the processor executes the three-dimensional imaging automatic calibration program, the steps of the three-dimensional imaging automatic calibration method of any one of the above-mentioned schemes are implemented.

[0047] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, wherein a three-dimensional imaging automatic calibration program is stored on the computer-readable storage medium, and when the three-dimensional imaging automatic calibration program is executed by a processor, the steps of the three-dimensional imaging automatic calibration method described in any one of the above schemes are implemented.

[0048] Beneficial effects: The present invention provides a three-dimensional imaging automatic calibration system. Compared with the prior art, the system includes: an infrared light source for emitting a light beam; a laser galvanometer, which can move along a preset direction, and which integrates a laser source and a galvanometer, and is used to emit laser light in a specific direction; a calibration plate, wherein the laser galvanometer is located between the infrared light source and the calibration plate, and the calibration plate is used to receive the light beam emitted by the infrared light source or the laser emitted by the laser galvanometer; and a camera, wherein the camera is located between the infrared light source and the calibration plate, and moves along a preset direction, and is used to collect images on the calibration plate. The three-dimensional imaging automatic calibration system provided by the present invention can stably, quickly and repeatably realize the automatic calibration of the camera and the laser galvanometer, and overcomes the problems of poor robustness and low efficiency caused by the reliance on the complex spatial arrangement of the calibration plate in the prior art; through the application of automation technology, the time required for manual adjustment is greatly reduced, while the requirements for the camera placement are relaxed, and the consistency and reliability of the calibration results are improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 This is a structural diagram of a three-dimensional imaging automatic calibration system provided in an embodiment of the present invention.

[0050] Figure 2 The present invention provides an overall flow chart of a three-dimensional imaging automatic calibration method based on a three-dimensional imaging automatic calibration system according to an embodiment of the present invention.

[0051] Figure 3 The present invention provides a flowchart of a specific implementation of a three-dimensional imaging automatic calibration method based on a three-dimensional imaging automatic calibration system.

[0052] Figure 4 A schematic diagram of the calibration of a laser galvanometer and a camera in a three-dimensional imaging automatic calibration method based on a three-dimensional imaging automatic calibration system provided in an embodiment of the present invention.

[0053] Figure 5 This is a block diagram of the internal structure principle of a terminal provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0054] In order to make the purpose, technical solution and effect of the present invention clearer and more specific, the present invention is further described in detail with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0055] In the field of three-dimensional calibration technology, calibration methods usually rely on accurately arranging calibration plates in different positions in space. However, this method has poor robustness, and because the position of the calibration plate needs to be adjusted multiple times to obtain data, it takes a lot of time, resulting in low overall efficiency. In addition, the imaging method of the prior art has strict requirements on the placement of the camera during the calibration process. Different camera combination placement methods will lead to different degrees of measurement accuracy errors, poor consistency, and difficulty in repeated implementation, which is not conducive to efficient and accurate three-dimensional reconstruction.

[0056] In order to solve the above problems, this embodiment provides a three-dimensional imaging automatic calibration system. When implemented specifically, the system includes: an infrared light source for emitting a light beam; a laser galvanometer, which can move along a preset direction, and the laser galvanometer integrates a laser source and a galvanometer, and is used to emit laser light in a specific direction; a calibration plate, the laser galvanometer is located between the infrared light source and the calibration plate, and the calibration plate is used to receive the light beam emitted by the infrared light source or the laser emitted by the laser galvanometer; a camera, the camera is located between the infrared light source and the calibration plate, and moves along a preset direction, and is used to collect images on the calibration plate. The three-dimensional imaging automatic calibration system provided by the present invention can stably, quickly and repeatedly realize the automatic calibration of the camera and the laser galvanometer, and overcomes the problems of poor robustness and low efficiency caused by relying on the complex spatial arrangement of the calibration plate in the existing method; through the application of automation technology, the time required for manual adjustment is greatly reduced, while the requirements for the camera placement are relaxed, and the consistency and reliability of the calibration results are improved.

[0057] This embodiment provides a three-dimensional imaging automatic calibration system. Figure 1 As shown, the system includes the following components: an infrared light source 110, which can emit infrared light with a wavelength of 850nm; a laser galvanometer, which can move along a preset direction, and the laser galvanometer integrates a laser source and a galvanometer, and is used to emit laser in a specific direction, and the preset direction includes front, back, left, and right. The galvanometer is fixed on a rotatable stable device, and the laser galvanometer includes a first laser galvanometer 150 placed horizontally and a second laser galvanometer 151 placed vertically, and the model parameters of the first laser galvanometer and the second laser galvanometer are consistent; a calibration plate 170, the laser galvanometer is located between the infrared light source and the calibration plate, and the calibration plate is used to receive the light beam emitted by the infrared light source or the laser emitted by the laser galvanometer; a camera 160, the camera is located between the infrared light source and the calibration plate, and moves along a preset direction, and is used to collect images on the calibration plate, and the preset direction includes front, back, left, and right. The lens of the camera has a filter for filtering out interference from external natural light.

[0058] In one implementation, the system also includes a motion platform, which is located between the infrared light source and the calibration plate, and the laser galvanometer and the camera are located on the motion platform, and the motion platform is used to control the movement of the laser galvanometer and the camera. The motion platform consists of a first high-precision motion axis 120, a second high-precision motion axis 121, a third high-precision motion axis 130, a first high-precision servo turntable 140, and a second high-precision servo turntable 141.

[0059] In one implementation, the system further includes a manipulator 180 , which is connected to the calibration plate and is used to move the calibration plate.

[0060] The three-dimensional imaging automatic calibration method based on the three-dimensional imaging automatic calibration system provided in this embodiment can be applied to a smart terminal. The overall flow chart is as follows: Figure 2 As shown in the flowchart of the specific implementation method, Figure 3 As shown, the specific steps include:

[0061] Step S100, obtaining the distance between the camera and the calibration plate, and the relative position parameters between the laser galvanometer and the camera, and moving the camera and the laser galvanometer to corresponding positions based on the distance and the relative position parameters.

[0062] In this embodiment, the system is first initialized, the distance between the camera and the calibration plate is set, and the camera's focus point is ensured to be on the calibration plate surface and the image is clear; one or more sets of relative position parameters between the laser galvanometer and the camera are set, and the relative position parameters include the relative distance and angle between the laser galvanometer and the camera; the distance and the relative position parameters can be automatically or manually input; a first motion trajectory is pre-set, and the first motion trajectory is used to assist in the calibration of the camera's internal parameters, and a second motion trajectory is pre-set, and the second motion trajectory is used to assist in the calibration of the camera and the laser galvanometer. Before calibrating the camera's internal parameters or calibrating the camera and the laser galvanometer, the distance between the camera and the calibration plate and the relative position parameters between the galvanometer and the laser camera are obtained. Based on the distance and the relative position parameters, the camera and the laser galvanometer are moved to the corresponding positions to ensure that the camera's focus point is on the calibration plate surface and the image is clear, providing an accurate spatial positioning reference for the subsequent calibration process.

[0063] Step S200: When the camera and the laser galvanometer are located at corresponding positions, a preset first motion trajectory is obtained, the calibration plate is moved based on the first motion trajectory, a first trajectory image of the calibration plate is captured by the camera during the movement, and the internal parameters of the camera are calibrated based on the first trajectory image.

[0064] In this embodiment, first, an infrared light source is turned on, and the infrared light source is used to emit a light beam to the calibration plate. Based on system initialization, the camera and the laser galvanometer are moved to corresponding positions. When the camera and the laser galvanometer are located at corresponding positions, a preset first motion trajectory is obtained. The first motion trajectory is used to assist in the calibration of the internal parameters of the camera. The first motion trajectory covers a sufficient range to ensure that the internal parameters of the camera can be fully detected. For example, a circular trajectory is set to allow the calibration plate to move along the circumference within a certain range to cover changes in different angles and positions. Then, the calibration plate is moved based on the first motion trajectory. The calibration plate has known geometric features, such as a standard checkerboard calibration plate with evenly distributed black and white squares. In the process of controlling the movement of the calibration plate, ensure that all positions and postures of the calibration plate are captured by the camera to obtain a variety of viewing angles. For example, the robot can move the calibration plate at a uniform speed according to the first motion trajectory and collect images at different positions and angles. In the process of the calibration plate moving along the first motion trajectory, a series of images containing different positions and postures of the calibration plate are collected by the camera, namely, the first trajectory images. The first trajectory images should cover the situation of the calibration plate at different positions and angles to ensure the diversity and effectiveness of the data. For example, an image is collected at a certain distance or angle to ensure that there are multiple images at different angles and positions for calibration. Finally, the collected first trajectory images are processed to extract feature points on the calibration plate, such as chessboard corner points, and these feature points are used to calibrate the internal parameters of the camera. The Zhang calibration method or other calibration algorithms are used to estimate the internal parameters of the camera, such as focal length, distortion coefficient, and principal point position. For example, the relevant functions in the OpenCV library can be used to complete the calibration process. This process not only eliminates the influence of lens distortion on image quality, but also establishes a mapping relationship between the image pixel coordinate system and the real world coordinate system, thereby providing the necessary basic data for the subsequent relative position calibration between the laser galvanometer and the camera, ensuring the measurement accuracy and reliability of the entire system.

[0065] In an implementation manner, the calibrating the camera internal parameters based on the first trajectory image further includes verifying the calibrated camera internal parameters.

[0066] In this embodiment, first, the first trajectory image is preprocessed, and the preprocessing method includes but is not limited to grayscale, histogram equalization and median filtering. The preprocessing helps to eliminate interference factors in the image, improve image quality, and make subsequent feature detection more accurate. Next, the feature points on the first trajectory image are identified, and the feature points include but are not limited to corners, edges or other significant visual features. These feature points are unique in the image, easy to track and easy to match under different viewing angles. Then, the image coordinates of the feature points are obtained. Subsequently, the world coordinates of the feature points are obtained, and the world coordinates are converted into pixel coordinates in the image coordinate system; finally, based on the image coordinates and the pixel coordinates, the root mean square error is calculated; if the root mean square error value is less than a preset threshold, it is determined that the calibration of the camera internal parameters is correct; if the root mean square error value is greater than the preset threshold, it is determined that the calibration of the camera internal parameters is abnormal, and the calibration plate 170 is moved on the first trajectory by increasing or decreasing a fixed value using the manipulator 180, and the calibration process of the camera internal parameters is performed again until the calibration of the camera internal parameters is correct.

[0067] In addition, by comparing the straightness of one or more edges in the original image before calibration and the image after the camera internal parameter calibration, it can be judged whether the calibration effect of the camera internal parameter reaches the expected result. Specifically, firstly, the original image before calibration is obtained. At this time, the distortion that may exist in the image will cause some straight lines to appear as curves; then, the original image is geometrically calibrated using the camera internal parameters obtained by calibration to obtain the calibrated image. In the calibrated image, the straight line that was originally bent due to distortion should be calibrated as a straight line. Therefore, by comparing the straightness of one or more edges in the original image before calibration and the calibrated image, it can be judged whether the calibration effect reaches the expected result; if the straight line in the calibrated image is obviously straighter than the straight line before calibration, it indicates that the calibration effect is good; otherwise, it is considered that the calibration effect is not good. At this time, the calibration plate 170 is moved on the first track by increasing or decreasing the fixed value using the manipulator 180, and the camera internal parameter calibration process is performed again until the calibration of the camera internal parameter reaches the expected result. This embodiment verifies the calibration of the internal parameters of the camera to ensure the accuracy and effectiveness of the calibration of the internal parameters of the camera, provides necessary basic data for the subsequent calibration of the camera and the laser galvanometer, and ensures the measurement accuracy and reliability of the entire system.

[0068] Step S300, obtaining a preset second motion trajectory, moving the calibration plate to a preset position based on the second motion trajectory, obtaining the posture information of the calibration plate relative to the camera at the preset position and a straight line image formed on the calibration plate by the laser lines generated by the laser galvanometer, and calibrating the camera and the laser galvanometer based on the posture information and the straight line image.

[0069] In this embodiment, after completing the calibration of the camera's internal parameters and confirming their accuracy, the camera and the laser galvanometer need to be calibrated next. First, based on system initialization, a preset second motion trajectory is obtained, and based on the second motion trajectory, the manipulator is controlled to move the calibration plate to a preset position; then, the posture information of the calibration plate relative to the camera at the preset position and the straight line image formed by the laser lines generated by the laser galvanometer on the calibration plate are obtained; finally, the camera and the laser galvanometer are calibrated based on the posture information and the straight line image to ensure the calibration accuracy and stability of the system.

[0070] During specific implementation, this embodiment performs calibration between the camera and the first laser galvanometer and the second laser galvanometer. The first laser galvanometer is placed horizontally, and the second laser galvanometer is placed vertically. There is no restriction on the calibration order between the camera and the first laser galvanometer and the second laser galvanometer. This embodiment can calibrate the behavioral characteristics of the system more comprehensively on a two-dimensional plane through the calibration of the camera and the first laser galvanometer and the second laser galvanometer, thereby significantly improving the positioning accuracy and robustness of the system. This comprehensive calibration method can not only correct errors in all directions more meticulously, but also simplify subsequent adjustments, ensuring the stability and reliability of the system in complex environments. However, it should be noted that the calibration of the camera and the laser galvanometer proposed in the present invention is not limited to the first laser galvanometer and the second laser galvanometer.

[0071] The specific implementation method of the camera and laser galvanometer calibration method proposed in this embodiment is as follows. It should be noted that the calibration methods of the camera and the first laser galvanometer or the second laser galvanometer are as follows, which will not be repeated in this embodiment. However, this embodiment calibrates both the camera and the first laser galvanometer and the camera and the second laser galvanometer. After completing the calibration of the camera and the first laser galvanometer and the camera and the second laser galvanometer, the calibration of the laser galvanometer and the camera is completed.

[0072] First, based on system initialization, a preset second motion trajectory is obtained, and the manipulator is controlled to move the calibration plate to the first position of the second motion trajectory. The camera intrinsic parameter matrix and distortion coefficient obtained after the camera internal parameter calibration are used to detect and extract the corner point position of the calibration plate in the image, and the first posture of the calibration plate relative to the camera at the first position is calculated by solving the PnP (Perspective-n-Point) problem, and the first posture is recorded. The first posture includes a first rotation matrix and a first translation vector. Next, the laser galvanometer is controlled to emit a laser line at the first position and form a first straight line on the calibration plate. Then, the first straight line is captured based on the camera to obtain a first straight line image. Subsequently, the manipulator is controlled to move the calibration plate to the second position of the second motion trajectory, and the camera intrinsic parameter matrix and distortion coefficient are used again to detect and extract the corner point position of the calibration plate in the image, and the second posture of the calibration plate relative to the camera at the second position is calculated by solving the PnP problem, and the second posture is recorded. The second posture includes a second rotation matrix and a second translation vector. Next, the laser galvanometer is controlled to emit a laser line at the second position and form a second straight line on the calibration plate; then, the second straight line is captured based on the camera to obtain a second straight line image. This embodiment uses posture information and straight line images to accurately calculate and adjust the geometric relationship between the camera and the laser galvanometer, thereby completing the calibration between the two and ensuring the consistency and accuracy of the image captured by the camera and the reflection path of the laser galvanometer in subsequent operations.

[0073] After obtaining the posture information and the straight line image, first, the edge of the laser line in the first straight line image and the second straight line image is identified by image processing technology, and the maximum continuous set of edge pixels is found, and this set is the pixel center of the laser line. The image processing technology includes but is not limited to edge detection algorithm. Then, based on the pixel center, the straight lines in the first straight line image and the second straight line image are fitted by a fitting algorithm to obtain a first fitting straight line and a second fitting straight line. The fitting algorithm includes but is not limited to the least squares method. Next, the second fitting straight line is converted to the coordinate system of the first posture to obtain a second reference straight line, or the first fitting straight line is converted to the coordinate system of the second posture to obtain a first reference straight line, for example: first, the image coordinates of multiple points are extracted from the second fitting straight line; then, the image coordinates of these points are inversely transformed by using the camera intrinsic parameter matrix to convert them from two-dimensional image coordinates to actual coordinates in a three-dimensional world coordinate system; then, the first rotation matrix and the first translation vector are used to convert these three-dimensional coordinate points from the world coordinate system to the local coordinate system of the first posture; finally, a new straight line equation is fitted based on the corresponding coordinate points in the local coordinate system of the first posture, that is, the second reference straight line, so as to complete the conversion of the second fitting straight line in the coordinate system of the first posture. Subsequently, based on the first fitting straight line and the second reference straight line, or based on the second fitting straight line and the first reference straight line, the equation and normal vector of the laser surface in the camera coordinate system are obtained, for example: first, the direction vectors of the first fitting straight line and the second reference straight line and any point on the first fitting straight line or the second reference straight line are transformed from the first posture coordinate system to the camera coordinate system using the first rotation matrix and the first translation vector; then, in the camera coordinate system, the direction vectors of the first fitting straight line and the second reference straight line are taken to calculate the cross product to obtain the normal vector of the laser surface; finally, based on the normal vector and any point transformed to the camera coordinate system, the equation of the laser surface is constructed. Then, based on the equation and the normal vector, the rotation matrix R and translation vector T of the laser surface relative to the camera coordinate system are obtained, that is, the [R|T] relationship of the laser surface relative to the camera coordinate system is established, for example: first, the normal vector of the laser surface is normalized to a unit vector; then, a standard basis vector is selected and cross-producted with the unit normal vector to obtain a first vertical vector, and normalized; then, the unit normal vector is cross-producted with the first vertical vector to obtain a second vertical vector; the rotation matrix R of the laser surface relative to the camera coordinate system is constructed based on the unit normal vector, the first vertical vector and the second vertical vector; finally, a point on the laser surface is selected as a reference point, and the position of the point in the camera coordinate system is the translation vector T of the laser surface relative to the camera coordinate system. Finally, based on the rotation matrix R and translation vector T of the laser surface relative to the camera coordinate system, the camera and the laser galvanometer are calibrated.For example, a homogeneous transformation matrix can be constructed based on the rotation matrix R and the translation vector T to accurately describe the spatial relationship between the laser surface and the camera coordinate system. The homogeneous transformation matrix is ​​used to adjust the parameter settings of the laser galvanometer to ensure that the laser lines emitted by the laser galvanometer are accurately aligned with the positions in the image captured by the camera. To ensure that the calibration is accurate and reliable, this calibration process can be repeated at different positions to verify the consistency of the projection of the laser lines in the camera image with the theoretical calculation results. Finally, the obtained calibration relationship is saved for subsequent data fusion and processing. The present invention can reduce cumulative errors, simplify the calculation process, and improve calibration accuracy by converting the fitting straight line into the attitude coordinate system and then converting it to the camera coordinate system to calibrate the camera and the laser galvanometer. Through step-by-step conversion and alignment, each step can be independently verified to ensure the consistency and accuracy of the data, thereby making the final calibration result more reliable. The schematic diagram of the laser galvanometer and camera calibration is as follows. Figure 4 As shown, 200 is a laser galvanometer, 210 is a laser surface formed by the laser galvanometer emitting laser lines, 220 is a camera, 230 is an image surface, 240 is a calibration plate 1, 241 is a calibration plate 2, 250 is a first straight line on the calibration plate 1, 251 is a second straight line on the calibration plate 2, 260 is an image of the first straight line in the image coordinate system, and 261 is an image of the second straight line in the image coordinate system, wherein X w -Y w -Z w is the world coordinate system, X C -Y C -Z C is the camera coordinate system, X L -Y L -Z L is the coordinate system of the first posture or the coordinate system of the second posture.

[0074] In one implementation, the calibration of the camera and the laser galvanometer further includes verifying the calibration results of the camera and the laser galvanometer.

[0075] In this embodiment, first, based on the calibration of the camera and the laser galvanometer, the calibration relationship between the camera and the laser galvanometer is obtained, and the manipulator is controlled to move the calibration plate to the target position, where the target position refers to a pre-set position, ensuring that the calibration plate is in a known and controllable position, thereby improving the accuracy and consistency of the calibration; then, at the target position, the laser galvanometer is controlled to emit a laser, and the laser is projected onto the calibration plate according to a set interval to obtain a laser pattern. This step ensures that the laser forms a structured and uniformly distributed pattern on the calibration plate to facilitate subsequent feature point extraction and calibration verification; then, the camera is used to capture the laser pattern, and the key feature points of the laser pattern are extracted through an image processing algorithm to effectively remove noise and improve the recognition accuracy of the feature points; then, the camera intrinsic matrix is ​​used to obtain the laser pattern. The key feature points are transformed from the image coordinate system to the camera coordinate system by a rotation matrix, and then the key feature point coordinates in the camera coordinate system are transformed into the world coordinate system by the rotation matrix and translation vector of the camera coordinate system relative to the world coordinate system, so as to obtain the spatial coordinates of the key feature points, and generate the corresponding point cloud image based on the spatial coordinates of the key feature points; finally, the calibration results of the camera and the laser galvanometer are verified by comparing the difference between the point cloud image and the theoretical geometric shape of the calibration plate. For example, if the calibration plate is a standard plane, then the point cloud image should also present a smooth plane. If there is any significant deviation, it indicates that there is an error in the calibration process, and each step in the calibration process needs to be rechecked, and the calibration process of the camera and the laser galvanometer needs to be repeated until the accuracy requirements are met. In this way, the accuracy and reliability of the calibration results can be ensured.

[0076] In summary, this embodiment provides a three-dimensional imaging automatic calibration system. When implemented specifically, the system includes: an infrared light source for emitting a light beam; a laser galvanometer, which can move along a preset direction, and the laser galvanometer integrates a laser source and a galvanometer, and is used to emit laser light in a specific direction; a calibration plate, the laser galvanometer is located between the infrared light source and the calibration plate, and the calibration plate is used to receive the light beam emitted by the infrared light source or the laser emitted by the laser galvanometer; a camera, the camera is located between the infrared light source and the calibration plate, and moves along a preset direction, and is used to collect images on the calibration plate. The three-dimensional imaging automatic calibration system provided by the present invention can stably, quickly and repeatably realize the automatic calibration of the camera and the laser galvanometer, and overcomes the problems of poor robustness and low efficiency caused by relying on the complex spatial arrangement of the calibration plate in the existing method; through the application of automation technology, the time required for manual adjustment is greatly reduced, while the requirements for the camera placement are relaxed, and the consistency and reliability of the calibration results are improved.

[0077] Based on the above embodiment, the present invention further provides a terminal, the principle block diagram of the terminal can be as follows: Figure 5 The terminal may include one or more processors 100 ( Figure 5 Only one is shown in the figure), a memory 101 and a computer program 102 stored in the memory 101 and executable on one or more processors 100, for example, a 3D imaging automatic calibration program. When one or more processors 100 execute the computer program 102, the various steps in the embodiment of the 3D imaging automatic calibration method can be implemented. Alternatively, when one or more processors 100 execute the computer program 102, the functions of the various modules / units in the embodiment of the 3D imaging automatic calibration method can be implemented, which is not limited here.

[0078] In one embodiment, the processor 100 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0079] In one embodiment, the memory 101 may be an internal storage unit of an electronic device, such as a hard disk or memory of the electronic device. The memory 101 may also be an external storage device of the electronic device, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device. Further, the memory 101 may also include both an internal storage unit of the electronic device and an external storage device. The memory 101 is used to store computer programs and other programs and data required by the terminal. The memory 101 may also be used to temporarily store data that has been output or is to be output.

[0080] Those skilled in the art will understand that Figure 5 The principle block diagram shown in the figure is only a block diagram of a partial structure related to the scheme of the present invention, and does not constitute a limitation on the terminal to which the scheme of the present invention is applied. The specific terminal may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0081] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, operating database or other media used in the embodiments provided by the present invention may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double operational data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0082] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A three-dimensional imaging automatic calibration system, characterized in that: The system comprises: an infrared light source for emitting a light beam; A laser galvanometer, which can move along a preset direction and integrates a laser source and a galvanometer, and is used to emit laser light in a specific direction; A calibration plate, wherein the laser galvanometer is located between the infrared light source and the calibration plate, and the calibration plate is used to receive the light beam emitted by the infrared light source or the laser emitted by the laser galvanometer; A camera is located between the infrared light source and the calibration plate and moves along a preset direction to collect an image on the calibration plate.

2. The three-dimensional imaging automatic calibration system according to claim 1, characterized in that: The system further comprises: A motion platform, the motion platform is located between the infrared light source and the calibration plate, and the laser galvanometer and the camera are located on the motion platform, and the motion platform is used to control the movement of the laser galvanometer and the camera.

3. The three-dimensional imaging automatic calibration system according to claim 2, characterized in that: The system further comprises: A manipulator is connected to the calibration plate and is used to move the calibration plate.

4. A three-dimensional imaging automatic calibration method based on the three-dimensional imaging automatic calibration system according to any one of claims 1 to 3, characterized in that: The method comprises: Acquire the distance between the camera and the calibration plate, and the relative position parameters between the laser galvanometer and the camera, and move the camera and the laser galvanometer to corresponding positions based on the distance and the relative position parameters; When the camera and the laser galvanometer are located at corresponding positions, a preset first motion trajectory is obtained, the calibration plate is moved based on the first motion trajectory, a first trajectory image of the calibration plate is collected by the camera during the movement, and the internal parameters of the camera are calibrated based on the first trajectory image; Acquire a preset second motion trajectory, move the calibration plate to a preset position based on the second motion trajectory, acquire posture information of the calibration plate relative to the camera at the preset position and a straight line image formed on the calibration plate by the laser lines generated by the laser galvanometer, and calibrate the camera and the laser galvanometer based on the posture information and the straight line image.

5. The three-dimensional imaging automatic calibration method according to claim 4, characterized in that: The step of calibrating the internal parameters of the camera based on the first trajectory image further comprises: Preprocessing the first trajectory image, identifying feature points on the first trajectory image, and obtaining image coordinates of the feature points; Obtaining the world coordinates of the feature point, and converting the world coordinates into pixel coordinates in an image coordinate system; Calculating a root mean square error based on the image coordinates and the pixel coordinates; If the RMS error value is less than a preset threshold, it is determined that the calibration of the camera internal parameters is correct.

6. The galvanometer-based three-dimensional imaging automatic calibration method according to claim 4, characterized in that: The step of acquiring the posture information of the calibration plate at the preset position relative to the camera and the straight line image formed on the calibration plate by the laser lines generated by the laser galvanometer includes: Moving the calibration plate to a first position of the second motion trajectory, and calculating and recording a first posture of the calibration plate relative to the camera at the first position; Controlling the laser galvanometer to emit a laser line at the first position, the laser is reflected by the galvanometer to form a laser line, and forms a first straight line on the calibration plate; Acquiring the first straight line based on the camera to obtain a first straight line image; Moving the calibration plate to a second position of the second motion trajectory, and calculating and recording a second posture of the calibration plate relative to the camera at the second position; Controlling the laser galvanometer to emit a laser line at the second position, the laser is reflected by the galvanometer to form a laser line, and forms a second straight line on the calibration plate; The second straight line is captured based on the camera to obtain a second straight line image.

7. The galvanometer-based three-dimensional imaging automatic calibration method according to claim 6, characterized in that: The calibrating the camera and the laser galvanometer based on the posture information and the straight line image includes: Acquire pixel centers of laser lines in the first straight line image and the second straight line image; Based on the pixel centers, fitting the straight lines in the first straight line image and the second straight line image using a fitting algorithm to obtain a first fitting straight line and a second fitting straight line; The second fitting straight line is converted into the coordinate system of the first posture to obtain a second reference straight line, or the first fitting straight line is converted into the coordinate system of the second posture to obtain a first reference straight line; The camera and the laser galvanometer are calibrated based on the first fitting straight line and the second reference straight line, or based on the second fitting straight line and the first reference straight line.

8. The galvanometer-based three-dimensional imaging automatic calibration method according to claim 7, characterized in that: The calibrating the camera and the laser galvanometer based on the first fitting straight line and the second reference straight line, or based on the second fitting straight line and the first reference straight line, includes: Based on the first fitting straight line and the second reference straight line, or based on the second fitting straight line and the first reference straight line, obtaining an equation and a normal vector of the laser surface in a camera coordinate system; Based on the equation and the normal vector, a rotation matrix and a translation vector of the laser surface relative to the camera coordinate system are obtained; The camera and the laser galvanometer are calibrated based on the rotation matrix and the translation vector.

9. The galvanometer-based three-dimensional imaging automatic calibration method according to claim 8, characterized in that: The calibration of the camera and the laser galvanometer further includes: Based on the calibration of the camera and the laser galvanometer, a calibration relationship between the camera and the laser galvanometer is obtained, and the calibration plate is moved to a target position; At the target position, controlling the laser galvanometer to emit laser light, and projecting the laser light onto the calibration plate through the galvanometer to obtain a laser pattern; Using the camera to capture the laser pattern, and extracting key feature points of the laser pattern through an image processing algorithm; Based on the calibration relationship, the spatial coordinates of the key feature points are calculated to generate a corresponding point cloud image; Based on the point cloud image, the calibration results of the camera and the laser galvanometer are verified.

10. A terminal, characterized in that: The terminal includes a memory, a processor, and a three-dimensional imaging automatic calibration program stored in the memory and executable on the processor. When the processor executes the three-dimensional imaging automatic calibration program, the steps of the three-dimensional imaging automatic calibration method as described in any one of claims 4 to 9 are implemented.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a three-dimensional imaging automatic calibration program, and when the three-dimensional imaging automatic calibration program is executed by the processor, the steps of the three-dimensional imaging automatic calibration method according to any one of claims 4 to 9 are implemented.

Images