Joint calibration method of laser plane and rotating shaft, three-dimensional imaging method, device and equipment

Abstract

The invention discloses a joint calibration method of a laser plane and a rotating shaft, a three-dimensional imaging method, a device and equipment, and relates to the technical field of three-dimensional imaging. The laser plane and the rotating shaft are calibrated simultaneously by using the same group of checkerboard images. Therefore, the obtained calibration data is consistent in time and space, the real characteristics of the system in the actual working state can be better reflected, and the calibration result is more reliable. According to the joint calibration, the feature point information used for laser plane calibration and rotating shaft calibration can be obtained at the same time only through one-time image acquisition operation, the operation process is simple, and complex switching and coordination between the laser plane calibration and the rotating shaft calibration are not needed. Operators only need to operate according to unified calibration steps, so that the efficiency of calibration work is improved. According to the joint calibration method, accumulative errors possibly generated when the laser plane and the rotating shaft are independently calibrated are avoided, and the calibration precision is improved.

Description

Technical Field

[0001] The present application relates to the field of three-dimensional imaging technology, and in particular to a joint calibration method of a laser plane and a rotation axis, a three-dimensional imaging method, a device and equipment. Background Art

[0002] In traditional 3D laser scanning technology, line laser rotation scanning is a commonly used method, which obtains the 3D coordinate data of the object by projecting a laser line onto the surface of the target object and capturing its reflected image. In order to achieve high-precision 3D reconstruction, the laser plane and rotation axis in the system must be accurately calibrated. However, in existing technologies, the calibration process of the laser plane and rotation axis is often carried out separately, and the steps are cumbersome and inefficient, which greatly limits the application and development of 3D scanning technology.

[0003] In the early calibration methods, the calibration of laser planes usually involves the use of specific targets, which usually requires a lot of manual operations and complex image processing algorithms to extract the laser line and match it with the feature points of the target. In addition, the calibration of the rotation axis often requires additional equipment and steps, such as the use of encoders or additional sensors to track the movement of the rotation axis, which not only increases the complexity of the system, but also increases the cost.

[0004] The method of separately calibrating the laser plane and the rotation axis lacks overall consideration and makes it difficult to fully utilize the intrinsic connection between the two, which further affects the calibration accuracy and the overall performance of the system, and affects the accuracy of the final 3D reconstruction. In addition, separate calibration also means that more time and resources are required to complete the calibration of the entire system, which is a significant disadvantage for application scenarios that require rapid deployment and operation. Summary of the invention

[0005] The purpose of this application is to provide a joint calibration method of a laser plane and a rotation axis, a three-dimensional imaging method, a device and equipment to achieve the joint calibration of a laser plane and a rotation axis and improve the accuracy and efficiency of the calibration.

[0006] To achieve the above objectives, this application provides the following solutions.

[0007] In a first aspect, the present application provides a joint calibration method for a laser plane and a rotation axis, the joint calibration method being applied to a line laser rotation scanning imaging system, the line laser rotation scanning imaging system comprising: a line laser, a camera and a pan-tilt rotating structure; the line laser and the camera are both arranged on the pan-tilt rotating structure, the joint calibration method comprising:

[0008] Get checkerboard images taken at different rotation angles;

[0009] Extract the intersection points of the laser stripes and the checkerboard lines in the checkerboard image as feature points;

[0010] Calibrate the laser plane of the line laser based on the pixel coordinates of each feature point in each checkerboard image to determine the laser plane equation of the line laser;

[0011] The rotation axis of the gimbal rotation structure is calibrated based on the checkerboard images taken at different rotation angles to determine the direction vector of the rotation axis.

[0012] In a second aspect, the present application provides a combined calibration device for a laser plane and a rotation axis, wherein the combined calibration device for a laser plane and a rotation axis applies the combined calibration method for a laser plane and a rotation axis, and the combined calibration device for a laser plane and a rotation axis comprises:

[0013] A chessboard image acquisition module is used to acquire chessboard images taken at different rotation angles;

[0014] A feature point extraction module is used to extract the intersection points of the laser stripes and the checkerboard lines in the checkerboard image as feature points;

[0015] A laser plane calibration module, used to calibrate the laser plane of the line laser based on the pixel coordinates of each feature point in each checkerboard image, and determine the laser plane equation of the line laser;

[0016] The rotation axis calibration module is used to calibrate the rotation axis of the pan / tilt rotating structure based on the checkerboard images taken at different rotation angles to determine the direction vector of the rotation axis.

[0017] In a third aspect, the present application provides a three-dimensional imaging method, the three-dimensional imaging method comprising:

[0018] Obtain images of the target object taken by the camera at different rotation angles;

[0019] Based on the laser plane equation of the line laser, the coordinates of each pixel in the target object image taken at different rotation angles are converted into three-dimensional coordinates in the camera coordinate system corresponding to each rotation angle;

[0020] Based on the external parameters of the camera at each rotation angle, the three-dimensional coordinates in the camera coordinate system corresponding to each rotation angle are converted to the world coordinate system to obtain the three-dimensional point cloud data at different rotation angles;

[0021] Based on the direction vector of the rotation axis, the three-dimensional point cloud data at different rotation angles are rotated and merged to obtain the three-dimensional point cloud data of the target object;

[0022] The laser plane equation of the line laser and the direction vector of the rotation axis are obtained by using the above-mentioned joint calibration method of the laser plane and the rotation axis.

[0023] In a fourth aspect, the present application provides a three-dimensional imaging device, the three-dimensional imaging device comprising:

[0024] A target object image acquisition module is used to acquire images of the target object taken by the camera at different rotation angles;

[0025] A first coordinate conversion module is used to convert each pixel coordinate in the target object image captured at different rotation angles into a three-dimensional coordinate in a camera coordinate system corresponding to each rotation angle based on the laser plane equation of the line laser;

[0026] The second coordinate conversion module is used to convert the three-dimensional coordinates in the camera coordinate system corresponding to each rotation angle into the world coordinate system based on the external parameters of the camera at each rotation angle, so as to obtain the three-dimensional point cloud data at different rotation angles;

[0027] A third coordinate conversion module is used to rotate and merge the three-dimensional point cloud data at different rotation angles based on the direction vector of the rotation axis to obtain the three-dimensional point cloud data of the target object;

[0028] The laser plane equation of the line laser and the direction vector of the rotation axis are obtained by using the above-mentioned joint calibration method of the laser plane and the rotation axis.

[0029] In a fifth aspect, the present application provides a computer device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the above-mentioned joint calibration method of the laser plane and the rotation axis or the three-dimensional imaging method.

[0030] According to the specific embodiments provided in this application, this application has the following technical effects.

[0031] The present application provides a joint calibration method for a laser plane and a rotation axis, a three-dimensional imaging method, a device and equipment. The present application uses the same set of checkerboard images to calibrate the laser plane and the rotation axis at the same time. The calibration data obtained in this way is consistent in time and space, and can better reflect the real characteristics of the system under actual working conditions, making the calibration results more reliable. The joint calibration only requires one image acquisition operation to simultaneously obtain feature point information for laser plane calibration and rotation axis calibration. The operation process is simple and does not require complex switching and coordination between laser plane calibration and rotation axis calibration. The operator only needs to follow the unified calibration steps to improve the efficiency of the calibration work. The joint calibration method avoids the cumulative errors that may be generated when the laser plane and rotation axis are calibrated separately, and improves the calibration accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0033] Figure 1 A schematic flow chart of a method for joint calibration of a laser plane and a rotation axis provided in one embodiment of the present application.

[0034] Figure 2 A schematic diagram of a joint calibration method for a laser plane and a rotation axis provided in one embodiment of the present application.

[0035] Figure 3 A schematic structural diagram of a linear laser rotation scanning imaging system provided in one embodiment of the present application.

[0036] Figure 4 A flowchart of a camera calibration process provided in one embodiment of the present application.

[0037] Figure 5 A schematic diagram of a camera calibration process provided in an embodiment of the present application.

[0038] Figure 6 This is an example diagram of obtaining a checkerboard image by joint calibration of the laser plane and the rotation axis provided in one embodiment of the present application.

[0039] Figure 7 This is an example diagram of the laser plane calibration results provided in one embodiment of the present application.

[0040] Figure 8 A schematic diagram of a 3D scan of a vernier caliper provided in an embodiment of the present application.

[0041] Fig. 9 A schematic 3D scan of a scratched PE water pipe provided in one embodiment of the present application.

[0042] Fig.10 A schematic diagram of the structure of a computer device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0043] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0044] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0045] In an exemplary embodiment, a joint calibration method for a laser plane and a rotation axis is provided, wherein the joint calibration method is applied to a line laser rotation scanning imaging system, such as Figure 3 As shown, the line laser rotation scanning imaging system includes: a line laser, a camera and a pan-tilt rotating structure; the line laser and the camera are both arranged on the pan-tilt rotating structure, as shown in FIG. Figure 1 and Figure 2 As shown, the joint calibration method includes the following steps 101 to 104.

[0046] Step 101, obtaining chessboard images taken at different rotation angles.

[0047] Step 102, extracting the intersection points of the laser stripes and the checkerboard lines in the checkerboard image as feature points;

[0048] Step 103, calibrating the laser plane of the line laser based on the pixel coordinates of each feature point in each checkerboard image, and determining the laser plane equation of the line laser;

[0049] Step 104 , calibrating the rotation axis of the pan / tilt rotating structure based on the checkerboard images captured at different rotation angles, and determining the direction vector of the rotation axis.

[0050] Implementation of the above steps 101 to 104 can realize the joint calibration of the laser plane and the rotation axis, thereby improving the accuracy and efficiency of the calibration.

[0051] The above-mentioned camera is one of the core devices of the line laser rotation scanning imaging system, which is responsible for capturing the image of the object under laser irradiation. In the embodiment of the present application, a black and white high-definition camera is selected and equipped with a lens with a focal length of 16mm to ensure that clear image data can be provided when acquiring the laser line. Its external signal triggering function enables the camera to work synchronously with the line laser and the pan-tilt rotation structure, thereby ensuring the coherence and consistency of the data.

[0052] The above-mentioned line laser is used to generate a laser plane. It projects a thin laser line onto the surface of the object to be measured, cooperates with the camera, and uses the principle of optical triangulation to obtain the three-dimensional coordinate data of the object surface. The line laser used in the embodiment of the present application has a wavelength of 520nm, a divergence angle of 45°, and a power of 150mW. This line laser forms a line of light on the surface of the object by emitting a line laser. This light is captured by the camera and the center position of the laser bar is extracted through subsequent image processing algorithms.

[0053] The above-mentioned pan-tilt rotating structure is equipped with a precise screw-worm structure. The pan-tilt rotating structure used in the embodiment of the present application performs rotational motion with a resolution of 0.001°. The core of the pan-tilt rotating structure is a 42 stepper motor with a step angle of 1.8°, which can ensure high-precision scanning of objects at different angles. The transmission ratio of the stepper motor is 90:1, which can accurately control the rotation speed of the system, and exchange data with a PC through the RS485-ModBus bus interface, realizing the integration of stepping, driving and control of the motion system, thereby ensuring synchronization during the entire scanning process.

[0054] The PC serves as the data processing and control center of the line laser rotation scanning imaging system. Through the image processing software and control algorithm, the PC receives the images taken by the camera in real time and performs a series of operations such as image preprocessing, laser strip extraction, and point cloud generation. At the same time, the PC controls the emission of the laser and the synchronous acquisition of the camera through an external interface. In addition, the PC is also responsible for interacting with the motion system and directing the motion system to move along a predetermined path during the scanning process. Due to the large amount of data generated by the 3D laser scanning system, the computing power of the PC determines the real-time performance of the system and the accuracy of data processing.

[0055] Laser plane calibration is a key step in laser 3D reconstruction systems, and its purpose is to determine the position and orientation of the laser plane in space. This involves determining the equation of the laser plane, which describes the geometric relationship of all points on the laser plane. By accurately determining the position of the laser plane, the accuracy of the conversion from 2D images to 3D space can be ensured, thereby improving the measurement accuracy of the entire 3D reconstruction system.

[0056] In another exemplary embodiment, the above step 103 may be replaced by the following steps 201 to 206 .

[0057] Step 201 : convert the pixel coordinates of each feature point in each checkerboard image into a world coordinate system to obtain the world coordinates of each feature point.

[0058] Step 202, calculating the mean of the world coordinates of each specific point as the world coordinate of the center point.

[0059] Step 203, centralizing the world coordinates of each specific point according to the world coordinates of the center point to obtain the world coordinates of each feature point after the centralization process.

[0060] Step 204 , according to the world coordinates of each feature point after centralization, the least square method is used to solve the following formula to obtain three plane parameters of the laser plane equation.

[0061]

[0062] Among them, a, b, c are the three plane parameters of the laser plane equation, x' i ,y' i and z' i They are the x-axis component, y-axis component, and z-axis component of the world coordinates of the specific point after centralization, and N is the number of feature points.

[0063] Step 205 , based on the world coordinates of the center point and the three plane parameters of the laser plane equation, the distance from the origin to the laser plane is calculated using the following formula as the distance parameter of the laser plane equation.

[0064] d = -n·p0;

[0065] Among them, d is the distance parameter of the laser plane equation, n is the right singular vector, n = [a, b, c], and p0 is the world coordinate of the center point.

[0066] Step 206, based on the three plane parameters and the distance parameter of the laser plane equation, the laser plane equation is determined to be:

[0067] ax+by+cz+d=0;

[0068] Among them, x, y, and z are the x-axis component, y-axis component, and z-axis component of the world coordinates of any point on the laser plane, respectively.

[0069] In the above step 201, in order to obtain the world coordinates of each feature point through coordinate transformation, the camera needs to be calibrated.

[0070] The embodiment of the present application adopts the Zhang Zhengyou calibration method for calibration. The Zhang Zhengyou calibration method is a commonly used camera intrinsic parameter calibration method. Its main idea is to calculate the internal parameters of the camera through points of specific geometric shapes on the calibration plate. Compared with other calibration methods, the Zhang Zhengyou calibration method has the advantages of simple calculation, high accuracy, and wide application range. It has been widely used in computer vision and robot vision. The specific calibration process is as follows: Figure 4 shown.

[0071] During the camera calibration process, a standard checkerboard calibration board was used, with a size of 12×9 and a width of 30 mm for each grid. Images of the calibration board were taken at different viewing angles and positions, and image processing techniques were used to extract corner point information, such as Figure 5 As shown, the embodiment of the present application achieves high-precision calibration of the camera.

[0072] The embodiment of the present application calibrates the experimental camera and obtains the camera's intrinsic parameter matrix. The calibration results show that the camera's intrinsic parameter matrix K is:

[0073]

[0074] These parameters accurately describe the intrinsic geometric and optical properties of the camera, providing a precise basis for subsequent image processing and 3D reconstruction.

[0075] In another exemplary embodiment, the checkerboard image in step 101 is obtained by placing the checkerboard pattern at a fixed position and projecting light stripes thereon. The pan-tilt rotating structure is used to rotate a fixed angle each time and take pictures, and a total of several checkerboard images are taken. In each checkerboard image, the feature points formed by the interaction of the light stripes and the checkerboard can be clearly captured and extracted.

[0076] In the subsequent calibration process, clearly capturing the light stripes on the checkerboard is crucial for subsequent data processing and analysis. If the checkerboard with light stripes is photographed directly, noise will be introduced due to laser reflection, ambient light or other factors, affecting the image quality. In order to reduce these noises and obtain clear light stripes, we first shoot the checkerboard without laser stripes using the original exposure setting to retain the details of the checkerboard, then turn on the laser and lower the exposure of the camera to make the image of the background checkerboard darker, while the light stripes will stand out more in the image due to their higher brightness. In this way, the loss of details due to overexposure can be reduced, ensuring the clarity and visibility of the light stripes in the image. Finally, the light stripe image taken after lowering the exposure is added to the original checkerboard image according to a certain weight. This method can effectively suppress noise while retaining the details of the light stripes and checkerboard, such as Figure 6 As shown, Figure 6 The first column in is a checkerboard image without laser stripes taken with the original exposure settings. Figure 6 The second column in the figure shows a checkerboard image with laser stripes taken after lowering the camera exposure. Figure 6 The third column in is the checkerboard image after weighted addition.

[0077] In the checkerboard image after weighted addition, the outline and structure of the checkerboard are preserved, while the details of the light streaks become clearer. The checkerboard image with clear light streaks obtained by this image fusion method can greatly improve the accuracy and reliability of the subsequent calibration process. In laser plane calibration, clearly visible light streaks help to more accurately identify feature points, thereby improving the accuracy of the calibration algorithm. In addition, this method can also reduce image quality fluctuations caused by changes in ambient light or improper camera settings, making the calibration process more stable and repeatable.

[0078] In another exemplary embodiment, in the above step 102, the intersection points of the light stripes and the checkerboard lines are extracted from the acquired checkerboard image by a centerline extraction algorithm as feature points.

[0079] In another exemplary embodiment, the above step 201 uses the obtained camera internal and external parameters and pixel coordinates to establish a projection model, and then accurately maps the feature points from the pixel coordinate system back to the world coordinate system.

[0080] Assume the world coordinates of the feature point are (X w ,Y w ,0), the camera coordinates are (X c ,Y c ,Z c ), the pixel coordinates are (u,v), and the camera extrinsic matrix is:

[0081]

[0082] Where R is the rotation matrix and t is the translation vector.

[0083] Transform the point in the world coordinate system to the camera coordinate system through the external parameter matrix:

[0084]

[0085] The points in the camera coordinate system are then projected onto the pixel coordinates through the intrinsic matrix:

[0086]

[0087] Combining the above two matrices and expanding them gives:

[0088]

[0089] X w and Y w Separation of the terms yields:

[0090] (uR 31 -c x R 31 -f x R 11 )X w +(uR 32 -c x R 32 -f x R 12 )Y w =f x t x -ut z +c x t z

[0091] (vR 31 -c y P 31 -f y R 21 )Xw +(vR 32 -c y R 32 -f y R 22 )Y w =f y t y -vt z +c y t z

[0092] Writing the above equations in matrix form and solving for world coordinates yields:

[0093]

[0094] According to the above steps, the three-dimensional coordinate data of multiple feature points are obtained. In the laser plane fitting, due to the presence of noise and measurement errors, the actual data points will not strictly fall on a plane. The goal of step 103 of the present application is to fit a plane through these points so that the plane is as close as possible to all observation points. This can be formalized as a problem of minimizing the sum of the vertical distances from the point to the plane. At this point, SVD can be used to fit an optimal plane so that the sum of the squares of the distances from all data points to the plane is minimized.

[0095] In another exemplary embodiment, to implement the above steps 202 and 203, the three-dimensional point set is {(x i ,y i ,z i )}, we want to fit a plane equation ax+by+cz+d=0, where a, b, c, d are the parameters to be determined. By centering the data, that is, subtracting the mean of the data set from each point, the processed data is symmetrically distributed around the origin, thereby improving numerical stability, reducing fitting errors, and improving the accuracy of the results. Let the center point be The centralized data is {(x' i ,y' i ,z' i )},but:

[0096]

[0097] In another exemplary embodiment, the goal of step 204 is to find a plane passing through the origin. This problem can be transformed into solving a least squares problem, namely:

[0098]

[0099] Solve the above formula and construct the following matrix:

[0100]

[0101] Use SVD to decompose the matrix A into A = UΣV T , through SVD decomposition, the problem of minimizing the error can be transformed into T Find a right singular vector n = [a, b, c] corresponding to the minimum singular value in . This vector will correspond to the plane parameters a, b, c. Therefore, the plane equation can be expressed as ax + by + cz = 0. Since the point has been centered, it is necessary to calculate the distance d from the origin to the laser plane. In order to simplify the problem, we re-express the laser plane equation as:

[0102] n·p0+d=0

[0103] but:

[0104] d=-n·p0

[0105] When calibrating the laser plane, the embodiment of the present application uses the known geometric structure of the chessboard and the image taken each time to find the feature points, and then uses the SVD decomposition method to fit these three-dimensional points to solve the parameters of the laser plane. The application of SVD not only provides a deep understanding of the data structure, but also ensures the numerical stability and accuracy of the calculation process.

[0106] Through the above steps, the optimal solution of laser plane fitting can be obtained. Through the above steps, the optimal solution of laser plane fitting is successfully obtained. The laser plane calibration process involves multiple key links, including the calibration of camera internal and external parameters, the extraction of calibration points, and the solution of laser plane equations. Figure 7 The characteristic points shown are Figure 7 (a) is an example of the fitting result in the world coordinate system. Figure 7 (b) is a schematic diagram of the fitting result in the camera coordinate system. During the calibration process, the embodiment of the present application uses a set of known feature points and obtains the laser plane equation by fitting through the SVD decomposition method.

[0107] In an embodiment of the present application, by defining a residual function and an initial direction and using a least squares optimization method, the optimal rotation axis direction parameters are successfully determined. First, by decomposing the extrinsic parameter matrix, the corresponding rotation matrix and translation vector are extracted. These rotation matrices and translation vectors represent the extrinsic parameters of the camera at different positions and directions. By calculating the relative transformation matrix between these extrinsic parameter matrices, the rotation axis and rotation angle are further extracted to describe the relative rotation between each pair of adjacent camera positions. Using the average value of the input rotation axis set as an estimate of the initial direction, the residual function is minimized, the difference between the rotation axis and the given axis is calculated, and these differences are minimized through an iterative process to obtain an accurate rotation axis direction.

[0108] In another exemplary embodiment, the above step 104 may be replaced by the following steps 301 and 302.

[0109] Step 301, based on the checkerboard images taken at different angles, determine the axis of relative rotation between two adjacent angles as a given axis;

[0110] Step 302: Based on the direction vector of the given axis, the least square method is used to solve the following formula to obtain the direction vector of the rotation axis;

[0111]

[0112] Among them, a k is the direction vector of the kth given axis, p is the direction vector of the rotation axis, and K is the number of given axes.

[0113] In the above step 301, the position and direction of the rotation axis can be calibrated by analyzing the position and posture of the camera at different rotation angles. The proposed optical plane-rotation axis joint calibration method can be used to calibrate the optical plane and the rotation axis while taking a set of pictures.

[0114] The present embodiment considers the problem of rotating axis fitting, and inputs a set of direction vectors a of a given axis. k , k = 1, 2, ..., K. Let the direction vector of the fitted rotation axis be p, and its normalized form is:

[0115]

[0116] In this embodiment of the present application, a residual function residuals(p,a k ) to measure the direction vector a of the input given axis k The difference between the direction vector p and the rotation axis is:

[0117]

[0118] To initialize the fitted direction p0, the average of all input rotation axes is taken:

[0119]

[0120] Finally, the least squares optimization method is used to find the rotation axis direction vector p that minimizes the sum of all residual squares (i.e., the optimal one): * :

[0121]

[0122] The position of the camera origin in the world coordinate system is extracted through the external parameter matrix. These position points will constitute a point set, which represents the distribution of the camera in different positions and directions. This application uses the mean of the points as the initial guess of the center of the circle, and uses the average value of the distance from the point to the center of the circle as the initial guess of the radius. These initial values ​​provide a reasonable starting point for iterative optimization, and then define the residual function to calculate the sum of the squares of the difference between the distance from each point to the fitted circle and the radius, thereby obtaining the residual of each point. Our goal is to find the center and radius of the sphere that minimizes the sum of squares of these residuals. The trajectory of these position points can be fitted by minimizing the residual method to find the best fitting rotation axis.

[0123] Based on the same inventive concept, the embodiment of the present application also provides a laser plane and rotation axis joint calibration device for implementing the laser plane and rotation axis joint calibration method involved in the above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in the embodiments of one or more laser plane and rotation axis joint calibration devices provided below can refer to the limitations of the laser plane and rotation axis joint calibration method above, and will not be repeated here.

[0124] In an exemplary embodiment, a combined calibration device for a laser plane and a rotation axis is provided, comprising:

[0125] A chessboard image acquisition module is used to acquire chessboard images taken at different rotation angles;

[0126] A feature point extraction module is used to extract the intersection points of the laser stripes and the checkerboard lines in the checkerboard image as feature points;

[0127] A laser plane calibration module, used to calibrate the laser plane of the line laser based on the pixel coordinates of each feature point in each checkerboard image, and determine the laser plane equation of the line laser;

[0128] The rotation axis calibration module is used to calibrate the rotation axis of the pan / tilt rotating structure based on the checkerboard images taken at different rotation angles to determine the direction vector of the rotation axis.

[0129] Converting the center point of the laser stripe from the pixel coordinate system to the world coordinate system is a key step in achieving high-precision 3D reconstruction. We first convert the points in the camera coordinate system to points in the world coordinate system based on the pinhole camera model and the given laser plane equation. Then, the rotation matrix is ​​calculated based on the calibrated rotation axis and the known rotation angle. Finally, the preliminary point cloud is rotated by the calculated rotation matrix, while ensuring that the direction of the point cloud is consistent with the required reference direction. Through this series of geometric transformations, we achieve precise rotation of the point cloud, align the point cloud data generated by multiple frames of images, reconstruct the 3D scene from a series of 2D images, and finally obtain a high-precision 3D point cloud model, which provides rich spatial information for further analysis and application.

[0130] In an exemplary embodiment, a three-dimensional imaging method is provided, and the three-dimensional imaging method includes the following steps 401 to 404.

[0131] Step 401, obtaining images of a target object captured by a camera at different rotation angles;

[0132] Step 402, based on the laser plane equation of the line laser, convert each pixel coordinate in the target object image captured at different rotation angles into a three-dimensional coordinate in the camera coordinate system corresponding to each rotation angle;

[0133] Step 403, based on the external parameters of the camera at each rotation angle, convert the three-dimensional coordinates in the camera coordinate system corresponding to each rotation angle into the world coordinate system to obtain three-dimensional point cloud data at different rotation angles;

[0134] Step 404, rotating and merging the three-dimensional point cloud data at different rotation angles based on the direction vector of the rotation axis to obtain the three-dimensional point cloud data of the target object;

[0135] The laser plane equation of the line laser and the direction vector of the rotation axis are obtained by using the above-mentioned joint calibration method of the laser plane and the rotation axis.

[0136] Suppose the pixel coordinates P in the target object image captured at a given rotation angle θ p (θ)=[u(θ),v(θ)] T , then the calculation formula of the normalized coordinates is:

[0137]

[0138] Assuming that the plane equation in the camera coordinate system is ax+by+bz+d=0, then Z c (θ) can be calculated by the following formula:

[0139]

[0140] The Z calculated in the previous step c (θ), and then combined with the normalized coordinates, we get the three-dimensional coordinates P in the camera coordinate system corresponding to the rotation angle θ c (θ) = [X c (θ),Y c (θ),Z c (θ)] T :

[0141]

[0142] Among them, u(θ) and v(θ) are the x-axis component and y-axis component of the pixel coordinate transformation in the target object image captured at the rotation angle θ, respectively, and c x 、c y 、f x 、f y is the internal parameter of the camera, a, b, c are the three plane parameters of the laser plane equation, d is the distance parameter of the laser plane equation, X c (θ), Y c (θ), Z c (θ) are the x-axis component, y-axis component, and z-axis component of the three-dimensional coordinates in the camera coordinate system corresponding to the rotation angle θ.

[0143] Finally, the 3D point in the camera coordinate system is converted to the world coordinate system through the inverse transformation of the extrinsic matrix:

[0144] P w (θ) = R1(θ) T (P c (θ)-t(θ))

[0145] Among them, P w (θ) is the 3D point cloud data under the rotation angle θ, P c (θ) is the three-dimensional coordinate transformation in the camera coordinate system corresponding to the rotation angle θ, R1(θ) is the rotation matrix of the camera at the rotation angle θ, and t(θ) is the translation matrix of the camera at the rotation angle θ.

[0146] Finally, the 3D point cloud needs to be rotated to merge the point cloud data at different angles. The specific steps are as follows:

[0147] Given the rotation axis p = [p x ,p y ,p z ] T and the rotation angle θ, can be represented by the rotation matrix R2:

[0148] R2(θ)=Icos(θ)+(1-cos(θ))pp T +sin(θ)A

[0149] Among them, I is the unit matrix, p is the normalized rotation axis vector, and A is the antisymmetric matrix of the rotation axis vector:

[0150]

[0151] Finally, for each point P in the point cloud generated by different photos w (θ), rotate the corresponding angle around the rotation axis.

[0152] The three-dimensional imaging method of this application successfully achieves high-precision point cloud generation and stitching through precise laser stripe center extraction, pixel coordinate to world coordinate conversion, and point cloud rotation transformation based on the rotation axis. This method is not only rigorous and reliable in theory, but also shows good results in practical applications. Through experimental verification, this method can generate consistent and accurate three-dimensional point clouds at different rotation angles, providing solid technical support for high-precision three-dimensional reconstruction and measurement.

[0153] Furthermore, in order to enhance the visualization of the point cloud, the embodiment of the present application introduces reflectivity for visualization, which uses color coding to more intuitively display surface features. Different materials reflect laser light in different ways, so different materials can be distinguished by reflectivity. Reflectivity information can also provide clues about the integrity of the scan data.

[0154] Using the above laser plane and rotation axis joint calibration method and 3D imaging method, a vernier caliper and a scratched PE water pipe were 3D scanned at a distance of 1m. Figure 8 and Fig. 9 As shown, Figure 8 (a) is a schematic diagram of a vernier caliper placed on a chessboard. Figure 8 (b) is the scanning result of the vernier caliper. Fig. 9 (a) is a schematic diagram of a scratched PE water pipe. Fig. 9 (b) is the scanning result of a scratched PE water pipe. This method not only clearly reconstructs the 3D shape of the object, but also captures the details of the tiny letters engraved on the vernier caliper and the scratches on the PE water pipe.

[0155] Based on the same inventive concept, the embodiment of the present application also provides a three-dimensional imaging device for implementing the three-dimensional imaging method involved above. The implementation solution provided by the device to solve the problem is similar to the implementation solution recorded in the above method, so the specific limitations in one or more three-dimensional imaging device embodiments provided below can refer to the limitations on the three-dimensional imaging method above, and will not be repeated here.

[0156] In an exemplary embodiment, a three-dimensional imaging device is provided, comprising:

[0157] A target object image acquisition module is used to acquire images of the target object taken by the camera at different rotation angles;

[0158] A first coordinate conversion module is used to convert each pixel coordinate in the target object image captured at different rotation angles into a three-dimensional coordinate in a camera coordinate system corresponding to each rotation angle based on the laser plane equation of the line laser;

[0159] The second coordinate conversion module is used to convert the three-dimensional coordinates in the camera coordinate system corresponding to each rotation angle into the world coordinate system based on the external parameters of the camera at each rotation angle, so as to obtain the three-dimensional point cloud data at different rotation angles;

[0160] A third coordinate conversion module is used to rotate and merge the three-dimensional point cloud data at different rotation angles based on the direction vector of the rotation axis to obtain the three-dimensional point cloud data of the target object;

[0161] The laser plane equation of the line laser and the direction vector of the rotation axis are obtained by using the above-mentioned joint calibration method of the laser plane and the rotation axis.

[0162] In an exemplary embodiment, a computer device is provided. The computer device may be a server or a terminal. The internal structure diagram thereof may be as follows: Fig.10 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a joint calibration method or a three-dimensional imaging method of a laser plane and a rotation axis is implemented.

[0163] Those skilled in the art will understand that Fig.10 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0164] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0165] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0166] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0167] The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. The non-relational database may include a distributed database based on blockchain, etc., but is not limited thereto. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but is not limited thereto.

[0168] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0169] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A joint calibration method for a laser plane and a rotation axis, characterized in that: The joint calibration method is applied to a line laser rotation scanning imaging system, which includes: a line laser, a camera and a pan-tilt rotating structure; the line laser and the camera are both arranged on the pan-tilt rotating structure, and the joint calibration method includes: Get checkerboard images taken at different rotation angles; Extract the intersection points of the laser stripes and the checkerboard lines in the checkerboard image as feature points; Calibrate the laser plane of the line laser based on the pixel coordinates of each feature point in each checkerboard image to determine the laser plane equation of the line laser; The rotation axis of the gimbal rotation structure is calibrated based on the checkerboard images taken at different rotation angles to determine the direction vector of the rotation axis.

2. The combined calibration method of laser plane and rotation axis according to claim 1, characterized in that: The laser plane of the line laser is calibrated based on the pixel coordinates of each feature point in each checkerboard image, and the laser plane equation of the line laser is determined, specifically including: The pixel coordinates of each feature point in each checkerboard image are converted to the world coordinate system to obtain the world coordinates of each feature point; Calculate the mean of the world coordinates of each specific point as the world coordinates of the center point; The world coordinates of each specific point are centralized according to the world coordinates of the center point to obtain the world coordinates of each feature point after the centralization process; According to the world coordinates of each feature point after centralization, the least square method is used to solve the following formula to obtain the three plane parameters of the laser plane equation; Among them, a, b, c are the three plane parameters of the laser plane equation, x' i ,y' i and z' i are the x-axis component, y-axis component, and z-axis component of the world coordinates of the specific point after centralization, and N is the number of feature points; Based on the world coordinates of the center point and the three plane parameters of the laser plane equation, the distance from the origin to the laser plane is calculated using the following formula as the distance parameter of the laser plane equation; d = -n·p0; Where d is the distance parameter of the laser plane equation, n is the right singular vector, n = [a, b, c], and p0 is the world coordinate of the center point; Based on the three plane parameters and distance parameters of the laser plane equation, the laser plane equation is determined as: ax+by+cz+d=0; Among them, x, y, and z are the x-axis component, y-axis component, and z-axis component of the world coordinates of any point on the laser plane, respectively.

3. The combined calibration method of laser plane and rotation axis according to claim 1, characterized in that: The rotation axis of the gimbal rotation structure is calibrated based on the checkerboard images taken at different rotation angles to determine the direction vector of the rotation axis, including: Based on the checkerboard images taken at different angles, the axis of relative rotation between two adjacent angles is determined as a given axis; Based on the direction vector of the given axis, the least square method is used to solve the following formula to obtain the direction vector of the rotation axis; Among them, a k is the direction vector of the kth given axis, p is the direction vector of the rotation axis, and K is the number of given axes.

4. A combined calibration device for a laser plane and a rotation axis, characterized in that: The combined calibration device for the laser plane and the rotation axis applies the combined calibration method for the laser plane and the rotation axis according to any one of claims 1 to 3, and the combined calibration device for the laser plane and the rotation axis comprises: A chessboard image acquisition module is used to acquire chessboard images taken at different rotation angles; A feature point extraction module is used to extract the intersection points of the laser stripes and the checkerboard lines in the checkerboard image as feature points; A laser plane calibration module, used to calibrate the laser plane of the line laser based on the pixel coordinates of each feature point in each checkerboard image, and determine the laser plane equation of the line laser; The rotation axis calibration module is used to calibrate the rotation axis of the pan / tilt rotating structure based on the checkerboard images taken at different rotation angles to determine the direction vector of the rotation axis.

5. A three-dimensional imaging method, characterized in that: The three-dimensional imaging method comprises: Obtain images of the target object taken by the camera at different rotation angles; Based on the laser plane equation of the line laser, the coordinates of each pixel in the target object image taken at different rotation angles are converted into three-dimensional coordinates in the camera coordinate system corresponding to each rotation angle; Based on the external parameters of the camera at each rotation angle, the three-dimensional coordinates in the camera coordinate system corresponding to each rotation angle are converted to the world coordinate system to obtain the three-dimensional point cloud data at different rotation angles; Based on the direction vector of the rotation axis, the three-dimensional point cloud data at different rotation angles are rotated and merged to obtain the three-dimensional point cloud data of the target object; The laser plane equation and the direction vector of the rotation axis of the line laser are obtained by using the joint calibration method of the laser plane and the rotation axis as described in any one of claims 1 to 3.

6. The three-dimensional imaging method according to claim 5, characterized in that: The formula for converting each pixel coordinate in the target object image taken at different rotation angles into the three-dimensional coordinate in the camera coordinate system corresponding to each rotation angle based on the laser plane equation of the line laser is: Among them, u(θ) and v(θ) are the x-axis component and y-axis component of the pixel coordinate transformation in the target object image captured at the rotation angle θ, respectively, and c x 、c y 、f x 、f y is the intrinsic parameter of the camera, a, b, c are the three plane parameters of the laser plane equation, d is the distance parameter of the laser plane equation, X c (θ), Y c (θ), Z c (θ) are the x-axis component, y-axis component, and z-axis component of the three-dimensional coordinates in the camera coordinate system corresponding to the rotation angle θ.

7. The three-dimensional imaging method according to claim 5, characterized in that: Based on the external parameters of the camera at each rotation angle, the three-dimensional coordinates in the camera coordinate system corresponding to each rotation angle are converted to the world coordinate system. The formula for obtaining the three-dimensional point cloud data at different rotation angles is: P w (θ)=R1(θ) T (P c (θ)-t(θ); Among them, P w (θ) is the 3D point cloud data under the rotation angle θ, P c (θ) is the three-dimensional coordinate transformation in the camera coordinate system corresponding to the rotation angle θ, R1(θ) is the rotation matrix of the camera at the rotation angle θ, and t(θ) is the translation matrix of the camera at the rotation angle θ.

8. The three-dimensional imaging method according to claim 5, characterized in that: Based on the direction vector of the rotation axis, the three-dimensional point cloud data at different rotation angles are rotated and merged to obtain the three-dimensional point cloud data of the target object, specifically including: Based on the direction vector of the rotation axis, the rotation matrix corresponding to each rotation angle is determined as: R2(θ)=Icos(θ)+(1-cos(θ))pp T +sin(θ)A; Where R2(θ) is the rotation matrix corresponding to the rotation angle θ, I is the unit matrix, p is the direction vector of the rotation axis, and A is the antisymmetric matrix of the direction vector of the rotation axis; Based on the second rotation matrix corresponding to each rotation angle, the three-dimensional point cloud data at each rotation angle is rotated to obtain the three-dimensional point cloud data of the target object.

9. A three-dimensional imaging device, characterized in that: The three-dimensional imaging device comprises: A target object image acquisition module is used to acquire images of the target object taken by the camera at different rotation angles; A first coordinate conversion module is used to convert each pixel coordinate in the target object image captured at different rotation angles into a three-dimensional coordinate in a camera coordinate system corresponding to each rotation angle based on the laser plane equation of the line laser; The second coordinate conversion module is used to convert the three-dimensional coordinates in the camera coordinate system corresponding to each rotation angle into the world coordinate system based on the external parameters of the camera at each rotation angle, so as to obtain the three-dimensional point cloud data at different rotation angles; A third coordinate conversion module is used to rotate and merge the three-dimensional point cloud data at different rotation angles based on the direction vector of the rotation axis to obtain the three-dimensional point cloud data of the target object; The laser plane equation and the direction vector of the rotation axis of the line laser are obtained by using the joint calibration method of the laser plane and the rotation axis as described in any one of claims 1 to 3.

10. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the joint calibration method of the laser plane and the rotation axis as described in any one of claims 1 to 3 or the three-dimensional imaging method as described in any one of claims 5 to 8.

Images