An error correction method and device for a line structured light 3D camera

Abstract

The present invention provides an error correction method and device for a line structured light 3D camera, effectively solving the problem of point cloud distortion of the line structured light 3D camera caused by errors and improving the imaging accuracy of the line structured light 3D camera. The method includes the following steps: placing a three-dimensional calibration board at different positions in the field of view of the line structured light 3D camera and performing relative motion scanning on the three-dimensional calibration board to obtain multiple groups of point cloud data at different positions; processing the obtained point cloud data to calculate the corner coordinates of the three-dimensional calibration board corresponding to each group of point cloud data; taking the inclination angle between the straight line where the relative motion direction caused by the error is located and the laser plane of the line structured light 3D camera as an error model correction parameter to construct an error correction model; solving the error correction model parameters through the spatial vector constraint relationship between the corner points on the three-dimensional calibration board; and applying the error correction model to the point cloud data of the object to be measured to obtain undistorted point cloud data.

Description

Technical Field

[0001] The present invention relates to the field of 3D vision technology, and particularly to an error correction method and device for a line structured light 3D camera. Background Art

[0002] With the rapid development of machine vision, the application of 3D vision technology has become more and more common. The existing 3D imaging technologies mainly include binocular stereo vision method, laser triangulation method, structured light 3D imaging, time-of-flight method ToF, light field imaging method, and holographic projection technology. Among them, the binocular stereo vision method, line structured light 3D, and surface structured light 3D imaging have higher accuracy, so 3D cameras based on these three principles are widely used in the industrial field. Among them, the 3D camera based on single-line structured light has a simple principle, high measurement accuracy, low cost, and high stability, so it has important engineering application value and good market prospects.

[0003] According to the included angle relationship between the laser plane and the straight line where the relative motion direction is located, the line structured light 3D camera can be divided into a vertical type and an oblique type. For the vertical single-line structured light 3D camera, after the line structured light 3D camera captures the laser line contour, the center contour of the laser line is obtained by the laser line center extraction algorithm, and then after the operation of the transformation matrix obtained by calibration, a single 3D contour line of the surface of the object to be measured is obtained. Finally, a complete 3D point cloud of the surface of the object to be measured is formed by combining multiple contour lines.

[0004] The imaging of a single-line structured light 3D camera requires relative motion between the object to be measured and the camera. The ideal installation position needs to satisfy that the straight line where the relative motion direction is located is completely perpendicular to the laser plane. However, in actual applications, it is very difficult to ensure that the 3D camera is installed at the ideal position. Due to the machining error of the mechanical components and the installation error of the 3D camera, it is very difficult to ensure that the straight line where the motion direction is located is completely perpendicular to the laser plane. Once the installation of the line structured light 3D camera has an angular inclination, it will cause the captured point cloud to be distorted, thereby affecting the measurement accuracy of the three-dimensional measurement system. Therefore, studying how to correct the installation error of the line structured light 3D camera is an important means to improve the reliability of the line structured light 3D measurement system and ensure the three-dimensional measurement accuracy. Summary of the Invention

[0005] In view of the above problems, the present invention provides an error correction method and device for a line structured light 3D camera, effectively solving the problem of point cloud distortion of the line structured light 3D camera caused by errors, improving the imaging accuracy of the line structured light 3D camera, and providing a guarantee for subsequent high-precision 3D image processing.

[0006] Its technical solution is as follows: An error correction method for a line structured light 3D camera, characterized by comprising the following steps:

[0007] The 3D calibration plate is placed at different positions in the field of view of the line structured light 3D camera. The line structured light 3D camera and the 3D calibration plate are relatively moved to scan the 3D calibration plate, and multiple sets of point cloud data of the 3D calibration plate at different positions are obtained.

[0008] Process the obtained point cloud data and calculate the coordinates of the corner points of the three-dimensional calibration plate corresponding to each set of point cloud data;

[0009] The inclination angle between the straight line where the relative motion direction caused by the error is located and the laser surface of the line structured light 3D camera is used as an error model correction parameter to construct an error correction model;

[0010] The error correction model parameters are solved through the spatial vector constraint relationship between the corner points on the three-dimensional calibration plate;

[0011] The error correction model is applied to the point cloud data of the measured object to obtain distortion-free point cloud data.

[0012] Furthermore, the three-dimensional calibration plate includes a square plane, the four sides of the square plane are respectively connected to side planes, and an angle is formed between the square plane and the side planes.

[0013] Further, the obtained point cloud data is processed to calculate the corner point coordinates of the three-dimensional calibration plate corresponding to each set of point cloud data, which specifically includes the following steps:

[0014] The obtained point cloud data is filtered and the five plane data of the three-dimensional calibration plate are separated into five independent 3D point sets. The plane fitting method based on the RANSAC algorithm is used to fit the plane equation of the three-dimensional calibration plate, which is expressed as:

[0015] A n x+B n y+C n z+D n =0

[0016] Among them A n , B n , C n , D n are the plane parameters of plane n, where n is an integer.

[0017] The three plane equations around the four corner points of the square plane are expressed as:

[0018]

[0019] A0, B0, C0, D0 are the plane parameters of the first plane around the corner points; A1, B1, C1, D1 are the plane parameters of the second plane around the corner points; A2, B2, C2, D2 are the plane parameters of the third plane around the corner points;

[0020] Solve the three plane equations around the corner points in sequence to obtain the coordinates of the four corner points of the square plane. Each set of point cloud data yields a set of coordinates of the four corner points of the square plane.

[0021] Furthermore, when the line structured light 3D camera captures the three-dimensional calibration board, there are tilting angles that affect the measurement accuracy, including the tilting angle formed by the line structured light 3D camera rotating α degrees around the Z-axis of the world coordinate system and the tilting angle formed by rotating β degrees around the X-axis of the world coordinate system, the tilting angle α between the X-axis of the world coordinate system and the tilting angle β between the Z-axis of the world coordinate system. The Y-axis of the world coordinate system is parallel to the relative movement direction when the line structured light 3D camera captures the three-dimensional calibration board. The error correction model is constructed as follows:

[0022]

[0023] where, [X Y Z] T are the distorted point coordinates, [X′ Y′ Z′] T are the corrected three-dimensional point coordinates, and α, β are the error correction model parameters respectively.

[0024] Furthermore, solve the error correction model parameters through the spatial vector constraint relationship between the corner points on the three-dimensional calibration board, including the following steps:

[0025] Determine the spatial vector constraint relationship between the corner points on the three-dimensional calibration board, including that the square plane of the three-dimensional calibration board satisfies the inner product of the diagonal vectors being 0 and the modulus lengths of the two diagonals of the square plane of the three-dimensional calibration board being equal to times the side length of the square plane;

[0026] Set the square of the inner product of the corrected diagonal vectors of a single set of three-dimensional calibration boards as the objective function, and the objective function is expressed through the corner point coordinates and the error correction model parameters;

[0027] Use the Lagrange multiplier method to solve the objective function, and set the optimization function based on the corner point data of multiple sets of three-dimensional calibration boards; take the fact that the square plane of the three-dimensional calibration board satisfies the inner product of the diagonal vectors being 0 and the modulus lengths of the two diagonals of the square plane of the three-dimensional calibration board being equal to times as the constraint condition in the Lagrange multiplier method;

[0028] Establish the Lagrange function, make the first-order partial derivatives of the Lagrange function with respect to each error correction model parameter and the Lagrange multiplier equal to zero, and solve to obtain the error correction model parameters.

[0029] Furthermore, solving for the error correction model parameters through the spatial vector constraint relationship between the corner points on the three-dimensional calibration board includes:

[0030] Let the error correction model be \(x_1 = \cos\alpha\), \(x_2 = \sin\alpha\), \(x_3 = \sin\beta\), \(x_4 = \cos\beta\). The four corner points of the square plane of the three-dimensional calibration board are arranged clockwise as A, B, C, and D. After calibration, the inner product of the diagonal vectors is 0, which is expressed as: And the magnitudes of the two diagonals are equal to times the side length of the square plane, which is expressed as: where \(L\) is the side length of the square plane of the three-dimensional calibration board;

[0031] Set the square of the inner product of the diagonal vectors after calibrating a single set of three-dimensional calibration boards as the objective function:

[0032]

[0033] \(\Delta X\) bd represents the difference in the X-axis coordinates of points B and D, \(\Delta Y\) bd represents the difference in the Y-axis coordinates of points B and D, \(\Delta Z\) bd represents the difference in the Z-axis coordinates of points B and D; \(\Delta X\) ac represents the difference in the X-axis coordinates of points A and C, \(\Delta Y\) ac represents the difference in the Y-axis coordinates of points A and C, \(\Delta Z\) ac represents the difference in the Z-axis coordinates of points A and C.

[0034] Solve the objective function based on the Lagrange multiplier method, and set the optimization function as:

[0035]

[0036] where \(j\) is the number of times of imaging at different positions of the three-dimensional calibration board, \(f\) i (x1,x2,x3,x4) is the objective function of the three-dimensional calibration board at the \(i\)th imaging;

[0037] The optimization function satisfies the constraint conditions:

[0038]

[0039] And satisfies

[0040] Establish the Lagrangian function

[0041] where \(\lambda\) i is called the Lagrange multiplier;

[0042] Derivatives are taken with respect to \(x_1\), \(x_2\), \(x_3\), \(x_4\), and \(\lambda\) respectively, and the derivatives are set to 0. The values of \(x_1\), \(x_2\), \(x_3\), and \(x_4\) are obtained as the extreme points of the optimization function under the constraint conditions, and are used as the error correction model parameters. i Derivatives are taken with respect to \(x_1\), \(x_2\), \(x_3\), \(x_4\), and \(\lambda\) respectively, and the derivatives are set to 0. The values of \(x_1\), \(x_2\), \(x_3\), and \(x_4\) are obtained as the extreme points of the optimization function under the constraint conditions, and are used as the error correction model parameters.

[0043] A computer device, characterized in that it includes: a processor, a memory, and a program;

[0044] The program is stored in the memory, and the processor calls the program stored in the memory to execute the above-mentioned error correction method for a line structured light 3D camera.

[0045] A computer-readable storage medium, characterized in that: the computer-readable storage medium is used to store a program, and the program is used to execute the above-mentioned error correction method for a line structured light 3D camera.

[0046] For the error correction method of the line structured light 3D camera of the present invention, a three-dimensional calibration board is designed. The three-dimensional calibration board at different positions is photographed by the line structured light 3D camera, and then the corner coordinates on the three-dimensional calibration board are obtained. According to the inclination angle that may actually exist when the line structured light 3D camera photographs the three-dimensional calibration board and affects the measurement accuracy, the error correction model parameters are set. An error correction model is constructed based on the error correction model parameters, and then the error correction model parameters are obtained by solving the spatial vector constraints between the corner points on the three-dimensional calibration board. Finally, the point cloud data of the object to be measured can be corrected by the error correction model to obtain distortion-free point cloud data. The method of the present invention has the following advantages:

[0047] 1. The three-dimensional calibration board designed by the present invention has a simple structure, is easy to process, and has strong versatility and can be reused.

[0048] 2. By using the three-dimensional calibration board designed by the present invention, only the intersection points of the plane intersection lines in space are used as the characteristic corner points, and the algorithm has low complexity, high robustness, and high accuracy.

[0049] 3. The method of the present invention is simple to operate, has high calibration efficiency, and has operability. It effectively solves the problem of point cloud distortion caused by the installation error of the line structured light 3D camera, improves the imaging accuracy of the line structured light 3D camera, and provides a guarantee for subsequent high-precision 3D image processing.

[0050] 4. The method of the present invention has versatility and scalability. It can be used in general push-broom 3D imaging occasions, and the method has versatility. At the same time, it can be extended to the application scenario of an obliquely incident line structured light 3D camera. Description of the Drawings

[0051] Figure 1Schematic diagram of the steps of an error correction method for a line structured light 3D camera in an embodiment of the present invention;

[0052] Figure 2 Schematic diagram of the three-dimensional calibration board in the embodiment;

[0053] Figure 3 Schematic diagram of the line structured light 3D camera shooting an object;

[0054] Figure 4 Top view schematic diagram of the actual installation error of the line structured light 3D camera rotating by α degrees around the Z axis;

[0055] Figure 5 Schematic diagram of the actual installation error of the line structured light 3D camera rotating by β degrees around the X axis;

[0056] Figure 6 Internal structure diagram of a computer device in an embodiment. Detailed implementation manners

[0057] When the line structured light 3D camera images, relative movement between the camera and the target is required. When building the imaging system of the line structured light 3D camera, most rely on mechanical tooling constraints to make the straight line where the relative movement direction is located parallel to the straight line where the Y axis of the 3D camera is located. This has high requirements for machining and installation accuracy, and is difficult and costly to implement in actual applications. Therefore, the present invention provides an error correction method for a line structured light 3D camera in an embodiment, including the following steps:

[0058] Step 1: Place the three-dimensional calibration board at different positions in the field of view of the line structured light 3D camera, and relative movement occurs between the line structured light 3D camera and the three-dimensional calibration board to scan the three-dimensional calibration board, obtaining multiple groups of point cloud data of the three-dimensional calibration board at different positions;

[0059] Step 2: Process the obtained point cloud data and calculate the corner coordinates of the three-dimensional calibration board corresponding to each group of point cloud data;

[0060] Step 3: Take the inclination angle between the straight line where the relative movement direction due to the error is located and the laser plane of the line structured light 3D camera as the error model correction parameter, and construct an error correction model;

[0061] Step 4: Solve the error correction model parameters through the spatial vector constraint relationship between the corner points on the three-dimensional calibration board;

[0062] Step 5: Apply the error correction model to the point cloud data of the object to be measured to obtain undistorted point cloud data.

[0063] Specifically, in one embodiment of the present invention, in step 1, before using the line structured light 3D camera, it is necessary to calibrate the line structured light 3D camera.

[0064] Use the line structured light 3D camera to capture the checkerboard calibration board, extract the pixel coordinates of the corner points of the checkerboard calibration board in the image, obtain the homography matrix through the corresponding relationship between the pixel coordinates and the three-dimensional world coordinates, and then obtain the internal parameter matrix, lens distortion parameters, and external parameter matrix of the camera. Finally, obtain the optimal parameters through the optimization method. The relationship between the pixel coordinate system and the world coordinate system is as follows:

[0065]

[0066] In the formula, f x = f / d x , f y = f / d y , f / d x represents the length of the focal length in the X-axis direction described by pixels, and f / d y represents the length of the focal length in the Y-axis direction described by pixels. u0 and v0 represent the actual optical center coordinates, and the unit is also pixels. M is the internal parameter of the camera, and R and T are the rotation matrix and translation vector between the camera coordinate system and the world coordinate system, which are called external parameters.

[0067] The image distortion model includes radial distortion and tangential distortion, and the distortion model is as follows:

[0068]

[0069] In the formula, ρ is the distance from the coordinate to the origin, k1 and k2 are the radial distortion coefficients, p1 and p2 are the tangential distortion coefficients, (x, y) is the coordinate in the ideal state, and (x′, y′) is the coordinate with distortion.

[0070] In this step, the Zhang Zhengyou calibration method is adopted, and the maximum likelihood estimation method is used to optimize the calibration result. Finally, the result obtained by the operation is used as the initial value, and the LM least squares optimization is used to obtain more accurate internal and external parameter matrices.

[0071] In step 1, before using the line structured light 3D camera, it is also necessary to calibrate the laser pose of the line structured light 3D camera.

[0072] The fan-shaped area projected by the laser in the line structured light 3D camera can be regarded as a light plane. If we know the laser plane equation, we can calculate the homography matrix between the image plane and the laser plane. Through the homography matrix, the three-dimensional coordinates of the surface of the object to be measured can be calculated, and the calculation relationship is as follows:

[0073]

[0074] Where \(u'\) and \(v'\) are the pixel coordinates after lens distortion correction, \(s\) is the scale factor, and \(X\) and \(Z\) are the three-dimensional coordinates in the camera coordinate system. is the homography matrix.

[0075] The plane equation of the laser plane is expressed as: \(APx + BPy + CPz + DP = 0\), where \(AP\), \(BP\), \(CP\), and \(DP\) are the plane coefficients of the laser plane.

[0076] Place the backlit checkerboard calibration board at different positions within the camera's field of view, and take a backlit calibration board image and an image with a laser line respectively. Use the COG algorithm to extract the contour of the laser center line, and then through the transformation relationship between the calibration board coordinate system and the camera coordinate system obtain the three-dimensional points of the laser line in the camera coordinate system. In theory, the laser plane equation can be solved through three non-collinear points. However, to reduce errors, we use multiple sets of three-dimensional points of the laser line to fit and obtain the plane coefficients of the laser plane.

[0077] See Figure 2 , the three-dimensional calibration board used in this step includes a square plane 1, and the four sides of the square plane are respectively connected with side planes 2. There is an angle between the square plane 1 and the side planes 2. In this embodiment, the side planes 2 are all rectangles, and the angle between the square plane 1 and the side planes 2 ranges from 15 to 45 degrees. The selection of the angle parameter should meet two requirements: 1) It can be distinguished from the middle square plane; 2) When the calibration board has a certain inclination angle in space, it can ensure that enough 3D points can be collected for the 5 planes in the figure. In this embodiment, the selected angle of the included angle is 30 degrees.

[0078] After calibration is completed, the three-dimensional calibration board designed in this embodiment can be placed in the field of view of the structured light 3D camera. The structured light 3D camera and the three-dimensional calibration board perform relative movement to scan the three-dimensional calibration board, and a set of 3D point cloud data of the three-dimensional calibration board can be obtained. Place the three-dimensional calibration board at other positions in the field of view of the structured light 3D camera, and perform relative movement again to scan the three-dimensional calibration board, and another set of 3D point cloud data of the three-dimensional calibration board can be obtained. Adjust the three-dimensional calibration board to multiple different positions, and multiple sets of point cloud data of the three-dimensional calibration board at different positions can be obtained.

[0079] In an embodiment of the present invention, in step 2, the obtained point cloud data is processed to calculate the corner coordinates of the three-dimensional calibration board corresponding to each set of point cloud data; specifically, it includes the following steps:

[0080] Step 201: Filter the obtained point cloud data: The scanned point cloud data may have some noise. In order to improve the subsequent calibration accuracy, the obtained point cloud is first filtered to remove sparse outliers. For each point, the average distance between it and all neighboring points is calculated. Assuming that the result distribution is a Gaussian distribution with a mean and standard deviation, all points whose average distance is outside the interval defined by the global distance mean and standard deviation can be regarded as sparse outliers, and the outliers are trimmed from the point cloud data set.

[0081] Step 202: Separate the five plane data of the three-dimensional calibration plate into five independent 3D point sets.

[0082] Step 203: Using a plane fitting method based on the RANSAC algorithm, the plane equations of the five planes of the three-dimensional calibration plate are obtained through fitting, which are expressed as:

[0083] A n x+B n y+C n z+D n =0

[0084] Among them A n , B n , C n , D n are the plane parameters of plane n, where n is an integer.

[0085] Step 204: Extraction of characteristic corner points of the three-dimensional calibration plate: The three plane equations around the four corner points of the square plane are respectively expressed as:

[0086]

[0087] A0, B0, C0, D0 are the plane parameters of the first plane around the corner point; A1, B1, C1, D1 are the plane parameters of the second plane around the corner point; A2, B2, C2, D2 are the plane parameters of the third plane around the corner point;

[0088] By solving the three plane equations around the corner points in turn, we can get the coordinates of the four corner points.

[0089] It is known that due to the processing errors of the mechanical components and the installation errors of the 3D camera, it is difficult to ensure that the straight line where the movement direction is located is completely perpendicular to the laser surface. Therefore, in practice, there is an inclination angle between the straight line where the relative movement direction is located and the laser surface of the line structured light 3D camera due to the error. For this reason, several inclination angles are analyzed in this embodiment to have an impact on the measurement accuracy.

[0090] The schematic diagram of vertical line structured light 3D camera scanning imaging is as follows Figure 3As shown: The ideal installation position of the line structured light 3D camera is such that the line in the direction of motion is perpendicular to the laser plane. However, due to errors, there are the following three cases:

[0091] Case 1: When the line structured light 3D camera is installed and rotated by α degrees around the Z-axis of the world coordinate system in the figure, the top view is as Figure 4 shown. The line where AC is located is the actual laser line. A and C are the imaging points when there is an inclination error of α, and B and D are the theoretically imaging points. From the geometric relationship in the figure, it is easy to know:

[0092]

[0093] In the formula, [X Y Z] T is the distorted point coordinate, and [X′ Y′ Z′] T is the corrected three-dimensional point coordinate.

[0094] Case 2: When the line structured light 3D camera is installed and rotated only around the Y-axis of the world coordinate system in the figure, it is only equivalent to different positions of the measured object in the field of view of the line structured light 3D camera, which has no impact on the three-dimensional measurement accuracy.

[0095] Case 3: When the line structured light 3D camera is installed and rotated by β degrees around the X-axis of the world coordinate system in the figure, its front view is as Figure 5 shown. The line where AC is located is the actual laser line. A and C are the imaging points when there is an inclination error of β, and B and D are the theoretically imaging points. From the geometric relationship in the figure, it is easy to know:

[0096]

[0097] In the formula, [X Y Z] T is the distorted point coordinate, and [X′ Y′ Z′] T is the corrected three-dimensional point coordinate.

[0098] Therefore, in step 3, when the line structured light 3D camera shoots the three-dimensional calibration board, the inclinations that affect the measurement accuracy include the inclination formed by the line structured light 3D camera rotating by α degrees around the Z-axis of the world coordinate system and the inclination formed by rotating by β degrees around the X-axis of the world coordinate system. The Y-axis of the world coordinate system is parallel to the relative motion direction when the line structured light 3D camera shoots the three-dimensional calibration board. The constructed error correction model is expressed as:

[0099]

[0100] Among them, [X Y Z] T is the distorted point coordinate, [X′ Y′ Z′] T is the corrected three-dimensional point coordinate, and α and β are the error correction model parameters respectively.

[0101] The corresponding point coordinate transformation formula is as follows:

[0102]

[0103] Because there will be certain errors in 3D reconstruction and the extraction of the corner points of the 3D calibration board, only the optimal solutions can be obtained for the error correction model parameters α and β.

[0104] In the present invention, in step 4, the following steps are included:

[0105] Since the middle of the 3D calibration board is a square plane, the spatial vector constraint relationship between the corner points on the 3D calibration board can be determined, including that the square plane of the 3D calibration board satisfies that the inner product of the diagonal vectors is 0 and the modulus lengths of the two diagonals of the square plane of the 3D calibration board are equal to times the side length of the square plane;

[0106] Set the square of the inner product of the diagonal vectors after the correction of a single set of 3D calibration boards as the objective function, and the objective function is expressed by the corner point coordinates and the error correction model parameters;

[0107] Use the Lagrange multiplier method to solve the objective function, and set the optimization function based on the corner point data of multiple sets of 3D calibration boards; take that the square plane of the 3D calibration board satisfies that the inner product of the diagonal vectors is 0 and the modulus lengths of the two diagonals of the square plane of the 3D calibration board are equal to times as the constraint condition in the Lagrange multiplier method;

[0108] Establish the Lagrangian function, make the first-order partial derivatives of the Lagrangian function with respect to each error correction model parameter and the Lagrange multiplier equal to zero, and solve to obtain the error correction model parameters.

[0109] Specifically, in one embodiment, step 4 includes:

[0110] Let the error correction model be x1 = cosα, x2 = sinα, x3 = sinβ, x4 = cosβ, and the four corner points of the square plane of the 3D calibration board are arranged clockwise as A, B, C, and D. After correction, it satisfies that the inner product of the diagonal vectors is 0, which is expressed as: And the modulus lengths of the two diagonals are equal to times the side length of the square plane, which is expressed as: where L is the side length of the square plane of the 3D calibration board;

[0111] Set the square of the inner product of the diagonal vectors after the correction of a single set of 3D calibration boards as the objective function:

[0112]

[0113] ΔX bdThe coordinate difference of the X-axis between points B and D, ΔY bd The coordinate difference of the Y-axis between points B and D, ΔZ bd The coordinate difference of the Z-axis between points B and D; ΔX ac The coordinate difference of the X-axis between points A and C, ΔY ac The coordinate difference of the Y-axis between points A and C, ΔZ ac The coordinate difference of the Z-axis between points A and C.

[0114] Solve the objective function based on the Lagrange multiplier method, and set the optimization function as:

[0115]

[0116] In the formula, j is the number of times of imaging at different positions of the three-dimensional calibration plate, f i (x1, x2, x3, x4) is the objective function of the three-dimensional calibration plate at the i-th imaging, and n can be set between 10 and 20;

[0117] The optimization function satisfies the constraint conditions:

[0118]

[0119] And satisfy

[0120] To find the extreme point of the optimization function F(X) under the constraint conditions, first make the Lagrangian function, which is expressed as:

[0121]

[0122] In the formula, λ i Is called the Lagrange multiplier;

[0123] Then take the partial derivatives of x1, x2, x3, x4, λ i respectively, and set the derivatives to 0 to obtain x1, x2, x3, x4. The obtained x1, x2, x3, x4 are the extreme points of the optimization function under the satisfied constraint conditions, and then the error correction model parameters α and β are calculated through the inverse trigonometric function.

[0124] The Lagrange multiplier method is a method for finding the extreme value of a multivariate function whose variables are restricted by one or more conditions. This method transforms an optimization problem with n variables and k constraint conditions into an extreme value problem of a system of equations with n + k variables. The application of the Lagrange multiplier method in calculating the error correction model parameters in the present invention can obtain relatively accurate solutions of the error correction model parameters α and β.

[0125] In step 5, the error correction model with the calculated error correction model parameters α and β is used for the point cloud data of the object to be measured, and the undistorted point cloud data can be obtained.

[0126] Due to the machining and assembly accuracy problems of each component during the construction of the line structured light 3D camera, it is very difficult to ensure that the straight line where the Y-axis of the 3D camera is located in space is parallel to the straight line where the relative motion direction is located. This will cause distortion of the point cloud obtained by the 3D camera, and the distortion will affect the positioning and measurement accuracy of the entire 3D system.

[0127] The method provided in this embodiment first calibrates the camera parameters and the laser plane parameters. After the calibration is completed, the single contour 3D data of the surface of the object to be measured can be reconstructed; the 3D calibration board is scanned relatively to obtain the 3D point cloud data on the surface of the calibration board; the RANSAC algorithm is used to fit multiple planes on the surface of the calibration board, and the intersection points of three adjacent planes are calculated as the characteristic corner points of the 3D calibration board; an error correction model is constructed according to the principle of laser triangulation imaging, and the Lagrange multiplier method is used to solve the correction model parameters. This method is of great practical significance for improving the accuracy and measurement precision of the measurement system for the vision system based on the line structured light 3D camera.

[0128] In an embodiment of the present invention, a computer device is further provided, which includes: a processor, a memory, and a program;

[0129] The program is stored in the memory, and the processor calls the program stored in the memory to execute the above-mentioned error correction method for the line structured light 3D camera.

[0130] The computer device may be a terminal, and its internal structure diagram may be as Figure 6 shown. The computer device includes a processor, a memory, a network interface, a display screen, and an input device connected through a bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. The computer program, when executed by the processor, implements the error correction method for the line structured light 3D camera. The display screen of the computer device may be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device may be a touch layer covering the display screen, or a button, a trackball, or a touchpad provided on the housing of the computer device, or an external keyboard, touchpad, or mouse, etc.

[0131] The memory can be, but is not limited to, Random Access Memory (RAM), Read Only Memory (ROM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), etc. Among them, the memory is used to store programs, and the processor executes the programs after receiving execution instructions.

[0132] The processor can be an integrated circuit chip with the ability to process signals. The above-mentioned processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc. The processor can also be other general-purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field-Programmable Gate Arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. It can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc.

[0133] Those skilled in the art can understand that Figure 6 the structure shown in

[0134] is only a block diagram of some structures related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0135] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a computer device, or a computer program product. Therefore, the embodiments of the present invention can take the form of an all-hardware embodiment, an all-software embodiment, or an embodiment combining software and hardware aspects. Moreover, the embodiments of the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memory, CD-ROM, optical memory, etc.) that contain computer-usable program code.

[0136] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of methods, computer devices, or computer program products according to the embodiments of the present invention. These computer program instructions can be provided to the processors of general-purpose computers, special-purpose computers, embedded processors, or other programmable data processing terminal devices to generate a machine, such that the instructions executed by the processors of the computer or other programmable data processing terminal devices generate a device for realizing the functions specified in the flowcharts and / or block diagrams.

[0137] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing terminal device to work in a specific manner, such that the instructions stored in the computer-readable memory generate a manufactured article including an instruction device that realizes the functions specified in the flowchart.

[0138] The above has introduced in detail the application of the error correction method, computer device, and computer-readable storage medium provided by the present invention in a line-structured light 3D camera. Specific examples are used herein to elaborate on the principle and implementation manner of the present invention. The description of the above embodiments is only used to help understand the method and its core idea of the present invention; at the same time, for those of ordinary skill in the art, according to the idea of the present invention, there will be changes in the specific implementation manner and application scope. In summary, the content of this specification should not be construed as a limitation to the present invention.

Claims

1. An error correction method for a line structured light 3D camera, characterized in that The following steps are involved: The 3D calibration plate is placed at different positions in the field of view of the line structured light 3D camera. The line structured light 3D camera and the 3D calibration plate move relative to each other and scan the 3D calibration plate to obtain multiple sets of point cloud data of the 3D calibration plate at different positions. Process the obtained point cloud data and calculate the coordinates of the corner points of the three-dimensional calibration plate corresponding to each set of point cloud data; The inclination angle between the straight line where the relative motion direction caused by the error is located and the laser surface of the line structured light 3D camera is used as an error model correction parameter to construct an error correction model; The error correction model parameters are solved through the spatial vector constraint relationship between the corner points on the three-dimensional calibration plate; The error correction model is applied to the point cloud data of the measured object to obtain distortion-free point cloud data.

2. The error correction method for a line structured light 3D camera according to claim 1, characterized in that: The three-dimensional calibration plate includes a square plane, the four sides of the square plane are respectively connected to side planes, and an angle is formed between the square plane and the side planes.

3. A method for error correction of a line structured light 3D camera according to claim 2, characterized in that: The angle ranges from 15 to 45 degrees.

4. A method for error correction of a line structured light 3D camera according to claim 2, characterized in that: The obtained point cloud data is processed to calculate the corner point coordinates of the three-dimensional calibration plate corresponding to each set of point cloud data, which specifically includes the following steps: The obtained point cloud data is filtered and the five plane data of the three-dimensional calibration plate are separated into five independent 3D point sets. The plane fitting method based on the RANSAC algorithm is used to fit the plane equation of the three-dimensional calibration plate, which is expressed as: A n x + B n y + C n z + D n = 0 Among them, A n , B n , C n , D n are respectively the plane parameters of plane n, where n is an integer; The three plane equations around the four corner points of the square plane are expressed as: A0, B0, C0, D0 are the plane parameters of the first plane around the corner point; A1, B1, C1, D1 are the plane parameters of the second plane around the corner point; A2, B2, C2, D2 are the plane parameters of the third plane around the corner point; The three plane equations around the corner points are solved in sequence to obtain the coordinates of the four corner points of the square plane. Each set of point cloud data obtains a set of coordinates of the four corner points of the square plane.

5. A method for error correction of a line structured light 3D camera according to claim 4, characterized in that: The point cloud data is filtered and processed, including: For each point, the average distance between the point and all neighboring points is calculated. If the average distance of the point is outside the interval defined by the global distance mean and standard deviation, it is regarded as a sparse outlier point and the sparse outlier point is removed from the point cloud dataset.

6. A method for error correction of a line structured light 3D camera according to claim 4, characterized in that: When the line structured light 3D camera shoots the 3D calibration plate, there are inclination angles that affect the measurement accuracy, including the inclination angle formed by the line structured light 3D camera rotating around the Z axis of the world coordinate system by α degrees and the inclination angle formed by the line structured light 3D camera rotating around the X axis of the world coordinate system by β degrees. The Y axis of the world coordinate system is parallel to the relative motion direction when the line structured light 3D camera shoots the 3D calibration plate. The error correction model is constructed as follows: Among them, [X Y Z] T are the coordinates of the distorted points, and [X′ Y′ Z′] T are the corrected three-dimensional point coordinates, and α and β are the error correction model parameters respectively.

7. A method for error correction of a line structured light 3D camera according to claim 6, characterized in that: Solving the error correction model parameters through the spatial vector constraint relationship between the corner points on the three-dimensional calibration plate includes the following steps: Determine the spatial vector constraint relationship between the corner points on the three-dimensional calibration board, including that the square plane of the three-dimensional calibration board satisfies that the inner product of the diagonal vectors is 0 and the moduli of the two diagonals of the square plane of the three-dimensional calibration board are equal to times the side length of the square plane; The square of the inner product of the diagonal vectors after correction of a single set of three-dimensional calibration plates is set as the objective function, and the objective function is represented by the corner point coordinates and the error correction model parameters; The Lagrange multiplier method is used to solve the objective function, and the optimization function is set based on the corner point data of multiple groups of three-dimensional calibration plates; the square plane of the three-dimensional calibration plate satisfies that the inner product of the diagonal vectors is 0 and the lengths of the two diagonals of the square plane of the three-dimensional calibration plate are equal to the side length of the square plane times as the constraint condition in the Lagrange multiplier method; A Lagrangian function is established, and the first-order partial derivatives of the Lagrangian function with respect to each error correction model parameter and the Lagrangian multiplier are set to zero, and the error correction model parameters are obtained by solving.

8. A method for error correction of a line structured light 3D camera according to claim 7, characterized in that: Solving the error correction model parameters through the spatial vector constraint relationship between the corner points on the three-dimensional calibration board includes: Let the error correction model be \(x_1 = \cos\alpha\), \(x_2 = \sin\alpha\), \(x_3 = \sin\beta\), \(x_4 = \cos\beta\). The four corner points of the square plane of the three-dimensional calibration plate are arranged clockwise as \(A\), \(B\), \(C\), and \(D\). After calibration, the inner product of the diagonal vectors is 0, which is expressed as: And the magnitudes of the two diagonals are equal to times the side length of the square plane, which is expressed as: where \(L\) is the side length of the square plane of the three-dimensional calibration plate; Setting the square of the inner product of the diagonal vectors after the single-group three-dimensional calibration board is corrected as the objective function: ΔX bd represents the difference in the X-axis coordinates between points B and D, ΔY bd represents the difference in the Y-axis coordinates between points B and D, ΔZ bd represents the difference in the Z-axis coordinates between points B and D; ΔX ac represents the difference in the X-axis coordinates between points A and C, ΔY ac represents the difference in the Y-axis coordinates between points A and C, ΔZ ac represents the difference in the Z-axis coordinates between points A and C; Solving the objective function based on the Lagrange multiplier method, and setting the optimization function as: where j is the number of times of imaging at different positions of the three-dimensional calibration board, and f i (x1, x2, x3, x4) is the objective function of the three-dimensional calibration board during the i-th imaging; The optimization function satisfies the constraint conditions: and satisfy Establish the Lagrangian function where λ i is called the Lagrange multiplier; Derive the partial derivatives with respect to \(x_1\), \(x_2\), \(x_3\), \(x_4\), and \(\lambda\) respectively, and set the derivatives to zero. Solve for \(x_1\), \(x_2\), \(x_3\), and \(x_4\), which are the extreme points of the optimization function under the given constraints, and use them as the parameters of the error correction model. i ​ 9. A computer device, characterized in that, It includes a processor, a memory, and a program; The program is stored in the memory, and the processor calls the program stored in the memory to execute an error correction method for a line structured light 3D camera according to claim 1.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store a program, and the program is used to execute an error correction method for a line structured light 3D camera according to claim 1.

Images