A full-automatic calibration method and device for a line laser 3D camera system

Abstract

The present application discloses a kind of line laser 3D camera system full-automatic calibration method and device, calibration parameter includes internal parameter, distortion coefficient, sensor tilt angle, laser surface parameter, rotation vector and translation vector, and calibration method is as follows: obtain the first disc calibration plate graph of different poses of first disc calibration plate, obtain the initial value of internal parameter, distortion coefficient and sensor tilt angle;Second disc calibration plate to line laser camera different distance second disc calibration plate graph and laser line graph L2 are obtained, laser surface calibration is carried out, and the initial value of laser surface parameter, rotation vector and translation vector is obtained;Grid plate to line laser camera different distance laser line graph L3 is obtained, the error of the calibration parameter is examined using the laser line graph L3, the calibration parameter is corrected according to error, and the calibration parameter after correction is obtained.The present application designs the method of calculating initial value first, then correcting initial value, improves the calibration precision of line laser 3D camera.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of line laser 3D camera calibration, in particular to a kind of line laser 3D camera system full-automatic calibration method. BACKGROUND

[0002] Line laser 3D camera is a kind of three-dimensional camera based on triangulation principle, which captures laser line information projected on the surface of object by image sensor, reconstructs the profile information of object surface, and is mainly composed of Shum camera and line laser generator.For the calibration of line laser 3D camera, mainly includes two steps: Shum camera calibration, laser surface calibration, calibration parameters include internal parameter, distortion coefficient, sensor tilt angle, laser surface parameter, rotation vector and translation vector of laser surface relative to camera coordinate system, the process is very complex, if only rely on manual to collect various images will spend a lot of time, this is unacceptable for mass production of line laser 3D camera in industry.In order to solve the above problems, the present application proposes a kind of line laser 3D camera full-automatic calibration method and process, which can realize all calibration process by one key, and obtain the final calibration parameter for three-dimensional reconstruction.

[0003] Based on line laser three-dimensional reconstruction technology, mainly through projecting laser to object surface, laser is modulated due to the change of object surface depth and possible gap, which is reflected in the change and discontinuity of light bar in image, the degree of change is proportional to depth, and discontinuity shows the physical gap of object surface.Through mathematical model to reconstruct this modulation process, so as to obtain three-dimensional coordinates according to two-dimensional laser stripe image of intersection line of laser plane and object surface.Because of its fast reconstruction speed, simple structure, high precision and strong anti-interference ability, it is widely used in product size detection, weld tracking, workpiece profile measurement, robot trajectory guidance and many other scenes in automobile production, electronic manufacturing and other fields.Based on line laser three-dimensional reconstruction technology mainly includes camera calibration, laser stripe center line extraction, laser plane calibration, etc., among them, laser stripe center line extraction has gray gravity center method, steger algorithm based on Hessian matrix and many mature solutions, and camera calibration is mainly completed by Zhang's calibration method of two-dimensional target and its improved method.When the calibration parameters obtained after camera and laser plane are completed are used for three-dimensional reconstruction, with the increase of range, the error gradually increases, so the measurement accuracy needs to be compensated.Shum camera complex calibration makes the time cost of calibration once is huge, and it is unrealistic to realize mass production by manual image shooting calibration mode.

[0004] A calibration method for an off-axis Scheimpflug 3D line laser camera is proposed in patent CN114359405A, which comprises: constructing an imaging model of the off-axis Scheimpflug 3D line laser camera; constructing a virtual image plane coordinate system parallel to the lens plane and passing through the imaging center of the optical axis, realizing the conversion between the image pixel coordinates and the coordinates in the world coordinate system; first calculating the initial values of the camera internal and external parameters, and then using a nonlinear optimization algorithm to calculate the optimal solution to obtain the camera calibration; collecting no less than 2 sets of calibration board pictures with and without laser for laser plane calibration, obtaining laser line data by the difference method of the calibration board pictures in each group, calculating the three-dimensional coordinates of the laser line center pixel coordinates in the camera coordinate system, and fitting to obtain the laser plane equation to obtain the calibration of the laser plane. The problem of difficult calibration of the off-axis Scheimpflug 3D line laser camera is solved. The technical problems solved by this technical solution are most close to those solved by the present solution, but the devices used are different, and the calibration methods and principles are also different.

[0005] In addition, patent CN113160339A discloses a projection instrument calibration method based on Scheimpflug law, which uses a higher-precision bidirectional multi-frequency multi-step phase shift method for calibration of a Scheimpflug projection instrument, improves the calculation accuracy of the phase, constructs a Scheimpflug projection instrument imaging model, calculates the initial values of the internal and external parameters by using the existing Zhang analytical algorithm, optimizes the internal and external parameters and distortion coefficients of the projection instrument by using the LM algorithm, and further optimizes the Scheimpflug angle by using the LM algorithm. This scheme does not involve the calibration of the laser plane, and the optimization method is different from the principle of the present application, and cannot realize the fast and accurate correction scheme in the present application. The core of this patent document is the bidirectional multi-frequency multi-step phase shift method, which cannot solve the full-automatic calibration process of the present application.

[0006] In summary, the disadvantages of the prior art are as follows:

[0007] (1) The calibration of the Scheimpflug camera is complex, and the time cost of one calibration is huge. It is unrealistic to achieve mass production through manual picture shooting calibration.

[0008] (2) The existing technology has insufficient accuracy in the calibration of the line laser 3D camera. SUMMARY

[0009] The line laser 3D camera system full-automatic calibration method and device proposed by the present application can at least solve one of the above technical problems.

[0010] To achieve the above-mentioned purposes, the present application proposes the following technical solutions:

[0011] A line laser 3D camera system full-automatic calibration method comprises:

[0012] A first disc calibration board picture in different poses is obtained, and the line laser 3D camera is calibrated by using the first disc calibration board picture to obtain internal parameters, distortion coefficients and sensor tilt angles;

[0013] acquire a second disc calibration plate image and a laser line image L2 of the second disc calibration plate at different distances from the line laser 3D camera, perform laser plane calibration using the laser line image L2 and the second disc calibration plate image, and obtain laser plane parameters, a rotation vector and a translation vector of the laser plane relative to a camera coordinate system;

[0014] The calibration parameters include intrinsic parameters, distortion coefficients, sensor tilt angles, laser plane parameters, a rotation vector and a translation vector of the laser plane relative to the camera coordinate system.

[0015] acquire a laser line image L3 of the grid plate at different distances from the line laser 3D camera, use the laser line image L3 to verify errors of three-dimensional coordinates reconstructed using the calibration parameters, correct the calibration parameters according to the errors, and obtain corrected calibration parameters.

[0016] Further, the method further comprises:

[0017] adjusting, by an automatic exposure algorithm, an exposure time when the line laser 3D camera captures the first disc calibration plate image at different poses, so that the brightness of each first disc calibration plate image is consistent.

[0018] The automatic exposure algorithm comprises:

[0019] presetting a gray value range, extracting the first disc calibration plate from the first disc calibration plate image by using adaptive threshold segmentation, and taking a region other than the black disc on the first disc calibration plate as a foreground.

[0020] calculating a mean value T1 of all pixel values in the foreground, and calculating a mean value T2 of pixel points with pixel values greater than T1 in the foreground.

[0021] If T2 is within the preset gray value range, the exposure time meets the requirements; if T2 is not within the preset gray value range, the exposure time is adjusted until T2 is within the preset gray value range.

[0022] Further, the acquiring of the second disc calibration plate image and the laser line image L2 of the second disc calibration plate at different distances from the line laser 3D camera comprises:

[0023] adjusting the position between the line laser 3D camera and the second disc calibration plate so that the laser line hits between two rows of discs on the second disc calibration plate, and the line laser 3D camera can capture the second disc calibration plate clearly.

[0024] respectively make the second disc calibration plate close to and away from the line laser 3D camera, and simultaneously acquire multiple second disc calibration plate images and laser line images L2.

[0025] The light is supplemented when the second disc calibration plate picture is acquired, and the light source of the light supplementing is the same as the wavelength of the light source of the line laser 3D camera; the laser line picture L2 is a picture of the line laser emitted by the line laser 3D camera on the second disc calibration plate.

[0026] Further, the laser line picture L3 of the grid plate at different distances from the line laser 3D camera is acquired, and the laser line picture L3 comprises:

[0027] The relative position between the line laser 3D camera and the grid plate is adjusted, so that the line laser emitted by the line laser 3D camera is parallel to the grid reference line on the grid plate, and the position of the laser line on the grid plate does not change when the grid plate is close to or far away from the line laser 3D camera;

[0028] The grid plate is respectively close to and far away from the line laser 3D camera, and a plurality of laser line pictures L3 are acquired simultaneously, and the relative position between the grid plate and the line laser 3D camera when each laser line picture L3 is acquired is recorded, and the laser line picture L3 is a picture of the line laser emitted by the line laser 3D camera on the grid plate.

[0029] Further, the line laser 3D camera is calibrated by using the first disc calibration plate picture, and the intrinsic parameters, the distortion coefficients and the sensor tilt angle are obtained, and the calibration comprises:

[0030] According to the first disc calibration plate picture, the initial intrinsic parameters, the initial distortion coefficients and the initial sensor tilt angle of the line laser 3D camera are calculated by using the Sharm camera calibration method, and the re-projection error of each feature point of each picture is recorded;

[0031] The first disc calibration plate picture with smaller re-projection error is selected, and the Sharm camera calibration method is used for further calibration, and the calibration result with the smallest re-projection error is selected as the intrinsic parameters, the distortion coefficients and the sensor tilt angle of the line laser 3D camera.

[0032] Further, the laser plane is calibrated by using the laser line picture L2 and the second disc calibration plate picture, and the laser plane parameters, the rotation vector and the translation vector of the laser plane relative to the camera coordinate system are obtained, and the calibration comprises:

[0033] A world coordinate system is established with the current second disc calibration plate as a reference, the X and Y axes are on the surface of the disc calibration plate, and the Z axis is perpendicular to the second disc calibration plate;

[0034] According to the current second disc calibration plate picture, the rotation matrix R and the translation matrix T of the current disc calibration plate relative to the line laser 3D camera coordinate system are calculated in combination with the camera intrinsic parameters and the distortion coefficients;

[0035] The 3D coordinates of each point on the laser line are obtained according to R, T, the intrinsic parameters and the transformation relationship between the normalized line laser 3D camera coordinates and the pixel coordinates;

[0036] Calculate 3D coordinates of each point on the laser line on each second disc calibration plate image, and fit to obtain laser plane parameters;

[0037] Use the Rodrigues transformation to rotate the laser plane to the XOZ plane of the world coordinate system for the first time, and obtain the rotation vector rot_plane; Then move the intersection of the laser plane and the Z axis to the center of the world coordinate system for the first time, and obtain the translation vector t_plane;

[0038] Secondly, rotate the laser plane so that the Z coordinates of the center points of the laser lines with the same X coordinates are the same, and obtain the rotation vector rot_vec of the laser plane relative to the camera coordinate system according to the first rotation and the second rotation, and obtain the translation vector t_vec of the laser plane relative to the camera coordinate system according to the first translation.

[0039] Further, the error of the three-dimensional coordinates reconstructed using the calibration parameters is verified by using the laser line image L3, and the calibration parameters are corrected according to the error to obtain corrected calibration parameters, comprising:

[0040] The Z coordinate calculation value of the laser line in each image in the laser line image L3 is calculated using the calibration parameters;

[0041] The position of each image in the laser line image L3 is known, and the measured value of the Z coordinate of the laser line in each image is obtained;

[0042] The Z coordinate calculation value of the laser line and the Z coordinate measured value of the laser line are compared to obtain a correction objective function;

[0043] The calibration parameters are corrected according to the correction objective function.

[0044] In another aspect, the present application also provides a full-automatic calibration device for a line laser 3D camera system, comprising:

[0045] A first disc calibration plate with adjustable pose, used for the line laser 3D camera to shoot different pose first disc calibration plate images;

[0046] A second disc calibration plate with adjustable relative position to the line laser 3D camera, used for the line laser 3D camera to shoot different position second disc calibration plate images and laser line image L2;

[0047] A grid plate with adjustable relative position to the line laser 3D camera, used for the line laser 3D camera to shoot different position laser line image L3;

[0048] A host computer for calculating calibration parameters, the calibration parameters including intrinsic parameters, distortion coefficients, sensor tilt angles, laser plane parameters, rotation vectors and translation vectors of the laser plane relative to the camera coordinate system;

[0049] The host computer obtains an intrinsic parameter, a distortion coefficient and a sensor tilt angle by using the first disc calibration board icon to calibrate a laser 3D camera; obtains a laser plane parameter, a rotation vector and a translation vector of the laser plane relative to a camera coordinate system by using a laser line diagram L2 and a second disc calibration board; and obtains a corrected calibration parameter by using a laser line diagram L3 to test a three-dimensional coordinate error after reconstruction, and correcting the calibration parameter according to the error.

[0050] Further, it further comprises:

[0051] A laser interferometer is used to measure the displacement of the grid plate between different positions of the laser line diagram L3 with high precision.

[0052] A light supplement lamp is used to supplement light when the first disc calibration board diagram and the second disc calibration board diagram are acquired, and the light supplement lamp has the same wavelength as the light source of the line laser 3D camera.

[0053] A six-axis mechanical arm is used to fix the first disc calibration board, and the six-axis mechanical arm drives the first disc calibration board to move, so that the line laser 3D camera can shoot the first disc calibration board diagram in different poses.

[0054] A three-axis fine adjustment platform is used to fix the line laser 3D camera and adjust the spatial position of the line laser 3D camera, and the three-axis fine adjustment platform comprises:

[0055] Before shooting the laser line diagram L3 at different distances, the first axis of the three-axis fine adjustment platform is adjusted so that the laser line is horizontal in the image.

[0056] The second axis of the three-axis fine adjustment platform is adjusted so that the slope of the laser line and the grid reference line is the same, and the laser line is parallel to the grid reference line.

[0057] The third axis of the three-axis fine adjustment platform is adjusted so that the distance between the laser line and the grid reference line remains unchanged whether the grid plate is moving away from or approaching the camera.

[0058] In another aspect, based on the same inventive concept, the present application also provides a computer readable storage medium, wherein the computer program is executed by a processor to make the processor execute the above-mentioned line laser 3D camera system full-automatic calibration method.

[0059] The beneficial effects of the present application are as follows:

[0060] (1) The device specially designed for line laser 3D camera calibration is combined with the calibration method to quickly and automatically complete the calibration of the line laser 3D camera.

[0061] (2) The line laser 3D camera calibration method calculates the initial value first, and then corrects the initial value, thereby improving the calibration accuracy of the line laser 3D camera.

[0062] (3) Full automatic, fast calibration process, not dependent on manual operation, more suitable for industrial production. BRIEF DESCRIPTION OF DRAWINGS

[0063] Figure 1 is the full automatic calibration method flow chart of the line laser 3D camera system in the embodiment of the application;

[0064] Figure 2 is the top view of the full automatic calibration device of the line laser 3D camera system in the embodiment of the application;

[0065] Figure 3 is the schematic diagram of the vertical image plane and the inclined image plane in the application;

[0066] Figure 4 is the grid plate schematic diagram in the embodiment of the application.

[0067] In the figure: 1-first disc calibration plate; 2-second disc calibration plate; 3-grid plate; 4-platform base; 5-three-axis fine adjustment platform; 6-line laser 3D camera; 7-light supplement lamp; 8-laser interferometer; 9-XY moving platform; 10-mover; 11-six-axis mechanical arm. DETAILED DESCRIPTION

[0068] In order to make the purpose, technical scheme and advantages of the embodiments of the application clearer, the technical scheme in the embodiments of the application will be described clearly and completely below in combination with the drawings in the embodiments of the application. Obviously, the described embodiments are part of the embodiments of the application, rather than all the embodiments.

[0069] The application provides a line laser 3D camera full automatic calibration method and device, which can realize all calibration processes of the line laser 3D camera by one key, and further corrects the calibration parameters through the laser line 3D coordinates on the grid plate 3 with accurate relative position, so as to obtain the final calibration parameters and compensation parameters for three-dimensional reconstruction.

[0070] The platform base 4 of the line laser 3D camera full automatic calibration device is provided with an XY moving platform 9, which is used to drive the mover 10 to move in the XY direction; for example, Figure 1As shown, the mover 10 is provided with a six-axis mechanical arm 11 with a known relative position, a second disc calibration plate 2, and a grid plate 3, wherein the six-axis mechanical arm 11 is installed with the first disc calibration plate 1, and the six-axis mechanical arm 11 is used to rotate the disc calibration plate to obtain different pose first disc calibration plate pictures. The XY moving platform 9 is provided with a three-axis fine adjustment platform 5 opposite, and the three-axis fine adjustment platform 5 is provided with a line laser 3D camera 6 to be calibrated, and the three-axis fine adjustment platform 5 is used to adjust the spatial position of the line laser 3D camera 6. The device is also provided with a fill light 7 located opposite the XY moving platform 9 and on the same side of the line laser 3D camera 6, and the fill light 7 is used for the fill light of the picture after the laser is turned off. It also includes a laser interferometer 8, which in this embodiment is arranged opposite the XY moving platform 9 and on the opposite side of the line laser 3D camera 6, and is used to obtain the accurate distance of the XY moving platform 9 along Y movement. The position of the laser interferometer 8 is not limited by this embodiment, as long as it can be used to measure the displacement of the XY moving platform 9 in the Y direction. In addition, the device is connected with the upper computer, and the switch and displacement of each part are controlled through the upper computer, and the information collected by each part is obtained.

[0071] Firstly, the position information of position 1, position 2 and position 3 needs to be obtained, position 1 is the position at which the line laser 3D camera 6 can clearly capture the first disc calibration plate 1 on the six-axis mechanical arm 11, position 2 is the position at which the line laser 3D camera 6 can clearly capture the second disc calibration plate 2 for laser plane calibration, and position 3 is the position at which the line laser 3D camera 6 can clearly capture the grid calibration plate. When the platform is powered on and reset, the XY moving platform 9 automatically reaches the reset zero point, and the above-mentioned positions 1, 2 and 3 are positioned according to the reset zero point.

[0072] The line laser 3D camera full-automatic calibration method and process of the application refers to the case that the relative positions of various modules are fixed in advance, and only one key is needed to complete the whole process of line laser 3D camera calibration, including: Sham camera calibration, laser plane calibration. The following steps are included:

[0073] Step 100, the calibration positions 1, 2 and 3 are obtained in advance, and the parallelism of the grid plate 3 on the XY moving platform 9 is adjusted through the laser interferometer 8, so that the XY moving platform is moved forward and backward along the Y axis direction, and the light spot position of the laser interferometer 8 reflected by the grid plate 3 is unchanged.

[0074] Step 200, the platform is powered on and reset, and the positions in step 100 are measured relative to the reset zero point.

[0075] Step 300, the XY moving platform 9 moves to the calibration position 1, the fill light 7 is turned on, the six-axis mechanical arm 11 is rotated, and at each rotation position, the line laser 3D camera 6 automatically exposes and collects the first disc calibration plate picture, and finally obtains different pose first disc calibration plate pictures.

[0076] Step 400, XY moving platform 9 moves to calibration position 2, XY moving platform 9 samples n positions between -y position and +y position along Y axis direction, wherein +y and -y are positions with position 2 as the origin, length y and sampling number n are preset values; at each position, automatically expose and shoot laser line diagram L2 and second disc calibration plate diagram, and at each position, the laser line is between the two rows of discs of the disc calibration plate.

[0077] Step 500, using laser line diagram L2 and second disc calibration plate diagram obtained in step 400, calibrate the Shum camera by using Shum camera calibration method, obtain laser plane equation by laser plane fitting, and obtain laser plane parameters.

[0078] Step 600, XY moving platform 9 moves to calibration position 3, adjusts 1-axis of three-axis fine adjustment platform 5 to make Z coordinates of both ends of the laser line the same, so that the laser line is horizontal in the image, adjusts 2-axis to make the slope of the laser line and the grid reference line in grid plate 3 the same, so that the laser line is parallel to the grid reference line, and adjusts 3-axis to make the distance between the laser line and the grid reference line below it remain unchanged when grid plate 3 moves away from or approaches line laser 3D camera 6.

[0079] Step 700, X-axis direction of XY moving platform 9 is stationary at target position 3, and 2k+1 positions are sampled from -z position to +z position along Y-axis direction, wherein k positions are sampled before and after 0 position, at each position, automatically expose and shoot the picture of the laser line hitting on grid plate 3, i.e. laser line diagram L3; wherein -z and +z are positions with position 3 as the origin, z and k are preset values, and during the movement, laser interferometer 8 accurately measures the movement distance of grid plate 3.

[0080] Step 800, with the help of laser interferometer 8, the coordinates of the laser line on grid plate 3 relative to position 3 can be accurately measured, and the calibration parameters are corrected by using laser line diagram L3.

[0081] Step 900, complete the calibration of line laser 3D camera 6, and obtain the final calibration parameters.

[0082] In the above step 100, grid plate 3 is a high-precision diffuse reflection plane, and the adjustment steps before calibration are as follows:

[0083] S101, tightly attach the mirror sheet with high reflectivity and high flatness to grid plate 3, and move XY moving platform 9 forward and backward along Y axis direction, so that the reflected light spot position of the mirror sheet remains unchanged.

[0084] S102, reset the XY moving platform 9, take the reset zero point as the zero point of each subsequent calibration, adjust the position of the XY moving platform 9 from the zero point, and find suitable 1, 2, and 3 calibration positions. For position 1, the initial position of the first disc calibration plate 1 is required to be clear in the image; for position 2, the XY moving platform 9 is required to move along the Y axis, and the laser line is always between the two rows of discs of the second disc calibration plate 2; for position 3, the grid plate 3 is required to be as much as possible in the middle of the field of view of the line laser 3D camera 6. The relative positions of the first disc calibration plate 1, the second disc calibration plate 2, and the grid plate 3 are fixed, so after the initial position is adjusted, subsequent calibration of a new camera only needs to adjust one position to meet the condition, and the remaining positions will automatically meet the condition.

[0085] At this point, the adjustment step before calibration is completed.

[0086] In the above step 300, in order to obtain high-precision Shum camera calibration parameters, multiple first disc calibration plate pictures of different poses need to be obtained, and the brightness of each picture is required to be substantially the same, so the exposure time of the line laser 3D camera 6 needs to be controlled to be different for each pose, so that the brightness of each picture is substantially consistent. The automatic exposure algorithm steps are as follows:

[0087] S301, set the gray scale range.

[0088] S302, use adaptive threshold segmentation to extract the disc calibration plate, at this time all regions on the calibration plate except the black discs will be divided into foreground P, and all other regions in the image will be background B.

[0089] S303, calculate the mean value of all pixels in P, denoted as T1.

[0090] S304, in order to prevent other bright areas in the image from being divided into foreground and causing interference, find all pixel points in P whose pixel values are greater than T1, and calculate the mean value of these pixel points again, denoted as T2. When T2 is within the pre-set gray scale range, it indicates that the brightness of the picture meets the requirements, and the exposure time meets the requirements. If T2 is not within the pre-set gray scale range, adjust the exposure time until T2 is within the pre-set gray scale range.

[0091] At this point, the automatic exposure picture taking is completed.

[0092] In the above step 500, the Shum camera needs to be calibrated, including the camera intrinsic parameters C, the extrinsic parameters O, the distortion coefficient D, and the sensor tilt angle A. The specific steps are as follows:

[0093] S501, use Zhang Zhengyou's calibration method to obtain the initial camera intrinsic and extrinsic parameters C and O, and the distortion coefficient D, and remove the first disc calibration plate pictures that cannot be detected.

[0094] S502, find the center point of each disc in the first disc calibration board picture, through the mutual transformation of the inclined image plane and the vertical image plane, the least squares method is used to continuously iterate and optimize the camera internal parameter, distortion coefficient and sensor tilt angle.

[0095] The above steps S501 and S502 are the Shum camera calibration method, wherein the vertical image plane is a virtual image plane, parallel to the lens plane of the Shum camera and the imaging center passes through the optical axis, and the inclined image plane is the actual imaging image plane, as shown in Figure 3 .

[0096] S503, record the re-projection error of each feature point of each picture in each iteration.

[0097] S504, arrange the errors at each feature point in each picture in descending order, take the average value E1 of the errors of the top five feature points, arrange E1 of each picture in descending order, and select the n pictures with the smallest E1.

[0098] S505, repeat steps 502 and 503 for the n pictures in S504, select the calibration result of the iteration with the smallest average error as the final calibration result, and obtain the initial value of the internal parameter C, distortion coefficient D and sensor tilt angle A of the camera.

[0099] At this point, the Shum camera calibration is completed.

[0100] In the above step 500, the laser plane is also calibrated, which uses the laser line diagram L2 at each position in step 400 and the second disc calibration board picture, and the steps are as follows:

[0101] S506, establish a world coordinate system with the current second disc calibration board as a reference, the X and Y axes are on the surface of the disc calibration board, and the Z axis is perpendicular to the disc calibration board; extract the center point of each disc of the disc calibration board, and record its pixel coordinates and the relative world coordinates .

[0102] S507, according to , , C, D, the PNP algorithm can be used to obtain the rotation matrix R and the translation matrix T of the current second disc calibration board relative to the camera coordinate system.

[0103] S508, extract the point P on the laser center line by the laser line extraction algorithm, and convert P to the point coordinates under the vertical image plane. Since the laser line hits the object and diverges, it is not a theoretical laser line, so the laser line extraction algorithm is used to extract the laser center line from the laser line presented on the object, that is, the theoretical laser line without any divergence.

[0104] S509, the 3D coordinates of each point on the laser line can be obtained according to R, T and C through the normalized transformation relationship between the camera coordinates and the pixel coordinates.

[0105] S510, the 3D coordinates of each point on the laser center line at each position are calculated repeatedly in S506-S509, and then the laser plane fitting is performed through the SVD method to obtain the laser plane equation, which is represented by the plane parameter coeff=(a, b, c, d).

[0106] S511, the laser plane obtained in step S510 is in the camera coordinate system, and needs to be rotated: the laser plane is rotated to the XOZ plane of the world coordinate system by using the Rodrigues transformation, and the intersection point of the laser plane and the Z axis is moved to the center position of the coordinate system to obtain the rotation and translation vectors rot_plane and t_plane, at this time the Y coordinate values of all the 3D coordinates of the points on the laser plane are 0.

[0107] S512, in actual 3D reconstruction, the points on the laser center line with the same X coordinate in the pixel coordinates before rotation in step S511 have the same Z axis coordinates in the world coordinate system after rotation. In actual operation, after the rotation and translation operation in the above step, it cannot be guaranteed that the same X coordinate in the pixel coordinates before rotation is the same Z axis coordinate in the world coordinate system after rotation. If this problem cannot be solved, the calibration accuracy of the calibration method of the present application cannot be guaranteed. To solve this problem, a horizontal black line graph is made in this embodiment, which has only one pixel in the horizontal direction, that is, the X coordinates are all the same. The 3D coordinates of the black line points are reconstructed through the coordinate transformation formula using the C, D, A, coeff, rot_plane and t_plane obtained above, the angle y_rot between the black line and the X axis is calculated according to the X and Z coordinates, and then the laser plane is rotated again around the Y axis by y_rot degrees to obtain the final rotation and translation vectors (rot_vec, t_vec). This method can well make the points on the laser center line with the same X coordinate in the pixel coordinates before rotation have the same Z axis coordinates in the world coordinate system after rotation, and has good effect in actual use.

[0108] Up to now, the laser plane calibration is completed.

[0109] In the above step 600, the adjustment of the three-axis fine adjustment platform 5 is involved, and the purpose is that when the grid plate 3 laser line graph is shot later, the laser lines are always on the same position on the grid plate 3 between the parallel laser lines, and when the laser 3D camera 6 is far away and close to the line, the laser lines are always on the same position on the grid plate 3. The specific implementation steps and methods are as follows:

[0110] S601, adjust the 1-axis of the three-axis fine adjustment platform 5: extract the coordinates of the center line of the laser line on the grid calibration plate, and obtain the 3D coordinates (X, Y, Z) thereof through the calibration parameters. At this time, the Z coordinate can be different, and the grid plate 3 has high flatness. Our goal is to adjust the Z coordinate of each point on the laser line to be the same. Therefore, a region is found at each end of the laser line, the Z mean values Z1 and Z2 are calculated, and the X coordinate centers X1 and X2 of the two regions are calculated. At this time, the offset angle can be calculated , the 1-axis motor is controlled to rotate by the angle, and the above process is repeated until the difference between Z1 and Z2 is small enough, and the 1-axis adjustment is completed.

[0111] S602, find a grid reference line above the laser line: the laser line needs to be found first, and then a region ROI is intercepted upward. For the ROI, the gray scale needs to be stretched first, so that the upper and lower gray scales are basically the same. Then, the entire ROI is inverted. Then, a cross kernel with a size of b*b is convolved, and the gray scale value is normalized to 0-255. At this time, the grid plate 3 black line intersection region is relatively bright, which can be extracted through threshold value. Then, straight line fitting is performed on all the intersection points extracted at present to obtain the grid reference line above the laser line.

[0112] S603, adjust the 2-axis: at this time, the slope k1 of the laser line and the slope k2 of the grid reference line are obtained. According to the following formula

[0113]

[0114] the angle by which the 2-axis needs to be rotated can be obtained. The above process is repeated until the slope difference between the laser line and the grid reference line is small enough, and the 2-axis adjustment is completed.

[0115] S604, adjust the 3-axis: the goal is to move the grid plate 3 forward and backward to keep the distance between the laser line and the grid reference line above unchanged. However, when the grid plate 3 moves away from and approaches the line laser 3D camera 6, even if the distance between the laser line and the grid line does not change, the pixel distance in the two images will change. Therefore, the pixel distance between the two grid lines is found through horizontal projection, the pixel distances d1 and d2 when moving away from and approaching the camera are obtained, the vertical change ratio d1 / d2 is calculated, and then the pixel distance between the two lines when approaching the camera is multiplied by the ratio to obtain the distances s1 and s2 between the two lines when moving away from and approaching the camera by a distance d. At this time, the offset angle can be calculated , the 3-axis is controlled to rotate by the angle, until the difference between the distances of the two lines is small enough, and the 3-axis adjustment is completed.

[0116] At this time, the 1-axis, 2-axis and 3-axis of the three-axis fine adjustment platform 5 are adjusted.

[0117] In step 800, the calibration parameters need to be corrected again according to the three-dimensional coordinates of the laser lines on the grid plate 3, and the specific correction steps are as follows:

[0118] S801, the coordinates of each point on the laser center line in the laser line image L3 are obtained by the laser center line extraction algorithm, and the three-dimensional coordinates (X, Y, Z) of each point on the laser center line are obtained by using the calibration parameters C, A, D, coeff, rot_vec, t_vec through the coordinate transformation formula, and it is known from step S511 that Y=0; it is known from step 700 that 2k+1 positions are sampled in the Y-axis direction from the -z position to the +z position, wherein k positions are sampled before and after the 0 position, the picture at the 0 position is selected, the 3D coordinates of the laser center line are calculated, and m points are randomly selected in the X-axis direction, and the average value of the Z coordinates within 2mm left and right of each point is calculated .

[0119] S802, select the laser image at position i, and obtain , the target function to be corrected can be obtained at this time:

[0120]

[0121] wherein represents the y coordinate of the jth sampling point of the laser image at the ith position, is a adjustable proportional coefficient, in order to make the Y coordinate not equal to 0 in the 3D coordinates calculated by using the corrected parameters, therefore, a Y coordinate constraint is needed for the result of each iteration. After obtaining the target function, the calibration parameters C, A, D, coeff, rot_vec, t_vec are corrected using the target function, and the least square method is used to correct the calibration parameters in the embodiment, and the corrected calibration parameters are obtained.

[0122] At this point, the correction of the calibration parameters by using the 3D coordinates of the laser lines on the grid plate 3 is completed.

[0123] The above correction process verifies the coordinate error of the reconstructed coordinates by using the obtained calibration parameters, and then an objective function is constructed by the error to correct the calibration parameters inversely. Different from the optimization technology principle in the background art, the correction in the present application is a direct correction of the error, and the target is to eliminate the error to the maximum extent, and better combined with the hardware, under the high-precision assistance of the laser interferometer 8, the appropriate calibration result is obtained.

[0124] The above steps can quickly complete the calibration process and obtain the calibration result by connecting the host computer and the electric control system of the calibration device, avoiding the tedious process of manual calibration, and the calibration result is more accurate.

[0125] The application further discloses a computer readable storage medium, and at least one program data is stored in the computer readable storage medium, and the program data is used for realizing the automatic calibration method of the line laser 3D camera system.

[0126] The technical features of the above embodiments can be combined in any manner, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but as long as the combinations of the technical features do not contradict, they should be considered as the scope of the present application.

[0127] The above embodiments are only used to illustrate the technical solutions of the present application, rather than limiting them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can still be modified, or some technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A full-automatic calibration method for a line laser 3D camera system, characterized in that, The method comprises the following steps: Obtain a first disc calibration plate image of different poses of a first disc calibration plate, and calibrate a line laser 3D camera by using the first disc calibration plate image to obtain intrinsic parameters, distortion coefficients and a sensor tilt angle; Obtain a second disc calibration plate image and a laser line image L2 of different distances between a second disc calibration plate and the line laser 3D camera, and perform laser plane calibration by using the laser line image L2 and the second disc calibration plate image to obtain laser plane parameters, a rotation vector and a translation vector of the laser plane relative to a camera coordinate system; The calibration parameters include intrinsic parameters, distortion coefficients, a sensor tilt angle, laser plane parameters, a rotation vector and a translation vector of the laser plane relative to the camera coordinate system; Obtain a laser line image L3 of different distances between a grid plate and the line laser 3D camera, and verify the error of the three-dimensional coordinates reconstructed by using the calibration parameters by using the laser line image L3, and correct the calibration parameters according to the error to obtain corrected calibration parameters; The method for obtaining the laser line image L3 of different distances between the grid plate and the line laser 3D camera comprises the following steps: Adjust the relative position between the line laser 3D camera and the grid plate, so that the line laser emitted by the line laser 3D camera is parallel to the grid reference line on the grid plate, and the position of the laser line on the grid plate does not change when the grid plate is close to or far away from the line laser 3D camera; Make the grid plate close to and far away from the line laser 3D camera respectively, and simultaneously obtain a plurality of laser line images L3, and record the relative position between the grid plate and the line laser 3D camera when each laser line image L3 is obtained, wherein the laser line image L3 is a picture of the line laser emitted by the line laser 3D camera on the grid plate; The method for verifying the error of the three-dimensional coordinates reconstructed by using the calibration parameters by using the laser line image L3, and correcting the calibration parameters according to the error to obtain corrected calibration parameters comprises the following steps: Calculate the calculated value of the Z coordinate of the laser line in each image in the laser line image L3 by using the calibration parameters; The position of each image in the laser line image L3 is known, and the measured value of the Z coordinate of the laser line in each image is obtained; Compare the calculated value of the Z coordinate of the laser line with the measured value of the Z coordinate of the laser line to obtain a correction objective function; Correct the calibration parameters according to the correction objective function.

2. The method of claim 1, wherein, Further comprising: Adjust the exposure time when the line laser 3D camera captures the first disc calibration plate image of different poses by using an automatic exposure algorithm, so that the brightness of each first disc calibration plate image is consistent; The automatic exposure algorithm comprises the following steps: Pre-set a gray value range, extract the first disc calibration plate from the first disc calibration plate image by using adaptive threshold segmentation, and take the area of the first disc calibration plate except the black disc as the foreground; Calculate the average value T1 of all pixel values in the foreground, and calculate the average value T2 of the pixel points whose pixel values are greater than T1 in the foreground; If T2 is within the preset gray value range, the exposure time meets the requirements; if T2 is not within the preset gray value range, adjust the exposure time until T2 is within the preset gray value range.

3. The method of claim 1, wherein, The method for obtaining the second disc calibration plate image and the laser line image L2 of different distances between the second disc calibration plate and the line laser 3D camera comprises the following steps: Adjust the position between the line laser 3D camera and the second disc calibration board, so that the laser line is on the two rows of discs of the second disc calibration board, and the line laser 3D camera can capture the second disc calibration board; Make the second disc calibration board close to and far away from the line laser 3D camera respectively, and capture multiple second disc calibration board images and laser line images L2; Wherein, light compensation is performed when capturing the second disc calibration board image, and the light source of the light compensation has the same wavelength as the light source of the line laser 3D camera; the laser line image L2 is a picture of the laser line emitted by the line laser 3D camera on the second disc calibration board.

4. The method of claim 1, wherein, The first disc calibration board image is used to calibrate the line laser 3D camera to obtain intrinsic parameters, distortion coefficients and sensor tilt angles, including: According to the first disc calibration board image, the initial intrinsic parameters, initial distortion coefficients and initial sensor tilt angle of the line laser 3D camera are calculated using the Sham camera calibration method, and the re-projection error of each feature point of each picture is recorded; Select the first disc calibration board image with smaller re-projection error, and continue to calibrate using the Sham camera calibration method, and select the calibration result with the smallest re-projection error as the intrinsic parameters, distortion coefficients and sensor tilt angle of the line laser 3D camera.

5. The method of claim 1, wherein, The laser plane calibration is performed using the laser line image L2 and the second disc calibration board image to obtain the laser plane parameters, the rotation vector and the translation vector of the laser plane relative to the camera coordinate system, including: A world coordinate system is established with the current second disc calibration board as a reference, the X and Y axes are on the surface of the disc calibration board, and the Z axis is perpendicular to the second disc calibration board; According to the current second disc calibration board image, the rotation matrix R and the translation matrix T of the current disc calibration board relative to the line laser 3D camera coordinate system are calculated in combination with the camera intrinsic parameters and distortion coefficients; Through the normalized transformation relationship between the line laser 3D camera coordinates and the pixel coordinates, the 3D coordinates of each point on the laser line are obtained according to R, T, the intrinsic parameters; The 3D coordinates of each point on the laser line of each second disc calibration board image are calculated, and the laser plane parameters are fitted; The laser plane is first rotated to the XOZ plane of the world coordinate system by using the Rodrigues transformation to obtain the rotation vector rot_plane; and then the intersection point of the laser plane and the Z axis is first translated to the center of the world coordinate system to obtain the translation vector t_plane; The laser plane is secondly rotated so that the Z coordinates of the center points of the laser lines with the same X coordinates are the same, and the rotation vector rot_vec of the laser plane relative to the camera coordinate system is obtained according to the first rotation and the second rotation, and the translation vector t_vec of the laser plane relative to the camera coordinate system is obtained according to the first translation.

6. A full-automatic calibration device for a line laser 3D camera system, characterized in that, It includes: The first disc calibration board, the pose of which is adjustable, is used to capture the first disc calibration board images in different poses by the line laser 3D camera; The second disc calibration board, the relative position of which to the line laser 3D camera is adjustable, is used to capture the second disc calibration board images and the laser line images L2 in different positions by the line laser 3D camera; The grid plate, the relative position of which to the line laser 3D camera is adjustable, is used to capture the laser line images L3 in different positions by the line laser 3D camera; The host computer is used to calculate calibration parameters, including intrinsic parameters, distortion coefficients, sensor tilt angles, laser plane parameters, and rotation vectors and translation vectors of the laser plane relative to the camera coordinate system. The host computer uses the first disc calibration plate icon to calibrate the linear laser 3D camera to obtain intrinsic parameters, distortion coefficients, and sensor tilt angles; uses the laser line diagram L2 and the second disc calibration plate to perform laser plane calibration to obtain laser plane parameters, rotation vectors, and translation vectors of the laser plane relative to the camera coordinate system; uses the laser line diagram L3 to verify the error of the three-dimensional coordinates reconstructed using the calibration parameters, and corrects the calibration parameters according to the error to obtain corrected calibration parameters. The laser line diagram L3 at different positions is obtained in the following way: Adjust the relative position between the linear laser 3D camera and the grid plate so that the linear laser emitted by the linear laser 3D camera is parallel to the grid reference line on the grid plate, and the position of the laser line on the grid plate does not change when the grid plate is close to or far away from the linear laser 3D camera; Make the grid plate close to and far away from the linear laser 3D camera respectively, and simultaneously acquire multiple laser line diagrams L3, record the relative position between the grid plate and the linear laser 3D camera when each laser line diagram L3 is acquired, and the laser line diagram L3 is the picture of the linear laser emitted by the linear laser 3D camera on the grid plate; Use the laser line diagram L3 to verify the error of the three-dimensional coordinates reconstructed using the calibration parameters, and correct the calibration parameters according to the error to obtain corrected calibration parameters, including: Calculate the calculated value of the Z coordinate of the laser line in each diagram in the laser line diagram L3 using the calibration parameters; The position of each diagram in the laser line diagram L3 is known, and the measured value of the Z coordinate of the laser line in each diagram is obtained; Compare the calculated value of the Z coordinate of the laser line with the measured value of the Z coordinate of the laser line to obtain a correction objective function; Correct the calibration parameters according to the correction objective function.

7. The apparatus according to claim 6, wherein, Further comprising: A laser interferometer for high-precision measurement of the displacement of the grid plate between different positions of the laser line diagram L3; A fill light, which has the same wavelength as the light source emitted by the linear laser 3D camera, is used to supplement light when acquiring the first disc calibration plate diagram and the second disc calibration plate diagram; A six-axis mechanical arm is used to fix the first disc calibration plate, and the six-axis mechanical arm drives the first disc calibration plate to move, so that the linear laser 3D camera can shoot the first disc calibration plate diagram at different poses; A three-axis fine adjustment platform is used to fix the linear laser 3D camera and adjust the spatial position of the linear laser 3D camera, including: Before shooting the laser line diagram L3 at different distances, adjust the first axis of the three-axis fine adjustment platform so that the laser line is horizontal in the image; Adjust the second axis of the three-axis fine adjustment platform so that the slope of the laser line is the same as that of the grid reference line, and the laser line is parallel to the grid reference line; Adjust the third axis of the three-axis fine adjustment platform so that the distance between the laser line and the grid reference line remains unchanged regardless of whether the grid plate is moving away from or approaching the camera.

8. A computer-readable storage medium, characterized in that, The computer program, when executed by a processor, causes the processor to perform the full-automatic calibration method of the linear laser 3D camera system according to any one of claims 1 to 5.

Images