Evaluation method and calibration method for optical plane calibration error of line structure optical sensor

Abstract

The invention discloses a line structured light sensor light plane calibration error evaluation method, and the calibration process of a light plane comprises the steps: obtaining a target image of which the surface is projected with a laser light bar, and extracting the pixel coordinates of the central point of the laser light bar; respectively converting the central point of the light bar and the target plane to a camera coordinate system by using calibration parameters of the camera; under a camera coordinate system, connecting the optical center with the central point of each light strip, and recording the intersection point of the optical center and the target plane as a reference point; fitting a light plane by using the reference point to complete calibration; the error evaluation method comprises the following steps: S1, marking intersection points of connecting lines between the optical center of the camera and each datum point and a light plane as theoretical points; and S2, calculating a deviation value between each theoretical point and the corresponding reference point, or calculating a deviation value between a back projection point of each theoretical point on the target image and a light strip center point in the target image, and taking the deviation value as an error of light plane calibration. The method is simple and reliable in evaluation process, and can improve the precision of light plane calibration.

Description

Technical Field

[0001] The present invention relates to the field of visual detection technology, and in particular to a light plane calibration error evaluation method and a calibration method for a line structured light sensor. Background Art

[0002] Line structured light sensor, a precision system that integrates line structured light projector, lens and camera, is increasingly used in the fields of 3D measurement, object recognition and morphology reconstruction. However, before this high-precision sensor is put into use, it must undergo a detailed and precise calibration process to ensure the accuracy and reliability of its measurement. The calibration work not only involves the calibration of the camera's intrinsic parameter matrix and distortion matrix, but also includes the precise determination of the light plane parameter matrix. This step is crucial to ensure that the sensor can accurately capture and interpret the line structured light information in space.

[0003] Among the many methods of sensor camera calibration, whether it is the traditional camera calibration method, the active vision camera calibration method, or the more flexible camera self-calibration method, the purpose is to obtain the most accurate calibration parameters to reduce the sensor measurement error and improve the sensor measurement accuracy. However, even with the assistance of these advanced calibration methods, due to various factors such as equipment accuracy, environmental interference, and improper operation, the calibration results of the light plane parameter matrix may still have certain errors.

[0004] The existence of this light plane calibration error will undoubtedly have a direct and significant impact on the measurement accuracy of the sensor. In the process of sensor accuracy verification, if the calibration error of the light plane cannot be accurately evaluated, the source of the sensor accuracy error cannot be clearly identified, and thus the error correction cannot be carried out in a targeted manner to improve the measurement accuracy and stability of the line structured light sensor. Therefore, how to evaluate the light plane calibration error is particularly important, but there is currently no effective method for evaluating the light plane calibration error. Summary of the invention

[0005] In order to solve the above technical problems, the present invention provides a light plane calibration error evaluation method and a calibration method for a line structured light sensor, which can intuitively evaluate the error of the light plane calibration and optimize the light plane calibration using the error evaluation result, thereby effectively improving the accuracy of the light plane calibration and providing support for subsequent sensor accuracy verification.

[0006] To this end, the technical solution is as follows:

[0007] A method for evaluating the calibration error of a light plane of a line structured light sensor, wherein the calibration process of the light plane is as follows: ① a target image with a laser light strip projected on the surface is obtained through a camera, and the pixel coordinates of the center point of the laser light strip are extracted on the target image; ② the center point of the light strip and the target plane are respectively converted to the camera coordinate system using the calibration parameters of the camera, and the coordinates of the center point of the light strip in the camera coordinate system and the target plane F in the camera coordinate system are obtained; ③ in the camera coordinate system, the optical center of the camera is connected with the center point of each light strip, and the intersection point with the target plane F is recorded as the reference point; ④ the light plane is fitted with the reference point to obtain the light plane I, and the calibration is completed;

[0008] The error evaluation method includes the following steps:

[0009] S1, connect the optical center of the camera with each reference point, and record the intersection points of the lines with the optical plane I as the theoretical points of each reference point;

[0010] S2. Calculate the deviation between each theoretical point and its corresponding reference point, or calculate the deviation between the back-projection point of each theoretical point on the target image and the center point of the light strip in the target image, and use it as the error of light plane calibration.

[0011] Furthermore, the method for calculating the deviation value between each theoretical point and its corresponding reference point in S2 is: respectively calculating the straight-line distance between each theoretical point and its corresponding reference point, finding the average of all straight-line distances, and taking the average as the error.

[0012] Furthermore, the method for S2 to calculate the deviation value between the back-projection point of each theoretical point on the target image and the center point of the light strip in the target image is: using a reprojection algorithm to calculate the coordinates of the back-projection point of each theoretical point on the target image; and calculating the deviation value between each back-projection point and the corresponding center point of the light strip, and obtaining the root mean square of all deviation values, and taking the root mean square as the error.

[0013] Further, the target position is adjusted, and steps ① to ③ are repeated to obtain multiple groups of reference points, and the light plane is fitted with the multiple groups of reference points.

[0014] Furthermore, when adjusting the target position, the target is only translated along the camera optical axis.

[0015] Furthermore, the target position is adjusted no less than 6 times.

[0016] Furthermore, the target is a planar target with a known positional relationship between adjacent feature points or a three-dimensional target with a planar relationship between adjacent feature points.

[0017] Furthermore, the method of acquiring the target image with the laser light strip projected on the surface by the camera is to simultaneously acquire the image containing the laser light strip and the target feature point, or to acquire the laser light strip and the target feature point image separately.

[0018] The present invention also discloses a method for assisting light plane calibration using the conclusions of the light plane calibration error evaluation method of a line structured light sensor. Based on the error value obtained by S2, a deviation threshold is set, and reference points whose deviation values ​​exceed the deviation threshold are deleted. The remaining reference points are used as reference points for fitting light plane I to perform light plane fitting.

[0019] This method provides a feasible light plane calibration error evaluation method, which uses the reference points in the light plane calibration process as the basis to obtain its theoretical points on the fitted light plane, and uses the reference points and theoretical points as the main body of calibration error evaluation, calculates the deviation of the theoretical points, and obtains the error value of the light plane calibration; this process avoids the influence of noise in the process of obtaining the error evaluation basis, and the acquisition of the error evaluation basis coincides with the process of light plane calibration, which simplifies the error evaluation process; at the same time, the calibration accuracy of the light plane is clarified before the sensor accuracy verification is performed, which eliminates the blind spot for the sensor accuracy verification and provides support;

[0020] In addition, this method also provides a method for optimizing light plane calibration using this error evaluation method. By processing the obtained deviation values, points with larger errors are screened out and deleted from the reference points. The remaining reference points are used for light plane fitting, which effectively improves the accuracy of light plane calibration and thereby improves the measurement accuracy of the line structured light sensor. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 It is a schematic diagram of the process flow of the light plane calibration error evaluation method of the present invention;

[0022] Figure 2 It is a schematic diagram of the principle of obtaining theoretical points through reference points;

[0023] Figure 3 It is a schematic diagram of an image including both the laser light strip and the target feature points;

[0024] Figure 4a It is a schematic diagram of the target image when the laser light strip and the target feature point image are collected respectively;

[0025] Figure 4b Schematic diagram of the laser light strip image when collecting the laser light strip and target feature point images respectively. DETAILED DESCRIPTION

[0026] The technical solution of the present invention is described in detail below in conjunction with the accompanying drawings and embodiments.

[0027] A method for evaluating the light plane calibration error of a line structured light sensor, such as Figure 1As shown in the figure, the calibration process of the light plane is as follows: ① The target image with laser light strips projected on the surface is obtained through the camera, and the pixel coordinates of the center point of the laser light strips are extracted on the target image; ② The center point of the light strips and the target plane are converted to the camera coordinate system using the calibration parameters of the camera, respectively, to obtain the coordinates of the center point of the light strips in the camera coordinate system and the target plane F in the camera coordinate system; ③ In the camera coordinate system, the optical center of the camera is connected with the center point of each light strip, and the intersection point with the target plane F is recorded as the reference point; ④ The light plane is fitted with the reference point to obtain the light plane I, and the calibration is completed;

[0028] The error evaluation method includes the following steps:

[0029] S1, connect the optical center of the camera with each reference point, and record the intersection points of the lines with the optical plane I as the theoretical points of each reference point; Figure 2 As shown;

[0030] S2. Calculate the deviation between each theoretical point and its corresponding reference point, or calculate the deviation between the back-projection point of each theoretical point on the target image and the center point of the light strip in the target image, and use it as the error of light plane calibration.

[0031] In specific implementation, the method of calculating the deviation value between each theoretical point and its corresponding reference point in S2 is: respectively calculating the straight-line distance between each theoretical point and its corresponding reference point, finding the average of all straight-line distances, and taking the average as the error.

[0032] As an optional implementation, S2 calculates the deviation value between the back-projection point of each theoretical point on the target image and the center point of the light strip in the target image by: using a reprojection algorithm to calculate the coordinates of the back-projection point of each theoretical point on the target image; and calculating the deviation value between each back-projection point and the corresponding center point of the light strip, and obtaining the root mean square of all deviation values, and taking the root mean square as the error.

[0033] In order to improve the accuracy of light plane fitting, adjust the target posture, repeat steps ① to ③, obtain multiple sets of reference points, fit the light plane with multiple sets of reference points, and obtain light plane I. The plane fitting can be achieved by fitting the reference points using methods such as least squares, optimization library ceres, optimization library g2o, or optimization library levmar.

[0034] In specific implementation, the adjustment of the target position only translates the target along the camera optical axis. The target position is adjusted no less than 6 times, preferably 6 to 10 times.

[0035] Specifically, the target is a plane target with a known positional relationship between adjacent feature points or a three-dimensional target with a plane relationship between adjacent feature points.

[0036] Furthermore, the method of acquiring the target image with the laser light strip projected on the surface by the camera can be to simultaneously capture the image containing the laser light strip and the target feature point, such as Figure 3 As shown; this method can obtain the laser light strip and target image at the same time through one image acquisition, saving time, but before acquiring the image, it is necessary to consider both the brightness of the target feature point and the brightness of the laser light strip;

[0037] It is also possible to collect laser light strips and target feature point images separately, such as Figure 4a , 4b As shown; this method does not need to take into account the brightness of the target feature points and the brightness of the laser light strip at the same time, but it needs to collect the laser light strip and target images separately, which increases the number of image collection times.

[0038] As an optional method for converting the target plane to the camera coordinate system, the characteristic points on the target are used to obtain the pose relationship T between the target coordinate system and the camera coordinate system according to the PNP algorithm; the coordinates of the characteristic points in the target coordinate system are converted to the camera coordinate system through the pose relationship T; the coordinates of the characteristic points in the camera coordinate system are used to fit the target plane F of the target in the camera coordinate system through the least squares method. Those skilled in the art may also select other existing conversion methods in combination with achieving the purpose.

[0039] As an optional method for converting the center point of the light bar to the camera coordinate system, the pixel coordinates of the center point of the light bar are normalized to the camera coordinate system using the camera intrinsic parameters to obtain the coordinates of the center point of the light bar in the camera coordinate system. Those skilled in the art may also select other existing conversion methods in combination with achieving the purpose.

[0040] The error evaluation method provided by this scheme is based on the reference point during light plane calibration. When the calibrated light plane is completed, the theoretical point corresponding to the reference point is obtained, and the light plane calibration error is obtained by calculating the deviation of the theoretical point. The theoretical point acquisition for the error calculation is based on the reference point of the light plane calibration, that is, the error acquisition process partially overlaps with the calibration process, which simplifies the process of separate error evaluation, saves resources and improves efficiency. At the same time, the calibration accuracy of the light plane is clarified before the sensor accuracy verification, which eliminates the blind spot for the sensor accuracy verification and provides support. When the error evaluation method of this scheme is implemented, the reference point and the theoretical point can be marked near the light plane I at the same time, forming a more intuitive fitting error field, so that the reference point with a larger error can be clearly seen, and then according to the error distribution, the mapping of the light plane calibration process can be better guided.

[0041] The present invention also discloses a method for assisting light plane calibration using the conclusions of the light plane calibration error evaluation method of a line structured light sensor. Based on the error value obtained by S2, a deviation threshold is set, and reference points whose deviation values ​​exceed the deviation threshold are deleted. The remaining reference points are used as reference points for fitting light plane I to perform light plane fitting.

[0042] In specific implementation, according to the characteristics of Gaussian distribution, the deviation threshold can be set to ±3σ, where σ is the standard deviation of all error values. That is, the mean and standard deviation σ are calculated for all deviations, and the theoretical points corresponding to the deviations greater than ±3σ from the mean are marked as error points, and the reference points corresponding to the error points are recorded as error reference points. When fitting the light plane, the error reference points are deleted from all reference points, and the remaining reference points are used as reference points for fitting the light plane I.

[0043] This method processes the obtained deviation values, screens out points with larger errors, deletes them from the reference points, and uses the remaining reference points for light plane fitting, which effectively improves the accuracy of light plane calibration and thus improves the measurement accuracy of the line structured light sensor.

[0044] In order to verify the application of this method in the evaluation of light plane calibration error, the light plane calibration of 8 different sensors was carried out and the mean deviation between each theoretical point and the corresponding reference point was calculated (i.e., calibration error value), as shown in Table 1 below:

[0045] sensor Calibration error value / mm Sensor 1 0.03654 Sensor 2 0.07181 Sensor 3 0.04256 Sensor 4 0.04469 Sensor 5 0.03746 Sensor 6 0.07171 Sensor 7 0.04180 Sensor 8 0.04483

[0046] Table 1

[0047] The foregoing description of the specific exemplary embodiments of the present invention is presented for the purpose of illustration and description. The foregoing description is not intended to be exhaustive, nor is it intended to limit the present invention to the precise form disclosed, and it is apparent that many changes and variations are possible in light of the above teachings. The exemplary embodiments are selected and described to explain the specific principles of the present invention and its practical application, so that other technicians in the field can realize and utilize the various exemplary embodiments of the present invention and its different selected forms and modified forms. The scope of the present invention is intended to be limited by the appended claims and their equivalents.

Claims

1. A method for evaluating the light plane calibration error of a line structured light sensor, characterized by: The calibration process of the light plane is as follows: ① The target image with the laser light strip projected on the surface is obtained through the camera, and the pixel coordinates of the center point of the laser light strip are extracted on the target image; ② The center point of the light strip and the target plane are respectively converted to the camera coordinate system using the calibration parameters of the camera, and the coordinates of the center point of the light strip in the camera coordinate system and the target plane F in the camera coordinate system are obtained; ③ In the camera coordinate system, the optical center of the camera is connected with the center point of each light strip, and the intersection point with the target plane F is recorded as the reference point; ④ The light plane is fitted with the reference point to obtain the light plane I, and the calibration is completed; The error evaluation method includes the following steps: S1, connect the optical center of the camera with each reference point, and record the intersection points of the lines with the optical plane I as the theoretical points of each reference point; S2. Calculate the deviation between each theoretical point and its corresponding reference point, or calculate the deviation between the back-projection point of each theoretical point on the target image and the center point of the light strip in the target image, and use it as the error of light plane calibration.

2. The method for evaluating light plane calibration error of a line structured light sensor according to claim 1, characterized in that: The method of calculating the deviation value between each theoretical point and its corresponding reference point in S2 is: respectively calculating the straight-line distance between each theoretical point and its corresponding reference point, finding the average of all straight-line distances, and taking the average as the error.

3. The method for evaluating light plane calibration error of a line structured light sensor according to claim 1, characterized in that: The method for S2 to calculate the deviation value between the back-projection point of each theoretical point on the target image and the center point of the light strip in the target image is: using a reprojection algorithm to calculate the coordinates of the back-projection point of each theoretical point on the target image; and calculating the deviation value between each back-projection point and the corresponding center point of the light strip, and obtaining the root mean square of all deviation values, and taking the root mean square as the error.

4. The method for evaluating light plane calibration error of a line structured light sensor according to claim 1, characterized in that: Adjust the target posture, repeat steps ① to ③, obtain multiple sets of reference points, and fit the light plane with the multiple sets of reference points.

5. The method for evaluating light plane calibration error of a line structured light sensor according to claim 4, characterized in that: When adjusting the target position, the target is only translated along the camera optical axis.

6. The method for evaluating light plane calibration error of a line structured light sensor according to claim 4, characterized in that: The target position is adjusted at least 6 times.

7. The method for evaluating light plane calibration error of a line structured light sensor according to claim 1, characterized in that: The target is a plane target with a known positional relationship between adjacent feature points or a three-dimensional target with a plane relationship between adjacent feature points.

8. The method for evaluating light plane calibration error of a line structured light sensor according to claim 1, characterized in that: The method of acquiring the target image with the laser light strip projected on the surface by the camera is to simultaneously acquire the image containing the laser light strip and the target feature point, or to separately acquire the image of the laser light strip and the target feature point.

9. A method for assisting light plane calibration using the conclusion of any of the light plane calibration error evaluation methods for line structured light sensors as claimed in claims 1 to 8, characterized in that: Based on the error value obtained by S2, a deviation threshold is set, and the reference points whose deviation values ​​exceed the deviation threshold are deleted. The remaining reference points are used as the reference points of the fitting light plane I to perform light plane fitting.

Images