Scheimpflug principle-based tilt-shift camera high-precision calibration method

Abstract

The invention discloses a tilt-shift camera high-precision calibration method based on a Scheimpflug principle. The method comprises the following steps: firstly, adjusting a tilt angle of a tilt shift to determine an optimal clear imaging area meeting a Scheimpflug condition, and constructing a micron-sized moving reference by utilizing a synchronous triggering mechanism of a sliding table and a grating ruler; in the internal reference calibration stage, a method for automatically positioning a real distortion center based on a point-line distance and a (LinePointsDis) index is provided, and a 14-parameter model containing thin prism distortion is adopted to optimize and solve the internal reference; in the external parameter calibration stage, a sliding table is controlled to drive a two-dimensional target to move in a fixed step length mode, a virtual three-dimensional target containing multi-layer depth information is constructed, and clear area angular points are screened to be combined with a PnP algorithm to solve the pose; in the light plane calibration stage, intersection points of laser stripe center lines and checkerboard grid lines are extracted, and geometric constraints of the grid lines are utilized to suppress noise and fit a light plane equation. According to the method, the problems of serious shift-axis imaging distortion center offset, insufficient field depth direction constraint, laser speckle noise interference and the like are effectively solved, and the calibration precision and robustness of the linear structure optical sensing system are remarkably improved.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and optical measurement technology, and in particular relates to a high-precision calibration method for tilt-shift cameras based on the Scheimpflug principle, applicable to line structured light 3D reconstruction systems. Background Technology

[0002] Structured light 3D measurement technology is widely used in industrial inspection and reverse engineering. To overcome the problem of insufficient depth of field in macro and large tilt angle scenes, a tilt-shift camera system based on the Scheimpflug principle is often used. By adjusting the optical path, the object plane, lens plane, and imaging plane intersect in a straight line, thereby expanding the depth of field. This system requires precise calibration to establish coordinate mapping relationships and correct optical distortion. However, existing calibration methods mostly use the traditional pinhole model, assuming that the distortion center coincides with the principal point and only considering radial and tangential distortion, ignoring the inherent principal point offset and thin prism distortion of the tilt-shift system, resulting in decreased accuracy of edge field-of-view correction. In addition, when calibrating extrinsic parameters and the optical plane, it is usually necessary to rely on a handheld 2D target, which has high sampling randomness and lacks strong geometric constraints in the depth direction. In light stripe extraction, direct extraction methods are mostly used, which are easily affected by environmental noise and lack geometric constraints based on target features, limiting the robustness and engineering practicality of the calibration. Therefore, there is an urgent need to develop a high-precision, robust, and easy-to-implement tilt-shift camera calibration method based on the Scheimpflug principle to meet the needs of high-precision industrial measurement. Summary of the Invention

[0003] This invention aims to address the shortcomings of existing tilt-shift camera calibration techniques by providing a high-precision, robust calibration method for tilt-shift cameras based on the Scheimpflug principle. The technical solution adopted in this invention is: a high-precision calibration method for tilt-shift cameras based on the Scheimpflug principle. This method includes: S10. Imaging Environment Construction: Adjust the angle between the lens plane and the imaging plane of the tilt-shift camera to meet the Scheimpflug condition to determine the optimal sharp imaging area; configure the synchronous triggering mechanism of the slide and the grating ruler to construct a moving reference with micron-level precision.

[0004] S20. Camera Intrinsic Calibration: Acquire chessboard images in different poses, automatically calculate the distortion center based on the point-to-line distance and (LinePointsDis) index, and optimize the solution of the camera intrinsic matrix and distortion parameters using a 14-parameter model that includes thin prism distortion.

[0005] S30. Camera extrinsic parameter calibration: A virtual 3D target is constructed by using a slide table to move the target in fixed steps. The pose relationship between the camera coordinate system and the world coordinate system is solved by combining the depth-constrained PnP algorithm.

[0006] S40. Laser plane calibration: Extract the intersection points of the laser stripe center line and the horizontal grid lines of the checkerboard pattern, use the geometric constraints of the grid lines to improve data accuracy, and then fit the laser plane equation.

[0007] Furthermore, S10 includes the following steps: S11. Place the calibration target parallel to the image sensor, and observe the changes in image sharpness by translating the target along the optical axis. Determine the front and rear boundary positions where the entire image is sharp. This area is the effective calibration space.

[0008] S12. Establish a sliding stage trigger synchronization mechanism based on grating ruler position feedback. The calibration target is vertically fixed on the precision electric sliding stage, and a trigger pulse is set so that the camera automatically triggers a data acquisition every 2,000 mm of movement of the sliding stage, ensuring the equidistant nature of spatial sampling.

[0009] Furthermore, S20 includes the following steps: S21. Acquire multiple checkerboard images from different angles within the clear imaging area and extract sub-pixel corner points.

[0010] S22. Calculate the distortion center. For each row and column corner point in the checkerboard image, fit a straight line and calculate the sum of the Euclidean distances from all corner points to the corresponding fitted lines, defined as LinePointsDis:

[0011] In the formula, These are the coordinates of the corner point. To fit the parameters of the straight line, iterate through all rows and columns, selecting... The smallest row and Find the smallest column and calculate the intersection of the two lines. This is used as the true distortion center of the camera.

[0012] S23. Solve for intrinsic parameters and distortion coefficients. A 14-parameter model incorporating radial distortion, tangential distortion, and thin prism distortion is used to calibrate the camera. This distortion model is described below:

[0013] In the formula, To normalize image coordinates, ; The radial distortion coefficient is... The tangential distortion coefficient is... The distortion coefficient is that of a thin prism. This higher-order model can effectively compensate for the optical asymmetry distortion caused by axis-shift imaging.

[0014] And establish the objective function to minimize the reprojection error:

[0015] During the optimization process, a fixed principal point strategy (CALIB_FIX_PRINCIPAL_POINT) is adopted to determine the distortion center calculated in S22. As fixed parameters, only the focal length and distortion coefficients are iteratively optimized to obtain a high-precision intrinsic parameter matrix. .

[0016] Furthermore, S30 includes the following steps: S31. Virtual 3D Target Construction. The control slide moves the checkerboard target along a direction perpendicular to the target plane. Next (in this embodiment) ), constructing in space with A virtual 3D target with layer depth information.

[0017] S32. Pose Calculation. Considering the unique imaging optical path of a tilt-shift camera, its depth distribution across the entire scene is uneven (i.e., the area near the focal plane satisfying the Scheimpflug condition is sharp, gradually blurring towards the sides). To ensure calibration accuracy, this step introduces a sharpness-based region filtering mechanism. From the extracted corner data, only specific columns of corners located within the optimal sharpness imaging area are retained. Subsequently, the intrinsic parameters obtained in S20 are used to perform distortion correction on these filtered high-precision corners. This is combined with the known parameters provided by the slide. Based on axis depth information, construct 3D-2D point pairs. Use the Perspective-n-Point (PnP) algorithm to solve for the rotation matrix of the camera coordinate system relative to the world coordinate system. Translation vector .

[0018] Furthermore, S40 includes the following steps: S41. Intersection Point Extraction. Acquire an image of the laser stripes projected onto the target. Extract the sub-pixel center lines of the laser stripes using skeleton thinning and grayscale centroid methods; fit the horizontal grid lines in the checkerboard image using the RANSAC algorithm; calculate the intersection points of the laser center lines with each horizontal grid line.

[0019] S42. Spatial coordinate reconstruction. Using the extrinsic parameters calculated in S30. By projecting the aforementioned "intersection points" on the image plane back onto the camera coordinate system using internal parameters, a three-dimensional scatter cloud on the laser light plane is obtained. .

[0020] S43. Plane fitting. For point clouds. By performing least-squares plane fitting (SVD decomposition), the laser light plane equation is obtained:

[0021] S44. Error Assessment. Calculate the root mean square error (RMS) from all observation points to the fitted plane. If the error is less than a preset threshold (e.g., 0.05 mm), the calibration is complete; otherwise, return to S10 to check the hardware alignment.

[0022] The tilt-shift camera calibration method of this invention, based on the Scheimpflug principle, is scientific, rigorous, and easy to implement. It only requires constructing a virtual 3D target using a precision slide and combining it with an intersection constraint algorithm. By following a standardized operating procedure, it can accurately calculate the camera's true distortion center, high-order intrinsic parameters including the thin prism model, extrinsic parameters with strong depth constraints, and the laser light plane equation with excellent noise resistance. This calibration method boasts high accuracy and robustness, effectively solving the problems of inaccurate distortion correction and unstable extrinsic parameter calculation in traditional methods under tilt-shift imaging conditions. It provides a more reliable and practical solution for industrial-grade high-precision 3D inspection, demonstrating significant engineering value. Attached Figure Description

[0023] Figure 1 This is an overall flowchart of the calibration method of the present invention; Figure 2 This is a schematic diagram of the principle of tilt-shift imaging; Figure 3 Extract the image of a corner point at a location calibrated by the camera's intrinsic parameters; Figure 4 A schematic diagram showing the location of the distortion center; Figure 5 A schematic diagram illustrating the construction and external parameter calibration of a virtual 3D target; Figure 6 This is a schematic diagram illustrating the effective calibration region selection based on Scheimpflug imaging characteristics.

[0024] Figure 7 A schematic diagram for extracting intersection points of the optical plane calibration. Detailed Implementation

[0025] High-precision intrinsic and extrinsic parameter calibration of tilt-shift cameras is fundamental to accurate optical plane calibration; conversely, high-precision optical plane calibration is crucial for realizing the system's three-dimensional measurement capabilities. These three elements together constitute a complete system calibration process. The following, in conjunction with the accompanying drawings, illustrates the implementation of the high-precision tilt-shift camera calibration method based on the Scheimpflug principle proposed in this invention through specific examples. Those skilled in the art can easily understand the advantages and effectiveness of this invention in solving the problem of tilt-shift distortion correction and improving the robustness of optical plane calibration from the content disclosed in this specification regarding automatic calculation of the distortion center, virtual stereo target construction, and optical plane intersection fitting. This invention can also be implemented or applied through different hardware combinations or algorithm optimizations. The details in this specification, such as the slide stepping parameters, distortion model order, and error evaluation indicators, can also be modified or changed based on different industrial measurement scenarios and accuracy requirements, without departing from the spirit of this invention.

[0026] like Figure 1 As shown in this embodiment, a high-precision calibration method for a tilt-shift camera based on the Scheimpflug principle includes the following steps: S10. Imaging Environment Construction: Adjust the angle between the lens plane and the imaging plane of the tilt-shift camera to meet the Scheimpflug condition to determine the optimal sharp imaging area; configure the synchronous triggering mechanism of the slide and the grating ruler to construct a moving reference with micron-level precision.

[0027] S20. Camera Intrinsic Calibration: Acquire chessboard images in different poses, automatically calculate the distortion center based on the point-to-line distance and (LinePointsDis) index, and optimize the solution of the camera intrinsic matrix and distortion parameters using a 14-parameter model that includes thin prism distortion.

[0028] S30. Camera extrinsic parameter calibration: A virtual 3D target is constructed by using a slide table to move the target in fixed steps. The pose relationship between the camera coordinate system and the world coordinate system is solved by combining the depth-constrained PnP algorithm.

[0029] S40. Laser plane calibration: Extract the intersection points of the laser stripe center line and the horizontal grid lines of the checkerboard pattern, use the geometric constraints of the grid lines to improve data accuracy, and then fit the laser plane equation.

[0030] Furthermore, S10 includes the following steps: S11. Place the calibration target parallel to the image sensor. By translating the target along the optical axis, observe the change in image sharpness to determine the front and rear boundary positions where the entire image is sharp. This area is the effective calibration space. Figure 2 As shown.

[0031] S12. Establish a sliding stage trigger synchronization mechanism based on grating ruler position feedback. The calibration target is vertically fixed on the precision electric sliding stage, and a trigger pulse is set so that the camera automatically triggers a data acquisition every 2,000 mm of movement of the sliding stage, ensuring the equidistant nature of spatial sampling.

[0032] Furthermore, S20 includes the following steps: S21. Acquire multiple checkerboard images from different angles within the clear imaging area. The acquired typical tilt-shift imaging checkerboard and corner point extraction results are as follows: Figure 3 As shown, corner points are clearly extracted within the depth of field.

[0033] S22. Calculate the distortion center. For example... Figure 4 As shown, for each row and column corner point in the chessboard image, a straight line is fitted, and the sum of the Euclidean distances from all corner points to the corresponding fitted lines is calculated, defined as LinePointsDis:

[0034] In the formula, These are the coordinates of the corner point. To fit the parameters of the straight line, iterate through all rows and columns, selecting... The smallest row and Find the smallest column and calculate the intersection of the two lines. This is used as the true distortion center of the camera.

[0035] S23. Solve for intrinsic parameters and distortion coefficients. A 14-parameter model incorporating radial distortion, tangential distortion, and thin prism distortion is used to calibrate the camera. This distortion model is described below:

[0036] In the formula, To normalize image coordinates, ; The radial distortion coefficient is... The tangential distortion coefficient is... The distortion coefficient is that of a thin prism. This higher-order model can effectively compensate for the optical asymmetry distortion caused by axis-shift imaging.

[0037] And establish the objective function to minimize the reprojection error:

[0038] During the optimization process, a fixed principal point strategy (CALIB_FIX_PRINCIPAL_POINT) is adopted to determine the distortion center calculated in S22. As fixed parameters, only the focal length and distortion coefficients are iteratively optimized to obtain a high-precision intrinsic parameter matrix. .

[0039] Furthermore, S30 includes the following steps: S31. Virtual 3D target construction. For example... Figure 5 As shown, the control slide moves the checkerboard target along a direction perpendicular to the target plane. Next (in this embodiment) ), constructing in space with A virtual 3D target with layer depth information.

[0040] S32. Pose Calculation. Considering the unique imaging optical path of a tilt-shift camera, its depth distribution across the entire scene is uneven (i.e., the area near the focal plane satisfying the Scheimpflug condition is sharp, gradually blurring towards the sides). To ensure calibration accuracy, this step introduces a region filtering mechanism based on sharpness. In the extracted corner data, only specific columns of corner points located within the optimal sharpness imaging area are retained (in this embodiment, the 4th and 5th columns of corner points in the center of the image are preferred, as this area is at the depth of field center, resulting in the highest corner point extraction accuracy, e.g., ...). Figure 6 (As shown). Subsequently, the intrinsic parameters obtained from S20 are used to perform distortion correction on these filtered high-precision corner points. Combined with the known parameters provided by the slide table... Based on axis depth information, construct 3D-2D point pairs. Use the Perspective-n-Point (PnP) algorithm to solve for the rotation matrix of the camera coordinate system relative to the world coordinate system. Translation vector .

[0041] Furthermore, such as Figure 7 As shown, S40 includes the following steps: S41. Intersection Point Extraction. Acquire an image of the laser stripes projected onto the target. Extract the sub-pixel center lines of the laser stripes using skeleton thinning and grayscale centroid methods; fit the horizontal grid lines in the checkerboard image using the RANSAC algorithm; calculate the intersection points of the laser center lines with each horizontal grid line.

[0042] S42. Spatial coordinate reconstruction. Using the extrinsic parameters calculated in S30. By projecting the aforementioned "intersection points" on the image plane back onto the camera coordinate system using internal parameters, a three-dimensional scatter cloud on the laser light plane is obtained. .

[0043] S43. Plane fitting. For point clouds. By performing least-squares plane fitting (SVD decomposition), the laser light plane equation is obtained:

[0044] S44. Error Assessment. Calculate the root mean square error (RMS) from all observation points to the fitted plane. If the error is less than a preset threshold (e.g., 0.05 mm), the calibration is complete; otherwise, return to S10 to check the hardware alignment.

[0045] It should be understood that the above embodiments are merely preferred embodiments of the present invention, intended to explain the technical principles and effects of the present invention, and are not intended to limit the scope of protection of the present invention. For those skilled in the art, any improvements, modifications, substitutions, or equivalent transformations made based on the technical essence of the present invention without departing from the concept of the present invention should be considered as included within the scope of protection of the claims of the present invention.

Claims

1. A high-precision calibration method for a tilt-shift camera based on the Scheimpflug principle, characterized in that, The calibration method includes the following steps: S10. Imaging Environment Construction: Adjust the angle between the lens plane and the imaging plane of the tilt-shift camera to meet the Scheimpflug condition to determine the optimal sharp imaging area; configure the synchronous triggering mechanism of the slide and the grating ruler to construct a moving reference with micron-level precision; S20. Camera intrinsic parameter calibration: Acquire chessboard images in different poses, automatically calculate the distortion center based on the point-to-line distance and (LinePointsDis) index, and optimize the solution of the camera intrinsic parameter matrix and distortion parameters using a 14-parameter model that includes thin prism distortion. S30. Camera extrinsic parameter calibration: Use a slide table to move the target in fixed steps to construct a virtual 3D target, and use the depth-constrained PnP algorithm to solve the pose relationship between the camera coordinate system and the world coordinate system. S40. Laser plane calibration: Extract the intersection points of the laser stripe center line and the horizontal grid lines of the checkerboard pattern, use the geometric constraints of the grid lines to improve data accuracy, and then fit the laser plane equation.

2. The high-precision calibration method for a tilt-shift camera based on the Scheimpflug principle according to claim 1, characterized in that, S10 includes the following steps: S11. Place the calibration target parallel to the image sensor, and observe the changes in image sharpness by translating the target along the optical axis. Determine the front and rear boundary positions of the image with full field sharpness. This area is the effective calibration space. S12. Establish a sliding stage trigger synchronization mechanism based on grating ruler position feedback. The calibration target is vertically fixed on the precision electric sliding stage. The trigger pulse is set so that the camera automatically triggers a data acquisition every 2.000mm of movement of the sliding stage, ensuring the equidistant nature of spatial sampling.

3. The high-precision calibration method for a tilt-shift camera based on the Scheimpflug principle according to claim 1, characterized in that, S20 includes the following steps: S21. Acquire multiple chessboard images from different angles within the clear imaging area and extract sub-pixel corner points; S22. Calculate the distortion center: For each row and column corner point in the checkerboard image, fit a straight line and calculate the sum of the Euclidean distances from all corner points to the corresponding fitted lines, defined as LinePointsDis. In the formula, These are the coordinates of the corner point. To fit the parameters of the straight line, iterate through all rows and columns, and select... The smallest row and Find the smallest column and calculate the intersection of the two lines. This is taken as the true distortion center of the camera; S23. Solving for intrinsic parameters and distortion coefficients: A 14-parameter model including radial distortion, tangential distortion, and thin prism distortion is used to calibrate the camera. This distortion model is described as follows: In the formula, To normalize image coordinates, ; The radial distortion coefficient is... The tangential distortion coefficient is... Let be the distortion coefficient of the thin prism; this higher-order model can effectively compensate for the optical asymmetric distortion caused by axis-shift imaging. Subsequently, an objective function for minimizing the reprojection error is established: During the optimization process, a fixed principal point strategy (CALIB_FIX_PRINCIPAL_POINT) is adopted to optimize the distortion center calculated in S22. As fixed parameters, only the focal length and distortion coefficients are iteratively optimized to obtain a high-precision intrinsic parameter matrix. .

4. The high-precision calibration method for a tilt-shift camera based on the Scheimpflug principle according to claim 1, characterized in that, S30 includes the following steps: S31. Virtual 3D Target Construction: Control the slide to move the checkerboard target along a direction perpendicular to the target plane. Next (in this embodiment) ), constructing in space with Virtual 3D target with layer depth information; S32. Pose Calculation: Considering the unique imaging optical path of a tilt-shift camera, its depth distribution across the entire scene is uneven (i.e., the area near the focal plane satisfying the Scheimpflug condition is sharp, gradually blurring towards the sides). To ensure calibration accuracy, this step introduces a sharpness-based region filtering mechanism. From the extracted corner data, only specific columns of corners located within the optimal sharpness imaging area are retained. Subsequently, the intrinsic parameters obtained in S20 are used to perform distortion correction on these filtered high-precision corners, combined with the known parameters provided by the slide. Using axis depth information, construct 3D-2D point pairs, and use the Perspective-n-Point (PnP) algorithm to solve for the rotation matrix of the camera coordinate system relative to the world coordinate system. Translation vector .

5. The high-precision calibration method for a tilt-shift camera based on the Scheimpflug principle according to claim 1, characterized in that, S40 includes the following steps: S41. Intersection point extraction: Acquire images of laser stripes projected onto the target, and extract the sub-pixel center lines of the laser stripes using skeleton thinning and grayscale centroid method; fit the horizontal grid lines in the checkerboard image using the RANSAC algorithm; calculate the intersection points of the laser center lines with each horizontal grid line. S42. Spatial coordinate reconstruction: using the extrinsic parameters calculated in S30 By back-projecting the aforementioned "intersection points" on the image plane onto the camera coordinate system using intrinsic parameters, a three-dimensional scatter cloud on the laser light plane is obtained. ; S43. Plane Fitting: For Point Clouds By performing least-squares plane fitting (SVD decomposition), the laser light plane equation is obtained: ; S44. Error Assessment: Calculate the root mean square error (RMS) of all observation points to the fitting plane. If the error is less than the preset threshold (e.g., 0.05 mm), the calibration is complete; otherwise, return to S10 to check the hardware alignment.

Images