Method and apparatus for double object determination of a shift camera

Abstract

The application discloses a method and device for double-target calibration of a shift camera, and the method comprises the following steps: acquiring calibration board images at different horizontal heights, wherein all the acquired calibration board images at least include a calibration board image at a horizontal height of Z=0; calling a Blob detection operator in opencv to extract feature point coordinates of all the calibration board images; constructing a shift camera model AH=B of the shift camera, and calculating a left camera internal parameter matrix M l and a right camera internal parameter matrix M r of the shift camera model AH=B; calling an opencv Zhang Zhengyou calibration function to solve distortion parameters of the calibration board image at Z=0; calling an opencv stereo calibration function to calculate a rotation matrix R and a translation vector T of the right camera relative to the left camera; calling an opencv stereo rectification function to calculate a re-projection matrix Q; inputting two-dimensional pixel coordinates and aligned disparity values d i to calculate 3D coordinates corresponding to the two-dimensional pixel coordinates; and the application can realize direct precision verification of double-target calibration of the shift camera and simplify a double-target calibration process of the shift camera.

Description

Technical Field

[0001] The present invention relates to the technical field of machine vision three-dimensional measurement, and in particular to a binocular positioning method and device for a tilt-shift camera. Background Art

[0002] In the field of three-dimensional measurement, limited depth of field makes it difficult for cameras to capture clear images of targets over a large area. This is especially true when using laser triangulation. Fringe images can easily become blurred due to loss of focus. A common solution in industrial measurement is to increase the depth of field by tilting the camera's optical axis at a significant angle to the sensor, satisfying Scheimpflug's law. These cameras are called tilt-shift cameras.

[0003] In industrial applications, high-precision calibration of tilt-shift cameras is the premise and basis for achieving high-precision measurement. Since their imaging models are different from those of general cameras, existing camera calibration methods are difficult to apply directly.

[0004] Existing camera calibration methods include the tilt-shift camera calibration method based on a generalized imaging model, the pinhole camera calibration method based on a three-dimensional checkerboard, and the Zhang Zhengyou calibration method. The Zhang Zhengyou calibration method is the most widely used, offering excellent robustness and accuracy. However, its target is defined based on an orthographic image, and the world coordinates of all feature points on the target lie in the same plane, resulting in a loss of coordinate dimension information and limiting calibration accuracy.

[0005] However, the problems of existing calibration methods can be summarized into the following three aspects:

[0006] 1. It is difficult to obtain clear images of the target using the pinhole camera model;

[0007] 2. The target feature points are all located in the same plane, which limits the calibration accuracy;

[0008] 3. The calculation of various parameters extracted by the tilt-shift camera in combination with the angle is complex.

[0009] In the improvement of the existing tilt-shift camera calibration technology, conventional Zhang calibration is first performed, and its result is used as the initial value to correct the original image, and then Zhang calibration is used for secondary calibration, which achieves a certain calibration accuracy. However, the calibration process is cumbersome. Summary of the Invention

[0010] The purpose of the present invention is to provide a binocular positioning method and device for a tilt-shift camera. The method fuses the angle matrix into the camera's intrinsic parameter matrix and uses a lifting platform to raise a fixed displacement value to obtain a small number of calibration plate images located in different planes. This method can directly verify the accuracy of binocular positioning of the tilt-shift camera. The method also uses the homography matrix to solve the intrinsic and extrinsic parameters and combines OpenCV to implement stereo vision calibration, thereby simplifying the binocular positioning process of the tilt-shift camera.

[0011] To achieve the above-mentioned object, the present invention discloses a dual-target positioning method for a tilt-shift camera, assuming that the rising direction of the lifting platform is the positive direction of the Z axis, and the tilt-shift camera includes a left camera and a right camera, which includes the following steps:

[0012] S1. Adjust the height of the tilt-shift camera by using a lifting platform to obtain calibration plate images at different plane heights, wherein all obtained calibration plate images include at least the calibration plate image when the horizontal height is Z=0;

[0013] S2. Call the Blob detection operator in opencv to extract the coordinates of feature points of all calibration plate images;

[0014] S3. Construct the tilt-shift camera model AH=B of the tilt-shift camera based on the coordinates of the feature points of all calibration plate images, and calculate the left camera intrinsic parameter matrix M of the tilt-shift camera model AH=B. l and the right camera intrinsic parameter matrix M r ;

[0015] S4. Call the opencv Zhang Zhengyou calibration function to solve the distortion parameters of the calibration plate image when Z=0;

[0016] S5. Call the opencv stereo calibration function to calculate the rotation matrix R and translation vector T of the right camera relative to the left camera;

[0017] S6. Call the opencv stereo correction function to calculate the reprojection matrix Q;

[0018] S7, input two-dimensional pixel coordinates and aligned disparity value d i , the 3D coordinates corresponding to the two-dimensional pixel coordinates are calculated according to the reprojection matrix Q.

[0019] Compared with the existing technology, the present invention integrates the angle matrix into the camera intrinsic parameter matrix and uses a lifting platform to increase the fixed displacement value to obtain a small number of calibration plate images located in different planes. By collecting images at different heights along the Z axis in the world coordinate system, the binocular positioning accuracy is improved and the calibration accuracy can be verified in real time. The angle matrix is ​​integrated into the camera intrinsic parameter matrix and the homography matrix is ​​used to solve the internal and external parameters. In combination with OpenCV, stereo vision calibration is realized, which simplifies the binocular positioning process of the tilt-shift camera.

[0020] Preferably, the step S1 further includes the following steps:

[0021] S101, selecting a circular calibration plate whose size is larger than the field of view of the tilt-shift camera as a calibration target;

[0022] S102, keep the feature points of the calibration board cover the entire field of view of the shift axis camera, the calibration board is provided with special points with a radius greater than the feature points of the calibration board, the special points are used for the left camera and the right camera to construct the same world coordinate system.

[0023] Preferably, all calibration board images also contain calibration board images with horizontal heights of Z=1, Z=2, Z=3, Z=4 and Z=5.

[0024] Preferably, the expression of the shift axis camera model AH=B is

[0025] Preferably, the step S3 specifically comprises the following steps:

[0026] S31, expand the shift axis camera model AH=B to obtain an expanded expression

[0027] S32, according to the feature point coordinates and the corresponding world coordinates of all calibration board images, solve the matrix H by singular value decomposition;

[0028] S33, the expression of the pinhole camera complete model is wherein s represents a proportionality coefficient, M represents a camera internal parameter matrix, R t represents a camera external parameter matrix, the vector expression R t =[R1 R2 R3 t] is calculated according to the pinhole camera complete model.

[0029] S34, according to the vector expression R t =[R1 R2 R3 t] and the constraint condition of the rotation vector, the following is calculated: ||R1||=||R2||=||R3||=1;

[0030] S35, according to the pinhole camera complete model, the shift axis camera model AH=B and the constraint condition of the rotation vector, the left camera internal parameter matrix M l and the right camera internal parameter matrix M r are calculated respectively as follows:

[0031]

[0032]

[0033]

[0034]

[0035] Preferably, the step S4 specifically includes the following steps:

[0036] S41. Call the opencv Zhang Zhengyou calibration function cv::calibrateCamera();

[0037] S42, using the pixel coordinates and world coordinates of the calibration plate image at Z=0 as input, calculate the distortion matrix D of the left camera l and the distortion matrix D of the right camera r .

[0038] Preferably, the step S5 specifically includes the following steps:

[0039] S51. Call the opencv stereo calibration function cv::stereoCalibrate();

[0040] S52, with M l 、M r 、D l 、D r 、R lr and T lr As a variable input to the reprojection matrix Q, the expression of the reprojection matrix Q is calculated as follows:

[0041] Preferably, the step S7 specifically includes the following steps:

[0042] S71, input two-dimensional pixel coordinates and aligned disparity value d i ;

[0043] S72. Based on the reprojection matrix Q, the expression of the reprojection matrix Q can be updated to

[0044] S73, calculate the 3D coordinates corresponding to the two-dimensional pixel coordinates according to the expression of the updated reprojection matrix Q:

[0045] Preferably, the step S7 further includes the following steps:

[0046] S8, the feature point coordinates of all aligned calibration plate images and the corresponding aligned disparity values ​​d i Substitute them into the expression of the updated reprojection matrix Q for calculation to obtain the depth Z of each layer of calibration plate image, where the calibration error of each layer of calibration plate image is E r =(Z i+1 -Z i )-1.

[0047] Accordingly, the present invention also discloses a binocular positioning device for a tilt-shift camera, comprising:

[0048] an acquisition unit configured to adjust the height of the tilt-shift camera by means of a lifting platform to acquire calibration plate images at different plane heights, wherein all acquired calibration plate images at least include an image of the calibration plate at a horizontal height of Z=0;

[0049] The calling unit is configured to call the Blob detection operator in OpenCV to extract the coordinates of feature points of all calibration plate images;

[0050] The first execution unit is configured to construct a tilt-shift camera model AH=B of the tilt-shift camera according to the feature point coordinates of all calibration plate images, and calculate the left camera intrinsic parameter matrix M of the tilt-shift camera model AH=B l and the right camera intrinsic parameter matrix M r ;

[0051] The second execution unit is configured to call the opencv Zhang Zhengyou calibration function to solve the distortion parameters of the calibration plate image when Z=0;

[0052] The third execution unit is configured to call an opencv stereo calibration function to calculate a rotation matrix R and a translation vector T of the right camera relative to the left camera;

[0053] The fourth execution unit is configured to call the opencv stereo correction function to calculate the reprojection matrix Q;

[0054] The fifth execution unit is configured to input the two-dimensional pixel coordinates and the aligned disparity value d i , the 3D coordinates corresponding to the two-dimensional pixel coordinates are calculated according to the reprojection matrix Q. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 is a flowchart of a binocular positioning method for a tilt-shift camera according to the present invention;

[0056] Figure 2 This is a schematic diagram of the difference in Y values ​​when the left and right images are aligned;

[0057] Figure 3 is a schematic diagram of calibration error distribution;

[0058] Figure 4 It is a schematic diagram of 3D coordinates displayed in stereo;

[0059] Figure 5 It is a structural schematic diagram of the binocular positioning device of the tilt-shift camera of the present invention. DETAILED DESCRIPTION

[0060] In order to explain the technical content, structural features, achieved objectives and effects of the present invention in detail, the following is a detailed description in conjunction with the embodiments and the accompanying drawings.

[0061] See also Figures 1-4 As shown, a dual-target positioning method for a tilt-shift camera of this embodiment is provided. Assuming that the rising direction of the lifting platform is the positive direction of the Z axis, the tilt-shift camera includes a left camera and a right camera, and the method includes the following steps:

[0062] S1. Adjust the height of the tilt-shift camera by using a lifting platform to obtain calibration plate images at different plane heights, wherein all obtained calibration plate images include at least the calibration plate image when the horizontal height is Z=0;

[0063] S2. Call the Blob detection operator in opencv to extract the coordinates of feature points of all calibration plate images;

[0064] S3. Construct the tilt-shift camera model AH=B of the tilt-shift camera based on the coordinates of the feature points of all calibration plate images, and calculate the left camera intrinsic parameter matrix M of the tilt-shift camera model AH=B. l and the right camera intrinsic parameter matrix M r ;

[0065] S4. Call the opencv Zhang Zhengyou calibration function to solve the distortion parameters of the calibration plate image when Z=0;

[0066] S5. Call the opencv stereo calibration function to calculate the rotation matrix R and translation vector T of the right camera relative to the left camera;

[0067] S6. Call the opencv stereo correction function to calculate the reprojection matrix Q;

[0068] S7, input two-dimensional pixel coordinates and aligned disparity value d i , the 3D coordinates corresponding to the two-dimensional pixel coordinates are calculated according to the reprojection matrix Q.

[0069] Preferably, the step S1 further includes the following steps:

[0070] S101, selecting a circular calibration plate whose size is larger than the field of view of the tilt-shift camera as a calibration target;

[0071] S102: Maintain that the feature points of the calibration plate cover the entire field of view of the tilt-shift camera, and provide special points on the calibration plate with a radius larger than the feature points of the calibration plate, wherein the special points are used for the left camera and the right camera to construct the same world coordinate system.

[0072] Preferably, all calibration board images also contain calibration board images with horizontal heights of Z=1, Z=2, Z=3, Z=4 and Z=5.

[0073] Preferably, the expression of the shift-axis camera model AH=B is

[0074] Preferably, the step S3 specifically comprises the following steps:

[0075] S31, unfolding the shift-axis camera model AH=B to obtain an unfolded expression

[0076] S32, according to the feature point coordinates and the corresponding world coordinates of all calibration board images, solving the matrix H by singular value decomposition;

[0077] S33, the expression of the pinhole camera complete model is

[0077] wherein s represents a scale factor, M represents a camera intrinsic parameter matrix, R t represents a camera extrinsic parameter matrix, and the vector expression R t =[R1 R2 R3 t] is calculated according to the pinhole camera complete model.

[0078] S34, according to the vector expression R t =[R1 R2 R3 t] and the constraint condition of the rotation vector, the following is calculated: ||R1||=||R2||=||R3||=1;

[0079] S35, according to the pinhole camera complete model, the shift-axis camera model AH=B and the constraint condition of the rotation vector, the left camera intrinsic parameter matrix M l and the right camera intrinsic parameter matrix M r are calculated respectively as follows:

[0080]

[0081]

[0082]

[0083]

[0084] Preferably, the step S4 specifically comprises the following steps:

[0085] S41, calling the opencv Zhang Zhengyou calibration function cv: calibrateCamera ().

[0086] S42, taking the pixel coordinates and the world coordinates of the calibration board image when Z=0 as inputs, calculate the distortion matrix D of the left camera l and the distortion matrix D of the right camera r .

[0087] Preferably, the step S5 specifically comprises the following steps:

[0088] S51, call the opencv stereo calibration function cv: stereoCalibrate ().

[0089] S52, take M l , M r , D l , D r , R lr and T lr as variable inputs into the re-projection matrix Q, and calculate the expression of the re-projection matrix Q as

[0090] Preferably, the step S7 specifically comprises the following steps:

[0091] S71, input the two-dimensional pixel coordinates and the aligned disparity value d i .

[0092] S72, according to the re-projection matrix Q, the expression of the re-projection matrix Q can be updated as

[0093] S73, according to the updated expression of the re-projection matrix Q, the 3D coordinates corresponding to the two-dimensional pixel coordinates are calculated as

[0094] Preferably, the step S7 further comprises the following steps:

[0095] S8, respectively substitute the feature point coordinates of all the aligned calibration board images and the corresponding aligned disparity values d i into the updated expression of the re-projection matrix Q to calculate the depth Z of each layer of calibration board image, wherein the calibration error of each layer of calibration board image is E r =(Z i+1 -Z i )-1.

[0096] Please refer to Figure 4 , the present application also discloses a double-target calibration device of a moving-axis camera, which comprises:

[0097] The acquisition unit 10 is configured to adjust the height of the tilt-shift camera by using a lifting platform to respectively acquire calibration plate images at different plane heights, wherein all acquired calibration plate images at least include the calibration plate image at a horizontal height of Z=0;

[0098] The calling unit 20 is configured to call the Blob detection operator in OpenCV to extract the coordinates of feature points of all calibration plate images;

[0099] The first execution unit 30 is configured to construct a tilt-shift camera model AH=B of the tilt-shift camera according to the coordinates of the feature points of all calibration plate images, and calculate the left camera intrinsic parameter matrix M of the tilt-shift camera model AH=B. l and the right camera intrinsic parameter matrix M r ;

[0100] The second execution unit 40 is configured to call the opencv Zhang Zhengyou calibration function to solve the distortion parameters of the calibration plate image when Z=0;

[0101] The third execution unit 50 is configured to call the opencv stereo calibration function to calculate the rotation matrix R and translation vector T of the right camera relative to the left camera;

[0102] The fourth execution unit 60 is configured to call the opencv stereo correction function to calculate the reprojection matrix Q;

[0103] The fifth execution unit 70 is configured to input the two-dimensional pixel coordinates and the aligned disparity value d i , the 3D coordinates corresponding to the two-dimensional pixel coordinates are calculated according to the reprojection matrix Q.

[0104] Combine Figures 1-5 The present invention integrates the angle matrix into the camera intrinsic parameter matrix and uses a lifting platform to increase the fixed displacement value to obtain a small number of calibration plate images located in different planes. It improves the binocular positioning accuracy by collecting images at different heights along the Z axis in the world coordinate system, and can realize real-time verification of the calibration accuracy. The angle matrix is ​​integrated into the camera intrinsic parameter matrix and the homography matrix is ​​used to solve the internal and external parameters, and stereo vision calibration is realized in combination with OpenCV, which simplifies the binocular positioning process of the tilt-shift camera.

[0105] The above disclosure is only the preferred embodiment of the present invention, which certainly cannot be used to limit the scope of the present invention. Therefore, equivalent changes made according to the scope of the patent application of the present invention are still within the scope of the present invention.

Claims

1. A method for binocular positioning of a tilt-shift camera, wherein the ascending direction of the lifting platform is the positive direction of the Z axis, and the tilt-shift camera includes a left camera and a right camera, characterized in that: The steps include: The height of the tilt-shift camera is adjusted by a lifting platform to obtain calibration plate images at different plane heights. All obtained calibration plate images at least include the calibration plate image when the horizontal height is Z=0. Call the Blob detection operator in opencv to extract the coordinates of feature points of all calibration plate images; According to the coordinates of the feature points of all calibration plate images, the tilt-shift camera model AH=B of the tilt-shift camera is constructed, and the left camera intrinsic parameter matrix M of the tilt-shift camera model AH=B is calculated. l and the right camera intrinsic parameter matrix M r ; Call the opencv Zhang Zhengyou calibration function to solve the distortion parameters of the calibration plate image when Z=0; Call the opencv stereo calibration function to calculate the rotation matrix R and translation vector T of the right camera relative to the left camera; Call the opencv stereo correction function to calculate the reprojection matrix Q; Input 2D pixel coordinates and aligned disparity values , the 3D coordinates corresponding to the two-dimensional pixel coordinates are calculated according to the reprojection matrix Q.

2. The binocular positioning method for a tilt-shift camera according to claim 1, wherein: The height of the tilt-shift camera is adjusted by the lifting platform to obtain calibration plate images at different plane heights, and all the obtained calibration plate images at least include the calibration plate image when the horizontal height is Z=0, and the steps of: Selecting a circular calibration plate with a plane size larger than the field of view of the tilt-shift camera as a calibration target; The characteristic points of the calibration plate are kept to cover the entire field of view of the tilt-shift camera. Special points with a radius larger than the characteristic points of the calibration plate are provided on the calibration plate. The special points are used for the left camera and the right camera to construct the same world coordinate system.

3. The binocular positioning method for a tilt-shift camera according to claim 1, wherein: All calibration plate images also include calibration plate images at horizontal heights of Z=1, Z=2, Z=3, Z=4, and Z=5.

4. The binocular positioning method for a tilt-shift camera according to claim 3, wherein: The expression of the tilt-shift camera model AH=B is: .

5. The binocular positioning method for a tilt-shift camera according to claim 4, wherein: The tilt-shift camera model AH=B of the tilt-shift camera is constructed based on the feature point coordinates of all calibration plate images, and the left camera intrinsic parameter matrix M of the tilt-shift camera model AH=B is calculated. l and the right camera intrinsic parameter matrix M r , specifically including the following steps: Expand the tilt-shift camera model AH=B to obtain the expanded expression: ; According to the coordinates of the feature points of all calibration plate images and the corresponding world coordinates, the singular value decomposition is used to solve the matrix H; Assume that the expression of the complete model of the pinhole camera is , where s represents the scale factor, M represents the camera internal parameter matrix, and R t Represents the camera extrinsic parameter matrix, which is calculated based on the complete pinhole camera model to obtain the vector expression ; According to the vector expression and the constraints of the rotation vector, we can calculate , , , , ; According to the pinhole camera complete model, the tilt-shift camera model AH=B and the constraints of the rotation vector, the left camera intrinsic parameter matrix M is calculated. l and the right camera intrinsic parameter matrix M r They are: ； ； ； 。 6. The binocular positioning method for a tilt-shift camera according to claim 5, wherein: The calling of the opencv Zhang Zhengyou calibration function to solve the distortion parameters of the calibration plate image when Z=0 specifically includes the following steps: Call opencv Zhang Zhengyou's calibration function cv::calibrateCamera(); The pixel coordinates and world coordinates of the calibration plate image at Z=0 are used as input to calculate the distortion matrix of the left camera. and the distortion matrix of the right camera .

7. The binocular positioning method for a tilt-shift camera according to claim 6, wherein: The calling of the opencv stereo calibration function to calculate the rotation matrix R and translation vector T of the right camera relative to the left camera specifically includes the following steps: Call the opencv stereo calibration function cv::stereoCalibrate(); by 、 、 、 、 and As a variable input to the reprojection matrix Q, the expression of the reprojection matrix Q is calculated as follows: .

8. The binocular positioning method for a tilt-shift camera according to claim 7, wherein: The input 2D pixel coordinates and the aligned disparity values , calculating the 3D coordinates corresponding to the two-dimensional pixel coordinates according to the reprojection matrix Q, specifically comprising the following steps: Input 2D pixel coordinates and aligned disparity values ; According to the reprojection matrix Q, the expression of the reprojection matrix Q can be updated as ; The 3D coordinates corresponding to the two-dimensional pixel coordinates are calculated based on the expression of the updated reprojection matrix Q: , , .

9. The binocular positioning method for a tilt-shift camera according to claim 8, wherein: The input 2D pixel coordinates and the aligned disparity values , the 3D coordinates corresponding to the two-dimensional pixel coordinates are calculated according to the reprojection matrix Q, and then the following steps are included: The feature point coordinates of all aligned calibration plate images and the corresponding aligned disparity values Substitute them into the expression of the updated reprojection matrix Q for calculation to obtain the depth Z of each layer of calibration plate image, where the calibration error of each layer of calibration plate image is .

10. A binocular positioning device for a tilt-shift camera, characterized in that: include: an acquisition unit configured to adjust the height of the tilt-shift camera by using a lifting platform to respectively acquire images of the calibration plate at different plane heights, wherein all acquired calibration plate images at least include an image of the calibration plate at a horizontal height of Z=0; The calling unit is configured to call the Blob detection operator in OpenCV to extract the coordinates of feature points of all calibration plate images; The first execution unit is configured to construct a tilt-shift camera model AH=B of the tilt-shift camera according to the coordinates of the feature points of all calibration plate images, and calculate the left camera intrinsic parameter matrix M of the tilt-shift camera model AH=B l and the right camera intrinsic parameter matrix M r ; The second execution unit is configured to call the opencv Zhang Zhengyou calibration function to solve the distortion parameters of the calibration plate image when Z=0; The third execution unit is configured to call an opencv stereo calibration function to calculate a rotation matrix R and a translation vector T of the right camera relative to the left camera; The fourth execution unit is configured to call the opencv stereo correction function to calculate the reprojection matrix Q; The fifth execution unit is configured to input the two-dimensional pixel coordinates and the aligned disparity value , the 3D coordinates corresponding to the two-dimensional pixel coordinates are calculated according to the reprojection matrix Q.

Images