Camera sight line parameter and display geometric pose calibration method

Abstract

The invention provides a camera sight parameter and display geometric pose calibration method without a camera model, aiming at the problem that the camera sight and the display geometric pose are difficult to accurately calibrate due to camera model defects. Structured light coding patterns on a plurality of groups of displays or display virtual images are shot by a camera, coordinates of light source points of the displays are obtained through decoding and unified to a world coordinate system, a sight line equation is used for representing the sight line of the camera, sight line equation fitting is carried out by the light source points of the displays, and finally nonlinear optimization is carried out according to fitting residual errors. And determining an optimal solution of the sight line parameter and the display pose parameter. The method has the advantages of high precision, high reliability, high flexibility, simultaneous calibration of camera line-of-sight parameters and display geometric poses and the like.

Description

Technical Field

[0001] The present invention relates to a calibration method for camera sight line parameters and display geometric posture of an optical system based on a camera and a display, and in particular to a technical solution for calibrating optical element surface shape and wavefront detection system based on deflectometry in the field of optical detection. Background Art

[0002] In the field of optical detection, especially the surface shape and wavefront detection of optical components based on deflectometry, an optical system composed of a camera and a display is often used. In order to complete high-precision surface shape detection or wavefront detection tasks, the user must accurately calibrate the camera's line of sight and the display's posture. The current common camera model uses a pinhole model plus a distortion model to compensate for the lens structure and distortion, thereby indirectly describing the direction of the camera's line of sight. This model ignores the high-order distortion, asymmetric distortion, processing and assembly errors in the actual optical imaging system, and the pinhole model assumes that all light passes through the projection center, which is also inconsistent with the actual optical imaging system. This defect makes it difficult to accurately calibrate the camera's line of sight and the display's geometric posture, limiting the performance of the optical system, especially the accuracy of optical component surface shape and wavefront detection based on deflectometry.

[0003] To solve this problem, the present invention proposes a calibration method for camera sight line parameters and display geometric posture without camera model. First, the camera captures the structured light coding patterns on multiple groups of displays or virtual images of displays, decodes and obtains the coordinates of the display light source points and unifies them into the world coordinate system, then fits the sight line equations of multiple groups of display light source points, and finally performs nonlinear optimization based on the fitting residual to determine the optimal solution of sight line parameters and display posture parameters. Summary of the invention

[0004] In order to overcome the shortcomings of the background technology, the present invention aims at the problem that the camera line of sight and the geometric pose of the display are difficult to accurately calibrate due to the defects of the camera model, and proposes a method for calibrating the camera line of sight parameters and the geometric pose of the display without the need for a camera model. The present invention directly characterizes the camera line of sight based on the line of sight equation, without the need for indirect characterization based on the camera model. The method has the advantages of high precision, high reliability, high flexibility, and simultaneous calibration of the camera line of sight parameters and the geometric pose of the display.

[0005] The present invention provides a method for calibrating camera sight parameters and display geometric posture without the need for a camera model. The camera shoots structured light coding patterns on multiple groups of displays or virtual images of displays, decodes and obtains the coordinates of the display light source points and unifies them into the world coordinate system, uses the sight line equation to characterize the camera sight line, and then fits the sight line equation with multiple groups of display light source points, and finally performs nonlinear optimization based on the fitting residual to determine the optimal solution of the sight line parameters and display posture parameters. The specific steps are as follows:

[0006] Step 1: The display displays the structured light coded pattern, and uses a camera to shoot multiple groups of structured light coded patterns on the display or the virtual image of the display according to the optical system structure to be calibrated: if the display is in the camera's field of view, the structured light coded pattern on the display is directly photographed; if the camera cannot directly observe the display, a plane mirror is used to make the virtual image of the display after mirror reflection appear in the camera's field of view.

[0007] Step 2: Decode each group of captured structured light coding patterns to obtain the display light source point in the display local coordinate system Axis and Axis coordinates; Use a two-dimensional polynomial to characterize the flatness of the display and determine the position of the display light source in the local coordinate system of the display Axis coordinates; the three-dimensional coordinates of the display light source point in the display local coordinate system are marked as ,in, Represents the coordinates of the light source point, subscript Indicates pixels, subscript Indicates A set of structured light coding patterns, Represents the local coordinate system of the display; through the coordinate transformation parameters Unify the display light source point to the world coordinate system, denoted as : , (1) in, represents the world coordinate system, The coordinate transformation parameters The obtained coordinate transformation matrix; if the display appears directly in the camera's field of view, then , and Respectively The rotation matrix and translation vector of the group: , (2) in, ; If the camera captures the virtual image of the display reflected by the plane mirror, then , and Respectively Plane mirror unit normal and distance for the group: , (3) in, .

[0008] Step 3: For each pixel, characterize the camera's line of sight using the line of sight equation: , (4) in, [ x p,o , y p,o ,0] is the line of sight and the world coordinate system - - The intersection of the planes, and They are The slope of the direction; Structured light coding pattern pixels, the coordinates Center and fit the line of sight equation, record With the line of sight equation exist Perpendicular to In the plane of the axis The deviation in direction is and : , (5) This deviation is the residual of the line of sight equation fitting.

[0009] Step 4: Use the fitting residual as the objective function for nonlinear optimization: , (6) The optimal solution Fit the sight line equation again to determine the optimal camera sight line parameters and ; If the display appears directly in the camera's field of view, the display position By the last group ( Group) coordinate transformation matrix is ​​determined by: , (7) If the camera shoots the display through a plane mirror, the display posture for: , (8) Thus, the calibration of the camera sight parameters and the display geometric posture is completed. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 It is a schematic diagram of the calibration principle of the present invention; DETAILED DESCRIPTION

[0011] In order to make the purpose and scheme of the present invention clearer, the present invention is described in detail by examples in conjunction with the accompanying drawings. It is necessary to point out that the following embodiments are only used to further illustrate the present invention and cannot be understood as limiting the scope of protection of the present invention. Those skilled in the art may make some non-essential improvements and adjustments to the present invention based on the above content of the present invention, which still fall within the scope of protection of the present invention.

[0012] The present invention provides a method for calibrating camera sight parameters and display geometric posture without the need for a camera model. The camera captures the structured light coding patterns on multiple groups of displays or virtual images of displays and decodes them to obtain the coordinates of the display light source points. The sight line equation is used to characterize the camera sight line, and then the sight line equation is fitted by multiple groups of display light source points. Finally, nonlinear optimization is performed based on the fitting residual to determine the optimal solution of the sight line parameters and display posture parameters. The specific steps are as follows:

[0013] Step 1: Display the structured light coded pattern on the display, and use a camera to capture the structured light coded pattern on multiple sets of displays or virtual images of the displays according to the optical system structure to be calibrated:

[0014] Reference Figure 1 (a), if the display 2 is within the field of view of the camera 1, the structured light coding pattern 4 is displayed on the display 2, and the camera 1 captures multiple groups of structured light coding patterns;

[0015] Reference Figure 1 (b) If the camera 1 cannot directly observe the display 2, a plane reflector 5 is used to make the virtual image of the display 2 reflected by the mirror 5 appear in the camera field of view, and the camera 1 captures a plurality of virtual images 6 of structured light coding patterns.

[0016] Step 2: Decode each group of captured structured light coding patterns to obtain the display light source point in the display local coordinate system Axis and Axis coordinates; Use a two-dimensional polynomial to characterize the flatness of the display and determine the position of the display light source in the local coordinate system of the display Axis coordinates; the three-dimensional coordinates of the display light source point in the display local coordinate system are marked as ,in, Represents the coordinates of the light source point, subscript Indicates pixels, subscript Indicates A set of structured light coding patterns, Represents the local coordinate system of the display; through the coordinate transformation parameters Unify the display light source point to the world coordinate system, denoted as : , (1) in, represents the world coordinate system, The coordinate transformation parameters The obtained coordinate transformation matrix; if the display appears directly in the camera's field of view, then , and Respectively The rotation matrix and translation vector of the group: , (2) in, ; If the camera captures the virtual image of the display reflected by the plane mirror, then , and Respectively Plane mirror unit normal and distance for the group: , (3) in, .

[0017] Step 3: For each pixel, characterize the camera's line of sight using the line of sight equation 3: , (4) in, [ x p,o , y p,o ,0] is the line of sight and the world coordinate system - - The intersection of the planes, and They are The slope of the direction; Structured light coding pattern Pixels, remember With the line of sight equation exist Perpendicular to In the plane of the axis The deviation in direction is and : , (5) The coordinates Centralized to: , (6) but and Rewritten as: , (7) and: , (8) , (9)

[0018] Step 4: Formula (7) calculates and That is, the residual of the straight line equation fitting, and the fitting residual is used as the objective function for nonlinear optimization: , (10) According to formulas (8) and (9), the optimal solution is Determine the camera's sight line parameters and ; If the display appears directly in the camera field of view, the display position is determined by the last group ( Group) coordinate transformation matrix is ​​determined by: , (11) If the camera shoots the display through a plane mirror, the display posture is: , (12) Thus, the calibration of the camera sight parameters and the display geometric posture is completed.

Claims

1. A method for calibrating camera sight parameters and display geometric pose, Features: The camera's line of sight is directly characterized according to the line of sight equation, without the need for indirect characterization based on the camera model. The camera shoots structured light coded patterns on multiple groups of displays or virtual images of displays, decodes and obtains the coordinates of the display light source points and unifies them into the world coordinate system. The line of sight equation is then fitted using the multiple groups of display light source points. Finally, nonlinear optimization is performed based on the fitting residuals to determine the optimal solutions for the line of sight parameters and display pose parameters, thus completing the calibration of the camera's line of sight parameters and display geometric pose.

2. A method for calibrating camera sight line parameters and display geometric posture according to claim 1, Features The coordinates of the display light source point obtained by the structured light coding pattern are unified into the world coordinate system. The specific form is as follows: The three-dimensional coordinates of the display light source point in the display local coordinate system are marked as ,in, Represents the coordinates of the light source point, subscript Indicates pixels, subscript Indicates A structured light coding pattern, Represents the local coordinate system of the display; through the coordinate transformation parameters Unify the display light source point to the world coordinate system, denoted as : , in, represents the world coordinate system, The coordinate transformation parameters The obtained coordinate transformation matrix; if the display appears directly in the camera's field of view, then , and Respectively The rotation matrix and translation vector of the group: , in, ; If the camera captures the virtual image of the display reflected by the plane mirror, then , and Respectively Plane mirror unit normal and distance for the group: , in, .

3. A method for calibrating camera sight line parameters and display geometric posture according to claim 1, Features The residuals of the line-of-sight equations of multiple display light source points are used as the objective function for nonlinear optimization to determine the optimal solutions of the line-of-sight parameters and display posture parameters. The specific form is as follows: Perform nonlinear optimization using the fitting residual as the objective function: , The optimal solution Fit the sight line equation again to determine the optimal camera sight line parameters and ; If the display appears directly in the camera's field of view, the display position By the last group ( Group) coordinate transformation matrix is ​​determined by: , If the camera shoots the display through a plane mirror, the display posture for: , Thus, the calibration of the camera sight parameters and the display geometric posture is completed.

Images