A multi-distance-based projector parameter calibration method and projector

Abstract

This invention belongs to the field of projector parameter calibration technology, specifically disclosing a projector parameter calibration method and projector based on multiple distances. First, the intrinsic and extrinsic parameters of the projector's optical engine and camera are calibrated at multiple different distances, obtaining calibration results at multiple distances. This allows for the fusion of calibration parameters from multiple distances to calculate the horizontal angle during actual keystone correction angle calculations, resulting in a more accurate fused horizontal angle. By fusing calibration parameters from multiple distances to calculate the calibration result at the current distance, this invention achieves adaptive calibration of the projector and camera's intrinsic and extrinsic parameters at different distances, improving the accuracy of calibration at different distances and resulting in more accurate horizontal angles and more precise keystone correction.

Description

Technical Field

[0001] This invention belongs to the field of projector parameter calibration technology, specifically relating to a multi-distance projector parameter calibration method and a projector. Background Technology

[0002] The calibration of projector parameters affects the projection quality of the projector. Therefore, improving the accuracy of projector calibration and reducing calibration costs are crucial for projector production.

[0003] Traditional methods for calibrating the intrinsic and extrinsic parameters of projector optical engines and cameras require placing a fixed-size checkerboard calibration board at the calibration station. The projector optical engine projects Gray codes containing coded information onto the checkerboard calibration board, and the camera captures these images to calibrate the intrinsic and extrinsic parameters of the projector optical engine and camera. This method requires an external, fixed-size checkerboard calibration board, which is costly. Furthermore, the calibration time for projecting Gray codes with coded information is relatively long. Additionally, since the intrinsic and extrinsic parameters of the projector optical engine vary at different distances, it cannot achieve adaptive calibration of intrinsic and extrinsic parameters at different distances, resulting in poor calibration performance.

[0004] Therefore, in existing technologies, some calibration methods project a checkerboard image onto a wall at a fixed distance. After the image is in focus, the camera is controlled to take a picture, which calculates the camera's intrinsic parameters and the extrinsic parameters between the camera and the projection engine at that distance. This allows for adaptive adjustment of the focal length in the projection engine's intrinsic parameters and the extrinsic parameters between the projection engine and the camera based on the number of motor steps after autofocus at different distances. This method calibrates the camera's intrinsic parameters at a single distance. However, because the camera's focal length and the extrinsic parameters between the projection engine and the camera vary at different distances, the calibration results will be inaccurate at other distances. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to overcome the defect that the existing methods for calibrating the intrinsic and extrinsic parameters of the projection optical engine and camera cannot effectively achieve the adaptive adaptation of intrinsic and extrinsic parameters at different distances, resulting in overlapping calibration effects. Therefore, the present invention provides a multi-distance-based projector parameter calibration method and projector.

[0006] A method for calibrating projector parameters based on multiple distances includes the following steps:

[0007] Set at least three reference zero distances. For each reference zero distance, move the projector so that the projector optical engine is facing the wall. The optical engine is at a distance from the wall from the reference zero distance. The projector optical engine projects a checkerboard pattern onto the wall. The camera of the projector takes a picture of the checkerboard pattern on the wall. The camera's intrinsic parameters and the extrinsic parameter rotation matrix between the projector optical engine and the camera are calibrated.

[0008] When the projector is in use, the projector optical engine projects a checkerboard image with multiple predefined corner coordinates onto the wall. The camera captures the corner pixel coordinates of the checkerboard image. Based on the camera's intrinsic parameters calibrated at a reference zero distance and the extrinsic rotation matrix between the projector optical engine and the camera, the distance from the current projector optical engine to the checkerboard image plane on the wall is calculated.

[0009] The projector's optical engine projects a checkerboard image with predefined coordinates of multiple corner points onto the wall. The horizontal rotation angle of the projector relative to the checkerboard image plane is calculated using the pixel coordinates of the checkerboard corner points captured by the camera, based on the camera's intrinsic parameters calibrated at multiple reference zero distances, as well as the extrinsic parameters of the projector and the camera.

[0010] Based on the current distance between the projector and the checkerboard image plane on the wall, the final horizontal rotation angle is calculated and determined. When the current distance between the projector and the checkerboard image plane on the wall is less than the minimum reference zero distance, the horizontal rotation angle calibrated at the minimum reference zero distance is determined as the final horizontal rotation angle. When the current distance between the projector and the checkerboard image plane on the wall is greater than the maximum reference zero distance, the horizontal rotation angle calibrated at the maximum reference zero distance is determined as the final horizontal rotation angle. Otherwise, the final horizontal rotation angle is calculated based on the two closest reference zero distances: one greater than the current distance between the projector and the checkerboard image plane on the wall, and the other less than the current distance between the projector and the checkerboard image plane on the wall.

[0011] Furthermore, for each of the aforementioned reference zero-position distances, the projector is moved so that its optical engine faces the wall, and the distance between the optical engine and the wall is the reference zero-position distance. A checkerboard pattern is projected onto the wall by the projector's optical engine, and the checkerboard pattern image on the wall is captured by the projector's camera. The intrinsic parameters of the camera and the extrinsic rotation matrix between the projector's optical engine and the camera are calibrated, including the following method steps:

[0012] The projector is positioned directly against the wall at the reference zero distance, projecting a checkerboard pattern, and the camera captures the corresponding checkerboard pattern.

[0013] Based on the correspondence between the world coordinate system coordinates of the chessboard image and the coordinates of the chessboard corner points detected by the camera in the chessboard image, calculate the homography matrix from the camera coordinate system to the world coordinate system of the chessboard on the wall.

[0014] Based on the homography matrix from the camera coordinate system to the checkerboard world coordinate system on the wall, calculate the rotation matrix from the camera coordinate system to the checkerboard world coordinate system on the wall, and define it as the rotation matrix from the camera coordinate system to the projection optical engine coordinate system.

[0015] Based on the intrinsic parameters of the projector and the distance from the projector to the wall, calculate the coordinates of the corner points of the checkerboard pattern projected onto the wall in the projector coordinate system.

[0016] Based on the rotation matrix and translation vector between the camera coordinate system and the projection optical engine coordinate system, the coordinates of the corner points of the chessboard grid projected onto the wall in the camera coordinate system are calculated.

[0017] By combining the pixel coordinates of the corner points in the checkerboard image captured by the camera with the coordinates of the corner points of the checkerboard on the wall in the camera coordinate system, the optimal camera intrinsic parameters, including the principal point and focal length, are calculated by the least squares method.

[0018] Based on the intrinsic parameters of the projector, the intrinsic parameters of the camera, the extrinsic parameters of the projector and the camera, the coordinates of multiple corner points of the predefined checkerboard pattern, the pixel coordinates of the corner points of the checkerboard captured by the camera, the distance from the projector to the checkerboard plane on the wall, and the horizontal rotation angle of the projector relative to the checkerboard plane on the wall are calculated.

[0019] When the distance from the projector to the checkerboard plane on the wall is within a preset threshold of the known distance from the projector to the wall, and the horizontal rotation angle of the projector relative to the checkerboard plane on the wall is within a preset threshold of the actual angle, determine the current camera intrinsic parameters and the extrinsic rotation matrix between the projector and the camera; otherwise, repeat the above steps.

[0020] Furthermore, the camera intrinsic principal points are iteratively combined and traversed according to a certain step size, and the x and y coordinates of the principal point are traversed based on the step size to traverse all possible combinations.

[0021] Furthermore, based on the correspondence between the world coordinate system coordinates of the checkerboard image and the coordinates of the checkerboard corner points detected in the checkerboard image captured by the camera, the homography matrix from the camera coordinate system to the world coordinate system of the checkerboard on the wall is calculated, including the following steps:

[0022] Let the chessboard grid be M rows and N columns, the size of the chessboard grid image projected onto the wall be length_chess, and the world coordinates of the chessboard grid on the wall be (length_chess* i , length_chess* j, 0). The homography matrix is ​​calculated using the findHomography function of OpenCV.

[0023] Furthermore, by combining the pixel coordinates of the corner points in the checkerboard image captured by the camera with the coordinates of the checkerboard corner points on the wall in the camera coordinate system, the optimal camera intrinsic parameters are calculated using the least squares method, including the following steps:

[0024] The normalized coordinates of the corner points of the checkerboard pattern on the wall in the camera coordinate system are expressed as follows: for 0 <= <n:

[0025] ;

[0026] ;

[0027] Where n represents the number of corner points on the chessboard. and Indicates the first Normalized coordinates of the three-dimensional coordinates of each corner point of the chessboard grid in the camera coordinate system. , and Indicates the first The three-dimensional coordinates of each corner point of the chessboard grid in the camera coordinate system;

[0028] Based on the pinhole imaging model, the pixel coordinates of the corner points of the checkerboard image captured by the camera. Establish a correspondence between the normalized coordinates of the checkerboard corner points on the wall in the camera coordinate system and the coordinates of the checkerboard.

[0029] ;

[0030] The optimal camera intrinsic parameters (Ci) are calculated using the least squares method based on the correspondence between point pairs. x _Camera, C y _Camera,F x _Camera, F y _Camera);

[0031] Among them, Cx_Camera and C y _Camera represents the principal point coordinates of the camera, F x _Camera and F y _Camera represents the focal length of the camera in the x and y axes.

[0032] Furthermore, based on the intrinsic parameters of the projection optical engine, the intrinsic parameters of the camera, the extrinsic parameters of the projection optical engine and the camera, the coordinates of multiple corner points of the predefined checkerboard pattern, the pixel coordinates of the checkerboard corner points captured by the camera, the distance from the projection optical engine to the checkerboard plane on the wall, and the horizontal rotation angle of the projection optical engine relative to the checkerboard plane on the wall are calculated, including the following method steps:

[0033] Based on the principle of binocular imaging, the coordinates of multiple corner points of a predefined checkerboard pattern are used. The spatial positions of the corner points of the checkerboard pattern projected onto the wall are calculated in the coordinate system of the projection optical engine using the pixel coordinates of the corner points captured by the camera. Then, based on the spatial positions of all the corner points of the checkerboard pattern in the coordinate system of the projection optical engine, the least squares method is used to fit the spatial plane, resulting in the spatial plane equation Z=AX+BY+C; the distance from the projection optical engine to the checkerboard plane is abs(C) / sqrt(A*A+B*B+1); and the horizontal rotation angle of the projection optical engine relative to the checkerboard plane on the wall is yaw=acos(A / sqrt(A*A+B*B+1))*180 / PI-90.

[0034] Where X, Y, and Z represent the three-dimensional coordinates of the corner points of the chessboard in the projection optical engine coordinate system, A, B, and C represent the plane parameters, abs() represents taking the absolute value, sqrt() represents taking the square root, acos() represents taking the inverse cosine, and PI represents pi.

[0035] Furthermore, three reference zero-position distances are selected, and for each reference zero-position distance, the intrinsic parameters of the camera and the extrinsic parameter rotation matrix between the projector optical engine and the camera are calibrated.

[0036] Furthermore, the projector's optical engine projects a checkerboard image with predefined corner coordinates onto the wall, and the camera captures the corner pixel coordinates of the checkerboard image. Based on the camera's intrinsic parameters calibrated at a reference zero distance and the extrinsic rotation matrix between the projector's optical engine and the camera, the distance from the current projector's optical engine to the checkerboard image plane on the wall is calculated, including the following steps:

[0037] Based on the principle of binocular imaging, the coordinates of multiple corner points of a predefined checkerboard pattern are used. The spatial positions of the corner points of the checkerboard pattern projected onto the wall in the projection optical engine coordinate system are calculated using the pixel coordinates of the corner points captured by the camera. Based on the spatial positions of all the corner points of the checkerboard pattern in the projection optical engine coordinate system, the least squares method is used to fit the spatial plane, resulting in the spatial plane equation Z=AX+BY+C. The distance from the projection optical engine to the checkerboard plane is calculated as abs(C) / sqrt(A*A+B*B+1).

[0038] Where X, Y, and Z represent the three-dimensional coordinates of the corner points of the chessboard in the projection optical engine coordinate system, A, B, and C represent the plane parameters, abs() represents taking the absolute value, and sqrt() represents taking the square root.

[0039] Furthermore, the projector's optical engine projects a checkerboard image with predefined corner coordinates onto the wall. Using the pixel coordinates of the checkerboard corners captured by the camera, and based on the camera's intrinsic parameters calibrated at multiple reference zero-position distances, as well as the extrinsic parameters of the projector and camera, the horizontal rotation angle of the projector relative to the checkerboard image plane on the wall is calculated. This includes the following steps:

[0040] Based on the principle of binocular imaging, the coordinates of multiple corner points of a predefined checkerboard pattern are used. The pixel coordinates of the checkerboard corner points captured by the camera are used to calculate the spatial position of the checkerboard corner points projected onto the wall in the coordinate system of the projection optical engine. The translation of the external parameters of the projector optical engine and the camera is calculated and adapted according to the number of motor steps. The focal length of the projection optical engine is calculated and adapted according to the number of motor steps. Based on the spatial position of all checkerboard corner points in the coordinate system of the projection optical engine, the least squares method is used to fit the spatial plane, and the spatial plane equation Z=AX+BY+C is obtained. The horizontal rotation angle of the projection optical engine relative to the checkerboard plane on the wall is expressed as acos(A / sqrt(A*A+B*B+1))*180 / PI-90.

[0041] A projector is calibrated using the aforementioned projector parameter calibration method.

[0042] Beneficial Effects: This invention discloses a projector parameter calibration method and projector based on multiple distances. First, the intrinsic and extrinsic parameters of the projector's optical engine and camera are calibrated at multiple different distances, obtaining calibration results at multiple distances. This allows for the fusion of calibration parameters from multiple distances to calculate the horizontal angle during actual keystone correction angle calculations, resulting in a more accurate fused horizontal angle. By fusing calibration parameters from multiple distances to calculate the calibration result at the current distance, this invention achieves adaptive calibration of the projector and camera's intrinsic and extrinsic parameters at different distances, improving the accuracy of calibration at different distances and resulting in more accurate horizontal angles and more precise keystone correction. Attached Figure Description

[0043] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0044] Explanation of reference numerals in the attached figures:

[0045] Figure 1 This is a schematic diagram of the main method steps of the present invention;

[0046] Figure 2This is a schematic diagram of the chessboard image of the present invention. Detailed Implementation

[0047] To make the above-mentioned objectives, features, and advantages of this application more apparent and understandable, the specific embodiments of this application are described in detail below with reference to the accompanying drawings. Many specific details are set forth in the following description to provide a thorough understanding of this application. However, this application can be implemented in many other ways different from those described herein, and those skilled in the art can make similar modifications without departing from the spirit of this application. Therefore, this application is not limited to the specific embodiments disclosed below.

[0048] In the description of this application, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", "clockwise", "counterclockwise", "axial", "radial", "circumferential", etc., indicating the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this application.

[0049] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0050] In this application, unless otherwise expressly specified and limited, the terms "installation," "connection," "joining," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components, unless otherwise expressly limited. Those skilled in the art can understand the specific meaning of the above terms in this application according to the specific circumstances.

[0051] Example 1:

[0052] In this embodiment, before calibrating the projector parameters, the following parameters are measured:

[0053] First, the projector's optical engine is positioned at a certain distance from the wall. After the projector autofocuses, let's denote the distance from the front of the projector's optical engine lens to the wall as distance0 (in millimeters). Let this distance be the baseline zero distance. Then, let's denote the width of the projected image after autofocus as width0 (in millimeters), and the position of the front of the optical engine lens relative to the projector's front cover as Location0. Finally, the optical engine resolution of the projector is Image. width and Image height Let the number of steps for the motor at this position be denoted as step0. Then the focal length of the zero-position surface optical engine is F0 = Fx0 = Fy0 = (distance0 / width0) * Image. width .

[0054] Move the projector's optical engine, changing its distance from the wall. After the projector autofocuses, the distance from the front of the projector's lens to the wall is recorded as distance1 (in millimeters). The width of the projected image after autofocus is recorded as width1 (in millimeters). The position of the front of the optical engine lens relative to the projector's front cover is Location1. The optical engine resolution is Image. width and Image height Let step1 be the number of steps the motor takes at this position. Then the focal length of the optical engine at this position is F1 = Fx1 = Fy1 = (distance1 / width1) * Image. width .

[0055] It can be calculated that the rate of change of the projector's focal length with respect to the number of motor steps is:

[0056] ΔFΔstep=(F1-F0) / (step1-step0);

[0057] The rate of change of the extrinsic Z-axis translation between the projector and the camera with respect to the number of motor steps is:

[0058] ΔTzΔstep=(Location1-Location0) / (step1-step0).

[0059] refer to Figure 1 As shown, this embodiment provides a multi-distance projector parameter calibration method, including the following steps:

[0060] At least three reference zero-position distances (distance0) are set. For each reference zero-position distance, the projector is moved so that its optical engine is directly facing the wall, and the optical engine is positioned at a distance from the reference zero-position distance from the wall. A checkerboard pattern is projected onto the wall by the projector's optical engine, and an image of the checkerboard pattern on the wall is captured by the projector's camera. The interval between each checkerboard pattern on the wall is assumed to be length.chess The intrinsic parameters of the camera and the extrinsic parameters of the rotation matrix between the projector's optical engine and the camera are calibrated. The checkerboard image is as follows: Figure 2 As shown.

[0061] In this embodiment, at the reference zero distance, the intrinsic principal point Cx of the projection optical engine is... Projector =Image width / 2, Cy Projector =Image height / 2. The focal length of the internal reference is as described in the relevant parameter measurements before calibration above, F Projector =Fx Projector =Fy Projector =F0. The translation parameters (Tx, Ty, Tz_0) of the extrinsic parameters between the projection optical engine and the camera can be obtained from the structural design parameters.

[0062] The calibration process requires solving the camera's intrinsic parameters (Cx). Camera Cy Camera Fx Camera Fy Camera This involves solving for the rotation matrix of the extrinsic parameters between the projection optical engine and the camera. The intrinsic parameters of the camera are solved iteratively, assuming the camera resolution is Image. width_camera and Image height_camera The camera pixel size is pixel_size (in millimeters), the effective focal length of the camera lens is EFL (in millimeters), and the initial intrinsic parameters of the camera are as follows: (Cx Camera =Image width_camera / 2, Cy Camera =Image height_camera / 2 Fx Camera =Fy Camera =EFL / pixel_size).

[0063] Specifically, in this embodiment, three reference zero-position distances are set: far, middle, and near. For each reference zero-position distance, the intrinsic parameters of the camera and the extrinsic rotation matrix between the projector optical engine and the camera are calibrated. The calibration method includes the following steps:

[0064] Step S1: The projector is positioned directly against the wall at the reference zero distance, projecting a checkerboard pattern image, and the camera captures the corresponding checkerboard pattern image.

[0065] Step S2: Based on the correspondence between the world coordinate system coordinates of the chessboard image and the coordinates of the chessboard corner points detected by the camera in the chessboard image, calculate the homography matrix H from the camera coordinate system to the world coordinate system of the chessboard on the wall.

[0066] Let the chessboard have M rows and N columns. The size of the chessboard image projected on the wall is length_chess, and the world coordinate system coordinates of the chessboard on the wall are (length_chess * i, length_chess * j, 0), where 0 <= i < M and 0 <= j < N. The homography matrix is obtained by calculating with the findHomography function of opencv.

[0067] Step S3: According to the homography matrix H from the camera coordinate system to the world coordinate system of the chessboard on the wall, calculate the rotation matrix from the camera coordinate system to the world coordinate system of the chessboard on the wall, defined as the rotation matrix M from the camera coordinate system to the projection optical machine coordinate system;

[0068] Step S4: According to the internal parameters of the projection optical machine and the distance from the projection optical machine to the wall, calculate the coordinates of the corner points of the chessboard projected on the wall in the projection optical machine coordinate system;

[0069] Step S5: According to the rotation matrix M and the translation vector (Tx, Ty, Tz_0) between the camera coordinate system and the projection optical machine coordinate system, calculate the coordinates of the corner points of the chessboard projected on the wall in the camera coordinate system;

[0070] Step S6: Combine the pixel coordinates of the corner points in the chessboard picture taken by the camera with the coordinates of the corner points of the chessboard on the wall in the camera coordinate system to establish a correspondence, and calculate the optimal camera internal parameters, including the principal point and the focal length, by the least squares method;

[0071] The normalized coordinate representation of the corner point coordinates of the chessboard on the wall in the camera coordinate system is, for 0 <= <n:

[0072] ;

[0073] ;

[0074] where n represents the number of corner points of the chessboard, and represent the normalized coordinates of the three-dimensional coordinates of the th corner point of the chessboard in the camera coordinate system, 、 and represent the three-dimensional coordinates of the th corner point of the chessboard in the camera coordinate system;

[0075] According to the pinhole imaging model, establish a correspondence between the pixel coordinates of the corner points of the chessboard picture taken by the camera and the normalized coordinates of the corner points of the chessboard on the wall in the camera coordinate system:

[0076] ;

[0077] The optimal camera intrinsic parameters (Ci) are calculated using the least squares method based on the correspondence between point pairs. x _Camera, C y _Camera,F x _Camera, F y _Camera);

[0078] Among them, Cx_Camera and C y _Camera represents the principal point coordinates of the camera, F x _Camera and F y _Camera represents the focal length of the camera in the x and y axes.

[0079] Step S7: Based on the intrinsic parameters of the projection optical engine, calculate the intrinsic parameters of the camera, the extrinsic parameters (rotation and translation) of the projection optical engine and the camera, the coordinates of the n corner points of the predefined checkerboard pattern, the pixel coordinates of the checkerboard corner points captured by the camera, calculate the distance from the projection optical engine to the checkerboard plane on the wall, and the horizontal rotation angle (yaw) of the projection optical engine relative to the checkerboard plane on the wall.

[0080] Based on the principle of binocular imaging, the coordinates of multiple corner points of a predefined checkerboard pattern are used. The spatial positions of the corner points of the checkerboard pattern projected onto the wall are calculated in the coordinate system of the projection optical engine using the pixel coordinates of the corner points captured by the camera. Then, based on the spatial positions of all the corner points of the checkerboard pattern in the coordinate system of the projection optical engine, the least squares method is used to fit the spatial plane, resulting in the spatial plane equation Z=AX+BY+C; the distance from the projection optical engine to the checkerboard plane is abs(C) / sqrt(A*A+B*B+1); and the horizontal rotation angle of the projection optical engine relative to the checkerboard plane on the wall is yaw=acos(A / sqrt(A*A+B*B+1))*180 / PI-90.

[0081] Where X, Y, and Z represent the three-dimensional coordinates of the corner points of the chessboard in the projection optical engine coordinate system, A, B, and C represent the plane parameters, abs() represents taking the absolute value, sqrt() represents taking the square root, acos() represents taking the inverse cosine, and PI represents pi.

[0082] Step S8: When the distance from the projector to the checkerboard plane on the wall is within a preset threshold of the known distance from the projector to the wall, and the horizontal rotation angle of the projector relative to the checkerboard plane on the wall is within a preset threshold of the actual angle, determine the current camera intrinsic parameters and the extrinsic rotation matrix between the projector and the camera; otherwise, repeat the above steps.

[0083] When calibrating the projector at the reference zero distance, the principal points of the camera's internal parameters are iteratively combined and traversed at a certain step length, and the abscissa and ordinate of the internal points of the principal parameters traverse all possible combinations based on the step length.

[0084] In this embodiment, the principal points of the camera's internal parameters are iteratively combined and traversed at a certain step length. Let the traversal step length be step. Starting from the initial internal principal point of the camera, the traversal of the principal point Cx is Cx Camera , Cx Camera ±step, Cx Camera ±2*step, Cx Camera ±3*step,......Cx Camera ±T*step, and the traversal of the principal point Cy is Cy Camera , CyCamera±step, Cy Camera ±2*step, Cy Camera ±3*step,......Cy Camera ±T*step, and combine all possible values of the traversal of Cx and the traversal of Cy. Preferably, in this embodiment, T is set according to the actual convergence situation.

[0085] If in step S7, the calculated distance from the projection optical machine to the checkerboard plane and the error of distance0 are within a certain threshold, that is, abs(distance from the projection optical machine to the checkerboard plane - distance0) < threshold_distance; and whether the difference between the horizontal rotation angle of the projection optical machine relative to the wall checkerboard plane and the actual angle of 0 degrees in step S7 is within a certain error threshold, that is, abs(horizontal rotation angle of the projection optical machine relative to the wall checkerboard plane - 0) < threshold_angle; then stop the iteration, and the currently calculated internal parameters of the camera and the external parameter rotation matrix M between the camera and the projection optical machine are used as the final result. If either of the two conditions is not satisfied, continue to iterate the principal points of the camera's internal parameters.

[0086] Preferably, in this embodiment, when calibrating at a distance of 200 mm from the projector to the wall, if the difference between the calculated distance from the projection optical machine to the projection wall and 2000 mm is within 30 mm, it is considered satisfied. The absolute value of the horizontal rotation angle of the projection optical machine relative to the projection wall is within 1.5 degrees, which is considered satisfied.

[0087] When the projector is in use, the projector's optical engine projects a checkerboard image with predefined corner coordinates onto the wall. The camera captures the corner pixel coordinates of the checkerboard image. Based on the camera's intrinsic parameters calibrated at a reference zero distance and the extrinsic rotation matrix between the projector's optical engine and the camera, the distance from the projector's optical engine to the checkerboard image plane on the wall is calculated. In this embodiment, there are three reference zero distances, defined as near, medium, and far calibration distances, specifically 3000mm, 2000mm, and 1000mm. The camera's intrinsic parameters and the extrinsic parameters of the projector's optical engine and camera are calibrated at each of these three distances. In other embodiments of the invention, a distance from the projector's actual placement position is selected as the reference zero distance, along with a closer and a farther reference zero distance.

[0088] In this embodiment, for calibration at near, intermediate, and long distances, the camera's intrinsic parameters calibrated at the middle 2000mm, along with the extrinsic parameters of the projection optical engine and camera, are used to calculate the distance from the projection optical engine to the checkerboard plane on the wall. In actual use, the projector is typically placed at a distance of around 2000mm, corresponding to the optical engine's focusing motor being positioned at the middle of its travel. The motor's focusing parameters minimize vertical nonlinear deviations, resulting in a more accurate distance calculation based on this position parameter. In other embodiments of the invention, a calibration distance closest to the projector's actual placement distance is selected to calculate the distance from the projection optical engine to the checkerboard plane on the wall across all calibration distances.

[0089] Based on the principle of binocular imaging, the coordinates of multiple corner points of the chessboard pattern are predefined. The spatial position of the chessboard corner points projected on the wall in the projection optical engine coordinate system is calculated by using the pixel coordinates of the chessboard corner points captured by the camera. Based on the spatial positions of all chessboard corner points in the projection optical engine coordinate system, the least squares method is used to fit the spatial plane, and the spatial plane equation Z=AX+BY+C is obtained. The distance from the projection optical engine to the chessboard plane is calculated as abs(C) / sqrt(A*A+B*B+1).

[0090] Where X, Y, and Z represent the three-dimensional coordinates of the corner points of the chessboard in the projection optical engine coordinate system, A, B, and C represent the plane parameters, abs() represents taking the absolute value, sqrt() represents taking the square root, acos() represents taking the inverse cosine, and PI represents pi.

[0091] When the distances to the three reference zero positions are obtained for far, middle, and near, the camera intrinsic parameters, projection optical engine and camera extrinsic parameters calibrated by the middle reference zero position are used to calculate the distances of the projection optical engine relative to the checkerboard plane as DistancFar_midparams, DistanceMid_midparams, and DistanceNear_midparams, respectively.

[0092] When used at the current distance, a checkerboard pattern with predefined coordinates of n corner points is projected. The pixel coordinates of the checkerboard corner points captured by the camera are used. The camera intrinsic parameters calibrated with the intermediate reference zero distance, as well as the extrinsic parameters of the projection optical engine and the camera, are used to calculate the distance from the projection optical engine to the checkerboard plane on the wall. The calculation method is the same as above.

[0093] Specifically, in this embodiment, the distance DistanceCurrent_midparams between the current projection optical engine and the checkerboard plane is calculated using the camera intrinsic parameters calibrated at the middle 2000mm, as well as the projection optical engine and camera extrinsic parameters.

[0094] When used at the current distance, the projector optical engine projects a checkerboard image with predefined coordinates of multiple corner points onto the wall. The horizontal rotation angle of the projector optical engine relative to the checkerboard image plane is calculated based on the pixel coordinates of the checkerboard corner points captured by the camera, the camera intrinsic parameters calibrated at multiple reference zero distances, and the extrinsic parameters of the projector optical engine and the camera.

[0095] Based on the principle of binocular imaging, the coordinates of multiple corner points of a predefined checkerboard pattern are used. The pixel coordinates of the checkerboard corner points captured by the camera are used to calculate the spatial position of the checkerboard corner points projected onto the wall in the coordinate system of the projection optical engine. The translation of the external parameters of the projector optical engine and the camera is calculated and adapted according to the number of motor steps. The focal length of the projection optical engine is calculated and adapted according to the number of motor steps. Based on the spatial position of all checkerboard corner points in the coordinate system of the projection optical engine, the least squares method is used to fit the spatial plane, and the spatial plane equation Z=AX+BY+C is obtained. The horizontal rotation angle of the projection optical engine relative to the checkerboard plane on the wall is expressed as acos(A / sqrt(A*A+B*B+1))*180 / PI-90.

[0096] In this embodiment, the horizontal rotation angles calculated based on the calibration parameters of the far, mid, and near distances are defined as yaw_farparams, yaw_midparams, and yaw_nearparams, respectively.

[0097] Specifically, the extrinsic translation Tz of the projection optical engine and camera, and the intrinsic focal length F of the projection optical engine. current The adaptation is calculated based on the number of motor steps, assuming the number of motor steps after autofocus is 1 step. now ;

[0098] The Tz of the extrinsic translation of the projector and camera needs to be calculated and adapted based on the number of motor steps:

[0099] Tz = ΔFΔstep *(step now -step0) + Tz0;

[0100] The internal reference focal length of the projection optical machine also needs to be calculated and adapted according to the motor steps:

[0101] F current = ΔFΔstep*(ste_ now -step0) +F0。

[0102] Based on the distance from the current projection optical machine to the wall chessboard image plane, calculate and determine the final horizontal rotation angle; when the distance from the current projection optical machine to the wall chessboard image plane is less than the minimum reference zero position distance, determine the horizontal rotation angle calibrated at the minimum reference zero position distance as the final horizontal rotation angle; when the distance from the current projection optical machine to the wall chessboard image plane is greater than the maximum reference zero position distance, determine the horizontal rotation angle calibrated at the maximum reference zero position distance as the final horizontal rotation angle; otherwise, based on the horizontal rotation angles calibrated at the two reference zero position distances that are greater than and less than the distance from the current projection optical machine to the wall chessboard image plane and are the closest, calculate the final horizontal rotation angle.

[0103] In this embodiment, three reference zero position distances of far, middle, and near are selected as 3000mm, 2000mm, and 1000mm respectively. Through the internal parameters of the camera calibrated at the reference zero position distance of 2000mm and the external parameters of the projection optical machine and the camera, the distances of the projection optical machine relative to the chessboard plane are calculated and expressed as DistancFar_midparams, DistanceMid_midparams, DistanceNear_midparams; the horizontal rotation angles calculated respectively from the calibration parameters of the three reference zero position distances of far, middle, and near are expressed as yaw_farparams, yaw_midparams, yaw_nearparams.

[0104] According to the situation of the distance DistanceCurrent_midparams from the current projection optical machine to the chessboard plane, perform the fusion calculation of the final horizontal rotation angle yaw. The method steps are expressed as:

[0105] When DistanceCurrent_midparams>=DistancFar_midparams:

[0106] yaw=yaw_farparams;

[0107] When DistanceMid_midparams<= DistanceCurrent_midparams<DistancFar_midparams:

[0108] ;

[0109] When DistanceNear_midparams < DistanceCurrent_midparams < DistancMid_midparams:

[0110] ;

[0111] When DistanceCurrent_midparams <= DistancNear_midparams:

[0112] yaw = yaw_nearparams.

[0113] In some other embodiments of the present invention, more than three reference zero distances are set, and the final horizontal rotation angle is obtained through the extended calculation by the above method steps.

[0114] In this embodiment, the rectangular correction of the projector is performed through the horizontal rotation angle obtained by fusion calculation.

[0115] Embodiment 2:

[0116] This embodiment provides a projector. The projector completes calibration through the projector parameter calibration method described in Embodiment 1 and realizes trapezoidal correction according to the calibration result.

[0117] The technical features of the above-described embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope described in this specification.

[0118] The above-described embodiments only represent several implementation manners of the present application. Their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the patent application. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application should be subject to the appended claims.

Claims

1. A method for calibrating projector parameters based on multiple distances, characterized in that, The following steps are included: Set at least three reference zero distances. For each reference zero distance, move the projector so that the projector optical engine is facing the wall. The optical engine is at a distance from the wall from the reference zero distance. The projector optical engine projects a checkerboard pattern onto the wall. The camera of the projector takes a picture of the checkerboard pattern on the wall. The camera's intrinsic parameters and the extrinsic parameter rotation matrix between the projector optical engine and the camera are calibrated. When the projector is in use, the projector optical engine projects a checkerboard image with multiple predefined corner coordinates onto the wall. The camera captures the corner pixel coordinates of the checkerboard image. Based on the camera's intrinsic parameters calibrated at a reference zero distance and the extrinsic rotation matrix between the projector optical engine and the camera, the distance from the current projector optical engine to the checkerboard image plane on the wall is calculated. The projector's optical engine projects a checkerboard image with predefined coordinates of multiple corner points onto the wall. The horizontal rotation angle of the projector relative to the checkerboard image plane is calculated using the pixel coordinates of the checkerboard corner points captured by the camera, based on the camera's intrinsic parameters calibrated at multiple reference zero distances, as well as the extrinsic parameters of the projector and the camera. Based on the current distance between the projector and the checkerboard image plane on the wall, the final horizontal rotation angle is calculated and determined. When the current distance between the projector and the checkerboard image plane on the wall is less than the minimum reference zero distance, the horizontal rotation angle calibrated at the minimum reference zero distance is determined as the final horizontal rotation angle. When the current distance between the projector and the checkerboard image plane on the wall is greater than the maximum reference zero distance, the horizontal rotation angle calibrated at the maximum reference zero distance is determined as the final horizontal rotation angle. Otherwise, the final horizontal rotation angle is calculated based on the horizontal rotation angles calibrated at the two closest reference zero distances, which are greater than the current distance between the projector and the checkerboard image plane on the wall and less than the current distance between the projector and the checkerboard image plane on the wall. When DistanceMid_midparams <= DistanceCurrent_midparams < DistancFar_midparams: ； Where yaw represents the final horizontal rotation angle, DistanceMid_midparams represents the distance from the current projection optical engine to the checkerboard plane at the reference zero distance in the middle, DistanceCurrent_midparams represents the distance from the projection optical engine to the checkerboard plane, and DistancFar_midparams represents the distance from the projection optical engine to the checkerboard plane at the far reference zero distance; yaw_farparams and yaw_midparams represent the horizontal rotation angles calculated from the calibration parameters of the far and middle reference zero distances, respectively.

2. The projector parameter calibration method based on multiple distances according to claim 1, characterized in that, For each reference zero-position distance, the projector is moved so that its optical engine faces the wall, and the distance between the optical engine and the wall is the reference zero-position distance. A checkerboard pattern is projected onto the wall by the projector's optical engine, and the checkerboard pattern image on the wall is captured by the projector's camera. The intrinsic parameters of the camera and the extrinsic parameter rotation matrix between the projector's optical engine and the camera are calibrated, including the following method steps: The projector is positioned directly against the wall at the reference zero distance, projecting a checkerboard pattern, and the camera captures the corresponding checkerboard pattern. Based on the correspondence between the world coordinate system coordinates of the chessboard image and the coordinates of the chessboard corner points detected by the camera in the chessboard image, calculate the homography matrix from the camera coordinate system to the world coordinate system of the chessboard on the wall. Based on the homography matrix from the camera coordinate system to the checkerboard world coordinate system on the wall, calculate the rotation matrix from the camera coordinate system to the checkerboard world coordinate system on the wall, and define it as the rotation matrix from the camera coordinate system to the projection optical engine coordinate system. Based on the intrinsic parameters of the projector and the distance from the projector to the wall, calculate the coordinates of the corner points of the checkerboard pattern projected onto the wall in the projector coordinate system. Based on the rotation matrix and translation vector between the camera coordinate system and the projection optical engine coordinate system, the coordinates of the corner points of the chessboard grid projected onto the wall in the camera coordinate system are calculated. By combining the pixel coordinates of the corner points in the checkerboard image captured by the camera with the coordinates of the corner points of the checkerboard on the wall in the camera coordinate system, the optimal camera intrinsic parameters, including the principal point and focal length, are calculated by the least squares method. Based on the intrinsic parameters of the projector, the intrinsic parameters of the camera, the extrinsic parameters of the projector and the camera, the coordinates of multiple corner points of the predefined checkerboard pattern, the pixel coordinates of the corner points of the checkerboard captured by the camera, the distance from the projector to the checkerboard plane on the wall, and the horizontal rotation angle of the projector relative to the checkerboard plane on the wall are calculated. When the distance from the projector to the checkerboard plane on the wall is within a preset threshold of the known distance from the projector to the wall, and the horizontal rotation angle of the projector relative to the checkerboard plane on the wall is within a preset threshold of the actual angle, determine the current camera intrinsic parameters and the extrinsic rotation matrix between the projector and the camera; otherwise, repeat the above steps.

3. The projector parameter calibration method based on multiple distances according to claim 2, characterized in that, The camera intrinsic point is iteratively combined and traversed according to a certain step size, and the x and y coordinates of the intrinsic point are traversed based on the step size to traverse all possible combinations.

4. The projector parameter calibration method based on multiple distances according to claim 2, characterized in that, Based on the correspondence between the world coordinate system coordinates of the checkerboard image and the coordinates of the checkerboard corner points detected by the camera in the checkerboard image, the homography matrix from the camera coordinate system to the world coordinate system of the checkerboard on the wall is calculated, including the following steps: Let the chessboard grid be M rows and N columns, the size of the chessboard grid image projected onto the wall be length_chess, and the world coordinates of the chessboard grid on the wall be (length_chess* i , length_chess* j, 0). The homography matrix is ​​calculated using the findHomography function of OpenCV.

5. The projector parameter calibration method based on multiple distances according to claim 2, characterized in that, By combining the pixel coordinates of the corner points in the checkerboard image captured by the camera with the coordinates of the checkerboard corner points on the wall in the camera coordinate system, the optimal camera intrinsic parameters are calculated using the least squares method, including the following steps: The normalized coordinates of the corner points of the checkerboard pattern on the wall in the camera coordinate system are expressed as follows: for 0 <= <n: ； ； Where n represents the number of corner points on the chessboard. and Indicates the first Normalized coordinates of the three-dimensional coordinates of each corner point of the chessboard grid in the camera coordinate system. , and Indicates the first The three-dimensional coordinates of each corner point of the chessboard grid in the camera coordinate system; Based on the pinhole imaging model, the pixel coordinates of the corner points of the checkerboard image captured by the camera. Establish a correspondence between the normalized coordinates of the checkerboard corner points on the wall in the camera coordinate system and the coordinates of the checkerboard. ； The optimal camera intrinsic parameters (Ci) are calculated using the least squares method based on the correspondence between point pairs. x _Camera, C y _Camera,F x _Camera, F y _Camera); Among them, Cx_Camera and C y _Camera represents the principal point coordinates of the camera, F x _Camera and F y _Camera represents the focal length of the camera in the x and y axes.

6. The projector parameter calibration method based on multiple distances according to claim 2, characterized in that, Based on the intrinsic parameters of the projection optical engine, the intrinsic parameters of the camera, the extrinsic parameters of the projection optical engine and the camera, the coordinates of multiple corner points of the predefined checkerboard pattern, the pixel coordinates of the checkerboard corner points captured by the camera, the distance from the projection optical engine to the checkerboard plane on the wall, and the horizontal rotation angle of the projection optical engine relative to the checkerboard plane on the wall are calculated, including the following steps: Based on the principle of binocular imaging, the coordinates of multiple corner points of a predefined checkerboard pattern are used. The spatial positions of the corner points of the checkerboard pattern projected onto the wall are calculated in the coordinate system of the projection optical engine using the pixel coordinates of the corner points captured by the camera. Then, based on the spatial positions of all the corner points of the checkerboard pattern in the coordinate system of the projection optical engine, the least squares method is used to fit the spatial plane, resulting in the spatial plane equation Z=AX+BY+C; the distance from the projection optical engine to the checkerboard plane is abs(C) / sqrt(A*A+B*B+1); and the horizontal rotation angle of the projection optical engine relative to the checkerboard plane on the wall is yaw=acos(A / sqrt(A*A+B*B+1))*180 / PI-90. Where X, Y, and Z represent the three-dimensional coordinates of the corner points of the chessboard in the projection optical engine coordinate system, A, B, and C represent the plane parameters, abs() represents taking the absolute value, sqrt() represents taking the square root, acos() represents taking the inverse cosine, and PI represents pi.

7. The projector parameter calibration method based on multiple distances according to claim 1, characterized in that, Three reference zero-position distances are selected. For each reference zero-position distance, the intrinsic parameters of the camera and the extrinsic parameter rotation matrix between the projector optical engine and the camera are calibrated.

8. The projector parameter calibration method based on multiple distances according to claim 1, characterized in that, The projector's optical engine projects a checkerboard image with predefined corner coordinates onto the wall. The camera captures the corner pixel coordinates of the checkerboard image. Based on the camera's intrinsic parameters calibrated at a reference zero-position distance and the extrinsic rotation matrix between the projector and the camera, the distance from the current projector to the checkerboard image plane on the wall is calculated, including the following steps: Based on the principle of binocular imaging, the coordinates of multiple corner points of a predefined checkerboard pattern are used. The spatial positions of the corner points of the checkerboard pattern projected onto the wall in the projection optical engine coordinate system are calculated using the pixel coordinates of the corner points captured by the camera. Based on the spatial positions of all the corner points of the checkerboard pattern in the projection optical engine coordinate system, the least squares method is used to fit the spatial plane, resulting in the spatial plane equation Z=AX+BY+C. The distance from the projection optical engine to the checkerboard plane is calculated as abs(C) / sqrt(A*A+B*B+1). Where X, Y, and Z represent the three-dimensional coordinates of the corner points of the chessboard in the projection optical engine coordinate system, A, B, and C represent the plane parameters, abs() represents taking the absolute value, and sqrt() represents taking the square root.

9. A method for calibrating projector parameters based on multiple distances according to claim 6, characterized in that, The projector's optical engine projects a checkerboard image with predefined corner coordinates onto a wall. Using the pixel coordinates of the checkerboard corners captured by a camera, and based on calibrated camera intrinsic parameters at multiple reference zero-position distances, as well as extrinsic parameters of the projector and camera, the horizontal rotation angle of the projector relative to the checkerboard image plane on the wall is calculated. This process includes the following steps: Based on the principle of binocular imaging, the coordinates of multiple corner points of a predefined checkerboard pattern are used. The pixel coordinates of the checkerboard corner points captured by the camera are used to calculate the spatial position of the checkerboard corner points projected onto the wall in the coordinate system of the projection optical engine. The translation of the external parameters of the projector optical engine and the camera is calculated and adapted according to the number of motor steps. The focal length of the projection optical engine is calculated and adapted according to the number of motor steps. Based on the spatial position of all checkerboard corner points in the coordinate system of the projection optical engine, the least squares method is used to fit the spatial plane, and the spatial plane equation Z=AX+BY+C is obtained. The horizontal rotation angle of the projection optical engine relative to the checkerboard plane on the wall is expressed as acos(A / sqrt(A*A+B*B+1))*180 / PI-90.

10. A projector, characterized in that, The projector is calibrated using the projector parameter calibration method described in any one of claims 1-9.

Images