A camera auto-calibration method for deflection measurement

Abstract

The application discloses a camera automatic calibration method for deflection measurement, and belongs to the technical field of visual measurement, and comprises the following steps: fixing a soft rack on a focusing ring of a camera, and installing a motor with a gear on one side of the camera, so that the focusing ring can be controlled by the motor; adjusting a focal plane position in a preset motor rotor angle range and performing manual camera calibration; collecting training data; and establishing a mapping model of the motor rotor angle and camera internal parameters through Gaussian process regression, so that the camera internal parameters corresponding to different focal plane positions are directly obtained by inputting the rotor angle of the motor into the trained regression model, and manual calibration is no longer needed. The application can solve the problems of complicated operation and difficulty caused by the need to frequently adjust the focal plane position to ensure the precision in deflection measurement, thereby effectively improving the deflection measurement efficiency.

Description

Technical Field

[0001] This invention relates to the field of deflection measurement technology, and in particular to an automatic camera calibration method for deflection measurement. Background Technology

[0002] Deflection measurement is an optical detection method with simple structure, large dynamic range, strong anti-interference ability and high measurement accuracy. In recent years, it has attracted widespread attention and is expected to become an effective solution for low-cost and high-precision optical detection [L. Huang, M. Idir, C. Zuo, and A. Asundi, "Review of phase measuring deflectometry", Optics and Lasers in Engineering 2018; 107:247-257.].

[0003] Since the actual camera used is not an ideal pinhole system, the screen and the workpiece under test cannot be focused by the camera at the same time. Therefore, "angle-position uncertainty" is an inherent problem in deflection measurement. Fortunately, there are many solutions to address the negative impact of screen defocus on measurement accuracy. This has made focusing the camera on the workpiece surface to ensure the positional accuracy of the measurement point the current mainstream solution [YNChen,XCZhang,T.Chen,R.Zhu,L.Ye,W.Lang,“Transition imaging phase measuring deflectometry for high-precision measurement of optical surfaces,”Measurement 2022;199:111589.].

[0004] However, height variations between different workpieces are unavoidable. Furthermore, when the curvature variations between different surfaces to be measured are significant, the relative positions of the camera and workpiece need to be adjusted to ensure the measurement range. This necessitates repeated adjustments to the camera's focal plane position during deflection measurements. Therefore, to ensure the reliability of the camera model, camera calibration is required after each focal plane position adjustment. High-precision camera calibration requires acquiring a dozen or more images of the calibration plate in different poses, which significantly impacts the efficiency of deflection measurements and places high demands on the operator's expertise. Moreover, especially in space-constrained scenarios, such as in-situ deflection measurement systems, ensuring sufficient differences between the various poses of the calibration plate to guarantee the fit of the problem is extremely difficult. Therefore, due to the requirements of camera calibration, high precision and high efficiency in deflection measurements are often mutually exclusive.

[0005] This invention is proposed to address at least partially the problems of high accuracy requirements for camera models in deflection measurements and the impact of camera calibration operations on measurement efficiency. It enables camera intrinsic parameters to be directly output from a mapping model, thereby eliminating cumbersome camera calibration steps and improving measurement efficiency while ensuring measurement accuracy. Summary of the Invention

[0006] To address the problem that camera calibration affects measurement efficiency in existing deflection measurements, the present invention aims to provide an automatic camera calibration method for deflection measurements, thereby at least partially solving the aforementioned problem.

[0007] To achieve the above objectives, the technical solution of the present invention is as follows:

[0008] In a first aspect, the present invention provides an automatic camera calibration method for deflection measurement, wherein the camera is a pinhole camera, and the focusing ring of the camera rotates under the drive of a motor to adjust the position of the focal plane of the camera. The method includes:

[0009] Obtain training samples:

[0010] S1. The camera's focus ring is rotated by a motor to adjust the camera's focal plane position, and the motor rotor angle corresponding to each focal plane position is recorded, as well as several photos of the calibration plate in different poses at each focal plane position.

[0011] S2. For each focal plane position, the camera model is fitted based on several photos of the calibration plate in different poses and Zhang Zhengyou's camera calibration method to obtain the camera intrinsic parameters at each focal plane position.

[0012] The camera model includes a linear part and a nonlinear part. The linear part of the camera model is represented as follows:

[0013]

[0014] The nonlinear part of the camera model, i.e., the distortion model, is represented as:

[0015]

[0016] Where (u,v) are the camera pixel coordinates, λ is the scale factor, P is the position of the 3D point in space, and R and T are the camera extrinsic parameters. d ,v d ) represents the camera pixel after distortion correction, r is the distance from the pixel to the origin of the pixel coordinate system, and c u c v f u f v k1, k2, k3, p1, and p2 are camera intrinsic parameters;

[0017] Constructing a mapping model:

[0018] S3. By setting random hyperparameters, establish the relationship between the motor rotor angle and the camera intrinsic parameter c based on single-output Gaussian process regression. u c v The first mapping relationship between k1, k2, k3, p1, and p2 is established, and the relationship between the motor rotor angle and the camera intrinsic parameter f is established based on multi-output Gaussian process regression. u f v The second mapping relationship between them;

[0019] S4. Using the negative log-marginal likelihood of single-output and multi-output Gaussian process regression as the objective function, optimize the values ​​of the hyperparameters through a gradient-based convex optimization method to obtain a mapping model representing the relationship between the motor rotor angle and the camera intrinsic parameters.

[0020] Camera calibration:

[0021] S5. During deflection measurement, after obtaining the electronic rotor angle, the corresponding camera intrinsic parameters are output based on the mapping model to achieve automatic camera calibration.

[0022] Preferably, in step S3, the f u f v The proportional relationship between them is determined by the inherent pixel size of the calibrated camera, so that by utilizing the f u f v The correlation between them is used as an additional constraint to increase the robustness of the second mapping relationship.

[0023] Preferably, in step S3, using a kernel function based on squared exponent operations and polynomial operations, the correlation k(α,α') between any two sets of input variables α and α' is expressed as:

[0024]

[0025] Here, σ1, σ2, l, and C are hyperparameters in the kernel function. The application of the squared exponent operation enables the extraction of infinite-dimensional features from the training samples, and the application of the polynomial operation improves the performance of the mapping model in the case of small training samples.

[0026] Preferably, a flexible rack or gear ring is fixedly mounted on the focusing ring of the camera, and a gear is correspondingly mounted on the output shaft of the motor.

[0027] The present invention also provides an electronic device, including a memory storing executable program code and a processor coupled to the memory; wherein the processor calls the executable program code stored in the memory to execute the method described above.

[0028] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the method described above.

[0029] The beneficial effects of the present invention by adopting the above technical solution are as follows: the camera calibration using the technical solution of the present invention can achieve camera intrinsic parameter output without additional operation, i.e., camera calibration, thereby achieving efficient camera calibration and meeting the accuracy requirements of deflection measurement, thus effectively improving measurement efficiency without affecting measurement accuracy. Attached Figure Description

[0030] Figure 1 This is a flowchart of the method according to Embodiment 1 of the present invention;

[0031] Figure 2 This is a schematic diagram of the connection structure between the camera and the motor in Embodiment 1 of the present invention;

[0032] Figure 3 This is a schematic diagram of the structure of an electronic device according to Embodiment 2 of the present invention. Detailed Implementation

[0033] The specific embodiments of the present invention will be further described below with reference to the accompanying drawings. It should be noted that these descriptions are for the purpose of aiding understanding the present invention, but do not constitute a limitation thereof. Furthermore, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0034] It should be noted that in the description of this invention, the terms "upper", "lower", "left", "right", "front", "rear", etc., indicate the orientation or positional relationship based on the description of the structure of this invention shown in the accompanying drawings. They are only for the convenience of describing this invention and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0035] The terms "first" and "second" in this technical solution are merely designations for corresponding structures that are identical or similar, or that perform similar functions. They do not represent an arrangement of the importance of these structures, nor do they imply any ranking, comparison of size, or other meaning.

[0036] Furthermore, unless otherwise explicitly specified and limited, the terms "installation" and "connection" should be interpreted broadly. For example, a connection can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium; it can be a connection within two structures. Those skilled in the art can understand the specific meaning of the above terms in this invention by considering the overall concept of the invention and the specific context of the solution.

[0037] Example 1

[0038] An automatic camera calibration method for deflection measurement is disclosed. This method automatically calibrates the camera during the deflection measurement process, thereby quickly outputting camera intrinsic parameters after the camera's focal plane position changes. The camera is configured as a pinhole camera, and the camera's focusing ring rotates under the drive of a motor to adjust the camera's focal plane position. Correspondingly, this method can directly output the camera intrinsic parameters corresponding to different motor rotor angles.

[0039] like Figure 1 As shown, the method provided in this embodiment includes the following steps:

[0040] Obtain training samples:

[0041] S1. The camera's focus ring is rotated by a motor to adjust the camera's focal plane position, and the motor rotor angle corresponding to each focal plane position is recorded. Several photos of the calibration plate in different poses are taken at each focal plane position.

[0042] In this embodiment, the camera used is a JAI SP-20000C-PMCL industrial camera with a resolution of 5120×3840 pixels, a frame rate of 16fps, and a lens focal length of 50mm. The motor used is a NiMotion STM2832B-485-MA-OFS integrated stepper servo motor. The motor rotor angle is calculated based on an integrated 18-bit absolute encoder. The motor's nominal dimensions are 28×28×47mm, allowing for convenient mounting on the side of the camera without affecting its operation. In this embodiment, the transmission between the motor's output shaft and the camera's focusing ring is achieved through a 0.8 module gear and a soft rack (or gear ring), such as... Figure 2 As shown.

[0043] Controlled by a motor, the calibration plate is photographed in different poses at eight different positions when the camera's focal plane is between 6 and 20 times the focal length.

[0044] S2. For each focal plane position, the camera model is fitted based on several photos of the calibration plate in different poses and Zhang Zhengyou's camera calibration method to obtain the camera intrinsic parameters at each focal plane position.

[0045] The camera model includes a linear part and a non-linear part. The linear part of the camera model is represented as follows:

[0046]

[0047] The nonlinear part of the camera model, i.e., the distortion model, is represented as:

[0048]

[0049] Where (u,v) are the camera pixel coordinates, λ is the scale factor, P is the position of the 3D point in space, and R and T are the camera extrinsic parameters. d ,v d ) represents the camera pixel after distortion correction, r is the distance from the pixel to the origin of the pixel coordinate system, and c u c v f u f v k1, k2, k3, p1, and p2 are camera intrinsic parameters.

[0050] In this way, at each focal plane position of the camera, the motor rotor angle and the corresponding camera intrinsic parameters can be obtained.

[0051] Constructing a mapping model:

[0052] S3. By setting random hyperparameters, establish the relationship between the motor rotor angle and the camera intrinsic parameter c based on single-output Gaussian process regression. u c v The first mapping relationship between k1, k2, k3, p1, and p2 is established, and the relationship between the motor rotor angle and the camera intrinsic parameter f is established based on multi-output Gaussian process regression. u f v The second mapping relationship between them.

[0053] Among them, f u f v The proportional relationship between them is determined by the inherent pixel size of the calibrated camera, so that by utilizing the f u f v The correlation between them is used as an additional constraint to increase the robustness of the second mapping relationship.

[0054] In step S3, the Gaussian process regression uses a kernel function based on squared exponential and polynomial operations. Therefore, for any two sets of input variables α and α', the correlation k(α,α') is expressed as:

[0055]

[0056] σ1, σ2, l, and C are hyperparameters in the kernel function. The application of squared exponent operation enables the extraction of infinite-dimensional features from the training samples, while the application of polynomial operation improves the performance of the mapping model in the case of small training samples.

[0057] S4. Using the negative log-marginal likelihood of single-output and multi-output Gaussian process regression as the objective function, the hyperparameter values ​​are optimized through gradient-based convex optimization methods to obtain a mapping model representing the relationship between the motor rotor angle and the camera intrinsic parameters.

[0058] Camera calibration:

[0059] S5. When performing deflection measurement, once the electronic rotor angle is obtained, the corresponding camera intrinsic parameters can be output based on the mapping model to achieve automatic camera calibration.

[0060] During verification:

[0061] A deflection measurement system was constructed using the aforementioned motor-assisted controlled camera and an iPad mini 2 screen with a resolution of 2048×1536 pixels and a pixel size of 0.0784mm. A plane mirror to be measured was placed in the camera's field of view, and the camera's focal plane was adjusted to the plane to be measured.

[0062] On one hand, surface reconstruction is performed based on the camera intrinsic parameters output by the aforementioned mapping model. On the other hand, by capturing 15 images of the checkerboard calibration board, the camera intrinsic parameters corresponding to the focal plane position are calculated using the traditional Zhang Zhengyou method, and surface reconstruction is also performed based on the calculated camera intrinsic parameters. Comparing the reconstructed surface shapes obtained from the two calibration methods, it is found that within a measurement range of 50×50mm, the root mean square values ​​of the surface shape errors obtained based on the two calibration methods are 96.992nm and 97.699nm, respectively, with an accuracy deviation of less than 1nm. This demonstrates that the automatic calibration method provided in this embodiment meets the accuracy requirements for deflection measurement.

[0063] Example 2

[0064] An electronic device, such as Figure 3 As shown, it includes a memory storing executable program code and a processor coupled to the memory; wherein the processor calls the executable program code stored in the memory to execute the method steps disclosed in the above embodiments.

[0065] Example 3

[0066] A computer storage medium storing a computer program, wherein the computer program is executed by a processor to perform the method steps disclosed in the above embodiments.

[0067] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0068] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0069] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0070] The embodiments of the present invention have been described in detail above with reference to the accompanying drawings, but the present invention is not limited to the described embodiments. For those skilled in the art, various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the present invention, and these variations still fall within the protection scope of the present invention.

Claims

1. An automatic camera calibration method for deflection measurement, characterized in that: The camera is a pinhole camera, and the camera's focus ring rotates under the drive of a motor to adjust the position of the camera's focal plane. The method includes: Obtain training samples: S1. The camera's focal plane position is adjusted by rotating the camera's focus ring via a motor, and the motor rotor angle corresponding to each focal plane position is recorded. Several photos of the calibration plate in different poses are also taken at each focal plane position. S2. For each focal plane position, the camera model is fitted based on several photos of the calibration plate in different poses and Zhang Zhengyou's camera calibration method to obtain the camera intrinsic parameters at each focal plane position. The camera model includes a linear part and a nonlinear part. The linear part of the camera model is represented as follows: (Equation 1); The nonlinear part of the camera model, i.e., the distortion model, is represented as: (Equation 2); Where (u, v) are the camera pixel coordinates, λ is the scale factor, P is the position of the 3D point in space, and R and T are the camera extrinsic parameters. d , v d ) represents the camera pixel after distortion correction, r is the distance from the pixel to the origin of the pixel coordinate system, and c u c v f u f v k1, k2, k3, p1, and p2 are camera intrinsic parameters; Constructing a mapping model: S3. By setting random hyperparameters, establish the relationship between the motor rotor angle and the camera intrinsic parameter c based on single-output Gaussian process regression. u c v The first mapping relationship between k1, k2, k3, p1, and p2 is established, and the relationship between the motor rotor angle and the camera intrinsic parameter f is established based on multi-output Gaussian process regression. u f v The second mapping relationship between them; S4. Using the negative log-marginal likelihood of single-output and multi-output Gaussian process regression as the objective function, optimize the values ​​of the hyperparameters through a gradient-based convex optimization method to obtain a mapping model representing the relationship between the motor rotor angle and the camera intrinsic parameters. Camera calibration: S5. During deflection measurement, after obtaining the electronic rotor angle, the corresponding camera intrinsic parameters are output based on the mapping model to achieve automatic calibration of the camera. In step S3, using a kernel function based on squared exponentiation and polynomial operations, the correlation k(α, α') between any two sets of input variables α and α' is expressed as: (Equation 3); Here, σ1, σ2, l, and C are hyperparameters in the kernel function. The application of the squared exponent operation enables the extraction of infinite-dimensional features from the training samples, and the application of the polynomial operation improves the performance of the mapping model in the case of small training samples.

2. The method according to claim 1, characterized in that: In step S3, the f u f v The proportional relationship between them is determined by the inherent pixel size of the calibrated camera, so that by utilizing the f u f v The correlation between them is used as an additional constraint to increase the robustness of the second mapping relationship.

3. The method according to claim 1, characterized in that: A flexible rack or gear ring is fixedly mounted on the focusing ring of the camera, and a corresponding gear is mounted on the output shaft of the motor.

4. An electronic device, characterized in that: The method includes a memory storing executable program code and a processor coupled to the memory; wherein the processor invokes the executable program code stored in the memory to perform the method as described in any one of claims 1-3.

5. A computer-readable storage medium storing a computer program, characterized in that: The computer program is executed by the processor to perform the method as described in any one of claims 1-3.

Images