An improved calibration method based on a grating projection measurement system

Abstract

An improved calibration method for a grating projection measurement system. In the present invention, the method for solving model parameters in a traditional grating projection measurement system is improved. Specifically, the four-step phase-shifting method is used to solve the phase difference, and the model parameters are solved through the proposed mathematical method and finally substituted into the phase-shift height model to realize the three-dimensional reconstruction of the object. This experiment is divided into two parts: mask topography reconstruction and metal block measurement experiment. The present invention proposes a method for extracting the expressions of the projection plane and the camera optical center in the world coordinate system during the projector calibration process and the single-object calibration process of the camera without the help of external objects, and directly calculating the distance between the two optical centers of the projector and the camera and the vertical distance from the projector optical center (CCD camera optical center) to the reference plane. This not only simplifies the cumbersome process and poor operability problems in the system calibration process, but also realizes the accuracy of the calibration process.

Description

Technical Field

[0001] The present invention relates to a calibration method for a traditional phase-shift height model in phase-shifting profilometry, and particularly to an improved calibration method for the distance between the optical centers of a projector and a camera and the vertical distance from the optical center of the projector (optical center of the CCD camera) to a reference plane. Background Art

[0002] Three-dimensional reconstruction has always been one of the important research fields in computer vision. In recent years, great progress has been made in the research on three-dimensional contour reconstruction technology, and new technologies and new methods have emerged continuously. Non-contact three-dimensional surface imaging technology has developed rapidly under the stimulation of market demands in industries, medical applications, and scientific research. Existing reconstruction systems can be roughly divided into: optical triangulation method, laser interferometry, time-of-flight method, structured light contour technology, etc.

[0003] Phase measurement profilometry (PMP) is a well-known optical metrology method based on active stereo vision. Since the 1980s, German optical researchers have proposed phase-shifting profilometry to measure the depth of an object. With the development and wide application of low-cost digital cameras and projectors, it has the characteristics of high precision, non-destructive, full-field, non-contact, etc., and has been widely used in machine vision, industrial inspection, contour modeling, biomedicine and other fields. A typical phase measurement contour system consists of a projector and a camera. The process is that the projector projects a set of grating stripes with a certain phase and periodic change onto the object to be measured. The grating stripes are modulated into deformed grating stripes according to the height change of the surface of the object to be measured. The camera captures the image of the modulated deformed grating stripes. After image processing, the phase information of the grating stripes can be obtained, and then the pixel coordinates of each point on the surface of the object to be measured in the image coordinate system of the projector can be obtained through decoding; according to the mapping relationship between the three-dimensional height (shape) and the phase, also known as phase-height mapping, finally the surface contour or deformation of the object is reconstructed in the image. Although the PMP measurement system has been widely used, there are some uncertainties in actual applications and measurements, and these uncertainties may further lead to large errors in shape construction. These uncertainties include that both the projector and the camera will generate noise; optical distortion and interference caused by light, etc.; the most important uncertainty is that the traditional phase-height mapping model may have a medium bid to build a grating projection measurement system. The projector and the camera must meet strict position constraint conditions, and the optical center of the camera and the center of the projection lens are respectively imaginary space points, and their specific positions cannot be accurately located in actual operation, which brings great errors to the experiment.

[0004] To address these uncertainties, many researchers have made efforts in recent years. Wang Zhenzhou from the State Key Laboratory of Robotics, Shenyang Institute of Automation, Chinese Academy of Sciences, proposed a calibration method for a three-dimensional surface imaging system based on multi-frequency phase-shifting profilometry with angle and pattern modeling, using a modeling method to remove noise in the calibration stage. Zhang Xu et al. discussed the internal relationship between two height models, namely geometric transformation and matrix transformation, in the PMP method, and explicitly explained the coefficients of RPHM and APHM using structure constants. Jong-Chol Kang et al. proposed a phase-height model with nine coefficients and a method to remove radial distortion of the camera lens. Jin Yong from the State Key Laboratory of Electronic Measurement Technology proposed an online measurement method based on digital grating projection and derived a mathematical model to describe the relationship between the degree of optical distortion and the phase of the fringe image. Zhou et al. proposed a connection line between phase-height models, where the exit pupil of the projector and the entrance pupil of the incident light are parallel to the reference plane, and the two optical axes of the CCD camera and the projector and the camera intersect at the same point on the reference plane. This model considers the linear relationship between the reciprocal of the phase difference and the out-of-plane height, but has not precisely established the optical structure of the PMP system. Mao et al. discussed that the connection line between the optical center of the projector and the optical center of the CCD camera does not have to be parallel to the reference plane, and the two optical axes do not have to intersect at the same point on the reference plane. Q. Ma et al. proposed a new phase-height model in comparison with the traditional PMP system, breaking the strict position constraint conditions for the projector and the camera in the traditional model. X. Bian et al. also recognized the limitations of the traditional model and proposed a new phase-height model in the generalized PMP system. For the case where the two pupil heights of the projection and imaging systems are not equal and the corresponding axes are not collinear, the experimental operability was improved by adding the rotation angle.

[0005] The above researchers have conducted a large number of experimental explorations on the phase-height model, all of which are improvements and breakthroughs in the restrictive conditions of the camera and the projector in the traditional model. For example, the deflection angle is added to relax the vertical and parallel conditions of the traditional phase-height model, or a multi-coefficient model is proposed to break the problem of the position arrangement of the camera and the projector. However, the introduction of these parameters has increased the complexity of the model. Due to the simplicity of the traditional model in calculation, the traditional model still has certain practical significance.

[0006] In the construction of the grating projection measurement system, since the optical center of the camera and the center of the projection lens are imaginary spatial points and their specific positions cannot be accurately located in actual operation, the method of using instruments for measurement has a large error. Summary of the Invention

[0007] The object of the present invention is to solve the problem of inaccurate solution of system parameters in traditional PMP systems, and to provide an improved calibration method based on a grating projection measurement system. Under the position conditions of the traditional model, the method proposes a strict method for solving system parameters, alleviates the problem of low measurement accuracy caused by improper solution of system parameters in the past, and ensures the accuracy of the model.

[0008] Under the basis of the traditional model, the restrictive conditions for the model remain unchanged in the present invention. Generally, there are two methods to obtain the vertical distance from the camera optical center to the projected object surface and the distance between the camera optical center and the projector optical center: 1. It is obtained by measuring with an optical precision instrument, but it is very difficult to control in the actual operation process; 2. It is through an implicit calibration method, setting a proportionality coefficient, and by giving a precision gauge block with a fixed height to obtain the calibration parameters. However, the proportional relationship in reality is not only a linear relationship. To solve this problem, the present invention proposes a method to calculate the distance from the camera optical center to the projection surface and the distance between the camera optical center and the projector optical center according to mathematical methods from the external parameters obtained in the process of single-object calibration of the camera and calibration of the projector.

[0009] To achieve the above object, the technical solution adopted by the present invention is as follows:

[0010] An improved calibration method based on a grating projection measurement system, the method specifically being:

[0011] Step 1: Introduce the parameters in the traditional phase-shift - height model

[0012]

[0013] Equation (1) is the height - phase mapping relation formula of a classic grating projection measurement system. Among them, L is the vertical distance from the projector optical center or the CCD camera optical center to the reference plane, D is the distance between the two optical centers of the projector and the camera, λ0 is the pitch of the grating lines, which is the pixel change value corresponding to a phase change of one period (2π) along the X-axis direction on the reference plane after the grating is projected onto the reference plane;

[0014] Step 2: Solve the parameter L

[0015] In the process of camera imaging and calibration, it involves the world coordinate system (O W , X W , Y W , Z W ), the camera coordinate system (O C , X C , Y C , Z C ), the T image physical coordinate system (O', x, y) and the image pixel coordinate system (O", u, v). The imaging process is also a series of transformation processes of the spatial object points in these four coordinate systems;

[0016] During the calibration process, the spatial attitude of the target plane can be arbitrarily transformed. When the target plane is restricted within the reference plane, the target plane becomes the reference plane. Then, L can be regarded as the distance between the optical center of the camera and the world coordinate plane. The spatial position relationship between the world coordinate system and the camera coordinate system is determined according to the external parameters. Camera calibration is to collect the position and internal parameters of the camera in space to establish the mapping relationship between the pixel points on the camera plane and the imaging plane;

[0017] During the calibration process, the vertical distance L between the optical center of the camera and the reference plane can be converted into the vertical distance between the optical center of the camera and the checkerboard calibration plane, that is, L is the length of the Z value of the optical center of the CCD camera in the world coordinate system; The conversion process from the world coordinate system to the camera coordinate system can be achieved through the rotation matrix R and the translation vector t; The conversion formula is:

[0018]

[0019] It can be obtained from Equation 2:

[0020]

[0021] When X c , Y c , Z c are all 0, calculate the absolute value of Z w using Equation (3), that is, the vertical distance L between the optical center of the camera and the reference plane;

[0022] Step 3: Solve for parameter D

[0023] In the binocular camera model, the spatial positions of the left and right cameras are related as follows, where P l and P r are the coordinates of the left and right cameras in the world coordinate system respectively, and R p and T p are the rotation and translation matrices of the right camera relative to the left camera;

[0024] P r = R p * P l + T p

[0025] The external parameters R p and T p are for the projector relative to the camera; Among them, the rotation matrix R p is in the form of The translation vector T p is in the form of When the line connecting the optical centers between the CCD camera and the projector is parallel to the reference plane, through the translation vector Tp The distance D is calculated, and the calculation formula is as follows:

[0026]

[0027] The beneficial effects of the present invention compared with the prior art are as follows: The present invention proposes a method for extracting the expressions of the projection plane and the camera optical center in the world coordinate system during the projector calibration process and the camera single-target calibration process without the help of external objects, and directly calculating the distance between the two optical centers of the projector and the camera and the vertical distance from the projector optical center (CCD camera optical center) to the reference plane. This method not only simplifies the cumbersome process and poor operability in the system calibration process but also realizes the accuracy of the calibration process. Brief Description of the Drawings

[0028] Figure 1 is a model diagram of a classical grating projection measurement system;

[0029] Figure 2 is a coordinate system conversion diagram of a camera linear model;

[0030] Figure 3 is a conversion model diagram of the world coordinate system and the camera coordinate system;

[0031] Figure 4 is a model diagram of a camera and a projector;

[0032] Figure 5 is an experimental picture of the four-step phase-shifting method;

[0033] Figure 6 is a process diagram of mask phase unwrapping;

[0034] Figure 7 is a three-dimensional model diagram of a mask and a model diagram;

[0035] Figure 8 is a point cloud diagram of a metal block;

[0036] Figure 9 is the upper and lower plane graphs fitted by the point cloud of the metal block. Detailed Embodiments

[0037] The technical solutions of the present invention will be further described below in conjunction with the drawings and embodiments, but are not limited thereto. Any modification or equivalent replacement of the technical solutions of the present invention without departing from the spirit scope of the technical solutions of the present invention shall be covered within the protection scope of the present invention.

[0038] The present invention improves the method for solving model parameters in a traditional grating projection measurement system. Specifically, the four-step phase-shift method is adopted to solve the phase difference. The model parameters are solved through the proposed mathematical method and finally substituted into the phase-shift height model to realize the three-dimensional reconstruction of the object. This experiment is divided into two parts: mask topography reconstruction and metal block measurement experiment.

[0039] Build a grating projection measurement system to meet the limiting conditions of the traditional phase-shift - height model. First, carry out the system calibration process. Place the camera and the projector in appropriate positions, and vertically place the reference plane plate directly in front of the projector. After the connection is completed, the first step: carry out the single-object calibration process of the camera and record the obtained data to calculate the vertical distance L from the camera optical center to the reference plane (innovation point). The second step: turn on the projector to carry out the calibration process of the projector. Stick the calibration board on the reference plane plate and project the checkerboard on the reference plane plate to obtain an appropriate number of photos to calculate the distance D from the camera optical center to the projector optical center. The third step: project the grating on the reference surface with phase shifts of 0, π / 2, π, and 3π / 2 respectively. After placing the object to be reconstructed in the grating area projected on the plane, use the camera to capture the deformed grating fringes modulated by the object to be measured and calculate the phase difference. Substitute the obtained vertical distance L from the camera optical center to the reference plane, the distance D from the camera optical center to the projector optical center, and the phase difference into the phase-shift - height formula to calculate the height value and complete the experiment.

[0040] Specific implementation manner 1: The method recorded in this implementation manner is an improved calibration method based on a grating projection measurement system. The method is specifically as follows:

[0041] Step 1: Introduce the parameters in the traditional phase-shift - height model.

[0042]

[0043] Equation (1) is the height-phase mapping relation of a classical grating projection measurement system. Here, L is the vertical distance from the optical center of the projector or the optical center of the CCD camera to the reference plane, D is the distance between the two optical centers of the projector and the camera, and λ0 is the pitch of the grating lines. After the grating is projected onto the reference plane, along the X-axis direction on the reference plane, it is the pixel change value corresponding to a phase change of one period (2π); L, D, and λ0 are system parameters, which are obtained through calibration. When building a grating projection measurement system, the projector and the camera must meet strict position constraint conditions. However, since the optical center of the camera and the center of the projection lens are imaginary spatial points, their specific positions cannot be accurately located in actual operation. In previous calibration processes, generally, the measurement method of precision instruments is used. In order to obtain more accurate measurement results, the present invention proposes a method for solving L and D through the coordinate system space conversion relationship in the monocular CCD camera calibration and the camera-projector calibration process.

[0044] Step 2: Solve the parameter L

[0045] In the process of camera imaging and calibration, it involves the world coordinate system (O W , X W , Y W , Z W ), the camera coordinate system (O C , X C , Y C , Z C ), the T image physical coordinate system (O’, x, y) and the image pixel coordinate system (O”, u, v). The imaging process is also a series of transformation processes of spatial object points in these four coordinate systems. As Figure 2 shown is the schematic diagram of the coordinate system transformation in the camera calibration process;

[0046] In the calibration process, the spatial attitude of the target plane can be arbitrarily changed. When the target plane is restricted within the reference plane, the target plane is the reference plane. Then L can be regarded as the distance between the optical center of the camera and the world coordinate plane. As Figure 4 shown, according to the external parameters, the spatial position relationship between the world coordinate system and the camera coordinate system is determined. Camera calibration is to collect the position and internal parameters of the camera in space to establish the mapping relationship between the pixel points on the camera plane and the imaging plane. The present invention selects the Zhang Zhengyou calibration method and uses the Bouguet Camera Calibration Toolbox for monocular camera calibration;

[0047] During the calibration process, the vertical distance L between the camera optical center and the reference plane can be converted into the vertical distance between the camera optical center and the checkerboard calibration plane, that is, L is the length of the Z value of the CCD camera optical center in the world coordinate system; the conversion process from the world coordinate system to the camera coordinate system can be achieved through the rotation matrix R and the translation vector t; the conversion formula is:

[0048]

[0049] From Equation 2, it can be obtained that:

[0050]

[0051] When X c 、Y c 、Z c are all 0, the absolute value of Z w is calculated using formula (3), that is, the vertical distance L between the camera optical center and the reference plane;

[0052] Step 3: Solve for parameter D

[0053] In the binocular camera model, the spatial positions of the left and right cameras are related as follows, where P l and P r are the coordinates of the left and right cameras in the world coordinate system respectively, R p and T p are the rotation and translation matrices of the right camera relative to the left camera;

[0054] P r = R p * P l + T p

[0055] In the measurement model, the projector is usually regarded as a reverse camera, so the model of the projector is the same as that of the camera, but the light is reversed. Therefore, the spatial positions of the CCD camera and the projector can also be represented by R p and T p ;

[0056] The external parameters R p and T p are for the projector relative to the camera; among them, the rotation matrix R p is in the form of The translation vector T p is in the form of When the optical center connection line between the CCD camera and the projector is parallel to the reference plane, the distance D is calculated through the translation vector T p , and the calculation formula is as follows:

[0057]

[0058] Embodiment 2 in detail: In the improved calibration method based on a grating projection measurement system described in Embodiment 1, in step 3, the present invention completes the calibration of the projector by using the secondary development method of the Bouguet Camera Calibration Toolbox proposed by Gabriel Falcao et al. The steps of the projector calibration experiment are as follows: The first step: Perform monocular camera calibration to record the internal and external parameters of the camera; The second step: Project the checkerboard target plane and obtain the corner point data; The third step: Calculate the internal and external parameters of the projector; The fourth step: Calculate the relative position relationship between the projector and the camera according to the external parameters of the projector.

[0059] Example 1:

[0060] In the present invention, after first fixing the positions of the camera and the projector, by actually measuring the vertical distance L from the optical center of the camera to the reference plane and the distance D from the optical center of the camera to the optical center of the projector, the first step of the calibration experiment is started. Select appropriate positions for the camera and the projector, start the monocular calibration and calculate the distance L from the optical center of the camera to the projection plane. The vertical distance L from the monocular camera to the target plane can be calculated. The second step of the calibration process is to calculate the distance D from the optical center of the projector to the optical center of the camera. By using the Projector-Camera Calibration Toolbox, the relative position relationship between the camera and the projector can be obtained, and thus the value of D can be calculated from the external parameters. The third step is to place the mask at the position of the reference plane of the grating projection. Since the surface of the mask is uneven, the projection grating is deformed on its surface, and the phase value at this time can be calculated, as Figure 5 shown in FIG. 6. Finally, the difference in the obtained phase values, that is, the principal phase value obtained, and the distance D from the optical center of the calibrated camera to the optical center of the projector and the distance D from the optical center of the camera to the projection plane are substituted into the formula to calculate the depth information of the object, and the three-dimensional point cloud data of the mask can be obtained. Figure 7 It is the three-dimensional point cloud data reconstructed for the mask. Figure 7 It is the point cloud map and the model map of the mask.

[0061] Example 2:

[0062] In order to further verify the accuracy of this method, the present invention measures a series of rectangular hexahedrons with known heights. The point cloud images of the rectangular hexahedrons are as Figure 8 shown in FIG. According to the point cloud data, the upper and lower planes of the rectangular hexahedron are fitted, as Figure 9 shown in FIG. Calculate the distance between the two planes to obtain the average height of the rectangular block. Using this method, rectangular hexahedrons of 5 mm, 12 mm, and 20 mm are repeatedly measured. For comparison, the results calculated by the traditional measurement method are also given.

[0063] In Table 1, h0 is the known height, h avg is the average height, MAE is the absolute average error obtained from multiple repeated measurements of the same rectangular hexahedron, and the word "traded" is abbreviated as "Tra". The average height (h avg ) and the mean absolute error (MAE) indicate the accuracy of the method. Generally speaking, the average height calculated by this method is closer to the true height than the traditional method, and the MAE value obtained by repeatedly calculating the object under test with this method is lower than that of the traditional method. Therefore, it can be concluded from Table 1 that the measurement method of the system parameters proposed in the present invention is more accurate than the traditional method.

[0064] Table 1 Experimental results of measuring different known heights by different methods

[0065]

Claims

1. An improved calibration method for a grating projection measurement system, characterized in that: The specific method is as follows: Step 1: Introduce the parameters in the traditional phase-shifting height model Equation (1) is the height-phase mapping relationship of a classic grating projection measurement system. Here, L is the vertical distance from the optical center of the projector or the optical center of the CCD camera to the reference plane, D is the distance between the two optical centers of the projector and the camera, and λ0 is the pitch of the grating lines. After the grating is projected onto the reference plane, along the X-axis direction on the reference plane, it is the pixel change value corresponding to a phase change of one cycle (2π). Step 2: Solve for the parameter L In the process of camera imaging and calibration, the world coordinate system (O W , X W , Y W , Z W ), the camera coordinate system (O C , X C , Y C , Z C ), the T image physical coordinate system (O', x, y) and the image pixel coordinate system (O'', u, v) are involved. The imaging process is also a series of transformation processes of spatial object points in these four coordinate systems; During the calibration process, the spatial attitude of the target plane can be arbitrarily changed. When the target plane is restricted within the reference plane, the target plane is the reference plane. Then L can be regarded as the distance between the optical center of the camera and the world coordinate plane. According to the external parameters, the spatial position relationship between the world coordinate system and the camera coordinate system is determined. Camera calibration is to collect the position and internal parameters of the camera in space to establish the mapping relationship between the pixel points on the camera plane and the imaging plane. During the calibration process, the vertical distance L between the optical center of the camera and the reference plane can be converted into the vertical distance between the optical center of the camera and the checkerboard calibration plane, that is, L is the length of the Z value of the optical center of the CCD camera in the world coordinate system. The conversion process from the world coordinate system to the camera coordinate system can be achieved through the rotation matrix R and the translation vector t. The conversion formula is: It can be obtained from Equation 2 that: When X c , Y c , Z c are all 0, calculate the absolute value of Z w using formula (3), which is the vertical distance L between the camera optical center and the reference plane; Step 3: Solve for the parameter D In the binocular camera model, the spatial positions of the left and right cameras are related as follows, where P l and P r are the coordinates of the left and right cameras in the world coordinate system, R p and T p are the rotation and translation matrices of the right camera relative to the left camera; P r = R p * P l + T p External parameter R p and T p is a projector relative to the camera; where the rotation matrix R p is in the form of The translation vector T p is in the form of When the connecting line of the optical centers between the CCD camera and the projector is parallel to the reference plane, the distance D is calculated through the translation vector T p The calculation formula is as follows:

2. An improved calibration method for a grating projection measurement system according to claim 1, characterized in that: In Step 3, the experimental steps for projector calibration are as follows: The first step: Perform monocular camera calibration and record the internal and external parameters of the camera. The second step: Project the checkerboard target plane and obtain the corner point data. The third step: Calculate the internal and external parameters of the projector. The fourth step: Calculate the relative position relationship between the projector and the camera according to the external parameters of the projector.

Images