Camera calibration method and method for restoring three-dimensional shape information of an object

Abstract

The present invention discloses a camera calibration method, including: constructing an orthogonal binary stripe target with periodic features; separating horizontal and vertical phase-shifted stripes by performing Fourier transform on the image, locating the positions of feature points through the phase diagram and extracting the pixel coordinates of the feature points; the present invention also provides a method for restoring three-dimensional topography information of an object; the present invention has strong robustness to camera defocus, is applicable to cameras with a small depth of field, and can expand its original depth of field.

Description

Technical Field

[0001] The present invention relates to the technical field of cameras, in particular to a camera calibration method and a method for restoring three-dimensional shape information of an object. Background Art

[0002] For precision instrument manufacturing and intelligent detection, the three-dimensional detection of complex microstructures and micro-devices is a key task, and the calibration of cameras is the first step in three-dimensional imaging. Currently, mainstream camera calibration methods usually require obtaining clear target images for accurate feature detection, which is then used for camera calibration. However, in the actual working environment, many cameras used for photographing precision parts have a small field of view and a small depth of field, and it is difficult to capture clear images within the limited depth of field. Dual telecentric cameras are widely used in the field of microscopic imaging. Because of their characteristics of constant magnification and small distortion in the depth direction, they are suitable for the measurement of precision instruments. However, their field of view and depth of field are small and difficult to calibrate.

[0003] Camera calibration is a key step in estimating the internal and external parameters of a stereo camera that links two-dimensional images and the three-dimensional world. Currently, it is mainly achieved through calibration targets with feature points. These targets are mainly divided into three categories: 3D targets, 2D targets, and 1D targets. Although 3D targets have high calibration accuracy, they are large in volume, expensive, and have high manufacturing costs, and are not suitable for the measurement and camera calibration in the microscopic small field of view. 1D targets have a simple structure but few feature points, and cannot guarantee calibration accuracy. Therefore, since the Zhang's calibration method based on a 2D calibration board was proposed, the originally complex camera calibration procedure has become simple and convenient. The 2D planar calibration board can be placed in any posture and direction, and this method has been widely used. The commonly used feature points of the 2D calibration board applicable to this method include square corner points, round points, and checkerboard corner points, etc. In addition to establishing a suitable calibration model, the coordinate extraction accuracy of the feature points also directly affects the calibration accuracy. In addition to the above three types of targets, some scholars have also proposed self-calibration methods that do not rely on calibration targets, but they require a large number of corresponding features and complex calculations, and the calibration results are unstable. Since traditional calibration methods directly extract feature points using the gray information of the target image, the image quality greatly affects the calibration accuracy of the camera. Summary of the Invention

[0004] To solve the problems existing in the prior art, the purpose of the present invention is to provide a camera calibration method and a method for restoring three-dimensional shape information of an object. The present invention has strong robustness to camera defocus, is applicable to cameras with a small depth of field, and can expand its original depth of field.

[0005] To achieve the above object, the technical solution adopted by the present invention is: a camera calibration method, including the following steps:

[0006] Step 1: Construct an orthogonal binary stripe target with periodic characteristics;

[0007] Step 2: By performing a Fourier transform on the image, separate the horizontal and vertical phase-shifted stripes from it, locate the positions of the feature points through the phase diagram, and extract the pixel coordinates of the feature points.

[0008] As a further improvement of the present invention, the specific steps of Step 1 are as follows:

[0009] The light intensity of the orthogonal binary stripe target with periodic characteristics is expressed as:

[0010]

[0011] where (X, Y) are the plane world coordinates, a, b1, b2 are constants, p x , p y are the periods in the X and Y directions respectively; φ X0 , φ Y0 is the additional phase value;

[0012] Perform binary encoding on the generated orthogonal stripes, introduce four parameters a1, a2, a3, a4, and through optimized diffusion, control the error diffusion ratio, thereby improving the sine similarity of the stripe pattern. The optimized kernel is expressed as:

[0013]

[0014] where - represents the previously processed pixel, x represents the currently processed pixel, and α1, α2, α3, α4 are the error diffusion ratios for optimization.

[0015] As a further improvement of the present invention, the specific steps of Step 2 are as follows:

[0016] Regard the periodic target as a periodic signal on a 2D plane, and obtain through Fourier transform:

[0017]

[0018] where E 0 is the zero-order spectrum, and represent the kth-order spectrum, E * represents the conjugate operation; f x , f y represent the frequencies in the x and y directions respectively, and Σ represents the summation;

[0019] After Fourier transform, select the main frequency components along the x-axis and y-axis from the spectrum, select a suitable window to separate the fundamental frequency components, and then perform inverse Fourier transform on the separated fundamental frequency components respectively to obtain two wrapped phases in the range of 0 to 2π;

[0020] In the obtained image, the pixel coordinates θ x = 2π and θ y = 2π are regarded as the coordinates of the feature points.

[0021] As a further improvement of the present invention, it further includes optimizing the coordinates regarded as the feature points, specifically as follows:

[0022] In the horizontal and vertical stripes, if the pixel points satisfying |θ x - 2π| < δ and |θ y - 2π| < δ are selected as candidate points, where δ represents the threshold;

[0023] The feature points on the same row or column of the target are on the same straight line. Nonlinear fitting is performed on the candidate points and their surroundings locally, and at the same time, constraint optimization is performed on the feature points on the same row or column of the target;

[0024] The feature points are incorporated into the global optimization, the wrapped phase is unwrapped, and the midpoint of the image plane is set as the starting point of phase unwrapping:

[0025]

[0026] c1 and c2 are constants directly related to the starting point of phase unwrapping; after obtaining the unwrapped phase, the candidate feature points are incorporated into the global linear fitting:

[0027]

[0028] where c vi , c hi (i = 1, 2, 3) are fitting coefficients, and the feature points are constrained to be on the same column or the same row; finally, after simultaneously constraining the local and global optimization of the feature points, the pixel coordinates of the feature points with sub-pixel accuracy are obtained.

[0029] As a further improvement of the present invention, the material of the orthogonal binary stripe target is a chrome-plated glass substrate, the thickness of the chrome-plated glass substrate is 1 mm, and it is formed by a laser lithography processing technology.

[0030] The present invention also provides a method for restoring the three-dimensional shape information of an object, which is implemented by using a camera calibrated by the above-mentioned camera calibration method, and at the same time, the three-dimensional shape information of the object is restored by using a reconstruction method based on phase height mapping.

[0031] As a further improvement of the present invention, the method for restoring the three-dimensional shape information of an object by using a reconstruction method based on phase height mapping is specifically as follows:

[0032] Control a high-precision translation stage to collect fringe structured light images of different reference planes, where the distance d between adjacent reference planes is equal. Obtain the phase map of each plane through phase unwrapping. Adopt the absolute phase height mapping method to fit a polynomial:

[0033]

[0034] In the formula, b0,..., b n are the polynomial coefficients obtained by fitting. Thus, the phase height mapping relationship in the measurement space is obtained. Then, collect the modulated fringes on the surface of the object to be measured, and use the same phase unwrapping algorithm to recover the phase map of the object. Combine the phase height mapping relationship and the calibration parameters to further obtain the three-dimensional shape information of the object to be measured.

[0035] According to the characteristics of the double telecentric camera, the present invention designs an orthogonal binary phase target for the calibration of a high-precision double telecentric camera. The method of the present invention paves the way for improving the calibration of the double telecentric camera and opens up a new direction in its various application research fields. First, according to the characteristics of the small field of view and small depth of field of the double telecentric camera, use the error diffusion optimization algorithm to encode the orthogonal fringes, and generate the orthogonal binary phase target through physical lithography. By doing so, filter out the encoding noise inherent in the encoding itself. The optimized orthogonal fringes have stronger sinusoidality, thus realizing the accurate extraction of feature points. In addition, adopt the method of global constraint to extract feature points, significantly improving the accuracy of feature point extraction. The feasibility of this method is verified through theoretical analysis and experiments.

[0036] The beneficial effects of the present invention are:

[0037] The calibration method of the present invention has strong robustness to the defocus of the camera, is applicable to cameras with small depth of field, and can expand its original depth of field. Comparative experiments with existing methods show that this method has advantages in reducing the reprojection error and calibrating the defocus camera. When the traditional calibration method fails due to severe defocus of the camera, the method of the present invention can still accurately calibrate the camera successfully, and the reprojection error is 0.08155 pixel. These improvements in camera calibration help to promote the development of three-dimensional reconstruction in the microscopic field and improve its accuracy in three-dimensional detection. Description of the Drawings

[0038] Figure 1 is a schematic diagram of the double telecentric lens model in the embodiment of the present invention;

[0039] Figure 2 is a schematic diagram of the orthogonal binary phase target pattern in the embodiment of the present invention;

[0040] Figure 3 is a flow chart of feature point extraction in the embodiment of the present invention;

[0041] Figure 4 It is the flowchart for optimizing feature point extraction in the embodiment of the present invention;

[0042] Figure 5 It is the schematic diagram of phase height mapping principle in the embodiment of the present invention;

[0043] Figure 6 It is the schematic diagram of the experimental device in the embodiment of the present invention;

[0044] Figure 7 It is the schematic diagram of system calibration in the embodiment of the present invention;

[0045] Figure 8 It is the photographed image of different targets in the embodiment of the present invention;

[0046] Figure 9 It is the reprojection error graph in the embodiment of the present invention;

[0047] Figure 10 It is the comparison graph of calibration clarity in the embodiment of the present invention;

[0048] Figure 11 It is the reprojection error graph of different defocus degrees in the embodiment of the present invention;

[0049] Figure 12 It is the schematic diagram of the distance between adjacent feature points in the embodiment of the present invention;

[0050] Figure 13 It is the histogram of the distance error between the checkerboard, dot target and orthogonal binary phase target in the focused and defocused cases in the embodiment of the present invention;

[0051] Figure 14 It is the fringe pattern photographed by the double telecentric camera in the embodiment of the present invention;

[0052] Figure 15 It is the measurement schematic diagram of the standard block in the embodiment of the present invention;

[0053] Figure 16 It is the schematic diagram of one of the reconstruction results in the embodiment of the present invention;

[0054] Figure 17 It is the schematic diagram of another reconstruction result in the embodiment of the present invention. Detailed implementation manners

[0055] The embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0056] Embodiment

[0057] A camera calibration method includes:

[0058] A. Double telecentric camera imaging model:

[0059] Camera calibration has always been a key area and the primary link in the field of computer vision. Due to its constant magnification, small distortion, and relatively small depth of field, the telecentric camera has been widely used in the field of microscopic imaging. As Figure 1 shown, in a double telecentric imaging system, the aperture is placed at the focal point, allowing only light rays parallel to the principal optical axis to enter the telecentric lens on the imaging side. Since the telecentric lens has a constant magnification, for the double telecentric imaging model, the imaging of an object does not change with the distance between the object and the telecentric lens on the object side.

[0060] (X w ,Y w ,Z w ) are the three-dimensional coordinates of the target point P in the world coordinate system, and (x c ,y c ,z c ) are the corresponding three-dimensional coordinates of the object point in the camera coordinate system. According to the camera imaging principle, (u, v) corresponds to the point p(x c ,y c ,z c ) in the image coordinate system. Since the double telecentric lens is based on orthographic projection, the projection of any point p in the image coordinate system can be expressed as:

[0061]

[0062] where m is the fixed magnification of the double telecentric camera, d u and d v represent the pixel sizes in the u and v directions respectively, and (u0, v0) represents the center of the image coordinate system. Through rigid body transformation, the world coordinate system P(X w ,Y w ,Z w ) is transformed into the image coordinate system p(x c ,y c ,z c ), R = {r ij}(i = 1, 2, 3; j = 1, 2, 3) is the rotation matrix, and T = [t x t y t z is the translation vector. Combining the above formula, the mathematical expression of the double telecentric imaging model can be obtained:

[0063]

[0064] Under the double telecentric imaging model, there are mainly two types of lens distortions, namely mirror distortion and tangential distortion. In the research of this embodiment, considering that the tangential distortion is much smaller than the radial distortion, and at the same time the distortion of the double telecentric lens is much smaller than that of the ordinary lens, only the radial distortion is considered. In the double telecentric imaging model, the first three coefficients of the radial distortion are selected. Therefore, the true coordinates with distortion can be expressed as:

[0065] u d = u + u(k1r 2 + k2r 4 + k3r 6 )

[0066] v d = v + v(k1r 2 + k2r 4 + k3r 6 )

[0067] where k1, k2, and k3 are the coefficients of the radial distortion; (u, v) are the ideal image coordinates, and (u d , v d ) are the true image coordinates with distortion.

[0068] Generally, when calibrating a double telecentric camera, six internal parameters and an external parameter matrix [R T] need to be calibrated. Among them, the six internal parameters include three internal parameters of the camera (effective magnification m, distortion center (u0, v0)) and three coefficients (k1, k2, k3) of the radial distortion.

[0069] B. Orthogonal binary phase target pattern:

[0070] In this embodiment, an orthogonal stripe target with periodic characteristics is used, as shown in (a) of Figure 2 , and its light intensity can be expressed as:

[0071]

[0072] where (X, Y) are the plane world coordinates, a, b1, b2 are constants, where a = 1 / 2, b1 = b2 = 1 / 4, p x , p y are the periods in the X and Y directions respectively. φ X0 , φ Y0 is the additional phase value. At the same time, the generated orthogonal stripes are binary encoded, as shown in Figure 2As shown in (b), the purpose of binary encoding the target is to ensure that the target image remains sinusoidal even when out of focus. At the same time, the projection curve, which is very similar to the sine curve, is made smoother and periodic, making it more suitable for Fourier transform. An optimized kernel method based on the traditional Floyd-steinberg dithering method is used. By introducing four parameters a1, a2, a3, a4, and through optimized diffusion, the error diffusion ratio is controlled, thereby improving the sine similarity of the fringe pattern. The optimized kernel can be expressed as:

[0073]

[0074] where - represents the previously processed pixel, x represents the currently processed pixel, and α1, α2, α3, α4 are the optimized error diffusion ratios.

[0075] C. Feature point extraction:

[0076] The periodic target designed in this embodiment can be regarded as a periodic signal in a 2D plane. Therefore, through Fourier transform, we can obtain: as follows:

[0077]

[0078] where E 0 is the zero-order spectrum, and represent the kth-order spectrum, and E * represents the conjugate operation. Figure 3 shows the process of feature point extraction. First, perform Fourier transform on the captured image Figure 3 in (a), and select the main frequency components along the x-axis and y-axis from the spectrum, as shown in Figure 3 in (b), where E 0 represents the zero-frequency component, and represent the fundamental frequency components. Select a suitable window to separate and Then perform inverse Fourier transform on the separated fundamental frequency components respectively to obtain two wrapped phases in the range of 0 to 2π, as shown in Figure 3 in (c). Therefore, as shown in Figure 3 in (e), in the obtained image, the pixel coordinates θ x = 2π and θ y = 2π are regarded as the coordinates of the feature points. However, this only achieves pixel-level accuracy. To achieve sub-pixel accuracy, this embodiment combines the characteristic of small distortion of the dual telecentric camera and proposes a new optimization algorithm. The optimization algorithm is as shown in Figure 4 shown.

[0079] First, among the horizontal and vertical stripes, if the pixel points satisfying |θ x - 2π| < δ and |θ y - 2π| < δ are selected as candidate points, as shown in (a) of Figure 4 , where δ represents an appropriate threshold. Secondly, the feature points on the same row or column of the target are all on the same straight line. Therefore, in this embodiment, not only the candidate points and their surroundings are nonlinearly fitted locally, but also the feature points on the same row or column of the target are constrained and optimized, as shown in (b) of Figure 4 .

[0080] As shown in (a) of Figure 4 , in the first step of this embodiment, a candidate region is determined for each feature point to be extracted. Different from only performing nonlinear fitting around a single feature point, this embodiment takes into account the integrity of the target and incorporates the feature points into the global optimization. First, the wrapped phase is unwrapped. Without loss of generality, the midpoint of the image plane is set as the starting point of phase unwrapping:

[0081]

[0082] As shown in (b) of Figure 4 , c1 and c2 are constants directly related to the starting point of phase unwrapping. After obtaining the unwrapped phase, the candidate feature points are incorporated into the global linear fitting:

[0083]

[0084] where c vi , c hi (i = 1, 2, 3) are fitting coefficients, which constrain the feature points to be on the same column or the same row. Finally, after simultaneously constraining the local and global optimization of the feature points, as shown in (c) of Figure 4 , the pixel coordinates of the feature points with sub-pixel accuracy are obtained.

[0085] D. Phase height mapping:

[0086] For a monocular structured light measurement system, there are generally two reconstruction models. One is to regard the projector as a reverse camera and use the principle of stereo vision as the reconstruction basis; the other is to establish a phase-height mapping relationship to reconstruct the three-dimensional information of the object. However, in the measurement system based on a dual telecentric camera, the imaging model does not conform to the ordinary perspective projection model, and the telecentric imaging model is insensitive to the depth change along the optical axis. Therefore, the reconstruction method based on the principle of stereo vision cannot be directly used in the microscopic three-dimensional measurement system with dual telecentric lenses. The reconstruction method based on phase-height mapping is relatively simple and does not need to consider the imaging model of the system, but it requires a high-precision displacement stage for auxiliary measurement. In this embodiment, the reconstruction method based on phase-height mapping is used to restore the three-dimensional shape information of the object. This method does not require calibration of the projector, but it is necessary to establish the mapping relationship between phase and height in the measurement space first.

[0087] First, control the high-precision translation stage to collect the fringe structured light images of different reference planes, where the distance between adjacent reference planes is equal to d. The system principle is as Figure 5 shown, and the phase map of each plane is obtained by phase unwrapping. The absolute phase-height mapping method is used to fit the polynomial:

[0088]

[0089] where b0,..., b n are the coefficients of the fitted polynomial, and thus the phase-height mapping relationship in the measurement space is obtained. Then, collect the modulated fringes on the surface of the object to be measured, and use the same phase-unwrapping algorithm to restore the phase map of the object. Combine the phase-height mapping relationship and the calibration parameters to further obtain the three-dimensional shape information of the object to be measured.

[0090] To verify the credibility and effectiveness of the method proposed in this embodiment, a set of precise experimental devices is built for physical measurement, as Figure 6 shown.

[0091] As Figure 6As shown in (a) of [reference], the measurement system includes various components, such as a double telecentric camera, an electrically controlled displacement stage, and a projector. To ensure accurate and low-distortion captured images, a Daheng MER2-301-125U3M camera with a resolution of 2048*1536 (pixel size 3.45um) is selected and paired with a high-precision double telecentric lens (magnification: 0.687, object-space depth of field ±0.9mm, object-space working distance 24mm, field of view 10.5mm*7.9mm). Meanwhile, to achieve precise positioning and controlled movement during the measurement, an OptoSigma OSMS80-20ZF-0B electrically controlled displacement stage with an accuracy of 0.1um is used. In the system, a projector equipped with a Texas Instruments DLP4500 chip with excellent performance is used and paired with the double telecentric camera to achieve precise and uniform projection onto the target surface for 3D reconstruction. At the same time, a target is designed for this small-field double telecentric lens, such as Figure 6 shown in (b) of [reference], a customized lithography calibration plate (resolution 2um), where the first row is binary targets with different periods, and the second row contains dot targets and checkerboard targets.

[0092] A. Calibration experiment: Comparative experiment under focusing and defocusing conditions:

[0093] In the first experiment, 50 pictures of different poses were taken with dot targets, checkerboards, and orthogonal binary phase targets placed within the depth of field of the camera. The calibration experiment was repeated multiple times, and 15 pictures were randomly selected for camera calibration in each camera calibration experiment, as Figure 7 shown.

[0094] Figure 8 (a)-(c) of [reference] respectively show the captured pictures of different targets. Among them, since the phase near the image boundary cannot be correctly restored, during the feature point extraction process, the feature points at the boundary are excluded, as Figure 8 shown in (c) of [reference]. For checkerboard and dot targets, the calibration tool [Opencv toolbox] in OpenCV is used to extract feature points and calibrate the camera. Table 1 shows the results of precise calibration of the three targets.

[0095]

[0096] Table 1

[0097] It can be seen from the reprojection error distribution diagram that, as Figure 9 shown, when the camera is in focus, all three targets can calibrate the camera well and obtain relatively small reprojection errors. Among them, the reprojection error obtained by the method proposed in this embodiment is 0.04631 pixel, which is better than the other two targets of checkerboard and dot targets.

[0098] The depth of field of the double telecentric lens used in this embodiment is only ±0.9 mm. Clear pictures can be captured within the depth of field only by controlling the displacement stage, so as to realize camera calibration under the condition of focusing. As Figure 10 shown, due to the small depth of field of the double telecentric lens, a small-angle tilt of the target will cause the picture to be blurred. The traditional dot target and checkerboard show out-of-focus blur, which affects the final calibration result. In the actual working environment, it is very difficult to control the whole target to be within the camera depth of field at the same time, and the calibration method designed in this embodiment solves this problem well. The orthogonal binary phase target has good robustness to out-of-focus blur and can successfully calibrate the camera.

[0099] When studying the calibration results of different targets under out-of-focus conditions, two groups of target images with different degrees of out-of-focus were taken. In order to keep the internal camera parameters constant, the camera was kept fixed during the experiment. For the first group, the target was controlled to be within the range of mild out-of-focus (0.9 mm - 1.2 mm) by using an electric control displacement stage. For the second group, the target was placed at a distance of severe out-of-focus (1.2 - 1.5 mm). The two groups of target pictures were respectively used for camera calibration. Table 2 summarizes the calibration results. In the case of mild out-of-focus, the reprojection error of the checkerboard is 0.74764 pixel, that of the dot target is 0.13899 pixel, and the reprojection error of the method in this embodiment is 0.0784 pixel, which is much smaller than the reprojection error obtained by calibrating the camera using the traditional target. In the case of severe out-of-focus, only the method in this embodiment can successfully calibrate the camera, and it has excellent robustness to image out-of-focus blur. Figure 11 The reprojection error maps obtained by the proposed method and the Zhang's method are shown. It can be seen that in the cases of mild and severe out-of-focus, the reprojection error obtained by calibrating the camera with the method in this embodiment still surrounds the origin, while the checkerboard and dot targets can only successfully calibrate the camera in the case of mild out-of-focus, and the reprojection error changes greatly compared with the case of focusing; in the case of severe out-of-focus of the camera, both the checkerboard and dot targets fail to calibrate, while the method in this embodiment successfully calibrates the camera, and the reprojection error is 0.08155 pixel.

[0100]

[0101] Table 2

[0102] B. Reconstruction experiment:

[0103] In order to verify that the method proposed in this embodiment can also improve the reconstruction accuracy of the object, the feature points of each pose of the three targets in the calibration experiment were reconstructed respectively, and the distance between adjacent feature points was measured at the same time. As Figure 12The orange line shown represents the distance between horizontally adjacent feature points, and the blue line represents the distance between vertically adjacent feature points. According to the experimental setup, a high-precision lithography calibration plate is described, with a manufacturing precision of 0.001 um. When making the target, the interval between feature points on the checkerboard and dot target is designed to be 0.4 mm, and the interval between feature points on the orthogonal binary phase target is designed to be 0.2 mm. The interval between feature points on the orthogonal fringe phase target is extracted and compared with the checkerboard and dot targets in the reconstruction experiment. Reconstruction experiments are carried out respectively within the focusing and defocusing ranges, and the experimental results are shown in Table 3. In the statistical data, the method of this embodiment has achieved better results both in the focusing and defocusing cases. The error distribution diagrams of the feature point spacing reconstructed with the parameters calibrated by the three targets in the focusing and defocusing cases are shown as Figure 13 shown. When the camera is focused, most of the errors measured by the three targets are concentrated around 0. When the camera is defocused and blurred, the spacing error measured by the calibration method proposed in this embodiment is much smaller than that of the other two targets, and most of the errors are still concentrated around 0, indicating that the parameters calibrated by the method of this embodiment are more accurate, have a higher tolerance for defocusing, and better robustness.

[0104]

[0105] Table 3

[0106] To verify that the camera calibrated using the method of this embodiment can still accurately perform three-dimensional reconstruction in the case of defocus blur, the three-frequency four-step phase-shifting method is used to reconstruct a plane at the same position, with the highest frequency using 10 steps. During the phase-height mapping process, phase packets on 8 planes are captured at intervals of 0.2 mm, and the order of the polynomial is set to n = 2, as shown in Figure 14 shown, and a white plane is measured. Figure 14 In (a)-(c) of

[0107] are the fringe images captured by the double telecentric camera, and a lookup table is established for measuring standard blocks, targets, and small objects. Figure 15 shown, standard blocks with heights of 1 mm and a plane with a height of 1.5 mm (flatness of 0.15 um, measurement uncertainty values of 0.06 um and 0.08 um respectively) are measured. The horizontal phase-shifting fringes are projected onto the standard block, and the double telecentric camera synchronously captures the fringe images modulated and deformed by the standard block. Using the standard block-modulated fringe pattern combined with the phase 3D lookup table, 2 standard block point clouds are calculated as shown in Figure 15 in (b) of Figure 15 and (c) of Figure 15The black lines in (b) and (c) provide a more intuitive observation. The error statistics of the plane fitting of the standard block surface for the 1000th row are shown in Table 4. The average absolute error of measuring the 1mm standard block is 0.00044mm, and the variance is 2.1027e-07mm. The average absolute error of measuring the 1.5mm height plane is 0.00061mm, and the variance is 3.4581e-07mm. Therefore, even when the reconstructed plane is outside the depth of field of the camera, the method of this embodiment still has a high measurement accuracy.

[0108]

[0109] Table 4

[0110] To test the performance of the method of this embodiment in measuring the surface topography of small objects in practical applications, the small object in (d) was measured. In total, 18 fringe patterns with different frequencies were projected. One pattern in each frequency is shown in (a)-(c). The physical diagram is presented in (d). The result after phase height mapping is shown in (e), where the unit of the Z-axis is only millimeters, while the units of the X and Y axes are pixels. Combining the internal and external parameters of the calibrated camera, the coordinates in the X and Y directions are converted into physical dimensions, and the unit becomes millimeters in (e). Figure 16 To test the performance of the method of this embodiment in measuring the surface topography of small objects in practical applications, the small object in (d) was measured. In total, 18 fringe patterns with different frequencies were projected. One pattern in each frequency is shown in (a)-(c). The physical diagram is presented in (d). The result after phase height mapping is shown in (e), where the unit of the Z-axis is only millimeters, while the units of the X and Y axes are pixels. Combining the internal and external parameters of the calibrated camera, the coordinates in the X and Y directions are converted into physical dimensions, and the unit becomes millimeters in (e). Figure 16 To test the performance of the method of this embodiment in measuring the surface topography of small objects in practical applications, the small object in (d) was measured. In total, 18 fringe patterns with different frequencies were projected. One pattern in each frequency is shown in (a)-(c). The physical diagram is presented in (d). The result after phase height mapping is shown in (e), where the unit of the Z-axis is only millimeters, while the units of the X and Y axes are pixels. Combining the internal and external parameters of the calibrated camera, the coordinates in the X and Y directions are converted into physical dimensions, and the unit becomes millimeters in (e). Figure 16 To test the performance of the method of this embodiment in measuring the surface topography of small objects in practical applications, the small object in (d) was measured. In total, 18 fringe patterns with different frequencies were projected. One pattern in each frequency is shown in (a)-(c). The physical diagram is presented in (d). The result after phase height mapping is shown in (e), where the unit of the Z-axis is only millimeters, while the units of the X and Y axes are pixels. Combining the internal and external parameters of the calibrated camera, the coordinates in the X and Y directions are converted into physical dimensions, and the unit becomes millimeters in (e). Figure 16 To test the performance of the method of this embodiment in measuring the surface topography of small objects in practical applications, the small object in (d) was measured. In total, 18 fringe patterns with different frequencies were projected. One pattern in each frequency is shown in (a)-(c). The physical diagram is presented in (d). The result after phase height mapping is shown in (e), where the unit of the Z-axis is only millimeters, while the units of the X and Y axes are pixels. Combining the internal and external parameters of the calibrated camera, the coordinates in the X and Y directions are converted into physical dimensions, and the unit becomes millimeters in (e). Figure 16 To test the performance of the method of this embodiment in measuring the surface topography of small objects in practical applications, the small object in (d) was measured. In total, 18 fringe patterns with different frequencies were projected. One pattern in each frequency is shown in (a)-(c). The physical diagram is presented in (d). The result after phase height mapping is shown in (e), where the unit of the Z-axis is only millimeters, while the units of the X and Y axes are pixels. Combining the internal and external parameters of the calibrated camera, the coordinates in the X and Y directions are converted into physical dimensions, and the unit becomes millimeters in (e).

[0111] The second test object is two small pendants. Figure 17 Each caption in (a)–(f) corresponds to the caption in Figure 16 In (f) of Figure 17 the diameter of the pendant is about 2.5mm, which is consistent with the actual situation.

[0112] Generally speaking, the research of this embodiment innovatively designs a target suitable for a dual telecentric camera with high resolution and small field of view and proposes a novel calibration method, which significantly reduces the calibration error and is applicable to the calibration of defocus cameras. By combining the Fourier transform with the designed novel orthogonal binary fringe target, the pixel coordinates of the feature points are accurately extracted, and the camera is successfully calibrated using the defocus blurred image, with a reprojection error of 0.07840pixel. Experiments prove that the proposed method can achieve accurate camera calibration and three-dimensional shape reconstruction in both defocused or focused environments. This method has made great improvements in camera calibration and has a positive promoting effect on promoting three-dimensional reconstruction in the microscopic field and improving the accuracy in three-dimensional detection.

[0113] The above-described embodiments merely represent specific implementation manners of the present invention. The description is relatively specific and detailed, but it should not be construed as a limitation on the scope of the patent for the present invention. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present invention, several modifications and improvements can still be made, and these all fall within the protection scope of the present invention.

Claims

1. A camera calibration method, characterized in that, It includes the following steps: Step 1: Construct an orthogonal binary stripe target with periodic characteristics; The specific content of Step 1 is as follows: The light intensity of the orthogonal binary stripe target with periodic characteristics is expressed as: where (X, Y) are the plane world coordinates, a, b1, b2 are constants, p x , p y are the periods in the X and Y directions respectively; φ X0 , φ Y0 is the additional phase value; Perform binary coding on the generated orthogonal stripes, introduce four parameters a1, a2, a3, a4, and through optimized diffusion, control the error diffusion ratio, so as to improve the sine similarity of the stripe pattern. The optimized kernel is expressed as: Among them, - represents the previously processed pixel, x represents the currently processed pixel, and α1, α2, α3, α4 are the error diffusion ratios for optimization; Step 2: Through Fourier transform of the image, separate the horizontal and vertical phase-shifted stripes, and locate the position of the feature points through the phase diagram and extract the pixel coordinates of the feature points; The specific content of Step 2 is as follows: Regard the periodic target as a periodic signal on a 2D plane, and obtain through Fourier transform: Among them, E 0 is the zero-order spectrum, and represent the k-th order spectrum, and E * represents the conjugate operation; After Fourier transform, select the main frequency components along the x-axis and y-axis from the spectrum, select a suitable window to separate the fundamental frequency components, and then perform inverse Fourier transform on the separated fundamental frequency components respectively to obtain two wrapped phases in the range of 0 to 2π; In the acquired image, the pixel coordinates θ x = 2π and θ y = 2π are regarded as the coordinates of the feature points.

2. The camera calibration method according to claim 1, wherein, It also includes optimizing the coordinates regarded as feature points, specifically as follows: In the vertical and horizontal stripes, if the pixels that satisfy |θ x - 2π| < δ and |θ y - 2π| < δ are selected as candidate points, where δ represents the threshold value; The feature points on the same row or column of the target are on the same straight line. Perform non-linear fitting on the candidate points and their surroundings locally, and at the same time perform constraint optimization on the feature points on the same row or column of the target; Incorporate the feature points into the global optimization, perform phase unwrapping on the wrapped phase, and set the midpoint of the image plane as the starting point of phase unwrapping: c1 and c2 are constants directly related to the starting point of phase unwrapping; after obtaining the unwrapped phase, incorporate the candidate feature points into the global linear fitting: where c vi , c hi (i = 1, 2, 3) are fitting coefficients, which constrain the feature points to be in the same column or the same row; finally, after simultaneously constraining the local and global optimization of the feature points, the pixel coordinates of the feature points with sub-pixel accuracy are obtained.

3. The camera calibration method according to claim 1, characterized in that The material of the orthogonal binary stripe target is a chrome-plated glass substrate, the thickness of the chrome-plated glass substrate is 1 mm, and it is formed by a laser lithography processing technology.

4. A method for restoring three-dimensional shape information of an object, characterized in that, It is realized by using the camera calibrated by the camera calibration method described in any one of claims 1-3, and at the same time, a reconstruction method based on phase-height mapping is used to restore the three-dimensional shape information of the object.

5. The method for restoring the three-dimensional shape information of an object according to claim 4, wherein The specific content of using the reconstruction method based on phase-height mapping to restore the three-dimensional shape information of the object is as follows: Control a high-precision translation stage to collect fringe structured light images of different reference planes, where the distance d between adjacent reference planes is equal, and obtain the phase map of each plane through phase unwrapping Adopt the absolute phase height mapping method to fit a polynomial: where b0, ..., b n are the fitting polynomial coefficients, from which the phase-height mapping relationship in the measurement space is obtained. Then, the modulated fringes on the surface of the object to be measured are collected, and the phase map of the object is recovered using the same phase-unwrapping algorithm. Combining the phase-height mapping relationship and the calibration parameters, the three-dimensional topography information of the object to be measured is further obtained.

Images