Camera calibration method and object three-dimensional shape information recovery method

Abstract

The invention discloses a camera calibration method. The method comprises the following steps: constructing an orthogonal binary stripe target with periodic characteristics; fourier transform is carried out on the image, horizontal and vertical phase shift fringes are separated from the image, the position of a feature point is positioned through a phase diagram, and pixel coordinates of the feature point are extracted; the invention further provides a method for recovering the three-dimensional shape information of the object. The method has high robustness for camera defocusing, is suitable for cameras with small depth of field, and can expand the original depth of field of the cameras.

Description

Technical Field

[0001] The present invention relates to the field of camera technology, and in particular to a camera calibration method and a method for restoring three-dimensional shape information of an object. Background Art

[0002] For precision instrument manufacturing and intelligent detection, three-dimensional detection of complex microstructures and micro devices is a key task, and camera calibration is the first step in three-dimensional imaging. The current mainstream camera calibration method usually requires a clear target image for accurate feature detection, which is then used for camera calibration. However, in actual working environments, many cameras used to shoot precision parts have a small field of view and a small depth of field, making it difficult to capture clear images within a limited depth of field. Bi-telecentric cameras are widely used in the field of microscopic imaging because they have the characteristics of constant magnification and slight distortion in the depth direction, making them suitable for the measurement of precision instruments. However, their field of view and depth of field are small, making them difficult to calibrate.

[0003] Camera calibration is a key step in estimating the internal and external parameters of a stereo camera that links two-dimensional images with the three-dimensional world. Currently, it is mainly achieved through calibration targets with feature points. These targets are mainly divided into three categories: 3D targets, 2D targets, and 1D targets. Although 3D targets have high calibration accuracy, they are large, expensive, and costly to produce, and are not suitable for measurement and camera calibration in the microscopic field of view. 1D targets have a simple structure but few feature points and cannot guarantee calibration accuracy. Therefore, since the Zhang calibration method based on the 2D calibration plate was proposed, the originally complex camera calibration procedure has become simple and convenient. The 2D plane calibration plate can be rotated and placed in any posture and direction, and this method has been widely used. The common feature points of the 2D calibration plate suitable for this method include square corner points, round points, and checkerboard corner points. In addition to establishing a suitable calibration model, the coordinate extraction accuracy of the feature points also directly affects the calibration accuracy. In addition to the above three targets, some scholars have also proposed self-calibration methods that do not rely on calibration targets, but they require a large number of corresponding features and complex calculations, and the calibration results are unstable. Since the traditional calibration method directly uses the grayscale information of the target image to extract feature points, the image quality greatly affects the calibration accuracy of the camera. Summary of the invention

[0004] In order to solve the problems existing in the prior art, the purpose of the present invention is to provide a camera calibration method and a method for restoring the three-dimensional morphology information of an object. The present invention has strong robustness against camera defocus, is suitable for cameras with small depth of field, and can extend their original depth of field.

[0005] To achieve the above object, the technical solution adopted by the present invention is: a camera calibration method, comprising the following steps:

[0006] Step 1, constructing an orthogonal binary fringe target with periodic characteristics;

[0007] Step 2: Separate the horizontal and vertical phase-shifted fringes from the image by Fourier transform, locate the feature point position through the phase map and extract the pixel coordinates of the feature point.

[0008] As a further improvement of the present invention, the step 1 is specifically as follows:

[0009] The light intensity of an orthogonal binary fringe target with periodic characteristics is expressed as:

[0010]

[0011] Among them, (X, Y) is the plane world coordinate, a, b1, b2 are constants, p x ,p y are the periods in the X and Y directions respectively; φ X0 ,φ Y0 is the additional phase value;

[0012] The generated orthogonal stripes are binary-encoded, and four parameters a1, a2, a3, and a4 are introduced. After optimized diffusion, the error diffusion ratio is controlled to improve the sinusoidal similarity of the stripe pattern. The optimized kernel is expressed as:

[0013]

[0014] Among them, - represents the previously processed pixel, x represents the pixel currently being processed, and α1, α2, α3, and α4 are the error diffusion ratios to be optimized.

[0015] As a further improvement of the present invention, the step 2 is specifically as follows:

[0016] Considering the periodic target as a periodic signal in a 2D plane, the following is obtained through Fourier transform:

[0017]

[0018] Among them, E 0 is the zero-order spectrum, and represents the k-th order spectrum, E * represents the conjugate operation; f x ,f y Respectively represent the frequencies in the x and y directions, and Σ represents the sum;

[0019] After Fourier transformation, the main frequency components along the x-axis and the y-axis are selected from the spectrum, and the fundamental frequency components are separated by using a suitable window. Then, the separated fundamental frequency components are subjected to inverse Fourier transformation to obtain two wrapped phases in the range of 0 to 2π.

[0020] In the obtained image, the pixel coordinates θ are x =2π and θ y =2π is regarded as the coordinate of the feature point.

[0021] As a further improvement of the present invention, the coordinates regarded as feature points are optimized as follows:

[0022] In the horizontal and vertical stripes, if |θ x -2π|<δ and |θ y Pixels with -2π|<δ are selected as candidate points, where δ represents the threshold;

[0023] The feature points in the same row or column on the target are on the same straight line, and nonlinear fitting is performed locally on the candidate points and their surroundings, while constrained optimization is performed on the feature points in the same row or column on the target;

[0024] Incorporate the feature points into the global optimization, unwrap the wrapped phase, and set the midpoint of the image plane as the starting point of the phase unwrapping:

[0025]

[0026] c1 and c2 are constants directly related to the starting point of phase unwrapping; after obtaining the unwrapped phase, the candidate feature points are incorporated into the global linear fit:

[0027]

[0028] where c vi ,c hi (i=1, 2, 3) are fitting coefficients, constraining the feature points to be in the same column or row; finally, after simultaneously constraining the local and global optimization of the feature points, the pixel coordinates of the feature points with sub-pixel accuracy are obtained.

[0029] As a further improvement of the present invention, the material of the orthogonal binary stripe target is a chrome-plated glass substrate, the thickness of the chrome-plated glass substrate is 1 mm, and it is formed by a laser photolithography process.

[0030] The present invention also provides a method for restoring the three-dimensional shape information of an object, which is implemented by using a camera calibrated by the camera calibration method as described above, and simultaneously using a reconstruction method based on phase height mapping to restore the three-dimensional shape information of the object.

[0031] As a further improvement of the present invention, the three-dimensional shape information of the object is restored by using a reconstruction method based on phase height mapping as follows:

[0032] Control the high-precision translation stage to collect stripe structured light images of different reference planes, where the distance between adjacent reference planes is equal to d, and the phase map of each plane is obtained by phase decomposition Fitting a polynomial using the absolute phase height mapping method:

[0033]

[0034] Where b0, ..., b n are the fitted polynomial coefficients, from which the phase-height mapping relationship in the measurement space is obtained, and then the modulated fringes on the surface of the object to be measured are collected, and the same phase deconvolution algorithm is used to restore the phase map of the object. The three-dimensional morphology information of the object to be measured is further obtained by combining the phase-height mapping relationship and the calibration parameters.

[0035] According to the characteristics of the dual-telecentric camera, the present invention designs an orthogonal binary phase target for the calibration of the high-precision dual-telecentric camera; the method of the present invention paves the way for improving the calibration of the dual-telecentric camera and opens up new directions in its various application research fields. First, according to the characteristics of the small field of view and small depth of field of the dual-telecentric camera, the error diffusion optimization algorithm is used to encode the orthogonal stripes, and the orthogonal binary phase target is generated by physical lithography. By doing so, the encoding noise inherent in the encoding itself is filtered out, and the optimized orthogonal stripes have stronger sinusoidality, thereby achieving accurate extraction of feature points. In addition, the feature points are extracted by using a global constraint method, which significantly improves the accuracy of feature point extraction. The feasibility of this method is confirmed by theoretical analysis and experiments.

[0036] The beneficial effects of the present invention are:

[0037] The calibration method of the present invention has strong robustness against camera defocus, is suitable for cameras with small depth of field, and can extend their original depth of field. A comparative experiment was conducted with the existing method, which showed that the method has advantages in reducing reprojection errors and defocus camera calibration. When the camera is severely defocused and the traditional calibration method fails to calibrate, the method of the present invention can still accurately calibrate the camera successfully, and the reprojection error is 0.08155pixel. These improvements in camera calibration help promote the development of three-dimensional reconstruction in the microscopy field and improve its accuracy in three-dimensional detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 is a schematic diagram of a double telecentric lens model in an embodiment of the present invention;

[0039] Figure 2 is a schematic diagram of an orthogonal binary phase target pattern in an embodiment of the present invention;

[0040] Figure 3 This is a flow chart of feature point extraction in an embodiment of the present invention;

[0041] Figure 4 This is a flowchart of feature point extraction optimization in an embodiment of the present invention;

[0042] Figure 5 This is a schematic diagram of the phase height mapping principle in an embodiment of the present invention;

[0043] Figure 6 Schematic diagram of an experimental device in an embodiment of the present invention;

[0044] Figure 7 This is a schematic diagram of system calibration in an embodiment of the present invention;

[0045] Figure 8 1 is a photograph of different targets in an embodiment of the present invention;

[0046] Fig. 9 is a reprojection error map in an embodiment of the present invention;

[0047] Fig.10 This is a calibration clarity comparison diagram in an embodiment of the present invention;

[0048] Fig.11 is a reprojection error diagram of different defocus levels in an embodiment of the present invention;

[0049] Fig.12 Schematic diagram of the distance between adjacent feature points in an embodiment of the present invention;

[0050] Fig.13 The histogram of the distance error between the chessboard, the dot target and the orthogonal binary phase target in the focused state and the defocused state in the embodiment of the present invention;

[0051] Fig.14 A fringe pattern captured by a dual telecentric camera in an embodiment of the present invention;

[0052] Fig.15 Schematic diagram of measuring a standard block in an embodiment of the present invention;

[0053] Fig.16 This is a schematic diagram of a reconstruction result in an embodiment of the present invention;

[0054] Fig.17 FIG. 4 is another schematic diagram of a reconstruction result in an embodiment of the present invention. DETAILED DESCRIPTION

[0055] The embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0056] Example

[0057] A camera calibration method, comprising:

[0058] A. Dual telecentric camera imaging model:

[0059] Camera calibration has always been a key area and the primary link in the field of computer vision. Telecentric cameras have been widely used in the field of microscopic imaging due to their constant magnification, slight distortion and relatively small depth of field. Figure 1 As shown in the figure, in the double telecentric imaging system, the aperture is placed at the focus, allowing only light parallel to the main optical axis to enter the telecentric lens on the imaging side. Since the telecentric lens has a constant magnification, for the double telecentric imaging model, the imaging of the object does not change with the distance between the object and the telecentric lens on the object side.

[0060] (X w ,Y w ,Z w ) is the three-dimensional coordinate of the target point P in the world coordinate system, (x c ,y c ,z c ) is the three-dimensional coordinate of the object point in the camera coordinate system. According to the camera imaging principle, (u, v) corresponds to the point p(x c ,y c ,z c ). Since the bi-telecentric lens is based on orthogonal projection, the projection of any point p in the image coordinate system can be expressed as:

[0061]

[0062] Where m is the fixed magnification of the bi-telecentric camera, d u With d v Represents the pixel size in the u and v directions respectively, and (u0, v0) represents the center of the image coordinate system. w ,Y w ,Z w ) is transformed to the image coordinate system p(x c ,y c ,z c ), R = {r ij}(i=1,2,3;j=1,2,3) is the rotation matrix, T=[t x t y t z ] is the translation vector. Combining the above formula, the mathematical expression of the double telecentric imaging model can be obtained:

[0063]

[0064] Under the dual telecentric imaging model, lens distortion mainly has two types of distortion, namely, mirror distortion and tangential distortion. In the study of this embodiment, it is considered that the tangential distortion is much smaller than the radial distortion, and the distortion of the dual telecentric lens is much smaller than that of the ordinary lens, so only the radial distortion is considered. In the dual telecentric imaging model, the first three coefficients of radial distortion are selected, so the real coordinate with distortion can be expressed as:

[0065] u d =u+u(k1r 2 +k2r 4 +k3r 6 )

[0066] v d =v+v(k1r 2 +k2r 4 +k3r 6 )

[0067] Where k1, k2 and k3 are the coefficients of radial distortion; (u,v) is the ideal image coordinate, (u d ,v d ) is the real image coordinate with distortion.

[0068] Usually, when calibrating a dual telecentric camera, six internal parameters and one external parameter matrix [RT] need to be calibrated. The six internal parameters include three internal parameters of the camera (effective magnification m, distortion center (u0, v0)) and three coefficients of radial distortion (k1, k2, k3).

[0069] B. Orthogonal binary phase target pattern:

[0070] This embodiment uses an orthogonal fringe target with periodic characteristics, such as Figure 2 As shown in (a), the light intensity can be expressed as:

[0071]

[0072] Where (X, Y) is the plane world coordinate, a, b1, b2 are constants, where a = 1 / 2, b1 = b2 = 1 / 4, p x ,p y are the periods in the X and Y directions respectively. φ X0 ,φ Y0 is an additional phase value. At the same time, the generated orthogonal fringes are binary encoded, such as Figure 2As shown in (b), the purpose of binary encoding the target is to make the image of the target maintain good sinusoidal properties when it is out of focus. At the same time, the projection curve that is very similar to the sine curve is made smoother and periodic, which is more suitable for Fourier transform. The optimized kernel method based on the traditional Floyd-steinberg dithering method is used. By introducing four parameters a1, a2, a3, and a4, the error diffusion ratio is controlled through optimized diffusion, thereby improving the sinusoidal similarity of the fringe pattern. The optimized kernel can be expressed as:

[0073]

[0074] Among them, - represents the previously processed pixel, x represents the pixel currently being processed, and α1, α2, α3, and α4 are the error diffusion ratios to be optimized.

[0075] C. Feature point extraction:

[0076] The periodic target designed in this embodiment can be regarded as a periodic signal in a 2D plane, so it can be obtained through Fourier transform as shown below:

[0077]

[0078] Among them, E 0 is the zero-order spectrum, and represents the k-th order spectrum, E * Represents the conjugate operation. Figure 3 The process of feature point extraction is shown. First, the captured image Figure 3 (a) in the figure is transformed by Fourier transform, and the main frequency components along the x-axis and the y-axis are selected from the spectrum, as shown in Figure 3 As shown in (b), where E 0 represents the zero-frequency component, and Represents the fundamental frequency component, and selects a suitable window to separate and Then, the separated fundamental frequency components are inverse Fourier transformed to obtain two wrapped phases in the range of 0 to 2π, such as Figure 3 As shown in (c) in . Therefore, Figure 3 As shown in (e), in the obtained image, the pixel coordinates θ are x =2π and θ y =2π is regarded as the coordinate of the feature point. However, this only achieves pixel-level accuracy. In order to achieve sub-pixel-level accuracy, this embodiment combines the small distortion characteristics of the dual telecentric camera and proposes a new optimization algorithm. The optimization algorithm is as follows Figure 4 shown.

[0079] First, in the horizontal and vertical stripes, if |θ x -2π|<δ and |θ y Pixels with -2π|<δ are selected as candidate points, such as Figure 4 As shown in (a) of FIG. 1 , δ represents an appropriate threshold. Secondly, the feature points in the same row or column on the target are all on the same straight line. Therefore, this embodiment not only performs nonlinear fitting on the candidate points and their surroundings locally, but also performs constrained optimization on the feature points in the same row or column on the target. Figure 4 As shown in (b) in .

[0080] like Figure 4 As shown in (a), in the first step, this embodiment determines the candidate region for each feature point to be extracted. Different from performing nonlinear fitting only around a single feature point, this embodiment takes the integrity of the target into account and incorporates the feature point into the global optimization. First, the wrapped phase is unwrapped. Without loss of generality, the midpoint of the image plane is set as the starting point of the phase unwrapping:

[0081]

[0082] like Figure 4 As shown in (b), c1 and c2 are constants directly related to the starting point of phase unwrapping. After obtaining the unwrapped phase, the candidate feature points are incorporated into the global linear fitting:

[0083]

[0084] where c vi ,c hi (i=1,2,3) are fitting coefficients, constraining feature points to be in the same column or row. Finally, after constraining feature points to be optimized locally and globally, Figure 4 As shown in (c) in the figure, the pixel coordinates of the feature points with sub-pixel accuracy are obtained.

[0085] D. Phase height mapping:

[0086] For monocular structured light measurement systems, there are generally two reconstruction models. One is to regard the projector as a reverse camera and use the principle of stereoscopic vision as the basis for reconstruction; the other is to establish a phase-height mapping relationship to reconstruct the three-dimensional information of the object. However, in a measurement system based on a dual-telecentric camera, the imaging model does not conform to the ordinary perspective projection model, and the telecentric imaging model is insensitive to depth changes along the optical axis. Therefore, the reconstruction method based on the principle of stereoscopic vision cannot be directly used in a microscopic three-dimensional measurement system with a dual-telecentric lens. The reconstruction method based on phase-height mapping is relatively simple and does not need to consider the imaging model of the system, but requires a high-precision translation stage to assist in measurement. This embodiment uses a reconstruction method based on phase-height mapping to restore the three-dimensional morphology information of the object. This method does not require calibration of the projector, but it is necessary to first establish a mapping relationship between phase and height in the measurement space.

[0087] First, the high-precision translation stage is controlled to collect stripe structured light images of different reference planes, where the distances between adjacent reference planes are equal to d. The system principle is as follows: Figure 5 As shown, the phase diagram of each plane is obtained by phase decomposition Fitting a polynomial using the absolute phase height mapping method:

[0088]

[0089] Where b0, ..., b n are the fitted polynomial coefficients, from which the phase-height mapping relationship in the measurement space is obtained, and then the modulated fringes on the surface of the object to be measured are collected, and the same phase deconvolution algorithm is used to restore the phase map of the object. The three-dimensional morphology information of the object to be measured is further obtained by combining the phase-height mapping relationship and the calibration parameters.

[0090] In order to verify the credibility and effectiveness of the method proposed in this embodiment, a set of precise experimental equipment was built in this embodiment to perform physical measurements, such as Figure 6 shown.

[0091] like Figure 6As shown in (a), the measurement system includes various components, including a dual telecentric camera, an electrically controlled translation stage, and a projector. In order to ensure that the captured images are accurate and have low distortion, a Daheng MER2-301-125U3M camera (pixel size 3.45um) with a resolution of 2048*1536 was selected, paired with a high-precision dual telecentric lens (magnification: 0.687, object depth of field ±0.9mm, object working distance 24mm, field of view 10.5mm*7.9mm). At the same time, in order to achieve precise positioning and control movement during the measurement process, an OptoSigma OSMS80-20ZF-0B electrically controlled translation stage with an accuracy of 0.1um was used. In the system, a high-performance projector equipped with a Texas Instruments DLP4500 chip is used, paired with a dual telecentric camera to achieve accurate and uniform projection of the target surface and realize three-dimensional reconstruction. At the same time, a target is designed for the small-field dual telecentric lens, such as Figure 6 As shown in (b), a customized photolithography calibration plate (resolution 2um) is provided, wherein the first row is binary targets of different periods, and the second row includes dot targets and checkerboard targets.

[0092] A. Calibration test: Comparative experiment under focusing and defocusing conditions:

[0093] In the first experiment, the dot target, checkerboard, and orthogonal binary phase target were placed within the depth of field of the camera and 50 pictures with different postures were taken. The calibration experiment was repeated many times. Each time the camera calibration experiment randomly selected 15 pictures for camera calibration. Figure 7 shown.

[0094] Figure 8 (a)-(c) show the pictures of different targets. Since the phase near the image boundary cannot be correctly restored, the feature points at the boundary are excluded in the feature point extraction process. Figure 8 As shown in (c) in the figure. For the checkerboard and dot targets, the calibration tool [Opencv toolbox] in OpenCV is used to extract feature points and calibrate the camera. Table 1 shows the results of the precise calibration of the three targets.

[0095]

[0096] Table 1

[0097] From the reprojection error distribution diagram, we can see that Fig. 9 As shown, when the camera is focused, the three targets can calibrate the camera well and obtain a small reprojection error. The reprojection error obtained by the method proposed in this embodiment is 0.04631 pixel, which is better than the two targets of the checkerboard and dot targets.

[0098] The depth of field of the bi-telecentric lens used in this embodiment is only ±0.9mm. Only by controlling the translation stage can a clear image be captured within the depth of field, thus achieving camera calibration under focusing conditions. Fig.10 As shown in the figure, due to the small double telecentric depth of field, the small tilt of the target will cause the image to be blurred, and the traditional dot target and checkerboard will show defocus blur, which will affect the final calibration result. In the actual working environment, it is difficult to control the entire target to be within the camera depth of field at the same time, and the calibration method designed in this embodiment solves this problem well. The orthogonal binary phase target has good robustness against defocus blur and can successfully calibrate the camera.

[0099] In order to study the calibration results of different targets under defocus conditions, two groups of target images with different degrees of defocus were taken. In order to maintain constant in-camera parameters, the camera remained fixed during the experiment. For the first group, the target was controlled to be within the range of slight defocus (0.9mm-1.2mm) using an electric-controlled translation stage. For the second group, the target was placed at a distance of severe defocus (1.2-1.5mm). The two groups of target images were used for camera calibration, and Table 2 summarizes the calibration results. In the case of slight defocus, the checkerboard reprojection error was 0.74764pixel, the dot target was 0.13899pixel, and the reprojection error of the method of this embodiment was 0.0784pixel, which is much smaller than the reprojection error obtained by calibrating the camera using a traditional target. In the case of severe defocus, only the method of this embodiment successfully calibrates the camera, and has excellent robustness to image defocus blur. Fig.11 The reprojection error diagrams obtained by the proposed method and Zhang's method are shown. It can be seen that in the cases of slight and severe defocus, the reprojection error obtained by calibrating the camera using the method of this embodiment is still around the origin, while the checkerboard and dot targets can only successfully calibrate the camera in the case of slight defocus, and the reprojection error changes greatly compared to the focused case; when the camera is severely defocused, both the checkerboard and dot targets fail to calibrate, while the method of this embodiment successfully calibrates the camera with a reprojection error of 0.08155 pixel.

[0100]

[0101] Table 2

[0102] B. Reconstruction experiment:

[0103] In order to verify that the method proposed in this embodiment can also improve the reconstruction accuracy of the object, the feature points of each position of the three targets in the calibration experiment are reconstructed respectively, and the distances between adjacent feature points are measured at the same time. Fig.12The orange line shown is the distance between adjacent feature points in the horizontal direction, and the blue line is the distance between adjacent feature points in the vertical direction. The high-precision photolithography calibration plate used in the experimental setup description has a manufacturing accuracy of 0.001um. When making the target, the feature point spacing on the checkerboard and dot targets is designed to be 0.4mm, and the feature point spacing on the orthogonal binary phase target is designed to be 0.2mm. The interval extraction of the spacing between feature points on the orthogonal stripe phase target is compared with the reconstruction experiment of the checkerboard and dot targets. The reconstruction experiments were carried out in the focus and defocus ranges respectively, and the experimental results are shown in Table 3. In the statistical data, the method of this embodiment achieved better results in both the focus and defocus conditions. The distribution diagram of the feature point spacing error reconstructed with the parameters obtained by calibrating the camera with three targets under the focus and defocus conditions is shown in the figure below. Fig.13 When the camera is focused, most of the errors measured by the three targets are concentrated around 0, while when the camera is out of focus, the calibration method proposed in this embodiment measures a spacing error that is much smaller than that of the other two targets, and most of the errors are still concentrated around 0, indicating that the parameters calibrated by the method of this embodiment are more accurate, have a higher tolerance to defocus, and are more robust.

[0104]

[0105] Table 3

[0106] In order to verify that the camera calibrated by the method of this embodiment can still accurately perform 3D reconstruction in the case of defocus blur, a three-frequency four-step phase shift method was used to reconstruct a plane at the same position, and the highest frequency was 10 steps. In the process of phase-height mapping, phase groups on 8 planes were captured at intervals of 0.2 mm, and the order of the polynomial was set to n = 2, as shown in Fig.14 As shown, a white plane is measured. Fig.14 (a)-(c) in the figure are fringe images taken by a dual telecentric camera. A lookup table is established to measure standard blocks, targets and small objects.

[0107] like Fig.15 As shown in the figure, the height of the standard block is 1mm, and the height of the plane is 1.5mm (the flatness is 0.15um, and the measurement uncertainty is 0.06um and 0.08um respectively). The horizontal phase-shifted stripes are projected onto the standard block, and the dual telecentric cameras synchronously capture the stripe image modulated by the standard block. The standard block modulated stripe pattern is combined with the phase 3D lookup table to calculate the point clouds of the two standard blocks as shown in the figure. Fig.15 (b) in 15 and (c) in 15. To quantitatively evaluate the 3D reconstruction accuracy, Fig.15 (d) and (e) in the figure give the height of the 1000th row along the x-axis, i.e. Fig.15The black lines in (b) and (c) provide a more intuitive observation. The error statistics of the 1000th row of the standard block surface plane fitting are shown in Table 4; the average absolute error of measuring the 1mm standard block is 0.00044mm, and the variance is 2.1027e-07mm; the average absolute error of measuring the 1.5mm height plane is 0.00061mm, and the variance is 3.4581e-07mm. Therefore, even if the reconstructed plane is outside the depth of field of the camera, the method of this embodiment still has a high measurement accuracy.

[0108]

[0109] Table 4

[0110] In order to test the performance of the method of this embodiment in measuring the surface morphology of small objects in practical applications, Fig.16 The small object in (d) was measured. In total, 18 fringe patterns of different frequencies were projected; one pattern at each frequency is shown in Fig.16 In (a)-(c) of the physical diagram. Fig.16 The result after phase height mapping is shown in (d). Fig.16 In (e), the unit of the Z axis is only millimeters, while the units of the X and Y axes are pixels. Combined with the calibrated camera internal and external parameters, the coordinates in the X and Y directions are converted to physical sizes. Fig.16 The unit in (e) is changed to millimeters.

[0111] The second test object was two small pendants; Fig.17 Each of the headings (a)–(f) in Fig.16 In the title. Fig.17 In (f), the diameter of the pendant is about 2.5 mm, which is consistent with the actual situation.

[0112] In general, the research of this embodiment innovatively designed a target suitable for high-resolution, small-field-of-view dual-telecentric cameras and proposed a novel calibration method, which significantly reduced the calibration error and is suitable for the calibration of defocused cameras. By combining Fourier transform with the design of a novel orthogonal binary stripe target, the pixel coordinates of the feature points are accurately extracted, and the camera is successfully calibrated using a defocused image with a reprojection error of 0.07840 pixels. Experiments have shown that the proposed method can achieve accurate camera calibration and three-dimensional morphology reconstruction in defocused or focused environments. This method has made great improvements in camera calibration and has a positive role in promoting three-dimensional reconstruction in the microscopic field and improving the accuracy of three-dimensional detection.

[0113] The above-mentioned embodiments only express the specific implementation of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the present invention. It should be pointed out that for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention.

Claims

1. A camera calibration method, characterized in that: The following steps are involved: Step 1, constructing an orthogonal binary fringe target with periodic characteristics; Step 2: Separate the horizontal and vertical phase-shifted fringes from the image by Fourier transform, locate the feature point position through the phase map and extract the pixel coordinates of the feature point.

2. The camera calibration method according to claim 1, characterized in that: The step 1 is specifically as follows: The light intensity of an orthogonal binary fringe target with periodic characteristics is expressed as: Among them, (X, Y) is the plane world coordinate, a, b1, b2 are constants, p x ,p y are the periods in the X and Y directions respectively; φ X0 ,φ Y0 is the additional phase value; The generated orthogonal stripes are binary-encoded, and four parameters a1, a2, a3, and a4 are introduced. After optimized diffusion, the error diffusion ratio is controlled to improve the sinusoidal similarity of the stripe pattern. The optimized kernel is expressed as: Among them, - represents the previously processed pixel, x represents the pixel currently being processed, and α1, α2, α3, and α4 are the error diffusion ratios to be optimized.

3. The camera calibration method according to claim 2, characterized in that: The step 2 is specifically as follows: Considering the periodic target as a periodic signal in a 2D plane, the following is obtained through Fourier transform: Among them, E 0 is the zero-order spectrum, and represents the k-th order spectrum, E * represents the conjugate operation; f x ,f y Respectively represent the frequencies in the x and y directions, and ∑ represents the sum; After Fourier transformation, the main frequency components along the x-axis and the y-axis are selected from the spectrum, and the fundamental frequency components are separated by using a suitable window. Then, the separated fundamental frequency components are subjected to inverse Fourier transformation to obtain two wrapped phases in the range of 0 to 2π. In the obtained image, the pixel coordinates θ are x =2π and θ y =2π is regarded as the coordinate of the feature point.

4. The camera calibration method according to claim 3, characterized in that: It also includes the optimization of the coordinates of the feature points, as follows: In the horizontal and vertical stripes, if |θ x -2π|<δ and |θ y Pixels with -2π|<δ are selected as candidate points, where δ represents the threshold; The feature points in the same row or column on the target are on the same straight line, and nonlinear fitting is performed locally on the candidate points and their surroundings, while constrained optimization is performed on the feature points in the same row or column on the target; Incorporate the feature points into the global optimization, unwrap the wrapped phase, and set the midpoint of the image plane as the starting point of the phase unwrapping: c1 and c2 are constants directly related to the starting point of phase unwrapping; after obtaining the unwrapped phase, the candidate feature points are incorporated into the global linear fit: where c vi ,c hi (i=1, 2, 3) are fitting coefficients, constraining the feature points to be in the same column or row; finally, after simultaneously constraining the local and global optimization of the feature points, the pixel coordinates of the feature points with sub-pixel accuracy are obtained.

5. The camera calibration method according to claim 1, characterized in that: The material of the orthogonal binary stripe target is a chrome-plated glass substrate, the thickness of the chrome-plated glass substrate is 1 mm, and the chrome-plated glass substrate is formed by laser photolithography process.

6. A method for restoring three-dimensional shape information of an object, characterized in that: The camera is calibrated by the camera calibration method according to any one of claims 1 to 5, and the three-dimensional shape information of the object is restored by a reconstruction method based on phase height mapping.

7. The method for restoring the three-dimensional shape information of an object according to claim 6, characterized in that: The three-dimensional shape information of the object is restored by the reconstruction method based on phase height mapping as follows: Control the high-precision translation stage to collect stripe structured light images of different reference planes, where the distance between adjacent reference planes is equal to d, and the phase map of each plane is obtained by phase decomposition Fitting a polynomial using the absolute phase height mapping method: Where b0, ..., b n are the fitted polynomial coefficients, from which the phase-height mapping relationship in the measurement space is obtained, and then the modulated fringes on the surface of the object to be measured are collected, and the same phase deconvolution algorithm is used to restore the phase map of the object. The three-dimensional morphology information of the object to be measured is further obtained by combining the phase-height mapping relationship and the calibration parameters.

Images