Optical perpendicularity measurement method, system and equipment based on double-camera automatic focusing and storage medium

Abstract

The invention discloses an optical verticality measurement method, system and device based on double-camera automatic focusing and a storage medium, and the method comprises the steps: enabling a camera a and a camera b to form a triangulation system, and completing the calibration of the triangulation system; calculating the serial number of the maximum value of the modulation degree corresponding to the focusing position; a dual-camera automatic focusing system formed by a projector, a camera a and a camera b is utilized, and the space coordinate value of each pixel point of each calibration surface in the triangulation system is completed through the assistance of N-step phase shift fringes; secondary fitting interpolation is carried out on the space coordinate value of each pixel point of the calibration surface in the triangulation system and the serial number corresponding to the maximum value of the modulation degree, and the mapping relation between the space coordinate value of each pixel point of the calibration surface and the serial number corresponding to the maximum value of the modulation degree is obtained; and reconstructing the three-dimensional surface shape of the measured object. According to the invention, the advantage of high precision of the projection structured light and the spatial information acquisition capability of the double cameras are combined, and stronger adaptability and effect are shown in scenes of intelligent detection, industrial measurement and the like.

Description

Technical Field

[0001] The present invention belongs to the technical field of structured light projection three-dimensional shape measurement, and particularly relates to an optical vertical measurement method, system, device, and storage medium based on dual-camera autofocus. Background Technique

[0002] With the rapid development of three-dimensional measurement technology, optical three-dimensional surface vertical measurement methods have been widely used in the fields of precision instruments, aerospace, and medicine due to their non-destructive, non-contact, and high-precision characteristics. This technology can efficiently measure objects with complex shapes, such as grooves, steps, and deep holes.

[0003] Three-dimensional sensing methods based on fringe projection can be classified into two categories according to the structure of the measurement system: one is an optical three-dimensional surface topography measurement technology based on the triangulation principle, and the other is an optical three-dimensional surface topography measurement technology based on the vertical measurement principle. In the technology based on the triangulation principle, it can be further divided into single-frame fringe processing technology and multi-frame fringe processing technology. Single-frame fringe processing technology includes Fourier transform, window Fourier transform, wavelet transform, and S transform methods, while multi-frame fringe processing technology mainly uses phase-shift methods. In this measurement system based on the triangulation principle, there is a certain angle between the projection optical axis and the observation optical axis, and generally, the measurement accuracy will increase as the angle increases. When a sinusoidal grating is projected onto the object surface, the detector captures the deformed fringes from another direction, and these deformed fringes contain the height information of the object surface. By using the fringe phase as the information carrier and combining phase demodulation technology, the three-dimensional topography of the object can be reconstructed. However, in this technology based on the triangulation principle, when measuring objects with sharp changes or discontinuities in surface height, due to the angle between the projection optical axis and the observation optical axis, shadow and occlusion problems are likely to occur. As the angle increases, the shadow and occlusion areas will also become larger, making it difficult for the system to accurately analyze and reconstruct the three-dimensional topography of the object, thus limiting the application scope of this technology.

[0004] The optical vertical measurement method can directly reconstruct the three-dimensional topography of the measured object without phase unwrapping. Compared with the triangulation method, this technology effectively avoids the possible shadow problems and the discontinuities encountered in the phase unwrapping process. However, the traditional optical vertical measurement method can only obtain the actual physical size of the object in the Z-axis direction, while the values in the XY direction only correspond to the position information of the camera pixels. Summary of the Invention

[0005] The purpose of the present invention is to provide an optical vertical measurement method, system, device, and storage medium based on dual-camera autofocus to solve the problem in the prior art that only the actual physical size of the measured object in the Z direction can be given.

[0006] The embodiments of the present application are implemented as follows. An optical vertical measurement method based on dual-camera autofocus is provided, including: Camera A and Camera B form a triangulation system, and the calibration of the triangulation system is completed;

[0007] Calculate the serial number of the zoom position corresponding to the maximum modulation value;

[0008] A dual-camera autofocus system composed of a projector, Camera A, and Camera B is used, and the spatial coordinate values of each pixel point on each calibration plane in the triangulation system are completed with the assistance of N-step phase-shifted fringes;

[0009] The spatial coordinate values of each pixel point on the calibration plane in the triangulation system and the serial number corresponding to the maximum modulation value are subjected to quadratic fitting interpolation to obtain the mapping relationship between the spatial coordinate values of each pixel point on the calibration plane and the serial number corresponding to the maximum modulation value;

[0010] Reconstruct the three-dimensional surface of the object to be measured.

[0011] In some embodiments, the conversion from the system pixel coordinates to the world coordinates and the calibration of the triangulation system include the following steps:

[0012] Establish the relationship between the 2D image pixel coordinates and the 2D image physical coordinates:

[0013]

[0014] In the formula, u represents the column number in the image pixel coordinates, with the unit of pixel, v represents the row number in the image pixel coordinates, with the unit of pixel; u0 represents the position of the origin of the column number of the image physical coordinates in the image pixel coordinates, and v0 represents the position of the origin of the row number of the image physical coordinates in the image pixel coordinates; x represents the column number in the image physical coordinates, with the unit of millimeter, and y represents the row number in the image physical coordinates, with the unit of millimeter;

[0015] Establish the mapping relationship between the 2D image physical coordinates and the points in the camera three-dimensional coordinate system:

[0016]

[0017] In the formula, x represents the column number in the image physical coordinates, with the unit of millimeter, y represents the row number in the image physical coordinates, with the unit of millimeter; f represents the focal length of the camera; X c , Y c , Z c are three mutually orthogonal axes of the camera coordinate system;

[0018] Convert to the following homogeneous coordinate expression:

[0019]

[0020] Wherein, x represents the number of columns in the physical coordinates of the image, with the unit of millimeter, and y represents the number of rows in the physical coordinates of the image, with the unit of millimeter; f represents the focal length of the camera; X c , Y c , Z c are three mutually orthogonal axes of the camera coordinate system;

[0021] Establish the conversion relationship between the camera coordinate system and the world coordinate system:

[0022] At this time, the calibration of the triangulation system in the ideal case is completed;

[0023] Wherein, u represents the number of columns in the pixel coordinates of the image, with the unit of pixel, and v represents the number of rows in the pixel coordinates of the image, with the unit of pixel; f represents the focal length of the camera; f x = f / d x , f y = f / d y , d x and d y respectively represent the physical sizes of the camera unit pixel in the horizontal and vertical directions; u0 and v0 represent the coordinates of the camera plane at the optical center; R1 represents a 3×3 orthogonal rotation matrix, and T1 = [T x , T y , T z is a three-dimensional translation vector; X W , Y W , Z W represent three mutually orthogonal axes in the world coordinate system; M1 represents the internal parameter matrix; K1 is the external parameter matrix;

[0024] In the actual triangulation system, the influence of tangential and radial distortion is:

[0025] At this time, the correction of the calibration of the actual triangulation system is completed;

[0026] Wherein, x represents the horizontal direction (number of columns) in the physical coordinates of the image, with the unit of millimeter, and y represents the vertical direction (number of rows) in the physical coordinates of the image, with the unit of millimeter; x' and y' respectively represent the coordinate values in the two directions after considering distortion, and the parameters p1 and p2 represent the tangential distortion coefficients, and ρ represents the distance from the coordinate point (x, y) to the origin.

[0027] In some embodiments, a projector is provided, and the current value of the electronically adjustable focus liquid lens is changed at equal intervals, so that the projector zooms the projection light field to different calibration planes. For changing the current value of the electronically adjustable focus liquid lens at equal intervals, N frames of fringe patterns with a fixed phase difference are projected onto the calibration plane. The light field distribution collected by the camera a on the focal plane is expressed as:

[0028]

[0029] The light field collected by the front and rear cameras a of the focal plane is expressed as:

[0030]

[0031] In the formula, and respectively represent the light field distributions on the focal plane and the image plane at a distance H from the focal plane; j represents the equally spaced zoom times of the projector b, and j = 1, 2, 3…J, where J represents the total number of movements; N represents N-step phase shift, and N≥3; n and n’ respectively represent the nth fringe pattern projected on the focal plane or the defocused plane at the j position, and n (or n’) = 0, 1, 2…, N - 1; σ H represents the standard deviation of the point spread function; u represents the number of columns in the image pixel coordinates, in pixels, and v represents the number of rows in the image pixel coordinates, in pixels; R2(u, v), B2(u, v) and C2(u, v) respectively represent the surface reflectivity of the object, the ambient light intensity and the contrast of the fringes; M2 represents the magnification of the measurement system; f0 represents the fringe frequency, and Φ(u, v) represents the initial phase of the fringes;

[0032] Extract the fringes on the focal plane respectively as in the formula

[0033]

[0034] The fringes on the defocused plane are as in the formula

[0035]

[0036] The modulation degree distributions of the fringes on the above-mentioned focal plane and defocused plane are as follows:

[0037]

[0038]

[0039] In the formula, M f (u, v) and M' f (u, v; σ H ) respectively represent the modulation degree distributions on the focal plane and the defocused plane; I(u, v) and I(u, v; σ H ) respectively represent the light field distributions on the focal plane and the image plane at a distance H from the focal plane; σ H represents the standard deviation of the point spread function; u represents the number of columns in the image pixel coordinates, in pixels, and v represents the number of rows in the image pixel coordinates, in pixels;

[0040] Perform quadratic fitting interpolation on the curve formed by the modulation values of the corresponding pixels in the modulation distribution at each scanning position, extract the maximum modulation value of the corresponding pixels, and the sequence number j(t, u, v) corresponding to the maximum modulation value of the corresponding pixels (u, v) can be obtained. max 。

[0041] In some embodiments, the current value of the electronically adjustable focus liquid lens is automatically set according to the sequence number corresponding to the maximum modulation value, so that the fringe pattern projected by the projector is in a focused state on each calibration plane. A dual-camera autofocus system composed of a projector, camera a, and camera b is used, and the spatial coordinate values of each pixel in the triangulation system on each calibration plane are assisted to be completed through N-step phase-shifted fringes.

[0042] In some embodiments, the projector projects vertical fringes and horizontal fringes respectively, and camera a and camera b collect synchronously. The nth fringe patterns of the collected vertical fringes and horizontal fringes are respectively represented mathematically as

[0043]

[0044]

[0045] In the formula, and respectively represent the light field distributions of the nth fringe patterns of the vertical fringes and horizontal fringes on the focal plane; u represents the column number in the image pixel coordinates, with the unit of pixel, and v represents the row number in the image pixel coordinates, with the unit of pixel; N represents N-step phase shift, and N≥3; n represents the nth fringe pattern projected, n = 0, 1, 2…, N - 1; R2(u, v), B2(u, v), and C2(u, v) are respectively the object surface reflectivity, ambient light intensity, and fringe contrast; M2 is the magnification of the measurement system; and respectively represent the vertical fringe frequency and the horizontal fringe frequency; Φ V (u, v) and Φ H (u, v) respectively represent the initial phases of the vertical fringes and horizontal fringes;

[0046] Calculate the respective phase values of and as follows:

[0047]

[0048]

[0049] In the formula, and respectively represent the truncated phases of the vertical fringes and horizontal fringes, with the value range from -π to π and having a discontinuity of 2π; and respectively represent the light field distributions of the vertical stripes and the horizontal stripes in the n-th frame of the fringe pattern on the focal plane; N represents N-step phase shift, and N≥3; n represents the n-th fringe pattern in the projection, n = 0, 1, 2…, N - 1;

[0050] Respectively add and multiples of 2π, and the formula is as follows:

[0051]

[0052]

[0053] In the formula, Φ V (u, v) and Φ H (u, v) respectively represent and the continuous phases obtained by adding and subtracting multiples of 2π; and respectively represent the truncated phases of the vertical stripes and the horizontal stripes; k V (u, v) and k H (u, v) respectively represent the orders of the vertical stripes and the horizontal stripes;

[0054] Thus,

[0055] In the formula, Φ V (u, v) and Φ H (u, v) respectively represent the absolute phases of the vertical stripes and the horizontal stripes; T V and T H respectively represent the periods of the vertical stripe frequency and the horizontal stripe frequency; u represents the column number in the image pixel coordinates, in pixels, and v represents the row number in the image pixel coordinates, in pixels;

[0056] Substitute into to obtain the values of X W (u, v), Y W (u, v), Z W (u, v) in the world coordinate system. By traversing each point on the calibration plane, the spatial coordinate values X W (t, u, v), Y W (t, u, v), Z W (t, u, v) of each pixel point on the calibration plane can be determined.

[0057] In some embodiments, within the measurement range determined by the farthest calibration plane and the nearest calibration plane in the vertical system, scan each calibration plane to obtain the spatial coordinate values X W (t, u, v), Y W(t, u, v), Z W (t, u, v), and respectively obtain the spatial coordinate values X of each pixel point of the calibration plane in the triangulation system W (t, u, v), Y W (t, u, v), Z W (t, u, v) and the corresponding sequence number of the maximum modulation degree are subjected to quadratic fitting interpolation, so as to establish the spatial coordinates X W (t, u, v), Y W (t, u, v), Z W (t, u, v) and the corresponding sequence number j(t, u, v) of the maximum modulation degree max between:

[0058] X W (t, u, v) = a(u, v) + b(u, v)j(t, u, v) max + c(u, v)j 2 (t, u, v) max

[0059] Y W (t, u, v) = a(u, v) + b(u, v)j(t, u, v) max + c(u, v)j 2 (t, u, v) max

[0060] Z W (t, u, v) = a(u, v) + b(u, v)j(t, u, v) max + c(u, v)j 2 (t, u, v) max

[0061] In the formula, t represents the t-th calibration plane, u and v respectively represent the row number and column number in the image pixel coordinates, X W (t, u, v), Y W (t, u, v), Z W (t, u, v) respectively represent the coordinate values in the X, Y, and Z coordinate axis directions of the t-th calibration plane at this row and column of the image, and j(t, u, v) max represents the sequence number of the maximum modulation degree of the t-th calibration plane, and a(u, v), b(u, v), and c(u, v) are respectively the fitting coefficients of the depth value - maximum modulation degree sequence number curve.

[0062] In some embodiments, the object to be measured is placed within the measurement range determined by the farthest calibration plane and the nearest calibration plane of the vertical system, and its position is kept unchanged. The current value of the electronically adjustable focus liquid lens is changed at equal intervals, so that the projector zooms the projection light field onto the object to be measured. At each current value of the electronically adjustable focus liquid lens, N fringe patterns with a fixed phase difference are projected onto the object to be measured, and the camera synchronously acquires the fringe patterns to obtain the sequence number j(t, u, v) corresponding to the maximum modulation degree. max , according to X W (t, u, v) = a(u, v) + b(u, v)j(t, u, v) max + c(u, v)j 2 (t, u, v) max Y W (t, u, v) = a(u, v) + b(u, v)j(t, u, v) max + c(u, v)j 2 (t, u, v) max , the coordinate values of the pixel points (u, v) of the object to be measured are obtained. Z W (t, u, v) = a(u, v) + b(u, v)j(t, u, v) max + c(u, v)j 2 (t, u, v) max

[0063] The calibration value X W (t, u, v), Y W (t, u, v), Z W (t, u, v). By traversing all pixel points, the height distribution of the object to be measured can be obtained, that is, the three-dimensional surface shape of the object to be measured is reconstructed and completed.

[0064] Correspondingly, the embodiment of the present application further provides an optical vertical measurement system based on dual-camera autofocus, including: a triangulation system module, where camera a and camera b form a triangulation system and complete the calibration of the triangulation system;

[0065] A sequence number module, which calculates the sequence number corresponding to the zoom position of the maximum modulation degree;

[0066] A module for obtaining the spatial coordinate values of each pixel point, which uses the dual-camera autofocus system composed of a projector, camera a, and camera b, and completes the spatial coordinate values of each pixel point on each calibration plane in the triangulation system with the assistance of N-step phase-shifted fringes;

[0067] A mapping relationship module, which performs quadratic fitting interpolation on the spatial coordinate values of each pixel point on the calibration plane in the triangulation system and the sequence number corresponding to the maximum modulation degree to obtain the mapping relationship between the spatial coordinate values of each pixel point on the calibration plane and the sequence number corresponding to the maximum modulation degree;

[0068] The three-dimensional surface shape module of the object to be measured is reconstructed to reconstruct the three-dimensional surface shape of the object to be measured.

[0069] Correspondingly, the embodiment of the present application further provides a computer device, including a storage and a processor. The storage stores a computer program. When the computer program is executed by the processor, the processor is enabled to execute the steps of the above method.

[0070] Correspondingly, the embodiment of the present application further provides a computer-readable storage medium, storing a computer program. When the computer program is executed by a processor, the processor is enabled to execute the steps of the above method.

[0071] In summary, due to the adoption of the above technical solutions, the beneficial effects of the present invention are as follows:

[0072] The present application first uses the maximum modulation value to realize the automatic focusing calibration plate of the projector, ensuring that the fringe projection in triangulation is clearer. The small depth of field of the projector can more accurately extract the maximum modulation value of the fringes, making the measurement accuracy higher. Secondly, it can directly provide high-precision XYZ data of the calibration plate, avoiding the error of epipolar line matching required by binocular stereo vision, and improving the anti-interference ability, so that it can also stably measure on weak texture or highly reflective surfaces. The present application combines the high-precision advantages of structured light projection and the spatial information acquisition ability of dual cameras, and will show stronger adaptability and effects in scenarios such as intelligent detection and industrial measurement. Description of the Drawings

[0073] Figure 1 It is a flowchart of an optical vertical measurement method based on dual-camera autofocus provided by an embodiment of the present invention;

[0074] Figure 2 It is a schematic diagram of the principle of an optical vertical measurement method based on dual-camera autofocus provided by an embodiment of the present invention;

[0075] Figure 3 It is a schematic diagram of the fringe pattern projected onto the object to be measured when the projector performs the 151st focal length transformation provided by an embodiment of the present invention;

[0076] Figure 4 It is an error distribution diagram of the system test of the standard sphere of the present invention and the difference from the fitted sphere;

[0077] Figure 5 It is the reconstruction result after calibration using the dual-camera autofocus triangulation system to assist the vertical measurement system involved in the present invention.

[0078] Description of the Reference Numerals:

[0079] 1 - Projector, 2 - Electronically tunable liquid lens, 3 - Half-transparent and half-reflective mirror, 4 - Camera a, 5 - Camera b, 6 - Object to be measured. Detailed implementation manners

[0080] In order to make the objectives, technical solutions and advantages of the present invention clearer and more understandable, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0081] The technical solution of this application is as follows:

[0082] As Figure 1 shown, in a first aspect, an optical vertical measurement method based on dual-camera autofocus provided by an embodiment of this application includes:

[0083] S01. Camera a and camera b form a triangulation system and complete the calibration of the triangulation system;

[0084] S02. Calculate the sequence number corresponding to the zoom position of the maximum modulation value;

[0085] S03. Use the dual-camera autofocus system composed of a projector, camera a, and camera b, and complete the spatial coordinate values of each pixel point on each calibration surface in the triangulation system with the assistance of N-step phase-shifted fringes;

[0086] S04. Perform quadratic fitting interpolation on the spatial coordinate values of each pixel point on the calibration surface in the triangulation system and the sequence number corresponding to the maximum modulation value to obtain the mapping relationship between the spatial coordinate values of each pixel point on the calibration surface and the sequence number corresponding to the maximum modulation value;

[0087] S05. Reconstruct the three-dimensional surface of the object to be measured.

[0088] This application first uses the maximum modulation value to realize the automatic focusing calibration plate of the projector, ensuring that the fringe projection in triangulation is clearer. The small depth of field of the projector can more accurately extract the maximum modulation value of the fringes, making the measurement accuracy higher. Secondly, it can directly provide the high-precision XYZ data of the calibration plate, avoid the error of epipolar matching required by binocular stereo vision, and improve the anti-interference ability, enabling stable measurement on weakly textured or highly reflective surfaces. This application combines the high-precision advantages of projection structured light and the spatial information acquisition ability of dual cameras, and will show stronger adaptability and effects in scenarios such as intelligent detection and industrial measurement.

[0089] In the said S01:

[0090] In some embodiments, the conversion from the system pixel coordinates to the world coordinates and the calibration of the triangulation system include the following steps:

[0091] S011. Establish the relationship between the 2D image pixel coordinates and the 2D image physical coordinates:

[0092]

[0093] In the formula, \(u\) represents the number of columns in the image pixel coordinates, with the unit of pixel, and \(v\) represents the number of rows in the image pixel coordinates, with the unit of pixel; \(u_0\) represents the position of the origin of the column number in the image physical coordinates in the image pixel coordinates, and \(v_0\) represents the position of the origin of the row number in the image physical coordinates in the image pixel coordinates; \(x\) represents the number of columns in the image physical coordinates, with the unit of millimeter, and \(y\) represents the number of rows in the image physical coordinates, with the unit of millimeter.

[0094] S012. Establish the mapping relationship between the 2D image physical coordinates and the points in the camera three-dimensional coordinate system:

[0095]

[0096] In the formula, \(x\) represents the number of columns in the image physical coordinates, with the unit of millimeter, \(y\) represents the number of rows in the image physical coordinates, with the unit of millimeter; \(f\) represents the focal length of the camera; \(X\) c , \(Y\) c , \(Z\) c are three mutually orthogonal axes of the camera coordinate system;

[0097] It is converted into the following homogeneous coordinate expression:

[0098]

[0099] In the formula, \(x\) represents the number of columns in the image physical coordinates, with the unit of millimeter, \(y\) represents the number of rows in the image physical coordinates, with the unit of millimeter; \(f\) represents the focal length of the camera; \(X\) c , \(Y\) c , \(Z\) c are three mutually orthogonal axes of the camera coordinate system;

[0100] S013. Establish the conversion relationship between the camera coordinate system and the world coordinate system:

[0101] At this time, the calibration of the triangulation system in the ideal case is completed;

[0102] In the formula, \(u\) represents the number of columns in the image pixel coordinates, with the unit of pixel, \(v\) represents the number of rows in the image pixel coordinates, with the unit of pixel; \(f\) represents the focal length of the camera; \(f\) x =f / d x , \(f\) y =f / d y , \(d\) x and \(d\) y respectively represent the physical sizes of the camera unit pixel in the horizontal and vertical directions; \(u_0\) and \(v_0\) represent the coordinates of the camera plane at the optical center; \(R_1\) represents a 3×3 orthogonal rotation matrix, \(T_1 = [T\) x , \(T\) y , \(T\)z is a three-dimensional translation vector; X W , Y W , Z W represent three mutually orthogonal axes in the world coordinate system; M1 represents the internal parameter matrix; K1 is the external parameter matrix;

[0103] S014. In an actual triangulation system, the effects of tangential and radial distortion are as follows:

[0104] At this time, the calibration correction of the actual triangulation system is completed;

[0105] In the formula, x represents the horizontal direction (number of columns) in the physical coordinates of the image, with the unit of millimeter, and y represents the vertical direction (number of rows) in the physical coordinates of the image, with the unit of millimeter; x' and y' respectively represent the coordinate values in the two directions after considering the distortion, the parameters p1 and p2 represent the tangential distortion coefficients, and ρ represents the distance from the coordinate point (x, y) to the origin.

[0106] In the above S02:

[0107] In some embodiments, a projector is provided to change the current value of the electronically tunable liquid lens at equal intervals, so that the projector zooms the projection light field to different calibration planes. For changing the current value of the electronically tunable liquid lens at equal intervals, project N frames of fringe patterns with a fixed phase difference onto the calibration plane. The light field distribution collected by camera a on the focal plane is expressed as:

[0108]

[0109] The light fields collected by the cameras before and after the focal plane are expressed as:

[0110]

[0111] In the formula, and respectively represent the light field distributions on the focal plane and the image plane at a distance H from the focal plane; j represents the equal-interval zooming times of projector b, and j = 1, 2, 3... J, where J represents the total number of movements; N represents N-step phase shift, and N ≥ 3; n and n' respectively represent the nth fringe pattern projected on the focal plane or the defocused plane at the j position, and n (or n') = 0, 1, 2..., N - 1; σ H represents the standard deviation of the point spread function; u represents the number of columns in the image pixel coordinates, with the unit of pixel, and v represents the number of rows in the image pixel coordinates, with the unit of pixel; R2(u, v), B2(u, v) and C2(u, v) respectively represent the object surface reflectivity, the ambient light intensity and the contrast of the fringe; M2 represents the magnification of the measurement system; f0 represents the fringe frequency, and Φ(u, v) represents the initial phase of the fringe;

[0112] Extract the fringes on the focal plane respectively as in the formula

[0113]

[0114] Extract the fringes on the defocused plane as in the formula

[0115]

[0116] The modulation degree distributions of the fringes on the above-mentioned focal plane and defocused plane are as follows:

[0117]

[0118] In the formula, M f (u, v) and M' f (u, v; σ H ) represent the modulation degree distributions on the focal plane and defocused plane respectively; I(u, v) and I(u, v; σ H ) represent the light field distributions on the focal plane and the image plane at a distance H from the focal plane respectively; σ H represents the standard deviation of the point spread function; u represents the column number in the image pixel coordinates, with the unit of pixel, and v represents the row number in the image pixel coordinates, with the unit of pixel;

[0119] It can be understood that since the current value of the electronically tunable liquid lens changes at equal intervals, the current value of the electronically tunable liquid lens can also be determined by j;

[0120] Perform quadratic fitting interpolation on the curve formed by the modulation degree values of the corresponding pixels of the modulation degree distributions at each scanning position, and extract the maximum modulation degree of the corresponding pixels, then the serial number j(t, u, v) corresponding to the maximum modulation degree of the corresponding pixels (u, v) can be obtained max .

[0121] It can be understood that the serial number j(t, u, v) max is also the number of times the electronically tunable liquid lens is at the j(t, u, v) max th current value. When the fringe pattern projected by the projector is in focus at the calibration plane pixel point (u, v), since the projector lens has a certain depth of field and the pose of the calibration plane is perpendicular to the coaxial line of the projector and camera a, at this time, take the maximum modulation degree serial number j(t, u mid , v mid ) max (u mid , v mid respectively represent the intermediate values of the total number of rows and columns of the image), then it can be ensured that the calibration plane is in the focused state.

[0122] In the above S03:

[0123] In some embodiments, the current value of the electronically adjustable liquid lens is automatically set according to the serial number corresponding to the maximum modulation degree, so that the fringe pattern projected by the projector is in a focused state on each calibration plane. A dual-camera autofocus system composed of a projector, camera a, and camera b is used, and the spatial coordinate values of each pixel point in the triangulation system on each calibration plane are completed with the assistance of N-step phase-shifted fringes.

[0124] Furthermore, the projector projects vertical fringes and horizontal fringes respectively, and camera a and camera b collect synchronously. The nth frame of the vertical and horizontal fringes collected are respectively expressed mathematically as

[0125]

[0126]

[0127] where and respectively represent the light field distributions of the nth frame of the vertical and horizontal fringes on the focal plane; u represents the number of columns in the image pixel coordinates, with the unit of pixel, v represents the number of rows in the image pixel coordinates, with the unit of pixel; N represents N-step phase shift, and N≥3; n represents the nth fringe pattern projected, n = 0, 1, 2…, N - 1; R2(u, v), B2(u, v), and C2(u, v) are respectively the object surface reflectivity, ambient light intensity, and fringe contrast; M2 is the magnification of the measurement system; and respectively represent the vertical fringe frequency and the horizontal fringe frequency; Φ V (u, v) and Φ H (u, v) respectively represent the initial phases of the vertical and horizontal fringes;

[0128] Calculate the and respective phase values as follows:

[0129]

[0130]

[0131] where and respectively represent the wrapped phases of the vertical and horizontal fringes, with the value range from -π to π and having a discontinuity of 2π; and respectively represent the light field distributions of the nth frame of the vertical and horizontal fringes on the focal plane; N represents N-step phase shift, and N≥3; n represents the nth fringe pattern projected, n = 0, 1, 2…, N - 1;

[0132] Respectively, and Add or subtract multiples of 2π, and the formula is as follows:

[0133]

[0134] In the formula, Φ V (u, v) and Φ H (u, v) respectively represent and the continuous phases obtained by adding or subtracting multiples of 2π; and respectively represent the truncated phases of vertical stripes and horizontal stripes; k V (u, v) and k H (u, v) respectively represent the orders of vertical stripes and horizontal stripes;

[0135] Thus,

[0136] In the formula, Φ V (u, v) and Φ H (u, v) respectively represent the absolute phases of vertical stripes and horizontal stripes; T V and T H respectively represent the periods of the vertical stripe frequency and the horizontal stripe frequency; u represents the column number in the image pixel coordinates, with the unit of pixel, and v represents the row number in the image pixel coordinates, with the unit of pixel;

[0137] Substitute into to obtain the values of X W (u, v), Y W (u, v), Z W (u, v) in the world coordinate system. By traversing each point on the calibration plane, the spatial coordinate values X W (t, u, v), Y W (t, u, v), Z W (t, u, v) of each pixel point on the calibration plane can be determined.

[0138] In the said S04:

[0139] In some embodiments, the spatial coordinate values of each pixel point on the calibration plane in the triangulation system and the sequence number corresponding to the maximum modulation value are subjected to quadratic fitting interpolation to obtain the mapping relationship between the spatial coordinate values of each pixel point on the calibration plane and the sequence number corresponding to the maximum modulation value, including the following steps:

[0140] Within the measurement range determined by the farthest calibration plane and the nearest calibration plane in the vertical system, scan each calibration plane to obtain the spatial coordinate values X W (t, u, v), Y W (t, u, v), Z W(t, u, v), and respectively obtain the spatial coordinate values X of each pixel point of the calibration plane in the triangulation system W (t, u, v), Y W (t, u, v), Z W (t, u, v) and the serial number corresponding to the maximum modulation degree are subjected to quadratic fitting interpolation, so as to establish the spatial coordinates X W (t, u, v), Y W (t, u, v), Z W (t, u, v) and the mapping relationship between the serial number j(t, u, v) corresponding to the maximum modulation degree max :

[0141] X W (t, u, v) = a(u, v) + b(u, v)j(t, u, v) max + c(u, v)j 2 (t, u, v) max

[0142] Y W (t, u, v) = a(u, v) + b(u, v)j(t, u, v) max + c(u, v)j 2 (t, u, v) max

[0143] Z W (t, u, v) = a(u, v) + b(u, v)j(t, u, v) max + c(u, v)j 2 (t, u, v) max

[0144] In the formula, t represents the t-th calibration plane, u and v respectively represent the row number and column number in the image pixel coordinates, X W (t, u, v), Y W (t, u, v), Z W (t, u, v) respectively represent the coordinate values in the X, Y, and Z axis directions of the t-th calibration plane at the row and column of the image, and j(t, u, v) max represents the serial number of the maximum modulation degree of the t-th calibration plane, and a(u, v), b(u, v), and c(u, v) are respectively the fitting coefficients of the depth value - maximum modulation degree serial number curve.

[0145] In S05:

[0146] In some embodiments, the projector zooms and scans the object to be measured, calculates the serial number values corresponding to the maximum modulation degree of each pixel point, and reconstructs the three-dimensional surface shape of the object to be measured according to the mapping relationship between the spatial coordinate values of each pixel point of the calibration plane and the serial number corresponding to the maximum modulation degree.

[0147] Further, place the object to be measured within the measurement range determined by the farthest calibration plane and the nearest calibration plane of the vertical system, and keep the position unchanged. Change the current value of the electronically tunable liquid lens at equal intervals, so that the projector zooms the projection light field onto the object to be measured. At each current value of the electronically tunable liquid lens, project N fringe patterns with a fixed phase difference onto the object to be measured, and the camera synchronously collects the fringe patterns to obtain the sequence number j(t, u, v) corresponding to the maximum modulation max , according to obtain the pixel point coordinate values X W (t, u, v), Y W (t, u, v), Z W (t, u, v) of the object to be measured. By traversing all pixel points, the height distribution of the object to be measured can be obtained, that is, the three-dimensional shape of the object to be measured is reconstructed and completed.

[0148] In a second aspect, an optical vertical measurement system based on dual-camera autofocus provided by an embodiment of the present application includes:

[0149] A triangulation system module, where camera a and camera b form a triangulation system and complete the calibration of the triangulation system;

[0150] A sequence number module, which calculates the sequence number corresponding to the zoom position of the maximum modulation;

[0151] A spatial coordinate value acquisition module for each pixel point, which uses a dual-camera autofocus system composed of a projector, camera a, and camera b, and completes the spatial coordinate values of each pixel point on each calibration plane in the triangulation system with the assistance of N-step phase-shifted fringes;

[0152] A mapping relationship module, which performs quadratic fitting interpolation on the spatial coordinate values of each pixel point on the calibration plane in the triangulation system and the sequence number corresponding to the maximum modulation to obtain the mapping relationship between the spatial coordinate values of each pixel point on the calibration plane and the sequence number corresponding to the maximum modulation;

[0153] A three-dimensional shape reconstruction module for the object to be measured, which reconstructs the three-dimensional shape of the object to be measured.

[0154] In the triangulation system module:

[0155] In some embodiments, the conversion from the system pixel coordinates to the world coordinates and the calibration of the triangulation system include the following steps:

[0156] S011. Establish the relationship between the 2D image pixel coordinates and the 2D image physical coordinates:

[0157]

[0158] Wherein, u represents the number of columns in the image pixel coordinates, with the unit of pixel, and v represents the number of rows in the image pixel coordinates, with the unit of pixel; u0 represents the position of the origin of the physical coordinate columns of the image in the image pixel coordinates, and v0 represents the position of the origin of the physical coordinate rows of the image in the image pixel coordinates; x represents the number of columns in the physical coordinates of the image, with the unit of millimeter, and y represents the number of rows in the physical coordinates of the image, with the unit of millimeter;

[0159] S012. Establish the mapping relationship between the 2D image physical coordinates and the points in the camera three-dimensional coordinate system:

[0160]

[0161] Wherein, x represents the number of columns in the physical coordinates of the image, with the unit of millimeter, and y represents the number of rows in the physical coordinates of the image, with the unit of millimeter; f represents the focal length of the camera; X c , Y c , Z c are three mutually orthogonal axes of the camera coordinate system;

[0162] It is converted into the following homogeneous coordinate expression:

[0163]

[0164] Wherein, x represents the number of columns in the physical coordinates of the image, with the unit of millimeter, and y represents the number of rows in the physical coordinates of the image, with the unit of millimeter; f represents the focal length of the camera; X c , Y c , Z c are three mutually orthogonal axes of the camera coordinate system;

[0165] S013. Establish the conversion relationship between the camera coordinate system and the world coordinate system:

[0166] At this time, the calibration of the triangulation system under ideal conditions is completed;

[0167] Wherein, u represents the number of columns in the image pixel coordinates, with the unit of pixel, and v represents the number of rows in the image pixel coordinates, with the unit of pixel; f represents the focal length of the camera; f x = f / d x , f y = f / d y , d x and d y respectively represent the physical sizes of the camera unit pixel in the horizontal and vertical directions; u0 and v0 represent the coordinates of the camera plane at the optical center; R1 represents a 3×3 orthogonal rotation matrix, and T1 = [T x , T y , T z is a three-dimensional translation vector; X W, Y W , Z W represent three mutually orthogonal axes in the world coordinate system; M1 represents the internal parameter matrix; K1 is the external parameter matrix;

[0168] S014. In an actual triangulation system, the effects of tangential and radial distortion are as follows:

[0169] At this time, the calibration correction of the actual triangulation system is completed;

[0170] In the formula, x represents the horizontal direction (number of columns) in the physical coordinates of the image, with the unit of millimeter, y represents the vertical direction (number of rows) in the physical coordinates of the image, with the unit of millimeter; x', y' respectively represent the coordinate values in the two directions after considering distortion, the parameters p1, p2 represent the tangential distortion coefficients, and ρ represents the distance from the coordinate point (x, y) to the origin.

[0171] In the serial number module:

[0172] In some embodiments, a projector is provided to change the current value of the electronically adjustable focus liquid lens at equal intervals, so that the projector zooms the projection light field to different calibration planes. For changing the current value of the electronically adjustable focus liquid lens at equal intervals, project N frames of fringe patterns with a fixed phase difference onto the calibration plane. The light field distribution collected by camera a on the focal plane is expressed as:

[0173]

[0174] The light fields collected by the cameras before and after the focal plane are expressed as:

[0175]

[0176] In the formula, and respectively represent the light field distributions on the focal plane and the image plane with a distance H from the focal plane; j represents the equal-interval zooming times of projector b, and j = 1, 2, 3…J, J represents the total number of movements; N represents N-step phase shift, and N≥3; n and n' respectively represent the nth fringe pattern projected on the focal plane or the defocused plane at the j position, n (or n') = 0, 1, 2…, N - 1; σ H represents the standard deviation of the point spread function; u represents the number of columns in the image pixel coordinates, with the unit of pixel, v represents the number of rows in the image pixel coordinates, with the unit of pixel; R2(u, v), B2(u, v) and C2(u, v) respectively represent the object surface reflectivity, the ambient light intensity and the contrast of the fringe; M2 represents the magnification of the measurement system; f0 represents the fringe frequency, and Φ(u, v) represents the initial phase of the fringe;

[0177] Extract the fringes on the focal plane respectively as in the formula

[0178]

[0179] The fringes on the defocused plane are as shown in Equation

[0180]

[0181] The modulation degree distributions of the fringes on the above-mentioned focal plane and defocused plane are as follows:

[0182]

[0183]

[0184] In the formula, M f (u, v) and M' f (u, v; σ H ) respectively represent the modulation degree distributions on the focal plane and the defocused plane; I(u, v) and I(u, v; σ H ) respectively represent the light field distributions on the focal plane and the image plane at a distance H from the focal plane; σ H represents the standard deviation of the point spread function; u represents the column number in the image pixel coordinates, with the unit of pixel, and v represents the row number in the image pixel coordinates, with the unit of pixel;

[0185] It can be understood that since the current value of the electronically tunable liquid lens changes at equal intervals, the current value of the electronically tunable liquid lens can also be determined by j;

[0186] Perform quadratic fitting interpolation on the curve formed by the modulation degree values of the corresponding pixels of the modulation degree distributions at each scanning position, and extract the maximum value of the modulation degree of the corresponding pixels, then the serial number j(t, u, v) corresponding to the maximum value of the modulation degree of the corresponding pixels (u, v) can be obtained max .

[0187] It can be understood that the serial number j(t, u, v) max is also the number of times the electronically tunable liquid lens is at the j(t, u, v) max th current value. At this time, the fringe pattern projected by the projector is in focus at the pixel point (u, v) on the calibration plane. Since the projector lens has a certain depth of field, and the pose of the calibration plane placed is perpendicular to the coaxial line of the projector and the camera a, at this time, take the serial number j(t, u mid , v mid ) max (u mid , v mid respectively represent the intermediate values of the total number of rows and columns of the image), then it can be ensured that the calibration plane is in focus.

[0188] In the spatial coordinate value acquisition module of each pixel point:

[0189] In some embodiments, the current value of the electronically adjustable liquid lens is automatically set according to the serial number corresponding to the maximum modulation degree, so that the fringe pattern projected by the projector is in a focused state on each calibration plane. A dual-camera autofocus system composed of the projector, camera a, and camera b is used, and the spatial coordinate values of each pixel point in the triangulation system on each calibration plane are assisted to be completed through N-step phase-shifted fringes.

[0190] Further, the projector projects vertical fringes and horizontal fringes respectively, and camera a and camera b collect synchronously. The nth fringe patterns of the collected vertical fringes and horizontal fringes are respectively expressed mathematically as

[0191]

[0192]

[0193] In the formula, and respectively represent the light field distributions of the nth fringe patterns of the vertical fringes and the horizontal fringes on the focal plane; u represents the number of columns in the image pixel coordinates, with the unit of pixel, v represents the number of rows in the image pixel coordinates, with the unit of pixel; N represents N-step phase shift, and N≥3; n represents the nth fringe pattern projected, n = 0, 1, 2…, N - 1; R2(u, v), B2(u, v) and C2(u, v) are respectively the object surface reflectivity, the ambient light intensity, and the contrast of the fringes; M2 is the magnification of the measurement system; and respectively represent the vertical fringe frequency and the horizontal fringe frequency; Φ V (u, v) and Φ H (u, v) respectively represent the initial phases of the vertical fringes and the horizontal fringes;

[0194] Calculate the respective phase values of and as follows:

[0195]

[0196]

[0197] In the formula, and respectively represent the truncated phases of the vertical fringes and the horizontal fringes, with the value range from -π to π and having a discontinuity of 2π; and respectively represent the light field distributions of the nth fringe patterns of the vertical fringes and the horizontal fringes on the focal plane; N represents N-step phase shift, and N≥3; n represents the nth fringe pattern projected, n = 0, 1, 2…, N - 1;

[0198] Respectively, and Add or subtract multiples of 2π, and the formula is as follows:

[0199]

[0200]

[0201] In the formula, Φ V (u, v) and Φ H (u, v) respectively represent and the continuous phases obtained by adding or subtracting multiples of 2π; and respectively represent the truncated phases of vertical stripes and horizontal stripes; k V (u, v) and k H (u, v) respectively represent the orders of vertical stripes and horizontal stripes;

[0202] Thus,

[0203] In the formula, Φ V (u, v) and Φ H (u, v) respectively represent the absolute phases of vertical stripes and horizontal stripes; T V and T H respectively represent the periods of vertical stripe frequency and horizontal stripe frequency; u represents the number of columns in the image pixel coordinates, with the unit of pixel, and v represents the number of rows in the image pixel coordinates, with the unit of pixel;

[0204] Substitute into to obtain the values of X W (u, v), Y W (u, v), Z W (u, v) in the world coordinate system. By traversing each point on the calibration plane, the spatial coordinate values X W (t, u, v), Y W (t, u, v), Z W (t, u, v) of each pixel point on the calibration plane can be determined.

[0205] In the mapping relationship module:

[0206] In some embodiments, the spatial coordinate values of each pixel point on the calibration plane in the triangulation system and the serial number corresponding to the maximum modulation value are subjected to quadratic fitting interpolation to obtain the mapping relationship between the spatial coordinate values of each pixel point on the calibration plane and the serial number corresponding to the maximum modulation value, including the following steps:

[0207] Within the measurement range determined by the farthest calibration plane and the nearest calibration plane in the vertical system, scan each calibration plane to obtain the spatial coordinate values X W(t, u, v), Y W (t, u, v), Z W (t, u, v), and respectively obtain the spatial coordinate values X of each pixel point of the calibration plane in the triangulation system W (t, u, v), Y W (t, u, v), Z W (t, u, v) and the serial number corresponding to the maximum modulation degree are subjected to quadratic fitting interpolation, so as to establish the spatial coordinate X W (t, u, v), Y W (t, u, v), Z W (t, u, v) and the serial number j(t, u, v) corresponding to the maximum modulation degree max The mapping relationship between them is:

[0208] X W X(t, u, v) = a(u, v) + b(u, v)j(t, u, v) max + c(u, v)j 2 (t, u, v) max

[0209] Y W Y(t, u, v) = a(u, v) + b(u, v)j(t, u, v) max + c(u, v)j 2 (t, u, v) max

[0210] Z W Z(t, u, v) = a(u, v) + b(u, v)j(t, u, v) max + c(u, v)j 2 (t, u, v) max

[0211] In the formula, t represents the t-th calibration plane, u and v respectively represent the row number and column number in the image pixel coordinates, X W (t, u, v), Y W (t, u, v), Z W (t, u, v) respectively represent the coordinate values in the X, Y, and Z axis directions of the t-th calibration plane at this row and column of the image, and j(t, u, v) max represents the serial number of the maximum modulation degree of the t-th calibration plane, and a(u, v), b(u, v), and c(u, v) are respectively the fitting coefficients of the depth value - maximum modulation degree serial number curve.

[0212] In the three-dimensional surface shape module of the reconstructed object to be measured:

[0213] In some embodiments, the projector zooms in on and scans the object to be measured, calculates the sequence number value corresponding to the maximum modulation value of each pixel point, and reconstructs the three-dimensional shape of the object to be measured according to the mapping relationship between the spatial coordinate values of each pixel point on the calibration plane and the sequence number corresponding to the maximum modulation value.

[0214] Further, place the object to be measured within the measurement range determined by the farthest calibration plane and the nearest calibration plane of the vertical system and keep its position unchanged. Change the current value of the electronically tunable liquid lens at equal intervals, so that the projector projects a zoomed light field onto the object to be measured. At each current value of the electronically tunable liquid lens, project N frames of fringe patterns with a fixed phase difference onto the object to be measured, and the camera synchronously captures the fringe patterns to obtain the sequence number j(t, u, v) corresponding to the maximum modulation value. max , according to obtain the pixel point coordinate values X W (t, u, v), Y W (t, u, v), Z W (t, u, v) of the object to be measured. By traversing all pixel points, the height distribution of the object to be measured can be obtained, that is, the reconstruction of the three-dimensional shape of the object to be measured is completed.

[0215] In a third aspect, the present application provides a computer device, including a storage and a processor. When the computer program stored in the storage is executed by the processor, the processor is caused to execute the steps of the optical vertical measurement method based on dual-camera autofocus as described above.

[0216] Among them, the computer device may be a computing device such as a desktop computer, a notebook, a palm computer, and a cloud server. The computer device can perform human-computer interaction with the user through a keyboard, a mouse, a remote control, a touchpad, or a voice control device, etc.

[0217] The memory at least includes one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (such as SD or D interface display memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disc, etc. In some embodiments, the memory may be an internal storage unit of the computer device, such as the hard disk or memory of the computer device. In other embodiments, the memory may also be an external storage device of the computer device, such as a plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, Flash Card, etc. equipped on the computer device. Of course, the memory may also include both the internal storage unit and the external storage device of the computer device. In this embodiment, the memory is commonly used to store the operating system and various application software installed on the computer device, such as the program code of the optical vertical measurement method based on dual-camera autofocus. In addition, the memory may also be used to temporarily store various data that have been output or will be output.

[0218] In some embodiments, the processor may be a central processing unit (CPU), controller, microcontroller, microprocessor, or other data processing chip. The processor is generally used to control the overall operation of the computer device. In this embodiment, the processor is used to run the program code stored in the memory or process data, such as running the program code of the optical vertical measurement method based on dual-camera autofocus.

[0219] Fourthly, an embodiment of the present application provides a computer-readable storage medium storing a computer program, and when the computer program is executed by a processor, the processor is caused to execute the steps of the optical vertical measurement method based on dual-camera autofocus as described above.

[0220] Wherein, the computer-readable storage medium stores an interface display program, and the interface display program can be executed by at least one processor, so that the at least one processor executes the steps of the optical vertical measurement method based on dual-camera autofocus as described above.

[0221] The schematic diagram of the optical vertical measurement system based on dual-camera autofocus of the present application is as Figure 2As shown, t is the calibration surface serial number, which is used to identify the calibration surfaces at different depth positions, wherein t=1 is calibration surface 1, and t=T is calibration surface T; the application includes a projector 1, an electronically adjustable focus liquid lens 2, a semi-transparent and semi-reflective mirror 3, and cameras a4 and b5 installed on the projector 1. According to the basic principle of imaging, Figure 2 As shown in the figure, after the light from the point light source passes through the ideal lens, it will be focused when the focus is on the imaging plane, thus forming a clear image. If the imaging plane deviates from the focal position, the image will become blurred whether it moves forward or backward. This degree of blur can be measured by "modulation", which reflects the sharpness and detail expression ability of the image. As the imaging plane gradually moves away from the focus, the image contour becomes blurred, the edge is unclear, and the modulation decreases accordingly; when the imaging plane gradually approaches the focus, the image details become more distinct, and the modulation value continues to increase. If the trend of modulation changing with the position of the imaging plane is plotted, its shape presents a symmetrical inverted "U" shape - the modulation reaches the highest value at the focus, and the farther away from the focus, the lower the value, and the descent process is symmetrically distributed. This relationship reveals the close connection between the imaging plane position and the image clarity, and also constitutes the theoretical basis for measuring the three-dimensional morphology of an object by the modulation method. By recording the position of each pixel when the modulation reaches the maximum value during the scanning process, the mapping relationship between the spatial coordinates corresponding to the pixel and the serial number corresponding to the maximum modulation value can be established, thereby realizing the reconstruction of the three-dimensional contour of the measured object 6.

[0222] Application Examples

[0223] The calibration depth is 0mm to 50mm, with a total of 11 calibration surfaces, and the distance between any two adjacent calibration surfaces is 5mm. The 3D object to be measured is a plaster head of a girl. A total of 300 different zoom positions are collected during the scanning of the object to be measured. Figure 3 It is the fringe pattern projected onto the object being measured when the projector changes its focal length for the 151st time. Figure 4 This is the error distribution diagram after the system tests the standard sphere and subtracts it from the fitted sphere. The root mean square error is 0.098mm. Figure 5 It is the reconstruction result after the calibration is completed by using the dual-camera auto-focus triangulation system to assist the vertical measurement system. It can be seen that the present invention can obtain a complete surface reconstruction result.

[0224] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. An optical vertical measurement method based on dual-camera autofocus, characterized in that, Including: Camera a and camera b form a triangulation system and complete the calibration of the triangulation system; Calculate the serial number corresponding to the zoom position at the maximum modulation depth; A dual-camera autofocus system composed of a projector, camera a, and camera b is used, and with the assistance of N-step phase-shifted fringes, the spatial coordinate values of each pixel point on each calibration plane in the triangulation system are completed; The spatial coordinate values of each pixel point on the calibration plane and the serial number corresponding to the maximum modulation depth are subjected to quadratic fitting interpolation to obtain the mapping relationship between the spatial coordinate values of each pixel point on the calibration plane and the serial number corresponding to the maximum modulation depth; Reconstruct the three-dimensional surface shape of the object to be measured.

2. The optical vertical measurement method based on dual-camera autofocus according to claim 1, wherein, The conversion from the system pixel coordinates to the world coordinates and the calibration of the triangulation system include the following steps: Establish the relationship between the 2D image pixel coordinates and the 2D image physical coordinates: Wherein, u represents the number of columns in the image pixel coordinates, with the unit of pixel, v represents the number of rows in the image pixel coordinates, with the unit of pixel; u0 represents the position of the origin of the column number of the image physical coordinates in the image pixel coordinates, and v0 represents the position of the origin of the row number of the image physical coordinates in the image pixel coordinates; x represents the number of columns in the image physical coordinates, with the unit of millimeter, and y represents the number of rows in the image physical coordinates, with the unit of millimeter; Establish the mapping relationship between the 2D image physical coordinates and the points in the camera three-dimensional coordinate system: Wherein, x represents the number of columns in the physical coordinates of the image, with the unit of millimeter, and y represents the number of rows in the physical coordinates of the image, with the unit of millimeter; f represents the focal length of the camera; X c , Y c , Z c are three mutually orthogonal axes of the camera coordinate system; It is converted into the following homogeneous coordinate expression: where x represents the number of columns in the physical coordinates of the image, in millimeters, y represents the number of rows in the physical coordinates of the image, in millimeters; f represents the focal length of the camera; X c , Y c , Z c are three mutually orthogonal axes of the camera coordinate system; Establish the conversion relationship between the camera coordinate system and the world coordinate system: At this time, the calibration of the triangulation system under ideal conditions is completed; Wherein, u represents the number of columns in the image pixel coordinates, with the unit of pixel, and v represents the number of rows in the image pixel coordinates, with the unit of pixel; f represents the focal length of the camera; f x = f / d x , f y = f / d y , d x and d y respectively represent the physical sizes of the camera unit pixel in the horizontal and vertical directions; u0 and v0 represent the coordinates of the camera optical center on the plane; R1 represents a 3×3 orthogonal rotation matrix, and T1 = [T x , T y , T z is a three-dimensional translation vector; X W , Y W , Z W represent three mutually orthogonal axes in the world coordinate system; M1 represents the internal parameter matrix; K1 is the external parameter matrix; In the actual triangulation system, the influence of tangential and radial distortions is: At this time, the correction of the calibration of the actual triangulation system is completed; Wherein, x represents the horizontal direction (number of columns) in the image physical coordinates, with the unit of millimeter, y represents the vertical direction (number of rows) in the image physical coordinates, with the unit of millimeter; x', y' respectively represent the coordinate values in the two directions after considering the distortion, and the parameters p1, p2 represent the tangential distortion coefficients, and ρ represents the distance from the coordinate point (x, y) to the origin.

3. The optical vertical measurement method based on dual-camera autofocus according to claim 1, wherein Provide a projector, and change the current value of the electronically tunable liquid lens at equal intervals, so that the projector zooms the projection light field to different calibration planes. For changing the current value of the electronically tunable liquid lens at equal intervals, project N frames of fringe patterns with a fixed phase difference onto the calibration plane. The light field distribution collected by camera a on the focal plane is expressed as: The light fields collected by camera a before and after the focal plane are expressed as: In the formula, and respectively represent the light field distributions on the focal plane and the image plane at a distance H from the focal plane; j represents the equal-interval zoom times of the projector b, and j = 1, 2, 3…J, where J represents the total number of movements; N represents N-step phase shift, and N≥3; n and n’ respectively represent the nth fringe pattern projected on the focal plane or the defocused plane at the j position, and n (or n’) = 0, 1, 2…, N-1; σ H represents the standard deviation of the point spread function; u represents the number of columns in the image pixel coordinates, with the unit of pixel, and v represents the number of rows in the image pixel coordinates, with the unit of pixel; R2(u, v), B2(u, v) and C2(u, v) respectively represent the surface reflectivity of the object, the ambient light intensity and the contrast of the fringes; M2 represents the magnification of the measurement system; f0 represents the fringe frequency, and Φ(u, v) represents the initial phase of the fringes; Extract the fringes on the focal plane respectively as in the formula The fringes on the defocused plane are as in the formula The modulation depth distributions of the fringes on the above-mentioned focal plane and defocused plane are as follows: Where, M f (u, v) and M' f (u, v; σ H ) represent the modulation transfer function distributions on the focal plane and the defocused plane, respectively; I(u, v) and I(u, v; σ H ) represent the light field distributions on the focal plane and the image plane at a distance H from the focal plane, respectively; σ H represents the standard deviation of the point spread function; u represents the column number in the image pixel coordinates, in pixels, and v represents the row number in the image pixel coordinates, in pixels; Perform quadratic fitting interpolation on the curve formed by the modulation values of the corresponding pixels in the modulation distribution at each scanning position, extract the maximum modulation value of the corresponding pixels, and then the sequence number j(t, u, v) corresponding to the maximum modulation value of the corresponding pixels (u, v) can be obtained. max 。 4. The optical vertical measurement method based on dual-camera autofocus according to claim 1, wherein Automatically set the current value of the electronically tunable liquid lens according to the serial number corresponding to the maximum modulation depth, so that the fringe pattern projected by the projector is in the focused state on each calibration plane. A dual-camera autofocus system composed of a projector, camera a, and camera b is used, and with the assistance of N-step phase-shifted fringes, the spatial coordinate values of each pixel point on each calibration plane in the triangulation system are completed.

5. The optical vertical measurement method based on dual-camera autofocus according to claim 4, wherein The projector projects vertical fringes and horizontal fringes respectively, and camera a and camera b collect synchronously. The nth fringe patterns of the collected vertical fringes and horizontal fringes are respectively expressed mathematically as In the formula, and respectively represent the light field distributions of the vertical and horizontal fringe patterns of the n-th frame on the focal plane; u represents the column number in the image pixel coordinates, with the unit of pixel, v represents the row number in the image pixel coordinates, with the unit of pixel; N represents N-step phase shift, and N≥3; n represents the n-th fringe pattern projected, n = 0, 1, 2…, N - 1; R2(u, v), B2(u, v) and C2(u, v) are the surface reflectivity of the object, the ambient light intensity and the contrast of the fringe pattern respectively; M2 is the magnification of the measurement system; and respectively represent the vertical fringe frequency and the horizontal fringe frequency; Φ V (u, v) and Φ H (u, v) respectively represent the initial phases of the vertical and horizontal fringe patterns; Calculate separately and their respective phase values as follows: In the formula, and respectively represent the truncated phases of vertical stripes and horizontal stripes, with a value range of -π to π and a 2π discontinuity; and respectively represent the light field distributions of the vertical stripes and horizontal stripes of the n-th frame of fringe patterns on the focal plane; N represents N-step phase shift, and N≥3; n represents the n-th fringe pattern of the projection, n = 0, 1, 2…, N - 1; Respectively, and are added or subtracted by multiples of 2π, and the formula is as follows: where Φ V (u, v) and Φ H (u, v) respectively represent and the continuous phases obtained by adding or subtracting multiples of 2π; and respectively represent the truncated phases of vertical stripes and horizontal stripes; k V (u, v) and k H (u, v) respectively represent the orders of vertical stripes and horizontal stripes; Thereby obtaining where Φ V (u, v) and Φ H (u, v) represent the absolute phases of the vertical and horizontal stripes respectively; T V and T H represent the periods of the vertical and horizontal stripe frequencies respectively; u represents the number of columns in the image pixel coordinates, in pixels, and v represents the number of rows in the image pixel coordinates, in pixels; Substitute into to obtain the values of X W (u, v), Y W (u, v), and Z W (u, v). By traversing each point on the calibration plane, the spatial coordinate values of each pixel point on the calibration plane, X W (t, u, v), Y W (t, u, v), and Z W (t, u, v) can be determined.

6. The optical vertical measurement method based on dual-camera autofocus according to claim 1, characterized in that, Within the measurement range determined by the farthest calibration plane and the nearest calibration plane in the vertical system, scan each calibration plane to obtain the spatial coordinate values X W (t, u, v), Y W (t, u, v), Z W (t, u, v), and respectively, for each pixel point on the calibration plane, the spatial coordinate values X W (t, u, v), Y W (t, u, v), Z W (t, u, v) and the serial number corresponding to the maximum modulation value are subjected to quadratic fitting interpolation, thereby establishing the mapping relationship between the spatial coordinates X W (t, u, v), Y W (t, u, v), Z W (t, u, v) and the serial number j(t, u, v) max corresponding to the maximum modulation value: X W (t, u, v) = a(u, v) + b(u, v)j(t, u, v) max + c(u, v)j 2 (t, u, v) max Y W (t, u, v) = a(u, v) + b(u, v)j(t, u, v) max + c(u, v)j 2 (t, u, v) max Z W (t, u, v) = a(u, v) + b(u, v)j(t, u, v) max + c(u, v)j 2 (t, u, v) max where t represents the t-th calibration plane, u and v represent the row number and column number in the image pixel coordinates, respectively, and X W (t, u, v), Y W (t, u, v), Z W (t, u, v) represent the coordinate values in the directions of the three coordinate axes X, Y, and Z of the t-th calibration plane at the row and column of the image, respectively, and j(t, u, v) max represents the modulation maximum sequence number of the t-th calibration plane, and a(u, v), b(u, v), and c(u, v) are the fitting coefficients of the depth value - maximum modulation sequence number curve, respectively.

7. The optical vertical measurement method based on dual-camera autofocus according to claim 1, wherein Place the object to be measured within the measurement range determined by the farthest calibration plane and the nearest calibration plane of the vertical system, and keep its position unchanged. Change the current value of the electronically adjustable focus liquid lens at equal intervals, so that the projector zooms the projection light field onto the object to be measured. At each current value of the electronically adjustable focus liquid lens, the camera synchronously acquires fringe patterns, and obtains the sequence number j(t, u, v) corresponding to the maximum modulation degree. max , according to X W (t, u, v) = a(u, v) + b(u, v)j(t, u, v) max + c(u, v)j 2 (t, u, v) max Y W (t, u, v) = a(u, v) + b(u, v)j(t, u, v) max + c(u, v)j 2 (t, u, v) max , obtain the pixel point coordinates of the object under test (u, v) Z W (t, u, v) = a(u, v) + b(u, v)j(t, u, v) max + c(u, v)j 2 (t, u, v) max The marked value X W (t, u, v), Y W (t, u, v), Z W (t, u, v). By traversing all pixel points, the height distribution of the object to be measured can be obtained, that is, the three-dimensional shape reconstruction of the object to be measured is completed.

8. An optical vertical measurement system based on dual-camera autofocus, characterized in that, Including: Triangulation system module, camera a and camera b form a triangulation system and complete the calibration of the triangulation system; Serial number module, calculate the serial number corresponding to the zoom position at the maximum modulation depth; The spatial coordinate value acquisition module for each pixel point utilizes a dual-camera autofocus system composed of a projector, Camera A, and Camera B, and completes the spatial coordinate values of each pixel point on each calibration plane in the triangulation system with the assistance of N-step phase-shifted fringes; The mapping relationship module performs quadratic fitting interpolation on the spatial coordinate values of each pixel point on the calibration plane in the triangulation system and the corresponding sequence number of the maximum modulation value, and obtains the mapping relationship between the spatial coordinate values of each pixel point on the calibration plane and the corresponding sequence number of the maximum modulation value; The three-dimensional surface shape reconstruction module for the object to be measured reconstructs the three-dimensional surface shape of the object to be measured.

9. A computer device, characterized in that, It includes a memory and a processor. When the computer program stored in the memory is executed by the processor, the processor executes the steps of the method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, A computer program is stored. When the computer program is executed by the processor, the processor executes the steps of the method according to any one of claims 1-7.

Images