Modulation degree measurement profilometry method, system and device and storage medium

Abstract

The invention discloses a modulation degree measurement profilometry method, system and device and a storage medium, and the method comprises the steps: enabling a projector a and a camera to form a triangulation system, and completing the calibration of the triangulation system; calculating the maximum value distribution of the modulation degree of each pixel point on any calibration surface, and obtaining a serial number corresponding to the maximum value of the modulation degree; obtaining any calibration surface in the vertical measurement system, and calculating the spatial coordinate value of each pixel point of the calibration surface in the triangulation system; performing quadratic fitting interpolation on the space coordinate value of each pixel point of the calibration surface in the triangulation system and the serial number corresponding to the maximum value of the modulation degree to obtain a mapping relation between the space coordinate value of each pixel point of the calibration surface and the serial number corresponding to the maximum value of the modulation degree; and reconstructing the three-dimensional surface shape of the measured object. In addition to the vertical measurement advantage of the original modulation degree measurement profilometry, the method can also provide the physical coordinate value of the three-dimensional surface shape of the measured object, and has a good application prospect in the aspect of the three-dimensional measurement technology.

Description

Technical Field

[0001] The present invention belongs to the technical field of structured light projection three-dimensional surface measurement, and in particular relates to a method, system, device and storage medium for modulation profilometry. Background Art

[0002] With the rapid development of 3D measurement technology, optical 3D surface vertical measurement methods have gained widespread attention in fields such as precision instruments, aerospace, and medicine. This technology offers the advantages of low loss, non-contact operation, and high precision, making it suitable for measuring complex objects with highly variable surface height distributions, such as grooves, steps, and deep holes.

[0003] Patent application number CN201110361261.2 discloses a method for rapid modulation profilometry using two orthogonal sinusoidal gratings. This patent describes a method for rapid modulation profilometry using two spaced-apart orthogonal sinusoidal gratings within optical 3D sensing technology. Two spaced-apart, orthogonal sinusoidal gratings are simultaneously imaged onto the object being measured, with the object positioned between the two grating imaging planes. A CCD camera captures images of the object's surface modulated by the orthogonal gratings from the same direction via a semi-transparent, semi-reflective mirror. Using Fourier transform, spatial frequency domain filtering, and inverse Fourier transform, the orthogonal grating images are separated to determine the modulation distribution of the two grating images on the object's surface. The object's height is then reconstructed using the previously calibrated relationship between the modulation ratio and height. In this patent, the grating projection direction and the CCD camera imaging direction are coaxial, enabling vertical measurement. This patent only requires a single image to recover the object's height. However, during the measurement process, this patent only provides the actual physical dimensions of the object in the Z coordinate. Summary of the Invention

[0004] The purpose of the present invention is to provide a method, system, device and storage medium for modulation profilometry to solve the problem that the prior art can only provide the actual physical size of the measured object in the Z direction.

[0005] The embodiment of the present application is implemented as follows: a method for modulation profilometry is provided, comprising: forming a triangulation system with a projector a and a camera, and completing calibration of the triangulation system;

[0006] Projector b and the camera form a vertical measurement system, which calculates the maximum modulation distribution of each pixel point on any calibration surface and obtains the serial number corresponding to the maximum modulation value;

[0007] Obtain any calibration surface in the vertical measurement system and calculate the spatial coordinate values ​​of each pixel point on the calibration surface in the triangulation system;

[0008] The spatial coordinate values ​​of each pixel point on the calibration surface in the triangulation system are respectively interpolated with the serial number corresponding to the maximum modulation index to obtain the mapping relationship between the spatial coordinate values ​​of each pixel point on the calibration surface and the serial number corresponding to the maximum modulation index;

[0009] Reconstruct the three-dimensional shape of the object being measured.

[0010] In some embodiments, the projector a and the camera form a triangulation system, and completing the triangulation system calibration includes the following steps:

[0011] Establish the relationship between 2D image pixel coordinates and 2D image physical coordinates:

[0012]

[0013] Where u represents the number of columns in the image pixel coordinates, in pixels; v represents the number of rows in the image pixel coordinates, in pixels; u0 represents the position of the origin of the image physical coordinate column in the image pixel coordinates, and v0 represents the position of the origin of the image physical coordinate row in the image pixel coordinates; x represents the number of columns in the image physical coordinates, in millimeters; y represents the number of rows in the image physical coordinates, in millimeters;

[0014] Establish the mapping relationship between the physical coordinates of the 2D image and the camera's 3D coordinate system points:

[0015]

[0016] Where x represents the number of columns in the physical coordinates of the image, in millimeters; y represents the number of rows in the physical coordinates of the image, in millimeters; f represents the focal length of the camera; X c , Y c , Z c are the three mutually orthogonal axes of the camera coordinate system;

[0017] Convert to the following homogeneous coordinate expression:

[0018]

[0019] Where x represents the number of columns in the physical coordinates of the image, in millimeters; y represents the number of rows in the physical coordinates of the image, in millimeters; f represents the focal length of the camera; X c , Y c , Z c are the three mutually orthogonal axes of the camera coordinate system;

[0020] Establish the transformation relationship between the camera coordinate system and the world coordinate system:

[0021]

[0022] Where u represents the number of columns in the image pixel coordinates, in pixels; v represents the number of rows in the image pixel coordinates, in pixels; f represents the focal length of the camera; f x =f / d x , f y =f / d y , d x and d y Respectively represent the physical size of the camera unit pixel in the horizontal and vertical directions; u0 and v0 represent the coordinates of the camera plane at the optical center; R1 represents a 3×3 orthogonal rotation matrix, T1 = [Tx, Ty, Tz] is a three-dimensional translation vector; X W , Y W , Z W Represents three mutually orthogonal axes in the world coordinate system; M1 represents the intrinsic parameter matrix; K1 is the external parameter matrix; at this point, the triangulation system calibration is completed under ideal conditions;

[0023] In a practical triangulation system, the effects of tangential and radial distortion are:

[0024] At this point, the correction of the actual triangulation system calibration is completed;

[0025] Where x represents the horizontal direction (number of columns) in the physical coordinates of the image, in millimeters; y represents the vertical direction (number of rows) in the physical coordinates of the image, in millimeters; x' and y' represent the coordinate values ​​in the two directions after considering the distortion, respectively; parameters p1 and p2 represent the tangential distortion coefficients; and ρ represents the distance from the coordinate point (x, y) to the origin.

[0026] In some embodiments, the projector b and the camera constitute a vertical measurement system, and calculating the maximum modulation distribution of each pixel point on any calibration surface and obtaining the serial number corresponding to the maximum modulation value includes the following steps:

[0027] Zoom the projector b along the optical axis at equal intervals, and project N frames of fringe patterns with a fixed phase difference onto the calibration surface at each zoom position. The light field distribution collected by the camera on the focal plane is expressed as:

[0028]

[0029] The light field collected by the camera before and after the focal plane is expressed as:

[0030]

[0031] Where, and represents the light field distribution on the focal plane and the image plane at a distance H from the focal plane, respectively; j represents the number of equally spaced zooms of projector b, and j = 1, 2, 3…J, J represents the total number of moves; N represents the N-step phase shift, and N ≥ 3; n and n' distributions represent the nth fringe pattern projected on the focal plane or the defocused plane at position j, n (or n') = 0, 1, 2…, N-1; σ H represents the standard deviation of the point spread function; u represents the number of columns in the image pixel coordinates, in pixels, and v represents the number of rows in the image pixel coordinates, in pixels; R2(u,v), B2(u,v) and C2(u,v) represent the surface reflectivity, ambient light intensity and fringe contrast, respectively; M2 represents the measurement system magnification; f0 represents the fringe frequency, and Φ(u,v) represents the initial phase of the fringe;

[0032] Extract the fringes on the focal plane as follows

[0033]

[0034] The fringes on the out-of-focus plane are as follows

[0035]

[0036] The modulation distribution of the fringes on the above focal plane and out-of-focus plane is as follows:

[0037]

[0038] Where M f (u,v) and M' f (u,v;σ H ) represent the modulation distribution on the focal plane and the defocus plane respectively; I(u,v) and I(u,v; σ H ) represent the light field distribution on the focal plane and the image plane at a distance H from the focal plane; σ H represents the standard deviation of the point spread function; u represents the number of columns in the image pixel coordinates, in pixels; v represents the number of rows in the image pixel coordinates, in pixels;

[0039] Perform quadratic fitting interpolation on the curve formed by the modulation values ​​of the pixels with the same name at each scanning position, extract the maximum modulation value of the pixels with the same name, and you can get the serial number j(t,u,v) corresponding to the maximum modulation value of the pixels with the same name (u,v) max , where t represents the tth calibration surface, u and v represent the number of rows and columns in the image pixel coordinates, respectively, and the serial number j(t,u,v) max A floating point number.

[0040] In some embodiments, obtaining an arbitrary calibration surface in a vertical measurement system and calculating the spatial coordinate values ​​of each pixel point of the calibration surface in a triangulation system includes the following steps:

[0041] The XYZ coordinate values ​​of each pixel point of the calibration surface in the triangulation system are calculated according to the Fourier transform method, wavelet transform method, S transform method or phase shift algorithm.

[0042] In some embodiments, calculating the XYZ coordinate values ​​of each pixel point of the calibration surface in the triangulation system according to the phase shift algorithm includes the following steps:

[0043] Projector a projects vertical stripes and horizontal stripes on the focal plane, and the camera collects them synchronously. The n-th frame of the vertical stripes and horizontal stripes collected are mathematically expressed as

[0044]

[0045]

[0046] Where, and represents the light field distribution of the nth fringe pattern of vertical and horizontal stripes on the focal plane respectively; u represents the number of columns in the image pixel coordinates, in pixels, and v represents the number of rows in the image pixel coordinates, in pixels; N represents the N-step phase shift, and N ≥ 3; n represents the nth fringe pattern of the projection, n = 0, 1, 2…, N-1; R2(u,v), B2(u,v) and C2(u,v) are the reflectivity of the object surface, the ambient light intensity and the contrast of the fringe, respectively; M2 is the magnification of the measurement system; and Respectively expressed as vertical stripe frequency and horizontal stripe frequency; Φ V (u, v) and Φ H (u, v) represent the initial phases of vertical and horizontal stripes, respectively;

[0047] Calculate separately and The respective phase values ​​are as follows:

[0048]

[0049]

[0050] Where, and They are expressed as the truncated phases of vertical and horizontal stripes, ranging from -π to π, with a discontinuity of 2π; and They represent the light field distribution of the n-th fringe pattern of vertical and horizontal stripes on the focal plane respectively; N represents the N-step phase shift, and N ≥ 3; n represents the n-th fringe pattern of the projection, n = 0, 1, 2…, N-1;

[0051] Respectively and To add or subtract multiples of 2π, the formula is as follows:

[0052]

[0053]

[0054] Where, Φ V (u, v) and Φ H (u,v) respectively represent and Continuous phase obtained by adding or subtracting multiples of 2π; and They are respectively represented as the truncated phases of vertical and horizontal stripes; k V (u,v) and k H (u, v) represent the order of vertical stripes and horizontal stripes respectively;

[0055] Thus obtaining

[0056] Where, Φ V (u, v) and Φ H (u, v) represent the absolute phases of vertical and horizontal stripes respectively; T V and T H They are respectively expressed as the period of vertical stripe frequency and horizontal stripe frequency; u represents the number of columns in the image pixel coordinates, in pixels; v represents the number of rows in the image pixel coordinates, in pixels;

[0057] Will Substitution You can get the X in the world coordinate system W (u,v),Y W (u,v), Z W (u,v) value, traverse each point on the calibration surface to determine the spatial coordinate value X of each pixel point on the calibration surface W (t,u,v),Y W (t,u,v),Z W (t,u,v).

[0058] In some embodiments, quadratic fitting interpolation is performed on the spatial coordinate values ​​of each pixel point on the calibration surface in the triangulation system and the serial number corresponding to the maximum modulation index to obtain a mapping relationship between the spatial coordinate value of each pixel point on the calibration surface and the serial number corresponding to the maximum modulation index, including the following steps:

[0059] Within the measurement range determined by the farthest calibration surface and the nearest calibration surface of the vertical system, complete the scanning of each calibration surface and obtain the spatial coordinate value X of each pixel point on the calibration surface. W (t,u,v),Y W (t,u,v),Z W (t,u,v), and the spatial coordinate value X of each pixel point on the calibration surface W (t,u,v),Y W (t,u,v),Z W (t,u,v) and the maximum modulation index corresponding to the serial number are interpolated by quadratic fitting to establish the spatial coordinate value X W (t,u,v),Y W (t,u,v),Z W (t,u,v) corresponds to the maximum value of the modulation index, the sequence number j(t,u,v) max The mapping relationship between them:

[0060] X W (t,u,v)=a(u,v)+b(u,v)j(t,u,v) max +c(u,v)j 2 (t,u,v) max

[0061] Y W (t,u,v)=a(u,v)+b(u,v)j(t,u,v) max +c(u,v)j 2 (t,u,v) max

[0062] Z W (t,u,v)=a(u,v)+b(u,v)j(t,u,v) max +c(u,v)j 2 (t,u,v) max

[0063] Where t represents the tth calibration surface, u and v represent the number of rows and columns in the image pixel coordinates, respectively, and X W (t,u,v),Y W (t,u,v),Z W (t,u,v) represent the coordinate values ​​of the t-th calibration surface in the row and column of the image in the directions of the three coordinate axes X, Y, and Z, respectively, j(t,u,v) max It represents the maximum modulation index of the t-th calibration surface, a(u,v), b(u,v), and c(u,v) are the fitting coefficients of the depth value-maximum modulation index curve.

[0064] In some embodiments, projector b zooms and scans the object to be measured, calculates the serial number value corresponding to the maximum modulation value of each pixel point, and reconstructs the three-dimensional surface shape of the object to be measured based on the mapping relationship between the spatial coordinate value of each pixel point on the calibration surface and the serial number corresponding to the maximum modulation value, specifically including:

[0065] Place the object to be measured within the calibration system and keep the position unchanged. Change the current value of the electronically adjustable focus liquid lens at equal intervals so that projector b zooms and projects the light field onto the object to be measured. At each current value of the electronically adjustable focus liquid lens, N frames of fringe patterns with a fixed phase difference are projected onto the object to be measured. The camera synchronously collects the fringe pattern and obtains the serial number j(t,u,v) corresponding to the maximum modulation index. max , according to X W (t,u,v)=a(u,v)+b(u,v)j(t,u,v) max +c(u,v)j 2 (t,u,v) max Y W (t,u,v)=a(u,v)+b(u,v)j(t,u,v) max +c(u,v)j 2 (t,u,v) max , obtain the pixel coordinate Z of the object being measured (u,v) W (t,u,v)=a(u,v)+b(u,v)j(t,u,v) max +c(u,v)j 2 (t,u,v) max

[0066] Value X W (t,u,v),Y W (t,u,v),Z W (t,u,v), traverse all pixel points to obtain the height distribution of the object being measured, that is, the reconstruction of the three-dimensional surface shape of the object being measured is completed.

[0067] Accordingly, an embodiment of the present application further provides a system for modulation profilometry, comprising:

[0068] A triangulation system calibration module is used to form a triangulation system with the projector a and the camera and complete the triangulation system calibration;

[0069] The maximum modulation index corresponding serial number module is used to form a vertical measurement system with projector b and camera, calculate the maximum modulation index distribution of each pixel point on any calibration surface, and obtain the serial number corresponding to the maximum modulation index;

[0070] The spatial coordinate value module of each pixel point is used to obtain any calibration surface in the vertical measurement system and calculate the spatial coordinate value of each pixel point of the calibration surface in the triangulation system;

[0071] A mapping relationship module is used to perform quadratic fitting interpolation on the spatial coordinate values ​​of each pixel point on the calibration surface in the triangulation system and the serial number corresponding to the maximum modulation index, thereby obtaining a mapping relationship between the spatial coordinate values ​​of each pixel point on the calibration surface and the serial number corresponding to the maximum modulation index;

[0072] The module for reconstructing the three-dimensional surface shape of the object under test is used to reconstruct the three-dimensional surface shape of the object under test.

[0073] Accordingly, an embodiment of the present application also provides a computer device, including a storage and a processor, wherein the storage stores a computer program, and when the computer program is executed by the processor, the processor performs the steps of the above method.

[0074] Accordingly, an embodiment of the present application further provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the processor executes the steps of the above method.

[0075] In summary, due to the adoption of the above-mentioned technical solution, the beneficial effects of the present invention are as follows: the present application only requires the addition of an additional projector b to the traditional modulation profilometry measurement system. By adding projector b and a camera, a triangulation system can be formed to assist the modulation profilometry measurement system in completing spatial system calibration. Because projector b has a large depth of field and can cover the entire measurement range, there is no need to adjust the focal length to ensure that the projected fringes remain clearly focused on each calibration surface. In addition, while retaining the vertical measurement advantages of the original modulation profilometry, it can also provide the physical coordinate values ​​of the three-dimensional surface shape of the measured object, enabling a more comprehensive reconstruction of the object's true three-dimensional physical dimensions. In aerospace and precision manufacturing, this technology can be used for defect detection, workpiece deformation analysis, and assembly accuracy assessment. In contrast, providing only Z-axis information without XY scale data makes it difficult to accurately determine the overall shape of the object, limiting its practicality. Therefore, the technology proposed in the present invention has broader application prospects in the field of three-dimensional measurement technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0076] Figure 1 A flow chart of a method for modulation profilometry provided by an embodiment of the present invention;

[0077] Figure 2 A schematic diagram of the modulation profilometry method according to an embodiment of the present invention;

[0078] Figure 3 A light field distribution diagram projected by a projector onto the object under test when the electronically adjustable focus lens provided by an embodiment of the present invention changes its current value for the 151st time;

[0079] Figure 4This is a reconstruction result diagram after the triangulation system is used to assist the vertical measurement system in completing calibration according to an embodiment of the present invention. Description of the drawings:

[0081] 1-projector a, 2-projector b, 3-electronically adjustable focus lens, 4-beam splitter, 5-camera, 6-object to be measured. DETAILED DESCRIPTION

[0082] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0083] The technical solution of this application is as follows:

[0084] like Figure 1 As shown, in a first aspect, embodiments of the present application provide a method for modulation profilometry, comprising:

[0085] S01, projector a and camera form a triangulation system, and complete the triangulation system calibration;

[0086] S02, projector b and camera form a vertical measurement system, calculate the maximum modulation distribution of each pixel point on any calibration surface, and obtain the serial number corresponding to the maximum modulation value;

[0087] S03, obtaining any calibration surface in the vertical measurement system, and calculating the spatial coordinate values ​​of each pixel point of the calibration surface in the triangulation system;

[0088] S04, performing quadratic fitting interpolation on the spatial coordinate values ​​of each pixel point on the calibration surface in the triangulation system and the serial number corresponding to the maximum modulation index, to obtain a mapping relationship between the spatial coordinate values ​​of each pixel point on the calibration surface and the serial number corresponding to the maximum modulation index;

[0089] S05. Reconstruct the three-dimensional shape of the object being measured.

[0090] This application only needs to add an additional projector b to the traditional modulation profilometry measurement system, and then a triangulation system can be formed by adding projector b and a camera to assist the modulation profilometry measurement system in completing the calibration of the spatial system. Since projector b has a large depth of field and can cover the entire measurement range, there is no need to adjust the focal length to ensure that the projected stripes remain in a clear and focused state on each calibration surface. In addition, in addition to retaining the vertical measurement advantages of the original modulation profilometry, it can also provide the physical coordinate values ​​of the three-dimensional surface shape of the object being measured, which can achieve a more comprehensive reconstruction of the true three-dimensional physical size of the object. In aerospace and precision manufacturing, this technology can be used for defect detection, workpiece deformation analysis, and assembly accuracy assessment. In contrast, only providing Z-axis information and lacking XY scale data makes it difficult to accurately determine the overall shape of the object, limiting its practicality. Therefore, the technology proposed in the present invention has a broader application prospect in the field of three-dimensional measurement technology. In the said S01:

[0091] In some embodiments, the projector a and the camera form a triangulation system, and completing the triangulation system calibration includes the following steps:

[0092] S011. Establish the relationship between the 2D image pixel coordinates and the 2D image physical coordinates:

[0093]

[0094] Where u represents the number of columns in the image pixel coordinates, in pixels; v represents the number of rows in the image pixel coordinates, in pixels; u0 represents the position of the origin of the image physical coordinate column in the image pixel coordinates; v0 represents the position of the origin of the image physical coordinate row in the image pixel coordinates; x represents the number of columns in the image physical coordinates, in millimeters; y represents the number of rows in the image physical coordinates, in millimeters;

[0095] S012. Establish the mapping relationship between the physical coordinates of the 2D image and the camera's 3D coordinate system points:

[0096]

[0097] Where x represents the number of columns in the physical coordinates of the image, in millimeters; y represents the number of rows in the physical coordinates of the image, in millimeters; f represents the focal length of the camera; X c , Y c , Z c are the three mutually orthogonal axes of the camera coordinate system;

[0098] Convert to the following homogeneous coordinate expression:

[0099]

[0100] Where x represents the number of columns in the physical coordinates of the image, in millimeters; y represents the number of rows in the physical coordinates of the image, in millimeters; f represents the focal length of the camera; X c , Y c , Z c are the three mutually orthogonal axes of the camera coordinate system;

[0101] S013. Establish the conversion relationship between the camera coordinate system and the world coordinate system:

[0102]

[0103] Where u represents the number of columns in the image pixel coordinates, in pixels; v represents the number of rows in the image pixel coordinates, in pixels; f represents the focal length of the camera; f x =f / d x , f y =f / d y , d x and d y Respectively represent the physical size of the camera unit pixel in the horizontal and vertical directions; u0 and v0 represent the coordinates of the camera plane at the optical center; R1 represents a 3×3 orthogonal rotation matrix, T1=[T x ,T y ,T z ] is the three-dimensional translation vector; X W , Y W , Z W Represents three mutually orthogonal axes in the world coordinate system; M1 represents the intrinsic parameter matrix; K1 is the external parameter matrix; at this point, the triangulation system calibration is completed under ideal conditions;

[0104] S014. In an actual triangulation system, the effects of tangential and radial distortion are:

[0105] At this point, the correction of the actual triangulation system calibration is completed;

[0106] Where x represents the horizontal direction (number of columns) in the physical coordinates of the image, in millimeters; y represents the vertical direction (number of rows) in the physical coordinates of the image, in millimeters; x' and y' represent the coordinate values ​​in the two directions after considering the distortion, respectively; parameters p1 and p2 represent the tangential distortion coefficients; and ρ represents the distance from the coordinate point (x, y) to the origin.

[0107] I understand. The relationship between image pixels and the world's three-dimensional coordinate system is established.

[0108] It can be understood that the intrinsic parameter matrix contains internal parameters such as the effective focal length of the camera and the coordinates of the camera's principal point; the external parameter matrix contains the rotation matrix and translation vector, which is determined by the position of the camera coordinate system relative to the world coordinate system.

[0109] It can be understood that in an actual triangulation system, to achieve high-precision measurement, it is also necessary to consider the thickness and distortion of the camera lens, that is, the influence of tangential and radial distortion.

[0110] It can be understood that the optical path is reversible, and accordingly, the projector a can be regarded as a reverse camera. The imaging system of the projector is usually described using the imaging system model of a camera.

[0111] It can be understood that projector a and the camera constitute a triangulation system, and the purpose of completing the triangulation system calibration is to provide system parameters for projector a and the camera, and to treat the calibration surface in the vertical measurement system as the object to be measured, which can provide the X, Y, and Z point cloud coordinate values ​​in the world coordinate system. It can also be understood that the calibration range of the triangulation system is slightly larger than that of the vertical measurement system.

[0112] In the S02:

[0113] In some embodiments, the projector b and the camera constitute a vertical measurement system, and calculating the maximum modulation distribution of each pixel point on any calibration surface and obtaining the serial number corresponding to the maximum modulation value includes the following steps:

[0114] S021. Zoom the projector b along the optical axis at equal intervals, and project N frames of fringe patterns with a fixed phase difference onto the calibration surface at each moving position. The light field distribution collected by the camera on the focal plane is expressed as:

[0115]

[0116] The light field collected by the camera before and after the focal plane is expressed as:

[0117]

[0118] Where, and represents the light field distribution on the focal plane and the image plane at a distance H from the focal plane, respectively; j represents the number of times the projector b zooms along the optical axis at equal intervals, and j = 1, 2, 3…J, J represents the total number of moves; N represents the N-step phase shift, and N ≥ 3; n and n' distributions represent the nth fringe pattern projected on the focal plane or the defocused plane at position j, n (or n') = 0, 1, 2…, N-1; σ H represents the standard deviation of the point spread function; u represents the number of columns in the image pixel coordinates, in pixels, and v represents the number of rows in the image pixel coordinates, in pixels; R2(u,v), B2(u,v) and C2(u,v) represent the surface reflectivity, ambient light intensity and fringe contrast, respectively; M2 represents the measurement system magnification; f0 represents the fringe frequency, and Φ(u,v) represents the initial phase of the fringe;

[0119] S022, extract the fringes on the focal plane as follows

[0120]

[0121] The fringes on the out-of-focus plane are as follows

[0122]

[0123] The modulation distribution of the fringes on the above focal plane and out-of-focus plane is as follows:

[0124]

[0125] Where M f (u,v) and M' f (u,v;σ H ) represent the modulation distribution on the focal plane and the defocus plane respectively; I(u,v) and I(u,v; σ H ) represent the light field distribution on the focal plane and the image plane at a distance H from the focal plane; σ H represents the standard deviation of the point spread function; u represents the number of columns in the image pixel coordinates, in pixels; v represents the number of rows in the image pixel coordinates, in pixels;

[0126] S023. Perform quadratic fitting interpolation on the curve formed by the modulation values ​​of the pixels with the same name at each scanning position, extract the maximum modulation value of the pixels with the same name, and obtain the serial number j(t,u,v) corresponding to the maximum modulation value of the pixels with the same name (u,v). max , where t represents the tth calibration surface, u and v represent the number of rows and columns in the image pixel coordinates, respectively, and the serial number j(t,u,v) max A floating point number.

[0127] In said S021:

[0128] Furthermore, the projector b is fixed on the optical platform, and the current value of the electronically adjustable focus liquid lens is changed at equal intervals, so that the projector b zooms the projected light field to different calibration surfaces.

[0129] In said S03:

[0130] In some embodiments, obtaining an arbitrary calibration surface in a vertical measurement system and calculating the spatial coordinate values ​​of each pixel point of the calibration surface in a triangulation system includes the following steps:

[0131] The spatial coordinate values ​​of each pixel point of the calibration surface in the triangulation system are calculated according to the Fourier transform method, wavelet transform method, S transform method or phase shift algorithm.

[0132] Furthermore, calculating the spatial coordinate values ​​of each pixel point of the calibration surface in the triangulation system according to the phase shift algorithm includes the following steps:

[0133] S031. Projector a projects vertical stripes and horizontal stripes on the focal plane, and the camera collects them synchronously. The n-th frame of the vertical stripes and horizontal stripes collected are mathematically expressed as

[0134]

[0135] Where, and represents the light field distribution of the nth fringe pattern of vertical and horizontal stripes on the focal plane respectively; u represents the number of columns in the image pixel coordinates, in pixels, and v represents the number of rows in the image pixel coordinates, in pixels; N represents the N-step phase shift, and N ≥ 3; n represents the nth fringe pattern of the projection, n = 0, 1, 2…, N-1; R2(u,v), B2(u,v) and C2(u,v) are the reflectivity of the object surface, the ambient light intensity and the contrast of the fringe, respectively; M2 is the magnification of the measurement system; and Respectively expressed as vertical stripe frequency and horizontal stripe frequency; Φ V (u, v) and Φ H (u, v) represent the initial phases of vertical and horizontal stripes, respectively;

[0136] S032, calculate separately and The respective phase values ​​are as follows:

[0137]

[0138]

[0139] Where, and They are expressed as the truncated phases of vertical and horizontal stripes, ranging from -π to π, with a discontinuity of 2π; and They represent the light field distribution of the n-th fringe pattern of vertical and horizontal stripes on the focal plane respectively; N represents the N-step phase shift, and N ≥ 3; n represents the n-th fringe pattern of the projection, n = 0, 1, 2…, N-1;

[0140] S033, respectively and To add or subtract multiples of 2π, the formula is as follows:

[0141]

[0142]

[0143] Where, Φ V (u, v) and Φ H (u,v) respectively represent and Continuous phase obtained by adding or subtracting multiples of 2π; and They are respectively represented as the truncated phases of vertical and horizontal stripes; k V (u,v) and k H (u, v) represent the order of vertical stripes and horizontal stripes respectively;

[0144] Thus obtaining

[0145] Where, Φ V (u, v) and Φ H (u, v) represent the absolute phases of vertical and horizontal stripes respectively; T V and T H They are respectively expressed as the period of vertical stripe frequency and horizontal stripe frequency; u represents the number of columns in the image pixel coordinates, in pixels; v represents the number of rows in the image pixel coordinates, in pixels;

[0146] Will Substitution You can get the X in the world coordinate system W (u,v),Y W (u,v), Z W (u,v) value, traverse each point on the calibration surface to determine the spatial coordinate value X of each pixel point on the calibration surface W (t,u,v),Y W (t,u,v),Z W (t,u,v).

[0147] It can be understood that the phase shift algorithm is an N-step (N≥3) phase shift algorithm with equal phase shifts.

[0148] Understandably, and Add or subtract multiples of 2π to eliminate the discontinuity of 2π.

[0149] In said S04:

[0150] The spatial coordinate values ​​of each pixel point on the calibration surface in the triangulation system are respectively interpolated with the serial number corresponding to the maximum modulation index to obtain a mapping relationship between the spatial coordinate values ​​of each pixel point on the calibration surface and the serial number corresponding to the maximum modulation index, including the following steps:

[0151] Within the measurement range determined by the farthest calibration surface and the nearest calibration surface of the vertical system, complete the scanning of each calibration surface and obtain the spatial coordinate value X of each pixel point on the calibration surface.W (t,u,v),Y W (t,u,v),Z W (t,u,v), and the spatial coordinate value X of each pixel point on the calibration surface W (t,u,v),Y W (t,u,v),Z W (t,u,v) and the maximum modulation index corresponding to the serial number are interpolated by quadratic fitting to establish the spatial coordinate value X W (t,u,v),Y W (t,u,v),Z W (t,u,v) corresponds to the maximum value of the modulation index, the sequence number j(t,u,v) max The mapping relationship between them:

[0152] X W (t,u,v)=a(u,v)+b(u,v)j(t,u,v) max +c(u,v)j 2 (t,u,v) max

[0153] Y W (t,u,v)=a(u,v)+b(u,v)j(t,u,v) max +c(u,v)j 2 (t,u,v) max

[0154] Z W (t,u,v)=a(u,v)+b(u,v)j(t,u,v) max +c(u,v)j 2 (t,u,v) max

[0155] Where t represents the tth calibration surface, u and v represent the number of rows and columns in the image pixel coordinates, respectively, and X W (t,u,v),Y W (t,u,v),Z W (t,u,v) represent the coordinate values ​​of the t-th calibration surface in the row and column of the image in the directions of the three coordinate axes X, Y, and Z, respectively, j(t,u,v) max It represents the maximum modulation index of the t-th calibration surface, a(u,v), b(u,v), and c(u,v) are the fitting coefficients of the depth value-maximum modulation index curve.

[0156] In said S05:

[0157] In some embodiments, projector b zooms and scans the object to be measured, calculates the serial number value corresponding to the maximum modulation value of each pixel point, and reconstructs the three-dimensional surface shape of the object to be measured based on the mapping relationship between the spatial coordinate value of each pixel point on the calibration surface and the serial number corresponding to the maximum modulation value, specifically including:

[0158] Place the object to be measured within the calibration system and keep the position unchanged. Change the current value of the electronically adjustable focus liquid lens at equal intervals so that projector b zooms and projects the light field onto the object to be measured. At each current value of the electronically adjustable focus liquid lens, N frames of fringe patterns with a fixed phase difference are projected onto the object to be measured. The camera synchronously collects the fringe pattern and obtains the serial number j(t,u,v) corresponding to the maximum modulation index. max , according to X W (t,u,v)=a(u,v)+b(u,v)j(t,u,v) max +c(u,v)j 2 (t,u,v) max Y W (t,u,v)=a(u,v)+b(u,v)j(t,u,v) max +c(u,v)j 2 (t,u,v) max , obtain the pixel coordinate Z of the object being measured (u,v) W (t,u,v)=a(u,v)+b(u,v)j(t,u,v) max +c(u,v)j 2 (t,u,v) max

[0159] Value X W (t,u,v),Y W (t,u,v),Z W (t,u,v), traverse all pixel points to obtain the height distribution of the object being measured, that is, the reconstruction of the three-dimensional surface shape of the object being measured is completed.

[0160] For example, Figure 2 As shown, the triangulation system includes a projector a1 and a camera 5, the vertical measurement system includes a projector b2 and a camera 5, the projector a1 and the electronic focus-adjustable lens 3 are installed on the projector a1, the serial number 4 is a semi-transparent and semi-reflective mirror, the object to be measured 6, t is the calibration surface serial number, which is used to identify the calibration surface of different depth values, where t=1 is calibration surface 1, and t=T represents calibration surface T.

[0161] According to the imaging principle, Figure 2As shown in the figure, light from a point source converges at the focal point on the imaging plane after passing through an ideal lens, forming a sharp image. However, when the imaging plane is positioned before or after the focal plane, the image becomes blurred. The degree of this blur can be described by modulation, which directly affects image sharpness and detail. As the distance between the imaging plane and the focal plane increases, the image becomes blurrier, details and edges become less clear, and the modulation value decreases accordingly. Conversely, as the imaging plane approaches the focal plane, the image becomes sharper, details and edges become sharper, and the modulation value increases accordingly. If the modulation value is plotted against the position of the imaging plane, it forms an inverted "U" graph (modulation on the vertical axis, imaging plane position on the horizontal axis). The curve reaches its peak when the imaging plane coincides with the focal plane; as the imaging plane moves forward or backward, the modulation value decreases symmetrically. This curve clearly demonstrates the effect of imaging plane position on modulation (image sharpness) and the relationship between modulation and imaging plane position. This is also the basic principle of reconstructing the three-dimensional surface shape of an object using modulation profilometry, that is, establishing a mapping relationship between the spatial coordinate value of each pixel point on the calibration surface and the serial number corresponding to the maximum modulation value.

[0162] In a second aspect, an embodiment of the present application provides a system for modulation profilometry, comprising:

[0163] A triangulation system calibration module is used to form a triangulation system with the projector a and the camera and complete the triangulation system calibration;

[0164] The maximum modulation index corresponding serial number module is used to form a vertical measurement system with projector b and camera, calculate the maximum modulation index distribution of each pixel point on any calibration surface, and obtain the serial number corresponding to the maximum modulation index;

[0165] The spatial coordinate value module of each pixel point is used to obtain any calibration surface in the vertical measurement system and calculate the spatial coordinate value of each pixel point of the calibration surface in the triangulation system;

[0166] A mapping relationship module is used to perform quadratic fitting interpolation on the spatial coordinate values ​​of each pixel point on the calibration surface in the triangulation system and the serial number corresponding to the maximum modulation index, thereby obtaining a mapping relationship between the spatial coordinate values ​​of each pixel point on the calibration surface and the serial number corresponding to the maximum modulation index;

[0167] The module for reconstructing the three-dimensional surface shape of the object under test is used to reconstruct the three-dimensional surface shape of the object under test.

[0168] In the triangulation system calibration module:

[0169] In some embodiments, the projector a and the camera form a triangulation system, and completing the triangulation system calibration includes the following steps:

[0170] S011. Establish the relationship between the 2D image pixel coordinates and the 2D image physical coordinates:

[0171]

[0172] Where u represents the number of columns in the image pixel coordinates, in pixels; v represents the number of rows in the image pixel coordinates, in pixels; u0 represents the position of the origin of the image physical coordinate column in the image pixel coordinates; v0 represents the position of the origin of the image physical coordinate row in the image pixel coordinates; x represents the number of columns in the image physical coordinates, in millimeters; y represents the number of rows in the image physical coordinates, in millimeters;

[0173] S012. Establish the mapping relationship between the physical coordinates of the 2D image and the camera's 3D coordinate system points:

[0174]

[0175] Where x represents the number of columns in the physical coordinates of the image, in millimeters; y represents the number of rows in the physical coordinates of the image, in millimeters; f represents the focal length of the camera; X c , Y c , Z c are the three mutually orthogonal axes of the camera coordinate system;

[0176] Convert to the following homogeneous coordinate expression:

[0177]

[0178] Where x represents the number of columns in the physical coordinates of the image, in millimeters; y represents the number of rows in the physical coordinates of the image, in millimeters; f represents the focal length of the camera; X c , Y c , Z c are the three mutually orthogonal axes of the camera coordinate system;

[0179] S013. Establish the conversion relationship between the camera coordinate system and the world coordinate system:

[0180]

[0181] Where u represents the number of columns in the image pixel coordinates, in pixels; v represents the number of rows in the image pixel coordinates, in pixels; f represents the focal length of the camera; f x =f / d x , f y =f / d y , d x and d yRespectively represent the physical size of the camera unit pixel in the horizontal and vertical directions; u0 and v0 represent the coordinates of the camera plane at the optical center; R1 represents a 3×3 orthogonal rotation matrix, T1=[T x ,T y ,T z ] is the three-dimensional translation vector; X W , Y W , Z W Represents three mutually orthogonal axes in the world coordinate system; M1 represents the intrinsic parameter matrix; K1 is the external parameter matrix; at this point, the triangulation system calibration is completed under ideal conditions;

[0182] S014. In an actual triangulation system, the effects of tangential and radial distortion are:

[0183] At this point, the correction of the actual triangulation system calibration is completed;

[0184] Where x represents the horizontal direction (number of columns) in the physical coordinates of the image, in millimeters; y represents the vertical direction (number of rows) in the physical coordinates of the image, in millimeters; x' and y' represent the coordinate values ​​in the two directions after considering the distortion, respectively; parameters p1 and p2 represent the tangential distortion coefficients; and ρ represents the distance from the coordinate point (x, y) to the origin.

[0185] I understand. The relationship between image pixels and the world's three-dimensional coordinate system is established.

[0186] It can be understood that the intrinsic parameter matrix contains internal parameters such as the effective focal length of the camera and the coordinates of the camera's principal point; the external parameter matrix contains the rotation matrix and translation vector, which is determined by the position of the camera coordinate system relative to the world coordinate system.

[0187] It can be understood that in an actual triangulation system, to achieve high-precision measurement, it is also necessary to consider the thickness and distortion of the camera lens, that is, the influence of tangential and radial distortion.

[0188] It can be understood that the optical path is reversible, and accordingly, the projector a can be regarded as a reverse camera. The imaging system of the projector is usually described using the imaging system model of a camera.

[0189] It can be understood that projector a and the camera constitute a triangulation system, and the purpose of completing the triangulation system calibration is to provide system parameters for projector a and the camera, and to treat the calibration surface in the vertical measurement system as the object to be measured, which can provide the X, Y, and Z point cloud coordinate values ​​in the world coordinate system. It can also be understood that the calibration range of the triangulation system is slightly larger than that of the vertical measurement system.

[0190] The maximum modulation index corresponds to the serial number module:

[0191] In some embodiments, the projector b and the camera constitute a vertical measurement system, and calculating the maximum modulation distribution of each pixel point on any calibration surface and obtaining the serial number corresponding to the maximum modulation value includes the following steps:

[0192] S021. Zoom the projector b along the optical axis at equal intervals, and project N frames of fringe patterns with a fixed phase difference onto the calibration surface at each moving position. The light field distribution collected by the camera on the focal plane is expressed as:

[0193]

[0194] The light field collected by the camera before and after the focal plane is expressed as:

[0195]

[0196] Where, and represents the light field distribution on the focal plane and the image plane at a distance H from the focal plane, respectively; j represents the number of times the projector b zooms along the optical axis at equal intervals, and j = 1, 2, 3…J, J represents the total number of moves; N represents the N-step phase shift, and N ≥ 3; n and n' distributions represent the nth fringe pattern projected on the focal plane or the defocused plane at position j, n (or n') = 0, 1, 2…, N-1; σ H represents the standard deviation of the point spread function; u represents the number of columns in the image pixel coordinates, in pixels, and v represents the number of rows in the image pixel coordinates, in pixels; R2(u,v), B2(u,v) and C2(u,v) represent the surface reflectivity, ambient light intensity and fringe contrast, respectively; M2 represents the measurement system magnification; f0 represents the fringe frequency, and Φ(u,v) represents the initial phase of the fringe;

[0197] S022, extract the fringes on the focal plane as follows

[0198]

[0199] The fringes on the out-of-focus plane are as follows

[0200]

[0201] The modulation distribution of the fringes on the above focal plane and out-of-focus plane is as follows:

[0202]

[0203]

[0204] Where M f (u,v) and M' f (u,v;σ H) represent the modulation distribution on the focal plane and the defocus plane respectively; I(u,v) and I(u,v; σ H ) represent the light field distribution on the focal plane and the image plane at a distance H from the focal plane; σ H represents the standard deviation of the point spread function; u represents the number of columns in the image pixel coordinates, in pixels; v represents the number of rows in the image pixel coordinates, in pixels;

[0205] S023. Perform quadratic fitting interpolation on the curve formed by the modulation values ​​of the pixels with the same name at each scanning position, extract the maximum modulation value of the pixels with the same name, and obtain the serial number j(t,u,v) corresponding to the maximum modulation value of the pixels with the same name (u,v). max Where t represents the t-th calibration surface, u and v represent the number of rows and columns in the image pixel coordinates, and the serial number j(t,u,v) max A floating point number.

[0206] In said S021:

[0207] Furthermore, the projector b is fixed on the optical platform, and the current value of the electronically adjustable focus liquid lens is changed at equal intervals, so that the projector b zooms the projected light field to different calibration surfaces.

[0208] In the spatial coordinate value module of each pixel point:

[0209] In some embodiments, obtaining an arbitrary calibration surface in a vertical measurement system and calculating the spatial coordinate values ​​of each pixel point of the calibration surface in a triangulation system includes the following steps:

[0210] The spatial coordinate values ​​of each pixel point of the calibration surface in the triangulation system are calculated according to the Fourier transform method, wavelet transform method, S transform method or phase shift algorithm.

[0211] Furthermore, calculating the spatial coordinate values ​​of each pixel point of the calibration surface in the triangulation system according to the phase shift algorithm includes the following steps:

[0212] S031. Projector a projects vertical stripes and horizontal stripes on the focal plane, and the camera collects them synchronously. The n-th frame of the vertical stripes and horizontal stripes collected are mathematically expressed as

[0213]

[0214] Where, and represents the light field distribution of the nth fringe pattern of vertical and horizontal stripes on the focal plane respectively; u represents the number of columns in the image pixel coordinates, in pixels, and v represents the number of rows in the image pixel coordinates, in pixels; N represents the N-step phase shift, and N ≥ 3; n represents the nth fringe pattern of the projection, n = 0, 1, 2…, N-1; R2(u,v), B2(u,v) and C2(u,v) are the reflectivity of the object surface, the ambient light intensity and the contrast of the fringe, respectively; M2 is the magnification of the measurement system; and Respectively expressed as vertical stripe frequency and horizontal stripe frequency; Φ V (u, v) and Φ H (u, v) represent the initial phases of vertical and horizontal stripes, respectively;

[0215] S032, calculate separately and

[0216]

[0217]

[0218]

[0219] Where, and They are expressed as the truncated phases of vertical and horizontal stripes, ranging from -π to π, with a discontinuity of 2π; and They represent the light field distribution of the n-th fringe pattern of vertical and horizontal stripes on the focal plane respectively; N represents the N-step phase shift, and N ≥ 3; n represents the n-th fringe pattern of the projection, n = 0, 1, 2…, N-1;

[0220] S033, respectively and To add or subtract multiples of 2π, the formula is as follows:

[0221]

[0222]

[0223] Where, Φ V (u, v) and Φ H (u,v) respectively represent and Continuous phase obtained by adding or subtracting multiples of 2π; and They are respectively represented as the truncated phases of vertical and horizontal stripes; k V (u,v) and k H (u, v) represent the order of vertical stripes and horizontal stripes respectively;

[0224] Thus obtaining

[0225] Where, Φ V (u, v) and Φ H (u, v) represent the absolute phases of vertical and horizontal stripes respectively; T V and T H They are respectively expressed as the period of vertical stripe frequency and horizontal stripe frequency; u represents the number of columns in the image pixel coordinates, in pixels; v represents the number of rows in the image pixel coordinates, in pixels;

[0226] Will Substitution You can get the X in the world coordinate system W (u,v),Y W (u,v), Z W (u,v) value, traverse each point on the calibration surface to determine the spatial coordinate value X of each pixel point on the calibration surface W (t,u,v),Y W (t,u,v),Z W (t,u,v).

[0227] It can be understood that the phase shift algorithm is an N-step (N≥3) phase shift algorithm with equal phase shifts.

[0228] Understandably, and Add or subtract multiples of 2π to eliminate the discontinuity of 2π.

[0229] In the mapping relationship module:

[0230] The spatial coordinate values ​​of each pixel point on the calibration surface in the triangulation system are respectively interpolated with the serial number corresponding to the maximum modulation index to obtain a mapping relationship between the spatial coordinate values ​​of each pixel point on the calibration surface and the serial number corresponding to the maximum modulation index, including the following steps:

[0231] Within the measurement range determined by the farthest calibration surface and the nearest calibration surface of the vertical system, complete the scanning of each calibration surface and obtain the spatial coordinate value X of each pixel point on the calibration surface. W (t,u,v),Y W (t,u,v),Z W (t,u,v), and the spatial coordinate value X of each pixel point on the calibration surface W (t,u,v),Y W (t,u,v),Z W (t,u,v) and the maximum modulation index corresponding to the serial number are interpolated by quadratic fitting to establish the spatial coordinate value X W (t,u,v),Y W(t,u,v),Z W (t,u,v) corresponds to the maximum value of the modulation index, the sequence number j(t,u,v) max The mapping relationship between them:

[0232] X W (t,u,v)=a(u,v)+b(u,v)j(t,u,v) max +c(u,v)j 2 (t,u,v) max

[0233] Y W (t,u,v)=a(u,v)+b(u,v)j(t,u,v) max +c(u,v)j 2 (t,u,v) max

[0234] Z W (t,u,v)=a(u,v)+b(u,v)j(t,u,v) max +c(u,v)j 2 (t,u,v) max

[0235] Where t represents the tth calibration surface, u and v represent the number of rows and columns in the image pixel coordinates, respectively, and X W (t,u,v),Y W (t,u,v),Z W (t,u,v) represent the coordinate values ​​of the t-th calibration surface in the row and column of the image in the directions of the three coordinate axes X, Y, and Z, respectively, j(t,u,v) max It represents the maximum modulation index of the t-th calibration surface, a(u,v), b(u,v), and c(u,v) are the fitting coefficients of the depth value-maximum modulation index curve.

[0236] In the module for reconstructing the three-dimensional surface shape of the object being measured:

[0237] In some embodiments, projector b zooms and scans the object to be measured, calculates the serial number value corresponding to the maximum modulation value of each pixel point, and reconstructs the three-dimensional surface shape of the object to be measured based on the mapping relationship between the spatial coordinate value of each pixel point on the calibration surface and the serial number corresponding to the maximum modulation value, specifically including:

[0238] Place the object to be measured within the calibration system and keep the position unchanged. Change the current value of the electronically adjustable focus liquid lens at equal intervals so that projector b zooms and projects the light field onto the object to be measured. At each current value of the electronically adjustable focus liquid lens, N frames of fringe patterns with a fixed phase difference are projected onto the object to be measured. The camera synchronously collects the fringe pattern and obtains the serial number j(t,u,v) corresponding to the maximum modulation index.max , according to X W (t,u,v)=a(u,v)+b(u,v)j(t,u,v) max +c(u,v)j 2 (t,u,v) max Y W (t,u,v)=a(u,v)+b(u,v)j(t,u,v) max +c(u,v)j 2 (t,u,v) max , obtain the pixel coordinate Z of the object being measured (u,v) W (t,u,v)=a(u,v)+b(u,v)j(t,u,v) max +c(u,v)j 2 (t,u,v) max

[0239] Value X W (t,u,v),Y W (t,u,v),Z W (t,u,v), traverse all pixel points to obtain the height distribution of the object being measured, that is, the reconstruction of the three-dimensional surface shape of the object being measured is completed.

[0240] In a third aspect, the present application provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the modulation profilometry method as described above.

[0241] The computer device may be a desktop computer, a notebook computer, a PDA, a cloud server, etc. The computer device may interact with the user via a keyboard, a mouse, a remote control, a touchpad, or a voice control device.

[0242] The memory includes at least one type of readable storage medium, including flash memory, a hard disk, a multimedia card, card-type memory (e.g., SD or D-interface display memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic storage, a magnetic disk, an optical disk, etc. In some embodiments, the memory may be an internal storage unit of the computer device, such as the computer device's hard disk or internal memory. In other embodiments, the memory may also be an external storage device of the computer device, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a flash memory card, etc. Of course, the memory may also include both the internal storage unit of the computer device and its external storage devices. In this embodiment, the memory is typically used to store the operating system and various application software installed on the computer device, such as the program code of the modulation profilometry method. In addition, the memory may also be used to temporarily store various types of data that have been output or are about to be output.

[0243] In some embodiments, the processor may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip. The processor is typically used to control the overall operation of the computer device. In this embodiment, the processor is used to execute program code stored in the memory or process data, such as executing the program code for the modulation profilometry method.

[0244] In a fourth aspect, the present application provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the processor executes the steps of the modulation profilometry method as described above.

[0245] The computer-readable storage medium stores an interface display program, and the interface display program can be executed by at least one processor to enable the at least one processor to perform the steps of the modulation profilometry method as described above.

[0246] Application Examples

[0247] This calibration ranges from 0mm to 50mm, with 11 calibration surfaces, and the distance between any two adjacent calibration surfaces is 5mm. The three-dimensional object being measured, a plaster head of Asma, was placed within the calibration range. During the scan, 300 different zoom positions were acquired, with current values ​​ranging from -15A to 3A, and with a step size of 0.06A. Figure 3 It is the light field distribution projected onto the object being measured by the projector when the electronic focus lens changes the current value for the 151st time. Figure 4 It is the reconstruction result after the calibration is completed by using the triangulation system to assist the vertical measurement system. It can be seen that the present invention can give the physical actual size of the reconstructed three-dimensional surface.

[0248] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method of modulation profilometry, characterized in that include: Projector a and the camera form a triangulation system, and the triangulation system calibration is completed; Projector b and the camera form a vertical measurement system, which calculates the maximum modulation distribution of each pixel point on any calibration surface and obtains the serial number corresponding to the maximum modulation value; Obtain any calibration surface in the vertical measurement system and calculate the spatial coordinate values ​​of each pixel point on the calibration surface in the triangulation system; The spatial coordinate values ​​of each pixel point on the calibration surface in the triangulation system are respectively interpolated with the serial number corresponding to the maximum modulation index to obtain the mapping relationship between the spatial coordinate values ​​of each pixel point on the calibration surface and the serial number corresponding to the maximum modulation index; Reconstruct the three-dimensional shape of the object being measured.

2. The method of modulation profilometry according to claim 1, characterized in that: Projector a and the camera form a triangulation system, and the triangulation system calibration is completed, including the following steps: Establish the relationship between 2D image pixel coordinates and 2D image physical coordinates: Where u represents the number of columns in the image pixel coordinates, in pixels; v represents the number of rows in the image pixel coordinates, in pixels; u0 represents the position of the origin of the image physical coordinate column in the image pixel coordinates, and v0 represents the position of the origin of the image physical coordinate row in the image pixel coordinates; x represents the number of columns in the image physical coordinates, in millimeters; y represents the number of rows in the image physical coordinates, in millimeters; Establish the mapping relationship between the physical coordinates of the 2D image and the camera's 3D coordinate system points: Where x represents the number of columns in the physical coordinates of the image, in millimeters; y represents the number of rows in the physical coordinates of the image, in millimeters; f represents the focal length of the camera; X c , Y c , Z c are the three mutually orthogonal axes of the camera coordinate system; Convert to the following homogeneous coordinate expression: Where x represents the number of columns in the physical coordinates of the image, in millimeters; y represents the number of rows in the physical coordinates of the image, in millimeters; f represents the focal length of the camera; X c , Y c , Z c are the three mutually orthogonal axes of the camera coordinate system; Establish the transformation relationship between the camera coordinate system and the world coordinate system: Where u represents the number of columns in the image pixel coordinates, in pixels; v represents the number of rows in the image pixel coordinates, in pixels; f represents the focal length of the camera; f x =f / d x , f y =f / d y , d x and d y Respectively represent the physical size of the camera unit pixel in the horizontal and vertical directions; u0 and v0 represent the coordinates of the camera plane at the optical center; R1 represents a 3×3 orthogonal rotation matrix, T1=[T x ,T y ,T z ] is the three-dimensional translation vector; X W , Y W , Z W Represents three mutually orthogonal axes in the world coordinate system; M1 represents the intrinsic parameter matrix; K1 is the external parameter matrix; at this point, the triangulation system calibration is completed under ideal conditions; In a practical triangulation system, the effects of tangential and radial distortion are: At this point, the correction of the actual triangulation system calibration is completed; Where x represents the horizontal direction (number of columns) in the physical coordinates of the image, in millimeters; y represents the vertical direction (number of rows) in the physical coordinates of the image, in millimeters; x' and y' represent the coordinate values ​​in the two directions after considering the distortion, respectively; parameters p1 and p2 represent the tangential distortion coefficients; and ρ represents the distance from the coordinate point (x, y) to the origin.

3. The method of modulation profilometry according to claim 1, characterized in that: Projector b and the camera form a vertical measurement system. Calculating the maximum modulation distribution of each pixel point on any calibration surface and obtaining the serial number corresponding to the maximum modulation value includes the following steps: Zoom the projector b along the optical axis at equal intervals, and project N frames of fringe patterns with a fixed phase difference onto the calibration surface at each zoom position. The light field distribution collected by the camera on the focal plane is expressed as: The light field collected by the camera before and after the focal plane is expressed as: Where, and represents the light field distribution on the focal plane and the image plane at a distance H from the focal plane, respectively; j represents the number of equally spaced zooms of projector b, and j = 1, 2, 3…J, J represents the total number of moves; N represents the N-step phase shift, and N ≥ 3; n and n' distributions represent the nth fringe pattern projected on the focal plane or the defocused plane at position j, n (or n') = 0, 1, 2…, N-1; σ H represents the standard deviation of the point spread function; u represents the number of columns in the image pixel coordinates, in pixels, and v represents the number of rows in the image pixel coordinates, in pixels; R2(u,v), B2(u,v) and C2(u,v) represent the surface reflectivity, ambient light intensity and fringe contrast, respectively; M2 represents the measurement system magnification; f0 represents the fringe frequency, and Φ(u,v) represents the initial phase of the fringe; Extract the fringes on the focal plane as follows The fringes on the out-of-focus plane are as follows The modulation distribution of the fringes on the above focal plane and out-of-focus plane is as follows: Where M f (u,v) and M' f (u,v;σ H ) represent the modulation distribution on the focal plane and the defocus plane respectively; I(u,v) and I(u,v; σ H ) represent the light field distribution on the focal plane and the image plane at a distance H from the focal plane; σ H represents the standard deviation of the point spread function; u represents the number of columns in the image pixel coordinates, in pixels; v represents the number of rows in the image pixel coordinates, in pixels; Perform quadratic fitting interpolation on the curve formed by the modulation values ​​of the pixels with the same name at each scanning position, extract the maximum modulation value of the pixels with the same name, and you can get the serial number j(t,u,v) corresponding to the maximum modulation value of the pixels with the same name (u,v) max , where t represents the tth calibration surface, u and v represent the number of rows and columns in the image pixel coordinates, respectively, and the serial number j(t,u,v) max A floating point number.

4. The method of modulation profilometry according to claim 1, wherein: Obtaining any calibration surface in the vertical measurement system and calculating the spatial coordinate values ​​of each pixel point on the calibration surface in the triangulation system includes the following steps: The XYZ coordinate values ​​of each pixel point of the calibration surface in the triangulation system are calculated according to the Fourier transform method, wavelet transform method, S transform method or phase shift algorithm.

5. The method of modulation profilometry according to claim 4, characterized in that: Calculating the XYZ coordinates of each pixel point on the calibration surface in the triangulation system using the phase shift algorithm includes the following steps: Projector a projects vertical stripes and horizontal stripes on the focal plane, and the camera collects them synchronously. The n-th frame of the vertical stripes and horizontal stripes collected are mathematically expressed as Where, and represents the light field distribution of the nth fringe pattern of vertical and horizontal stripes on the focal plane respectively; u represents the number of columns in the image pixel coordinates, in pixels, and v represents the number of rows in the image pixel coordinates, in pixels; N represents the N-step phase shift, and N ≥ 3; n represents the nth fringe pattern of the projection, n = 0, 1, 2…, N-1; R2(u,v), B2(u,v) and C2(u,v) are the reflectivity of the object surface, the ambient light intensity and the contrast of the fringe, respectively; M2 is the magnification of the measurement system; and Respectively expressed as vertical stripe frequency and horizontal stripe frequency; Φ V (u,v) and Φ H (u, v) represent the initial phases of vertical and horizontal stripes, respectively; Calculate separately and The respective phase values ​​are as follows: Where, and They are expressed as the truncated phases of vertical and horizontal stripes, ranging from -π to π, with a discontinuity of 2π; and They represent the light field distribution of the n-th fringe pattern of vertical and horizontal stripes on the focal plane respectively; N represents the N-step phase shift, and N ≥ 3; n represents the n-th fringe pattern of the projection, n = 0, 1, 2…, N-1; Respectively and To add or subtract multiples of 2π, the formula is as follows: Where, Φ V (u,v) and Φ H (u,v) respectively represent and Continuous phase obtained by adding or subtracting multiples of 2π; and They are respectively represented as the truncated phases of vertical and horizontal stripes; k V (u,v) and k H (u, v) represent the order of vertical stripes and horizontal stripes respectively; Thus obtaining Where, Φ V (u,v) and Φ H (u, v) represent the absolute phases of vertical and horizontal stripes respectively; T V and T H They are respectively expressed as the period of vertical stripe frequency and horizontal stripe frequency; u represents the number of columns in the image pixel coordinates, in pixels; v represents the number of rows in the image pixel coordinates, in pixels; Will Substitution You can get X in the world coordinate system W (u,v),Y W (u,v), Z W (u,v) value, traverse each point on the calibration surface to determine the spatial coordinate value X of each pixel point on the calibration surface W (t,u,v),Y W (t,u,v),Z W (t,u,v).

6. The method of modulation profilometry according to claim 1, wherein: The spatial coordinate values ​​of each pixel point on the calibration surface in the triangulation system are respectively interpolated with the serial number corresponding to the maximum modulation index to obtain a mapping relationship between the spatial coordinate values ​​of each pixel point on the calibration surface and the serial number corresponding to the maximum modulation index, including the following steps: Within the measurement range determined by the farthest calibration surface and the nearest calibration surface of the vertical system, complete the scanning of each calibration surface and obtain the spatial coordinate value X of each pixel point on the calibration surface. W (t,u,v),Y W (t,u,v),Z W (t,u,v), and the spatial coordinate value X of each pixel point on the calibration surface W (t,u,v),Y W (t,u,v),Z W (t,u,v) and the maximum modulation index corresponding to the serial number are interpolated by quadratic fitting to establish the spatial coordinate value X W (t,u,v),Y W (t,u,v),Z W (t,u,v) corresponds to the maximum value of the modulation index, the sequence number j(t,u,v) max The mapping relationship between them: X W (t,u,v)=a(u,v)+b(u,v)j(t,u,v) max +c(u,v)j 2 (t,u,v) max Y W (t,u,v)=a(u,v)+b(u,v)j(t,u,v) max +c(u,v)j 2 (t,u,v) max Z W (t,u,v)=a(u,v)+b(u,v)j(t,u,v) max +c(u,v)j 2 (t,u,v) max Where t represents the tth calibration surface, u and v represent the number of rows and columns in the image pixel coordinates, respectively, and X W (t,u,v),Y W (t,u,v),Z W (t,u,v) represent the coordinate values ​​of the t-th calibration surface in the row and column of the image in the directions of the three coordinate axes X, Y, and Z, respectively, j(t,u,v) max It represents the maximum modulation index of the t-th calibration surface, a(u,v), b(u,v), and c(u,v) are the fitting coefficients of the depth value-maximum modulation index curve.

7. The method of modulation profilometry according to claim 1, wherein: Projector b zooms and scans the object to be measured, calculates the serial number value corresponding to the maximum modulation index of each pixel point, and reconstructs the three-dimensional surface shape of the object to be measured based on the mapping relationship between the spatial coordinate value of each pixel point on the calibration surface and the serial number corresponding to the maximum modulation index. Specifically, it includes: Place the object to be measured within the calibration system and keep the position unchanged. Change the current value of the electronically adjustable focus liquid lens at equal intervals so that projector b zooms and projects the light field onto the object to be measured. At each current value of the electronically adjustable focus liquid lens, N frames of fringe patterns with a fixed phase difference are projected onto the object to be measured. The camera synchronously collects the fringe pattern and obtains the serial number j(t,u,v) corresponding to the maximum modulation index. max , according to X W (t,u,v)=a(u,v)+b(u,v)j(t,u,v) max +c(u,v)j 2 (t,u,v) max Y W (t,u,v)=a(u,v)+b(u,v)j(t,u,v) max +c(u,v)j 2 (t,u,v) max , obtain the pixel coordinates of the object being measured (u,v) Z W (t,u,v)=a(u,v)+b(u,v)j(t,u,v) max +c(u,v)j 2 (t,u,v) max Value X W (t,u,v),Y W (t,u,v),Z W (t,u,v), traverse all pixel points to obtain the height distribution of the object being measured, that is, the reconstruction of the three-dimensional surface shape of the object being measured is completed.

8. A system for modulation profilometry, characterized in that include: A triangulation system calibration module is used to form a triangulation system with the projector a and the camera and complete the triangulation system calibration; The maximum modulation index corresponding serial number module is used to form a vertical measurement system with projector b and camera, calculate the maximum modulation index distribution of each pixel point on any calibration surface, and obtain the serial number corresponding to the maximum modulation index; The spatial coordinate value module of each pixel point is used to obtain any calibration surface in the vertical measurement system and calculate the spatial coordinate value of each pixel point of the calibration surface in the triangulation system; A mapping relationship module is used to perform quadratic fitting interpolation on the spatial coordinate values ​​of each pixel point on the calibration surface in the triangulation system and the serial number corresponding to the maximum modulation index, thereby obtaining a mapping relationship between the spatial coordinate values ​​of each pixel point on the calibration surface and the serial number corresponding to the maximum modulation index; The module for reconstructing the three-dimensional surface shape of the object under test is used to reconstruct the three-dimensional surface shape of the object under test.

9. Computer device, characterized in that The method comprises a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that A computer program is stored, and when the computer program is executed by a processor, the processor is caused to perform the steps of the method according to any one of claims 1 to 7.

Images