Optical perpendicularity measurement method, system and equipment based on binocular stereoscopic vision and storage medium

Abstract

The invention discloses an optical verticality measurement method, system and device based on binocular stereoscopic vision, and a storage medium. The method comprises the following steps: completing epipolar correction of a binocular stereoscopic vision system; obtaining a serial number corresponding to the maximum value of the modulation degree; calculating space coordinate values of all pixel points of each calibration surface in the binocular stereoscopic vision system; establishing a mapping relation; the projector scans the measured object in a zooming mode, the serial number value corresponding to the maximum value of the modulation degree of each pixel point is calculated, and the three-dimensional surface shape of the measured object is reconstructed according to the mapping relation. According to the method, the limitation that only the size of the measured object in the Z-axis (height) direction can be measured in a traditional vertical measurement technology is broken through, complete three-dimensional geometric information (physical sizes in the X, Y and Z directions) can be reconstructed, and wide and profound application values are brought to the fields of intelligent manufacturing, quality detection and the like.

Description

Technical Field

[0001] The present invention belongs to the technical field of structured light projection three-dimensional surface measurement, and in particular relates to an optical vertical measurement method, system, equipment and storage medium based on binocular stereo vision. Background Art

[0002] With the continued development of information technology, optical electronics, and related high-tech fields, three-dimensional surface structure measurement has gradually gained attention across various industries and is widely used in precision manufacturing, material performance analysis, product quality testing, and medical applications. Three-dimensional surface structure measurement can be divided into two categories: contact and non-contact methods.

[0003] Common contact measurement methods include three-dimensional coordinate measurement systems (CMMs) and scanning probe microscopy (SPM) technologies. CMMs establish a spatial reference system by setting up three mutually perpendicular motion axes and use a probe to accurately measure the target's position in three-dimensional space. SPMs rely on an extremely fine probe sliding across the sample surface, recording information about the interaction between the probe and the surface to obtain its microstructure, with resolution down to the atomic scale. Although these methods offer superior accuracy, the testing process is relatively slow and carries the risk of abrasion to the sample surface.

[0004] With the continuous advancement of optoelectronics technology, non-contact measurement techniques have effectively overcome the potential for sample damage associated with traditional contact measurement. These methods have been widely used in various fields due to their rapidity, non-destructive nature, and ease of use. Typical techniques include confocal laser microscopy (CLSM), interferometry, laser triangulation (LTM), and structured light measurement. CLSM is a high-spatial-resolution optical imaging technique that captures information from only a single focal point at each moment, achieving two-dimensional or three-dimensional imaging of the sample through point-by-point scanning. Interferometry techniques are subdivided into single-wavelength interferometry, dual-wavelength interferometry, and white-light interferometry. Single-wavelength interferometry is prone to phase uncertainty when dealing with large height variations; dual-wavelength interferometry has a narrow measurement range; and while white-light interferometry has achieved breakthroughs in increasing the measurement range, it places high demands on environmental stability and the reflectivity of the sample surface. Laser triangulation, based on the relationship between light reflection and geometry, acquires three-dimensional contour data of an object. However, practical applications of this technique face challenges such as complex algorithms.

[0005] Structured light measurement methods can be divided into two categories based on their measurement principles: one based on triangular geometry and the other based on vertical modulation. The former, which includes phase-shift profilometry, Fourier transform profilometry, wavelet transform profilometry, moiré fringe technology, and S-transform profilometry, encodes the object's height information into the phase of the fringe pattern and uses phase decoding to restore its three-dimensional shape. However, these profilometry techniques have certain limitations when dealing with areas with dramatic height changes, such as deep holes and trenches.

[0006] To address this issue, a vertical measurement method based on the modulation principle has been proposed. This method encodes depth information in the fringe modulation, allowing 3D contour reconstruction through modulation analysis. However, existing vertical measurement methods based on the modulation principle can only provide the depth dimension of the measured object in the Z direction. Summary of the Invention

[0007] The purpose of the present invention is to provide an optical vertical measurement method, system, device and storage medium based on binocular stereo vision to solve the problem that the existing technology can only provide the actual physical size of the measured object in the Z direction.

[0008] The embodiment of the present application is implemented as follows: providing an optical vertical measurement method based on binocular stereo vision, comprising: cameras a and b forming a binocular stereo vision system, and performing epipolar correction of the binocular stereo vision system;

[0009] The projector and camera a form a vertical measurement system, calculate the maximum modulation distribution of each pixel point on each calibration surface, and obtain the serial number corresponding to the maximum modulation value;

[0010] Use a projector to project a fringe pattern to assist the binocular stereo vision system in completing stereo matching and calculate the spatial coordinate values ​​of all pixel points on each calibration surface in the binocular stereo vision system;

[0011] Establish a mapping relationship between the serial number corresponding to the maximum modulation value of all pixel points on each calibration surface in the vertical measurement system and the spatial coordinate values ​​of all pixel points on each calibration surface in the binocular stereo vision system;

[0012] The projector zooms and scans the object to be measured, calculates the serial number value corresponding to the maximum modulation index of each pixel point, and reconstructs the three-dimensional surface shape of the object to be measured based on the above mapping relationship.

[0013] In some embodiments, camera a and camera b form a binocular stereo vision system, and the conversion of system pixel coordinates to world coordinates includes the following steps:

[0014] Establish the relationship between 2D image pixel coordinates and 2D image physical coordinates:

[0015]

[0016] Where u represents the number of columns in the image pixel coordinates, in pixels; v represents the number of rows in the image pixel coordinates, in pixels; u0 represents the position of the origin of the image physical coordinate column in the image pixel coordinates, and v0 represents the position of the origin of the image physical coordinate row in the image pixel coordinates; x represents the number of columns in the image physical coordinates, in millimeters; y represents the number of rows in the image physical coordinates, in millimeters;

[0017] Establish the mapping relationship between the physical coordinates of the 2D image and the camera's 3D coordinate system points:

[0018]

[0019] Where x represents the number of columns in the physical coordinates of the image, in millimeters; y represents the number of rows in the physical coordinates of the image, in millimeters; f represents the focal length of the camera; X c , Y c , Z c are the three mutually orthogonal axes of the camera coordinate system;

[0020] Will Convert to the following homogeneous coordinate expression:

[0021]

[0022] Where x represents the number of columns in the physical coordinates of the image, in millimeters; y represents the number of rows in the physical coordinates of the image, in millimeters; f represents the focal length of the camera; X c , Y c , Z c are the three mutually orthogonal axes of the camera coordinate system;

[0023] Establish the transformation relationship between the camera coordinate system and the world coordinate system:

[0024]

[0025] Where u represents the number of columns in the image pixel coordinates, in pixels; v represents the number of rows in the image pixel coordinates, in pixels; f represents the focal length of the camera; f x =f / d x , f y =f / d y , d x and d y Respectively represent the physical size of the camera unit pixel in the horizontal and vertical directions; u0 and v0 represent the coordinates of the camera plane at the optical center; R1 represents a 3×3 orthogonal rotation matrix, T1=[T x ,T y ,T z ] is the three-dimensional translation vector; X W , YW , Z W Represents three mutually orthogonal axes in the world coordinate system; M1 represents the intrinsic parameter matrix; K1 is the external parameter matrix.

[0026] In some embodiments, epipolar calibration of a binocular stereo vision system includes the following steps:

[0027] Based on the Bouguet algorithm, assuming that the right camera b is used as the reference, the binocular camera external parameter rotation matrix R and translation vector T are obtained. The left camera a coordinate system is rotated half along the positive direction of the rotation vector corresponding to the rotation matrix R, and converted into a matrix form recorded as R l , the right camera b coordinate system rotates half in the opposite direction of the rotation vector corresponding to the rotation matrix R, and is converted into a matrix form and recorded as R r ,At this time, the coordinate systems of camera a and camera b are parallel, but the baseline is not parallel to the imaging plane;

[0028] Construct a transformation matrix to make the baseline parallel to the imaging plane. The construction formula is as follows:

[0029]

[0030] Where t is the translation vector T through R r The rotated vector, t = R r *T, e1 represents the normalized vector t, e2 represents the projection perpendicular to t on the xy plane, e3, e1 and e2 form an orthogonal right-handed coordinate system, t x and t y Both represent the components of vector t in the x and y directions respectively;

[0031] By obtaining e1, e2, and e3, the corrected rotation matrix can be obtained as follows:

[0032]

[0033] Where R rect represents the transformation matrix, e1 T 、e2 T 、e3 T Represent the transpose of e1, e2, and e3 respectively;

[0034] Combining the transformation matrix and the rotation matrix of the left and right cameras, the correction matrix of the binocular camera is calculated as:

[0035]

[0036] Where R lrect Represents the rotation matrix of the left camera after stereo correction, R rrect Represents the rotation matrix of the right camera after stereo correction, R rect Represents the corrected rotation matrix, Rl and R r Represent the rotation matrices of the left camera and the right camera respectively; at this point, the epipolar correction of the binocular stereo vision system is completed.

[0037] In some embodiments, the projector and camera a form a vertical measurement system, and calculating the maximum modulation distribution of each pixel point on any calibration surface and obtaining the serial number corresponding to the maximum modulation value includes the following steps:

[0038] The projector is zoomed along the optical axis at equal intervals, and N frames of fringe patterns with a fixed phase difference are projected onto the calibration surface at each moving position. The light field distribution collected by camera a on the focal plane is expressed as:

[0039]

[0040] The light field collected by camera a before and after the focal plane is expressed as:

[0041]

[0042] Where, and represents the light field distribution on the focal plane and the image plane at a distance H from the focal plane, respectively; j represents the number of times the projector zooms along the optical axis at equal intervals, and j = 1, 2, 3…J, J represents the total number of moves; N represents the N-step phase shift, and N ≥ 3; n and n' distribution represent the nth fringe pattern projected on the focal plane or the defocused plane at position j, n (or n') = 0, 1, 2…, N-1; σ H represents the standard deviation of the point spread function; u represents the number of columns in the image pixel coordinates, in pixels, and v represents the number of rows in the image pixel coordinates, in pixels; R2(u,v), B2(u,v) and C2(u,v) represent the surface reflectivity, ambient light intensity and fringe contrast, respectively; M2 represents the measurement system magnification; f0 represents the fringe frequency, and Φ(u,v) represents the initial phase of the fringe;

[0043] Extract the fringes on the focal plane as follows

[0044]

[0045] The fringes on the out-of-focus plane are as follows

[0046]

[0047] The modulation distribution of the fringes on the above focal plane and out-of-focus plane is as follows:

[0048]

[0049]

[0050] Where M f (u,v) and M' f (u,v;σ H ) represent the modulation index distribution on the focal plane and the defocus plane respectively; I(u,v) and I(u,v; σ H ) represent the light field distribution on the focal plane and the image plane at a distance H from the focal plane; σ H represents the standard deviation of the point spread function; u represents the number of columns in the image pixel coordinates, in pixels; v represents the number of rows in the image pixel coordinates, in pixels;

[0051] Perform quadratic interpolation fitting on the curve formed by the modulation values ​​of the same-named pixels at each scanning position, extract the maximum modulation value of the same-named pixels, and obtain the serial number corresponding to the maximum modulation value of the same-named pixels (u, v), where i represents the i-th calibration surface, u and v represent the number of rows and columns in the image pixel coordinates, respectively, and j represents the serial number, which is a floating-point number.

[0052] In some embodiments, a projector is used to project a fringe pattern to assist a binocular stereo vision system in completing stereo matching, and the spatial coordinate values ​​of all pixel points on each calibration surface in the binocular stereo vision system are calculated, including the following steps:

[0053] The fringe pattern is projected by a projector, and phase information is introduced to assist the binocular stereo vision system to more accurately match the same-name points and complete stereo matching. The introduction of phase can be achieved by projecting an N-step phase shift pattern. The light field of the N-step phase shift pattern can be expressed as:

[0054]

[0055] Where G n (u,v) represents the light field distribution, u and v represent the number of rows and columns in the image pixel coordinates, α(u,v) and β(u,v) represent the ambient light intensity and the contrast of the stripes, respectively, f0 is the stripe frequency, is the phase of the fringe, N is an integer, indicating N-step phase shift, n represents the nth fringe pattern, and n = 0, 1, 2…, N-1;

[0056] Image phase information extraction is calculated using the following formula:

[0057]

[0058] Where, is the truncated phase of the coordinate point (u, v), where u and v represent the row and column numbers in the image pixel coordinates, respectively. n Represents the light field distribution, N is an integer, representing N-step phase shift, n represents the nth fringe pattern, and n=0,1,2…,N-1;

[0059] The above formula cuts the phase information between [-π,π], and its absolute phase information is

[0060]

[0061] Where, Φ uw (u,v) is the absolute phase value of the coordinate point (u,v), is the truncated phase at the coordinate point (u, v), where u and v represent the row and column numbers in the image pixel coordinates, and k represents the fringe order;

[0062] The spatial coordinate value X of any point in space W (u,v),Y W (u,v), Z W The reconstruction of (u,v) is done by the following formula

[0063]

[0064]

[0065]

[0066] Among them, X W (u,v),Y W (u,v), Z W (u, v) represent the spatial coordinate values ​​of the pixel point (u, v), u L ,v L Respectively represent the number of rows and columns in the left camera, u R ,v R They represent the number of rows and columns in the right camera, d represents the baseline distance between the left and right cameras, and f represents the focal length of the camera.

[0067] In some embodiments, establishing a mapping relationship between the serial numbers corresponding to the maximum modulation values ​​of all pixels on each calibration surface in the vertical measurement system and the spatial coordinate values ​​of all pixels on each calibration surface in the binocular stereo vision system includes the following steps:

[0068] Within the measurement range of the vertical system, complete the scanning of each calibration surface and obtain the spatial coordinate value X of each pixel point on the calibration surface. W (i,u,v),Y W (i,u,v),Z W (i,u,v), and the spatial coordinate value X of each pixel point on the calibration surface W (i,u,v),Y W (i,u,v),Z W (i,u,v) and the maximum modulation index corresponding to the serial number are fitted by quadratic interpolation to establish the spatial coordinate value X W(i,u,v),Y W (i,u,v),Z W (i,u,v) corresponds to the sequence number j(i,u,v) with the maximum modulation index max The mapping relationship between them:

[0069] X W (i,u,v)=a(u,v)+b(u,v)j(i,u,v) max +c(u,v)j 2 (i,u,v) max

[0070] Y W (i,u,v)=a(u,v)+b(u,v)j(i,u,v) max +c(u,v)j 2 (i,u,v) max

[0071] Z W (i,u,v)=a(u,v)+b(u,v)j(i,u,v) max +c(u,v)j 2 (i,u,v) max

[0072] Where i represents the i-th calibration surface, u and v represent the number of rows and columns in the image pixel coordinates, respectively, and X W (i,u,v),Y W (i,u,v),Z W (i,u,v) represents the coordinate values ​​of the i-th calibration surface in the row and column of the image in the directions of the three coordinate axes X, Y, and Z, j(i,u,v) max represents the maximum modulation index of the i-th calibration surface, a(u,v), b(u,v), and c(u,v) are the fitting coefficients of the depth value-maximum modulation index curve.

[0073] In some embodiments, the projector zooms and scans the object to be measured, calculates the serial number value corresponding to the maximum modulation degree of each pixel point, and reconstructs the three-dimensional surface shape of the object to be measured based on the above mapping relationship, including the following steps:

[0074] Place the object to be measured within the calibration system and keep the position unchanged. Change the current value of the electronically adjustable focus liquid lens at equal intervals so that the projector zooms and projects the light field onto the object to be measured. At each current value of the electronically adjustable focus liquid lens, N frames of fringe patterns with a fixed phase difference are projected onto the object to be measured. The camera synchronously collects the fringe pattern and obtains the serial number j(i,u,v) corresponding to the maximum modulation index. max , according to X W(i,u,v)=a(u,v)+b(u,v)j(i,u,v) max +c(u,v)j 2 (i,u,v) max

[0075] Y W (i,u,v)=a(u,v)+b(u,v)j(i,u,v) max +c(u,v)j 2 (i,u,v) max , obtain the pixel coordinate Z of the object being measured (u,v) W (i,u,v)=a(u,v)+b(u,v)j(i,u,v) max +c(u,v)j 2 (i,u,v) max

[0076] Value X W (i,u,v),Y W (i,u,v),Z W (i,u,v), traverse all pixel points to obtain the height distribution of the object being measured, that is, the reconstruction of the three-dimensional surface shape of the object being measured is completed.

[0077] Accordingly, the embodiment of the present application further provides an optical vertical measurement system based on binocular stereo vision, including:

[0078] Binocular stereo vision system module, camera a and camera b form a binocular stereo vision system, and complete the epipolar line correction of the binocular stereo vision system;

[0079] The maximum modulation index corresponds to the serial number module. The projector and camera a form a vertical measurement system. The maximum modulation index distribution of each pixel point on each calibration surface is calculated to obtain the serial number corresponding to the maximum modulation index;

[0080] The spatial coordinate value module uses a projector to project a fringe pattern to assist the binocular stereo vision system in completing stereo matching and calculate the spatial coordinate values ​​of all pixel points on each calibration surface in the binocular stereo vision system;

[0081] A mapping relationship module establishes a mapping relationship between the serial number corresponding to the maximum modulation value of all pixel points on each calibration surface in the vertical measurement system and the spatial coordinate values ​​of all pixel points on each calibration surface in the binocular stereo vision system;

[0082] The module for reconstructing the three-dimensional surface shape of the object under test uses a projector to zoom and scan the object under test, calculates the serial number value corresponding to the maximum modulation index of each pixel point, and reconstructs the three-dimensional surface shape of the object under test based on the above mapping relationship.

[0083] Accordingly, an embodiment of the present application also provides a computer device, including a storage and a processor, wherein the storage stores a computer program, and when the computer program is executed by the processor, the processor performs the steps of the above method.

[0084] Accordingly, an embodiment of the present application further provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the processor executes the steps of the above method.

[0085] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are:

[0086] This application only needs to add an additional camera b to the traditional measurement system, and by projecting a set of phase-shifted vertical stripes, the spatial coordinate values ​​of the calibration surface can be reconstructed according to the binocular stereo vision composed of camera a and camera b, and then the vertical measurement system composed of the projector and camera a scans the calibration surface, and obtains the serial number values ​​corresponding to the maximum modulation values ​​of different pixel points on each calibration surface, and establishes a mapping relationship between the serial number value corresponding to the maximum modulation value and the spatial coordinate value. When measuring the object to be measured, it is only necessary to place the object to be measured in the calibration space range, scan the object to be measured using the vertical measurement system, and obtain the serial number value of the maximum modulation value of each pixel point, and then reconstruct the three-dimensional surface shape of the object to be measured according to the mapping relationship between the serial number value corresponding to the maximum modulation value and the spatial coordinate value. This application breaks through the limitation of traditional vertical measurement technology that can only measure the size of the object to be measured in the Z-axis (height) direction, and can reconstruct complete three-dimensional geometric information (physical dimensions in the X, Y, and Z directions), which will bring extensive and far-reaching application value to fields such as intelligent manufacturing and quality inspection, such as three-dimensional digital modeling and process optimization, online three-dimensional detection and feedback control, quality inspection and evaluation, etc. in the field of intelligent manufacturing. BRIEF DESCRIPTION OF THE DRAWINGS

[0087] Figure 1 A flow chart of an optical vertical measurement method based on binocular stereo vision provided by an embodiment of the present invention;

[0088] Figure 2 A schematic diagram of an optical vertical measurement method based on binocular stereo vision is provided for an embodiment of the present invention;

[0089] Figure 3 A schematic diagram of epipolar geometry provided in an embodiment of the present invention;

[0090] Figure 4 A schematic diagram of a fringe pattern projected onto a measured object during the 151st focal length change of a projector is provided for an embodiment of the present invention;

[0091] Figure 5 A schematic diagram of the reconstruction results of the optical vertical measurement method based on binocular stereo vision provided by an embodiment of the present invention. Figure 6 The optical vertical measurement method based on binocular stereo vision provided in the embodiment of the present invention reconstructs a three-dimensional surface image of a measured object.

[0092] Description of reference numerals:

[0093] 1-projector, 2-electronically adjustable focus liquid lens, 3-semi-transparent and semi-reflective mirror, 4-camera a, 5-camera b, 6-object to be measured. DETAILED DESCRIPTION

[0094] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0095] The technical solution of this application is as follows:

[0096] like Figure 1 As shown, in a first aspect, an embodiment of the present application provides an optical vertical measurement method based on binocular stereo vision, comprising:

[0097] S01, camera a and camera b form a binocular stereo vision system and complete the epipolar correction of the binocular stereo vision system;

[0098] S02, the projector and camera a form a vertical measurement system, calculate the maximum value distribution of modulation of each pixel point on each calibration surface, and obtain the serial number corresponding to the maximum value of modulation;

[0099] S03, using a projector to project a fringe pattern to assist the binocular stereo vision system in completing stereo matching, and calculating the spatial coordinate values ​​of all pixel points on each calibration surface in the binocular stereo vision system;

[0100] S04, establishing a mapping relationship between the serial numbers corresponding to the maximum modulation values ​​of all pixel points on each calibration surface in the vertical measurement system and the spatial coordinate values ​​of all pixel points on each calibration surface in the binocular stereo vision system;

[0101] S05. The projector zooms and scans the object to be measured, calculates the serial number value corresponding to the maximum value of the modulation degree of each pixel point, and reconstructs the three-dimensional surface shape of the object to be measured based on the above mapping relationship.

[0102] This application only needs to add an additional camera b to the traditional measurement system, and by projecting a set of phase-shifted vertical stripes, the spatial coordinate values ​​of the calibration surface can be reconstructed according to the binocular stereo vision composed of camera a and camera b, and then the vertical measurement system composed of the projector and camera a scans the calibration surface, and obtains the serial number values ​​corresponding to the maximum modulation values ​​of different pixel points on each calibration surface, and establishes a mapping relationship between the serial number value corresponding to the maximum modulation value and the spatial coordinate value. When measuring the object to be measured, it is only necessary to place the object to be measured in the calibration space range, scan the object to be measured using the vertical measurement system, and obtain the serial number value of the maximum modulation value of each pixel point, and then reconstruct the three-dimensional surface shape of the object to be measured according to the mapping relationship between the serial number value corresponding to the maximum modulation value and the spatial coordinate value. This application breaks through the limitation of traditional vertical measurement technology that can only measure the size of the object to be measured in the Z-axis (height) direction, and can reconstruct complete three-dimensional geometric information (physical dimensions in the X, Y, and Z directions), which will bring extensive and far-reaching application value to fields such as intelligent manufacturing and quality inspection, such as three-dimensional digital modeling and process optimization, online three-dimensional detection and feedback control, quality inspection and evaluation, etc. in the field of intelligent manufacturing.

[0103] In the S01:

[0104] It can be understood that the conversion from system pixel coordinates to world coordinates is completed first, and then the epipolar correction of the binocular stereo vision system is completed.

[0105] In some embodiments, camera a and camera b form a binocular stereo vision system, and the conversion of system pixel coordinates to world coordinates includes the following steps:

[0106] S011. Establish the relationship between the 2D image pixel coordinates and the 2D image physical coordinates:

[0107]

[0108] Where u represents the number of columns in the image pixel coordinates, in pixels; v represents the number of rows in the image pixel coordinates, in pixels; u0 represents the position of the origin of the image physical coordinate column in the image pixel coordinates, and v0 represents the position of the origin of the image physical coordinate row in the image pixel coordinates; x represents the number of columns in the image physical coordinates, in millimeters; y represents the number of rows in the image physical coordinates, in millimeters;

[0109] S012. Establish the mapping relationship between the physical coordinates of the 2D image and the camera's 3D coordinate system points:

[0110]

[0111] Where x represents the number of columns in the physical coordinates of the image, in millimeters; y represents the number of rows in the physical coordinates of the image, in millimeters; f represents the focal length of the camera; X c , Yc , Z c are the three mutually orthogonal axes of the camera coordinate system;

[0112] Will Convert to the following homogeneous coordinate expression:

[0113]

[0114] Where x represents the number of columns in the physical coordinates of the image, in millimeters; y represents the number of rows in the physical coordinates of the image, in millimeters; f represents the focal length of the camera; X c , Y c , Z c are the three mutually orthogonal axes of the camera coordinate system;

[0115] S013. Establish the conversion relationship between the camera coordinate system and the world coordinate system:

[0116]

[0117] Where u represents the number of columns in the image pixel coordinates, in pixels; v represents the number of rows in the image pixel coordinates, in pixels; f represents the focal length of the camera; f x =f / d x , f y =f / d y , d x and d y Respectively represent the physical size of the camera unit pixel in the horizontal and vertical directions; u0 and v0 represent the coordinates of the camera plane at the optical center; R1 represents a 3×3 orthogonal rotation matrix, T1=[T x ,T y ,T z ] is the three-dimensional translation vector; X W , Y W , Z W Represents three mutually orthogonal axes in the world coordinate system; M1 represents the intrinsic parameter matrix; K1 is the external parameter matrix.

[0118] like Figure 2As shown, it can be understood that a geometric constraint exists between the images captured by camera a and camera b (left and right cameras) in a binocular stereo vision system. This constraint is represented by epipolar lines, and epipolar geometry is the geometric relationship between the left and right images. Specifically, there is a point M in three-dimensional space, which is projected as image points m1 and m2 on the imaging planes of the left and right cameras, respectively. Due to the fixed relative position and orientation between the two cameras, these two image points are not randomly distributed but satisfy the epipolar constraint: given image point m1, its corresponding image point m2 must lie on a specific straight line in the right image, called the epipolar line. When performing stereo matching, directly matching homonymous points requires calculating the corresponding epipolar line for each search, and then searching along the entire epipolar line. This computation is relatively complex. However, before stereo matching, a certain image transformation is performed to ensure that the left and right imaging planes are coplanar and the corresponding epipolar lines are parallel and at the same height. This allows searching for homonymous points to be performed only on the same line, greatly reducing the difficulty of stereo matching.

[0119] In some embodiments, the epipolar correction algorithm comprises a Bouguet algorithm or a Fusiello algorithm.

[0120] It can be understood that the conversion coordinates are the spatial coordinate transformation relationship between the two cameras (i.e., external parameters), so that the points in one camera coordinate system can be accurately converted to another coordinate system. Epipolar correction is a geometric transformation operation performed on binocular images. Its purpose is to align the images so that the matching points of the same spatial point in the left and right images are on the same horizontal line. In this way, the epipolar lines that were originally distributed in any direction are "straightened" to the horizontal lines of the image, which significantly simplifies the image matching process. The main reason for epipolar correction is to improve the efficiency and accuracy of stereo matching. After correction, the search for corresponding points only needs to scan on the same line, which greatly reduces the amount of calculation; it also lays an accurate and unified foundation for subsequent disparity map calculation, depth estimation and three-dimensional reconstruction.

[0121] Furthermore, epipolar line correction of the binocular stereo vision system includes the following steps:

[0122] S014. Based on the Bouguet algorithm, assuming that the right camera b is used as the reference, the binocular camera external parameter rotation matrix R and translation vector T are obtained. The left camera a coordinate system is rotated half along the positive direction of the rotation vector corresponding to the rotation matrix R, and converted into a matrix form and recorded as R l , the right camera b coordinate system rotates half in the opposite direction of the rotation vector corresponding to the rotation matrix R, and is converted into a matrix form and recorded as R r ,At this time, the coordinate systems of camera a and camera b are parallel, but the baseline is not parallel to the imaging plane;

[0123] S015. Construct a transformation matrix to make the baseline parallel to the imaging plane. The construction formula is as follows:

[0124]

[0125] Where t is the translation vector T through R r The rotated vector, t = R r *T, e1 represents the normalized vector t, e2 represents the projection perpendicular to t on the xy plane, e3, e1 and e2 form an orthogonal right-handed coordinate system, t x and t y Both represent the components of vector t in the x and y directions respectively;

[0126] S016. Obtain e1, e2, and e3 to obtain the corrected rotation matrix as follows:

[0127]

[0128] Where R rect represents the transformation matrix, e1 T 、e2 T 、e3 T Represent the transpose of e1, e2, and e3 respectively;

[0129] S017. Combine the transformation matrix and the rotation matrix of the left and right cameras to calculate the correction matrix of the binocular camera:

[0130]

[0131] Where R lrect Represents the rotation matrix of the left camera after stereo correction, R rrect Represents the rotation matrix of the right camera after stereo correction, R rect Represents the corrected rotation matrix, R l and R r Represent the rotation matrices of the left camera and the right camera respectively; at this point, the epipolar correction of the binocular stereo vision system is completed.

[0132] It can be understood that during the correction process, the new pixel coordinates may not be integers, and they need to be interpolated using the pixel values ​​of the original image. In addition, the corrected left and right images need to be cropped and the center and boundary areas of the images need to be redefined to ensure that the repeated overlapping area of ​​the left and right images is increased.

[0133] In the S02:

[0134] In some embodiments, the projector and camera a form a vertical measurement system, and calculating the maximum modulation distribution of each pixel point on any calibration surface and obtaining the serial number corresponding to the maximum modulation value includes the following steps:

[0135] S021. Zoom the projector along the optical axis at equal intervals, and project N frames of fringe patterns with a fixed phase difference onto the calibration surface at each moving position. The light field distribution collected by camera a on the focal plane is expressed as:

[0136]

[0137] The light field collected by camera a before and after the focal plane is expressed as:

[0138]

[0139] Where, and represents the light field distribution on the focal plane and the image plane at a distance H from the focal plane, respectively; j represents the number of times the projector zooms along the optical axis at equal intervals, and j = 1, 2, 3…J, J represents the total number of moves; N represents the N-step phase shift, and N ≥ 3; n and n' distribution represent the nth fringe pattern projected on the focal plane or the defocused plane at position j, n (or n') = 0, 1, 2…, N-1; σ H represents the standard deviation of the point spread function; u represents the number of columns in the image pixel coordinates, in pixels, and v represents the number of rows in the image pixel coordinates, in pixels; R2(u,v), B2(u,v) and C2(u,v) represent the surface reflectivity, ambient light intensity and fringe contrast, respectively; M2 represents the measurement system magnification; f0 represents the fringe frequency, and Φ(u,v) represents the initial phase of the fringe;

[0140] S022, extract the fringes on the focal plane as follows

[0141]

[0142] The fringes on the out-of-focus plane are as follows

[0143]

[0144] The modulation distribution of the fringes on the above focal plane and out-of-focus plane is as follows:

[0145]

[0146]

[0147] Where M f (u,v) and M' f (u,v;σ H ) represent the modulation index distribution on the focal plane and the defocus plane respectively; I(u,v) and I(u,v; σ H ) represent the light field distribution on the focal plane and the image plane at a distance H from the focal plane; σ Hrepresents the standard deviation of the point spread function; u represents the number of columns in the image pixel coordinates, in pixels; v represents the number of rows in the image pixel coordinates, in pixels;

[0148] S023. Perform quadratic interpolation fitting on the curve formed by the modulation values ​​of the pixels with the same name at each scanning position, extract the maximum modulation value of the pixels with the same name, and obtain the serial number corresponding to the maximum modulation value of the pixels with the same name (u, v), where i represents the i-th calibration surface, u and v represent the number of rows and columns in the image pixel coordinates, respectively, and j represents the serial number, which is a floating point number.

[0149] In said S021:

[0150] Furthermore, the projector is fixed on the optical platform, and the current value of the electronically adjustable focus liquid lens is changed at equal intervals, so that the projector zooms the projected light field to different calibration surfaces.

[0151] In said S03:

[0152] In some embodiments, a projector is used to project a fringe pattern to assist a binocular stereo vision system in completing stereo matching, and the spatial coordinate values ​​of all pixel points on each calibration surface in the binocular stereo vision system are calculated, including the following steps:

[0153] S031. Use a projector to project a fringe pattern and introduce phase information to assist the binocular stereo vision system in more accurately matching the same-name points and completing stereo matching. The introduction of phase can be achieved by projecting an N-step phase shift pattern. The light field of the N-step phase shift pattern can be expressed as:

[0154]

[0155] Where G n (u,v) represents the light field distribution, u and v represent the number of rows and columns in the image pixel coordinates, α(u,v) and β(u,v) represent the ambient light intensity and the contrast of the stripes, respectively, f0 is the stripe frequency, is the phase of the fringe, N is an integer, indicating N-step phase shift, n represents the nth fringe pattern, and n = 0, 1, 2…, N-1;

[0156] Image phase information extraction is calculated using the following formula:

[0157]

[0158] Where, is the truncated phase of the coordinate point (u, v), where u and v represent the row and column numbers in the image pixel coordinates, respectively. n Represents the light field distribution, N is an integer, representing N-step phase shift, n represents the nth fringe pattern, and n=0,1,2…,N-1;

[0159] The above formula cuts the phase information between [-π,π], and its absolute phase information is

[0160]

[0161] Where, Φ uw (u,v) is the absolute phase value of the coordinate point (u,v), is the truncated phase at the coordinate point (u, v), where u and v represent the row and column numbers in the image pixel coordinates, and k represents the fringe order;

[0162] It can be understood from the above formula that after epipolar correction, according to the monotonically increasing property of the absolute phase value on the epipolar line, for a pixel point (u L ,v L ), according to the epipolar constraint, search for the pixel point with the closest phase value on the same row of the right image (u R ,v R ) as the same-name points, thereby quickly matching the same-name points on the two camera images to complete binocular stereo matching.

[0163] S032, spatial coordinate value X of any point in space W (u,v),Y W (u,v), Z W The reconstruction of (u,v) is done by the following formula

[0164]

[0165]

[0166]

[0167] Among them, X W (u,v),Y W (u,v), Z W (u, v) represent the spatial coordinate values ​​of the pixel point (u, v), u L ,v L Respectively represent the number of rows and columns in the left camera, u R ,v R They represent the number of rows and columns in the right camera, d represents the baseline distance between the left and right cameras, and f represents the focal length of the camera.

[0168] In said S04:

[0169] In some embodiments, establishing a mapping relationship between the serial numbers corresponding to the maximum modulation values ​​of all pixels on each calibration surface in the vertical measurement system and the spatial coordinate values ​​of all pixels on each calibration surface in the binocular stereo vision system includes the following steps:

[0170] Within the measurement range of the vertical system, complete the scanning of each calibration surface and obtain the spatial coordinate value X of each pixel point on the calibration surface. W (i,u,v),Y W (i,u,v),Z W (i,u,v), and the spatial coordinate value X of each pixel point on the calibration surface W (i,u,v),Y W (i,u,v),Z W (i,u,v) and the maximum modulation index corresponding to the serial number are fitted by quadratic interpolation to establish the spatial coordinate value X W (i,u,v),Y W (i,u,v),Z W (i,u,v) corresponds to the sequence number j(i,u,v) with the maximum modulation index max The mapping relationship between them:

[0171] X W (i,u,v)=a(u,v)+b(u,v)j(i,u,v) max +c(u,v)j 2 (i,u,v) max

[0172] Y W (i,u,v)=a(u,v)+b(u,v)j(i,u,v) max +c(u,v)j 2 (i,u,v) max

[0173] Z W (i,u,v)=a(u,v)+b(u,v)j(i,u,v) max +c(u,v)j 2 (i,u,v) max

[0174] Where i represents the i-th calibration surface, u and v represent the number of rows and columns in the image pixel coordinates, respectively, and X W (i,u,v),Y W (i,u,v),Z W (i,u,v) represents the coordinate values ​​of the i-th calibration surface in the row and column of the image in the directions of the three coordinate axes X, Y, and Z, j(i,u,v) max represents the maximum modulation index of the i-th calibration surface, a(u,v), b(u,v), and c(u,v) are the fitting coefficients of the depth value-maximum modulation index curve.

[0175] In said S05:

[0176] In some embodiments, a projector zooms and scans an object to be measured, calculates a serial number value corresponding to the maximum modulation index of each pixel point, and reconstructs a three-dimensional surface shape of the object to be measured based on the above mapping relationship, including the following steps: placing the object to be measured within the range of a calibration system and keeping the position unchanged, changing the current value of an electronically adjustable focus liquid lens at equal intervals so that the projector zooms and projects a light field onto the object to be measured, and at each current value of the electronically adjustable focus liquid lens, projects N frames of fringe patterns with a fixed phase difference onto the object to be measured, and synchronously captures the fringe patterns with a camera to obtain a serial number j(i, u, v) corresponding to the maximum modulation index. max , according to X W (i,u,v)=a(u,v)+b(u,v)j(i,u,v) max +c(u,v)j 2 (i,u,v) max

[0177] Y W (i,u,v)=a(u,v)+b(u,v)j(i,u,v) max +c(u,v)j 2 (i,u,v) max , obtain the pixel coordinate Z of the object being measured (u,v) W (i,u,v)=a(u,v)+b(u,v)j(i,u,v) max +c(u,v)j 2 (i,u,v) max

[0178] Value X W (i,u,v),Y W (i,u,v),Z W (i,u,v), traverse all pixel points to obtain the height distribution of the object being measured, that is, the reconstruction of the three-dimensional surface shape of the object being measured is completed.

[0179] In a second aspect, an embodiment of the present application provides an optical vertical measurement system based on binocular stereo vision, comprising:

[0180] Binocular stereo vision system module, camera a and camera b form a binocular stereo vision system, and complete the epipolar line correction of the binocular stereo vision system;

[0181] The maximum modulation index corresponds to the serial number module. The projector and camera a form a vertical measurement system. The maximum modulation index distribution of each pixel point on each calibration surface is calculated to obtain the serial number corresponding to the maximum modulation index;

[0182] The spatial coordinate value module uses a projector to project a fringe pattern to assist the binocular stereo vision system in completing stereo matching and calculate the spatial coordinate values ​​of all pixel points on each calibration surface in the binocular stereo vision system;

[0183] A mapping relationship module establishes a mapping relationship between the serial number corresponding to the maximum modulation value of all pixel points on each calibration surface in the vertical measurement system and the spatial coordinate values ​​of all pixel points on each calibration surface in the binocular stereo vision system;

[0184] The module for reconstructing the three-dimensional surface shape of the object under test uses a projector to zoom and scan the object under test, calculates the serial number value corresponding to the maximum modulation index of each pixel point, and reconstructs the three-dimensional surface shape of the object under test based on the above mapping relationship.

[0185] In the binocular stereo vision system module:

[0186] The maximum modulation index corresponds to the serial number module:

[0187] It can be understood that the conversion from system pixel coordinates to world coordinates is completed first, and then the epipolar correction of the binocular stereo vision system is completed.

[0188] In some embodiments, camera a and camera b form a binocular stereo vision system, and the conversion of system pixel coordinates to world coordinates includes the following steps:

[0189] S011. Establish the relationship between the 2D image pixel coordinates and the 2D image physical coordinates:

[0190]

[0191] Where u represents the number of columns in the image pixel coordinates, in pixels; v represents the number of rows in the image pixel coordinates, in pixels; u0 represents the position of the origin of the image physical coordinate column in the image pixel coordinates, and v0 represents the position of the origin of the image physical coordinate row in the image pixel coordinates; x represents the number of columns in the image physical coordinates, in millimeters; y represents the number of rows in the image physical coordinates, in millimeters;

[0192] S012. Establish the mapping relationship between the physical coordinates of the 2D image and the camera's 3D coordinate system points:

[0193]

[0194] Where x represents the number of columns in the physical coordinates of the image, in millimeters; y represents the number of rows in the physical coordinates of the image, in millimeters; f represents the focal length of the camera; X c , Y c , Z c are the three mutually orthogonal axes of the camera coordinate system;

[0195] Will Convert to the following homogeneous coordinate expression:

[0196]

[0197] Where x represents the number of columns in the physical coordinates of the image, in millimeters; y represents the number of rows in the physical coordinates of the image, in millimeters; f represents the focal length of the camera; X c , Y c , Z c are the three mutually orthogonal axes of the camera coordinate system;

[0198] S013. Establish the conversion relationship between the camera coordinate system and the world coordinate system:

[0199]

[0200] Where u represents the number of columns in the image pixel coordinates, in pixels; v represents the number of rows in the image pixel coordinates, in pixels; f represents the focal length of the camera; f x =f / d x , f y =f / d y , d x and d y Respectively represent the physical size of the camera unit pixel in the horizontal and vertical directions; u0 and v0 represent the coordinates of the camera plane at the optical center; R1 represents a 3×3 orthogonal rotation matrix, T1=[T x ,T y ,T z ] is the three-dimensional translation vector; X W , Y W , Z W Represents three mutually orthogonal axes in the world coordinate system; M1 represents the intrinsic parameter matrix; K1 is the external parameter matrix.

[0201] like Figure 2 As shown, it can be understood that a geometric constraint exists between the images captured by camera a and camera b (left and right cameras) in a binocular stereo vision system. This constraint is represented by epipolar lines, and epipolar geometry is the geometric relationship between the left and right images. Specifically, there is a point M in three-dimensional space, which is projected as image points m1 and m2 on the imaging planes of the left and right cameras, respectively. Due to the fixed relative position and orientation between the two cameras, these two image points are not randomly distributed but satisfy the epipolar constraint: given image point m1, its corresponding image point m2 must lie on a specific straight line in the right image, called the epipolar line. When performing stereo matching, directly matching homonymous points requires calculating the corresponding epipolar line for each search, and then searching along the entire epipolar line. This computation is relatively complex. However, before stereo matching, a certain image transformation is performed to ensure that the left and right imaging planes are coplanar and the corresponding epipolar lines are parallel and at the same height. This allows searching for homonymous points to be performed only on the same line, greatly reducing the difficulty of stereo matching.

[0202] In some embodiments, the epipolar correction algorithm comprises a Bouguet algorithm or a Fusiello algorithm.

[0203] It can be understood that the conversion coordinates are the spatial coordinate transformation relationship between the two cameras (i.e., external parameters), so that the points in one camera coordinate system can be accurately converted to another coordinate system. Epipolar correction is a geometric transformation operation performed on binocular images. Its purpose is to align the images so that the matching points of the same spatial point in the left and right images are on the same horizontal line. In this way, the epipolar lines that were originally distributed in any direction are "straightened" to the horizontal lines of the image, which significantly simplifies the image matching process. The main reason for epipolar correction is to improve the efficiency and accuracy of stereo matching. After correction, the search for corresponding points only needs to scan on the same line, which greatly reduces the amount of calculation; it also lays an accurate and unified foundation for subsequent disparity map calculation, depth estimation and three-dimensional reconstruction.

[0204] Furthermore, epipolar line correction of the binocular stereo vision system includes the following steps:

[0205] S014. Based on the Bouguet algorithm, assuming that the right camera b is used as the reference, the binocular camera external parameter rotation matrix R and translation vector T are obtained. The left camera a coordinate system is rotated half along the positive direction of the rotation vector corresponding to the rotation matrix R, and converted into a matrix form and recorded as R l , the right camera b coordinate system rotates half in the opposite direction of the rotation vector corresponding to the rotation matrix R, and is converted into a matrix form and recorded as R r ,At this time, the coordinate systems of camera a and camera b are parallel, but the baseline is not parallel to the imaging plane;

[0206] S015. Construct a transformation matrix to make the baseline parallel to the imaging plane. The construction formula is as follows:

[0207]

[0208] Where t is the translation vector T through R r The rotated vector, t = R r *T, e1 represents the normalized vector t, e2 represents the projection perpendicular to t on the xy plane, e3, e1 and e2 form an orthogonal right-handed coordinate system, t x and t y Both represent the components of vector t in the x and y directions respectively;

[0209] S016. Obtain e1, e2, and e3 to obtain the corrected rotation matrix as follows:

[0210]

[0211] Where R rect represents the transformation matrix, e1 T 、e2T 、e3 T Represent the transpose of e1, e2, and e3 respectively;

[0212] S017. Combine the transformation matrix and the rotation matrix of the left and right cameras to calculate the correction matrix of the binocular camera:

[0213]

[0214] Where R lrect Represents the rotation matrix of the left camera after stereo correction, R rrect Represents the rotation matrix of the right camera after stereo correction, R rect Represents the corrected rotation matrix, R l and R r Represent the rotation matrices of the left camera and the right camera respectively; at this point, the epipolar correction of the binocular stereo vision system is completed.

[0215] It can be understood that during the correction process, the new pixel coordinates may not be integers, and they need to be interpolated using the pixel values ​​of the original image. In addition, the corrected left and right images need to be cropped and the center and boundary areas of the images need to be redefined to ensure that the repeated overlapping area of ​​the left and right images is increased.

[0216] The maximum modulation index corresponds to the serial number module:

[0217] In some embodiments, the projector and camera a form a vertical measurement system, and calculating the maximum modulation distribution of each pixel point on any calibration surface and obtaining the serial number corresponding to the maximum modulation value includes the following steps:

[0218] S021. Zoom the projector along the optical axis at equal intervals, and project N frames of fringe patterns with a fixed phase difference onto the calibration surface at each moving position. The light field distribution collected by camera a on the focal plane is expressed as:

[0219]

[0220] The light field collected by camera a before and after the focal plane is expressed as:

[0221]

[0222] Where, and represents the light field distribution on the focal plane and the image plane at a distance H from the focal plane, respectively; j represents the number of times the projector zooms along the optical axis at equal intervals, and j = 1, 2, 3…J, J represents the total number of moves; N represents the N-step phase shift, and N ≥ 3; n and n' distribution represent the nth fringe pattern projected on the focal plane or the defocused plane at position j, n (or n') = 0, 1, 2…, N-1; σ Hrepresents the standard deviation of the point spread function; u represents the number of columns in the image pixel coordinates, in pixels, and v represents the number of rows in the image pixel coordinates, in pixels; R2(u,v), B2(u,v) and C2(u,v) represent the surface reflectivity, ambient light intensity and fringe contrast, respectively; M2 represents the measurement system magnification; f0 represents the fringe frequency, and Φ(u,v) represents the initial phase of the fringe;

[0223] S022, extract the fringes on the focal plane as follows

[0224]

[0225] The fringes on the out-of-focus plane are as follows

[0226]

[0227] The modulation distribution of the fringes on the above focal plane and out-of-focus plane is as follows:

[0228]

[0229]

[0230] Where M f (u,v) and M' f (u,v;σ H ) represent the modulation index distribution on the focal plane and the defocus plane respectively; I(u,v) and I(u,v; σ H ) represent the light field distribution on the focal plane and the image plane at a distance H from the focal plane; σ H represents the standard deviation of the point spread function; u represents the number of columns in the image pixel coordinates, in pixels; v represents the number of rows in the image pixel coordinates, in pixels;

[0231] S023. Perform quadratic interpolation fitting on the curve formed by the modulation values ​​of the pixels with the same name at each scanning position, extract the maximum modulation value of the pixels with the same name, and obtain the serial number corresponding to the maximum modulation value of the pixels with the same name (u, v), where i represents the i-th calibration surface, u and v represent the number of rows and columns in the image pixel coordinates, respectively, and j represents the serial number, which is a floating point number.

[0232] In said S021:

[0233] Furthermore, the projector is fixed on the optical platform, and the current value of the electronically adjustable focus liquid lens is changed at equal intervals, so that the projector zooms the projected light field to different calibration surfaces.

[0234] In the spatial coordinate value module:

[0235] In some embodiments, a projector is used to project a fringe pattern to assist a binocular stereo vision system in completing stereo matching, and the spatial coordinate values ​​of all pixel points on each calibration surface in the binocular stereo vision system are calculated, including the following steps:

[0236] S031. Use a projector to project a fringe pattern and introduce phase information to assist the binocular stereo vision system in more accurately matching the same-name points and completing stereo matching. The introduction of phase can be achieved by projecting an N-step phase shift pattern. The light field of the N-step phase shift pattern can be expressed as:

[0237]

[0238] Where G n (u,v) represents the light field distribution, u and v represent the number of rows and columns in the image pixel coordinates, α(u,v) and β(u,v) represent the ambient light intensity and the contrast of the stripes, respectively, f0 is the stripe frequency, is the phase of the fringe, N is an integer, indicating N-step phase shift, n represents the nth fringe pattern, and n = 0, 1, 2…, N-1;

[0239] Image phase information extraction is calculated using the following formula:

[0240]

[0241] Where, is the truncated phase of the coordinate point (u, v), where u and v represent the row and column numbers in the image pixel coordinates, respectively. n Represents the light field distribution, N is an integer, representing N-step phase shift, n represents the nth fringe pattern, and n=0,1,2…,N-1;

[0242] The above formula cuts the phase information between [-π,π], and its absolute phase information is

[0243]

[0244] Where, Φ uw (u,v) is the absolute phase value of the coordinate point (u,v), is the truncated phase at the coordinate point (u, v), where u and v represent the row and column numbers in the image pixel coordinates, and k represents the fringe order;

[0245] It can be understood from the above formula that after epipolar correction, according to the monotonically increasing property of the absolute phase value on the epipolar line, for a pixel point (u L ,v L ), according to the epipolar constraint, search for the pixel point with the closest phase value on the same row of the right image (u R ,v R) as the same-name points, thereby quickly matching the same-name points on the two camera images to complete binocular stereo matching.

[0246] S032, spatial coordinate value X of any point in space W (u,v),Y W (u,v), Z W The reconstruction of (u,v) is done by the following formula

[0247]

[0248]

[0249]

[0250] Among them, X W (u,v),Y W (u,v), Z W (u, v) represent the spatial coordinate values ​​of the pixel point (u, v), u L ,v L Respectively represent the number of rows and columns in the left camera, u R ,v R They represent the number of rows and columns in the right camera, d represents the baseline distance between the left and right cameras, and f represents the focal length of the camera.

[0251] In the mapping relationship module:

[0252] In some embodiments, establishing a mapping relationship between the serial numbers corresponding to the maximum modulation values ​​of all pixels on each calibration surface in the vertical measurement system and the spatial coordinate values ​​of all pixels on each calibration surface in the binocular stereo vision system includes the following steps:

[0253] Within the measurement range of the vertical system, complete the scanning of each calibration surface and obtain the spatial coordinate value X of each pixel point on the calibration surface. W (i,u,v),Y W (i,u,v),Z W (i,u,v), and the spatial coordinate value X of each pixel point on the calibration surface W (i,u,v),Y W (i,u,v),Z W (i,u,v) and the maximum modulation index corresponding to the serial number are fitted by quadratic interpolation to establish the spatial coordinate value X W (i,u,v),Y W (i,u,v),Z W (i,u,v) corresponds to the sequence number j(i,u,v) with the maximum modulation index max The mapping relationship between them:

[0254] XW (i,u,v)=a(u,v)+b(u,v)j(i,u,v) max +c(u,v)j 2 (i,u,v) max

[0255] Y W (i,u,v)=a(u,v)+b(u,v)j(i,u,v) max +c(u,v)j 2 (i,u,v) max

[0256] Z W (i,u,v)=a(u,v)+b(u,v)j(i,u,v) max +c(u,v)j 2 (i,u,v) max

[0257] Where i represents the i-th calibration surface, u and v represent the number of rows and columns in the image pixel coordinates, respectively, and X W (i,u,v),Y W (i,u,v),Z W (i,u,v) represents the coordinate values ​​of the i-th calibration surface in the row and column of the image in the directions of the three coordinate axes X, Y, and Z, j(i,u,v) max represents the maximum modulation index of the i-th calibration surface, a(u,v), b(u,v), and c(u,v) are the fitting coefficients of the depth value-maximum modulation index curve.

[0258] In the module for reconstructing the three-dimensional surface shape of the object being measured:

[0259] In some embodiments, a projector zooms and scans an object to be measured, calculates a serial number value corresponding to the maximum modulation index of each pixel point, and reconstructs a three-dimensional surface shape of the object to be measured based on the above mapping relationship, including the following steps: placing the object to be measured within the range of a calibration system and keeping the position unchanged, changing the current value of an electronically adjustable focus liquid lens at equal intervals so that the projector zooms and projects a light field onto the object to be measured, and at each current value of the electronically adjustable focus liquid lens, projects N frames of fringe patterns with a fixed phase difference onto the object to be measured, and synchronously captures the fringe patterns with a camera to obtain a serial number j(i, u, v) corresponding to the maximum modulation index. max , according to X W (i,u,v)=a(u,v)+b(u,v)j(i,u,v) max +c(u,v)j 2 (i,u,v) max Y W (i,u,v)=a(u,v)+b(u,v)j(i,u,v)max +c(u,v)j 2 (i,u,v) max , obtain the pixel coordinate Z of the object being measured (u,v) W (i,u,v)=a(u,v)+b(u,v)j(i,u,v) max +c(u,v)j 2 (i,u,v) max

[0260] Value X W (i,u,v),Y W (i,u,v),Z W (i,u,v), traverse all pixel points to obtain the height distribution of the object being measured, that is, the reconstruction of the three-dimensional surface shape of the object being measured is completed.

[0261] In a third aspect, the present application provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the optical vertical measurement method based on binocular stereo vision as described above.

[0262] The computer device may be a desktop computer, a notebook computer, a PDA, a cloud server, etc. The computer device may interact with the user via a keyboard, a mouse, a remote control, a touchpad, or a voice control device.

[0263] The memory includes at least one type of readable storage medium, including flash memory, hard disk, multimedia card, card-type memory (e.g., SD or D interface display memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory can be an internal storage unit of the computer device, such as the hard disk or memory of the computer device. In other embodiments, the memory can also be an external storage device of the computer device, such as a plug-in hard disk equipped with the computer device, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card, etc. Of course, the memory can also include both the internal storage unit of the computer device and its external storage device. In this embodiment, the memory is often used to store the operating system and various application software installed on the computer device, such as the program code of the optical vertical measurement method based on binocular stereo vision. In addition, the memory can also be used to temporarily store various types of data that have been output or are about to be output.

[0264] In some embodiments, the processor may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip. The processor is generally used to control the overall operation of the computer device. In this embodiment, the processor is used to run the program code stored in the memory or process data, such as running the program code of the optical vertical measurement method based on binocular stereo vision.

[0265] In a fourth aspect, the present application provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the processor executes the steps of the optical vertical measurement method based on binocular stereo vision as described above.

[0266] The computer-readable storage medium stores an interface display program, and the interface display program can be executed by at least one processor to enable the at least one processor to perform the steps of the above-mentioned optical vertical measurement method based on binocular stereo vision.

[0267] The principle diagram of the optical vertical measurement system based on binocular stereo vision in this application is as follows Figure 3As shown, i represents the serial number of the calibration surface corresponding to different depths, i=1 represents the reference surface, that is, the height is 0mm, and i=I represents the calibration height farthest from the reference surface. During the experiment, in order to facilitate the verification of the accuracy of the mapping relationship, the calibration surfaces are placed at equal intervals between the calibration intervals i=1 and i=I, that is, the calibration surfaces are placed at equal intervals within the calibration interval. In fact, within the calibration space, the calibration surfaces can be placed in the calibration interval at any posture. When calibrating the system, the calibration surface is moved from t=1 to t=T in sequence. For any calibration surface, when the projector 1 zooms and scans the calibration surface, the current value of the electronic adjustable focus liquid lens 2 will be evenly spaced from G1 to G J , changing J values ​​in total. Correspondingly, the focal length of projector 1 will change from f1 to f J , camera a4 performs synchronous acquisition based on the light field distribution on the surface of the measured object 6 reflected by the semi-transparent and semi-reflective mirror 3. At this point, the vertical measurement system image acquisition work is completed. According to S02, the modulation degree distribution on different focal planes when the calibration surface i = 1 can be calculated. For any pixel point (u, v) on the calibration surface, an approximate Gaussian distribution curve can be obtained. The maximum value of each curve will correspond to a serial number j(i,u,v). max The serial number value is also the position where the projected grating of projector 1 is clearest on the calibration surface. Set the current value of electronic adjustable focus liquid lens 2 to G j(i,u,v)max , projector 1 projects N frames of fringe pattern, camera a4 and camera b5 shoot at the same time, at this time the binocular stereo vision system image acquisition work is completed, and the spatial coordinate value X of each pixel point on the calibration surface is calculated according to the formula in S03 W (i,u,v),Y W (i,u,v),Z W (i,u,v). Any point on the same calibration surface will get the maximum modulation index serial number and spatial coordinate value. Move the calibration surface to i=2, repeat the above operation until the calibration surface at i=I reaches the maximum value. Set the serial number j(I,u,v) max And the spatial coordinate value X of each pixel W (I,u,v),Y W (I,u,v),Z W (I,u,v) calculation. You can establish the spatial coordinate value X according to the formula in S04 W (i,u,v),Y W (i,u,v),Z W (i,u,v) corresponds to the sequence number j(i,u,v) with the maximum modulation index max The mapping relationship between them.

[0268] Application Examples

[0269] The calibration depth is from 0mm to 50mm, with a total of 11 calibration surfaces, and the distance between any two adjacent calibration surfaces is 5mm. The three-dimensional object being measured is a plaster head of David. A total of 300 different zoom positions are collected during the scanning of the object being measured. Figure 4 When the projector changes its focal length for the 151st time, the fringe pattern is projected by the projection device onto the light field distribution of the object being measured. Figure 5 This is the error distribution diagram after the system tests the standard surface and subtracts it from the fitting surface. The root mean square error is 0.1009mm. Figure 6 In order to reconstruct the three-dimensional surface image of the measured object using the present invention, it can be seen that the present invention can provide the physical actual size of the reconstructed three-dimensional surface image.

[0270] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. An optical vertical measurement method based on binocular stereo vision, characterized in that: include: Camera a and camera b form a binocular stereo vision system and complete the epipolar correction of the binocular stereo vision system; The projector and camera a form a vertical measurement system, calculate the maximum modulation distribution of each pixel point on each calibration surface, and obtain the serial number corresponding to the maximum modulation value; Use a projector to project a fringe pattern to assist the binocular stereo vision system in completing stereo matching and calculate the spatial coordinate values ​​of all pixel points on each calibration surface in the binocular stereo vision system; Establish a mapping relationship between the serial number corresponding to the maximum modulation value of all pixel points on each calibration surface in the vertical measurement system and the spatial coordinate values ​​of all pixel points on each calibration surface in the binocular stereo vision system; The projector zooms and scans the object to be measured, calculates the serial number value corresponding to the maximum modulation index of each pixel point, and reconstructs the three-dimensional surface shape of the object to be measured based on the above mapping relationship.

2. The optical vertical measurement method based on binocular stereo vision according to claim 1, characterized in that: Cameras a and b form a binocular stereo vision system. The conversion of system pixel coordinates to world coordinates includes the following steps: Establish the relationship between 2D image pixel coordinates and 2D image physical coordinates: Where u represents the number of columns in the image pixel coordinates, in pixels; v represents the number of rows in the image pixel coordinates, in pixels; u0 represents the position of the origin of the image physical coordinate column in the image pixel coordinates, and v0 represents the position of the origin of the image physical coordinate row in the image pixel coordinates; x represents the number of columns in the image physical coordinates, in millimeters; y represents the number of rows in the image physical coordinates, in millimeters; Establish the mapping relationship between the physical coordinates of the 2D image and the camera's 3D coordinate system points: Where x represents the number of columns in the physical coordinates of the image, in millimeters; y represents the number of rows in the physical coordinates of the image, in millimeters; f represents the focal length of the camera; X c , Y c , Z c are the three mutually orthogonal axes of the camera coordinate system; Will Convert to the following homogeneous coordinate expression: Where x represents the number of columns in the physical coordinates of the image, in millimeters; y represents the number of rows in the physical coordinates of the image, in millimeters; f represents the focal length of the camera; X c , Y c , Z c are the three mutually orthogonal axes of the camera coordinate system; Establish the transformation relationship between the camera coordinate system and the world coordinate system: Where u represents the number of columns in the image pixel coordinates, in pixels; v represents the number of rows in the image pixel coordinates, in pixels; f represents the focal length of the camera; f x =f / d x , f y =f / d y , d x and d y Respectively represent the physical size of the camera unit pixel in the horizontal and vertical directions; u0 and v0 represent the coordinates of the camera plane at the optical center; R1 represents a 3×3 orthogonal rotation matrix, T1=[T x ,T y ,T z ] is the three-dimensional translation vector; X W , Y W , Z W Represents three mutually orthogonal axes in the world coordinate system; M1 represents the intrinsic parameter matrix; K1 is the external parameter matrix.

3. The optical vertical measurement method based on binocular stereo vision according to claim 1, characterized in that: Epipolar line correction of binocular stereo vision system includes the following steps: Based on the Bouguet algorithm, assuming that the right camera b is used as the reference, the binocular camera external parameter rotation matrix R and translation vector T are obtained. The left camera a coordinate system is rotated half along the positive direction of the rotation vector corresponding to the rotation matrix R, and converted into a matrix form recorded as R l , the right camera b coordinate system rotates half in the opposite direction of the rotation vector corresponding to the rotation matrix R, and is converted into a matrix form and recorded as R r ,At this time, the coordinate systems of camera a and camera b are parallel, but the baseline is not parallel to the imaging plane; Construct a transformation matrix to make the baseline parallel to the imaging plane. The construction formula is as follows: Where t is the translation vector T through R r The rotated vector, t = R r *T, e1 represents the normalized vector t, e2 represents the projection perpendicular to t on the xy plane, e3, e1 and e2 form an orthogonal right-handed coordinate system, t x and t y Both represent the components of vector t in the x and y directions respectively; By obtaining e1, e2, and e3, the corrected rotation matrix can be obtained as follows: Where R rect represents the transformation matrix, e1 T 、e2 T 、e3 T Represent the transpose of e1, e2, and e3 respectively; Combining the transformation matrix and the rotation matrix of the left and right cameras, the correction matrix of the binocular camera is calculated as: Where R lrect Represents the rotation matrix of the left camera after stereo correction, R rrect Represents the rotation matrix of the right camera after stereo correction, R rect Represents the corrected rotation matrix, R l and R r Represent the rotation matrices of the left camera and the right camera respectively; at this point, the epipolar correction of the binocular stereo vision system is completed.

4. The optical vertical measurement method based on binocular stereo vision according to claim 1, characterized in that: The projector and camera a form a vertical measurement system, which calculates the maximum modulation distribution of each pixel point on any calibration surface and obtains the serial number corresponding to the maximum modulation value, including the following steps: The projector is zoomed along the optical axis at equal intervals, and N frames of fringe patterns with a fixed phase difference are projected onto the calibration surface at each moving position. The light field distribution collected by camera a on the focal plane is expressed as: The light field collected by camera a before and after the focal plane is expressed as: Where, and represents the light field distribution on the focal plane and the image plane at a distance H from the focal plane, respectively; j represents the number of times the projector zooms along the optical axis at equal intervals, and j = 1, 2, 3…J, J represents the total number of moves; N represents the N-step phase shift, and N ≥ 3; n and n' distribution represent the nth fringe pattern projected on the focal plane or the defocused plane at position j, n (or n') = 0, 1, 2…, N-1; σ H represents the standard deviation of the point spread function; u represents the number of columns in the image pixel coordinates, in pixels, and v represents the number of rows in the image pixel coordinates, in pixels; R2(u,v), B2(u,v) and C2(u,v) represent the surface reflectivity, ambient light intensity and fringe contrast, respectively; M2 represents the measurement system magnification; f0 represents the fringe frequency, and Φ(u,v) represents the initial phase of the fringe; Extract the fringes on the focal plane as follows The fringes on the out-of-focus plane are as follows The modulation distribution of the fringes on the above focal plane and out-of-focus plane is as follows: Where M f (u,v) and M' f (u,v;σ H ) represent the modulation index distribution on the focal plane and the defocus plane respectively; I(u,v) and I(u,v; σ H ) represent the light field distribution on the focal plane and the image plane at a distance H from the focal plane; σ H represents the standard deviation of the point spread function; u represents the number of columns in the image pixel coordinates, in pixels; v represents the number of rows in the image pixel coordinates, in pixels; Perform quadratic interpolation fitting on the curve formed by the modulation values ​​of the same-named pixels at each scanning position, extract the maximum modulation value of the same-named pixels, and obtain the serial number corresponding to the maximum modulation value of the same-named pixels (u, v), where i represents the i-th calibration surface, u and v represent the number of rows and columns in the image pixel coordinates, respectively, and j represents the serial number, which is a floating-point number.

5. The optical vertical measurement method based on binocular stereo vision according to claim 1, characterized in that: The fringe pattern is projected by a projector to assist the binocular stereo vision system in completing stereo matching. The spatial coordinate values ​​of all pixel points on each calibration surface in the binocular stereo vision system are calculated, including the following steps: The fringe pattern is projected by a projector, and phase information is introduced to assist the binocular stereo vision system to more accurately match the same-name points and complete stereo matching. The introduction of phase can be achieved by projecting an N-step phase shift pattern. The light field of the N-step phase shift pattern can be expressed as: Where G n (u,v) represents the light field distribution, u and v represent the number of rows and columns in the image pixel coordinates, α(u,v) and β(u,v) represent the ambient light intensity and the contrast of the stripes, respectively, f0 is the stripe frequency, is the phase of the fringe, N is an integer, indicating N-step phase shift, n represents the nth fringe pattern, and n = 0, 1, 2…, N-1; Image phase information extraction is calculated using the following formula: Where, is the truncated phase of the coordinate point (u, v), where u and v represent the row and column numbers in the image pixel coordinates, respectively. n Represents the light field distribution, N is an integer, representing N-step phase shift, n represents the nth fringe pattern, and n=0,1,2…,N-1; The above formula cuts the phase information between [-π,π], and its absolute phase information is Where, Φ uw (u,v) is the absolute phase value of the coordinate point (u,v), is the truncated phase at the coordinate point (u, v), where u and v represent the row and column numbers in the image pixel coordinates, and k represents the fringe order; The spatial coordinate value X of any point in space W (u,v),Y W (u,v), Z W The reconstruction of (u,v) is done by the following formula Among them, X W (u,v),Y W (u,v), Z W (u, v) represent the spatial coordinate values ​​of the pixel point (u, v), u L ,v L Respectively represent the number of rows and columns in the left camera, u R ,v R They represent the number of rows and columns in the right camera, d represents the baseline distance between the left and right cameras, and f represents the focal length of the camera.

6. The optical vertical measurement method based on binocular stereo vision according to claim 1, characterized in that: Establishing a mapping relationship between the maximum modulation values ​​of all pixel points on each calibration surface in the vertical measurement system and the spatial coordinate values ​​of all pixel points on each calibration surface in the binocular stereo vision system includes the following steps: Within the measurement range of the vertical system, complete the scanning of each calibration surface and obtain the spatial coordinate value X of each pixel point on the calibration surface. W (i,u,v),Y W (i,u,v),Z W (i,u,v), and the spatial coordinate value X of each pixel point on the calibration surface W (i,u,v),Y W (i,u,v),Z W (i,u,v) and the maximum modulation index corresponding to the serial number are fitted by quadratic interpolation to establish the spatial coordinate value X W (i,u,v),Y W (i,u,v),Z W (i,u,v) corresponds to the sequence number j(i,u,v) with the maximum modulation index max The mapping relationship between them: X W (i,u,v)=a(u,v)+b(u,v)j(i,u,v) max +c(u,v)j 2 (i,u,v) max Y W (i,u,v)=a(u,v)+b(u,v)j(i,u,v) max +c(u,v)j 2 (i,u,v) max Z W (i,u,v)=a(u,v)+b(u,v)j(i,u,v) max +c(u,v)j 2 (i,u,v) max Where i represents the i-th calibration surface, u and v represent the number of rows and columns in the image pixel coordinates, respectively, and X W (i,u,v),Y W (i,u,v),Z W (i,u,v) represents the coordinate values ​​of the i-th calibration surface in the row and column of the image in the directions of the three coordinate axes X, Y, and Z, j(i,u,v) max represents the maximum modulation index of the i-th calibration surface, a(u,v), b(u,v), and c(u,v) are the fitting coefficients of the depth value-maximum modulation index curve.

7. The optical vertical measurement method based on binocular stereo vision according to claim 1, characterized in that: The projector zooms and scans the object to be measured, calculates the serial number value corresponding to the maximum modulation degree of each pixel point, and reconstructs the three-dimensional surface shape of the object to be measured based on the above mapping relationship, including the following steps: Place the object to be measured within the calibration system and keep the position unchanged. Change the current value of the electronically adjustable focus liquid lens at equal intervals so that the projector zooms and projects the light field onto the object to be measured. At each current value of the electronically adjustable focus liquid lens, N frames of fringe patterns with a fixed phase difference are projected onto the object to be measured. The camera synchronously collects the fringe pattern and obtains the serial number j(i,u,v) corresponding to the maximum modulation index. max , according to X W (i,u,v)=a(u,v)+b(u,v)j(i,u,v) max +c(u,v)j 2 (i,u,v) max Y W (i,u,v)=a(u,v)+b(u,v)j(i,u,v) max +c(u,v)j 2 (i,u,v) max , obtained object (u,v) image element point Z W (i,u,v)=a(u,v)+b(u,v)j(i,u,v) max +c(u,v)j 2 (i,u,v) max Value X W (i,u,v),Y W (i,u,v),Z W (i,u,v), traverse all pixel points to obtain the height distribution of the object being measured, that is, the reconstruction of the three-dimensional surface shape of the object being measured is completed.

8. An optical vertical measurement system based on binocular stereo vision, characterized in that: include: Binocular stereo vision system module, camera a and camera b form a binocular stereo vision system, and complete the epipolar line correction of the binocular stereo vision system; The maximum modulation index corresponds to the serial number module. The projector and camera a form a vertical measurement system. The maximum modulation index distribution of each pixel point on each calibration surface is calculated to obtain the serial number corresponding to the maximum modulation index; The spatial coordinate value module uses a projector to project a fringe pattern to assist the binocular stereo vision system in completing stereo matching and calculate the spatial coordinate values ​​of all pixel points on each calibration surface in the binocular stereo vision system; A mapping relationship module establishes a mapping relationship between the serial number corresponding to the maximum modulation value of all pixel points on each calibration surface in the vertical measurement system and the spatial coordinate values ​​of all pixel points on each calibration surface in the binocular stereo vision system; The module for reconstructing the three-dimensional surface shape of the object under test uses a projector to zoom and scan the object under test, calculates the serial number value corresponding to the maximum modulation index of each pixel point, and reconstructs the three-dimensional surface shape of the object under test based on the above mapping relationship.

9. Computer device, characterized in that The method comprises a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that A computer program is stored, and when the computer program is executed by a processor, the processor is caused to perform the steps of the method according to any one of claims 1 to 7.

Images