Transparent glass surface shape measuring device and method based on three-dimensional deflection technology

Abstract

The invention discloses a transparent glass surface shape measuring device and a transparent glass surface shape measuring method based on stereoscopic deflection, a plane mirror is introduced to complete calibration work, global system parameters are optimized according to re-projection results of the plane mirror, system calibration flexibility is improved, and calibration precision is improved. Glass to be detected can be placed at any position in a to-be-detected area on a carrying platform without fixing, a Gray code auxiliary line shift fringe technology is adopted as a coding and unfolding scheme, the coding scheme is completely based on a binary coding method, a modulation graph has the characteristic of spatial separation, coding information is easier to extract, and the coding efficiency is improved. The reflection information of the upper surface of the glass is extracted through a self-adaptive method based on a clustering algorithm, threshold parameters do not need to be set manually, a pure vision scheme is adopted, the method has the advantages of being low in cost, large in dynamic measurement range and the like, and the measurement precision meets the actual industrial measurement requirement.

Description

Technical Field

[0001] The invention relates to the technical field of three-dimensional profile measurement, and in particular to a transparent glass surface shape measurement device and method based on stereo deflectometry. Background Art

[0002] Transparent free-form glass is a common material that is widely used in industries such as display, automobile, construction, and electronic information. However, due to the limitations of industrial processing technology, it is difficult to avoid errors in the produced glass, which will affect the optical properties and mechanical strength of the glass. Among them, the detection of glass surface shape is crucial to evaluating glass quality because it provides rich three-dimensional morphological information that can be used to analyze and evaluate the quality of the glass surface, such as flatness, warping, contour and other indicators. Especially in the field of high-precision manufacturing, such as automotive HUD (head-up display) systems, tiny defects on the glass surface may cause light to distort or scatter when passing through the glass, thereby affecting the display effect of the HUD system.

[0003] Traditional glass quality inspection methods mainly rely on manual measurement, but this method has problems such as large human resource investment, low efficiency, low measurement accuracy and sparse data. Classical line laser, structured light and other methods are difficult to apply due to the transparency of glass, while spectral confocal method and interference law are difficult to measure large-sized transparent glass. Phase deflectometry is an efficient, flexible and robust measurement technology for complex surfaces. Its dynamic range is about 1000 times higher than that of traditional interference method and is suitable for large-sized surface measurement. However, transparent glass is a refractive or mixed reflection / refractive element. Directly using phase deflectometry to measure the surface shape of transparent glass will introduce the noise influence caused by the reflection component of the lower surface of the glass. The current coding and extraction algorithm is used to demodulate the reflection component of the upper surface of glass, which is usually low in robustness or high in computational complexity. Summary of the invention

[0004] In order to overcome the phenomenon of aliasing of refraction and reflection information when measuring the surface of transparent glass, a large field of view and high-precision three-dimensional surface detection is achieved. This invention first provides a mechanical structure device that can be applied to the measurement of the surface of free-form transparent glass, which mainly solves the problem of additional reflection on the lower surface of the glass generated during the projection of the diffuse reflection display screen. Secondly, by adding a standard plane mirror, the problem of calibrating the position and posture of the display screen is completed, which reduces the cost of calibration and improves the flexibility of calibration. In addition, by reprojecting the standard plane mirror, the global calibration parameters are optimized and the detection accuracy of the system is improved. The present invention is achieved through the following technical solutions:

[0005] The invention discloses a transparent glass surface shape measuring device based on stereo deflectometry, comprising:

[0006] Display screen: used as a diffuse reflection light source to project structured light information onto the glass surface to be tested;

[0007] Standard plane mirror: used to assist in calibrating the display screen;

[0008] Lifting platform: used to place the glass to be tested and adjust the height of the glass plane to be tested;

[0009] Image acquisition device: including two industrial area array cameras and corresponding lenses, as left and right binocular cameras of stereo deflectometry, set in the direction of reflected light of the glass to be tested, used to capture the modulation image information reflected by the surface of the glass to be tested;

[0010] Fixing device: used to fix the camera and display screen, and adjust the display screen position and field of view;

[0011] Computer: used to generate corresponding surface structured light modulation information, and synchronously collect multiple images of the measured object with modulated structured light through an image acquisition device, and used to calculate the transparent glass gradient information using the measurement method according to the images collected by the image acquisition device at different times and different positions, so as to realize the three-dimensional reconstruction of the surface of the transparent glass;

[0012] The computer is connected to the image acquisition device and the display screen respectively, and the display screen faces the glass to be tested.

[0013] As a further improvement, the display screen of the present invention has an angle of 30 to 60° with the horizontal line, a vertical distance of 10 to 30 cm from the glass to be tested, and ensures that the projected image completely covers the glass to be tested; the binocular camera faces the glass to be tested, the angle between the optical axis and the horizontal line is 30 to 60°, the baseline length of the binocular camera is 60 to 200 mm, and ensures that both cameras can completely obtain the image of the glass to be tested.

[0014] The present invention also discloses a transparent glass surface shape measurement method based on stereo deflectometry, comprising:

[0015] S1 Camera calibration: The binocular cameras synchronously collect multiple checkerboard images with different postures in their common field of view, and accurately calibrate the internal parameters of each camera of the image acquisition device according to the Zhang Zhengyou calibration method to ensure that there is an accurate mapping relationship between the image data captured by the camera and the three-dimensional coordinates of the actual object. Then, the precise posture relationship between the left camera and the right camera is calculated and the binocular cameras are corrected by Bouguet to maximize the common field of view of the binocular cameras;

[0016] S2 display screen calibration: Generate a checkerboard pattern on the display screen, place a plane mirror on the stage, ensure that the binocular camera can directly obtain the virtual image of the display screen in the mirror, adjust the plane mirror posture at least three times, control the binocular camera to synchronously collect the corresponding checkerboard virtual image on the display screen, and solve the position relationship between the binocular camera and the display screen;

[0017] S3 global parameter optimization: Use the rough calibration results of S1 and S2 to reproject the plane mirror, and further optimize the global calibration parameters by calculating the flatness error, distance error and angle error to obtain the calibration parameters;

[0018] S4 obtains the modulation image: Gray code-assisted line shift technology and phase shift stripes are used as the surface structured light projection pattern to control the display screen to project onto the glass surface, and synchronously controls the binocular camera to collect the reflection pattern of the glass surface to be measured at the corresponding moment to obtain the modulation image;

[0019] S5 extracts the upper surface reflection component map: normalizes the gray value of the line shift modulation map obtained in S4, and adaptively selects the threshold through the K-means clustering algorithm, and extracts the upper surface reflection component map by adaptive threshold segmentation;

[0020] S6 wrapping code information unfolding: Kalman filtering is performed on the upper surface reflection component map obtained in S5; the center line of the line shift stripe in S5 is extracted using the Steger algorithm based on the Hessian matrix; the complementary Gray code unwrapping algorithm is used to unfold the relative code to obtain the coding information of the binocular camera in the horizontal and vertical directions respectively; further, the calibration parameters obtained in S3 can be used to obtain the correspondence between the pixel coordinates of the left and right cameras and the pixel coordinates of the display screen;

[0021] S7 gradient information acquisition: according to the correspondence between the pixel coordinates of the left and right cameras and the pixel coordinates of the display screen obtained in S6, the gradient information of the glass surface is acquired through a binocular matching method based on phase difference minimization;

[0022] S8 Gradient integral reconstruction: Based on the gradient information obtained in S7, the local RBF is used to replace the global RBF, namely RBF-FD, and the multivariate Hermite data, namely the gradient, is combined with the constraints of the point cloud data to complete the accurate reconstruction of the glass surface shape.

[0023] As a further improvement, in step S5 described in the present invention, the specific implementation steps of adaptively selecting the threshold include: taking the grayscale value of each pixel of the modulation image captured by the camera as the maximum grayscale value of the corresponding pixel in the modulation image, and normalizing it; using the normalized grayscale value as the characteristic value, clustering each pixel through the K-means algorithm; since the grayscale value change process presents four stages, corresponding to the quaternary coding, that is, K=4; the threshold of the Gray code modulation information on the upper surface of the glass after the grayscale value is normalized can be calculated as the average value of the centers of the two coding clusters with larger grayscale values, and the grayscale threshold is reversely normalized after Kalman smoothing filtering to obtain the actual grayscale threshold distribution.

[0024] As a further improvement, in step S6 of the present invention, the Hessian matrix is ​​expressed as:

[0025]

[0026] Among them, Z(x, y) is a two-dimensional image, g(x, y) is a two-dimensional Gaussian function,

[0027] Taking (x0, y0) as the base point, let the unit vector obtained by the Hessian matrix be (n x , n y ), and perform a second-order Taylor expansion along the bright stripe section to obtain the point on the section (x0+tn x ,y0+tn y )’s grayscale value:

[0028] z(x0+tn x ,y0+tn y )=Z(x0,y0)+N(r x , r y ) T +NH(x,y)N T / 2

[0029] Where N = (tn x ,tn y ), the image pixel z(x, y) is convolved with its corresponding partial differential form of Gaussian kernel to obtain (r x , r y ), as follows:

[0030]

[0031] according to It can be calculated that:

[0032]

[0033] The exact position of the center point of the bright stripe is (x0+tn x ,y0+tn y ).

[0034] As a further improvement, the specific steps in step S7 of the present invention are:

[0035] 1) Select the phase value in the first camera image plane to be For any given pixel A1, A1 is compared with the first camera optical center Concatenate to construct the incident vector Here, the camera is considered a pinhole camera;

[0036] 2) Find the same phase value on the display Pixels Assumptions A point P1 on the surface is the reflection point, and the corresponding reflection vector is obtained. and normal vector

[0037] 3) Align P1 with the optical center of the second pinhole camera Concatenate to construct the incident vector The intersection pixel B1 with the second camera image plane has a phase value of

[0038] 4) According to the normal vector determined in 2), we can obtain The corresponding reflection vector Intersection point with LCD plane Its phase value B is

[0039] 5) Set along Search until ε is less than the set threshold, and this point is the true reflection point;

[0040] 6) Traverse all pixel points and repeat the above steps to obtain the gradient information of the glass surface.

[0041] As a further improvement, in step S8 of the present invention, RBF-FD uses local RBF instead of global RBF to improve the calculation efficiency. x c The result calculated by the linear operator L is approximately a linear combination of n neighboring nodes in the same template:

[0042]

[0043] Among them, in order to calculate the n neighboring nodes in the template, compared with the method of calculating the distance between each two points and then sorting, the KD tree is used to achieve lower time complexity. Under the conditions that the glass surface is continuous and second-order differentiable, the interpolation of the glass surface can be defined as a weighted combination of weights calculated according to the RBF-FD in the case of a first-order derivative operator:

[0044]

[0045] Where N is the number of sampling points, s(x) is the depth information calculated at sampling point x, and W x (xx i ) and W y (xx i ) are the partial derivative operators in the X direction respectively by the RBF-FD method and the Y-direction partial derivative operator In this case, according to the evaluation point x i The calculated weight value of the test point x, α i and β i W x (xx i ) and W y (xx i )’s weighting coefficient;

[0046] Weight W x and W y It can be obtained by RBF-FD weight calculation method. The depth information s(x) also requires the coefficient matrix [α1, ..., α N ] T and [β1, ..., β N ] T To solve, in order to calculate the coefficient matrix, the analytical derivatives of the interpolation equations are matched with the measured gradients to obtain the optimization objective:

[0047]

[0048] As a further improvement, in step S8 of the present invention, the constraints of multivariate Hermite data, i.e., gradient and point cloud data, are introduced to improve the optimization objective to:

[0049]

[0050] A system of linear equations can be described in matrix form:

[0051]

[0052] in:

[0053]

[0054] W xx , W xy and W yy is the second-order partial derivative operator of the weight W. The weight matrix of the current sampling point is calculated according to the sampling points in the template. By solving the above linear equations, the coefficient matrix [α1, ..., α N ] T and [β1, ..., β N ] T , then the glass surface height information can be calculated accordingly.

[0055] The beneficial effects of the present invention are as follows:

[0056] 1. A plane mirror is introduced to complete the calibration work, and the global system parameters are optimized with the plane mirror reprojection results, which improves the flexibility of system calibration and the accuracy of calibration.

[0057] 2. When measuring glass, this device can place the glass to be tested at any position within the test area on the loading platform without fixing it.

[0058] 3. Gray code assisted line shift stripe technology is used as the encoding and expansion scheme. This encoding scheme is completely based on the binary encoding method. Its modulation diagram has the characteristics of spatial separation, and the encoded information is easier to extract.

[0059] 4. Binary stripes have an advantage in projection speed and do not require gamma nonlinear correction of the display screen, making them simple and fast.

[0060] 5. Extract the reflection information of the glass surface through an adaptive method based on clustering algorithm, without manually setting threshold parameters

[0061] 6. The edge error problem of Gray code itself is eliminated by supplementing the complementary Gray code pattern, and the center line extraction algorithm is used to make the line shift stripe coding achieve sub-pixel accuracy.

[0062] 7. The present invention adopts a pure visual solution, which has the characteristics of low cost and large dynamic measurement range, and the measurement accuracy meets the actual industrial measurement needs. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1 Schematic diagram of the structure of the glass surface measurement device;

[0064] Figure 2 Glass surface detection flow chart;

[0065] In the figure, 1 is the lifting platform, 2 is the glass to be tested, 3 is the left camera, 4 is the right camera, and 5 is the display screen. DETAILED DESCRIPTION

[0066] The present invention is further described and illustrated below in conjunction with specific embodiments. The embodiments are merely exemplary of the present disclosure and do not define the scope of limitation. The technical features of each embodiment of the present invention can be combined accordingly without conflicting with each other. The embodiment of the present invention demonstrates the detection process of the three-dimensional surface features of the triangular window glass, and the specific contents are as follows:

[0067] The present invention provides a transparent glass surface shape measuring device based on stereo deflectometry, comprising:

[0068] Display screen: used as a diffuse reflection light source to project structured light information onto the glass surface to be tested;

[0069] Standard plane mirror: used to assist in calibrating the display screen;

[0070] Lifting platform: used to place the glass to be tested and adjust the height of the glass plane to be tested;

[0071] Image acquisition device: including two industrial area array cameras and corresponding lenses, as left and right binocular cameras of stereo deflectometry, set in the direction of reflected light of the glass to be tested, used to capture the modulation image information reflected by the surface of the glass to be tested;

[0072] Fixing device: used to fix the camera and display screen, and adjust the display screen position and field of view;

[0073] Computer: used to generate corresponding surface structured light modulation information, and synchronously collect multiple images of the measured object with modulated structured light through an image acquisition device, and used to calculate the transparent glass gradient information using the measurement method according to the images collected by the image acquisition device at different times and different positions, so as to realize the three-dimensional reconstruction of the surface of the transparent glass;

[0074] The computer is connected to the image acquisition device and the display screen respectively, and the display screen faces the glass to be tested; the angle between the display screen and the horizontal line is about 45°, and the vertical distance between the display screen and the glass to be tested is 20 cm, and it is ensured that the projected image completely covers the glass to be tested; the binocular camera faces the glass to be tested, the angle between the optical axis and the horizontal line is 45°, the baseline length of the binocular camera is 100 mm, and it is ensured that both cameras can completely obtain the image of the glass to be tested.

[0075] The present invention provides a transparent glass surface shape measurement method based on stereo deflectometry, comprising:

[0076] 1) Binocular camera calibration: The binocular cameras synchronously collect 20 chessboard images of different positions in their common field of view, and accurately calibrate the intrinsic parameters of each camera of the image acquisition device according to the Zhang Zhengyou calibration method to ensure that there is an accurate mapping relationship between the image data captured by the camera and the three-dimensional coordinates of the actual object. After calibration, the intrinsic parameter matrix K of the left camera can be obtained. L and its distortion coefficient D L , and the intrinsic parameter matrix K of the right camera R and its distortion coefficient D R :

[0077]

[0078] D L =[k1,k2,k3,p1,p2] L =[-0.0888,-0.1784,-0.5099,0,0]

[0079]

[0080] D R =[k1,k2,k3,p1,p2] R=[-0.0966,-0.2864,-0.0495,0,0]

[0081] So far, the calibration of the left and right cameras has been completed, and their intrinsic parameter matrices and distortion coefficients have been obtained respectively. Their reprojection root mean square errors are 0.056 pixels and 0.076 pixels respectively. The poses of the left and right cameras are fixed, and the pose relationship of the left camera coordinate system to the right camera coordinate system is transformed by the transformation matrix [R LR |t LR ] to describe:

[0082]

[0083] The calculated root mean square error of the binocular camera reprojection is 0.298 pixels. After completing the calibration of the binocular camera, in order to improve the efficiency of stereo vision reconstruction, the binocular camera needs to be corrected. The embodiment of the present invention uses Bouguet epipolar correction, which can maximize the common field of view of the binocular camera and calculate the transformation matrix [R L |T L ] and [R R |T R ] are:

[0084]

[0085] Combining the calculated parameters, a reprojection table for epipolar correction is calculated, and the image is reprojected using the reprojection table;

[0086] 2) Display screen position calibration: The display screen is used to project a chessboard with a resolution of 1920×1080 and a size of 0.2475mm×0.2475mm for each pixel. Generate an 11×9 chessboard calibration pattern on the display screen, place the plane mirror on the stage to ensure that the binocular camera can directly obtain the virtual image of the display screen in the mirror, adjust the plane mirror posture 12 times, and control the binocular camera to synchronously capture the corresponding chessboard virtual image on the display screen. According to the Harris corner detection algorithm and combined with the PnP method, the transformation matrix of the display screen and the left camera can be further calculated. And the transformation matrix of the display screen and the right camera The calculation results are as follows:

[0087]

[0088] 3) Global parameter optimization: The plane mirror is reprojected using the rough calibration results obtained above, and the global calibration parameters are further optimized by calculating the flatness error, distance error and angle error of the reprojection to obtain the optimized calibration parameters;

[0089] 4) Six Gray code patterns and one complementary Gray code pattern are projected in the horizontal and vertical directions respectively. Among them, the six-bit Gray code can distinguish 64 areas. Then, in each area, the area is subdivided by line shift stripes with a width of 3, and the display screen is controlled to project a ten-step line shift stripe pattern and a six-step line shift stripe pattern on the glass in the horizontal and vertical directions respectively, and the binocular camera is synchronously controlled to collect the reflection pattern of the glass surface to be tested at the corresponding time to obtain the modulation map;

[0090] 5) The modulation images captured by the camera are recorded as I0, I1, I2, ..., I n , the gray value of each pixel is the maximum gray value of the corresponding pixel in the modulation image, and I m , normalize the i-th modulation image to the interval (-0.5, 0.5) to obtain Its expression is: Taking the normalized gray value as the feature value, each pixel is clustered by the K-means algorithm, and the cluster centers of 0, 1, 2, and 3 codes are c0, c1, c2, and c3 respectively. The threshold of the gray code modulation information on the glass surface after gray value normalization can be calculated as the average value of the two coding cluster centers c1 and c2 of 1 and 2. After K-means clustering, each pixel in the image is divided into one of the four coding types 0, 1, 2, and 3. Pixels with different codes are marked with different colors, and pixels with the same color belong to the same coding type. The grayscale threshold is reversely normalized to obtain the actual grayscale threshold distribution.

[0091] The Kalman filter is used to smooth the one-dimensional threshold distribution and the Steger algorithm based on the Hessian matrix is ​​used to extract the center line of the line shift fringes; the Hessian matrix is ​​expressed as:

[0092]

[0093] Among them, Z(x, y) is a two-dimensional image, g(x, y) is a two-dimensional Gaussian function,

[0094] Taking (x0, y0) as the base point, let the unit vector obtained by the Hessian matrix be (n x , n y ), and perform a second-order Taylor expansion along the bright stripe section to obtain the point on the section (x0+tn x ,y0+tn y )’s grayscale value:

[0095] z(x0+tn x ,y0+tn y )=Z(x0,y0)+N(rx , r y ) T +NH(x,y)N T / 2

[0096] Where N = (tn x ,tn y ), the image pixel z(x, y) is convolved with its corresponding partial differential form of Gaussian kernel to obtain (r x , r y ), as follows:

[0097]

[0098] according to It can be calculated that:

[0099]

[0100] The exact position of the center point of the bright stripe is (x0+tn x ,y0+tn y ). Then, the relative coding is unfolded by using the complementary Gray code unpacking algorithm to obtain the coding information of the binocular camera in the horizontal direction and the vertical direction respectively; further, the correspondence relationship between the pixel coordinates of the binocular camera and the pixel coordinates of the display screen is calculated by the calibration parameters obtained in step 3);

[0101] 6) Based on the correspondence between the pixel coordinates of the binocular camera and the pixel coordinates of the display screen and the principle of normal consistency, the phase value is selected in the first camera image plane. For any given pixel A1, A1 is compared with the first camera optical center Concatenate to construct the incident vector Here, the camera is treated as a pinhole camera; find the same phase value on the display Pixels Assumptions A point P1 on the surface is the reflection point, and the corresponding reflection vector is obtained. and normal vector Place P1 with the optical center of the second pinhole camera Concatenate to construct the incident vector The intersection pixel B1 with the second camera image plane has a phase value of According to the determined normal vector, The corresponding reflection vector Intersection point with LCD plane Its phase value B is set up along Search until ε is less than the set threshold, and this point is the real reflection point; traverse all pixel points and repeat the above steps to obtain the gradient information of the glass surface.

[0102] According to the obtained gradient information, the local RBF is used to replace the global RBF, that is, RBF-FD, and the multivariate Hermite data, that is, the gradient and point cloud data constraints are combined to complete the accurate reconstruction of the glass surface shape. RBF-FD uses local RBF to replace the global RBF to improve the calculation efficiency. x c The result calculated by the linear operator L is approximately a linear combination of n neighboring nodes in the same template:

[0103]

[0104] Among them, in order to calculate the n neighboring nodes in the template, KD is used to achieve lower time complexity compared to the method of calculating the distance between each two points and then sorting them. Under the conditions that the glass surface is continuous and second-order differentiable, the interpolation of the glass surface can be defined as a weighted combination of weights calculated according to RBF-FD in the case of a first-order derivative operator:

[0105]

[0106] Where N is the number of sampling points, s(x) is the depth information calculated at sampling point x, and W x (xx i ) and W y (xx i ) are the partial derivative operators in the X direction respectively by the RBF-FD method and the Y-direction partial derivative operator In this case, according to the evaluation point x i The calculated weight value of the test point x, α i and β i W x (xx i ) and W y (xx i )’s weighting coefficient;

[0107] Weight W x and W y It can be obtained by RBF-FD weight calculation method. The depth information s(x) also requires the coefficient matrix [α1, ..., α N ] T and [β1, ..., β N ] T To solve, in order to calculate the coefficient matrix, the analytical derivatives of the interpolation equations are matched with the measured gradients to obtain the optimization objective:

[0108]

[0109] By introducing the constraints of multivariate Hermite data, namely gradient and point cloud data, the optimization objective is improved to:

[0110]

[0111] A system of linear equations can be described in matrix form:

[0112]

[0113] in:

[0114]

[0115] W xx , W xy and W yy is the second-order partial derivative operator of weight W, and the weight matrix of the current sampling point is calculated based on the sampling points in the template.

[0116] By solving the above linear equations, we can obtain the coefficient matrix [α1, ..., α N ] T and [β1, ..., β N ] T , then the glass surface height information can be calculated accordingly.

[0117] 7) Repeated measurement experiments are performed on the surface shape of the triangular window glass sample. The surface shape and relative surface height distribution are calculated through active projection and measurement by the experimental system. According to actual measurement requirements, the curvature, smoothness, flatness and other indicators can be further calculated based on the surface shape information.

[0118] The above-mentioned embodiment only expresses one implementation mode of the present invention, and its description is relatively specific and detailed, but it cannot be understood as limiting the scope of the present invention. For ordinary technicians in this field, several modifications and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention.

[0119] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A transparent glass surface shape measuring device based on stereo deflectometry, characterized in that: include: Display screen: used as a diffuse reflection light source to project structured light information onto the glass surface to be tested; Standard plane mirror: used to assist in calibrating the display screen; Lifting platform: used to place the glass to be tested and adjust the height of the glass plane to be tested; Image acquisition device: including two industrial area array cameras and corresponding lenses, as left and right binocular cameras of stereo deflectometry, set in the direction of reflected light of the glass to be tested, used to capture the modulation image information reflected by the surface of the glass to be tested; Fixing device: used to fix the camera and display screen, and adjust the display screen position and field of view; Computer: used to generate corresponding surface structured light modulation information, and synchronously collect multiple images of the measured object with modulated structured light through an image acquisition device, and used to calculate the transparent glass gradient information using the measurement method according to the images collected by the image acquisition device at different times and different positions, so as to realize the three-dimensional reconstruction of the surface of the transparent glass; The computer is connected to the image acquisition device and the display screen respectively, and the display screen faces the glass to be tested.

2. The transparent glass surface shape measuring device based on stereo deflectometry according to claim 1, characterized in that: The display screen has an angle of 30 to 60° with the horizontal line, a vertical distance of 10 to 30 cm with the glass to be tested, and ensures that the projected image completely covers the glass to be tested; the binocular camera faces the glass to be tested, the angle between the optical axis and the horizontal line is 30 to 60°, the baseline length of the binocular camera is 60 to 200 mm, and ensures that both cameras can completely obtain the image of the glass to be tested.

3. A transparent glass surface shape measurement method based on stereo deflectometry, characterized in that: include: S1 Camera calibration: The binocular cameras synchronously collect multiple checkerboard images of different postures in their common field of view, and accurately calibrate the internal parameters of each camera of the image acquisition device according to the Zhang Zhengyou calibration method to ensure that there is an accurate mapping relationship between the image data captured by the camera and the three-dimensional coordinates of the actual object. Then, the precise posture relationship between the left camera and the right camera is calculated and the binocular cameras are corrected by Bouguet to maximize the common field of view of the binocular cameras. S2 display screen calibration: Generate a checkerboard pattern on the display screen, place a plane mirror on the stage, ensure that the binocular camera can directly obtain the virtual image of the display screen in the mirror, adjust the plane mirror posture at least three times, control the binocular camera to synchronously collect the corresponding checkerboard virtual image on the display screen, and solve the position relationship between the binocular camera and the display screen; S3 global parameter optimization: Use the rough calibration results of S1 and S2 to reproject the plane mirror, and further optimize the global calibration parameters by calculating the flatness error, distance error and angle error to obtain the calibration parameters; S4 obtains the modulation image: Gray code-assisted line shift technology and phase shift stripes are used as the surface structured light projection pattern to control the display screen to project onto the glass surface, and synchronously controls the binocular camera to collect the reflection pattern of the glass surface to be measured at the corresponding moment to obtain the modulation image; S5 extracts the upper surface reflection component map: normalizes the gray value of the line shift modulation map obtained in S4, and adaptively selects the threshold through the K-means clustering algorithm, and extracts the upper surface reflection component map by adaptive threshold segmentation; S6 wrapping code information unfolding: Kalman filtering is performed on the upper surface reflection component map obtained in S5; the center line of the line shift stripe in S5 is extracted using the Steger algorithm based on the Hessian matrix; the complementary Gray code unwrapping algorithm is used to unfold the relative code to obtain the coding information of the binocular camera in the horizontal and vertical directions respectively; further, the calibration parameters obtained in S3 can be used to obtain the correspondence between the pixel coordinates of the left and right cameras and the pixel coordinates of the display screen; S7 gradient information acquisition: according to the correspondence between the pixel coordinates of the left and right cameras and the pixel coordinates of the display screen obtained in S6, the gradient information of the glass surface is acquired through a binocular matching method based on phase difference minimization; S8 Gradient integral reconstruction: Based on the gradient information obtained in S7, the local RBF is used to replace the global RBF, namely RBF-FD, and the multivariate Hermite data, namely the gradient, is combined with the constraints of the point cloud data to complete the accurate reconstruction of the glass surface shape.

4. The detection method of the transparent glass surface shape detection device based on stereo deflectometry according to claim 3, characterized in that: In step S5, the specific implementation steps of adaptively selecting the threshold value include: taking the grayscale value of each pixel of the modulation image captured by the camera as the maximum grayscale value of the corresponding pixel in the modulation image, and normalizing it; using the normalized grayscale value as the characteristic value, clustering each pixel through the K-means algorithm; since the grayscale value change process presents four stages, corresponding to the quaternary encoding, that is, K=4; the threshold of the Gray code modulation information on the upper surface of the glass after the grayscale value is normalized can be calculated as the average value of the centers of the two coding clusters with larger grayscale values, and the grayscale threshold is reversely normalized after Kalman smoothing filtering to obtain the actual grayscale threshold distribution.

5. The detection method of the transparent glass surface shape detection device based on stereo deflectometry according to claim 3, characterized in that: In step S6, the Hessian matrix is ​​expressed as: Among them, Z(x,y) is a two-dimensional image, g(x,y) is a two-dimensional Gaussian function, Taking (x0, y0) as the base point, let the unit vector obtained by the Hessian matrix be (n x ,n y ), and perform a second-order Taylor expansion along the bright stripe section to obtain the point on the section (x0+tn x ,y0+tn y )’s grayscale value: z(x0+tn x ,y0+tn y )=Z(x0,y0)+N(r x ,r y ) T +NH(x,y)N T / 2 Where N = (tn x ,tn y ), the image pixel z(x,y) is convolved with its corresponding partial differential form of Gaussian kernel to obtain (r x ,r y ), as follows: according to It can be calculated that: The exact position of the center point of the bright stripe is (x0+tn x ,y0+tn y ).

6. The detection method of the transparent glass surface shape detection device based on stereo deflectometry according to claim 3, characterized in that: The specific steps in step S7 are: 1) Select the phase value in the first camera image plane to be For any given pixel A1, A1 is compared with the first camera optical center Concatenate to construct the incident vector Here, the camera is considered a pinhole camera; 2) Find the same phase value on the display Pixels Assumptions A point P1 on the surface is the reflection point, and the corresponding reflection vector is obtained. and normal vector 3) Align P1 with the optical center of the second pinhole camera Concatenate to construct the incident vector The intersection pixel B1 with the second camera image plane has a phase value of 4) According to the normal vector determined in 2), we can obtain The corresponding reflection vector Intersection point with LCD plane Its phase value B is 5) Set along Search until ε is less than the set threshold, and this point is the true reflection point; 6) Traverse all pixel points and repeat the above steps to obtain the gradient information of the glass surface.

7. The detection method of the transparent glass surface shape detection device based on stereo deflectometry according to claim 3, characterized in that: In step S8, RBF-FD uses local RBF instead of global RBF to improve the calculation efficiency. x c The result calculated by the linear operator L is approximately a linear combination of n neighboring nodes in the same template: Among them, in order to calculate the n neighboring nodes in the template, compared with the method of calculating the distance between each two points and then sorting, the KD tree is used to achieve lower time complexity. Under the conditions that the glass surface is continuous and second-order differentiable, the interpolation of the glass surface can be defined as a weighted combination of weights calculated according to the RBF-FD in the case of a first-order derivative operator: Where N is the number of sampling points, s(x) is the depth information calculated at sampling point x, and W x (xx i ) and W y (xx i ) are the partial derivative operators in the X direction respectively by the RBF-FD method and the Y-direction partial derivative operator In this case, according to the evaluation point x i The calculated weight value of the test point x, α i and β i W x (xx i ) and W y (xx i )’s weighting coefficient; Weight W x and W y It can be obtained by RBF-FD weight calculation method. The depth information s(x) also requires the coefficient matrix [α1,…,α N ] T and [β1,…,β N ] T To solve, in order to calculate the coefficient matrix, the analytical derivatives of the interpolation equations are matched with the measured gradients to obtain the optimization objective:

8. The detection method of the transparent glass surface shape detection device based on stereo deflectometry according to claim 3, characterized in that: In step S8, the multivariate Hermite data, i.e., the constraints of the gradient and the point cloud data, are introduced to improve the optimization objective to: A system of linear equations can be described in matrix form: in: W xx , W xy and W yy is the second-order partial derivative operator of the weight W. The weight matrix of the current sampling point is calculated according to the sampling points in the template. By solving the above linear equations, the coefficient matrix [α1,…,α N ] T and [β1,…,β N ] T , then the glass surface height information can be calculated accordingly.

Images