A high reflectivity sample dynamic three-dimensional measurement method and system

Abstract

The application discloses a high-reflectivity sample dynamic three-dimensional measurement method and system, which can realize high-dynamic-range three-dimensional measurement and reconstruction only by single-frame imaging, and lays a foundation for high-dynamic-range dynamic three-dimensional measurement and reconstruction. The method comprises the following steps: (1) a double-vision measurement system is formed by combining a speckle projection module with an imaging module, speckle projection is generated by a laser transmission diffraction element and projected onto the surface of a measured object, and then imaging is performed on a polarization camera after the surface of the measured object is modulated; (2) the polarization camera is calibrated, and synchronous acquisition is performed by using the double-vision polarization camera; (3) when I=255, it is indicated that the pixel of the point is overexposed, and the point needs to be processed; (4) if the high-reflectivity area is overexposed after polarization enhancement, polarization full-light processing is adopted; (5) if the high-reflectivity area is still overexposed, polarization fusion processing is adopted; (6) polar line correction is performed on the left fusion image and the right strategy fusion image after processing; (7) after being fused into a high-dynamic-range image, pixel points are matched in the right image with the same fusion strategy; and (8) three-dimensional reconstruction is performed.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of optical three-dimensional measurement, in particular to a high-reflectivity sample dynamic three-dimensional measurement method and a high-reflectivity sample dynamic three-dimensional measurement system. BACKGROUND

[0002] The acquisition of the three-dimensional appearance of an object is of great significance, and optical three-dimensional measurement technology has the advantages of non-contact, rapidity, simple system and easy operation, and is the most widely used technology in non-contact measurement. It has a relatively mature application in reverse engineering, cultural heritage protection, deformation detection and industrial part quality monitoring. The speckle projection three-dimensional measurement is the most mature method. This method projects a random speckle pattern onto the surface of the measured object, synchronously collects the speckle patterns at different viewing angles using a camera, and then uses stereo matching technology for matching processing to obtain a parallax map, so as to recover three-dimensional information using the triangulation method. However, due to the limited imaging dynamic range of the camera, the structured light three-dimensional measurement method will cause the camera imaging to be oversaturated when measuring high-reflectivity measured objects such as ceramic bottles, metal shells of airplanes and milled industrial parts. At this time, the use of the traditional structured light three-dimensional measurement method will cause the loss of structured light information at the glare area, resulting in a decrease in reconstruction accuracy and even large-area data voids.

[0003] To solve this problem, in current industrial applications, workers often use the method of spraying degradable powder on the surface of high-reflectivity workpieces to cover the high-light part. However, the disadvantages of this method are obvious. Spraying powder is not only time-consuming and labor-intensive, but also increases the thickness of the measured object surface, and the measurement accuracy will be affected by the uniformity of the sprayed powder. The most mature and effective method is the multiple-exposure method, which refers to a series of imaging under different exposure times of the camera for the same field of view, so as to fuse into an image that avoids image saturation. This method requires a certain number of projections or imaging, which limits its application in dynamic three-dimensional measurement. SUMMARY

[0004] To overcome the defects of the prior art, the technical problem to be solved by the present application is to provide a high-reflectivity sample dynamic three-dimensional measurement method, which can realize high-dynamic-range three-dimensional measurement and reconstruction with only a single frame of imaging, thereby laying a foundation for high-dynamic-range dynamic three-dimensional measurement and reconstruction.

[0005] The technical scheme of the present application is as follows: the high-reflectivity sample dynamic three-dimensional measurement method comprises the following steps:

[0006] (1) a speckle projection module is combined with an imaging module to form a binocular vision measurement system, a laser transmission diffraction element is used to generate a speckle projection onto the surface of the measured object, and the speckle projection is imaged in a polarized camera after being modulated by the surface of the measured object;

[0007] (2) Camera calibration is performed on the polarization camera to obtain camera intrinsic and extrinsic parameters, a tunable laser is turned on to irradiate a diffractive optical element to generate speckle projection on the surface of the measured object, and a binocular polarization camera is used for synchronous collection;

[0008] (3) The high dynamic range image captured by the camera has two regions of overexposure and underexposure, and the pixel gray value of the 8bit camera imaging is 255 at most, so when I = 255, it indicates that the pixel is overexposed and needs to be processed;

[0009] (4) If the high-reflectivity region is overexposed after polarization enhancement, I = 255, then polarization plenoptic processing is adopted;

[0010] (5) If it is still overexposed, I = 255, then polarization fusion processing is adopted, and the pixel point in the optimal polarization channel in the matching window is matched;

[0011] (6) Polar line correction is performed on the left fusion image and the right strategy fusion image after processing;

[0012] (7) After fusion into a high dynamic range image, each pixel point is matched in the right image with the same fusion strategy;

[0013] (8) After matching, a disparity map is obtained, combined with the system calibration parameters, and according to the triangular position relationship between the binocular camera and the measured object, the three-dimensional data is obtained by using the triangular method combined with the camera intrinsic and extrinsic parameter matrix obtained by calibration, and three-dimensional reconstruction is performed.

[0014] The method for synthesizing a high dynamic range speckle image by using laser speckle projection combined with binocular polarization camera imaging according to the application can perform polarization enhancement and polarization plenoptic processing on dark regions, and can adopt window adaptive polarization fusion on over-bright regions and exposure regions, select a suitable polarization channel for the matching window of each pixel point, and realize matching, so that high dynamic range three-dimensional measurement and reconstruction can be realized by only one frame of imaging, thereby laying a foundation for high dynamic range dynamic three-dimensional measurement and reconstruction.

[0015] A high-reflectivity sample dynamic three-dimensional measurement system is also provided, which comprises a projection module and an imaging module, after the polarization camera in the imaging module is calibrated, the projection module generates stable random distribution speckles by irradiating a diffractive optical element with a tunable laser, and projects the speckles onto the surface of the measured object, then the left polarization camera and the right polarization camera in the imaging module synchronously collect the multi-polarization channel speckle patterns modulated by the measured object from different viewing angles, then the left polarization image is input into high dynamic range adaptive fusion to obtain a fusion image and a fusion strategy, the right polarization image is subjected to polar line correction according to the fusion strategy and the left polarization fusion image, and the corrected image is matched pixel by pixel to obtain a disparity map, and a reconstruction result is obtained by a three-dimensional reconstruction method. BRIEF DESCRIPTION OF DRAWINGS

[0016] Figure 1 This is a schematic diagram of the structure of the dynamic three-dimensional measurement system for high reflectivity samples according to the present invention, wherein 1 is an adjustable laser, 2 is a diffractive optical element, 3 is a right polarization camera, 4 is a left polarization camera, and 5 is the object being measured.

[0017] Figure 2 This is a flowchart of the three-dimensional reconstruction method of the present invention.

[0018] Figure 3 This is a flowchart of the high dynamic range adaptive fusion method of the present invention.

[0019] Figure 4 It is a grayscale image of the scene being tested obtained in the experiment of this invention.

[0020] Figure 5 This invention is an experimental fusion of grayscale images.

[0021] Figure 6 This is a schematic diagram of the three-dimensional reconstructed point cloud generated in the experiment of this invention. Detailed Implementation

[0022] like Figure 2 As shown, this dynamic three-dimensional measurement method for high reflectivity samples includes the following steps:

[0023] (1) A binocular vision measurement system is formed by combining a speckle projection module with an imaging module. The speckle projection generated by the laser transmission diffraction element is projected onto the surface of the object being measured and then imaged in the polarization camera after being modulated by the surface of the object being measured.

[0024] (2) Calibrate the polarization camera, obtain the camera's intrinsic and extrinsic parameters, turn on the adjustable laser to irradiate the diffraction optical element to generate speckle projection onto the surface of the object under test, and use a binocular polarization camera to collect data synchronously.

[0025] (3) The high dynamic range images captured by the camera have two areas: overexposure and underexposure. The maximum pixel gray value of the 8-bit camera is 255. When I = 255, it means that the pixel is overexposed and needs to be processed.

[0026] (4) If the high-reflection area is overexposed after polarization enhancement, I = 255, then polarized full-light processing shall be adopted;

[0027] (5) If it is still overexposed, I=255, then polarization fusion processing is used to select the pixel in the optimal polarization channel of the matching window for matching;

[0028] (6) Perform epipolar correction on the processed left fused image and right strategy fused image;

[0029] (7) After fusing into a single high dynamic range image, match each pixel in the right image with the same fusion strategy;

[0030] (8) After matching, the disparity map is obtained, combined with the system calibration parameters, and the three-dimensional data is obtained by using the triangulation method combined with the camera internal and external parameter matrix obtained by calibration according to the triangular position relationship of the binocular camera and the measured object, and three-dimensional reconstruction is performed.

[0031] The method for synthesizing a high dynamic range speckle image by means of laser speckle projection combined with binocular polarization camera imaging according to the application realizes polarization enhancement and polarization plenoptic processing for dark areas, adopts window adaptive polarization fusion for over-bright areas and exposure areas, selects a suitable polarization channel for each pixel point, realizes matching, and therefore high dynamic range three-dimensional measurement and reconstruction can be realized by means of single-frame imaging, thereby laying a foundation for high dynamic range dynamic three-dimensional measurement and reconstruction.

[0032] Preferably, in the step (1), the speckle projection module comprises an adjustable laser and a diffractive optical element, and the imaging module comprises a first polarization camera and a second polarization camera; the polarization camera uses a multi-polarization channel sensor to acquire four polarization speckle images of 0°, 45°, 90° and 135° in a single frame.

[0033] Preferably, in the step (2), the internal and external parameters of the camera are acquired by using Zhang Zhengyou chessboard calibration method for polarization camera calibration, wherein the polarization camera is selected to be in an imaging mode of 0°, 45°, 90° and 135° polarization channels.

[0034] Preferably, in the step (3), the four acquired polarization images are subjected to polarization enhancement processing of a multi-sensor, and the principle is that:

[0035]

[0036] wherein represents superposition of polarization Stokes vectors, I0, I 45 , I 90 , and I 135 respectively represent

[0037] 0°, 45°, 90° and 135° polarization images.

[0038] Preferably, in the step (4), the principle of polarization plenoptic processing is that:

[0039]

[0040] Preferably, in the step (5), the optimization index is a window evaluation gray gradient of a speckle image:

[0041]

[0042] The window size is selected to be 5*5.

[0043] Preferably, in step (6), the polar line correction adopts the Fusiello method.

[0044] Preferably, in step (7), the ZNCC algorithm is used for image matching, and the spatial matching correlation coefficient is:

[0045]

[0046] Where w is the correlation coefficient, the larger w is, the more correlated the left and right images of that pixel are; the correlation window size is selected as: (2w m +1)×(2w m +1); This represents the average grayscale value of all pixels within the reference sub-region in the left image and the matching sub-region in the right image; I L (u,v), I R (u,v) represents the grayscale value of a pixel within the reference sub-region in the left image and the matching sub-region in the right image.

[0047] Preferably, in step (8), the disparity map is matched and combined with the internal parameters obtained from camera calibration, and the three-dimensional information is obtained using triangulation. The specific formula used is as follows:

[0048]

[0049] Where f is the calibrated focal length of the polarization camera, and B is the baseline length between the left and right cameras, in mm. This indicates the coordinates of the imaging center of the right-polarizing camera. The image center coordinates of the left polarizing camera, in pixels; [u L ,v L [x, y, z] represents the pixel coordinates on the left image, in pixels; [x, y, z] represents the pixel coordinates. L ,v L The three-dimensional coordinate information is in mm.

[0050] like Figure 1 As shown, a dynamic three-dimensional measurement system for high reflectivity samples is also provided, which includes a projection module and an imaging module. After the polarization camera in the imaging module is calibrated, the projection module uses an adjustable laser to illuminate the diffraction optical element to generate a stable random distribution speckle pattern, which is projected onto the surface of the object under test. Then, the left polarization camera and the right polarization camera in the imaging module simultaneously acquire the multi-polarization channel speckle pattern modulated by the object under test from different perspectives. Then, the left polarization image is input into a high dynamic range adaptive fusion to obtain a fused image and a fusion strategy. The right polarization image is epipolarized with the left polarization fused image according to the fusion strategy. After correction, the image is matched pixel by pixel to obtain a disparity map. The reconstruction result is obtained through a three-dimensional reconstruction method.

[0051] One specific embodiment of the present application is described in detail below.

[0052] The specific implementation steps of a high dynamic range three-dimensional measurement method based on a polarization camera single-frame imaging are as follows:

[0053] Step 1: A speckle projection module composed of an adjustable laser 1 and a diffractive optical element 2 is combined with an imaging module of a polarization camera 3 and a polarization camera 4 to form a binocular vision measurement system. The measurement principle is that speckle projection is generated by laser transmission of the diffractive element to the surface of the measured object 5, and after modulation by the surface of the measured object, imaging is performed in the polarization camera. The polarization camera uses a multi-polarization channel sensor to collect multi-polarization speckle images at 0°, 45°, 90°, and 135° in a single frame.

[0054] Step 2: For the binocular vision system described above, the following processing is performed: Figure 2 First, camera calibration is performed on the system polarization camera to obtain camera intrinsic and extrinsic parameters. The adjustable laser is turned on to irradiate the diffractive optical element to generate speckle projection onto the surface of the measured object. The binocular polarization camera is used to synchronously collect images. After collection, the images are subjected to step 3 high dynamic range adaptive fusion, step 4 epipolar rectification and stereo matching, and step 5 three-dimensional reconstruction.

[0055] Step 3: The high dynamic range images captured by the camera have two regions of overexposure and underexposure. The maximum pixel gray value of an 8-bit camera image is 255, so when I = 255, it indicates that the pixel is overexposed and needs to be processed. The processing method is as shown in Figure 3 Based on the four polarization images collected by the camera in step 1, the images are subjected to multi-sensor polarization enhancement processing. The principle is as follows:

[0056]

[0057] In the formula, represents the superposition of the polarization Stokes vector;

[0058] Step 4: If the high-reflectance region after increase is overexposed, i.e., I = 255, then polarization plenoptic processing is used. The principle is as follows:

[0059]

[0060] Step 5: If it is still overexposed, i.e., I = 255, then polarization fusion processing is used. The pixel point in the optimal polarization channel in the matching window is selected for matching, and the optimization index is the window evaluation gray scale gradient of the speckle image:

[0061]

[0062] In this example, the window size is selected to be 5*5.

[0063] Step 6, polar correction is performed on the processed left fusion image and right strategy fusion image;

[0064] Step 7, the collected image in the present example is shown in Fig. 6, and after fusion into a high dynamic range image, as shown in Fig. 7, pixel points are matched in the right image with the same fusion strategy, and the matching formula is as follows: Figure 4 Figure 5

[0065]

[0066] wherein w is a correlation coefficient, and the greater w indicates that the left and right images of the pixel point are more relevant; the correlation window size is selected as (2w+1)×(2w+1); m m I L (u,v) and I R (u,v) represent the average gray value of all pixel points in the reference sub-region of the left image and the to-be-matched sub-region of the right image. In the present example, the matching window size is selected as 5*5.

[0067] Step 8, after matching, the disparity map is obtained, combined with the system calibration parameters, and according to the triangular position relationship between the binocular camera and the measured object, the three-dimensional data is obtained by using the triangulation method combined with the camera internal and external parameter matrix obtained by calibration, three-dimensional reconstruction is performed, and the three-dimensional reconstruction result in the present example is shown in Fig. 8. Figure 6

[0068] The above is only a preferred embodiment of the present application, and does not limit the present application in any form. Any simple modification, equivalent change and modification made according to the technical essence of the present application to the above embodiment are still within the protection scope of the technical solution of the present application.​​​​​

Claims

1. A high reflectivity sample dynamic three-dimensional measurement method, characterized by: It comprises the following steps: (1) A speckle projection module is combined with an imaging module to form a binocular vision measurement system, speckle projection is generated by a laser transmission diffraction element to the surface of a measured object, and after modulation by the surface of the measured object, imaging is performed in a polarization camera; (2) Camera calibration is performed on the polarization camera to obtain camera intrinsic and extrinsic parameters, an adjustable laser is turned on to irradiate the diffraction optical element to generate speckle projection to the surface of the measured object, and synchronous acquisition is performed by using the binocular polarization camera; (3) The high dynamic range image captured by the camera has two regions of overexposure and underexposure, the maximum pixel gray value of the 8-bit camera imaging is 255, when I = 255, it indicates that the pixel is overexposed, and needs to be processed; (4) If the high-reflectivity region is overexposed after polarization enhancement, I = 255, then polarization plenoptic processing is adopted; (5) If it is still overexposed, I = 255, then polarization fusion processing is adopted, and the pixel point in the optimal polarization channel in the matching window is matched; (6) Polar line correction is performed on the processed left fusion image and right strategy fusion image; (7) After fusion into a high dynamic range image, each pixel point is matched in the right image with the same fusion strategy; (8) The disparity map is obtained after matching, combined with the system calibration parameters, and the three-dimensional data is obtained by using the triangulation method combined with the camera intrinsic and extrinsic parameter matrices obtained by calibration according to the triangular position relationship between the binocular camera and the measured object, and three-dimensional reconstruction is performed.

2. The high-reflectivity sample dynamic three-dimensional measurement method of claim 1, wherein: In the step (1), the speckle projection module comprises an adjustable laser and a diffraction optical element, and the imaging module comprises a first polarization camera and a second polarization camera; the polarization camera uses a multi-polarization channel sensor to collect four polarization speckle images of 0°, 45°, 90° and 135° in a single frame.

3. The high-reflectivity sample dynamic three-dimensional measurement method of claim 2, wherein: In the step (2), the internal and external parameters of the camera are obtained by using Zhang Zhengyou chessboard calibration method for polarization camera calibration, wherein the polarization camera collection is selected as 0°, 45°, 90° and 135° polarization channel imaging mode. In the step (3), the four collected polarization images are subjected to polarization enhancement processing of multiple sensors, and the principle is as follows:

4. The high-reflectivity sample dynamic three-dimensional measurement method of claim 3, wherein: In the step (4), the principle of polarization plenoptic processing is as follows: wherein represents the superposition of polarized Stokes vectors, I0, I 45 , 90 , and I 135 represent 0°, 45°, 90°, and 135° polarized images, respectively.

5. The high-reflectivity sample dynamic three-dimensional measurement method of claim 4, wherein: In the step (5), the optimization index is the window evaluation gray gradient of the speckle image:

6. The high-reflectivity sample dynamic three-dimensional measurement method of claim 5, wherein: The window size is selected as 5*5. In the step (6), the polar line correction adopts Fusiello method.

7. The high-reflectivity sample dynamic three-dimensional measurement method of claim 6, wherein: In the step (7), the image matching is performed by using ZNCC algorithm, and the spatial matching correlation coefficient is as follows:

8. The high-reflectivity sample dynamic three-dimensional measurement method of claim 7, wherein: In the step (8), the three-dimensional information is obtained by using the triangular method with the internal parameters obtained by matching the disparity map and the camera calibration, and the specific formula used is as follows: Where w is the correlation coefficient, the larger w is, the more correlated the left and right images of that pixel are; the correlation window size is selected as: (2w m +1)×(2w m +1); This represents the average grayscale value of all pixels within the reference sub-region in the left image and the matching sub-region in the right image; I L (u,v), I R (u,v) represents the grayscale value of a pixel within the reference sub-region in the left image and the matching sub-region in the right image.

9. The high-reflectivity sample dynamic three-dimensional measurement method of claim 8, wherein: It comprises: where f is the focal length of the polarized camera obtained by calibration, and B is the baseline length between the left and right cameras, in mm; represents the imaging center coordinates of the right polarized camera, represents the imaging center coordinates of the left polarized camera, in pixels; L L represents the pixel coordinates on the left imaging image, in pixels; L L represents the three-dimensional coordinate information of the pixel, in mm.​​ 10. The system for high reflectance sample dynamic three-dimensional measurement method of claim 1, wherein: ​ The projection module and the imaging module, after the polarization camera in the imaging module is calibrated, the projection module irradiates the diffractive optical element by the adjustable laser to generate stable random distribution speckle, which is projected to the surface of the measured object, then the left polarization camera and the right polarization camera in the imaging module synchronously collect, from different visual angles, the multi-polarization channel speckle patterns modulated by the measured object, then the left polarization image is input into the high dynamic range adaptive fusion to obtain the fusion image and the fusion strategy, the right polarization image is subjected to polar line correction according to the fusion strategy and the left polarization fusion image, the corrected image is matched pixel by pixel to obtain the parallax map, and the reconstruction result is obtained through the three-dimensional reconstruction method.

Images