A polarization-based high dynamic range surface three-dimensional reconstruction method

Abstract

For high dynamic range surface, a simple and effective three-dimensional reconstruction method is proposed, which combines the features of non-overexposure area in polarized image and non-polarized image, aiming to improve the reconstruction quality of high dynamic range surface object. The non-overexposure area in non-polarized image usually contains complete fringe information and is less affected by high light interference. The best fringe area of different polarized image groups is extracted, and then the best fringe information of different polarization directions is gradually replaced by the incomplete and unclear fringe information in the non-polarized image. Finally, the overexposure area in the non-polarized image is replaced by complete fringe information, so that the global complete fringe information can be obtained. Experimental results show that when the object with strong reflective surface is reconstructed, the negative influence of overexposure area on absolute phase calculation and three-dimensional reconstruction can be effectively solved.

Description

Technical Field

[0001] This invention relates to the field of optical three-dimensional measurement, and more particularly to the recovery of stripe information in overexposed areas during three-dimensional reconstruction of high dynamic range surface objects. Background Technology

[0002] Given its advantages of high speed, non-contact operation, and simple hardware configuration, visual 3D shape measurement is widely used in various fields, such as defect detection, industrial parts quality control, and reverse engineering. Existing structured light methods are generally suitable for measuring the topography of diffuse reflective surfaces. When measuring high dynamic range (HDR) surfaces, the limitations of the camera's intensity response range make it difficult to balance the high dynamic range and low response areas in a single exposure. Handling overexposed areas has always been a challenge in 3D shape reconstruction of HDR surface objects. Overexposure typically occurs in HDR scenes, causing the loss of detail in certain areas of the image, severely affecting the accuracy of the reconstruction results. Traditional reconstruction techniques often fail to effectively recover the geometric features of these areas, thus limiting their applicability in practical applications. Therefore, there is an urgent need to develop a method for 3D reconstruction of HDR surface objects. Summary of the Invention

[0003] This invention proposes a simple method for reconstructing surface objects with high dynamic range. By combining the features of non-highlight regions in polarized and unpolarized images, it overcomes problems such as absolute phase calculation errors and point cloud loss caused by overexposed areas.

[0004] To achieve the above objectives, the present invention employs the following technical solution: a priori-guided repair method for strong highlights in striped projection contour images, comprising the following steps:

[0005] S1: Take unpolarized images and several sets of polarized images using a calibrated system;

[0006] S2: Dephase the unpolarized image and identify areas where dephase demodulation was unsuccessful;

[0007] S3: Further identify unsuccessful dephase-dissociation regions in brighter polarized images for unpolarized images;

[0008] S4: For areas in the brighter polarized image where phase unresolved is unsuccessful, extract the phase information of the corresponding area in the darker polarized image. If the brighter polarized image successfully resolves the entire area in the unpolarized image where phase unresolved is unsuccessful, then directly perform phase fusion.

[0009] S5: Fuse the phase information of the successfully deconstructed regions of different polarization images into the unsuccessfully deconstructed regions of the unpolarization image;

[0010] S6: Calculate the absolute phase of the fused phase information and convert it into a 3D point cloud.

[0011] The common method used is to obtain the stripe information of the overexposed area through multiple exposures. However, the present invention uses a polarizer placed in front of the camera and projector. By rotating the polarizer, the stripe information of the overexposed area can be obtained. The image acquisition method of the present invention is much more convenient than the multiple exposure method.

[0012] This invention only requires determining the projected light intensity and camera exposure time once before acquiring an image of an object, and can be achieved without complex calculations and cumbersome operating steps.

[0013] The accuracy of the method proposed in this invention is related to the accuracy of the system used. The higher the accuracy of the system, the higher the accuracy of the three-dimensional reconstruction result of the method proposed in this invention. This method will not affect the accuracy of the original system.

[0014] Traditional multi-exposure fusion techniques rely on sophisticated and complex algorithms to fuse relevant phase information together, while the method proposed in this invention only requires a second substitution function to recover the phase information of overexposed areas.

[0015] Through experimental process and results, it can be seen that the total number of images required by the present invention is less than that required by the common multi-exposure fusion method. Although the method proposed in this invention uses fewer images, the absolute phase map and three-dimensional point cloud map obtained by the method of this invention show that this method can also restore the geometric features of overexposed areas. Attached Figure Description

[0016] Figure 1 This is a flowchart of the prior-guided repair method for strong highlights in striped projection contour images according to the present invention.

[0017] Figure 2 The image shows the overexposed area of ​​an unpolarized image and the non-overexposed portion of a polarized image in that area.

[0018] Figure 3 This is a four-step phase shift diagram.

[0019] Figure 4 This is a non-polarized image containing stripe information, along with its calculated absolute phase map and 3D point cloud map.

[0020] Figure 5 This describes the general process of phase information fusion.

[0021] Figure 6 The absolute phase map and three-dimensional point cloud map are obtained by the method proposed in this invention.

[0022] Figure 7 This is a comparison diagram between the traditional method and the method proposed in this invention. Detailed Implementation

[0023] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0024] like Figure 1 As shown, a simple method for reconstructing high dynamic range surface objects includes the following steps:

[0025] S1: Take unpolarized images and several sets of polarized images using a calibrated system. The specific method is as follows:

[0026] The object under test and the system calibrated according to this invention are positioned on the experimental platform. After fixing the position, a sinusoidal fringe pattern is projected by a projector, projecting the blue fringe pattern onto the surface of the object under test. With the camera exposure time fixed (t = 20000 μs), the standard for judging light intensity is that the fringe information of the non-overexposed areas of the object can be accurately and completely acquired by the camera. After determining the light intensity, a set of unpolarized fringe images is taken. Subsequently, with the camera exposure time, projection brightness, and other object positions unchanged, a polarizer is placed in front of the camera and projector. When taking brighter polarized image sets, the two polarization angles need to be fixed at 0° for acquisition; while when taking darker sets, the polarizer in front of the camera is rotated. The number of polarized image sets is determined to ensure that each set, when combined, can completely supplement the fringe information of the overexposed areas in the unpolarized image. Figure 2 As shown, the non-overexposed area is R. F Overexposed area is region R G .

[0027] S2: Dephase the unpolarized image and identify areas where dephase resolution was unsuccessful.

[0028] S2.1: This invention uses a three-frequency four-step phase-shifting method for three-dimensional reconstruction, and the intensity of its projected pattern is shown in formula (1):

[0029]

[0030] In the formula, A(x,y) is the background intensity of the striped image, and B(x,y) is the modulation amplitude. Here, n is the phase value, n is the phase shift index, N is the phase shift step number, and (x, y) is the position of the pixel in the projected image. In a four-step phase shift, n = 0, 1, 2, 3, and N = 4. Therefore, the phase shift amounts of the four stripe images are 0, π / 2, π, and 3π / 2, respectively. Figure 3 As shown, its strength is as indicated by formula (2):

[0031]

[0032] Based on the striped projection image acquired by the camera, its relevant values ​​can be solved using the following formula, as shown in formula (3):

[0033] From formulas (4) and (5), we can obtain the results in the four-step phase shift. As shown in formula (4):

[0034]

[0035] Solving the above formula yields The value range is from -π to π. Unwrapping the wrapped phase allows us to obtain the globally unambiguous absolute phase. The phase expansion algorithm used in this invention is the multi-frequency phase expansion method. From the above formula, the phase information of the three frequencies is obtained as follows: That is, calculate the phase difference between the first group and the second group, as shown in formula (5):

[0036]

[0037] Similarly, the phase difference between the second and third groups is shown in formula (6):

[0038]

[0039] Combining formulas (5) and (6), the final phase difference can be obtained, as shown in formula (7):

[0040]

[0041] Phase unrolling algorithms increase the measurement range by reducing their signal-to-noise ratio, thus synthesizing the phase map. Used only as a reference phase to assist in phase unwrapping, the synthesized wavelength λ ed With a smaller wavelength λ d A scaling factor is defined and applied to the phase at the beat frequency to determine the fringe order, ultimately yielding the final absolute phase. As shown in formula (8):

[0042]

[0043] After calculation, an absolute phase diagram can be obtained. Experiments were conducted on the metal plate of this invention, and its unpolarized image is shown below. Figure 4 As shown in (a) above, the absolute phase diagram is as follows: Figure 4 As shown in (b) above, the final 3D point cloud map is as follows: Figure 4 As shown in (c), the range of unexposed regions can be determined by the absolute phase diagram.

[0044] S3: Further identify unsuccessful dephase-dissociation regions in brighter polarized images by means of the following method;

[0045] Similar to step S2, the unresolved areas of the unpolarized image are further identified in a brighter polarized image. A polarizer is added to reduce the area of ​​the overexposed region, thus obtaining a portion of the correct and complete stripe information of the unresolved areas of the unpolarized image.

[0046] S4: For areas in the brighter polarized image where phase unresolved failure occurs, extract the phase information of the corresponding areas in the darker polarized image. If the brighter polarized image successfully resolves the entire area in the unpolarized image where phase unresolved failure occurs, then directly perform phase fusion. The specific method is as follows:

[0047] For areas in the brighter polarized image where phase unresolved is not successfully resolved, the phase information of the corresponding areas is extracted from the darker polarized image. Then, the phase information of the successfully resolved areas is fused together. If the phase information of the overexposed areas of the unpolarized image has been fully supplemented in the brighter polarized image, phase fusion is performed directly. The specific fusion method will be explained in detail in S5.

[0048] S5: The phase information of the successfully deconstructed regions of different polarization images is fused into the unsuccessfully deconstructed regions of the unpolarization image. The specific method is as follows:

[0049] Images captured by a camera cannot be directly used to determine the exact extent of overexposure, nor can the loss of fringe information be directly identified. Overexposure areas vary slightly depending on the frequency and phase shift of the image. Therefore, the best approach is to determine the unresolved areas based on the final resolution result. This method is also used in brighter polarized images, but the determination area is the unresolved region in the image taken without a polarizer. For darker polarized images, based on the brighter polarized images, smaller, successfully resolved areas are selected sequentially from larger areas until no unresolved areas remain in the object region. Finally, all successfully resolved areas are merged into a single image set. Phase fusion is a step-by-step process of replacing phase information in overexposure areas. To better understand this, let's assume that the final phase-incorrect area in a set of unpolarized images of an object is... The region with incorrect phase in the final brighter polarized image group is Darker polarized image groups need to be in the region The final phase information is correct, and then the darker polarization image group region is... The phase information of the brighter polarized image group is replaced with the phase information of this region. Finally, the fused brighter polarized image group region is... The phase information of the unpolarized image group is replaced with the phase information of this region, so that a complete global phase information can be obtained. If I1(x,y) represents the phase information of the first image at position (x,y) and I2(x,y) represents the pixel value of the second image at the same position, then the pixel information of a region can be replaced by the formula, as shown in formula (9):

[0050]

[0051] Where f is a restoration model, and region R represents the unresolved phase information region corresponding to the unpolarized image. The phase information of the first image pixel is used to replace the phase information of the second image pixel. Through these two replacements, the phase information of the unresolved phase information region is replaced with the correct phase information, thus obtaining the complete phase information. The general process is as follows: Figure 5 As shown.

[0052] S6: Calculate the absolute phase of the fused phase information and convert it into a 3D point cloud.

[0053] The fused phase information is processed through step S2 to obtain the absolute phase of the globally deconstructed phase, as shown below. Figure 6 As shown in (a), the absolute phase is obtained through the three-frequency four-step method, from which the phase value of each pixel can be obtained. Then, the depth value of each pixel is calculated using the known camera parameters. Next, the pixel coordinates are converted into three-dimensional world coordinates according to the camera's intrinsic parameters. Finally, a three-dimensional point cloud map is obtained from this correct absolute phase information, as shown in (a). Figure 6 As shown in (b), to verify that the method proposed in this invention does not affect the accuracy of the original system, this invention uses traditional methods to reconstruct and compare the fused region at different shooting angles, as follows: Figure 7 As shown, Figure 7 (a) in the figure is a point cloud map obtained by reconstructing the fused region of the present invention using traditional methods. Figure 7 (b) in the figure is a point cloud image obtained using the method of the present invention. Figure 7 (c) highlights the area without stripe information, which corresponds to the area fused by the present invention. Figure 7 (d) in the figure shows a comparison of the same area in the two point cloud images. Although there are slight differences in details due to different shooting angles, the reconstruction results of the more obvious scratches on the metal plate show that the method proposed in this invention has almost no impact on the accuracy of the original system.

Claims

1. A high dynamic range surface 3D reconstruction method based on polarization, characterized in that, Includes the following steps: S1: The calibrated system captures an unpolarized image and several sets of polarized images, specifically: The object to be measured and the 3D measurement system are positioned on the experimental platform. After fixing the position, a sinusoidal fringe pattern is projected onto the surface of the object. Under the premise that the camera exposure time remains constant, the standard for judging the light intensity is that the fringe information of the non-overexposed area of ​​the object can be accurately and completely acquired by the camera. After determining the light intensity, a set of unpolarized fringe images are taken. Subsequently, under the premise that the camera exposure time, projection brightness and other object positions remain unchanged, a polarizer is placed in front of the camera and the projector. When taking the brighter polarized image set, the two polarization angles need to be fixed at 0° for acquisition; while when taking the darker set, the polarizer in front of the camera is rotated. The number of polarized image sets is determined to ensure that the combination of each set can completely fill the fringe information of the overexposed area in the unpolarized image. The total number of image sets is 3. S2: Dephase the unpolarized image and identify areas where dephase demodulation was unsuccessful; S3: Further identify unsuccessful dephase-dissociation regions in brighter polarized images for unpolarized images; S4: For areas in the brighter polarized image where phase unresolved is unsuccessful, extract the phase information of the corresponding area in the darker polarized image. If the brighter polarized image successfully resolves the entire area in the unpolarized image where phase unresolved is unsuccessful, then directly perform phase fusion. S5: The phase information of successfully deconstructed regions of different polarization images is fused into the undeconstructed regions of the unpolarization image, specifically: When performing phase fusion, the phase information of the region corresponding to the unpolarized image is replaced with the phase information of the region corresponding to the polarized image. If I1(x,y) represents the phase information of the first image at position (x,y) and I2(x,y) represents the pixel value of the second image at the same position, then formula (1) represents the replacement of the pixel information of a region. Where f is a repair model, and region R represents the unresolved phase information region corresponding to the unpolarized image. The phase information of the first image pixel is replaced with the phase information of the second image pixel. Through two replacements, the phase information of the unresolved phase information region is replaced with the correct phase information, thus obtaining the complete phase information. S6: Calculate the absolute phase of the fused phase information and convert it into a 3D point cloud.

2. The polarization-based high dynamic range surface 3D reconstruction method according to claim 1, characterized in that, The method for deconverting an unpolarized image and identifying regions where deconverting was unsuccessful is as follows: (1) Three-dimensional reconstruction is carried out using the three-frequency four-step phase shift method. In the fringe projection profile measurement system, the generated image containing sinusoidal fringe information is imported into the projector. Then, the grating pattern of the projector is projected onto the surface of the measured object and the fringe information of the surface of the measured object is obtained through the camera. Finally, the data is processed by the MATLAB program to obtain the phase information. (2) After obtaining the phase information, the wrapped phase is unwrapped to obtain the globally unambiguous absolute phase. The phase expansion algorithm used is the multi-frequency phase expansion method, which yields phase information for three frequencies. The phase difference between the first and second groups and the phase difference between the second and third groups are calculated. The final phase difference is then obtained from these two phase differences. This phase unfolding algorithm increases the measurement range, reduces the signal-to-noise ratio of the phase data, and synthesizes the final phase difference. Used only as a reference phase to assist in phase unwrapping, the synthesized wavelength λ ed With a smaller wavelength λ d A scaling factor is defined and applied to the phase at the beat frequency to determine the fringe order, ultimately yielding the final absolute phase. (3) Analyze the phase resolution results of the unpolarized image and determine the areas where the absolute phase is incorrect based on the areas where the phase is lost, that is, determine the areas where phase resolution is unsuccessful in the absolute phase diagram.

3. The polarization-based high dynamic range surface 3D reconstruction method according to claim 1, characterized in that, The method for further identifying unresolved regions in a brighter polarized image from unpolarized images is as follows: For areas where unresolved phase separation was unsuccessful in the unpolarized image, further identification of these areas was performed in a brighter polarized image. A polarizer was then added to reduce the area of ​​overexposed regions, resulting in a portion of correct and complete fringe information from the unresolved phase separation areas of the unpolarized image.

4. The high dynamic range surface 3D reconstruction method based on polarization according to claim 1, characterized in that, The method for extracting phase information of corresponding regions in a darker polarized image from regions where phase unresolved in a brighter polarized image is not specified. If phase unresolved successfully occurs in the entire region where phase unresolved in the unpolarized image, the method for direct phase fusion is as follows: For areas in the brighter polarized image where phase unresolved is not successfully resolved, the phase information of the corresponding area is extracted from the darker polarized image. Then, the phase information of each group of successfully resolved areas is fused together step by step. If the brighter polarized image successfully resolves the entire area in the unpolarized image where phase unresolved is not successful, that is, if the fringe information of the brighter polarized image can complete the phase information of the overexposed area in the unpolarized image, then phase fusion can be performed directly if this condition is met.

5. The high dynamic range surface 3D reconstruction method based on polarization according to claim 1, characterized in that, The method for calculating the absolute phase from the fused phase information and converting it into a three-dimensional point cloud is as follows: The absolute phase is obtained through a three-frequency four-step method, the phase value of each pixel is obtained, the depth value of each pixel is calculated using known camera parameters, and then the pixel coordinates are converted into three-dimensional world coordinates based on the camera's intrinsic parameters, ultimately generating a three-dimensional point cloud map.

Images