Single-frame phase unwrapping method for fringe images

Abstract

The application discloses a single-frame phase unwrapping method of a fringe image, and the method comprises the following steps: collecting a color fringe image; performing RGB three-channel separation on the color fringe image to obtain three separated images; then fusing the three separated images according to a preset fusion algorithm to obtain a composite image; finally inputting the composite image into a trained neural network capable of phase unwrapping to perform phase unwrapping; wherein the preset fusion algorithm is to obtain the composite image by weighting and summing the pixel values of the pixel points at the same position of the three separated images with the scalar weight value obtained by fusing the luminance value and the contrast value of the separated image as a coefficient. The single-frame phase unwrapping method improves the dynamic range of the image, guarantees the real-time performance, has stability and accuracy in global phase unwrapping, and effectively improves the phase unwrapping quality.

Description

Technical Field

[0001] The present invention relates to the field of optical measurement technology, and in particular to a single-frame dephasing method for a fringe image. Background Art

[0002] The projection and capture of fringe patterns typically involve sophisticated optical systems and complex image processing algorithms. In optical metrology, such as digital fringe projection and fringe photography, the clarity and contrast of the fringe pattern directly impact the accuracy and precision of the measurement results. However, due to variations in surface reflectivity, ambient light interference, and the performance limitations of cameras and projectors, overexposure and underexposure are unavoidable. If the surface of the object being measured has a high dynamic range (combining highly reflective and dark areas), it can be challenging to find the appropriate exposure that maintains sharp fringes in both reflective and dark areas. If the camera exposure is too low, the fringes in dark areas will be unclear; conversely, if the camera exposure is too high, the fringes in reflective areas will be unclear. This phenomenon not only leads to loss of image detail but can also introduce noise, compromising the accuracy of subsequent image processing and analysis. Therefore, high-quality deinterlacing of the fringe pattern in a single frame is crucial for achieving 3D measurement of objects in high-speed motion.

[0003] Traditional Fourier methods convert grating fringe images into the frequency domain, filter out zero-frequency and high-frequency components to eliminate background and noise, and obtain the fundamental frequency component. Through inverse transformation and complex operations, the parcel phase information is extracted from the fringe pattern, enabling single-frame dephasing. However, this method is a global transformation and cannot directly provide local temporal (or spatial) information about the signal. Therefore, when dealing with overexposure or darkening, this method cannot accurately reflect the phase changes in these local areas, resulting in localized distortion and error accumulation of phase information.

[0004] To address this issue, existing technologies project stripes of varying intensities onto the surface of the object being measured, take multiple shots, and fuse appropriate images to obtain a complete stripe image of the object, enabling three-dimensional reconstruction of the object. While this method improves the dynamic range of the image, it sacrifices real-time performance.

[0005] In addition, there is also a single-frame grating fringe analysis method based on a deep learning framework in the existing technology. The idea of ​​this method is to use only one fringe image as input and use a neural network to simulate the phase demodulation process of the time domain phase shift method. Although this method can dephase a single-frame phase image in real time, it only takes one image to be dephased, and there is currently no suitable method to obtain a single-frame high dynamic range image through a single camera shot, which limits the dephase quality of this method. Summary of the Invention

[0006] In order to solve the above technical problems, the present invention provides a single-frame dephasing method for fringe images, which improves the dynamic range of the image, ensures real-time performance, and has the stability and accuracy of dephasing in a global range, effectively improving the dephasing quality.

[0007] To this end, the technical solution is as follows:

[0008] A single-frame dephasing method for a fringe image comprises the following steps:

[0009] S1, collect a color stripe image;

[0010] S2, performing RGB three-channel separation on the color stripe image to obtain three separated images;

[0011] S3, fusing the three separated images according to a preset fusion algorithm to obtain a composite image;

[0012] S4, inputting the composite image obtained in S3 into a trained neural network capable of dephasing for dephasing;

[0013] The preset fusion algorithm in S3 includes the following steps:

[0014] S3.1. By using the preset weight coefficients of the separated images, the brightness and contrast values ​​of individual pixels in a single separated image are combined to obtain a scalar weight value for that pixel. After traversing all pixels in the separated image, a scalar weight matrix for the separated image is obtained. The scalar weight value is the comprehensive weight of the pixel and remains unchanged during the coordinate transformation process.

[0015] S3.2, repeat S3.1 to obtain the scalar weight matrices of the three separated images respectively;

[0016] S3.3. Normalize the scalar weight matrices of the three separated images obtained in S3.2 so that the sum of the scalar weight values ​​at the same pixel position in the three separated images is 1;

[0017] S3.4. The pixel values ​​of the pixel points at the same position in the three separated images are weightedly summed using the corresponding scalar weight values ​​in the normalized scalar weight matrix as coefficients to obtain the fused pixel value of the point. After traversing all the pixel points, the composite image to be deconstructed is obtained.

[0018] Furthermore, the contrast value of a single pixel point in S3.1 is obtained by performing Laplace convolution on the pixel value of the point.

[0019] Furthermore, the calculation method of the brightness value of a single pixel in S3.1 is:

[0020]

[0021] Where x and y represent the horizontal and vertical coordinates of a single pixel in the image coordinate system, E(x, y) represents the brightness value of the pixel at (x, y), I(x, y) represents the pixel value of the pixel at (x, y), and I mean represents the average grayscale value of the image, σ is the standard deviation of the Gaussian curve; exp() represents the exponential function with the natural constant e as the base.

[0022] Furthermore, the method for fusing the brightness value and contrast value of a single pixel in S3.1 is:

[0023] W(x,y)=S c C(x,y)×S e E(%y)

[0024] Where x and y represent the horizontal and vertical coordinates of a single pixel in the image coordinate system, W(x, y) represents the scalar weight value of the pixel (x, y), C(x, y) represents the contrast value of the pixel (x, y); E(x, y) represents the brightness value of the pixel (x, y); S c Represents the weight coefficient of the contrast value of the separated image, S e Represents the weight coefficient for separating the brightness values ​​of the image.

[0025] Furthermore, the neural network is a UNet neural network, a convolutional neural network or a generative adversarial network.

[0026] Furthermore, the UNet neural network includes n downsampling convolution layers and n upsampling convolution layers; the nth downsampling convolution layer is jump-connected to the n-i+1th upsampling convolution layer, 1≤i≤n; each downsampling convolution layer is followed by a maximum pooling layer; each upsampling convolution layer is preceded by a transposed convolution layer; the last upsampling convolution layer is connected to a convolution layer with a convolution kernel size of 1×1, and outputs the absolute phase solution result.

[0027] Furthermore, in order to obtain stripe features of different scales, each downsampling convolution layer and upsampling convolution layer includes three convolution operation layers, and the convolution kernels of the three convolution operation layers are not equal; three convolution operations are also provided before the upsampling operation of the first upsampling convolution layer.

[0028] The present invention first separates three images from a single color stripe image. These images are then fused using a pre-defined image fusion method to optimize overexposed and underexposed areas in the original image, resulting in a composite image with high dynamic range. Finally, the composite image is fed into a neural network for phase analysis, yielding an accurate phase map. This solves the existing problem of a lack of suitable methods for capturing a single-frame high dynamic range image and performing accurate phase analysis using a single camera shot.

[0029] The beneficial effects of the present application are:

[0030] 1. By acquiring a single frame of color fringe image through the camera, after RGB channel separation and preset fusion algorithm processing, a high dynamic range fringe image can be directly obtained, overcoming the problem of lack of real-time caused by multiple camera shooting in the prior art to obtain high dynamic range fringe image;

[0031] 2. Using the composite image processed by the preset fusion algorithm as the input of the neural network for phase unwrapping, the defects of traditional deep learning methods in input image quality are compensated;

[0032] 3. By constructing and training the improved UNet neural network, the deep learning method can retain more detailed information during phase unwrapping, and the phase unwrapping accuracy is higher. BRIEF DESCRIPTION OF DRAWINGS

[0033] Figure 1 is a schematic diagram of the method flow of the present application;

[0034] Figure 2a is a schematic diagram of the R channel separated image separated from the RGB three channels;

[0035] Figure 2b is a schematic diagram of the G channel separated image separated from the RGB three channels;

[0036] Figure 2c is a schematic diagram of the B channel separated image separated from the RGB three channels;

[0037] Figure 2d is a schematic diagram of the RGB three channel separated image fusion composite image;

[0038] Figure 3 is a structural schematic diagram of the UNet neural network used in the phase unwrapping of the method of the present application;

[0039] Figure 4 is an experimental comparison schematic diagram of the phase unwrapping results of the method of the present application

[0040] Figure 5 is a comparison of Figure 4 is an experimental comparison schematic diagram of the phase unwrapping results of the same picture using Fourier algorithm in the prior art;

[0041] Figure 6 is a comparison of Figure 4 is an experimental comparison schematic diagram of the phase unwrapping results of the same picture using traditional deep learning in the prior art. DETAILED DESCRIPTION

[0042] The technical solutions of the present application are described in detail below in combination with the drawings and specific embodiments.

[0043] A single-frame dephasing method for fringe images, such as Figure 1 As shown, the following steps are included:

[0044] S1, collect a color stripe image;

[0045] S2, performing RGB three-channel separation on the color stripe image to obtain three separated images;

[0046] S3, fusing the three separated images according to a preset fusion algorithm to obtain a composite image;

[0047] S4, inputting the composite image obtained in S3 into a trained neural network capable of dephasing for dephasing;

[0048] The preset fusion algorithm in S3 includes the following steps:

[0049] S3.1. By using the preset weight coefficients of the separated images, the brightness and contrast values ​​of individual pixels in a single separated image are combined to obtain a scalar weight value for that pixel. After traversing all pixels in the separated image, a scalar weight matrix for the separated image is obtained. The scalar weight value is the comprehensive weight of the pixel and remains unchanged during the coordinate transformation process.

[0050] S3.2, repeat S3.1 to obtain the scalar weight matrices of the three separated images respectively;

[0051] S3.3. Normalize the scalar weight matrices of the three separated images obtained in S3.2 so that the sum of the scalar weight values ​​at the same pixel position in the three separated images is 1;

[0052] S3.4. The pixel values ​​of the pixel points at the same position in the three separated images are weightedly summed using the corresponding scalar weight values ​​in the normalized scalar weight matrix as coefficients to obtain the fused pixel value of the point. After traversing all the pixel points, the composite image to be deconstructed is obtained.

[0053] In the specific implementation of S1, a 3-CMOS camera can be used to capture color images. Before using the 3-CMOS camera to capture images, different gain coefficients are set for the three RGB channels of the camera.

[0054] The three-channel separation image separated from the color image is a striped grayscale image with different exposure levels. Due to underexposure and overexposure, there will be too bright and too dark areas in the image. Therefore, from the perspective of fusion effect, the scalar weight value in S3.1 needs to be obtained by combining the contrast value and brightness value of a single pixel. The contrast value of a single pixel is obtained by performing Laplace convolution on the pixel value of the point; the brightness value of a single pixel is calculated as follows:

[0055]

[0056] Where x and y represent the horizontal and vertical coordinates of a single pixel in the image coordinate system, E(x, y) represents the brightness value of the pixel at (x, y), I(x, y) represents the pixel value of the pixel at (x, y), and I mean represents the average grayscale value of the image, σ is the standard deviation of the Gaussian curve; exp() represents the exponential function with the natural constant e as the base.

[0057] In specific implementation, the method for fusing the brightness value and contrast value of a single pixel in S3.1 is as follows:

[0058] W(x,y)=SC(r,y)×SE(%y)

[0059] Where x and y represent the horizontal and vertical coordinates of a single pixel in the image coordinate system, W(x, y) represents the scalar weight value of the pixel (x, y), C(x, y) represents the contrast value of the pixel (x, y); E(x, y) represents the brightness value of the pixel (x, y); S c Represents the weight coefficient of the contrast value of the separated image, S e The weight coefficient representing the brightness value of the separated image. Those skilled in the art may also select other existing fusion methods in combination with achieving the purpose.

[0060] In specific implementation, the neural network for dephasing can be a UNet neural network, a convolutional neural network or a generative adversarial network.

[0061] Specifically, taking the UNet neural network model as an example, it includes n downsampling convolution layers and n upsampling convolution layers; the nth downsampling convolution layer is jump-connected to the n-i+1th upsampling convolution layer, 1≤i≤n; each downsampling convolution layer is followed by a maximum pooling layer; each upsampling convolution layer is preceded by a transposed convolution layer; the last upsampling convolution layer is connected to a convolution layer with a convolution kernel size of 1×1, and outputs the absolute phase solution result.

[0062] In more detail, in order to obtain stripe features of different scales, each downsampling convolution layer and upsampling convolution layer includes three convolution operation layers, and the convolution kernels of the three convolution operation layers are not equal; there are three convolution operations before the upsampling operation of the first upsampling convolution layer.

[0063] like Figure 2a 、 2b ,2c, shows three grayscale images with different exposure levels separated from a three-channel color picture taken by a single frame of a 3-CMOS camera;

[0064] Perform Laplace convolution on the three separated images, that is, use the formula |C(x, y) = I(x, y) × L|, where L is the Laplace convolution kernel: L = [0, 1, 0; 1, -4, 1; 0, 1, 0] to obtain the contrast value corresponding to each pixel in the three images;

[0065] Set σ to 0.2 and use the formula Get the brightness value of each pixel in the three separated images respectively;

[0066] Setting S c 0.22, S e is 0.78, using the formula W(x, y) = S c C(x, y)×S e E(x, y) obtains the scalar weight value of each pixel in the three separated images;

[0067] Afterwards, using the formula Normalize the scalar weight values ​​at the same pixel position in the three separated images; where N = 3;

[0068] Finally, using the formula N=3, the fusion pixel value of each pixel is obtained; after traversing all the pixels, a composite image is obtained, that is, a high dynamic range fringe image, such as Figure 2d As shown, the composite image has uniform exposure overall and good fringe contrast globally.

[0069] like Figure 3 As shown in the figure, a specific structure of a UNet neural network is given, with n=4, and the number of input channels and output channels are both 1; the convolution kernel sizes of the three convolution operation layers are 3×3, 5×5 and 7×7, respectively, which capture stripe features of different scales, and introduce batch normalization layer (BN) and ReLU activation function, which helps to reduce the spatial dimension of the data (i.e., the image size is halved) while retaining important features, reducing the amount of calculation and controlling overfitting; the convolution kernel size of the maximum pooling layer is 2×2; the convolution kernel size of the transposed convolution layer is 2×2; the number of convolution kernels in the convolution operation layer of each downsampling layer is 32, 64, 128 and 256 respectively; the number of convolution kernels in the transposed convolution layer of each upsampling layer is 256+256=512, 128+128=256, 64+64=128 and 32+32=64 respectively.

[0070] The three grayscale images with different exposure levels separated from the three-channel color image taken by a single frame of a 3-CMOS camera are synthesized by a preset fusion algorithm to obtain a high dynamic range fringe image as the input of the neural network. In this embodiment, the synthesized image enters the processing flow with a resolution of 2048*1536 pixels, which is the starting point of the entire neural network deconvolution process. Figure 3 The UNet neural network shown in the figure outputs an absolute phase map with a resolution of 2048*1536 pixels after four downsampling and four upsampling, completing the phase separation process. The results are shown in Figure 4 .

[0071] In order to verify the effect of dephasing high dynamic range images in this embodiment, a comparative experiment was conducted. For the same image, the traditional Fourier dephasing method and the single-frame grating fringe analysis method based on the deep learning framework were used respectively. The results are as follows: Figure 5 、 6 shown.

[0072] from Figures 4-6 It can be seen from the comparison and magnification of local details that the method provided by the present invention can retain more image details and has higher accuracy in deconvolution; the traditional Fourier deconvolution method, such as Figure 5 As shown in the figure, there are obvious line-shaped cumulative dephase errors; the single-frame grating fringe analysis method based on the deep learning framework, such as Figure 6 As shown in , although the overall deconvolution of the image is correct, there are still pixels similar to salt and pepper noise at the positions of local A and local B; the results of this method are as follows Figure 4 As shown, the image phase decomposition is error-free, with no noise at locations A and B. The overall grayscale transition is smooth, and the phase decomposition is more accurate and stable. This demonstrates that this method has a more stable and accurate phase decomposition capability globally, enabling high-quality, real-time phase map decomposition.

[0073] The foregoing descriptions of specific exemplary embodiments of the present invention have been presented for purposes of illustration and description. The foregoing descriptions are not intended to be exhaustive or to limit the invention to the precise forms disclosed, and it is apparent that many variations and modifications are possible in light of the foregoing teachings. The exemplary embodiments have been chosen and described in order to explain the specific principles of the invention and their practical application, thereby enabling others skilled in the art to make and utilize the various exemplary embodiments of the invention and various alternatives and modifications thereof. The scope of the invention is intended to be defined by the appended claims and their equivalents.

Claims

1. A single-frame dephasing method for a fringe image, characterized by: The method comprises the following steps: S1, collect a color stripe image; S2, performing RGB three-channel separation on the color stripe image to obtain three separated images; S3, fusing the three separated images according to a preset fusion algorithm to obtain a composite image; S4, inputting the composite image obtained in S3 into a trained neural network capable of dephasing for dephasing; The preset fusion algorithm in S3 includes the following steps: S3.

1. Using the preset weight coefficients of the separated images, fuse the brightness and contrast values ​​of individual pixels in a single separated image to obtain a scalar weight value for that pixel; after traversing all pixels in the separated image, obtain a scalar weight matrix for the separated image; S3.2, repeat S3.1 to obtain the scalar weight matrices of the three separated images respectively; S3.

3. Normalize the scalar weight matrices of the three separated images obtained in S3.2 so that the sum of the scalar weight values ​​at the same pixel position in the three separated images is 1; S3.

4. The pixel values ​​of the pixel points at the same position in the three separated images are weightedly summed using the corresponding scalar weight values ​​in the normalized scalar weight matrix as coefficients to obtain the fused pixel value of the point. After traversing all the pixel points, the composite image to be deconstructed is obtained.

2. The single-frame dephasing method for a fringe image according to claim 1, wherein: The contrast value of a single pixel in S3.1 is obtained by performing Laplace convolution on the pixel value of the pixel.

3. The single-frame dephasing method for a fringe image according to claim 1, wherein: The calculation method of the brightness value of a single pixel in S3.1 is: Where x and y represent the horizontal and vertical coordinates of a single pixel in the image coordinate system, E(x, y) represents the brightness value of the pixel at (x, y), I(x, y) represents the pixel value of the pixel at (x, y), and I mean represents the average grayscale value of the image, σ is the standard deviation of the Gaussian curve; exp() represents the exponential function with the natural constant e as the base.

4. The single-frame dephasing method for a fringe image according to claim 1, wherein: The method for fusing the brightness value and contrast value of a single pixel in S3.1 is: W(x,y)=S c C(x,y)×S e E(x,y) Where x and y represent the horizontal and vertical coordinates of a single pixel in the image coordinate system, W(x, y) represents the scalar weight value of the pixel (x, y), C(x, y) represents the contrast value of the pixel (x, y); E(x, y) represents the brightness value of the pixel (x, y); S c Represents the weight coefficient of the contrast value of the separated image, S e Represents the weight coefficient for separating the brightness values ​​of the image.

5. The single-frame dephasing method for a fringe image according to claim 1, wherein: The neural network is a UNet neural network or a generative adversarial network.

6. The single-frame dephasing method for a fringe image according to claim 5, wherein: The UNet neural network includes n downsampling convolution layers and n upsampling convolution layers; the nth downsampling convolution layer is jump-connected to the n-i+1th upsampling convolution layer, 1≤i≤n; each downsampling convolution layer is followed by a maximum pooling layer; each upsampling convolution layer is preceded by a transposed convolution layer; the last upsampling convolution layer is connected to a convolution layer with a convolution kernel size of 1×1 and outputs the absolute phase solution result.

7. The single-frame dephasing method for a fringe image according to claim 6, wherein: Each of the downsampling convolution layer and the upsampling convolution layer includes three convolution operation layers, and the convolution kernels of the three convolution operation layers are not equal; and three convolution operations are further provided before the upsampling operation of the first upsampling convolution layer.

Images