Single-frame dephasing method for stripe image

Abstract

The invention discloses a single-frame dephasing method for a stripe image. The method comprises the following steps: acquiring a color stripe image; carrying out RGB three-channel separation on the color stripe image to obtain three separated images; fusing the three separated images according to a preset fusion algorithm to obtain a composite image; and finally, inputting the synthesized image into a trained neural network capable of dephasing for dephasing. Wherein the preset fusion algorithm is to perform weighted summation on pixel values of pixel points at the same position of the three separated images by taking a scalar weight value obtained by fusing brightness values and contrast values of the separated images as a coefficient to obtain a composite image. According to the single-frame dephasing method, the dynamic range of the image is enlarged, the real-time performance is ensured, the stability and accuracy of dephasing in the global range are achieved, and the dephasing quality is effectively improved.

Description

Technical Field

[0001] The invention relates to the technical field of optical measurement, and in particular to a single-frame dephasing method for a fringe image. Background Art

[0002] The projection and acquisition of fringe patterns usually involve precise optical systems and complex image processing algorithms. In optical measurement, such as digital fringe projection technology and fringe photography, the clarity and contrast of the fringe pattern directly affect the accuracy and precision of the measurement results. However, due to the difference in reflectivity of the object surface, ambient light interference, and the performance limitations of the camera and projector, overexposure and dark phenomena are difficult to avoid. If the surface dynamic range of the object to be measured is high (with both highly reflective areas and dark areas), it is difficult to find an appropriate exposure so that the stripes in the reflective and dark areas on the surface of the object to be measured remain clear at the same time. If the camera exposure is too low, the stripes in the dark area are not clear; conversely, if the camera exposure is too high, the stripes in the reflective area are not clear. This phenomenon not only leads to the loss of image details, but also may introduce noise, affecting the accuracy of subsequent image processing and analysis. Therefore, in order to achieve three-dimensional measurement of objects in high-speed motion scenes, high-quality phase resolution of single-frame fringe patterns is extremely important.

[0003] The traditional Fourier method converts the grating fringe image into the frequency domain, filters out the zero-frequency and high-frequency components to remove the background and noise, and obtains the fundamental frequency component; then, through inverse transformation and complex operations, the wrapped phase information is extracted from the fringe image to achieve single-frame dephasing; however, this method is a global transformation method and cannot directly provide the local time (or space) information of the signal. Therefore, when dealing with overexposure or darkening, this method cannot accurately reflect the phase changes in these local areas, which leads to local distortion and error accumulation of phase information.

[0004] To solve this problem, the existing technology projects stripes of different intensities onto the surface of the object to be measured, takes multiple shots, and selects appropriate images for fusion to obtain a complete object stripe image, thereby achieving three-dimensional reconstruction of the object to be measured. Although this method improves the dynamic range of the image, it sacrifices real-time performance.

[0005] In addition, there is also a single-frame grating fringe analysis method based on a deep learning framework in the prior art. The idea of ​​this method is to use only one fringe image as input and use a neural network to simulate the phase demodulation process of the time domain phase shift method. Although this method can dephase a single-frame phase image in real time, it only takes one image to be dephased, and there is currently no suitable method to obtain a single-frame high dynamic range image through a single camera shot, which limits the dephase quality of this method. Summary of the invention

[0006] In order to solve the above technical problems, the present invention provides a single-frame dephasing method for a fringe image, which improves the dynamic range of the image, ensures real-time performance, and has the stability and accuracy of dephasing in a global range, effectively improving the dephasing quality.

[0007] To this end, the technical solution is as follows:

[0008] A single-frame dephasing method for a fringe image comprises the following steps:

[0009] S1, collect a color stripe image;

[0010] S2, performing RGB three-channel separation on the color stripe image to obtain three separated images;

[0011] S3, fusing the three separated images according to a preset fusion algorithm to obtain a composite image;

[0012] S4, inputting the composite image obtained in S3 into a trained neural network capable of deconstructing phases for deconstructing phases;

[0013] The preset fusion algorithm in S3 includes the following steps:

[0014] S3.1. By presetting the weight coefficient of the separated image, the brightness value and contrast value of a single pixel in a single separated image are merged to obtain a scalar weight value of the point; after traversing all the pixel points in the separated image, a scalar weight matrix of the separated image is obtained; the scalar weight value is the comprehensive weight of the point, which remains unchanged during the coordinate transformation process;

[0015] S3.2, repeat S3.1 to obtain the scalar weight matrices of the three separated images respectively;

[0016] S3.3, normalizing the scalar weight matrices of the three separated images obtained in S3.2 so that the sum of the scalar weight values ​​at the same pixel position of the three separated images is 1;

[0017] S3.4. The pixel values ​​of the pixel points at the same position in the three separated images are weightedly summed with the corresponding scalar weight values ​​in the normalized scalar weight matrix as coefficients to obtain the fused pixel value of the point. After traversing all the pixel points, the composite image to be deconstructed is obtained.

[0018] Furthermore, the contrast value of a single pixel point in S3.1 is obtained by performing Laplace convolution on the pixel value of the point.

[0019] Furthermore, the calculation method of the brightness value of a single pixel in S3.1 is:

[0020]

[0021] Where x and y represent the horizontal and vertical coordinates of a single pixel in the image coordinate system, E(x, y) represents the brightness value of the pixel (x, y), I(x, y) represents the pixel value of the pixel (x, y), and I mean represents the average gray value of the image, σ is the standard deviation of the Gaussian curve; exp() represents the exponential function with the natural constant e as the base.

[0022] Furthermore, the method for fusing the brightness value and contrast value of a single pixel in S3.1 is:

[0023] W(x,y)=S c C(x,y)×S e E(%y)

[0024] Where x and y represent the horizontal and vertical coordinates of a single pixel in the image coordinate system, W(x, y) represents the scalar weight value of the pixel (x, y), C(x, y) represents the contrast value of the pixel (x, y); E(x, y) represents the brightness value of the pixel (x, y); S c Represents the weight coefficient of the contrast value of the separated image, S e Represents the weight coefficient of the brightness value of the separated image.

[0025] Furthermore, the neural network is a UNet neural network, a convolutional neural network or a generative adversarial network.

[0026] Furthermore, the UNet neural network includes n downsampling convolution layers and n upsampling convolution layers; the nth downsampling convolution layer is jump-connected to the n-i+1th upsampling convolution layer, 1≤i≤n; each downsampling convolution layer is followed by a maximum pooling layer; each upsampling convolution layer is preceded by a transposed convolution layer; the last upsampling convolution layer is connected to a convolution layer with a convolution kernel size of 1×1, and outputs an absolute phase solution result.

[0027] Furthermore, in order to obtain stripe features of different scales, each of the downsampling convolution layer and the upsampling convolution layer includes three convolution operation layers, and the convolution kernels of the three convolution operation layers are not equal; three convolution operations are also arranged before the upsampling operation of the first upsampling convolution layer.

[0028] The present invention first separates three separated images based on a color stripe image, and then fuses the three separated images using a preset image fusion method to optimize the overexposed and dark areas in the original image to obtain a high dynamic range composite image, and finally inputs the composite image into a neural network for phase resolution to obtain an accurate phase map. This solves the problem in the prior art that there is no suitable method to obtain a single-frame high dynamic range image through a single camera shot and perform accurate phase resolution.

[0029] The beneficial effects of the present invention are:

[0030] 1. The camera collects a single-frame color fringe image, and after RGB channel separation and preset fusion algorithm processing, a high dynamic range fringe image can be directly obtained, which overcomes the problem of insufficient real-time performance caused by multiple camera shots in the prior art to obtain a high dynamic range fringe image;

[0031] 2. Use the synthetic image processed by the preset fusion algorithm as the input of the neural network for phase resolution, which makes up for the defects of the traditional deep learning method in input image quality;

[0032] 3. By constructing and training an improved UNet neural network, the deep learning method can retain more detailed information during the deconvolution process, and the deconvolution accuracy is higher. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 It is a schematic diagram of the process of the present invention;

[0034] Figure 2a It is a schematic diagram of the R channel separated image separated from the RGB three channels;

[0035] Figure 2b Schematic diagram of the G channel separated image separated from the RGB three channels;

[0036] Figure 2c Schematic diagram of the B channel separated image separated from the RGB three channels;

[0037] Figure 2d It is a schematic diagram of fusing the RGB three-channel separated images into a synthetic image;

[0038] Figure 3 A schematic diagram of the structure of the UNet neural network used in the phase solving method of the present invention;

[0039] Figure 4 Schematic diagram of experimental comparison of phase separation results of the method of the present invention

[0040] Figure 5 For Figure 4 Schematic diagram of experimental comparison of the deconvolution results of the same image using Fourier algorithm;

[0041] Figure 6 For Figure 4 Schematic diagram of the experimental comparison of the decomposition results of the same image using traditional deep learning. DETAILED DESCRIPTION

[0042] The technical solution of the present invention is described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0043] A single-frame dephasing method for fringe images, such as Figure 1 As shown, the following steps are included:

[0044] S1, collect a color stripe image;

[0045] S2, performing RGB three-channel separation on the color stripe image to obtain three separated images;

[0046] S3, fusing the three separated images according to a preset fusion algorithm to obtain a composite image;

[0047] S4, inputting the composite image obtained in S3 into a trained neural network capable of deconstructing phases for deconstructing phases;

[0048] The preset fusion algorithm in S3 includes the following steps:

[0049] S3.1. By presetting the weight coefficient of the separated image, the brightness value and contrast value of a single pixel in a single separated image are merged to obtain a scalar weight value of the point; after traversing all the pixel points in the separated image, a scalar weight matrix of the separated image is obtained; the scalar weight value is the comprehensive weight of the point, which remains unchanged during the coordinate transformation process;

[0050] S3.2, repeat S3.1 to obtain the scalar weight matrices of the three separated images respectively;

[0051] S3.3, normalizing the scalar weight matrices of the three separated images obtained in S3.2 so that the sum of the scalar weight values ​​at the same pixel position of the three separated images is 1;

[0052] S3.4. The pixel values ​​of the pixel points at the same position in the three separated images are weightedly summed with the corresponding scalar weight values ​​in the normalized scalar weight matrix as coefficients to obtain the fused pixel value of the point. After traversing all the pixel points, the composite image to be deconstructed is obtained.

[0053] In the specific implementation of S1, a 3-CMOS camera can be used to collect color images. Before using the 3-CMOS camera to collect images, different gain coefficients are set for the three RGB channels of the camera respectively.

[0054] The three-channel separation image separated from the color image is a stripe grayscale image with different exposure levels. Due to underexposure and overexposure, there will be too bright and too dark areas in the image. Therefore, from the perspective of fusion effect, the scalar weight value in S3.1 needs to be obtained by combining the contrast value and brightness value of a single pixel. Among them, the contrast value of a single pixel is obtained by performing Laplace convolution on the pixel value of the point; the calculation method of the brightness value of a single pixel is:

[0055]

[0056] Where x and y represent the horizontal and vertical coordinates of a single pixel in the image coordinate system, E(x, y) represents the brightness value of the pixel (x, y), I(x, y) represents the pixel value of the pixel (x, y), and I mean represents the average gray value of the image, σ is the standard deviation of the Gaussian curve; exp() represents the exponential function with the natural constant e as the base.

[0057] In specific implementation, the method for fusing the brightness value and contrast value of a single pixel in S3.1 is:

[0058] W(x,y)=SC(r,y)×SE(%y)

[0059] Where x and y represent the horizontal and vertical coordinates of a single pixel in the image coordinate system, W(x, y) represents the scalar weight value of the pixel (x, y), C(x, y) represents the contrast value of the pixel (x, y); E(x, y) represents the brightness value of the pixel (x, y); S c Represents the weight coefficient of the contrast value of the separated image, S e The weight coefficient representing the brightness value of the separated image. Those skilled in the art may also select other existing fusion methods in combination with achieving the purpose.

[0060] In specific implementation, the neural network for de-phase can be a UNet neural network, a convolutional neural network or a generative adversarial network.

[0061] Specifically, taking the UNet neural network model as an example, it includes n downsampling convolution layers and n upsampling convolution layers; the nth downsampling convolution layer is jump-connected to the n-i+1th upsampling convolution layer, 1≤i≤n; each downsampling convolution layer is followed by a maximum pooling layer; each upsampling convolution layer is preceded by a transposed convolution layer; the last upsampling convolution layer is connected to a convolution layer with a convolution kernel size of 1×1, and outputs the absolute phase solution result.

[0062] In more detail, in order to obtain stripe features of different scales, each downsampling convolution layer and upsampling convolution layer includes three convolution operation layers, and the convolution kernels of the three convolution operation layers are not equal; there are three convolution operations before the upsampling operation of the first upsampling convolution layer.

[0063] like Figure 2a , 2b , 2c, shows three grayscale images with different exposure levels separated from a three-channel color picture taken by a single frame of a 3-CMOS camera;

[0064] The three separated images are subjected to Laplace convolution respectively, that is, the contrast value corresponding to each pixel in the three images is obtained by using the formula |C(x, y) = I(x, y) × L|, where L is the Laplace convolution kernel: L = [0, 1, 0; 1, -4, 1; 0, 1, 0];

[0065] Set σ to 0.2 and use the formula Get the brightness value of each pixel in the three separated images respectively;

[0066] Setting S c 0.22, S e is 0.78, using the formula W(x, y) = S c C(x,y)×S e E(x, y) obtains the scalar weight value of each pixel in the three separated images;

[0067] Afterwards, using the formula Normalize the scalar weight values ​​of the three separated images at the same pixel position; where N = 3;

[0068] Finally, using the formula N = 3, and the fused pixel value of each pixel is obtained; after traversing all the pixels, a composite image is obtained, that is, a high dynamic range fringe image, such as Figure 2d As shown, the composite image is uniformly exposed overall and the fringe contrast is also good globally.

[0069] like Figure 3 As shown in the figure, a specific structure of a UNet neural network is given, with n=4, and the number of input channels and the number of output channels are both 1; the convolution kernel sizes of the three convolution operation layers are 3×3, 5×5 and 7×7, respectively, capturing stripe features of different scales, and introducing a batch normalization layer (BN) and a ReLU activation function, which helps to reduce the spatial dimension of the data (i.e., the image size is halved) while retaining important features, reducing the amount of calculation and controlling overfitting; the convolution kernel size of the maximum pooling layer is 2×2; the convolution kernel size of the transposed convolution layer is 2×2; the number of convolution kernels in the convolution operation layer of each downsampling layer is 32, 64, 128 and 256, respectively; the number of convolution kernels in the transposed convolution layer of each upsampling layer is 256+256=512, 128+128=256, 64+64=128 and 32+32=64, respectively.

[0070] A high dynamic range fringe image is obtained by synthesizing three grayscale images with different exposure levels separated from a three-channel color picture taken by a single frame of a 3-CMOS camera through a preset fusion algorithm as the input of the neural network. In this embodiment, the synthesized image enters the processing flow with a resolution of 2048*1536 pixels, which is the starting point of the entire neural network deconvolution process. Figure 3 The UNet neural network shown in the figure outputs an absolute phase map with a resolution of 2048*1536 pixels after four downsampling and four upsampling, completing the phase deconvolution process. The results are shown in Figure 4 .

[0071] In order to verify the effect of high dynamic range image dephasing achieved by this embodiment, a comparative experiment was conducted. For the same image, the traditional Fourier dephasing method and the single-frame grating fringe analysis method based on the deep learning framework were used respectively. The results are as follows: Figure 5 , 6 shown.

[0072] from Figures 4 to 6 It can be seen from the comparison and local detail magnification that the method provided by the present invention can retain more image details and has higher accuracy in deconvolution; the traditional Fourier deconvolution method, such as Figure 5 As shown in the figure, there are obvious linear cumulative phase-decomposition errors; the single-frame grating fringe analysis method based on the deep learning framework, such as Figure 6 As shown in , although the overall deconvolution of the image is correct, there are still pixels similar to salt and pepper noise at the positions of local A and local B; the results of this method are as follows Figure 4 As shown in the figure, there is no error in the image phase decomposition, no noise in the local A and local B positions, and the overall grayscale transition is smooth, and the phase decomposition accuracy and stability are higher. This shows that this method has a more stable and accurate phase decomposition capability in the global range, and can achieve high-quality, real-time phase map solution.

[0073] The foregoing description of the specific exemplary embodiments of the present invention is presented for the purpose of illustration and description. The foregoing description is not intended to be exhaustive, nor is it intended to limit the present invention to the precise form disclosed, and it is apparent that many changes and variations are possible in light of the above teachings. The exemplary embodiments are selected and described to explain the specific principles of the present invention and its practical application, so that other technicians in the field can realize and utilize the various exemplary embodiments of the present invention and its different selected forms and modified forms. The scope of the present invention is intended to be limited by the appended claims and their equivalents.

Claims

1. A single-frame dephasing method for a fringe image, characterized in that: The method comprises the following steps: S1, collect a color stripe image; S2, performing RGB three-channel separation on the color stripe image to obtain three separated images; S3, fusing the three separated images according to a preset fusion algorithm to obtain a composite image; S4, inputting the composite image obtained in S3 into a trained neural network capable of deconstructing phases for deconstructing phases; The preset fusion algorithm in S3 includes the following steps: S3.

1. By presetting the weight coefficient of the separated image, the brightness value and contrast value of a single pixel in a single separated image are merged to obtain a scalar weight value of the point; after traversing all the pixels in the separated image, a scalar weight matrix of the separated image is obtained; S3.2, repeat S3.1 to obtain the scalar weight matrices of the three separated images respectively; S3.3, normalizing the scalar weight matrices of the three separated images obtained in S3.2 so that the sum of the scalar weight values ​​at the same pixel position of the three separated images is 1; S3.

4. The pixel values ​​of the pixel points at the same position in the three separated images are weightedly summed with the corresponding scalar weight values ​​in the normalized scalar weight matrix as coefficients to obtain the fused pixel value of the point. After traversing all the pixel points, the composite image to be deconstructed is obtained.

2. The single-frame dephasing method of fringe images as claimed in claim 1, characterized in that: The contrast value of a single pixel point in S3.1 is obtained by performing Laplace convolution on the pixel value of the point.

3. The single-frame dephasing method of fringe images as claimed in claim 1, characterized in that: The calculation method of the brightness value of a single pixel in S3.1 is: Where x and y represent the horizontal and vertical coordinates of a single pixel in the image coordinate system, E(x, y) represents the brightness value of the pixel (x, y), I(x, y) represents the pixel value of the pixel (x, y), and I mean represents the average gray value of the image, σ is the standard deviation of the Gaussian curve; exp() represents the exponential function with the natural constant e as the base.

4. The single-frame dephasing method of fringe images as claimed in claim 1, characterized in that: The method for fusing the brightness value and contrast value of a single pixel in S3.1 is: W(x,y)=S c C(x,y)×S e E(x,,y) Where x and y represent the horizontal and vertical coordinates of a single pixel in the image coordinate system, W(x, y) represents the scalar weight value of the pixel (x, y), C(x, y) represents the contrast value of the pixel (x, y); E(x, y) represents the brightness value of the pixel (x, y); S c Represents the weight coefficient of the contrast value of the separated image, S e Represents the weight coefficient of the brightness value of the separated image.

5. The single-frame dephasing method of fringe images as claimed in claim 1, characterized in that: The neural network is a UNet neural network, a convolutional neural network or a generative adversarial network.

6. The single-frame dephasing method of fringe images as claimed in claim 5, characterized in that: The UNet neural network includes n downsampling convolution layers and n upsampling convolution layers; the nth downsampling convolution layer is jump-connected to the n-i+1th upsampling convolution layer, 1≤i≤n; each downsampling convolution layer is followed by a maximum pooling layer; each upsampling convolution layer is preceded by a transposed convolution layer; the last upsampling convolution layer is connected to a convolution layer with a convolution kernel size of 1×1, and outputs an absolute phase solution result.

7. The single-frame dephasing method of fringe images as claimed in claim 6, characterized in that: Each of the downsampling convolution layer and the upsampling convolution layer includes three convolution operation layers, and the convolution kernels of the three convolution operation layers are not equal; and three convolution operations are arranged before the upsampling operation of the first upsampling convolution layer.

Images