An interference fringe filtering method based on deep learning

Abstract

The application discloses an interference fringe filtering method based on deep learning, which comprises the following steps: S1, designing a GAN neural network for interference fringe image denoising and a DnCNN neural network for noise modeling; S2, generating an interference image sample through a program, adding noise generated by the DnCNN to obtain a clear-noise point pair image dataset; S3, training the network through the generated dataset to obtain an interference fringe filtering model based on the GAN; S4, obtaining a real noisy interference fringe image captured by an optical system; and S5, outputting a clear interference image without noise points through the GAN network. According to the application, a clean and clear interference fringe image is obtained, the good denoising effect is ensured, the computer operation time is shortened, no additional hardware device is needed, and the cost is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a method for implementing interference fringe filtering based on deep learning. Background Art

[0002] Interferometers play a very important role in the field of three-dimensional shape measurement. By performing phase unwrapping on the interference fringes, the three-dimensional shape of an object can be obtained in a non-contact, high-precision manner.

[0003] A true noisy image can be regarded as a clean and clear image with a noise offset added to each pixel.

[0004] During the acquisition of interference fringes, the experimental instrument is subject to ambient light, disturbances, camera hardware, and acquisition card errors, which can cause the obtained interference fringes to contain a large amount of noise information. When the interference fringe pattern is disturbed by noise, this noise will affect the accuracy of phase unwrapping and even cause the unwrapping process to fail, ultimately affecting the accuracy of the object's 3D topography measurement.

[0005] Traditional methods for filtering and denoising interference fringes include mean filtering, Gaussian filtering, median filtering, and bilateral filtering. While some of these methods can achieve good denoising results, they take too long to compute, impacting measurement efficiency. Alternatively, while they meet the time requirement, they blur the image during the denoising process, losing detail and thus affecting measurement accuracy. Summary of the Invention

[0006] To address the shortcomings of the prior art, the present invention aims to provide a method for implementing interference fringe filtering based on deep learning to obtain clean, clear interference fringe patterns. The present invention ensures good denoising effects, shortens computer processing time, and eliminates the need for additional hardware, reducing costs. To achieve the above-mentioned objectives and other advantages of the present invention, a method for implementing interference fringe filtering based on deep learning is provided, comprising:

[0007] S1. Design a GAN neural network for interference fringe image denoising and a DnCNN neural network for noise modeling;

[0008] S2. Generate interference image samples through the program and add noise generated by DnCNN to obtain a clear-noise pair image dataset;

[0009] S3. Generate a dataset for network training to obtain a GAN-based interference fringe filter model.

[0010] S4, obtaining a real noisy interference fringe pattern captured by the optical system;

[0011] S5. Output a clear interference image without noise through the GAN network.

[0012] Preferably, in step S1, the DnCNN neural network adds a short-circuit design on the basis of the convolution, BN, and ReLU cascade structure to prevent network overfitting and improve network convergence accuracy. The network input is a noisy picture and the output is a noisy image. At the same time, a threshold module is designed in the output layer to filter out invalid noise and improve network accuracy; the GAN neural network input is a noisy picture and the output is a clear sample.

[0013] Preferably, in step S2, a program is used to generate interference fringe images of random patterns, a discrimination mechanism is set, interference fringe images that are close to the actual situation are retained, and the improved DnCNN network described in claim 2 is used to generate paired clear-noise interference fringe images, which are divided into training sets and test sets in proportion.

[0014] Preferably, in step S3, the GAN network includes a generator and a discriminator. The GAN network inputs a noisy picture to the network, first generates a noise-free picture through the generator, and then inputs the generated noise-free picture and the known noise-free picture into the discriminator to let the discriminator determine which of the two is the known noise-free picture, and feeds back the parameters to the network. The generator needs to make the generated picture deceive the discriminator, and the output of the generator is input into the circular noise filtering module to continuously improve the clarity of the generator output picture, thereby continuously improving the generation and updating the network; and the discriminator needs to successfully judge the authenticity of the picture, and adjust the network in real time through threshold setting.

[0015] Preferably, the optical system comprises a semiconductor laser (1), a beam expander (2) arranged on one side of the semiconductor laser (1), a beam splitter prism (3) arranged on one side of the beam splitter prism (2), an imaging lens (8) arranged above the beam splitter prism (3), a collimating lens (4) arranged on one side of the beam splitter prism (3), a plane crystal (5) arranged on one side of the collimating lens (4), and a measured surface (7) arranged on one side of the plane crystal (5), wherein the end face of the plane crystal (5) adjacent to the measured surface (7) is a standard surface (6), and a camera is arranged on one side of the imaging lens (8).

[0016] Preferably, in step S5, a real noisy interference fringe pattern is obtained through an interference optical system, and an attention mechanism is added to the GAN network in step S3 to enhance the network's restoration performance. The noisy interference fringe pattern obtained in step S5 is input into the GAN denoising model trained in step S3, and a clean, noise-free interference fringe pattern is output.

[0017] Compared with the existing technology, the present invention has the following advantages: it solves the difficulty of obtaining a clear-noise interference fringe dataset, and uses the DnCNN network to train real noisy images to obtain an image noise model, which can effectively simulate the noise situation of images in a real environment and provide a better model training set for the GAN denoising network. By programmatically generating interference fringe images and adding a CNN noise extraction model, a set of image denoising training sets can be well constructed. At the same time, because the CNN network can simulate the noise in a real environment, the GAN denoising network can achieve better training accuracy. The deep learning denoising method based on GAN can improve the restoration accuracy through mutual learning between the generator and the discriminator. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 This is a structural diagram of the DnCnn model for implementing the interference fringe filtering method based on deep learning according to the present invention;

[0019] Figure 2 A structural diagram of a GAN model for implementing an interference fringe filtering method based on deep learning according to the present invention;

[0020] Figure 3 This is a structural diagram of a GAN model generator for implementing an interference fringe filtering method based on deep learning according to the present invention;

[0021] Figure 4 This is a structural diagram of a GAN model discriminator for implementing an interference fringe filtering method based on deep learning according to the present invention;

[0022] Figure 5 This is a structural diagram of an optical experimental system for implementing an interference fringe filtering method based on deep learning according to the present invention;

[0023] Figure 6 This is a flowchart of a method for implementing interference fringe filtering based on deep learning according to the present invention. DETAILED DESCRIPTION

[0024] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0025] Reference Figure 1-6 , a method for implementing interference fringe filtering based on deep learning, comprising the following steps: S1, designing a GAN neural network for interference fringe image denoising and a DnCNN neural network for noise modeling;

[0026] S2, generate interference image samples through the program, and add noise generated by DnCNN to obtain a clear-noise pair image dataset;

[0027] S3. Generate a dataset for network training to obtain a GAN-based interference fringe filter model.

[0028] S4, obtaining a real noisy interference fringe pattern captured by the optical system;

[0029] S5. Output a clear interference image without noise through the GAN network.

[0030] Example 1

[0031] Step 1: Design neural network: Design the GAN neural network for interference fringe image denoising and the DnCNN neural network for noise modeling. First, design the DnCNN neural network for noise modeling. Figure 1 As shown in the figure, DnCNN is improved from the VGG network. The network layer is a convolution, BN, and ReLU cascade structure, which is finally output through multi-layer connections. The network input is a noisy picture. The noise image first passes through a layer of modules composed of convolution and ReLU functions, and then accesses a block module composed of 20 layers of Conv+BN+ReLU. Finally, it passes through a layer of convolution to extract the output of the noise module. The convolution layer can be used to extract the feature information of the image. The BN layer normalizes the data to prevent the gradient from disappearing. The ReLU function performs activation processing. The loss function used by the network is MSE (mean square error). Subsequently, a filtering denoising network based on the GAN network is designed, as shown in the figure. Figure 2 As shown in Figure 2. The network is composed of a set of generator networks and a set of discriminator networks, where G represents the generator and D represents the discriminator. Figure 3 As shown in the figure, the input is a noisy image. After several layers, the convolution layer consists of Conv, BN, and ReLU, which are used to extract image features. The residual block is similar to the convolution layer, but with a residual structure to eliminate gradient vanishing and explosion. Finally, the extracted feature map is decoded by the deconvolution layer to output a noise-free image. The network structure of the discriminator is as follows: Figure 4 As shown, the network consists of several convolutional layers, each composed of Conv+BN+ReLU. The input is a denoised image generated by the generator, which is then compared with a known clear image to determine if it is a real image. This competition between the two continuously optimizes network performance and improves accuracy.

[0032] Step 2: Generate interference image samples using a program and add noise generated by DnCNN to obtain a dataset of clear-noise pairs. First, use the program to generate interference fringe samples. Then, input the real-world noisy images into the DnCNN network to train the noise extraction model. Then, add the real-world noise images generated by the DnCNN network to the clean, clear interference fringe samples generated by the program to obtain interference fringe images with real noise. The obtained noise-clear image pairs serve as the training set for GAN network training.

[0033] Step 3: Use the generated dataset to train the network and obtain a GAN-based interference fringe filter model. The dataset is input into the GAN neural network for training. The noisy interference fringe pattern is first input into the GAN generator network. The generator network extracts image features through the convolution layer, and then enters the residual module for calculation. The residual module effectively prevents the network's gradient from vanishing or exploding. Finally, the extracted feature map is decoded through the deconvolution layer and output as a noise-free image. The image generated by the generator is then input into the discriminator. The discriminator determines the probability of the generated image being a real image. If the probability is 1, it is a real image, and if it is 0, it is a fake image. The generator needs to make the generated image deceive the discriminator, while the discriminator needs to determine whether it is generated by the generator. The two compete with each other and continuously adjust parameters until the discriminator network is unable to determine whether the output of the generator network is real. The weight parameters at this time are recorded, and the GAN denoising model is obtained.

[0034] Step 4: Obtain the real noise interference fringe pattern captured by the optical system. Figure 5 As shown in the figure, in the interference system, a single-shot light beam is partially reflected on the standard plane as the reference beam, and partially transmitted and passed through the device under test as the detection beam. The detection beam returns by self-collimation and overlaps with the reference beam to form interference fringes. The CCD obtains the light on the imaging lens and inputs the interference fringe pattern into the computer for processing, thus obtaining a true interference fringe pattern with noise.

[0035] Step 5: Use the GAN network to output a clear, noise-free interference image. Obtain a real, noisy interference fringe pattern through an interferometric optical system. Input this into the GAN denoising model trained in step 3, and output a clean, noise-free interference fringe pattern.

[0036] In summary, during the acquisition of interference fringes, the experimental instrument is subject to a large amount of noise due to factors such as ambient light, disturbances, camera hardware, and acquisition card errors. When the interference fringe pattern is affected by noise, this noise can affect the accuracy of phase unwrapping and even cause the unwrapping process to fail, ultimately affecting the accuracy of the object's three-dimensional topography measurement. To address this issue, the present invention has discovered a method for implementing interference fringe filtering based on deep learning. This method ensures excellent denoising effects, shortens computer processing time, and does not require additional hardware equipment, reducing costs.

[0037] The number of devices and processing scales described herein are intended to simplify the description of the present invention, and applications, modifications, and variations of the present invention will be apparent to those skilled in the art.

[0038] Although the embodiments of the present invention have been disclosed above, they are not limited to the applications listed in the description and implementation methods. They can be fully applied to various fields suitable for the present invention. For those familiar with the art, additional modifications can be easily implemented. Therefore, without departing from the general concept defined by the claims and the scope of equivalents, the present invention is not limited to the specific details and illustrations shown and described herein.

Claims

1. A method for implementing interference fringe filtering based on deep learning, characterized in that: The following steps are involved: S1. Design a GAN neural network for interference fringe image denoising and a DnCNN neural network for noise modeling; S2. Generate interference image samples through the program and add noise generated by DnCNN to obtain a clear-noise pair image dataset; S3. Generate a dataset for network training to obtain a GAN-based interference fringe filter model. S4, obtaining a real noisy interference fringe pattern captured by the optical system; S5. Output a clear interference image without noise through the GAN network.

2. The method for implementing interference fringe filtering based on deep learning according to claim 1, characterized in that: In step S1, the DnCNN neural network adds a short-circuit design based on the convolution, BN, and ReLU cascade structure to prevent network overfitting and improve network convergence accuracy. The network input is a noisy picture and the output is a noisy image. At the same time, a threshold module is designed in the output layer to filter out invalid noise and improve network accuracy. The GAN neural network input is a noisy picture and the output is a clear sample.

3. The method for implementing interference fringe filtering based on deep learning according to claim 2, characterized in that: In step S2, a program is used to generate interference fringe images of random patterns, a discrimination mechanism is set, interference fringe images that are close to the actual situation are retained, and the DnCNN neural network in claim 2 is used to generate paired clear-noise interference fringe images, which are divided into training sets and test sets in proportion.

4. The method for implementing interference fringe filtering based on deep learning according to claim 3, wherein: In step S3, the GAN network includes a generator and a discriminator. The GAN network inputs a noisy picture to the network. First, the generator generates a noise-free picture. Then, the generated noise-free picture and the known noise-free picture are input into the discriminator to let the discriminator determine which of the two is the known noise-free picture and feed back the parameters to the network. The generator needs to make the generated picture deceive the discriminator. The output of the generator is input into the circular noise filtering module to continuously improve the clarity of the generator output image, thereby continuously improving the generation and updating the network; the discriminator needs to successfully judge the authenticity of the picture and adjust the network in real time through threshold setting.

5. The method for implementing interference fringe filtering based on deep learning according to claim 4, characterized in that: The optical system comprises a semiconductor laser (1), a beam expander (2) arranged on one side of the semiconductor laser (1), a beam splitter prism (3) arranged on one side of the beam splitter prism (2), an imaging lens (8) arranged above the beam splitter prism (3), a collimating objective lens (4) arranged on one side of the beam splitter prism (3), a plane crystal (5) arranged on one side of the collimating objective lens (4), and a measured surface (7) arranged on one side of the plane crystal (5), wherein the end face of the plane crystal (5) adjacent to the measured surface (7) is a standard face (6), and a camera is arranged on one side of the imaging lens (8).

6. The method for implementing interference fringe filtering based on deep learning according to claim 1, wherein: In step S5, a real noisy interference fringe pattern is obtained through an interference optical system, and the GAN network in step S3 adds an attention mechanism to enhance the network's restoration performance; the noisy interference fringe pattern obtained in step S5 is input into the GAN denoising model trained in step S3, and a clean noise-free interference fringe pattern is output.

Images