Phase contrast OCT image denoising method based on generative adversarial network

Abstract

The invention discloses a phase contrast OCT (Optical Coherence Tomography) image denoising method based on a generative adversarial network, which utilizes the training characteristic that the generative adversarial network continuously generates adversarial, and selects a phase image with a good vector method denoising effect to provide learning training of the generative adversarial network, so that the generative adversarial network can well learn the vector method denoising effect. Meanwhile, the generative adversarial network is based on a data-driven model, model parameters are automatically adjusted through a training process, manual parameter adjustment is not needed, the adaptive capacity to complex and diversified images and noise is high, the denoising effect is better, and universality is higher.

Description

Technical Field

[0001] The present invention relates to the technical field of image denoising, and in particular to a phase contrast OCT image denoising method based on a generative adversarial network. Background Art

[0002] Optical coherence tomography (OCT) is a tomographic imaging method based on the principle of low-coherence interference. It has the advantages of non-invasiveness and rapid imaging, and can be used for two-dimensional cross-sectional and three-dimensional volume imaging inside transparent and semi-transparent objects. The depth resolution and range of OCT are micrometers and millimeters, respectively, and can be used for tomographic imaging of the internal microstructures of various biological tissues and non-biological materials such as cornea, skin, tumors, polymers, and composite materials. However, since OCT uses coherent illumination, speckle noise is the main noise source that reduces the quality of OCTB scanning images. Speckle noise is the multiplicative noise inherent in coherent imaging, caused by multiple forward and backward scattering of light waves. It often reduces contrast, and granular speckle noise limits the axial and lateral effective image resolution.

[0003] In traditional denoising methods, median filtering: determine a neighborhood for each pixel, sort the pixel values ​​in the neighborhood by size, and replace the center pixel value with the sorted middle value. Wavelet transform: perform discrete wavelet transform (DWT) on the image and decompose it into multiple subbands (such as low-frequency subbands and high-frequency subbands). Threshold the coefficients of the high-frequency subband to remove noise. Perform inverse wavelet transform (IDWT) to reconstruct the denoised image. Gaussian filtering: select a suitable Gaussian kernel (such as 3x3 or 5x5) and calculate the weight of each position. For each pixel, use the Gaussian kernel to weight the pixel values ​​in the neighborhood and calculate the weighted average, and replace the center pixel value with the weighted average. Mean filtering: determine a neighborhood for each pixel (such as a 3x3 window), calculate the average value of all pixel values ​​in the neighborhood, and replace the center pixel value with this average value.

[0004] The above-mentioned traditional denoising methods are designed based on the principles of general image processing and do not take the characteristics of OCT signals into consideration. Therefore, they are not effective in removing the phase noise of OCT signals.

[0005] CNN denoising: Based on a data-driven model, it automatically learns appropriate features and denoising strategies through a large amount of training data. However, it still does not target the characteristics of OCT signals and noise. It only learns and denoises phase noise images, and the denoising effect is not good.

[0006] Although the vector method performs signal denoising in the complex domain for the OCT signal model, it still follows the model of traditional denoising methods. It does not use deep learning methods, is based on predefined mathematical models and rules, relies on manual parameter adjustment, lacks the ability to adapt to different images, and has poor denoising effects on complex and diverse noises. Summary of the invention

[0007] The purpose of the present invention is to overcome the shortcomings of the prior art and provide a phase contrast OCT image denoising method based on a generative adversarial network.

[0008] To achieve the above purpose, the technical solution provided by the present invention is:

[0009] A phase contrast OCT image denoising method based on a generative adversarial network, comprising:

[0010] A large number of simulated noisy phase images with different fringes and noise levels;

[0011] Process the noisy phase image obtained by simulation by using the vector method, and select the denoised phase image with good denoising effect and the noisy phase image corresponding to the denoised phase image;

[0012] A generative adversarial network is established, and the denoised phase image with good denoising effect and the noisy phase image corresponding to the denoised phase image are selected to train the network, so as to obtain a trained denoising model;

[0013] Experimental data collection, obtaining noisy phase images with different fringe densities;

[0014] The noisy phase image obtained by collecting experimental data is input into the trained denoising model for denoising, and finally the denoised phase image is obtained.

[0015] Furthermore, experimental data collection to obtain a noisy phase diagram includes:

[0016] The OCT system is used to collect the interference spectral signals of the sample before and after deformation, and CZT transformation is performed to obtain the wrapped phase of the interference spectral signals before and after deformation. The difference between the two is then calculated to obtain the wrapped phase image. By changing the selection of the interference image before and after the displacement, noisy phase images with different fringe densities can be obtained.

[0017] Furthermore, the generative adversarial network is trained by selecting the denoised phase image with good denoising effect and the noisy phase image corresponding to the denoised phase image, and the process includes:

[0018] A large number of noisy and denoised phase images with good denoising effect after vector method processing are selected as training data and input into the generator. The generator learns to generate denoised images, and then the denoised images generated by the generator and the original noise-free images are input into the discriminator. The discriminator distinguishes the difference between the denoised images and the original images, and then calculates the loss function between the denoised images and the original images, the loss between the generator and the discriminator, and the discriminant loss. The loss function is then fed back to the generator, and the generator adjusts the learning parameters for a new round of training. Each round of learning thereafter is a continuous cycle, with the generator network and the discriminator network alternately trained, and the parameters of the correction network are adjusted through continuous confrontation between the two networks.

[0019] Furthermore, the basic principle of generating adversarial networks is expressed as follows:

[0020]

[0021] In the formula, G and D represent the generator and discriminator respectively, E represents the expectation, L is the loss function, and p data (u) represents the distribution of real data, p f (f) represents the noise distribution, logD(u) is the probability that the discriminator D correctly identifies the real image, and log(1-D(G(f))) is the probability that D identifies the generated image as a false image; Represents the adversarial process between the generator G and the discriminator D. The generator G minimizes the loss function L in order to generate more realistic denoised images, and the discriminator D maximizes the loss function L in order to more accurately distinguish between real images and generated images.

[0022] Furthermore, the loss function is used to measure the difference between the generated image and the target real image. It calculates the absolute difference between the pixels of the generated image and the target image and uses it as the loss to promote the similarity between the generated image and the target image. The calculation formula is as follows:

[0023] L1Loss=(1 / (H*W)∑|S[i,j]-T[i,j]| (2)

[0024] Among them, H and W represent the height and width of the image respectively, S[i,j] represents the pixel value of the generated image at position [i,j], and T[i,j] represents the pixel value of the target real image at position [i,j].

[0025] Furthermore, the discriminant loss is a binary cross entropy loss function, which is used to calculate the loss of the discriminator for real images and generated images, and to determine the probability that the generated image is a real image. The formula is as follows:

[0026]

[0027] Among them, N represents the number of samples, ∑ represents the sum of all samples, and the output of the discriminator for the real image is D r The output of the generated image is D f ; The target value of 1 represents a real image, and the target value of 0 represents a generated image; by minimizing BCELoss, the discriminator is prompted to output a real image close to 1 and a generated image close to 0, thereby achieving the distinction between real images and generated images.

[0028] Furthermore, in the downsampling part of the generator's network architecture, the initial downsampling layer consists of a convolutional layer, a batch normalization layer, and a LeakyReLU activation function, which are used to perform initial feature extraction and downsampling of the input image; followed by a series of Block modules for multiple downsamplings; each Block module contains a convolutional layer, a batch normalization layer, and a LeakyReLU activation function, and the step size of the convolutional layer in the Block class is 2 in each dimension, so that the size of the feature map is halved, the number of input channels gradually increases, and the number of output channels gradually doubles, thereby realizing layer-by-layer feature extraction and downsampling; the bottom is the bottleneck layer, which consists of a convolutional layer and a ReLU activation function, which is used to generate a bottleneck feature and reduce the dimensionality of the input feature map.

[0029] Furthermore, the upsampling part of the network architecture of the generator includes continuous Block modules for multiple upsampling; each Block module contains a transposed convolution layer, a batch normalization layer and a ReLU activation function, and uses a transposed convolution layer to achieve upsampling by inserting zero values ​​and applying a convolution kernel to increase the size of the feature map without losing details; the number of input channels is gradually halved, and the number of output channels is gradually reduced to achieve layer-by-layer upsampling of features; the final upsampling part consists of a transposed convolution layer and a Tanh activation function, which is used to upsample the last layer of feature maps to the same size as the input image and output the generated image; skip connection, each upsampling block concatenates its input with the downsampling part of the corresponding layer before upsampling to establish a skip connection.

[0030] Furthermore, in the network architecture of the discriminator, the number of input image channels is 3. The initial convolution part consists of a convolution layer and a LeakyReLU activation function, which is used to perform initial feature extraction and downsampling of the input image, and the number of output channels is 32; the intermediate convolution part consists of multiple CNNBlocks, each of which is used to perform convolution operations and decide whether to downsample and use dropout based on whether it is the last layer. These CNNBlocks are constructed according to the specified number of features, and the number of features gradually increases to achieve layer-by-layer feature extraction; the last convolution layer maps the feature map of the last layer to a scalar for discrimination of binary classification tasks.

[0031] Compared with the prior art, the principles and advantages of this technical solution are as follows:

[0032] 1. Utilizing the training characteristics of the generative adversarial network that continuously generates adversarial images, by selecting phase images with good vector method denoising effects, the generative adversarial network learning training is provided, so that the generative adversarial network can learn the effect of vector method denoising very well. At the same time, the generative adversarial network is based on a data-driven model, and the model parameters are automatically adjusted through the training process without manual parameter adjustment. It has strong adaptability to complex and diverse images and noises, and has better denoising effects and stronger versatility. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the services required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0034] Figure 1 It is a principle flow chart of a phase contrast OCT image denoising method based on a generative adversarial network of the present invention;

[0035] Figure 2 A framework diagram of a generative adversarial network in a phase contrast OCT image denoising method based on a generative adversarial network according to the present invention;

[0036] Figure 3 This is the architecture diagram of the generator network in the generative adversarial network;

[0037] Figure 4 This is the architecture diagram of the discriminator network in the generative adversarial network;

[0038] Figure 5 The following is a comparison of the denoising effects of the vector method (a is a simulated phase image, and b is a real experimental phase image);

[0039] Figure 6 A comparison chart of the denoising effect based on the generative adversarial network (c is the simulated phase image, d is the actual collected experimental phase image). DETAILED DESCRIPTION

[0040] The present invention will be further described below in conjunction with specific embodiments:

[0041] like Figure 1 As shown, the phase contrast OCT image denoising method based on a generative adversarial network described in this embodiment includes the following steps:

[0042] S1, a large number of simulated noisy phase images with different fringes and noise sizes;

[0043] Through experiments, it is found that speckle noise can be modeled as Gaussian noise (μ = 0, σ = 1) multiplied by uniform distribution (range 0.02 to 0.5). Therefore, in this step, by randomly simulating different original phase images through matlab, and then adding Gaussian noise, a large number of phase images with different stripes and noise sizes can be obtained.

[0044] S2, processing the noisy phase image obtained by simulation by the vector method, and selecting the denoised phase image with good denoising effect and the noisy phase image corresponding to the denoised phase image;

[0045] S3, establishing a generative adversarial network, and training it through the selected denoised phase images with good denoising effect and the noisy phase images corresponding to the denoised phase images, to obtain a trained denoising model;

[0046] In this step, the training process includes:

[0047] like Figure 2 As shown in the figure, a large number of noisy and denoised phase images with good denoising effect processed by the vector method are selected as training data and input into the generator. The generator learns to generate denoised images, and then the denoised images generated by the generator and the original noise-free images are input into the discriminator. The discriminator distinguishes the difference between the denoised images and the original images, and then calculates the loss function between the generated denoised images and the original images (i.e., L1 loss), the loss between the generator and the discriminator (i.e., adversarial loss), and the discriminant loss; the loss function is then fed back to the generator, and the generator adjusts the learning parameters for a new round of training. Each round of learning thereafter is a continuous cycle, with the generator network and the discriminator network alternately iteratively trained, and the parameters of the correction network are adjusted through continuous confrontation between the two networks.

[0048] The basic principle of generating adversarial networks is expressed as follows:

[0049]

[0050] In the formula, G and D represent the generator and discriminator respectively, E represents the expectation, L is the loss function, and p data (u) represents the distribution of real data, p f (f) represents the noise distribution, logD(u) is the probability that the discriminator D correctly identifies the real image, and log(1-D(G(f))) is the probability that D identifies the generated image as a false image; Represents the adversarial process between the generator G and the discriminator D. The generator G minimizes the loss function L in order to generate more realistic denoised images, and the discriminator D maximizes the loss function L in order to more accurately distinguish between real images and generated images.

[0051] The loss function is used to measure the difference between the generated image and the target real image. It calculates the absolute difference between the pixels of the generated image and the target image and uses it as the loss to promote the similarity between the generated image and the target image. The calculation formula is as follows:

[0052] L1Loss=(1 / (H*W)∑|S[i,j]-T[i,j]| (2)

[0053] Among them, H and W represent the height and width of the image respectively, S[i,j] represents the pixel value of the generated image at position [i,j], and T[i,j] represents the pixel value of the target real image at position [i,j].

[0054] The discriminant loss is a binary cross entropy loss function, which is used to calculate the loss of the discriminator for real images and generated images, and to determine the probability that the generated image is a real image. The formula is as follows:

[0055]

[0056] Among them, N represents the number of samples, ∑ represents the sum of all samples, and the output of the discriminator for the real image is D r The output of the generated image is D f ; The target value of 1 represents a real image, and the target value of 0 represents a generated image; by minimizing BCELoss, the discriminator is prompted to output a real image close to 1 and a generated image close to 0, thereby achieving the distinction between real images and generated images, which helps to improve the generator's ability to generate more realistic images.

[0057] like Figure 3 As shown in the figure, the downsampling part of the network architecture of the generator, the initial downsampling layer consists of a convolutional layer, a batch normalization layer and a LeakyReLU activation function, which is used to perform initial feature extraction and downsampling of the input image; followed by a series of Block modules for multiple downsampling; each Block module contains a convolutional layer, a batch normalization layer and a LeakyReLU activation function, and the step size of the convolutional layer in the Block class is 2 in each dimension, so that the size of the feature map is halved, the number of input channels gradually increases, and the number of output channels gradually doubles, realizing layer-by-layer feature extraction and downsampling; the bottom is the bottleneck layer, which consists of a convolutional layer and a ReLU activation function, which is used to generate a bottleneck feature and reduce the dimension of the input feature map.

[0058] The upsampling part of the generator's network architecture includes consecutive Block modules for multiple upsampling; each Block module contains a transposed convolution layer, a batch normalization layer, and a ReLU activation function. The transposed convolution layer is used to achieve upsampling by inserting zero values ​​and applying convolution kernels to increase the size of the feature map without losing details; the number of input channels is gradually halved, and the number of output channels is gradually reduced to achieve layer-by-layer upsampling of features; the final upsampling part consists of a transposed convolution layer and a Tanh activation function, which is used to upsample the last layer of feature maps to the same size as the input image and output the generated image; jump connection, each upsampling block concatenates its input with the downsampling part of the corresponding layer before upsampling to establish a jump connection. In this way, fine-grained features can be passed to higher resolution layers during the upsampling process, thereby helping the generator to better restore details.

[0059] like Figure 4 As shown in the figure, in the network architecture of the discriminator, the number of input image channels is 3. The initial convolution part consists of a convolution layer and a LeakyReLU activation function, which is used to perform initial feature extraction and downsampling of the input image. The number of output channels is 32; the intermediate convolution part consists of multiple CNNBlocks, each of which is used to perform convolution operations and decide whether to downsample and use dropout based on whether it is the last layer. These CNNBlocks are constructed according to the specified number of features, and the number of features gradually increases to achieve layer-by-layer feature extraction; the last convolution layer maps the feature map of the last layer to a scalar for discrimination of binary classification tasks.

[0060] S4, experimental data collection, obtaining noisy phase images with different fringe densities;

[0061] In this step, the interference spectrum signal of the sample before and after deformation is collected through the OCT system, and CZT transformation (Chirp Z-Transform) is performed to obtain the wrapped phase of the interference spectrum signal before and after deformation. The difference between the two is then calculated to obtain the wrapped phase image. By changing the selection of the interference image before and after the displacement, the noisy phase image with different fringe densities can be obtained.

[0062] S5. The noisy phase image obtained by collecting experimental data is input into the trained denoising model for denoising, and finally a denoised phase image is obtained.

[0063] from Figure 5 and Figure 6 It can be seen that by utilizing the training characteristics of the generative adversarial network that continuously generates adversarial effects, phase images with good vector method denoising effects are selected to provide learning training for the generative adversarial network, so that the generative adversarial network can learn the effect of vector method denoising well.

[0064] At the same time, the generative adversarial network is based on a data-driven model, and the model parameters are automatically adjusted through the training process, without manual parameter adjustment. It has strong adaptive ability to complex and diverse images and noise, better denoising effect and stronger versatility. The above-mentioned implementation examples are only preferred embodiments of the present invention, and are not intended to limit the scope of implementation of the present invention. Therefore, all changes made according to the shape and principle of the present invention should be covered within the protection scope of the present invention.

Claims

1. A phase contrast OCT image denoising method based on generative adversarial network, characterized in that: include: A large number of simulated noisy phase images with different fringes and noise levels; Process the noisy phase image obtained by simulation by using the vector method, and select the denoised phase image with good denoising effect and the noisy phase image corresponding to the denoised phase image; A generative adversarial network is established, and the denoised phase image with good denoising effect and the noisy phase image corresponding to the denoised phase image are selected to train the network, so as to obtain a trained denoising model; Experimental data collection, obtaining noisy phase images with different fringe densities; The noisy phase image obtained by collecting experimental data is input into the trained denoising model for denoising, and finally the denoised phase image is obtained.

2. The method for denoising phase-contrast OCT images based on a generative adversarial network according to claim 1, characterized in that: Experimental data collection, obtaining noisy phase diagrams, including: The OCT system is used to collect the interference spectral signals of the sample before and after deformation, and CZT transformation is performed to obtain the wrapped phase of the interference spectral signals before and after deformation. The difference between the two is then calculated to obtain the wrapped phase image. By changing the selection of the interference image before and after the displacement, noisy phase images with different fringe densities can be obtained.

3. The method for denoising phase-contrast OCT images based on a generative adversarial network according to claim 1, characterized in that: The generative adversarial network is trained by selecting the denoised phase image with good denoising effect and the noisy phase image corresponding to the denoised phase image. The process includes: A large number of noisy and denoised phase images with good denoising effect after vector method processing are selected as training data and input into the generator. The generator learns to generate denoised images, and then the denoised images generated by the generator and the original noise-free images are input into the discriminator. The discriminator distinguishes the difference between the denoised images and the original images, and then calculates the loss function between the denoised images and the original images, the loss between the generator and the discriminator, and the discriminant loss. The loss function is then fed back to the generator, and the generator adjusts the learning parameters for a new round of training. Each round of learning thereafter is a continuous cycle, with the generator network and the discriminator network alternately trained, and the parameters of the correction network are adjusted through continuous confrontation between the two networks.

4. The method for denoising a phase contrast OCT image based on a generative adversarial network according to claim 3, characterized in that: The basic principle of generating adversarial networks is expressed as follows: In the formula, G and D represent the generator and discriminator respectively, E represents the expectation, L is the loss function, and p data (u) represents the distribution of real data, p f (f) represents the noise distribution, logD(u) is the probability that the discriminator D correctly identifies the real image, and log(1-D(G(f))) is the probability that D identifies the generated image as a false image; Represents the adversarial process between the generator G and the discriminator D. The generator G minimizes the loss function L in order to generate more realistic denoised images, and the discriminator D maximizes the loss function L in order to more accurately distinguish between real images and generated images.

5. The method for denoising phase-contrast OCT images based on a generative adversarial network according to claim 3, characterized in that: The loss function is used to measure the difference between the generated image and the target real image. It calculates the absolute difference between the pixels of the generated image and the target image and uses it as the loss to promote the similarity between the generated image and the target image. The calculation formula is as follows: L1Loss=(1 / (H*W)∑|S[i,j]-T[i,j]|(2) Among them, H and W represent the height and width of the image respectively, S[i,j] represents the pixel value of the generated image at position [i,j], and T[i,j] represents the pixel value of the target real image at position [i,j].

6. The method for denoising phase-contrast OCT images based on a generative adversarial network according to claim 3, characterized in that: The discriminant loss is a binary cross entropy loss function, which is used to calculate the loss of the discriminator for real images and generated images, and to determine the probability that the generated image is a real image. The formula is as follows: Among them, N represents the number of samples, ∑ represents the sum of all samples, and the output of the discriminator for the real image is D r The output of the generated image is D f ; The target value of 1 represents a real image, and the target value of 0 represents a generated image; by minimizing BCELoss, the discriminator is prompted to output a real image close to 1 and a generated image close to 0, thereby achieving the distinction between real images and generated images.

7. The method for denoising phase-contrast OCT images based on a generative adversarial network according to claim 3, characterized in that: In the downsampling part of the generator's network architecture, the initial downsampling layer consists of a convolutional layer, a batch normalization layer, and a LeakyReLU activation function, which are used to perform initial feature extraction and downsampling of the input image; followed by a series of Block modules for multiple downsamplings; each Block module contains a convolutional layer, a batch normalization layer, and a LeakyReLU activation function, and the step size of the convolutional layer in the Block class is 2 in each dimension, so that the size of the feature map is halved, the number of input channels gradually increases, and the number of output channels gradually doubles, realizing layer-by-layer feature extraction and downsampling; the bottom is the bottleneck layer, which consists of a convolutional layer and a ReLU activation function, which is used to generate a bottleneck feature and reduce the dimensionality of the input feature map.

8. The method for denoising phase contrast OCT images based on generative adversarial networks according to claim 7, characterized in that: The upsampling part of the generator's network architecture includes consecutive Block modules for multiple upsampling; each Block module contains a transposed convolution layer, a batch normalization layer, and a ReLU activation function. The transposed convolution layer is used to achieve upsampling by inserting zero values ​​and applying convolution kernels to increase the size of the feature map without losing details; the number of input channels is gradually halved, and the number of output channels is gradually reduced to achieve layer-by-layer upsampling of features; the final upsampling part consists of a transposed convolution layer and a Tanh activation function, which is used to upsample the last layer of feature maps to the same size as the input image and output the generated image; jump connection, each upsampling block concatenates its input with the downsampling part of the corresponding layer before upsampling to establish a jump connection.

9. The method for denoising phase-contrast OCT images based on a generative adversarial network according to claim 3, characterized in that: In the network architecture of the discriminator, the number of input image channels is 3. The initial convolution part consists of a convolution layer and a LeakyReLU activation function, which is used to perform initial feature extraction and downsample the input image to a number of 32 output channels. The intermediate convolution part consists of multiple CNNBlocks, each of which is used to perform convolution operations and decide whether to downsample and use dropout based on whether it is the last layer. These CNNBlocks are constructed based on the specified number of features, and the number of features gradually increases to achieve layer-by-layer feature extraction. The last convolution layer maps the feature map of the last layer to a scalar for discrimination of binary classification tasks.

Images