OCT image unpaired denoising methods, devices, and media

Abstract

The application relates to an OCT image unpaired denoising method, device and medium, comprising the following steps: obtaining a noisy OCT image, inputting a trained unpaired denoising model, and obtaining a denoised OCT image; the unpaired denoising model comprises a self-noise auxiliary module, a noise change learning module, a generator and a discriminator, the unpaired denoising model is based on unpaired OTC image data, and is trained with the minimum model loss as the target; the self-noise auxiliary module is used for extracting multiple noises from the noise OCT image, the noise change learning module is used for generating a mixed basic noise according to the multiple noises extracted by the self-noise auxiliary module, the model loss comprises L1 loss between multiple groups of OTC image pairs, each group of OTC image pairs comprises a clean OTC image and a corresponding denoised OTC image of the clean OTC image, and the corresponding denoised OTC image of the clean OTC image is generated based on the mixed basic noise and the generator. Compared with the prior art, the denoising capacity and generalization capacity of the application are stronger.

Description

Technical Field

[0001] This invention belongs to the field of medical image processing technology, and in particular relates to a method, device and medium for unpaired noise reduction of OCT images. Background Technology

[0002] Optical coherence tomography (OCT) uses the principle of weakly coherent light interference to perform two-dimensional or three-dimensional imaging based on the different light reflections of different tissues. It is a non-invasive and non-destructive imaging method widely used in clinical diagnosis in ophthalmology and dermatology. However, due to the high scattering properties of biological tissues, speckle noise is inevitably introduced during imaging, reducing image quality and affecting subsequent clinical diagnosis.

[0003] Mainstream OCT denoising methods can be divided into two categories: those based on hardware improvements and those based on image processing. Since improving the OCT structure to reduce imaging noise inevitably leads to increased structural complexity and cost, current developments tend to favor image processing methods for denoising. When paired data is available, supervised learning methods can be used for denoising. However, large-scale datasets of paired noisy and clean OCT images suitable for deep learning are difficult to obtain. Training with small datasets carries the risk of overfitting, resulting in poor generalization performance and difficulty in effectively denoising.

[0004] When large-scale paired datasets are unavailable, training with unpaired datasets or unsupervised methods is a more ideal approach. Unsupervised methods attempt to utilize the inherent information of noisy images for denoising, assuming that the background noise of an OCT image is pure noise and consistent with the noise distribution in the details of the OCT image, thus using background noise to guide the denoising of the entire OCT image. However, this approach presents two problems in practical applications: first, distinguishing between background and details requires manual screening, which is very time-consuming and labor-intensive; second, the assumption that the background is pure noise is uncertain, and without correctly learning the noise distribution, denoising may be incomplete or structural information may be incorrectly removed. Unpaired methods utilize unrelated clean and noisy images for training. Currently, most unpaired OCT denoising methods are based on the CycleGAN framework. Researchers use CycleGAN and its variants to transform noisy OCT styles to clean OCT styles. However, this method requires two generators and two discriminators. For example, Chinese invention patent CN113781366B discloses a deep learning-based method for sharpening eye OCT images. The CycleGAN model consists of two generator models and two discriminator models forming a bidirectional ring structure, which is difficult to train. Furthermore, relying solely on neural networks to extract noise can lead to the neural network having no concept of noise distribution, resulting in incomplete noise extraction and incorrect extraction of structural information, while completely failing to utilize the image's inherent information.

[0005] In summary, existing OCT image denoising methods all have some drawbacks, which can lead to problems such as poor generalization performance, incomplete denoising, incorrect removal of structural information, difficulty in model training, and difficulty in finding the correct learning direction for neural networks. Therefore, it is necessary to design an OCT image unpaired denoising method with stronger denoising and generalization capabilities. Summary of the Invention

[0006] The purpose of this invention is to overcome the shortcomings of the prior art by providing an unpaired noise reduction method, device, and medium for OCT images, thereby better eliminating noise in OCT images.

[0007] The objective of this invention can be achieved through the following technical solutions:

[0008] This invention provides a method for unpaired noise reduction of OCT images, comprising the following steps:

[0009] Noisy OCT images are acquired, preprocessed, and then input into a trained unpaired denoising model to obtain denoised OCT images.

[0010] The unpaired denoising model is built on a generative adversarial network and includes a self-noise assist module, a noise change learning module, a generator, and a discriminator. The unpaired denoising model is iteratively trained based on noisy OTC images and clean OTC images with no pairing relationship, with the goal of minimizing model loss.

[0011] The self-noise assist module is used to extract various noises from the noisy OCT image. The noise change learning module is used to weight and sum the various noises extracted by the self-noise assist module to generate a mixed base noise. The generator is used to predict the speckle noise corresponding to its input image. The discriminator is used to determine the authenticity of its input noise. The model loss includes L1 loss between multiple OTC image pairs. Each OTC image pair includes a clean OTC image and its corresponding denoised OTC image. The denoised OTC image corresponding to the clean OTC image is generated based on the mixed base noise and the generator.

[0012] Furthermore, the preprocessing includes image normalization.

[0013] Furthermore, the noise-assisted module extracts noise from the noisy OCT image through wavelet transform and singular value decomposition to obtain various types of noise.

[0014] Furthermore, after reconstructing the extracted noise using a pixel block scrambling method, the noise is then input into the noise change learning module to generate mixed base noise.

[0015] Furthermore, in the noise change learning module, the sum of the weight coefficients of various noises is 1, and the variation range of each weight coefficient is [0,1].

[0016] Furthermore, a bias is set for the range of variation of each weighting coefficient.

[0017] Furthermore, the generator is constructed based on a deep convolutional neural network without residual structures, and the discriminator is PatchGAN.

[0018] Furthermore, the generator is used to predict speckle noise corresponding to the input image, and the model loss also includes the loss of the generator and the loss of the discriminator.

[0019] The present invention also provides an electronic device, including a memory, a processor, and a program stored in the memory, wherein the processor executes the program to implement the above-described method.

[0020] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described method.

[0021] Compared with the prior art, the present invention has the following beneficial effects:

[0022] 1. This invention achieves OCT image denoising through an unpaired denoising model. The unpaired denoising model is built upon a generative adversarial network and includes a self-noise auxiliary module, a noise variation learning module, a generator, and a discriminator. The self-noise auxiliary module extracts various noises from noisy OCT images, enabling the model to learn the mathematical distribution of speckle noise in OCT images more quickly and accurately. The noise variation learning module weights and sums the various noises extracted by the self-noise auxiliary module to generate a mixed base noise, which enhances the model's generalization ability. During the training process of the unpaired denoising model, there is no need to use paired noisy and clean OCT images. The denoised OCT image corresponding to the clean OCT image can be generated based on the mixed base noise and the generator, forming an image pair with the clean OCT image. The model loss is constructed based on the L1 loss between the image pairs, and iterative training is performed with the goal of minimizing the model loss. The unpaired denoising model built and trained using the above method has better adaptability to unknown or changing noise conditions and can further improve the denoising ability of noisy OCT images.

[0023] 2. This invention extracts noise from noisy OCT images by using wavelet transform and singular value decomposition to obtain various types of noise. It can utilize the information of the noisy OCT image itself to extract noise that is close to the mathematical distribution of the real speckle noise, thus assisting in the training of the unpaired denoising model.

[0024] 3. In the noise change learning module of the present invention, the sum of the weight coefficients of various noises is 1, the variation range of each weight coefficient is [0,1], and a bias is set for the variation range of each weight coefficient, which can ensure that the mixed noise is not a single type of noise, and can improve the generalization performance of the unpaired denoising model in extracting noise. Attached Figure Description

[0025] Figure 1 This is a schematic diagram of the structure and training process of an unpaired denoising model.

[0026] Where NI is the noisy OTC image, noise w is the noise extracted by wavelet transform, noise s is the noise extracted by singular value decomposition, and α and β are coefficients;

[0027] Figure 2 This is a schematic diagram of the generator's structure;

[0028] Figure 3 This is a schematic diagram of the discriminator's structure;

[0029] Figure 4The images are before and after denoising using the method of the present invention, where (4a) is the OTC image before denoising and (4b) is the OTC image after denoising. Detailed Implementation

[0030] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. These embodiments are based on the technical solution of the present invention and provide detailed implementation methods and specific operating procedures. However, the scope of protection of the present invention is not limited to the following embodiments.

[0031] Example:

[0032] This embodiment provides a method for unpaired noise reduction of OCT images, including the following steps:

[0033] Noisy OCT images are acquired, preprocessed, and then input into a trained unpaired denoising model to obtain denoised OCT images.

[0034] Unpaired denoising models are built based on generative adversarial networks, such as Figure 1 As shown, it includes a self-noise assist module, a noise variation learning module, a generator, and a discriminator. The self-noise assist module denoises the noisy OCT image using wavelet transform and singular value decomposition, respectively, and then subtracts the denoised image from the noisy OCT image to obtain the self-noise. w and noise s noise s and noise w The size is consistent with the image size. The two types of noise mentioned above are very similar to the distribution of speckle noise in OCT images, but they contain structural information. Therefore, this embodiment uses pixel patch shuffle to reconstruct the noise, eliminating the structural information and ultimately obtaining the self-noise used to assist neural network training. w and noise s It should be noted that the self-noise assist module is not limited to wavelet transform and singular value decomposition. Theoretically, other denoising methods that utilize the image's own information are also applicable to the self-noise assist module.

[0035] If the noise remains unchanged, the noise extracted by the neural network will be biased towards a mixture of noise extracted by wavelet transform and singular value decomposition. Therefore, the final optimal state of the removed noise can only infinitely approximate that mixed noise, failing to become a more accurate noise denoising method and instead resembling an accelerated denoising method. This invention solves this problem through a noise change learning module, which assigns weights to and sums the various noises extracted by the self-noise auxiliary module to generate a mixed basic noise. baseThis is used to subsequently add to clean OCT images for model training, mixing in basic noise. base The specific expression is as follows:

[0036] noise base =α*noise s +β*noise w

[0037] Here, α and β are random values. The noise added is changed by two variable coefficients α and β so that it is not limited to a certain type of speckle noise. β is equal to (1-α) and takes values ​​in the range [0,1].

[0038] Considering that this could lead to α and β taking the value of 0, the noise guiding the neural network would be pure noise. s or noise w In order to make the added noise base It won't be complete noise. s or noise w In this embodiment, a bias of 0.4 is added to α and β:

[0039] noise base = (α + 0.4 * noise) s +(β+0.4*noise w

[0040] The generator G predicts the speckle noise corresponding to its input image, and the discriminator D determines the authenticity of the input noise. The generator is a deep convolutional neural network without residual structures, and its structure is as follows: Figure 2 As shown. The discriminator chosen is PatchGAN, whose structure is as follows. Figure 3 As shown, the advantages of PatchGAN are that it can consider the relationship between local image patches, model high-frequency information, improve computational efficiency, and have flexibility.

[0041] The unpaired denoising model is iteratively trained to minimize the model loss, with a learning rate of 1×10⁻⁴. The Adam optimizer is used to optimize the neural network of the model, and training is performed for 100 epochs. The specific training process is as follows:

[0042] First, obtain a set of noisy OTC images NI and a set of clean OTC images CI without pairing relationships, and adjust the number of images in the two sets to be the same (based on the number of noisy OTC images; if there are more clean OTC images, select them randomly; if there are fewer clean OTC images, copy and add them randomly to the set). Adjust the numerical range of the OTC images to between 0 and 1, and adjust the image size to a fixed value. In this embodiment, 640×640 is used as an example.

[0043] Secondly, the self-noise auxiliary module processes the input noisy image NI and outputs noise. w and noise s In each training round, noise will be w and noise s Input noise variation learning module, output random noise. base Based on noise base The generator generates three sets of image pairs: a clean image CI and a first predicted image CI_BN_C, a clean image CI and a second predicted image CI_NG_C, and a predicted clean image NI_C and a third predicted image NI_C_BN_C.

[0044] The generation process of the first predicted image CI_BN_C is as follows:

[0045] Mixed basic noise base Add the noisy OTC image CI to the clean OTC image CI to obtain the noisy OTC image CI_BN. Input the noisy OTC image CI_BN into the generator to generate the corresponding speckle noise G(CI_BN). Subtract the speckle noise G(CI_BN) from the noisy OTC image CI_BN to obtain the first predicted image CI_BN_C.

[0046] The generation process of the second predicted image CI_NG_C is as follows:

[0047] The noisy OTC image NI is input into the generator to generate the corresponding speckle noise G(NI). The speckle noise G(NI) is added to the clean OTC image CI to obtain the noisy OTC image CI_NG. The noisy OTC image CI_NG is input into the generator to generate the corresponding speckle noise G(CI_NG). The speckle noise G(CI_NG) is subtracted from the noisy OTC image CI_NG to obtain the second predicted image CI_NG_C.

[0048] The process of generating the predicted clean image NI_C is as follows:

[0049] Input the noisy OTC image NI into the generator to generate the corresponding speckle noise G(NI). Subtract the speckle noise G(NI) from the noisy OTC image NI to obtain the predicted clean image NI_C.

[0050] The generation process of the third predicted image NI_C_BN_C is as follows:

[0051] Mixed basic noise basa Add the noisy OTC image NI_C to the clean prediction image NI_C to obtain the noisy OTC image NI_C_BN. Input the noisy OTC image NI_C_BN into the generator to generate the corresponding speckle noise G(NI_C_BN). Subtract the speckle noise G(NI_C_BN) from the noisy OTC image NI_C_BN to obtain the third prediction image NI_C_BN_C.

[0052] The three image groups CI and CI_BN_C, CI and CI_NG_C, and NI_C and NI_C_BN_C are image pairs between clean images (or predicted clean images) and their corresponding predicted clean images. The image pair loss is calculated using the following formula:

[0053] Loss=|image clean -image pred_ |

[0054] The model loss includes not only the image pair loss, but also the generator loss and the discriminator loss. The calculation method is the same as that for GAN loss calculation, that is, the generator wants to minimize the loss and the discriminator wants to maximize the loss.

[0055] LOSS GAN =E[logD(image) real )]+E[log 1-D(image fake ))]

[0056] The model training process can be represented by the following algorithm:

[0057]

[0058]

[0059] After model training, the noisy OCT image to be processed is normalized, that is, the numerical range is adjusted to between 0 and 1. The size of the noisy OCT image can be arbitrary. The image is input into the generator, and the generator outputs the predicted noise. The predicted noise is subtracted from the noisy OCT image to obtain a clean OCT image. The numerical range of the clean OCT image is adjusted to between 0 and 255, and it is saved as a TIFF image to obtain the final inference result. The OCT images before and after denoising are shown below. Figure 4 As shown in (4a) and (4b).

[0060] Compared to existing methods, the proposed method does not require additional data preprocessing and can be trained using arbitrary unpaired data, improving the denoising capability of noisy OCT images. It can better remove speckle noise while preserving clear structure. The proposed self-noise auxiliary module enables the neural network to learn the mathematical distribution of speckle noise faster and more accurately, while the noise variation learning module enhances the generalization ability of the neural network.

[0061] If the above methods are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0062] The above description of the embodiments is provided to enable those skilled in the art to understand and use the invention. It will be apparent to those skilled in the art that various modifications can be made to these embodiments, and the general principles described herein can be applied to other embodiments without inventive effort. Therefore, the present invention is not limited to the above embodiments, and any improvements and modifications made by those skilled in the art based on the disclosure of the present invention without departing from the scope of the invention should be within the protection scope of the present invention.

Claims

1. A method for unpaired noise reduction of OCT images, characterized in that, Includes the following steps: Noisy OCT images are acquired, preprocessed, and then input into a trained unpaired denoising model to obtain denoised OCT images. The unpaired denoising model is built on a generative adversarial network and includes a self-noise assist module, a noise change learning module, a generator, and a discriminator. The unpaired denoising model is iteratively trained based on noisy OTC images and clean OTC images with no pairing relationship, with the goal of minimizing model loss. The self-noise assist module is used to extract various noises from the noisy OCT image. The noise change learning module is used to weight and sum the various noises extracted by the self-noise assist module to generate a mixed base noise. The generator is used to predict the speckle noise corresponding to its input image. The discriminator is used to determine the authenticity of its input noise. The model loss includes L1 loss between multiple OTC image pairs. Each OTC image pair includes a clean OTC image and its corresponding denoised OTC image. The denoised OTC image corresponding to the clean OTC image is generated based on the mixed base noise and the generator.

2. The OCT image unpaired noise reduction method according to claim 1, characterized in that, The preprocessing includes image normalization.

3. The OCT image unpaired noise reduction method according to claim 1, characterized in that, The noise-assisted module extracts noise from the noisy OCT image through wavelet transform and singular value decomposition to obtain various types of noise.

4. The OCT image unpaired noise reduction method according to claim 3, characterized in that, After reconstructing the extracted noise using a pixel block scrambling method, the noise is then input into the noise change learning module to generate mixed base noise.

5. The OCT image unpaired noise reduction method according to claim 1, characterized in that, In the noise change learning module, the sum of the weight coefficients of various noises is 1, and the range of each weight coefficient is [0,1].

6. The OCT image unpaired noise reduction method according to claim 5, characterized in that, Set a bias for the range of variation of each weight coefficient.

7. The OCT image unpaired noise reduction method according to claim 1, characterized in that, The generator is built on a deep convolutional neural network without residual structure, and the discriminator is PatchGAN.

8. The OCT image unpaired noise reduction method according to claim 1, characterized in that, The model loss also includes the generator loss and the discriminator loss.

9. An electronic device comprising a memory, a processor, and a program stored in the memory, characterized in that, When the processor executes the program, it implements the method as described in any one of claims 1-8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1-8.

Images