OCT image simultaneous noise reduction and super-resolution reconstruction method based on neural network

Abstract

The invention belongs to the technical field of biomedicine and computer vision, and discloses an OCT image simultaneous noise reduction and super-resolution reconstruction method based on a neural network. An SD-OCT imaging system is used for collecting an image, and an original image is obtained. And obtaining a label image through an image registration and averaging algorithm. The RD-SRGAN network is used for carrying out noise reduction and super-resolution reconstruction on an original image, comparing a result with a label image, and feeding back the result to the network for learning, thereby finally obtaining a high-quality clear image. The RD-SRGAN network designed by the method is low in complexity, can be migrated to a computer or mobile equipment, and is high in reasoning speed, and the FMDB network and the enhanced channel attention module can extract the most useful features to a certain extent, namely, the RD-SRGAN network has relatively high reasoning precision. According to the method, the OCT image can be subjected to noise reduction and super-resolution reconstruction at the same time at a high speed.

Description

Technical Field

[0001] The present invention relates to the fields of biomedicine and computer vision technology, and in particular to a method for simultaneous noise reduction and super-resolution reconstruction of OCT images based on a neural network. Background Art

[0002] Optical coherence tomography (OCT) is a non-contact, non-invasive imaging technology used in ophthalmology, dermatology and other fields. Its role in the diagnosis of retinal diseases has become indispensable. High resolution is one of the significant advantages of OCT, but low-coherence interference imaging is greatly affected by speckle noise, which reduces the contrast of the image and blurs the structural features. At the same time, in order to reduce the impact of involuntary eye movement on imaging, OCT usually needs to reduce the spatial sampling rate to increase the acquisition speed. Low spatial sampling rate will lead to loss of image details. Therefore, the research on denoising and super-resolution reconstruction of OCT images is of great significance. It can not only improve the clarity of the image, but also amplify the texture details and edge structure information of the image, thereby improving the accuracy and reliability of clinical diagnosis and treatment.

[0003] In general, there are two main ways to reduce noise in OCT images: hardware improvement and software optimization. In terms of hardware, the main approach is to improve the detectors and light sources in the scanning imaging system to improve the imaging quality, including spatial compounding and frequency compounding, but this will undoubtedly increase the complexity and maintenance cost of the equipment. In terms of software, there are mainly the following methods: statistical filtering suppresses noise by sorting or averaging pixels in the local neighborhood, but texture details will be lost; wavelet transform decomposes the signal into sub-bands of different frequencies to achieve signal analysis and noise reduction, but there are boundary effects; non-local mean (NLM) uses global information to reduce image noise, while better retaining image details, but the computational complexity is high.

[0004] In essence, traditional image denoising algorithms aim to remove high-frequency noise in images to improve image quality. Noise reduction will inevitably make image edges smooth and details blurred, resulting in a decrease in resolution. Therefore, denoising based on traditional algorithms and high-resolution reconstruction are mutually exclusive to a certain extent. However, simultaneous denoising and super-resolution reconstruction based on deep learning have significant advantages.

[0005] Based on the above difficulties, further research is needed on how to use deep learning methods to further improve the quality of OCT images. Summary of the invention

[0006] The purpose of the present invention is to propose a method for simultaneous denoising and super-resolution reconstruction of OCT images based on a neural network.

[0007] The technical solution adopted by the present invention is specifically as follows:

[0008] Use the built SD-OCT imaging system to collect images and obtain original OCT images;

[0009] Several original OCT images are registered and averaged to obtain labeled OCT images;

[0010] The generator of the RD-SRGAN network is used to extract features from original OCT images and generate denoised and super-resolution images; the discriminator compares the generated images with high-quality images and feeds the differences back to the generator, allowing it to learn to generate more realistic denoised and super-resolution images.

[0011] The RD-SRGAN network includes a GAN backbone network, an FMDB feature extraction module, an upsampling module, an enhanced channel attention module, and a relative average discriminator;

[0012] The principle of GAN is based on the zero-sum game idea in game theory. The model is trained by letting the generator and the discriminator compete with each other. The goal of the generator is to generate realistic data samples, while the goal of the discriminator is to distinguish between samples generated by the generator and real samples. These two modules compete and co-evolve with each other through repeated iterative training processes.

[0013] The generator of the classic GAN model receives a random vector as input and generates a data sample similar to the real sample. With the emergence of super-resolution generative adversarial network (SRGAN), low-resolution images (LR) are used as input and high-resolution images (HR) are used as real samples. The goal of the model is to find the mapping between LR and HR, which can be expressed as:

[0014] G:LR→HR

[0015] The LR image is reconstructed into a super-resolution (SR) image through the learned mapping relationship, and then the discriminator D is used to determine whether the SR is more realistic than the HR. The process can be described as:

[0016]

[0017] Among them, y is the HR image, x is the LR image, E is the expected operator, P h and P l Data distributions of HR and LR images, respectively.

[0018] First, the noisy low-resolution (NLR) image is used as the input of the generator. After the generator extracts features, it is upsampled to obtain a denoised super-resolution (DSR) image. The discriminator compares it with the clean high-resolution (CHR) image to determine whether the CHR image is more realistic than the DSR image. By continuously training and optimizing the model, good DSR image results can be obtained.

[0019] The generator of the RD-SRGAN network consists of a shallow feature extraction module, a deep feature extraction module, and an upsampling module. The input of the generator is the NLR-OCT image. First, a 3x3 convolution block and a ReLU activation function are used to extract the shallow features of the OCT retinal image, including low-frequency information such as the retinal contour and layer boundaries. In terms of deep feature extraction of the image, a feature multi-distillation module (FMDB) is proposed, which has two characteristics: channel separation and feature distillation. It can fully capture and utilize the spatial and channel information of multi-scale feature maps, increase the ability to extract retinal features layer by layer, and significantly increase the details after super-resolution reconstruction of the image. FMDB uses an attention mechanism to guide the GAN to train the denoising model, and uses the learning results of the previous stage to guide the noise learning of the current stage, so as to improve the denoising performance and achieve the best denoising effect.

[0020] As the network depth increases, the model learns more abstract and complex features, and its generalization ability is also enhanced. However, problems such as gradient vanishing also follow. Dense connection is a method often used in practical applications to alleviate gradient vanishing. Each FMDB in the model will accept the output of all previous FMDBs as its additional input to achieve parameter information fusion and feature reuse. This method can not only reduce the number of parameters and the complexity of the model, but also effectively prevent overfitting, help enhance the model's multi-scale expression ability of the input OCT image, extract richer and more useful feature representations, and enhance the denoising effect. Finally, the number of upsampling modules is set according to the size of the magnification factor to obtain the DSR image.

[0021] In order to improve the sharpening ability of OCT images and highlight the pathological structure of key parts of the retina, the present invention optimizes the discriminator in the classic GAN network and uses a relative average discriminator to identify the generated images. Both the DSR image and the CHR image are input into the relative average discriminator to determine the probability that the given labeled CHR image is more realistic than the DSR image after super-resolution reconstruction, and the image difference is fed back to the generator for learning, so as to continuously optimize it, learn the mapping relationship between different image noise levels and resolutions, and make the generated image more fit the distribution space of CHR image data; at the same time, the discriminator continuously improves itself by continuously judging which of the gradually optimized DSR image and CHR image is more realistic. The ultimate goal is that the generator can output a denoised super-resolution OCT retinal image, thereby improving image quality and retaining more detail information, and realizing simultaneous denoising and super-resolution reconstruction of the original OCT image. This training method can help the generator and the discriminator learn from each other to generate better results. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 A framework diagram of simultaneous noise reduction and super-resolution reconstruction of OCT images based on a neural network in an example of the present invention;

[0023] Figure 2 Schematic diagram of the structure of SD-OCT in an example of the present invention;

[0024] Figure 3 Schematic diagram of the RD-SRGAN network structure in an example of the present invention;

[0025] Figure 4 It is the feature multiple distillation module (FMDB);

[0026] Figure 5 To strengthen the channel attention module. DETAILED DESCRIPTION

[0027] The embodiments of the present invention are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and are not to be construed as limitations of the present invention. For the step numbers in the following embodiments, they are only provided for the convenience of explanation, and the order between the steps is not limited in any way, and the execution order of each step in the embodiment can be adaptively adjusted according to the understanding of those skilled in the art.

[0028] In the description of the present invention, it should be understood that descriptions involving orientations, such as up, down, front, back, left, right, etc., and orientations or positional relationships indicated are based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present invention.

[0029] In the description of the present invention, "several" means one or more, "more" means more than two, "greater than", "less than", "exceed" etc. are understood as not including the number itself, and "above", "below", "within" etc. are understood as including the number itself. If there is a description of "first" or "second", it is only used for the purpose of distinguishing the technical features, and cannot be understood as indicating or implying the relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features.

[0030] In the description of the present invention, unless otherwise clearly defined, terms such as setting, installing, connecting, etc. should be understood in a broad sense, and technicians in the relevant technical field can reasonably determine the specific meanings of the above terms in the present invention based on the specific content of the technical solution.

[0031] Using SD-OCT to acquire images and implement simultaneous noise reduction and super-resolution reconstruction of OCT images includes the following steps:

[0032] Step 1: Build the SD-OCT imaging system: A broadband light source emits broadband light, which is coupled into a fiber-based Michelson interferometer through a fiber circulator. The optical interference signal between the light backscatter from the probe and the reference arm is transmitted to the spectrometer. Through the spectrometer, the interference spectrum is imaged into a line scan camera (CCD), and the Doppler angle θ is quantified using a graduated divider head. This process ultimately obtains cardiac OFT structural and flow velocity maps at several locations.

[0033] The broadband light source is a broadband superluminescent diode (SLD) with a central wavelength of 1310nm and a bandwidth of 52nm, which can provide an axial resolution of about 14μm when irradiating a sample exposed to air. The probe includes a collimator, an X-Y2D galvanometer scanning system and an objective lens (focal length 50mm, lateral resolution 16μm); the spectrometer consists of a collimator (f=5mm), a 1145line / mm transmission grating and a Fourier lens (f=100mm); the line scan camera is a 1024-pixel infrared InGaAs.

[0034] Step 2: Construct the RD-SRGAN network, which includes a GAN backbone network, an FMDB feature extraction module, an upsampling module, an enhanced channel attention module, and a relative average discriminator.

[0035] The FMDB feature extraction module includes a convolution layer and a residual connection layer, wherein the convolution layer includes a 3×3 convolution, a Relu activation function and a batch normalization, and the residual connection layer includes a 1×1 convolution, a Relu activation function and a batch normalization.

[0036] The Enhanced Channel Attention (ECA) module is an attention mechanism for improving the performance of neural networks. It aims to improve the network's ability to model channel features, which helps to improve the network's performance in various visual tasks. The ECA module optimizes the feature extraction process by enhancing the attention between channels by introducing global context information and the association between channels. Specifically, the ECA module consists of two main parts: an adaptive global average pooling operation and a channel attention mechanism. The global average pooling operation is used to capture global context information, while the channel attention mechanism is used to calculate the importance weight of each channel, so that the network can better focus on features that are beneficial to the current task. By introducing the ECA module, the number of parameters and computational cost of the network can be effectively reduced, and the perception and generalization capabilities of the network can be improved.

[0037] The relative average discriminator (RelativisticAverage Discriminator) is a technique for improving the training of generative adversarial networks (GANs). In the relative average discriminator, the goal of the discriminator is to evaluate the authenticity of generated samples relative to real samples and output a measure of this relativity. Specifically, for a given real sample x and a generated sample G(z), the relative average discriminator no longer simply outputs a scalar discriminant score, but outputs a relative discriminant score indicating the authenticity of the generated sample relative to the real sample. This relative processing can help the generator better understand the difference between real samples and generated samples, thereby improving the training effect. The introduction of the relative average discriminator is believed to accelerate the convergence speed of GAN and improve the diversity and fidelity of generated samples.

[0038] Step 3: Set the loss function for network training. The loss function has three parts: content loss, adversarial loss, and regression loss. The regression loss uses L1 loss.

[0039] Step 4: Preprocess the structure image collected in step 1 and resize it to a uniform size. This example can use but is not limited to images of 450×450 size, and align and average the four images collected at the same position to obtain a label image.

[0040] Step 5: Encapsulate the original image and the labeled image into a data set, iterate the training until the network converges, and obtain the trained RD-SRGAN network.

[0041] Step 6: Load the trained RD-SRGAN network weights, input the low-resolution noisy image, and perform denoising and super-resolution reconstruction on it.

Claims

1. A method for simultaneous denoising and super-resolution reconstruction of OCT images based on a neural network, characterized in that: The details are as follows: Use the built SD-OCT imaging system to collect images and obtain original OCT images; Several original OCT images are registered and averaged to obtain labeled OCT images; The generator of the RD-SRGAN network is used to extract features from original OCT images and generate denoised and super-resolution images; the discriminator compares the generated images with high-quality images and feeds the differences back to the generator, allowing it to learn to generate more realistic denoised and super-resolution images.

2. The method for simultaneous denoising and super-resolution reconstruction of OCT images based on a neural network according to claim 1, characterized in that: The RD-SRGAN network includes a GAN backbone network, an FMDB feature extraction module, an upsampling module, an enhanced channel attention module, and a relative average discriminator; the noisy low-resolution (NLR) image is used as the input of the generator, and after the generator extracts features, it is upsampled to obtain a denoised super-resolution (DSR) image, which is compared with a clean high-resolution (CHR) image through the discriminator to determine whether the CHR image is more realistic than the DSR image. Good DSR image results can be obtained by continuously training and optimizing the model. The generator of the RD-SRGAN network consists of a shallow feature extraction module, a deep feature extraction module, and an upsampling module. The input of the generator is the NLR-OCT image. First, a 3x3 convolution block and a ReLU activation function are used to extract the shallow features of the OCT retinal image, including low-frequency information such as the retinal contour and layer boundaries. In terms of deep feature extraction of the image, a feature multi-distillation module (FMDB) is proposed. FMDB uses an attention mechanism to guide the GAN to train the denoising model, and uses the learning results of the previous stage to guide the noise learning of the current stage, so as to improve the denoising performance and achieve the best denoising effect.

3. The method for simultaneous denoising and super-resolution reconstruction of OCT images based on a neural network according to claim 2, characterized in that: The FMDB module has two characteristics: channel separation and feature distillation. It can fully capture and utilize the spatial and channel information of multi-scale feature maps, increase the ability to extract retinal features layer by layer, and significantly increase the details of the image after super-resolution reconstruction.

4. The method for simultaneous denoising and super-resolution reconstruction of OCT images based on a neural network according to claim 2, characterized in that: The enhanced channel attention module enhances the modeling capability of the relationship between different channels in the feature map by introducing the channel attention mechanism. It mainly includes two key components: global average pooling first performs a global average pooling operation on the input feature map to capture global context information, which helps to provide a wider context; Channel attention uses global information to calculate the importance weight of each channel so that the network can better focus on features that are beneficial to the current task. The whole process enables the network to better understand the relationship between different channels while maintaining parameter efficiency and to use feature representation more effectively.

5. The method for simultaneous denoising and super-resolution reconstruction of OCT images based on a neural network according to claim 2, characterized in that: The relative average discriminator calculates the relative authenticity between the generated samples and the real samples and outputs a relative discrimination score. This score represents the authenticity of the generated samples relative to the real samples. This relative processing helps the training process to be more stable, because the generator needs to generate samples that not only deceive the discriminator, but also deceive the discriminator into thinking that the generated samples are more real than the real samples.

Images