A neural network-based OCT image simultaneous denoising and super-resolution reconstruction method

Abstract

The application belongs to the technical field of biomedical and computer vision, and discloses a kind of OCT image denoising and super-resolution reconstruction method based on neural network simultaneously. Image is collected using SD-OCT imaging system, and original image is obtained. Label image is obtained by image registration and averaging algorithm. RD-SRGAN network is used for denoising and super-resolution reconstruction of the original image, and the result is compared with the label image, and then fed back to the network for learning, to finally obtain a high-quality clear image. The RD-SRGAN network designed by the method has small complexity, can be migrated to a computer or a mobile device, and has fast inference speed. The FMDB network and the reinforced channel attention module can extract the most useful features to a certain extent, i.e. have high inference accuracy. The method can simultaneously denoise and super-resolution reconstruct the OCT image at a faster speed.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of biomedical and computer vision, and particularly relates to a method for simultaneously denoising and super-resolution reconstruction of OCT images based on a neural network. BACKGROUND

[0002] Optical coherence tomography (OCT) is a non-contact and non-invasive imaging technology, which is applied in ophthalmology, dermatology and other fields. Its role in retinal disease diagnosis is indispensable. High resolution is one of the significant advantages of OCT, but low coherence interference imaging is greatly affected by speckle noise, which reduces the contrast of the image and blurs the structural features. At the same time, in order to reduce the influence of eyeball non-autonomous movement on imaging, OCT usually needs to reduce the spatial sampling rate to improve the acquisition speed. Low spatial sampling rate will lead to the loss of image details. Therefore, the research on denoising and super-resolution reconstruction of OCT images is of great significance. Not only can it improve the clarity of the image, but also can magnify the texture details and edge structure information of the image, thereby improving the accuracy and reliability of clinical diagnosis and treatment.

[0003] Generally, there are two ways to denoise OCT images: hardware improvement and software optimization. In terms of hardware, the main method is to improve the detector, light source and other devices in the scanning imaging system to improve the imaging quality, including spatial compounding and frequency compounding, etc., but it will undoubtedly increase the complexity and maintenance cost of the device. In terms of software, there are mainly the following methods: statistical filtering, which sorts or averages the pixels in the local neighborhood to suppress noise, but it will lose texture details; wavelet transform, which decomposes the signal into subbands of different frequencies, thereby realizing the analysis and denoising of the signal, but there is a boundary effect; non-local mean (NLM) uses global information to reduce image noise while better preserving image details, but the computational complexity is high.

[0004] Essentially, the traditional image denoising algorithm aims to eliminate high-frequency noise in the image to improve the quality of the image. After denoising, the image edges will inevitably be smoothed and the details will be blurred, resulting in a decrease in resolution. Therefore, denoising and high-resolution reconstruction based on traditional algorithms are to some extent mutually opposed. Simultaneous denoising and super-resolution reconstruction based on deep learning has a significant advantage.

[0005] Based on the above difficulties, how to use the method of deep learning to further improve the quality of OCT images needs further research. SUMMARY

[0006] The purpose of the present application is to provide a method for simultaneously denoising and super-resolution reconstruction of OCT images based on a neural network.

[0007] The technical scheme adopted by the present application is as follows:

[0008] Using the built SD-OCT imaging system to collect images, obtain the original OCT image;

[0009] A number of original OCT images are obtained by registration and averaging algorithm to obtain a label OCT image;

[0010] The generator of the RD-SRGAN network is used to extract the features of the original OCT image and generate a denoising and super-resolution image; the discriminator compares the generated image with the high-quality image, and feeds back the difference to the generator, so that it learns to generate more realistic denoising and super-resolution images.

[0011] The RD-SRGAN network includes a GAN backbone network, an FMDB feature extraction module, an up-sampling module, a reinforced channel attention module, and a relative average discriminator.

[0012] The principle of GAN is based on the zero-sum game idea in game theory. The model is trained by letting the generator and the discriminator play against each other. The goal of the generator is to generate realistic data samples, while the goal of the discriminator is to distinguish between the samples generated by the generator and the real samples. Through the iterative training process, the two modules compete and co-evolve with each other.

[0013] The generator of the classic GAN model receives a random vector as input and generates a data sample similar to the real sample. With the advent of super-resolution generative adversarial network (SRGAN), low-resolution image (LR) is taken as input, and high-resolution image (HR) is taken as real sample. The goal of the model is to find the mapping between LR and HR, which can be represented as:

[0014] G: LR→HR

[0015] The LR image is reconstructed into a super-resolution (SR) image through the learned mapping relationship, and then the discriminator D is used to judge whether the SR is more real than the HR. The process can be described as:

[0016]

[0017] Where y is the HR image, x is the LR image, E is the expectation operator, P h and P l are the data distributions of the HR and LR images, respectively.

[0018] First, the noisy low-resolution (NLR) image is taken as the input of the generator, and after the generator extracts the features, the denoising super-resolution (DSR) image is obtained by up-sampling. Through the discriminator, it is compared with the clean high-resolution (CHR) image to judge whether the CHR image is more real than the DSR image. Through continuous training and optimization of the model, a good DSR image result can be obtained.

[0019] The generator of the RD-SRGAN network is composed of a shallow feature extraction module, a deep feature extraction module, and an up-sampling module, respectively. The input of the generator is an NLR-OCT image. First, a 3x3 convolution block and a ReLU activation function are used to extract the shallow features of the OCT retinal image, including low-frequency information such as retinal contours and layer boundaries. In terms of deep feature extraction of the image, a feature multi-distillation module (FMDB) is proposed, which has two characteristics of channel separation and feature distillation, can fully capture and utilize the spatial and channel information of multi-scale feature maps, increase the extraction ability of retinal features layer by layer, and significantly increase the details after image super-resolution reconstruction. FMDB uses an attention mechanism to guide the GAN training denoising model, uses the learning results of the previous stage to guide the noise learning of the current stage, so as to improve the denoising performance and achieve the best denoising effect.

[0020] With the increase of network depth, the model learns more abstract and complex features, and the generalization ability is also enhanced. However, problems such as gradient disappearance also come with it. Dense connection is a method often used in practical applications to alleviate gradient disappearance, and each FMDB in the model accepts the output of all previous FMDBs as additional input to realize parameter information fusion and feature reuse. This way not only reduces the number of parameters and reduces the complexity of the model, but also effectively prevents overfitting, helps to enhance the model's multi-scale expression ability for input OCT images, extract richer and more useful feature representations, and enhance the denoising effect. Finally, the number of up-sampling modules is set according to the size of the magnification factor, and the DSR image is obtained.

[0021] In order to improve the sharpening ability of the OCT image and highlight the pathological structure of the key part of the retina, the discriminator in the classic GAN network is optimized in the present application, and a relative average discriminator is used to identify the generated image. The DSR image and the CHR image are input into the relative average discriminator, which judges the probability that the given label CHR image is more real than the DSR image after super-resolution reconstruction, and feeds back the difference between the images to the generator learning, so as to continuously optimize it and learn the mapping relationship between different image noise levels and resolutions, so that the generated image is more fitted to the distribution space of the CHR image data; At the same time, the discriminator continuously improves itself by continuously judging which one of the gradually optimized DSR image and CHR image is more real. The ultimate goal is for the generator to output a denoising super-resolution OCT retinal image, thereby improving the image quality and retaining more detailed information, and achieving simultaneous denoising and super-resolution reconstruction of the original OCT image. This training method can help the generator and the discriminator learn from each other to generate better results. BRIEF DESCRIPTION OF DRAWINGS

[0022] Figure 1 A framework diagram for simultaneous denoising and super-resolution reconstruction of OCT images based on a neural network in an embodiment of the present application;

[0023] Figure 2 A structural diagram of SD-OCT in an embodiment of the present application;

[0024] Figure 3 A structural diagram of an RD-SRGAN network in an embodiment of the present application;

[0025] Figure 4 A feature multi-distillation module (FMDB);

[0026] Figure 5 A reinforced channel attention module. DETAILED DESCRIPTION

[0027] Embodiments of the present application are described in detail below with reference to the accompanying drawings, in which the same or similar elements or elements having the same or similar functions are denoted by the same or similar reference numerals throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are intended only for the purpose of explaining the present application, and should not be understood as limiting the present application. For the step numbers in the following embodiments, they are only set for the convenience of explanation, and the order between the steps is not limited in any way, and the execution order of each step in the embodiments can be adaptively adjusted according to the understanding of those skilled in the art.

[0028] In the description of the present application, it should be understood that the orientation description, such as the orientation or position relationship indicated by up, down, front, back, left, right, etc. is based on the orientation or position relationship shown in the drawings, and is only for the convenience of describing the present application and simplifying the description, and does not indicate or imply that the device or element referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore should not be understood as limiting the present application.

[0029] In the description of the present application, the meaning of several is one or more, and the meaning of multiple is more than two, greater than, less than, more than, etc. are understood as not including the number, and above, below, etc. are understood as including the number. If it is described as first, second, it is only for the purpose of distinguishing technical features, and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated or the order of technical features indicated.

[0030] In the description of the present application, unless otherwise explicitly limited, the words such as setting, installing, connecting, etc. should be understood broadly, and those skilled in the art can reasonably determine the specific meaning of the above words in the present application in combination with the specific content of the technical solution.

[0031] Collecting images and achieving simultaneous noise reduction and super-resolution reconstruction of OCT images using SD-OCT, comprising the following steps:

[0032] Step 1: Build an SD-OCT imaging system: a broadband light source emits broadband light, which is coupled into a fiber-based Michelson interferometer through a fiber optic circulator. The optical interference signal between the backscattered light from the probe and the reference arm is transmitted to a spectrometer. By spectrometer, the interference spectrum is imaged into a line-scan camera (CCD), and the Doppler angle θ is quantified using a graduated separation head. This process ultimately obtains several position cardiac OFT structure maps and flow velocity maps.

[0033] The broadband light source is a broadband superluminescent diode (SLD) with a center wavelength of 1310 nm and a bandwidth of 52 nm, which can provide an axial resolution of about 14 μm when illuminating a sample exposed to air. The probe contains a collimator, an X-Y 2D galvanometer scanning system and an objective lens (focal length 50 mm, lateral resolution 16 μm); the spectrometer consists of a collimating mirror (f = 5 mm), a 1145 line / mm transmission grating and a Fourier lens (f = 100 mm); the line-scan camera is a 1024-pixel infrared InGaAs.

[0034] Step 2: Construct the RD-SRGAN network, which includes a GAN backbone network, an FMDB feature extraction module, an up-sampling module, an enhanced channel attention module, and a relative average discriminator.

[0035] The FMDB feature extraction module includes a convolution layer and a residual connection layer, wherein the convolution layer includes 3x3 convolution, Relu activation function and batch normalization, and the residual connection layer includes 1x1 convolution, Relu activation function and batch normalization.

[0036] The enhanced channel attention module (ECA) is an attention mechanism used to improve the performance of neural networks. It aims to improve the network's ability to model channel features and help improve the network's performance in various visual tasks. The ECA module enhances the attention between channels by introducing global context information and the correlation between channels to optimize the feature extraction process. Specifically, the ECA module consists of two main parts: an adaptive global average pooling operation and a channel attention mechanism. The global average pooling operation is used to capture global context information, while the channel attention mechanism is used to calculate the importance weight of each channel, so that the network can better focus on features that are beneficial to the current task. By introducing the ECA module, the number of network parameters and computational cost can be effectively reduced, and the network's perception ability and generalization ability can be improved.

[0037] The relative average discriminator is a technique for improving the training of a generative adversarial network (GAN). In the relative average discriminator, the objective of the discriminator is to assess the authenticity of the generated samples relative to the real samples and output a measure of this relativity. Specifically, for a given real sample x and a generated sample G(z), the relative average discriminator no longer simply outputs a scalar discriminant score, but outputs a relative discriminant score representing the authenticity of the generated sample relative to the real sample. This relative treatment can help the generator better understand the difference between real samples and generated samples, thereby improving the effect of training. The introduction of the relative average discriminator is believed to be able to accelerate the convergence speed of the GAN and improve the diversity and realism of the generated samples.

[0038] Step 3: Set the loss function for network training, which has three parts: content loss, adversarial loss, and regression loss. The regression loss uses L1 loss.

[0039] Step 4: Preprocess and resize the structural diagram collected in step 1 to a uniform size. In this example, images of size 450x450 can be used, but are not limited to, and the four images collected at the same position are registered and averaged to obtain the label image.

[0040] Step 5: Encapsulate the original image and the label image into a dataset, and iteratively train until the network converges to obtain the trained RD-SRGAN network.

[0041] Step 6: Load the trained RD-SRGAN network weight, input the low-resolution noisy image, and perform noise reduction and super-resolution reconstruction.

Claims

1. A method for simultaneous denoising and super-resolution reconstruction of OCT images based on neural networks, characterized in that, Specifically as follows: Images were acquired using the constructed SD-OCT imaging system to obtain raw OCT images; Labeled OCT images are obtained from several original OCT images using registration and averaging algorithms; The RD-SRGAN network's generator extracts features from the original OCT image to generate denoised and super-resolution images. The discriminator compares the generated image with a high-quality image, feeding the differences back to the generator so it can learn to generate more realistic denoised and super-resolution images. The network mainly includes a GAN backbone network, an FMDB feature extraction module, an upsampling module, an enhanced channel attention module, and a relative averaging discriminator. The noisy low-resolution (NLR) image is used as the generator's input. After the generator extracts features, it is upsampled to obtain the denoised super-resolution (DSR) image. The discriminator compares this image with a clean high-resolution (CHR) image to determine whether the CHR image is more realistic than the DSR image. By continuously training and optimizing the model, good DSR image results can be obtained.

2. The method for simultaneous denoising and super-resolution reconstruction of OCT images based on neural networks according to claim 1, characterized in that, The generator of the RD-SRGAN network consists of a shallow feature extraction module, a deep feature extraction module, and an upsampling module. The input to the generator is an NLR-OCT image. First, a 3x3 convolutional block and a ReLU activation function are used to extract shallow features of the OCT retinal image, including low-frequency information such as retinal contours and layer boundaries. For deep feature extraction, a feature multiple distillation module, FMDB, is proposed. FMDB uses an attention mechanism to guide the GAN training of the noise reduction model, using the learning results of the previous stage to guide the noise learning in the current stage, thereby improving the noise reduction performance and achieving the best noise reduction effect.

3. The method for simultaneous denoising and super-resolution reconstruction of OCT images based on neural networks according to claim 2, characterized in that, The feature multiple distillation module has two characteristics: channel separation and feature distillation. It can fully capture and utilize the spatial and channel information of multi-scale feature maps, increase the ability to extract retinal features layer by layer, and significantly increase the details after image super-resolution reconstruction.

4. The method for simultaneous denoising and super-resolution reconstruction of OCT images based on neural networks according to claim 2, characterized in that, The enhanced channel attention module enhances the ability to model the relationship between different channels in the feature map by introducing a channel attention mechanism. It mainly includes two key components: global average pooling first performs a global average pooling operation on the input feature map to capture global contextual information, which helps to provide a wider range of contexts. Channel attention uses global information to calculate the importance weight of each channel, so that the network can better focus on features that are beneficial to the current task. The whole process enables the network to better understand the relationship between different channels while maintaining parameter efficiency, and to make more effective use of feature representations.

5. The method for simultaneous denoising and super-resolution reconstruction of OCT images based on neural networks according to claim 2, characterized in that, The relative average discriminator calculates the relative authenticity between generated samples and real samples and outputs a relative discrimination score, which represents the authenticity of the generated sample relative to the real sample.

Images