Retina OCT image denoising method based on zero sample learning

Abstract

The invention discloses a retina OCT (Optical Coherence Tomography) image denoising method based on zero sample learning. The method comprises the following steps: generating a noise independent image pair for a single noisy image by adopting a CDIS (Coherent Discrete Identifier) The method comprises the following steps of: carrying out de-noising by using an MLFSnet network (AF-MSDSConv, Aap-LSM, SA, a reconstruction module); using RMSE symmetry and consistency joint loss zero sample training; in the reasoning stage, an original image is directly input, and a same-resolution de-noising result is output. The speckle noise can be significantly suppressed and the layered structure of the retina can be retained without noise-free true values or pairwise data, so that the retina can be used while being shot.

Description

Technical Field

[0001] This invention relates to a denoising method for retinal OCT images based on zero-shot learning. Background Technology

[0002] Optical coherence tomography (OCT) has become the standard imaging method for the clinical diagnosis and treatment of retinal diseases due to its non-contact and high-resolution advantages. However, OCT uses a coherent imaging mechanism, and speckle noise is unavoidable, leading to blurred image details and reduced contrast of layered structures, directly affecting doctors' quantitative judgment of the thickness of each retinal layer and the morphology of lesion areas. Traditional OCT image denoising methods can be divided into hardware and software categories. Hardware methods reduce noise by optimizing the OCT system hardware architecture, but usually increase the system's complexity and cost. Software methods rely on post-processing algorithms: traditional filtering (median, Gaussian, Wiener) is prone to losing small structures while removing noise; methods based on sparse representation, low-rank matrices, or transform domains (wavelet, curvelet) are computationally complex and have poor adaptability to speckle statistical characteristics.

[0003] While deep learning methods have achieved significant performance improvements, supervised learning schemes require a large amount of noise-free ground truth data, which is extremely costly to obtain. Unsupervised learning schemes, while not requiring ground truth images as labels for training, still require a lot of noisy image data for training.

[0004] Therefore, how to achieve high-quality noise reduction using only a single acquired image without any training data has become a pressing technical challenge in the field of OCT image processing. Summary of the Invention

[0005] The present invention provides a retinal OCT image denoising method based on zero-shot learning to solve the problems existing in the prior art.

[0006] The technical solutions adopted in this invention are as follows:

[0007] A retinal OCT image denoising method based on zero-shot learning includes the following steps:

[0008] S1: Input only one noisy raw retinal OCT image, and perform two complementary downsampling operations on the raw retinal OCT image through the feature differential image sampling module to obtain two low-resolution image pairs with independent noise distributions and shared noise-free target content;

[0009] S2: Input the image pair into a pre-constructed MLFSnet network, the network comprising: a multi-scale convolutional feature extraction module AF-MSDSConv, an improved local sparsity module Adap-LSM, a spatial attention module SA, and a reconstruction module;

[0010] S3: The MLFSnet network is trained with zero samples using a joint loss function consisting of symmetry loss and consistency loss, both of which are measured by root mean square error. After training, the original retinal OCT image is input into the MLFSnet network, and a denoised retinal OCT image with the same resolution as the original image is output.

[0011] Furthermore, the feature-differentiated image sampling module performs first downsampling and second downsampling on a single original retinal OCT image using a complementary checkerboard downsampling method, generating two low-resolution noise images, a first low-resolution noise image and a second low-resolution noise image, with spatial resolution reduced by half, noise distributions independent of each other, and sharing noise-free target content. The first low-resolution noise image and the second low-resolution noise image form the low-resolution image pair in S1.

[0012] Furthermore, the multi-scale convolutional feature extraction module AF-MSDSConv sets up three branches in parallel: 3×3, 5×5, and 7×7 depth separable dilated convolutions. Each branch is followed by a 1×1 convolution with a uniform 64 channels, and the features are weighted and fused by learnable weights normalized by SoftMax to output multi-scale features.

[0013] Furthermore, the improved local sparsity module Adap-LSM is an improvement on the original LSM module, and the improvements are as follows:

[0014] Improvement 1: In the fixed threshold generation method of the LSM module, introduce a learnable parameter, adaptive. param This allows the threshold to adapt to the input data.

[0015] Improvement 2: Replace the single sigmoid mask of the LSM module with a sigmoid mask adapted by learnable parameters. param A dynamic mask, scaled in conjunction with the channel mean, enables dynamic sparsity adjustment;

[0016] Improvement 3: Add a clamp operation after the LSM module to force the mask value to the [0,1] interval, ensuring the effectiveness of the sparse mask.

[0017] Furthermore, the learnable parameter adaptive param It is a single learnable scalar with an initial value of 1. Its numerical range is automatically updated through backpropagation and is multiplied with the channel mean to adjust the sparsity of the sparse mask.

[0018] Furthermore, the spatial attention module SA takes the H×W×256 feature map inside the MLFSnet network as input, obtains the spatial weight map through global average pooling, 1×1 convolution and Sigmoid, and multiplies it pixel by pixel with the input H×W×256 feature map.

[0019] Furthermore, the reconstruction module consists of five 3×3 convolutional layers connected in series, each followed by a ReLU activation function. The final layer outputs 1 channel with a stride of 1, which is used to map the feature map into a single-channel denoised image with the same resolution as the original.

[0020] The present invention has the following beneficial effects:

[0021] Beneficial effects of the present invention

[0022] (1) Only a single clinically acquired noisy OCT image is needed, without any noise-free ground truth or paired noise data, to achieve high-quality denoising and meet the requirement of "instant use".

[0023] (2) The CDIS sampling strategy designed for the statistical characteristics of OCT speckle generates “noise-independent-content-consistent” image pairs, making the N2N idea applicable for the first time in a single-image scenario, and the speckle suppression rate is better than that of traditional unsupervised methods.

[0024] (3) AF-MSDSConv multi-scale hole separable convolution + adaptive weighting can maintain the sharpness of the boundaries of each layer of the retina while denoising; Adap-LSM dynamic sparsification further removes redundancy and improves contrast; SA spatial attention strengthens key areas to ensure that small lesions are not lost. Attached Figure Description

[0025] Figure 1a This is a schematic diagram illustrating the principle of feature-differentiated image sampling.

[0026] Figure 1b Two low-resolution noisy images with different noise distributions are obtained by sampling the original noisy image using two different templates.

[0027] Figure 2 This is a diagram of the MLFSnet network structure.

[0028] Figure 3 This is the overall flowchart for unsupervised training.

[0029] Figure 4 The denoising results for the ORI retinal OCT image dataset are shown below:

[0030] (a) Original noisy image; (b) Multi-frame average ground truth image; (c) BM3D; (d) U-net; (e) ZS-N2N; (f) U-net-ZD denoised image; (g) MLFS-ZD.

[0031] Figure 5 The noise reduction results are for the PKU37 retinal OCT image dataset.

[0032] (a) Original noisy image; (b) Multi-frame average ground truth image; (c) BM3D; (d) U-net; (e) ZS-N2N; (f) U-net-ZD; (g) MLFS-ZD. Detailed Implementation

[0033] The invention will now be further described with reference to the accompanying drawings.

[0034] This invention discloses a retinal OCT image denoising method based on zero-shot learning, comprising the following steps:

[0035] S1: Input only one noisy raw retinal OCT image, and perform two complementary downsampling operations on the raw retinal OCT image through the feature differential image sampling module to obtain two low-resolution image pairs with independent noise distributions and shared noise-free target content;

[0036] S2: Input the image pair into a pre-constructed MLFSnet network, the network comprising: a multi-scale convolutional feature extraction module AF-MSDSConv, an improved local sparsity module Adap-LSM, a spatial attention module SA, and a reconstruction module;

[0037] S3: The MLFSnet network is trained with zero samples using a joint loss function consisting of symmetry loss and consistency loss, both of which are measured by root mean square error. After training, the original retinal OCT image is input into the MLFSnet network, and a denoised retinal OCT image with the same resolution as the original image is output.

[0038] The steps of the present invention will be described in further detail below.

[0039] 1) Zero-shot denoising principle;

[0040] Image denoising tasks based on deep learning are considered to be regression models, which can be represented as:

[0041] ,

[0042] in, This means minimizing the objective function to obtain the corresponding variables. Values, The mapping function representing the neural network. This represents an input noisy image. Represents the truth image. This represents the expected value of the observed object. Represents the loss function. Represents the parameters of the neural network. Ignore noisy input image. With the truth image The relationship between them can be defined as follows:

[0043] ,

[0044] As the formula shows, theoretically, neural networks can transform noisy images. and truth image The optimization process of the loss function is independent. Therefore, it can be represented by an image where the expected value equals the true value. Replacing the true value with noisy data can ensure that the fitting effect of the neural network remains unchanged.

[0045] The noise in OCT images has a mean of 0 in large samples, making it possible to use noisy images... For the true image Replacement is performed. At this point, if both the input and label images are noisy images, the network's objective function can be expressed as:

[0046] ,

[0047] in, and This refers to pairs of noisy images that are independent of each other and share the same noise-free target. This invention addresses this by analyzing single noisy images... Two downsampling operations in different directions were performed to obtain... and This process can be represented as:

[0048] ,

[0049] like Figure 1a As shown, to adapt to the speckle noise denoising unique to OCT images, this invention designs a Characteristic-Divergent Image Sampling (CDIS) method to achieve zero-sample denoising. Among other things, and These represent two different downsampling modules. (Noisy image) pass and A set of noisy image pairs was obtained and If the noise is independent and there is the same noise-free target, noise can be removed using an N2N denoising strategy.

[0050] The original noisy image is processed through two different sampling templates and This yields two low-resolution noisy images (N1 and N2) with different noise distributions, as follows: Figure 1b .

[0051] 2) MLFSnet network;

[0052] This invention proposes a novel denoising network architecture, MLFSnet (Multi-Scale Feature Spatial Attention Network), which aims to combine with the proposed zero-shot training strategy to achieve efficient zero-shot denoising tasks.

[0053] The network design of this invention combines the multi-scale convolutional feature extraction module AF-MSDSConv, the improved local sparsity module Adap-LSM, the spatial attention module SA, and the reconstruction module. Through the synergistic effect of these innovative modules, the denoising effect is significantly improved while preserving the details and texture of the image.

[0054] (21) Multi-scale convolutional feature extraction module AF-MSDSConv;

[0055] The multi-scale convolutional feature extraction module AF-MSDSConv, based on the traditional multi-scale convolutional feature extraction module, incorporates dilated convolution and separable convolution, and employs adaptive fusion weights to fuse features from different scales, further enhancing the network's adaptability to noise at different scales. Specifically, it includes the following components:

[0056] Input layer: The input image has dimensions H×W×C, where H and W are the height and width of the image, and C is the number of channels in the image.

[0057] Convolutional Layer 1: First, a 3×3 depthwise separable convolution kernel is used to process the input image. This depthwise separable convolution separates the channels, reducing the number of parameters. Next, a 1×1 convolution kernel is used to adjust the number of channels to 64. Finally, the ReLU activation function is used to introduce non-linearity, aiming to extract low-level features of the image while preserving finer-grained information.

[0058] Convolutional Layer 2: A 5×5 dilated convolutional kernel with a dilation rate of 2 is used. This allows the kernel to expand the receptive field without increasing the number of parameters, enabling it to capture features from larger image regions and adapt to larger-scale noise. Similarly, this depthwise separable convolution processes the channels separately. A 1×1 convolutional kernel is then used to adjust the number of channels to 64. Finally, a non-linearity is introduced through the ReLU activation function.

[0059] Convolutional layer 3: A 7×7 dilated convolution kernel with a dilation rate of 2 is used to further expand the receptive field, capture more local information, and ensure the ability to handle more complex noise patterns. This is also a depthwise separable convolution. A 1×1 convolution kernel is then used to adjust the number of channels to 64. Finally, the ReLU activation function is used to introduce nonlinearity.

[0060] Adaptive fusion weights: Defines a learnable parameter `self.fusion`. weights It is initialized as a tensor with all 1s and a length of 3, corresponding to the output of the three convolution branches.

[0061] Feature fusion: In the previous method, the three convolutional branches process the input x to obtain out1, out2, and out3 respectively. Then, self.fusion is applied... weights Perform a softmax operation to obtain adaptive fusion weights, and then sum the outputs of the three branches according to these weights (weights[0] * out1 + weights[1] * out2 + weights[2] * out3) to finally obtain the fused feature output.

[0062] (22) Improved local sparsification module Adap-LSM;

[0063] This part improves upon the Local Sparserization (LSM) module by incorporating adaptive sparsity adjustment. This aims to more effectively reduce redundant information in images, further enhancing the network's computational efficiency and generalization ability. Specifically, it includes the following components:

[0064] Input layer: The size of the input feature map is H×W×256.

[0065] Sparse convolutional layer: Uses a 1×1 convolutional kernel, with 256 input and 256 output channels. This convolutional layer is used for preliminary processing of the input feature map, preparing it for the subsequent generation of the sparse mask.

[0066] Adaptive parameter adjustment: A learnable adaptive parameter, `adaptive`, is added to the module. paramThe initial value is 1. This parameter is used to adaptively adjust the sparsity when generating sparse masks to better adapt to different data features.

[0067] Sparse mask generation: By applying the sigmoid function to the output of the sparse convolutional layer, a sparse mask in the range [0, 1] is generated. This mask will be used to control which parts of the input feature map need to be preserved or suppressed.

[0068] Feature mean calculation: Calculate the mean of the input feature map in the height and width dimensions to obtain a mean feature in the channel dimension. This mean feature will participate in the subsequent sparsity adjustment process.

[0069] Adaptive sparsity adjustment: The generated sparse mask is multiplied by the adaptive parameters and the mean feature to obtain the adjusted mask. This allows the sparsity to be adaptively adjusted according to the mean of the input feature map, enabling the module to dynamically control the degree of information retention based on the characteristics of the data.

[0070] Mask value range constraint: Use the clamp function to restrict the adjusted mask value to the range [0, 1] to ensure the validity of the mask.

[0071] Output layer: The input feature map is multiplied element-wise with the adjusted mask to obtain the final output feature map. This process removes some redundant information from the input feature map, allowing the network to focus more on the important signals.

[0072] The improved local sparsity module Adap-LSM introduces an adaptive sparsity adjustment mechanism, which can dynamically adjust the sparsity based on the characteristics of the input data, thereby more flexibly reducing redundant information and improving network performance.

[0073] (23) Spatial attention mechanism module (SA);

[0074] The Spatial Attention (SA) module dynamically adjusts the feature weights of different spatial regions, enabling the network to adaptively focus on key information regions in the image, thereby improving denoising performance. Specifically, it includes the following components:

[0075] Input layer: The feature map F (size H×W×256) input from the previous layer is used as input.

[0076] Global average pooling: First, perform global average pooling to obtain global information of the image and obtain a feature map of size 1×1×256.

[0077] Convolution operation: A 1×1 convolution kernel is applied to the feature map after global average pooling, and the number of output channels is 256, which is used to generate spatial attention weights.

[0078] Sigmoid activation: After convolution, the Sigmoid activation function is used to generate a weight map between 0 and 1, with a size of H×W×256.

[0079] Weighted Feature Map: The generated spatial attention weight map is multiplied by the input feature map to obtain a weighted feature map. This allows the network to focus on important regions in the image and suppress the influence of irrelevant regions.

[0080] The Spatial Attention (SA) module significantly improves the model's flexibility and adaptability in denoising tasks.

[0081] (24) Rebuild module;

[0082] The reconstruction module's role is to map the denoised feature maps back to the spatial domain of the original image. The reconstruction module consists of five identical convolutional layers with a ReLU activation function:

[0083] Convolutional layer: A 3×3 convolution kernel with a stride of 1 and 1 output channel is used to obtain the final denoised image.

[0084] Activation function: The ReLU activation function is used in the last layer to map the output to a non-negative range, ensuring that the pixel values ​​of the recovered image are within the valid range.

[0085] The reconstruction module successfully combines the denoising process with the image reconstruction process by compressing multi-channel feature maps into a single-channel output image, ensuring that the details and textures of the denoised image are preserved to the greatest extent.

[0086] Figure 2 This is a diagram of the MLFSnet network structure.

[0087] (3) Loss function;

[0088] The Neighbor2Neighbor unsupervised denoising strategy works by combining the consistency loss and symmetry loss of the noisy image to obtain a joint loss function. This invention builds upon this by using RMSE to construct the loss function, as follows:

[0089] Input noisy image Through the network of this invention Generating simulated image noise ,use and The difference can represent the output denoised image. This process can be represented as:

[0090] ,

[0091] This is essentially a residual learning process. This invention uses a network to train and fit the noise distribution of the input noisy image, and then performs subtraction to achieve the purpose of denoising.

[0092] ,

[0093] As shown in the above equation, the loss function of this invention consists of a symmetric loss: Consistency loss: It consists of two parts.

[0094] ,

[0095] Symmetry loss: This invention uses the root mean square error (RMSE) to calculate the loss between the denoised image obtained after one downsampling and the noisy image after another downsampling. This is believed to play a regularization role during training.

[0096] ,

[0097] Consistency loss: Similarly, the root mean square error (RMSE) is used to calculate the loss between downsampling first and then denoising through the network versus denoising through the network first and then downsampling under the two downsampling methods. The purpose of this is to ensure that a common denoising target is achieved. Figure 3 This is the overall process for unsupervised training.

[0098] 4) Comparison of experimental results;

[0099] To verify the denoising effect of the proposed MLFS-ZD zero-shot denoising scheme on retinal OCT images, this invention compares it with the following typical methods: BM3D, ZS-N2N, U-net-ZD using the proposed zero-shot strategy, and U-net trained with supervised learning. In the experimental results below, bold data indicates the best results, and underlined data indicates the second best results.

[0100] Table 1 Comparison of experimental results for the ORI retinal OCT image dataset

[0101]

[0102] Based on the data shown in Table 1, the proposed zero-shot denoising method MLFS-ZD demonstrates superior performance across several key metrics: it outperforms the zero-shot method ZS-N2N in PSNR (33.84dB), SSIM (0.9168), ENL (5766.38), and EPI (0.87), with ENL and EPI being the best. Despite the lack of supervised learning benchmarks, MLFS-ZD still shows an advantage in SSIM compared to the supervised learning method U-net (0.9072), and its ENL is significantly higher than U-net, demonstrating its excellent noise suppression and structure recovery capabilities. Compared to the zero-shot variant U-net-ZD, MLFS-ZD shows significant advantages in PSNR, ENL, and EPI, further validating the effectiveness of the proposed network architecture. This method achieves denoising performance close to supervised learning through zero-shot constraints. Figure 4 Judging from the results, MLFS-ZD has a more significant effect on denoising the background area, while other methods still show some noise in the background area after magnification.

[0103] Table 2 Comparison of experimental results for the PKU37 retinal OCT image dataset

[0104]

[0105] According to the data in Table 2, MLFS-ZD demonstrated excellent performance on the PKU37 small-sample retinal OCT dataset: PSNR (29.47 dB) was the best across the entire dataset, representing a 14.5% improvement over ZS-N2N (25.73 dB), another zero-sample method; ENL (975.74) was 6.9 times that of ZS-N2N, indicating strong noise suppression capabilities even in data-scarce scenarios. On the edge preservation index EPI (0.66), this method surpassed supervised learning U-net (0.58), achieving a 13.8% improvement, demonstrating MLFS-ZD's advantage in structure restoration. In medical small-sample scenarios, MLFS-ZD, through a multi-scale feature fusion mechanism, achieved a synergistic improvement in noise suppression (ENL), edge preservation (EPI), and structure restoration (SSIM), providing an efficient solution for rare disease image analysis without labeled data. Figure 5 The results show that MLFS-ZD has excellent edge preservation, very good background noise reduction, and complete preservation of the structural information of the retinal layer.

[0106] The above description is only a preferred embodiment of the present invention. It should be noted that those skilled in the art can make several improvements without departing from the principle of the present invention, and these improvements should also be considered within the scope of protection of the present invention.

Claims

1. A retinal OCT image denoising method based on zero-shot learning, characterized in that: Includes the following steps: S1: Input only one noisy raw retinal OCT image, and perform two complementary downsampling operations on the raw retinal OCT image through the feature differential image sampling module to obtain two low-resolution image pairs with independent noise distributions and shared noise-free target content; S2: Input the image pair into a pre-constructed MLFSnet network, the network comprising: a multi-scale convolutional feature extraction module AF-MSDSConv, an improved local sparsity module Adap-LSM, a spatial attention module SA, and a reconstruction module; S3: The MLFSnet network is trained with zero samples using a joint loss function consisting of symmetry loss and consistency loss, both of which are measured by root mean square error. After training, the original retinal OCT image is input into the MLFSnet network, and a denoised retinal OCT image with the same resolution as the original image is output.

2. The retinal OCT image denoising method based on zero-shot learning as described in claim 1, characterized in that: The feature-differentiated image sampling module performs first and second downsampling on a single original retinal OCT image using a complementary checkerboard downsampling method, generating two low-resolution noise images, a first low-resolution noise image and a second low-resolution noise image, with their spatial resolution reduced by half, noise distributions independent of each other, and sharing noise-free target content. The first low-resolution noise image and the second low-resolution noise image form the low-resolution image pair in S1.

3. The retinal OCT image denoising method based on zero-shot learning as described in claim 1, characterized in that: The multi-scale convolutional feature extraction module AF-MSDSConv sets up three branches in parallel: 3×3, 5×5, and 7×7 depth separable dilated convolutions. Each branch is followed by a 1×1 convolution with a uniform 64 channels, and then weighted and fused by learnable weights normalized by SoftMax to output multi-scale features.

4. The retinal OCT image denoising method based on zero-shot learning as described in claim 1, characterized in that: The improved local sparsity module Adap-LSM is an improvement on the original LSM module, and the improvements are as follows: Improvement 1: In the fixed threshold generation method of the LSM module, introduce a learnable parameter, adaptive. param This allows the threshold to adapt to the input data. Improvement 2: Replace the single sigmoid mask of the LSM module with a sigmoid mask adapted by learnable parameters. param A dynamic mask, scaled in conjunction with the channel mean, enables dynamic sparsity adjustment; Improvement 3: Add a clamp operation after the LSM module to force the mask value to the [0,1] interval, ensuring the effectiveness of the sparse mask.

5. The retinal OCT image denoising method based on zero-shot learning as described in claim 4, characterized in that: The learnable parameter adaptive param It is a single learnable scalar with an initial value of 1. Its numerical range is automatically updated through backpropagation and is multiplied with the channel mean to adjust the sparsity of the sparse mask.

6. The retinal OCT image denoising method based on zero-shot learning as described in claim 1, characterized in that: The spatial attention module (SA) takes the H×W×256 feature map inside the MLFSnet network as input, and obtains a spatial weight map through global average pooling, 1×1 convolution and sigmoid, and multiplies it pixel by pixel with the input H×W×256 feature map.

7. The retinal OCT image denoising method based on zero-shot learning as described in claim 1, characterized in that: The reconstruction module consists of five 3×3 convolutional layers connected in series. Each convolutional layer is followed by a ReLU activation function. The last layer outputs 1 channel with a stride of 1, which is used to map the feature map into a single-channel denoised image with the same resolution as the original.

Images