Motion artifact reduction in optical coherence tomography angiography images

Abstract

Optical coherence tomography angiography (OCTA) imaging is susceptible to artifacts caused by eye movements, including displacement artifacts, repeated-scan artifacts, and white-line artifacts. A two-stage framework for sequentially removing the three types of artifacts in degraded OCTA images is developed. In the first stage, a hierarchical transformer model, termed HFormer, leverages global features, vertical line features, and intra-vertical line features of the degraded OCTA image to remove displacement artifacts and repeated-scan artifacts, resulting in an intermediate OCTA image. In the second stage, a residual conditional diffusion model (RCDM) removes white-line artifacts by reconstructing the artifact regions of the intermediate OCTA image that overlap with white-line artifacts while avoiding modifying the residual regions of the intermediate OCTA image that do not overlap with white-line artifacts.

Description

Cross-references to related applications

[0001] This application claims priority and benefit to U.S. Provisional Patent Application Serial No. 63 / 624,295, filed January 24, 2024, the disclosure of which is incorporated herein by reference in its entirety. Abbreviations

[0002] A-MSA Axial Multi-Head Self-Attention C-MSA Channel Multi-Head Self-Attention CNN (Convolutional Neural Network) Computer Vision (CV) FFN feedforward network HTM hierarchical converter model IA-MSA In-Axis Multi-Head Self-Attention LFFN (Local Feedforward Network) MAE (Mean Absolute Error) MLP (Multilayer Perceptron) MSA Bullish Self-Attention Optical coherence tomography (OCT) OCTA (Optical Coherence Tomography) angiography OSAT proceeds gradually PSNR (Peak Signal-to-Noise Ratio) RCDM residual conditional diffusion model SSIM Structural Similarity Index Technical Field

[0003] This disclosure generally relates to image processing of medical images. In particular, this disclosure relates to a two-stage deep learning-based image processing technique for mitigating eye movement artifacts in OCTA images. Background Technology

[0004] OCTA is a non-invasive imaging technique widely used for retinal vascular imaging[1]. This technique utilizes motion-contrast imaging to rapidly generate angiographic images based on high-resolution volumetric blood flow data. In particular, OCTA is obtained by extracting the variance within multiple repeated B-scans at cross-sectional locations[2],[3]. The application of OCTA has shown great potential in the diagnosis of various retinal diseases, including diabetic retinopathy[4], age-related macular degeneration[5], and glaucoma[6].

[0005] However, the limited raster scanning speed of B-scans makes them susceptible to motion artifacts caused by various movements, including eye movements and blinking[7]. These motion artifacts have been observed in OCTA images with an incidence of 93.1%[8], which can significantly affect the quality of frontal OCTA images, potentially interfering with their visualization, interpretation and analysis, and ultimately affecting the accuracy of clinical diagnosis[9].

[0006] Based on differences in imaging principles, motion artifacts can be classified into displacement artifacts, repetition artifacts, and white line artifacts

[10] . Displacement artifacts occur when eye movements occur during raster scanning, resulting in discontinuities in vertical image segments originating from different retinal regions

[11] . Repetition artifacts arise from repeated scanning of the same retinal region, resulting in repeated vertical image segments[8]. White line artifacts represent themselves as vertical lines in OCTA images, which may lead to misinterpretations of underlying vascular structures

[12] ,

[13] .

[0007] Recently, much research has been dedicated to addressing the problems caused by various artifacts in OCTA by improving the scanning stability and speed, including the development of high-speed cameras

[14] , advanced scanning mirrors

[15] and eye-tracking devices

[16] . However, these hardware-based solutions are often associated with implementation complexity and high acquisition costs, which have limited their widespread adoption

[17] . In contrast, software-based methods offer a cost-effective and competitive alternative. Classical software-based methods utilize statistical analysis

[18] -

[20] and registration techniques

[21] ,

[22] to improve the quality of OCTA images. However, these classical methods often exhibit limited generalization ability to motion artifacts and may even introduce additional artifacts in some cases

[21] ,

[23] . Recent advances in deep learning have led to innovative strategies aimed at mitigating white line artifacts in OCTA images

[12] ,

[24] -

[26] . Although these deep learning methods outperform classical methods in performance, some of these methods require additional vessel segmentation masks to obtain clear OCTA images

[25] ,

[26] . It is noteworthy that all these deep learning methods to date have primarily targeted white line artifacts

[12] ,

[24] -

[26] . This specific focus is a significant limitation, as existing deep learning techniques often neglect other essential motion artifacts in OCTA images, such as displacement and repetitive scan artifacts. Therefore, there is an urgent need to create a holistic solution that addresses all three types of motion artifacts to generate high-quality OCTA images.

[0008] One potential solution to displacement and rescan artifacts is to employ an improved Unet

[27] ,

[28] or transformer model

[29] -

[31] from computer vision. However, as shown in Table I, these methods perform poorly because they ignore the unique properties of these artifacts, such as orientation mismatch and discontinuities. To mitigate these problems, we developed HFormer, a novel model specifically designed to better represent orientation and local features in OCTA images. This makes it very effective in handling displacement and rescan artifacts.

[0009] For white line artifact removal, well-designed generative models in computer vision, such as transformers

[32] and diffusion models

[33] , have shown promise. However, as shown in Table I (which contains experimental results obtained for some existing deep learning models), the effectiveness of these generative models is limited when applied directly to OCTA images. These methods often struggle to capture the critical nuances necessary for accurate white line artifact removal, resulting in oversmoothing.

[0010] There is a need in this field for a unique deep learning technique that can be applied to mitigate displacement artifacts, repetitive scan artifacts, and white line artifacts in OCTA images in order to generate high-quality OCTA images. Summary of the Invention

[0011] The mathematical equations referenced in this invention can be found in the specific implementation methods.

[0012] One aspect of this disclosure provides a computer-implemented method for reducing eye movement artifacts in degraded OCTA images to produce enhanced quality OCTA images. The eye movement artifacts to be reduced include displacement artifacts, repetition artifacts, and white line artifacts.

[0013] The method includes: setting an HTM configured to remove displacement artifacts and duplicate scan artifacts from a first input OCTA image to produce a first output OCTA image; setting an RCDM configured to remove white line artifacts from a second input OCTA image to produce a second output OCTA image; using the HTM to remove any displacement artifacts and any duplicate scan artifacts from the degraded OCTA image to produce an intermediate OCTA image; and using the RCDM to remove any white line artifacts from the intermediate OCTA image to produce a quality-enhanced OCTA image.

[0014] The HTM includes multiple transformer blocks. These transformer blocks are configured to employ self-attention to capture multiple features that can be used to remove the displacement artifacts and repetition artifacts from the first input OCTA image. Preferably and advantageously, the multiple features include global features, vertical line features, and in-vertical line features of the first input OCTA image.

[0015] In summary, the HTM is configured to use the plurality of features (including global features, vertical line features, and in-vertical line features of the first input OCTA image) to remove displacement artifacts and duplicate scan artifacts from the first input OCTA image.

[0016] In some embodiments, the HTM has a U-shaped hierarchical network structure that incorporates skip connections between the encoder block and decoder block of the HTM.

[0017] In some embodiments, the input feature map X is processed to produce an output feature map. The individual converter block comprises an MSA module and an LFFN, such that the individual converter block is calculated from X according to equations (6) and (7). .

[0018] In some embodiments, the LFFN is formed by two parallel convolutional blocks, each of which includes a 1x1 point-wise convolution layer and a 3x3 depth-wise convolution layer.

[0019] In some embodiments, the plurality of transformer blocks include: a first plurality of transformer blocks for capturing the global features; a second plurality of transformer blocks for capturing the vertical line features; and a third plurality of transformer blocks for capturing features within the vertical line.

[0020] In some embodiments, each of the first plurality of converter blocks is configured to use channel MSA to capture the global features.

[0021] In some embodiments, each of the second plurality of transformer blocks is configured to use axial MSA to capture the vertical line feature.

[0022] In some embodiments, each of the third plurality of transformer blocks is configured to use in-axial MSA to capture the in-vertical feature.

[0023] In some embodiments, the HTM is trained to minimize the loss function given by equation (1).

[0024] In some embodiments, the setup of the HTM includes training the HTM before it is used to process the degraded OCTA image.

[0025] Preferably, the HTM is implemented as HFormer.

[0026] In some embodiments, the RCDM is a diffusion model configured to optimize the likelihood of correctly reconstructing a first region of the second input OCTA image that overlaps with the white line artifact, wherein the reconstruction of the first region is conditioned on image data of a second region of the second input OCTA that does not overlap with the white line artifact.

[0027] In summary, the RCDM is configured to reconstruct the first region while avoiding modification of the second region to remove the white line artifact from the second input OCTA image.

[0028] In some embodiments, the RCDM setup includes training the RCDM before it is used to process intermediate OCTA images.

[0029] A computational system for reducing eye movement artifacts in degraded OCTA images to produce enhanced quality OCTA images can be implemented based on the disclosed method. Specifically, the computational system includes one or more computers configured to perform the process of reducing the eye movement artifacts in the degraded OCTA images to produce the enhanced quality OCTA images, according to any embodiment of the disclosed method.

[0030] Other aspects of this disclosure are illustrated by the examples described below. Attached Figure Description

[0031] Figure 1 A schematic diagram illustrating the OSAT framework is shown, in which: the left side of the diagram shows the architecture of HFormer in stage 1, the specific global transformer block and the axial transformer block; and the right side shows the architecture of RCDM in stage 2, as well as the residual block and the self-attention block.

[0032] Figure 2 Subgraphs (a)-(d) depict schematic diagrams illustrating exemplary architectures of self-attention, wherein: subgraph (a) relates to pixel-wise self-attention; subgraph (b) relates to channel self-attention; subgraph (c) relates to axial self-attention; and subgraph (d) relates to in-axial self-attention.

[0033] Figure 3 The diagram illustrates the forward diffusion process q (from left to right) of gradually adding Gaussian noise to the white line region, and the process with x... c The reverse reasoning process p (from right to left) for conditionally iteratively denoising the white line region.

[0034] Figure 4 Exemplary schematic diagrams of the FFN and LFFN disclosed herein are depicted.

[0035] Figure 5 The results of different motion artifact removal methods on real clinically damaged OCTA images are shown graphically.

[0036] Figure 6 A flowchart is depicted, illustrating exemplary steps of the computer-implemented method disclosed herein for reducing eye movement artifacts in degraded OCTA images to produce quality-enhanced OCTA images.

[0037] Those skilled in the art will understand that the elements in the accompanying drawings are shown for simplicity and clarity and are not necessarily drawn to scale. Detailed Implementation

[0038] As used herein, the term “avoid” (or “avoiding”) means any method of partially or completely preventing, deterring, eliminating, preemptively stopping, halting, hindering, or delaying the occurrence of the consequences or phenomena that follow the word “avoid”. The term “avoid” does not imply that it is necessarily absolute, but rather that it is effective in providing some degree of avoidance, prevention, or improvement of the consequences or phenomena that follow the word “avoid”.

[0039] To address the various challenges mentioned above in removing displacement artifacts, repetition artifacts, and white line artifacts from OCTA images, this disclosure develops a two-stage framework for sequentially removing these three types of eye movement artifacts. This two-stage framework is named OSAT (i.e., Stepwise). The disclosed OSAT includes several novel designs, including an HFormer for removing displacement and repetition artifacts and an RCDM for removing white line artifacts.

[0040] The HFormer is tailored for OCTA images by enhancing the overall feature representation across global, local, and vertical dimensions. This design enables the HFormer to capture fine details and complex textures, integrating information from multiple viewpoints and levels of detail for more effective artifact removal.

[0041] As mentioned above, existing methods for white line artifact removal often struggle to capture the critical subtle details necessary for accurate removal, resulting in over-smoothing. To address this challenge, a novel method, RCDM, has been developed that integrates contextual information to enhance attention to detail reconstruction within white line artifacts in OCTA images. RCDM effectively utilizes contextual information in OCTA images during iterative removal of white line artifacts, ensuring the preservation of critical pathological details while effectively targeting and removing artifacts.

[0042] This disclosure brings advancements to the field in three aspects. First, a two-stage framework, OSAT, is proposed for sequentially removing displacement artifacts, repetitive scan artifacts, and whiteline artifacts. Second, a novel transformer, HFormer, is proposed. HFormer is used as the first-stage method in OSAT to remove displacement and repetitive scan artifacts. The hierarchical self-attention module in HFormer enhances the global feature representation across global, local, and vertical extents. Third, in the second stage of OSAT, a novel diffusion model called RCDM is proposed. RCDM is used to remove whiteline artifacts. RCDM strategically utilizes contextual information as a condition, focusing its attention on whiteline regions and enhancing its ability to fill damaged areas.

[0043] 1. Work related to issuing OSAT 1.1. Deep Learning for Motion Artifact Removal in OCTA Deep learning methods have gained significant attention for motion artifact removal in OCTA images. Gao et al.

[24] adopted a modified U-Net architecture and incorporated low-rank matrix constraints and total variation norm constraints into the loss function, effectively mitigating white line artifacts. Similar methods using low-rank approximation and total variation have also been proposed for denoising OCT images

[34] and OCTA images

[35] . Under the ComNet framework

[25] , a two-stage convolutional neural network was designed for white line artifact removal and vessel segmentation in OCTA images. Ren et al.

[26] proposed a self-supervised method called CABR, in which the reconstruction model is trained on clean regions and vessel segmentation annotations are used during inference to efficiently remove white line artifacts. Meanwhile, SR-Net

[12] directly processes white line artifacts using a simplified U-net and employs an additional thinning network to further improve image quality. These methods mainly target white line artifacts but ignore displacement and scan repetition artifacts. In addition, these methods often rely on complex vessel segmentation annotations, making them less practical for real-world applications. In contrast, this disclosure addresses all three types of motion artifacts without using vessel segmentation annotation, marking a pioneering effort in this direction.

[0044] 1.2. Transformers for Image Reconstruction Transformers have demonstrated superior performance to CNNs because they capture nonlocal information. Consequently, many efforts have been made to explore the application of transformers to image reconstruction

[29] ,

[31] ,

[36] -

[40] . The Swin transformer

[41] uses local windows to focus attention on regions and employs shift operations to enhance the interaction within these windows. This approach, exemplified by SwinIR

[37] , is effective for image reconstruction. DaViT

[42] combines a dual self-attention mechanism to efficiently capture global context with linear complexity, enhancing image reconstruction. SFormer

[31] integrates self-attention into the U-Net framework, combining two paradigms for robust performance. Restormer

[29] removes spatial self-attention, maintains performance, and demonstrates transform adaptability. Uformer

[30] addresses high-resolution maps by using shift windows and adapting the transformer to computational constraints without affecting the reconstruction results. In this disclosure, a hierarchical transformer is proposed to enhance feature representations across global, local, and vertical ranges, resulting in better motion artifact removal.

[0045] 1.3. Diffusion Model for Image Reconstruction Diffusion models have demonstrated remarkable ability to generate high-quality images. Their versatility and effectiveness have recently been extended to the field of image restoration tasks

[43] ,

[44] . These tasks cover a wide range of challenges, including denoising, super-resolution, and image inpainting

[33] ,

[45] -

[47] . Notably, SR3

[47] and cascaded diffusion models

[48] have illustrated the great potential of diffusion models in super-resolution tasks. Furthermore, Palette

[49] drew inspiration from conditional generative models to introduce conditional diffusion models tailored to a variety of image reconstruction tasks. Latent diffusion models

[50] have greatly improved the efficiency of image restoration processes in latent space. RePaint

[33] has improved image inpainting techniques by re-evaluating iterations in diffusion models. However, due to the inherent output diversity of diffusion models, reconstructing the entire image in RePaint may introduce additional artifacts. In contrast, this disclosure takes the sharp regions in OCTA images as conditions and focuses more on completion within the white line regions.

[0046] 2. OSAT Development The following section presents a two-stage method, OSAT, designed to remove three types of motion artifacts. In the first stage, Section 2.1, a U-shaped hierarchical transformer model, HFormer, is proposed to handle displacement and repeat scan artifacts. In the second stage, Section 2.2, a conditional diffusion model, RCDM, is proposed to mitigate whiteline artifacts and the negative impact of additional white areas introduced after removing displacement and repeat scan artifacts.

[0047] 2.1. Stage 1: Hierarchical Converter like Figure 1 As shown, the overall architecture of the proposed HFormer follows a U-shaped hierarchical network structure, incorporating skip connections between encoder and decoder blocks. To clarify, from the representation... Starting with a degraded OCTA image as input, we begin the process with convolutional layers to derive low-level feature representations. Subsequently, X0 undergoes four consecutive encoder stages. Each stage includes a downsampling layer that halves the feature map size while doubling the feature channels, and a stack of HFormer blocks. The number of blocks gradually increases from the top to the bottom. For the feature reconstruction stage, the decoder includes an upsampling layer that halves the feature channels while doubling the feature map size, and a stack of transformer blocks similar to those used in the encoder. Furthermore, encoder features are fused with decoder features via skip connections. A 3x3 convolutional layer is then applied to obtain the representation as... The residual image. Finally, the restored image is obtained as... The HFormer is trained using a subsequent loss function: (1) in: This represents the first-stage ground truth image with white line artifacts. It is given by the following formula

[51] (2) and Defined as (3) In equations (2) and (3), for all experiments, ε is empirically set to 1x10. -3 And Δ represents the Laplacian operator. The Laplacian operator Δ(x,y) for an image with pixel density I(x,y) is given by the following equation: (4) This can be calculated using the Laplacian operator of a Gaussian kernel on an OCTA image, and the kernel function is... (5) The edge-based loss function ensures that the reconstructed OCTA image retains the original anatomical boundaries of the vessels. By focusing on the edges, we can preserve fine details and prevent any structural deformation that may occur during the refinement process. Compared to methods that require detailed vessel segmentation masks, our edge-preserving loss function does not require pre-training another segmentation model. This advantage simplifies the process and improves the efficiency of our method, making it more practical for real-world applications. The parameter λ in Equation (1) controls the relative importance of the two loss terms. The value of λ can be set to 0.05. In Equation (5), δ can be set to 0.5. Other values ​​of λ and δ can be used depending on the specific circumstances.

[0048] 2.1.1. Hierarchical Self-Attention Applying transforms to motion artifact removal presents three main challenges. First, such as Figure 2 As shown in subfigure (a), traditional transformers compute spatial self-attention, leading to a quadratic increase in computational cost proportional to the feature map size, making them unsuitable for high-resolution feature maps. Secondly, preserving local contextual information is crucial for artifact removal, as the surrounding regions of displaced or re-scanned pixels can help recover their original state. However, transformers have limitations in effectively capturing these local dependencies. Finally, vertical feature representations and correlations play a significant role in removing displacement and re-scan artifacts, but self-attention struggles to adequately capture these aspects. To address these three issues, we introduce a hierarchical transformer block in Stage 1. This block leverages self-attention to capture dependencies across long and vertical range elements and integrates depthwise convolution into the self-attention module and FFN to improve the capture of critical local range context. When processing feature map X, the HFormer block consists of three types of transformer blocks with MSA modules and LFFNs. The computation of each transformer block is as follows: and (7) in, and These are the outputs of the MSA module and the LFFN module, respectively. In the following sections, we elaborate on three types of converter blocks with MSA and LFFN modules.

[0049] Channel Multi-Head Self-Attention (C-MSA). Instead of using pixel-by-pixel self-attention to capture global features, we perform self-attention operations across channels. For example... Figure 2 As shown in subgraph (b), consider the input tensor. We denote the number of heads as h, and the head dimension as D = C / h. For the i-th head, the input feature X... i C-MSA then performs the following operations in sequence: layer standardization , Pointwise convolutional layer as well as Lightweight deep convolutional layers To obtain projections respectively , and The projection obtained by using 3x3 depth convolution. , and It is enriched with local contextual information. Subsequently, the channel attention of the i-th head can be represented as follows: (8) in, It represents single-channel self-attention, and it captures relationships across channel dimensions. Used as an adaptive and trainable scaling parameter, it allows us to adjust the size of the dot product before applying the softmax function. Channel multi-head self-attention can be represented as follows: (9) in, This represents a 1x1 convolution operation.

[0050] Axial Multi-Head Self-Attention (A-MSA). To remove displacement and duplicate scan artifacts in OCTA images, capturing vertical features is crucial. Therefore, we introduce axial self-attention, where each vertical line feature is treated as a token, allowing the model to learn the relationships between these vertical lines. For example... Figure 2 As shown in subgraph (c), consider the input tensor. We divide X into W vertical segments, where each vertical segment is represented as... Let j = 1, ..., W. Let the number of heads be d, and the head dimension be D = W / d. For X, we represent the i-th head feature as X. i Similar to the projection operation in C-MSA, A-MSA also applies operations. , and To calculate the projection: , and Subsequently, after feature reshaping, the resulting projection is... , and Therefore, the formula for the i-th single-head axial attention can be expressed as follows: (10) in, It is a single-channel self-attention. These are learnable scaling parameters. The axial multi-head self-attention is described as follows: (11) in, This represents a 1x1 convolution operation.

[0051] In-Axis Multi-Head Self-Attention (IA-MSA). AMSA captures the relationships between different vertical lines but ignores the correlations between pixels within each individual vertical line, which are important for translation and duplication artifact removal. Therefore, we introduce an additional self-attention module to improve the feature representation within each vertical line. Figure 2 As shown in subgraph (d), consider the input tensor. We further divide X into W vertical lines. Each vertical line is characterized as follows: (Where j=1,…,W), consisting of H units (tokens) of dimension C. Assuming there are d heads, and each head has a dimension of D=H / d, we will... j The i-th head feature is represented as X ji Similar to the projection and reshaping operations in C-MSA and A-MSA, IA-MSA also employs operations... , and To generate projection , and Therefore, X j The i-th single-head axial inward self-attention mechanism can be defined as follows: (12) in, This represents the single-headed inward self-attention in the j-th vertical line. These are learnable scaling parameters. The formula for in-axis multi-head self-attention is as follows: (13) in, This represents a 1x1 convolution operation.

[0052] 2.1.2. LFFN In the standard vanilla transformer, FFN ( Figure 4 The subgraph (a) shown uses an MLP incorporating nonlinearity to generate the output. , where X represents the input tensor. However, MLPs have significantly lower image-specific inductive biases than CNNs and are not very good at utilizing local context. To address this issue, we propose LFFN, such as Figure 4Subgraph (b) is shown. Specifically, we replace the standard MLP in FFN with two parallel convolutional blocks, each containing a 1x1 pointwise convolutional layer and a 3x3 depthwise convolutional layer, instead of the ordinary MLP in FFN. This modification produces two projections. F d1 and F d2 This represents a convolution operation. The final output is derived from the dot product of these two projections, which can be represented as follows: (14) in, This represents a 1x1 convolutional layer. Represents the dot product. This represents the GeLU activation function.

[0053] 2.2. Phase 2: RCDM While HFormer in Stage 1 effectively handles displacement and repetition artifacts, it's important to note that the post-processed image from Stage 1 may still contain white patches along image boundaries and artifacts from the original white lines. Therefore, RCDM is designed to remove these in a second stage. In the context of removing white line artifacts, we define the following terms for Stage 2: input image x, target clean image x0, white line region m, and region within the white lines m. And the known clean area is The classic denoising diffusion probability model

[52] utilizes a forward process that employs a Markov diffusion kernel (scheduled by a fixed variance). (Definition) Transform the data distribution q(x0) sequentially into ,in: (15) This formula allows for switching from x0 to x at any time step t. t Perform closed-loop sampling: (16) Subsequently, a parameterized Markov chain is trained to reverse this forward process, effectively denoising arbitrary Gaussian noise into data samples. This reversal is described by the following equation: (17) The training process involves maximizing the model's log-likelihood using appropriate parameterization and simplification: (18) However, noting that areas outside the white line region do not require reconstruction, there is no need to apply the diffusion model to the entire image. Therefore, we design RCDM to address the conditional data distribution following... Optimize likelihood under constraints Since the forward process is determined by a Markov chain involving the addition of Gaussian noise, we are able to determine the forward process given a known region x. c In any time point Sampling intermediate images This allows us to consistently preserve the known region x. c This prevents the introduction of external noise and allows us to focus more intently on the white line region at each time step t. Within the framework of RCDM, we define the reverse process and the forward process (e.g., Figure 3 (As shown). The reverse process is represented as... It is characterized as a Markov chain, which has a distribution from the initial distribution The learned Gaussian transition is shown in the following formula: (19) and (20) The forward process is represented as Its characteristic is that it gradually introduces Gaussian noise into the data, following variance scheduling. It is represented as follows: (twenty one) Using symbols We can express the following equation: (twenty two) We take for Furthermore, the optimization objective can be viewed as a denoising operation specifically targeting the white line region, as shown below: (twenty three) 3. Experiments and Analysis 3.1. Implementation Details Implementation details of HFormer in OSAT. For the HFormer architecture, there are four levels of encoder-decoder transform blocks. The number of transform blocks ranges from level 1 to level 4 as follows: [4, 6, 6, 8]. The initial number of channels is 48. The number of attention heads for C-MSA, A-MSA, and IA-MSA at each level is specified as [1, 2, 4, 8], [2, 2, 1, 1], and [2, 2, 1, 1], respectively. We train the model with a batch size of 8. We use the AdamW optimizer with the following parameters: And the weight decays to 1x10 -4 Our training process spans 300,000 iterations and includes two phases. Initially, we use 3x10... -4We initially trained the model for 10,000 iterations using the initial learning rate. Subsequently, we used cosine annealing to gradually reduce the learning rate to 1x10^6 over the remaining 290,000 iterations. -6 .

[0054] Implementation details of RCDM in OSAT. We adopt the U-Net architecture

[52] as our foundation, incorporating various modifications influenced by recent research

[47] . In particular, we adopt the residual blocks introduced in

[49] and utilize the adaptive group normalization technique

[54] . We set the mini-batch size to 8, and the experimental setup follows previous work

[47] ,

[52] . We set T=2000 for training and T=1000 for inference.

[0055] Evaluation metrics. To evaluate the effectiveness of different methods in removing motion artifacts, we calculated MAE (mean absolute error), PSNR (peak signal-to-noise ratio), and SSIM (structural similarity index) scores on synthetic datasets. For real-world clinical data, we invited physicians to rate the results of different methods.

[0056] 3.2. Dataset In this study, we have curated two comprehensive synthetic datasets, OCTA-6MM and OCTA-3MM, for training and testing motion artifact removal models in OCTA images. Furthermore, we have incorporated real-world clinical image datasets to support visual representation and qualitative analysis using different methods.

[0057] Synthetic Data. We introduce various motion artifacts into sharp OCTA images to generate data pairs. To create displacement artifacts, we begin by dividing the input image I into a variable number of vertical blocks (denoted as M). The specific value of M is randomly selected from two to ten. These blocks are then labeled as I... m Each of these vertical blocks undergoes one of three operations: remains stationary, moves upward along the vertical axis, or moves downward along the vertical axis. These operations simulate displacement artifacts that may occur during image acquisition. Based on the doctor's advice, we set the maximum value of the movement to 25. To create displacement artifacts, we randomly divide the input image I into a variable number of vertical blocks, denoted as S, where S is randomly selected from the range of two to ten. Then, we... sThe image is copied to its right to generate the desired scan artifacts. For white line artifacts, we randomly select bands within the image and replace them with bands containing white lines. Therefore, OCTA-6MM first collects 4151 sharp 6mm x 6mm OCTA images from COINPS. These images are then augmented to synthesize 67660 image pairs for training and 768 image pairs for testing. Similarly, OCTA-3MM first collects 4891 sharp 3mm x 3mm OCTA images from COINPS, then synthesizes 63582 image pairs for training and 824 image pairs for testing.

[0058] Real-world clinical data. A comprehensive cohort of 70 patients from the Ophthalmology Department of the Sixth Affiliated Hospital of South China University of Technology was examined. Images were acquired using a high-speed spectral-domain OCTA machine with a scanning frequency of 120 kHz, provided by Microhuman Medical Technology (model C3000). It covered a variety of scanning ranges, including 6mm x 6mm foveal scans and a wider 12mm x 12mm wide-angle scans. The dataset obtained from these scans includes retinal images under both normal and diseased conditions, including diseases such as diabetic retinopathy, age-related macular degeneration, and others.

[0059] 3.3. Comparison Methods To our knowledge, there is currently no existing method that directly addresses the challenge of removing the three different types of motion artifacts in OCTA images. Therefore, we have conducted a comprehensive evaluation of two OCTA artifact removal methods (SR-NET

[12] and RUNet

[24] ) and six state-of-the-art image restoration methods in the field of computer vision (DeblurGAN

[53] , MIMOUnet

[27] , MPRNet

[28] , SFormer

[31] , Uformer

[30] and Restormer

[29] ) to compare with the proposed OSAT. SRNET

[12] is designed to remove white line noise in OCTA images and further improves the quality of OCTA images by using a thinning network. RU-Net

[24] modifies the typical UNet and uses low-rank regularization to remove white line artifacts during model training. DeblurGAN

[53] incorporates a feature pyramid network into its generator and uses a multi-scale discriminator to enhance image restoration. MIMO

[27] employs a coarse-to-fine approach and leverages a multi-input multi-output U-net for superior feature representation. MPRNet

[28] employs a multi-stage architecture for progressive learning of the restoration function. SFormer

[31] integrates an efficient self-attention module into the U-net to handle high-level features for image restoration. Uformer

[30] incorporates a shift window into self-attention to enhance local features in the transformer block. Restormer

[30] utilizes a single efficient self-attention module to enhance global features. Notably, of these six methods, DeblurGAN, MIMO-Unet, and MPRNet are CNN-based, while SFormer, Uformer, and Restormer are transformer-based. In Stage 2, we extend our comparison to include two state-of-the-art methods for mitigating white line artifacts: the transformer-based method T-Former

[32] and the diffusion method RePaint

[33] .

[0060] Table I: Comparison of OSAT with various methods on OCTA-6MM and OCTA-3MM, where SR-Net

[12] and RU-Net

[24] are specifically designed for OCTA images. Table II: User studies based on real clinical data.

[0061] Table III: Comparison of different stages of the OSAT method.

[0062] Table IV: Comparison of different stages of the OSAT method.

[0063] Table V: Comparison of OSAT artifact removal results with different self-attention settings in HFormer.

[0064] Table VI: Comparison of OSAT artifact removal for different FFNs in HFormer.

[0065] Table VII: The effect of HFormer loss function design.

[0066] The effects of the loss function in HFormer. We compared them in Table VII. and and and The effectiveness of the study was demonstrated. outperformed and Loss. Furthermore, when we remove... At that time, we noticed a decrease in artifact removal performance, highlighting the issue. Importance and effectiveness in Phase 1.

[0067] 3.4. Results of Motion Artifact Removal Results on synthetic datasets. Table I summarizes the artifact removal results on OCTA-6MM and OCTA-3MM. It first presents a comparative analysis of OSAT with eight single-stage methods, involving two recent OCTA-specific models SR-NET

[12] and RU-Net

[24] , as well as six state-of-the-art image restoration methods in the CV field, including DeblurGAN

[53] , MIMO-Unet

[27] , MPRNet

[28] , SFormer

[31] , Uformer

[30] , and Restormer

[29] . In addition, we also compare our OSAT with different two-stage methods. As shown in Table I, although the six CV methods outperform SR-NET and RU-Net, these single-stage methods still struggle to handle displacement, repetitive scans, and white line artifacts simultaneously, and achieve unsatisfactory performance on MAE, PSNR, and SSIM metrics. Furthermore, Restormer and Uformer outperform other CNN-based methods, demonstrating the superiority of transformer methods. MPRNet outperforms SFormer due to its multi-scale aggregation scheme and additional refinement network, and these complex operations enhance MPRNet's capabilities. In addition, SFormer still uses a CNN backbone and self-attention blocks as additional attention modules, which limits the global representation of SFormer. The proposed OSAT sequentially removes three types of motion artifacts and outperforms all single-stage methods, demonstrating the superiority of this two-stage framework. Within the two-stage framework, we evaluate six CV methods and our proposed HFormer during the first stage. In the second stage, we compare the proposed RCDM with two other different image inpainting techniques, T-former

[32] and Repaint

[33] . The results of the two-stage method are summarized in Table I, demonstrating the superiority of OSAT in artifact removal, where it achieves significantly better MAE, PSNR, and SSIM results for both the OCTA-6MM and OCTA-3MM datasets. HFormer in OSAT captures feature representations across local, global, and vertical scales, significantly enhancing its potential to improve the efficiency of motion artifact removal. Furthermore, the guidance of clear condition regions makes RCDM effective in removing white line artifacts. To further analyze the impact of HFormer and RCDM, we replaced them with other alternative state-of-the-art methods, and the results are summarized in Tables III and IV, respectively.

[0068] Results on Real-World Clinical Data. We comprehensively evaluated our OSAT using a dataset of 70 real-world clinical OCTA images from a hospital setting. To ensure the robustness and clinical applicability of our method, we sought input from experienced medical professionals who carefully evaluated and scored the outputs produced when various techniques were applied to this real-world dataset. We compared OSAT with other two-stage methods, where DeblurGAN, MIMO, MPRNet, SFormer, UFormer, and Restormer participated in Stage 1, and RCDM served as the Stage 2 method. The results of this evaluation are summarized in Table II, which clearly highlights the superior mean quality score (p-value < 0.05) achieved by OSAT compared to other two-stage methods. Figure 5 The visualizations show the performance of various combinations, with regions containing motion artifacts clearly outlined by red rectangles and key points indicated by yellow arrows. These visualizations demonstrate the significant ability of HFormer and RCDM in OSAT to effectively generalize to complex motion artifacts in OCTA images.

[0069] 3.5. Ablation Studies Performance of HFormer in OSAT. We evaluated the performance of the proposed HFormer relative to six other methods, including DeblurGAN

[53] , MIMO-Unet

[27] , MPRNet

[28] , SFormer

[31] , Uformer

[30] , and Restormer

[29] in stage 1, all of which use RCDM as the stage 2 model. As shown in Table III, HFormer outperforms the other methods on all MAE, PSNR, and SSIM metrics on the OCTA-6MM and OCTA-3MM datasets. By capturing long-range dependencies in OCTA images, the transformer effectively removes motion artifacts and achieves better performance than CNN-based methods in motion artifact removal. Furthermore, the proposed HFormer employs a hierarchical self-attention module to capture the full range of feature representations, thereby enhancing the model's capabilities.

[0070] The performance of RCDM in OSAT. We also evaluated the performance of RCDM relative to T-Former and RePaint in Stage 2. T-Former is an image completion method based on the SOTA transform, and RePaint is a diffusion model-based method. In this case, all frameworks utilize HFormer as the Stage 1 method. The results are summarized in Table IV. The proposed RCDM outperforms the two compared methods, while RePaint achieves better performance than T-Former. Thousands of forward and backward passes enhance RePaint's ability to remove artifacts, while RCDM outperforms RePaint due to its greater focus on white line regions.

[0071] Effects of Different Self-Attention Settings. We conducted an ablation study to evaluate the effectiveness of different self-attention modules within the HFormer architecture. We maintained the base self-attention module C-MSA throughout the study. Furthermore, we examined the effects of two other modules, A-MSA and IA-MSA, respectively. To ensure fairness in the comparison, we replaced any removed modules with C-MSA instances to maintain a consistent model size. The results in Table V show that removing IA-MSA resulted in a decrease in PSNR of 0.53 and 0.65 for OCTA-6MM and OCTA-3MM, respectively. In contrast, excluding A-MSA resulted in a more significant decrease in PSNR, decreasing by 1.00 and 0.98 for OCTA-6MM and OCTA-3MM, respectively. This highlights the greater importance of A-MSA compared to IA-MSA, emphasizing that inter-strip relationships are more important than relationships within individual vertical strips. We also conducted additional experiments to explore the effect of sequence ordering of IA-MSA and A-MSA. When IA-MSA is applied followed by A-MSA (IA ⇒ A), as shown in Table V, the results show a superior representation compared to the reverse order (A ⇒ IA).

[0072] Performance of FFN. We compared LFFN with conventional MLP FFN using an expansion ratio of 4. The results are summarized in Table VI, clearly showing that LFFN outperforms conventional MLP on all metrics, including MAE, PSNR, and SSIM. This provides strong evidence for the superior performance of LFFN in artifact removal, highlighting the effectiveness of local representation.

[0073] The effects of the loss function in HFormer. We compared them in Table VII. and and and The effectiveness of the study was demonstrated. outperformed and Loss. Furthermore, when we remove... At that time, we noticed a decline in artifact removal performance, highlighting the... Importance and effectiveness in Phase 1.

[0074] 4. Embodiments of this disclosure Based on the above details, examples, applications, and possible generalizations and extensions of OSAT, the following embodiments of this disclosure have been developed.

[0075] One aspect of this disclosure provides a computer-implemented method for reducing eye movement artifacts in degraded OCTA images to produce enhanced quality OCTA images.

[0076] The disclosed method utilizes Figure 6 To explain, Figure 6 A flowchart 600 depicts exemplary steps of the disclosed method. Exemplarily, the disclosed method includes steps 610, 620, 630, and 640. Steps 610 and 620 are initialization steps. Steps 630 and 640 collectively implement OSAT.

[0077] In step 610, an HTM is configured to remove displacement artifacts and duplicate scan artifacts from the first input OCTA image. The HTM is arranged to receive the first input OCTA image and, as a result, produce a first output OCTA image as the output of the HTM. Preferably and advantageously, the HTM is implemented as any embodiment of the HFormer disclosed above.

[0078] The HTM includes multiple transformer blocks. These transformer blocks are configured to employ self-attention to capture multiple features that can be used to remove displacement artifacts and repetition artifacts from the first input OCTA image. When setting up the HTM, as demonstrated above for HFormer, it is preferred and advantageous that the multiple features include global features, vertical line features, and in-vertical line features of the first input OCTA image. Therefore, the multiple transformer blocks include a first plurality of transformer blocks for capturing global features, a second plurality of transformer blocks for capturing vertical line features, and a third plurality of transformer blocks for capturing in-vertical line features.

[0079] In summary, HTM is configured to utilize the multiple features (including the global features, vertical line features, and vertical line in-line features) to remove displacement artifacts and duplicate scan artifacts from the first input OCTA image.

[0080] In some embodiments, the first, second, and third plurality of transformer blocks are configured as follows: The first transformer block of the first plurality of transformer blocks is configured to perform a first plurality of self-attention operations (which are channel self-attention operations) on channels of a first input feature map received by the first transformer block. The second transformer block of the second plurality of transformer blocks is configured to perform a second plurality of self-attention operations (which are axial self-attention operations) on vertical lines of a second input feature map received by the second transformer block, such that the relationships between the vertical lines are learned. The third transformer block of the third plurality of transformer blocks is configured to perform a third plurality of self-attention operations (which are in-axial self-attention operations) on different pixels within individual vertical lines of a third input feature map received by the third transformer block, such that information about the correlations between pixels within individual vertical lines is extracted. Further details regarding the first, second, and third plurality of self-attention operations are provided in Section 2.1.1 and... Figure 2 The subgraphs (b)-(d) are provided.

[0081] In some embodiments of implementing the first, second, and third converter blocks, a separate converter block among the multiple converter blocks is formed by including an MSA module and an LFFN module. Based on the MSA module and the LFFN module, the computation performed by the separate converter block is governed by equations (6) and (7).

[0082] Note that each of the first converter blocks in the first plurality of converter blocks uses channel MSA to capture global features. Similarly, each of the second converter blocks in the second plurality of converter blocks uses axial MSA to capture vertical line features. Furthermore, similarly, each of the third converter blocks in the third plurality of converter blocks uses axial in-line MSA to capture in-line features.

[0083] In some embodiments, the LFFN module is formed by two parallel convolutional blocks. Each of the two parallel convolutional blocks contains a 1x1 pointwise convolutional layer and a 3x3 depthwise convolutional layer. As mentioned above, using the LFFN module has advantages over using an MLP because MLPs have a significantly lower image-specific inductive bias than CNNs (which are used in the LFFN module) and are less adept at utilizing local context.

[0084] The HTM employs a U-shaped hierarchical network following an encoder-decoder architecture. Therefore, the individual converter blocks in the multiple converter blocks are typically classified as encoder blocks and decoder blocks. In some embodiments, the U-shaped hierarchical network includes skip connections between encoder blocks and decoder blocks. Alternatively, skip connections may not exist in the U-shaped hierarchical network employed by the HTM.

[0085] The HTM can be a pre-trained model, such that the model parameters of the HTM are pre-stored, and the pre-stored model parameters are loaded into the HTM in step 610. Alternatively, the HTM can be trained during the setup of the HTM in step 610. In some embodiments, step 610 includes training the HTM. In some embodiments, the HTM is trained according to the loss function given by equation (1).

[0086] In step 620, an RCDM is set up to remove white line artifacts from the second input OCTA image. The RCDM is arranged to receive the second input OCTA image and thereby produce a second output OCTA image as the output of the RDCM.

[0087] RCDM is a diffusion model used to reconstruct a first region of the second input OCTA image that overlaps with the white line artifact, while avoiding modification of a second region of the second input OCTA image that does not overlap with the white line artifact. The first and second regions can be named the artifact region and the residual region, respectively. Preferably and advantageously, the diffusion model is configured to optimize the likelihood of correctly reconstructing the first region while conditioned on the image data of the second region.

[0088] In summary, RCDM is configured to reconstruct the first region while avoiding modification of the second region in order to remove white line artifacts from the second input OCTA image.

[0089] Similar to HTM, RCDM can be a pre-trained model, such that the model parameters of RCDM are pre-stored, and the pre-stored model parameters are loaded into RCDM in step 620. Alternatively, RCDM can be trained during RCDM setup in step 620. In some embodiments, step 620 includes training RCDM.

[0090] Step 630 reflects the implementation of the first stage of OSAT. In step 630, HTM is used to remove any displacement artifacts and any duplicate scan artifacts from the degraded OCTA image to produce an intermediate OCTA image. Note that HTM is used to process the degraded OCTA image in step 630 after HTM has been trained.

[0091] Step 640 reflects the implementation of the second phase of OSAT. In step 640, any white line artifacts are removed from the intermediate OCTA image obtained in step 630 using RCDM, resulting in a quality-enhanced OCTA image. Note that RCDM is used to process the intermediate OCTA image in step 640 after RCDM has been trained.

[0092] A computational system for reducing eye movement artifacts in degraded OCTA images to produce enhanced quality OCTA images may be implemented by including one or more computers, wherein the one or more computers are configured to perform the process of reducing eye movement artifacts in degraded OCTA images to produce enhanced quality OCTA images according to any embodiment of the disclosed method. The individual computers may be general-purpose computers, special-purpose computers (e.g., computers implemented using one or more artificial intelligence processors or one or more graphics processing units), desktop computers, physical computing servers, distributed computing servers, or mobile computing devices such as smartphones and tablets.

[0093] This disclosure may be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, these embodiments should be considered illustrative rather than restrictive in all respects. The scope of the invention is indicated by the appended claims rather than the foregoing description, and thus all variations falling within the meaning and equivalents of the claims are intended to be included therein.

[0094] References The following is a list of references occasionally cited in this manual. The full content of each of these references is incorporated into this text through citation.

[0095] [1] C. Wang, H. Ning, X. Chen, and S. Li, "Db-unet: Mlp based dualbranch unet for accurate vessel segmentation in octa images," in ICASSP,2023.[2] R. F. Spaide, J. G. Fujimoto, N. K. Waheed, S. R. Sadda, and G.Staurenghi, "Optical coherence tomography angiography," Prog. Retin. EyeRes., 2018.[3] W. Yang, Y. Shao, and Y. Xu, "Guidelines on clinical researchevaluation of artificial intelligence in ophthalmology," in Int J Ophthalmol,2023.[4] C.-H. Hua, K. Kim, T. Huynh-The, J. I. You, S.-Y. Yu, T. Le-Tien,S.-H. Bae, and S. Lee, "Convolutional network with twofold featureaugmentation for diabetic retinopathy recognition from multi-modal images,"IEEE J. Biomed. Health., 2020.[5] E. W. Schneider and S. C. Fowler, "Opticalcoherence tomography angiography in the management of age-related maculardegeneration," Curr Opin Ophthalmol, 2018.[6] Y. Jia, J. M. Simonett, J.Wang, X. Hua, L. Liu, T. S. Hwang, and D.Huang, "Wide-field oct angiographyinvestigation of the relationship between radial peripapillary capillaryplexus density and nerve fiber layer thickness," Investig. Ophthalmol. Vis.Sci., 2017.[7] T. T. Hormel, D. Huang, and Y. Jia, "Artifacts and artifactremoval in optical coherence tomographic angiography," Quant. Imaging. Med.Surg., 2021.[8] I. C. Holmen, S. M. Konda, J. W. Pak, K. W. McDaniel, B.Blodi, K. E. Stepien, and A. Domalpally, "Prevalence and severity ofartifacts in optical coherence tomographic angiograms," JAMA Ophthalmol.,2020.[9] Y. Zhou, K. Yu, M. Wang, Y. Ma, Y. Peng, Z. Chen, W. Zhu, F. Shi,and X. Chen, "Speckle noise reduction for oct images based on image styletransfer and conditional gan," IEEE J. Biomed. Health., 2021.

[10] R. F.Spaide, J. G. Fujimoto, and N. K. Waheed, "Image artifacts in opticalcoherence angiography," Retina, 2015.

[11] P. Anvari, M. Ashrafkhorasani, A.Habibi, and K. G. Falavarjani, "Artifacts in optical coherence tomographyangiography," J.Ophthalmic Vis. Res., 2021.

[12] J. Cao, Z. Xu, M. Xu, Y. Ma,and Y. Zhao, "A two-stage framework for optical coherence tomographyangiography image quality improvement," Front. Med., 2023.

[13] K. Hu, S.Jiang, Y. Zhang, X. Li, and X. Gao, "Joint-seg: Treat foveal avascular zoneand retinal vessel segmentation in octa images as a joint task," IEEE Trans.Instrum. Meas., 2022.

[14] S. Chen, B. Potsaid, Y. Li, J. Lin, Y. Hwang, E. M.Moult, J. Zhang, D. Huang, and J. G. Fujimoto, "High speed, long range, deeppenetration swept source oct for structural and angiographic imaging of theanterior eye," Sci. Rep., 2022.

[15] A. Yasin Alibhai, C. Or, and A. J.Witkin, "Swept source optical coherence tomography: a review," Curr.Ophthalmol. Rep., 2018.

[16] M. F. Kraus, B. Potsaid, M. A. Mayer, R. Bock, B.Baumann, J. J. Liu, J. Hornegger, and J. G. Fujimoto, "Motion correction inoptical coherence tomography volumes on a per a-scan basis using orthogonalscan patterns," Biomed. Opt. Express, 2012.

[17] A.Wolf, K. Tripanpitak, S.Umeda, and M. Otake-Matsuura, "Eyetracking paradigms for the assessment ofmild cognitive impairment: a systematic review," Front. Psychol., 2023.

[18] A. Uji, S. Balasubramanian, J. Lei, E. Baghdasaryan, M. Al-Sheikh, and S. R.Sadda, "Choriocapillaris imaging using multiple en face optical coherencetomography angiography image averaging," JAMA Ophthalmol., 2017.

[19] X. Wei,A. Camino, S. Pi, W. Cepurna, D. Huang, J. C. Morrison, and Y. Jia, "Fast androbust standard-deviation-based method for bulk motion compensation in phase-based functional oct," Opt. Lett., 2018.

[20] Y. Zhang, W. Gao, and C. Xie, "Fourier spatial transform-based method of suppressing motion noises in octa,"Opt. Lett., 2022.

[21] A. Camino, Y. Jia, G. Liu, J. Wang, and D. Huang, "Regression-based algorithm for bulk motion subtraction in optical coherencetomography angiography," Biomed. Opt. Express, 2017.

[22] P. Zang, G. Liu, M.Zhang, C. Dongye, J. Wang, A. D. Pechauer, T. S. Hwang, D. J.Wilson, D.Huang, D. Li et al., "Automated motion correction using parallel-stripregistration for wide-field en face oct angiogram," Biomed. Opt. Express,2016.

[23] A. Camino, M. Zhang, C. Dongye, A. D. Pechauer, T. S. Hwang, S. T.Bailey, B. Lujan, D. J. Wilson, D. Huang, and Y. Jia, "Automated registrationand enhanced processing of clinical optical coherence tomographyangiography," Quant. Imaging. Med. Surg., 2016.

[24] D. Gao, N. Celik, X. Wu,B. M. Williams, A. Stylianides, and Y. Zheng, "A novel deep learning basedocta de-striping method," in MIUA, 2020.

[25] A. Li, C. Du, and Y. Pan, "Deep-learning-based motion correction in optical coherence tomographyangiography," J. Biophotonics, 2021.

[26] J. Ren, K. Park, Y. Pan, and H.Ling, "Self-supervised bulk motion artifact removal in optical coherencetomography angiography," in CVPR, 2022.

[27] S.-J. Cho, S.-W. Ji, J.-P. Hong,S.-W. Jung, and S.-J. Ko, "Rethinking coarse-to-fine approach in single imagedeblurring," in ICCV, 2021.

[28] S. W.Zamir, A. Arora, S. Khan, M. Hayat, F.S. Khan, M.-H. Yang, and L. Shao, "Multi-stage progressive imagerestoration," in CVPR, 2021.

[29] S. W. Zamir, A. Arora, S. Khan, M. Hayat, F.S. Khan, and M.-H. Yang, "Restormer: Efficient transformer for high-resolution image restoration," in CVPR, 2022.

[30] Z. Wang, X. Cun, J. Bao, W.Zhou, J. Liu, and H. Li, "Uformer: A general u-shaped transformer for imagerestoration," in CVPR, 2022.

[31] F.-J. Tsai, Y.-T. Peng, Y.-Y. Lin, C.-C.Tsai, and C.-W. Lin, "Stripformer: Strip transformer for fast imagedeblurring," in ECCV, 2022.

[32] Y. Deng, S. Hui, S. Zhou, D. Meng, and J.Wang, "T-former: An efficient transformer for image inpainting," in ACM MM,2022.

[33] A. Lugmayr, M. Danelljan, A. Romero, F. Yu, R. Timofte, and L. VanGool, "Repaint: Inpainting using denoising diffusion probabilistic models,"in CVPR, 2022.

[34] P. G. Daneshmand, A. Mehridehnavi, and H.Rabbani, "Reconstruction of optical coherence tomography images using mixed low rankapproximation and second order tensor based total variation method," IEEETrans. Med. Imaging, 2020.

[35] X. Wu, D. Gao, D. Borroni, S. Madhusudhan, Z.Jin, and Y. Zheng, "Cooperative low-rank models for removing stripe noisefrom octa images," IEEE J. Biomed. Health., 2020.

[36] F. Yang, H. Yang, J.Fu, H. Lu, and B. Guo, "Learning texture transformer network for image super-resolution," in CVPR, 2020.

[37] J. Liang, J. Cao, G. Sun, K. Zhang, L. VanGool, and R. Timofte, "SwinIR: Image restoration using swin transformer," inICCV Workshops, 2021.

[38] M. Kumar, D. Weissenborn, and N. Kalchbrenner, "Colorization transformer," in ICLR, 2021.

[39] H. Chen, Y. Wang, T. Guo, C.Xu, Y. Deng, Z. Liu, S. Ma, C. Xu, C. Xu, and W. Gao, "Pre-trained imageprocessing transformer," in CVPR, 2021.

[40] Z. Wang, X. Cun, J. Bao, and J.Liu, "Uformer: A general u-shaped transformer for image restoration," arXiv:2106.03106, 2021.

[41] Z. Liu, Y. Lin, Y. Cao, H. Hu, Y. Wei, Z. Zhang, S.Lin, and B. Guo, "Swin transformer: Hierarchical vision transformer usingshifted windows," arXiv:2103.14030, 2021.

[42] M. Ding, B. Xiao, N. Codella,P. Luo, J. Wang, and L. Yuan, "Davit: Dual attention vision transformers," inECCV, 2022.

[43] Z.-X. Cui, C. Cao, S. Liu, Q. Zhu, J. Cheng, H. Wang, Y. Zhu,and D. Liang, "Self-score: Self-supervised learning on score-based models formri reconstruction," arXiv:2209.00835, 2022.

[44] C. Cao, Z.-X. Cui, S. Liu,H. Zheng, D. Liang, and Y. Zhu, "High-frequency space diffusion models foraccelerated mri," arXiv:2208.05481, 2022.

[45] W. Xia, Q. Lyu, and G. Wang, "Low-dose ct using denoising diffusion probabilistic model for 20x speedup,"arXiv:2209.15136, 2022.

[46] Y. Shi and G. Wang, "Conversion of the mayo ldctdata to synthetic equivalent through the diffusion model for trainingdenoising networks with a theoretically perfect privacy," arXiv:2301.06604,2023.

[47] C. Saharia, J. Ho, W. Chan, T.Salimans, D. J. Fleet, and M.Norouzi, "Image super-resolution via iterative refinement," IEEE Trans.Pattern Anal. Mach. Intell., 2022.

[48] J. Ho, C. Saharia, W. Chan, D. J.Fleet, M. Norouzi, and T. Salimans, "Cascaded diffusion models for highfidelity image generation," J. Mach. Learn. Res., 2022.

[49] C. Saharia, W.Chan, H. Chang, C. Lee, J. Ho, T. Salimans, D. Fleet, and M. Norouzi, "Palette: Image-to-image diffusion models," in SIGGRAPH, 2022.

[50] R. Rombach,A. Blattmann, D. Lorenz, P. Esser, and B. Ommer, "High-resolution imagesynthesis with latent diffusion models," in CVPR, 2022.

[51] P. Charbonnier,L. Blanc-Feraud, G. Aubert, and M. Barlaud, "Two deterministic half-quadraticregularization algorithms for computed imaging," in ICIP, 1994.

[52] J. Ho, A.Jain, and P. Abbeel, "Denoising diffusion probabilistic models," NeurIPS,2020.

[53] O. Kupyn, T. Martyniuk, J. Wu, and Z. Wang, "Deblurgan-v2:Deblurring (orders-of-magnitude) faster and better," in ICCV, 2019.

[54] P.Dhariwal and A.Nichol, "Diffusion models beat gans on image synthesis,"NeurIPS, 2021.

Claims

1. A computer-implemented method for reducing eye movement artifacts in degraded optical coherence tomography (OCTA) angiography images to produce enhanced quality OCTA images, the method comprising: A hierarchical transformer model (HTM) is set up to remove displacement artifacts and duplicate scan artifacts from a first input OCTA image, thereby producing a first output OCTA image. A residual conditional diffusion model (RCDM) is set up to remove white line artifacts from a second input OCTA image, thereby producing a second output OCTA image; The HTM is used to remove any displacement artifacts and any duplicate scan artifacts from the degraded OCTA image to produce an intermediate OCTA image; as well as The RCDM is used to remove any white line artifacts from the intermediate OCTA image to produce the enhanced OCTA image.

2. The method according to claim 1, wherein, The HTM is also configured to use multiple features to remove the displacement artifacts and duplicate scan artifacts from the first input OCTA image, wherein the multiple features include global features, vertical line features, and in-vertical line features of the first input OCTA image.

3. The method according to claim 1, wherein, The HTM includes multiple transformer blocks configured to employ self-attention to capture multiple features that can be used to remove the displacement artifacts and repetitive scan artifacts from the first input OCTA image, wherein the multiple features include global features, vertical line features, and in-vertical line features of the first input OCTA image.

4. The method according to claim 1, wherein, The HTM has a U-shaped hierarchical network structure, which incorporates skip connections between the encoder block and the decoder block of the HTM.

5. The method according to claim 3, wherein, Used to process the input feature map X to produce the output feature map. The individual converter block comprises a multi-head self-attention (MSA) module and a local feedforward network (LFFN), enabling the individual converter block to compute: ， and ， MSA(.) and LFFN(.) are mathematical functions executed by the MSA module and the LFFN, respectively.

6. The method according to claim 5, wherein, The LFFN is formed by two parallel convolutional blocks, and each of the two parallel convolutional blocks includes a 1x1 pointwise convolutional layer and a 3x3 depthwise convolutional layer.

7. The method according to claim 5, wherein, The plurality of converter blocks include: First plurality of transformer blocks used to capture the global features; A second plurality of transformer blocks for capturing the vertical line feature; and A third plurality of transformer blocks are used to capture features within the vertical line.

8. The method according to claim 7, wherein, Each of the first plurality of converter blocks is configured to use channel MSA to capture the global features.

9. The method according to claim 7, wherein, Each of the second plurality of converter blocks is configured to use axial MSA to capture the vertical line feature.

10. The method according to claim 7, wherein, Each of the third plurality of transformer blocks is configured to use in-axial MSA to capture the features within the vertical line.

11. The method according to claim 1, wherein, The HTM is trained to minimize the loss function. It is given by the following formula: in: This indicates the restored OCTA image due to the use of the HTM; A ground truth OCTA image corrupted by white line artifacts; It is given by the following formula: ε is the predetermined offset; It is given by the following formula: Δ denotes the Laplace operator; and λ is the control. and Predetermined parameters for the relative importance between them.

12. The method according to claim 1, wherein, The HTM setup includes training the HTM before it is used to process the degraded OCTA image.

13. The method according to claim 1, wherein, The HTM is implemented as HFormer.

14. The method according to claim 1, wherein, The RCDM is configured to reconstruct a first region of the second input OCTA image that overlaps with the white line artifact, while avoiding modification of a second region of the second input OCTA that does not overlap with the white line artifact, in order to remove the white line artifact from the second input OCTA image.

15. The method according to claim 1, wherein, The RCDM is a diffusion model configured to optimize the likelihood of correct reconstruction of a first region of the second input OCTA image that overlaps with the white line artifact, wherein the reconstruction of the first region is conditioned on image data of a second region of the second input OCTA that does not overlap with the white line artifact.

16. The method according to claim 1, wherein, The RCDM setup includes training the RCDM before it is used to process the intermediate OCTA image.

17. A computing system for reducing eye motion artifacts in degraded optical coherence tomography (OCTA) images to produce quality-enhanced OCTA images, said computing system comprising one or more computers configured to perform a process of reducing the eye motion artifacts in the degraded OCTA images to produce the quality-enhanced OCTA images according to any one of the preceding claims.

Images