Retinal OCTA multi-object segmentation method based on double-branch attention network

Abstract

This invention discloses a multi-target segmentation method for retinal OCTA based on a bi-branch attention network, belonging to the interdisciplinary field of medical information processing and medical image processing. The method constructs a bi-branch network containing 3D and 2D paths. A three-scale aggregation channel attention mechanism is introduced in the 2D path encoder stage to dynamically fuse multi-scale features while preserving high-frequency details. Further, an extended frequency-space attention module is designed to achieve synergistic optimization of frequency domain filtering and spatial domain awareness. A Mamba-inspired linear attention module is integrated to model the global context with linear complexity. Finally, a boundary-aware shared feature decoupling module is used to achieve joint segmentation of retinal capillaries, arteries, veins, and avascular areas in the fovea. This invention effectively solves the problems of multi-scale lesion adaptation, loss of high-frequency details, and low global modeling efficiency in existing methods.

Description

Technical Field

[0001] This invention belongs to the interdisciplinary field of medical information processing and medical image processing, and specifically relates to a multi-target segmentation method for retinal OCTA based on a bi-branch attention network. Background Technology

[0002] Optical coherence tomography (OCTA) is an important non-invasive imaging technique in ophthalmology. Utilizing the low-coherence interference principle of optical coherence tomography, it detects blood flow by analyzing the signal decorrelation through repeated B-scans, thereby studying the vascular structure and microcirculation of the retina and choroid. In the field of ophthalmic disease detection and diagnosis, OCTA exhibits significant advantages over traditional imaging techniques such as color fundus imaging and fluorescein angiography. OCTA can acquire high-resolution three-dimensional information of the retinal vascular system, accurately presenting the structural features of the retinal arteries (RA), retinal capillaries (RC), and retinal veins (RV), while also clearly imaging subtle morphological changes in the avascular zone (FAZ) of the macula. With these unique technical characteristics, OCTA occupies an important position in ophthalmic clinical and research work, becoming a highly valuable new imaging tool that has attracted considerable attention from researchers.

[0003] Structural changes at key targets within the retina are closely associated with numerous eye diseases. In the retinal region, morphological abnormalities in the retinal retinopathy (RA) and retinal vein (RV) (such as tortuosity and diameter changes) are early markers of diabetic retinopathy (DR) and retinal vein occlusion (RVO), and are also associated with systemic diseases such as hypertension. The retinal retinopathy (RC) is divided into superficial vascular plexus (SVP) and deep vascular plexus (DVP), and decreased density or the formation of non-perfusion areas is common in patients with DR and glaucoma. Furthermore, abnormal blood flow in the choroidal capillaries is closely associated with dry age-related macular degeneration (AMD). The retinal plexus (FAZ), an avascular area in the center of the macula, shows significant enlargement or irregular morphology, which is significantly associated with the degree of ischemia in DR and diabetic macular edema (DME), and can be used to assess diseases such as retinal vein occlusion. In addition, the phenotype of retinal vessels is an important biomarker for pathological conditions such as hypertension and Parkinson's disease. Therefore, quantitative analysis of these structures in optical coherence tomography (OCTA) volumetric scanning is crucial for effectively assessing the progression of eye diseases. For a long time, researchers have been seeking a precise and efficient method to accurately segment retinal structures from optical coherence tomography (OCTA) images of blood vessels.

[0004] In retinal structure segmentation, existing segmentation methods can be categorized into single-object segmentation methods and multi-object segmentation methods. The development of single-object segmentation methods in optical coherence tomography (OCTA) image analysis reflects the technological leap from traditional algorithms to deep learning. Early traditional methods based on thresholding and region growing, while computationally simple, struggled to handle complex vascular structures and noise interference. With the rise of deep learning, convolutional neural network (CNN) models, represented by UNet and ResNet, achieved end-to-end segmentation through an encoder-decoder architecture, while the introduction of Transformer further optimized global feature extraction capabilities through a self-attention mechanism. These methods performed well in 2D OCT projection image processing, but had significant limitations, namely, the inability to effectively utilize 3D spatial information. While 3D to 2D methods (such as OCT2Former) can directly extract features from 3D OCT data, they still fall short of meeting the needs of joint analysis of multiple structures in clinical practice. For example, the arterial-venous (AV) ratio, a key indicator of diabetic retinopathy, requires the simultaneous analysis of two vascular features, but existing single-object segmentation methods can only handle a single structure independently. Furthermore, although multiple single-object segmentation models can be used simultaneously, integrating multiple models significantly increases memory and computational burden and reduces computational efficiency.

[0005] For multi-object segmentation methods, the aim is to segment multiple categories of targets simultaneously in a single image. By sharing feature extraction networks and joint optimization strategies, the spatial correlation between multiple targets is modeled while reducing redundant computation, significantly improving segmentation efficiency and consistency. Compared with single-object segmentation methods, it avoids the error accumulation problem of class-by-class processing. Recently, a 3D-2D method (IPN-V2) was proposed, which for the first time simultaneously integrates and segments RC, RA, RV and FAZ. Subsequently, ODDF-Net enhances discriminability by introducing optical density features and combines decoupled segmentation heads with disease features to assist in optimizing multi-target segmentation performance, effectively solving the problems of segmentation confusion and insufficient accuracy caused by the high morphological similarity between arteries and veins and structural variations caused by ophthalmic diseases. However, existing methods still face multiple challenges: (1) Large-scale lesions and small lesions may coexist in the same image, requiring the network to dynamically fuse features of different scales; (2) Accurate segmentation of lesion boundaries and textures in medical images depends on high-frequency information, but existing methods are difficult to preserve such details; (3) Traditional CNN models have limited receptive fields and are difficult to capture global contextual information in images. Summary of the Invention

[0006] To address the aforementioned issues, this invention presents a meticulously designed MSA-ODDF-Net network based on the ODDF-Net multi-target segmentation network for multi-target segmentation in optical coherence tomography (OCTA) angiography. This network not only effectively extracts RC, RA, RV, and FAZ parameters, but also enables multi-scale dynamic lesion modeling, frequency-space collaborative perception, and efficient global context modeling, significantly improving segmentation robustness under complex pathological conditions.

[0007] The present invention provides a retinal OCTA multi-target segmentation method based on a dual-branch attention network, comprising the following steps: A multi-object segmentation network was constructed to segment retinal optical coherence tomography (OCTA) angiography data; The multi-target segmentation network includes a 3D path, a 2D path, and a boundary-aware shared feature decoupling module. The 3D path is used to extract 2D features after full projection as features output by the 3D path, and to extract multiple projection features and integrate the projection features into the 2D path. The features output from the 2D path and the 3D path are input into the boundary-aware shared feature decoupling module, which outputs the segmentation results of retinal capillaries, retinal arteries, retinal veins and avascular areas of the fovea.

[0008] Furthermore, the 3D path includes convolutional layers and three feature compression modules (FPMs). The 3D path receives 3D OCTA volume data and 3D optical density volume data as input, which are then convolved by convolutional layers and stitched together. After passing through three fast projection modules (FPM) for continuous feature dimensionality reduction and projection, the fully projected 2D features are obtained and used as the feature input of the 3D path output to the boundary-aware shared feature decoupling module. The extraction of multiple projection features and the integration of these features into the 2D path specifically refers to the following: each Fast Projection Module (FPM) outputs the projection features for that stage, and the projection features from different stages are integrated into the encoder of the 2D path through the Cross-Dimensional Feature Fusion Module (CFF).

[0009] Furthermore, in the encoder stage of the 2D path, a three-scale aggregation channel attention mechanism (TACA) is introduced to dynamically fuse the feature maps output from adjacent levels to obtain fused features. The fused features are then input into the extended frequency-space attention module to output features optimized by the dual branches in the frequency and spatial domains.

[0010] Furthermore, the encoder stage includes a multi-layer encoder, employing a three-scale aggregated channel attention mechanism (TACA) to dynamically fuse the feature maps output from three adjacent layers in the encoder, resulting in fused features. The specific steps include: Feature maps output by the encoders at layers (i-1), i, and i+1) , , Global average pooling and 1×1 convolution are used to achieve channel alignment; The aligned features are concatenated along the channel dimension, and an attention weight matrix M is generated by Sigmoid activation. The attention matrix M is divided into three sub-matrices, and the feature maps input to TACA are processed based on these three sub-matrices respectively. , , Channel weighting is performed to obtain shallow features. ,feature Deep features ; shallow features Perform downsampling and deep feature extraction. After upsampling, then with features The fused features are obtained through residual connections. .

[0011] Furthermore, the fused features are input into the extended frequency-spatial attention module, which outputs features optimized by dual branches in the frequency and spatial domains. Specific steps include: (3.1) Fusion features of the input Perform a 2D Fourier transform to obtain the frequency domain feature F. i ; (3.2) The frequency domain features are separated by low-frequency and high-frequency masks to obtain the separated high-frequency features. and low frequency characteristics and the separated low-frequency features By applying a learnable filter for weighting, the weighted low-frequency features are obtained. ; (3.3) The weighted low-frequency features High frequency characteristics After superposition, a 2D inverse Fourier transform is performed to restore the spatial domain, yielding the reconstructed frequency domain features. ; (3.4) Simultaneously, the fusion features of the input A spatial weight map is generated by using a spatial attention branch to fuse the input features. Weighting is performed to obtain spatial attention features. ; (3.5) Reconstructing features in the frequency domain Spatial attention characteristics Adaptive fusion is performed using learnable parameters to output features optimized by dual branches in the frequency and spatial domains. .

[0012] Furthermore, a first Mamba-inspired linear attention module is introduced before the first downsampling stage in the 2D path, and a second Mamba-inspired linear attention module is introduced after the last upsampling stage. The Mamba-inspired linear attention module is as follows. The features input to the Mamba-inspired linear attention module are first subjected to linear transformation, convolution, and channel multiplication to generate intermediate features. , and , means as follows: ; in, For convolution operations, For linear transformation, This represents channel-by-channel multiplication;

[0013]

[0014] Output is the final output of the MILA module. Attention mechanisms that represent linear computational complexity.

[0015] Furthermore, the boundary-aware shared feature decoupling module includes multiple decoupling branches, which respectively output segmentation probability maps of retinal capillaries, arteries, veins, and avascular regions of the fovea.

[0016] Furthermore, a hybrid loss function is used to train the multi-objective segmentation network model, including cross-entropy loss for segmentation tasks and cross-entropy loss for disease classification tasks, expressed as: ; It is the cross-entropy loss used for the segmentation task. It is the cross-entropy loss used for classification tasks. Represents the mixed loss function;

[0017] Furthermore, the learnable filters in the Extended Frequency-Spatial Attention (EFSD) module are initialized according to a normal distribution.

[0018] Beneficial Effects: The MSA-ODDF-Net proposed in this invention solves the core pain points of existing OCTA (Optical Coherence Tomography) multi-target segmentation, namely "difficulty in adapting multi-scale lesions, loss of high-frequency details, and low efficiency of global modeling," by integrating multi-scale attention mechanisms, frequency-spatial domain collaborative perception, and efficient global context modeling technology. It achieves high-precision joint segmentation of RC (retinal capillaries), RA (retinal arteries), RV (retinal veins), and FAZ (foveal avascular zone). Its technical advantages are specifically reflected in the following aspects:

[0019] (1) Three-scale aggregation channel attention mechanism TACA dynamically fuses features of adjacent layers through a progressive multi-scale channel attention mechanism, reducing redundancy and enhancing the ability to model multi-scale lesions, while avoiding the loss of high-frequency details, thereby improving segmentation accuracy.

[0020] (2) EFSD-Attention decomposes features into texture and global structure components by introducing 2D Fourier transform, and uses learnable filters to adaptively adjust low-frequency weights to enhance global perception capabilities. At the same time, it directly preserves the original high-frequency components through mask separation, avoiding the loss of details during inverse transform and ensuring high-frequency integrity. In addition, EFSD-Attention adopts a dual-branch structure: the spatial attention branch focuses on local details, and the frequency domain branch complements global modeling through frequency domain filtering. Finally, feature fusion is used to achieve the synergistic optimization of local details in the spatial domain and global structure in the frequency domain.

[0021] (3) After MILA is integrated into UNet, by introducing the local bias and position information provided by the forget gate and the optimized block structure, it achieves complementarity between global modeling and CNN local perception, with linear complexity. It efficiently processes high-resolution images, avoiding the secondary computational overhead of traditional attention. Its parallel design significantly improves training and inference speed, while outperforming the Mamba model in OCTA segmentation tasks, achieving a balance between lightweight design and accuracy.

[0022] (4) The method proposed in this study shows significant advantages in the field of multi-target segmentation of optical coherence tomography (OCTA) for vascular imaging. Based on two seed sets of the OCTA-500 dataset, the method achieved an average crossover ratio of 87.95% and an average crossover ratio of 82.67% on the 3M dataset and the 6M dataset, respectively, which verifies its efficiency in complex structure segmentation tasks. Attached Figure Description

[0023] The accompanying drawings illustrate various embodiments generally by way of example rather than limitation, and are used, together with the specification and claims, to explain embodiments of the invention. Where appropriate, the same reference numerals are used in all drawings to refer to the same or similar parts. Such embodiments are illustrative and are not intended to be exhaustive or exclusive embodiments of the apparatus or method.

[0024] Figure 1 This is a schematic diagram of the method of the present invention; Figure 2 This is a schematic diagram of the connection relationship between the modules in this invention. Detailed Implementation

[0025] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0026] This invention presents a multi-target segmentation method for retinal optical coherence tomography (OCTA) angiography based on a dual-branch attention network. This method utilizes a deep learning network fusion of the three-scale aggregated channel attention mechanism (TACA), the extended frequency-spatial attention (EFSD-Attention) of the dilated kernel, and the Mamba-inspired linear attention module (MILA). Applied to multi-target segmentation of OCTA images, this method provides precise image analysis support for the clinical diagnosis and disease assessment of ophthalmic diseases such as diabetic retinopathy, retinal vein occlusion, and age-related macular degeneration.

[0027] This invention provides a multi-target segmentation method for retinal optical coherence tomography (OCTA) angiography based on MSA-ODDF-Net, such as... Figure 1 As shown, the specific steps are as follows:

[0028] Step 1: Acquire optical coherence tomography (OCTA) data for vascular imaging and construct a dual-branch network architecture that includes 3D and 2D paths;

[0029] 3D optical coherence tomography (OCTA) volumetric data of blood vessels were obtained from the OCTA-500 dataset. The data were divided into 3mm×3mm and 6mm×6mm subsets according to the field of view, and into training, validation and test sets according to the number. The 3D data were standardized by bilinear interpolation to generate 3D optical density (OD) volumetric data based on the optical density calculation model.

[0030] like Figure 2 As shown, a dual-branch network architecture including 3D and 2D paths is constructed: the 3D path contains multiple 3×3×3 convolutional layers and three feature compression modules (FPM). In the spatial feature extraction stage of the 3D path, the 3D path receives 3D OCTA volume data and generated 3D optical density (OD) volume data as input. Then, it uses three fast projection modules (FPM) to perform feature dimensionality reduction and projection, gradually compressing the complex 3D image features from 3D space to 2D space to obtain fully projected 2D features. The features extracted by this path play a key role in subsequent processes: on the one hand, the fully projected 2D features are input to the boundary-aware shared feature decoupling module for region segmentation or to the auxiliary classification head (ACH) to assist in disease prediction. The diseases include age-related macular degeneration (AMD), choroidal neovascularization (CNV), diabetic retinopathy (DR), and normal non-retinopathy (NOR). On the other hand, the projected features from different stages are directly integrated into the encoder of the 2D path through the cross-dimensional feature fusion (CFF) module, thereby achieving efficient cross-dimensional information integration.

[0031] The 2D path uses a U-Net architecture. The cross-dimensional feature fusion module (CFF) processes the 3D features output by the feature compression module (FPM). Specifically, it extracts the maximum and average values ​​of the 3D projected features along each channel dimension, fuses them using 1×1 convolutions, and aligns them spatially using max pooling. Then, it hierarchically concatenates these features with the corresponding three scale-level features from the 2D path encoder, achieving efficient cross-dimensional information integration and convolution feature reuse. In this specific implementation, the cross-dimensional feature fusion module (CFF) adopts the CFF structure from the paper ODDF-Net: Multi-object segmentation in 3Dretinal OCTA using optical density and disease features (Knowledge-Based Systems).

[0032] Integrating 3D projection features into a 2D encoder enhances spatial awareness, thereby enabling cross-path feature fusion.

[0033] Step 2: Multi-scale feature optimization of the Three-Scale Aggregated Channel Attention Mechanism (TACA) module

[0034] The 2D path encoder , , The feature map output from the layer is input to the three-scale aggregation channel attention mechanism TACA. After preprocessing, attention modeling and fusion, optimized features are generated. The specific steps are as follows:

[0035] Step 2.1, given the encoder's first... Layer output feature map and its next layer features and features of the previous layer ,in Indicates the feature map height. Indicates the width of the feature map. This represents the number of channels in the feature map. First, feature normalization is performed:

[0036]

[0037]

[0038]

[0039] in This represents a global average pooling operation. Channel alignment is then achieved using a 1×1 convolutional layer.

[0040]

[0041]

[0042]

[0043] in This represents a 1×1 convolutional layer.

[0044] Step 2.2: Concatenate the aligned features along the channel dimension and generate an attention weight matrix through a non-linear transformation. :

[0045]

[0046] in This indicates a channel splicing operation. This is the Sigmoid activation function.

[0047] Step 2.3: Divide the attention matrix into three sub-matrices:

[0048]

[0049] in This represents a matrix segmentation operation, where the dimensions of the segmented matrix must be compared with those of the original features. and Channel number matching, i.e. , , .

[0050] Step 2.4: Perform channel-weighted processing on the original features:

[0051]

[0052] in This indicates channel-by-channel multiplication.

[0053] shallow features Perform transposed convolution downsampling:

[0054]

[0055] For deep features Perform transposed convolution upsampling:

[0056]

[0057] The final fused features are obtained through residual connections:

[0058]

[0059] In the formula This represents element-wise addition. Represents downsampling, Represents upsampling, This represents the final fused feature matrix output by the module.

[0060] Step 3: Frequency-spatial collaborative processing of the extended kernel's frequency-spatial attention (EFSD-Attention).

[0061] The output features of the three-scale aggregated channel attention mechanism (TACA) are input into the extended frequency-spatial attention (EFSD) module of the extended kernel. Through filtering in the frequency domain branch and local enhancement in the spatial domain branch, the frequency-spatial co-function features are output. The specific steps are as follows:

[0062] Step 3.1, the output features of the three-scale convergent channel attention mechanism (TACA) are: Perform a 2D Fourier transform (2D-DFT) on each channel:

[0063]

[0064] in, This represents the pixel value (or feature value) at coordinates (x, y) in the input feature space domain. Represents the frequency domain feature tensor. For frequency domain coordinates, These are the spatial coordinates corresponding to the frequency domain. It is the imaginary unit.

[0065] Step 3.2, Define the low-frequency mask central area , 1 for all others; high-frequency mask Separate frequency domain features:

[0066]

[0067]

[0068] This represents the separated high-frequency features. This indicates the separated low-frequency characteristics;

[0069] Step 3.3, Introduce a learnable filter The initialization follows a normal distribution. Weighting of low-frequency features:

[0070]

[0071] Step 3.4, and After superposition, perform a 2D inverse Fourier transform (2D-iDFT) to restore the spatial domain:

[0072]

[0073] Step 3.5, for Perform global average pooling, input a layer containing a 5×5 dilated convolution and a sigmoid activation, and generate a spatial weight map. :

[0074]

[0075] in, This indicates a global average pooling operation. This represents a 5×5 convolution kernel. This is the Sigmoid activation function.

[0076] Step 3.6, for Perform spatial feature weighting to generate spatial attention features :

[0077]

[0078] in This is a channel-by-channel multiplication.

[0079] Step 3.7, Frequency Domain Reconstruction Features Spatial attention characteristics Feature fusion employs an adaptive weight fusion strategy:

[0080]

[0081] in These are learnable parameters that ensure the coordinated enhancement of the global structure in the frequency domain and the local details in the spatial domain.

[0082] Step 4: Global Context Modeling of the MAMBA-Inspired Linear Attention Module MILA

[0083] In the 2D path, before the encoder's first downsampling and after the decoder's last upsampling, a MAMBA-inspired linear attention module MILA is embedded. Through feature transformation, linear attention calculation and positional encoding fusion, global feature association is enhanced. The steps are as follows:

[0084] Step 4.1, Input Features ,in Number of pixels Given the number of channels, intermediate features are generated through three paths:

[0085]

[0086] in For convolution operations, For linear transformation, This indicates channel-by-channel multiplication.

[0087] Step 4.2: Generate the final output through linear transformation and residual connection:

[0088]

[0089]

[0090] Output is the final output of the MILA module. Attention mechanisms that represent linear computational complexity.

[0091] Step 5: Model Training and Multi-Object Segmentation Output

[0092] A hybrid loss function is defined, and a two-stage training strategy is adopted to optimize the model. The fully projected 2D features from the 3D path and the output features from the 2D path decoder are input into the boundary-aware shared feature decoupled DSH module. The segmentation results of four types of retinal structures are output through the boundary-aware shared feature decoupled DSH module. The performance is verified using metrics such as mIoU, Dice, and HD95. The specific implementation includes the following sub-steps:

[0093] Step 5.1, Total Loss Due to segmentation loss Disease classification loss constitute:

[0094]

[0095]

[0096] in, , Let n be the total number of voxels, m be the total number of categories, and n be the total number of voxels. and Let r represent the probability that voxel r belongs to class t in the 3D input image and the true label, respectively. The classification loss weight was set to 0.2. It is the cross-entropy loss used for the segmentation task. It is the cross-entropy loss used for classification tasks.

[0097] Step 5.2, Two-stage training strategy. The PyTorch framework is used, based on an NVIDIA RTX 2080 Ti with 11GB of VRAM. The activation function is LeakyReLU with a negative slope of 0.2, and the optimizer is Adam with momentum parameters... , .

[0098] Step 5.2.1, first stage training for 250 epochs. Remove ACH module parameters, optimize only the segmentation network; 3mm subset batch size is 4, 6mm subset batch size is 2; initial learning rate is... .

[0099] Step 5.2.2, Second phase training (20 rounds): For the first 10 rounds, freeze all parameters and train only ACH, with an initial learning rate of... The 3D path parameters were fine-tuned in the last 10 rounds of unfreezing; the batch size was uniformly set to 4; and the learning rate was reduced to... The 2D path parameters are fixed throughout the process to avoid segmentation performance degradation.

[0100] Step 5.3, Multi-target Segmentation and Performance Validation: The DSH module contains 5 decoupled branches (retinal capillaries, arteries, veins, avascular areas of the fovea, and background). The output segmentation probability map is evaluated using the following metrics: Dice coefficient, Intersection over Union (IoU), 95% Hausdorff distance (HD95), Accuracy (ACC), Sensitivity (SE), Specificity (SP), Precision-Recall (PR) curve, and Average Precision (AP). Precision quantifies the proportion of true positive samples among all predicted positive samples, while recall measures the proportion of correctly predicted positive samples among all actual positive samples. The definitions of each metric are as follows:

[0101]

[0102]

[0103]

[0104]

[0105]

[0106]

[0107]

[0108]

[0109] in, This represents the set of segmentation boundary points predicted by the model. This represents the set of true boundary points labeled. Represents the prediction point set Any boundary pixel on the surface, Represents the real point set Any boundary pixel on the surface, It is a true positive result. It's a false positive. It is a true negative. It is a false negative. Defined as and The 95th percentile of the Hausdorff distance between them. To measure the overall segmentation performance of the model for all retinal objects, the mean intersection-union ratio (mIoU) is used as an important evaluation metric, and its calculation formula is as follows:

[0110]

[0111] in, The number of targets to be segmented.

[0112] The above description is merely a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the technical scope disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A multi-target segmentation method for retinal OCTA based on a dual-branch attention network, characterized in that, Includes the following steps: A multi-object segmentation network was constructed to segment retinal optical coherence tomography (OCTA) angiography data; The multi-target segmentation network includes a 3D path, a 2D path, and a boundary-aware shared feature decoupling module. The 3D path is used to extract 2D features after full projection as features output by the 3D path, and to extract multiple projection features and integrate the projection features into the 2D path. The features output from the 2D path and the 3D path are input into the boundary-aware shared feature decoupling module, which outputs the segmentation results of retinal capillaries, retinal arteries, retinal veins and avascular areas of the fovea.

2. The method according to claim 1, characterized in that, The 3D path includes convolutional layers and three feature compression modules (FPMs). The 3D path receives 3D OCTA volume data and 3D optical density volume data as input, which are then convolved by convolutional layers and stitched together. After passing through three fast projection modules (FPM) for continuous feature dimensionality reduction and projection, the fully projected 2D features are obtained and used as the feature input of the 3D path output to the boundary-aware shared feature decoupling module. The extraction of multiple projection features and the integration of these features into the 2D path specifically refers to the following: each Fast Projection Module (FPM) outputs the projection features for that stage, and the projection features from different stages are integrated into the encoder of the 2D path through the Cross-Dimensional Feature Fusion Module (CFF).

3. The method according to claim 1, characterized in that, In the encoder stage of the 2D path, a three-scale aggregation channel attention mechanism (TACA) is introduced to dynamically fuse the feature maps output from adjacent levels to obtain fused features. The fused features are then input into the extended frequency-space attention module to output features optimized by the dual branches in the frequency and spatial domains.

4. The method according to claim 3, characterized in that, The encoder stage includes a multi-layer encoder, and a three-scale aggregated channel attention mechanism (TACA) is used to dynamically fuse the feature maps output from three adjacent layers in the encoder to obtain the fused features. The specific steps include: Feature maps output by the encoders at layers (i-1), i, and i+1) , , Global average pooling and 1×1 convolution are used to achieve channel alignment; The aligned features are concatenated along the channel dimension, and an attention weight matrix M is generated by Sigmoid activation. The attention matrix M is divided into three sub-matrices, and the feature maps input to TACA are processed based on these three sub-matrices respectively. , , Channel weighting is performed to obtain shallow features. ,feature Deep features ; shallow features Perform downsampling and deep feature extraction. After upsampling, then with features The fused features are obtained through residual connections. .

5. The method according to claim 3, characterized in that, The fused features are input into the extended frequency-spatial attention module, which outputs features optimized by dual branches in the frequency and spatial domains. Specific steps include: (3.1) Fusion features of the input Perform a 2D Fourier transform to obtain the frequency domain feature F. i ; (3.2) The frequency domain features are separated by low-frequency and high-frequency masks to obtain the separated high-frequency features. and low frequency characteristics and the separated low-frequency features By applying a learnable filter for weighting, the weighted low-frequency features are obtained. ; (3.3) The weighted low-frequency features High frequency characteristics After superposition, a 2D inverse Fourier transform is performed to restore the spatial domain, yielding the reconstructed frequency domain features. ; (3.4) Simultaneously, the fusion features of the input A spatial weight map is generated by using a spatial attention branch to fuse the input features. Weighting is performed to obtain spatial attention features. ; (3.5) Reconstructing features in the frequency domain Spatial attention characteristics Adaptive fusion is performed using learnable parameters to output features optimized by dual branches in the frequency and spatial domains. .

6. The method according to claim 1, characterized in that, A first Mamba-inspired linear attention module is introduced before the first downsampling stage in the 2D path, and a second Mamba-inspired linear attention module is introduced after the last upsampling stage. The specific Mamba-inspired linear attention modules are as follows. The features input to the Mamba-inspired linear attention module are first subjected to linear transformation, convolution, and channel multiplication to generate intermediate features. , and , means as follows: ； in, For convolution operations, For linear transformation, This represents channel-by-channel multiplication; Output is the final output of the MILA module. Attention mechanisms that represent linear computational complexity.

7. The method according to claim 1, characterized in that, The boundary-aware shared feature decoupling module contains multiple decoupling branches, which respectively output segmentation probability maps of retinal capillaries, arteries, veins, and avascular areas of the fovea.

8. The method according to claim 1, characterized in that, It also includes inputting the features output by the 3D path into the Auxiliary Classification Head (ACH) to assist in disease classification.

9. The method according to claim 8, characterized in that, A hybrid loss function is used to train the multi-objective segmentation network model, including cross-entropy loss for segmentation tasks and cross-entropy loss for disease classification tasks, expressed as: It is the cross-entropy loss used for the segmentation task. It is the cross-entropy loss used for classification tasks. This represents the mixed loss function.

10. The method according to claim 1, characterized in that, The learnable filters in the Extended Frequency-Spatial Attention (EFSD) module are initialized according to a normal distribution.

Images