Retinal neovascularization segmentation method based on double-path multi-scale attention network

Abstract

The application discloses a retinal neovascularization segmentation method based on a double-path multi-scale attention network, and belongs to the technical field of medical information processing. The method comprises the following steps: determining a WF-OCTA image of retinal neovascularization, drawing a segmentation mask on the WF-OCTA image, and obtaining input data; extracting multi-scale features of the input data through an encoder-decoder and establishing a skip connection to obtain a multi-level input feature tensor; performing double-path multi-scale feature mining on the input feature tensor, capturing morphological information and context information in parallel, and obtaining a residual enhanced output feature tensor; performing statistical spatial-spectral attention enhancement on the residual enhanced output feature tensor, performing channel re-labeling and spatial fine positioning based on a statistical quantity, and obtaining a statistical enhanced output feature tensor; fusing the statistical enhanced output feature tensor in a decoding path and gradually reconstructing to obtain a pixel-level probability map, training the pixel-level probability map through a hybrid loss, and outputting a segmentation result.

Description

Technical Field

[0001] This application belongs to the field of medical information processing technology, specifically relating to a method for retinal neovascularization segmentation based on a dual-path multi-scale attention network. Background Technology

[0002] Retinal neovascularization is a key marker of the progression to the proliferative phase of various retinal vascular diseases, including diabetic retinopathy, retinal vein occlusion, and retinopathy of prematurity. These vessels have thin walls, high permeability, and are often accompanied by fibrous proliferation, easily leading to vitreous hemorrhage and tractional retinal detachment, ultimately causing severe and irreversible visual impairment. Clinical evidence-based studies have shown that timely identification and intervention during the non-proliferative phase or early stages of neovascularization can significantly reduce the risk of vision loss and the burden of treatment. Therefore, early and accurate identification and quantification of neovascularization are of clear clinical necessity.

[0003] In terms of imaging techniques, traditional early assessment mainly relies on fluorescein fundus angiography (FFA). FFA can show leakage and perfusion dynamics, making it an important tool for assessing lesion activity; however, it is an invasive examination with the risk of allergies, and leakage and hemorrhage may obscure small lesions. It also has limited coverage of the peripheral retina, and the accessibility of objective quantification and repeated follow-up is limited. Complementing this, optical coherence tomography (OCTA) has the advantages of being non-invasive, high-resolution, and capable of layered imaging. It can clearly show the microvascular network and support objective quantitative indicators such as lesion area, skeleton length, number of branches / rings, tortuosity, and fractal dimension, facilitating follow-up management. Building on this, wide-angle optical coherence tomography (WF-OCTA) further expands the field of view to the peripheral retina. Through large-area scanning or multi-field stitching, it significantly improves the detection capability of peripheral RNVs and alleviates the blind spots in central visual field imaging. However, wide-angle scenes are more susceptible to motion and projection artifacts, segmentation faults, low signal-to-noise ratios in the surrounding area, and cross-device differences, which increases the workload and inconsistencies of manual interpretation and makes stable quantization more difficult. Summary of the Invention

[0004] Purpose of the invention: This application develops a method for retinal neovascularization segmentation based on a dual-path multi-scale attention network, aiming to solve the technical problem in the prior art that neovascularization cannot be accurately distinguished and segmented under conditions of complex artifacts and extremely sparse foreground.

[0005] Technical solution: Embodiments of this application provide a method for retinal neovascularization segmentation based on a dual-path multi-scale attention network, including:

[0006] Determine the WF-OCTA image of retinal neovascularization, draw a segmentation mask on the WF-OCTA image, and obtain input data;

[0007] Multi-scale features of the input data are extracted by an encoder-decoder and skip connections are established to obtain multi-level input feature tensors.

[0008] Dual-path multi-scale feature mining is performed on the input feature tensor to capture morphological and contextual information in parallel, and residual-enhanced output feature tensor is obtained.

[0009] The residual enhanced output feature tensor is subjected to statistical spatial-spectral attention enhancement, channel recalibration and spatial fine localization based on statistics to obtain the statistically enhanced output feature tensor;

[0010] The statistically enhanced output feature tensor is fused in the decoding path and reconstructed step by step to obtain a pixel-level probability map. The map is then trained with a hybrid loss and the segmentation result is output.

[0011] In some embodiments, the WF-OCTA image is characterized as follows:

[0012] ;

[0013] in, This is the input tensor used to characterize the WF-OCTA image; Constructed from real numbers, possessing One channel, height is Width is tensor;

[0014] The characterization formula for the segmentation mask includes:

[0015] ;

[0016] in, The segmentation mask; 255 represents a black pixel; 255 represents a white pixel.

[0017] In some embodiments, the step of obtaining input data includes:

[0018] Spatial consistency resampling is performed on the WF-OCTA image and the segmentation mask to unify the size of the WF-OCTA image and the segmentation mask;

[0019] The intensity of the WF-OCTA image is scaled to a preset numerical range to obtain the input data, the representation formula of the input data including:

[0020] ;

[0021] in, The input data is used to characterize the WF-OCTA image after spatial consistency resampling and intensity scaling.

[0022] In some embodiments, the step of extracting multi-scale features from the input data via an encoder-decoder and establishing skip connections to obtain a multi-scale feature pyramid includes:

[0023] The input data is downsampled step by step by an encoder to obtain downsampled data.

[0024] The downsampled data is upsampled step by step by the decoder, and the high-level features are scale-aligned with the low-level features.

[0025] The high-level features and the corresponding scale-based encoding features are skip-connected in the channel dimension and fused through convolutional blocks to obtain the multi-level input feature tensor. The representation formula of the multi-level input feature tensor includes:

[0026] ;

[0027] ;

[0028] in, It is a set of feature tensors containing S+1 levels, used to characterize the multi-level input feature tensors; Let s be the input feature tensor of level s; The number of channels for the s-th level input feature tensor; It is the set of real numbers; and Let be the resolution of the s-th level input feature tensor. and It decreases as s increases, in order to carry hierarchical representations from fine-grained to high semantic levels.

[0029] In some embodiments, the step of performing dual-path multi-scale feature mining on the input feature tensor, capturing morphological information and contextual information in parallel, and obtaining a residual-enhanced output feature tensor includes:

[0030] The number of channels in the input feature tensor is mapped to the number of intermediate channels based on the channel compression function;

[0031] Based on the number of intermediate channels, a two-dimensional convolution is applied to the input feature tensor to obtain the morphological branch output feature tensor. The representation formula of the morphological branch output tensor includes:

[0032] ;

[0033] in, Output a tensor for the morphological branch; for Two-dimensional convolution; The number of channels of the input feature tensor; This refers to the number of intermediate channels; For the first Level input feature tensor; It is a finite set of convolution kernel sizes;

[0034] Dilated convolution is applied to the input feature tensor based on the number of intermediate channels to obtain the context branch output feature tensor, the representation formula of the context branch output feature tensor includes:

[0035] ;

[0036] in, Output a feature tensor for the context branch; The void ratio is of Two-dimensional dilated convolution; Let be a finite set of positive integers representing the void ratio;

[0037] By concatenating the morphological branch output feature tensor and the context branch output feature tensor along the channel dimension, a multi-branch aggregated feature tensor is obtained. The representation formula of the multi-branch aggregated feature tensor includes:

[0038] ;

[0039] in, For the multi-branch aggregation feature tensor; For splicing operations;

[0040] The multi-branch aggregated feature tensor is fused using two-dimensional convolution to obtain a channel-fused feature tensor. The representation formula of the channel-fused feature tensor includes:

[0041] ;

[0042] in, The channel fusion feature tensor; To change the number of channels from Mapped to of Two-dimensional convolution, The number of channels of the multi-branch aggregated feature tensor;

[0043] The residual enhanced output feature tensor is obtained based on the channel fusion feature tensor and the input feature tensor. The representation formula of the residual enhanced output feature tensor includes:

[0044] ;

[0045] in, The residual is used to enhance the output feature tensor.

[0046] In some embodiments, the steps of performing statistical spatial-spectral attention enhancement on the residual enhanced output feature tensor, performing channel recalibration and statistically based spatial fine localization, and obtaining the statistically enhanced output feature tensor include:

[0047] A statistical pooling operator is applied to the residual enhancement output feature tensor, and the average value is calculated after nonlinear mapping through a two-layer perceptron to obtain the channel attention map. The representation formula of the channel attention map includes:

[0048] ;

[0049] in, This is the channel attention map; For Sigmoid activation operation; The set of statistical pooling operators used. for Cardinality; for The index variable in the set is used to characterize each specific pooling operator in the set of statistical pooling operators; The two-layer sensor; This refers to the channel reduction ratio; For statistical pooling operators; The residual enhancement output feature tensor; Composed of real numbers, having Each channel, resolution is The tensor.

[0050] Based on the residual enhancement output feature tensor and the channel attention map, a channel recalibration feature tensor is obtained. The representation formula of the channel recalibration feature tensor includes:

[0051] ;

[0052] in, Recalibrate the feature tensor for the channel; This is element-wise multiplication;

[0053] The channel recalibrated feature tensor is subjected to average pooling to obtain the channel mean map;

[0054] Max pooling is performed on the channel recalibration feature tensor to obtain the channel maximum value map;

[0055] Based on the channel recalibration feature tensor, a spatial location mean and standard deviation map are obtained. The representation formulas for the spatial location mean and the standard deviation map include:

[0056] ;

[0057] ;

[0058] in, The average value of the spatial location; for In the passage Spatial location The value at that location; The standard deviation plot; It is a tiny constant. This is used to ensure numerical stability;

[0059] The channel mean map, the channel maximum map, and the standard deviation map are concatenated along the channel dimension, and then subjected to two-dimensional convolution to obtain a spatial attention map. The representation formula of the spatial attention map includes:

[0060] ;

[0061] in, This is the spatial attention map; for Two-dimensional convolution is used to represent the mapping of three input channels to one output channel; For splicing operations; This is the mean value plot of the channel; This is a graph showing the maximum value of the channel; It consists of real numbers, has 1 channel, and has a resolution of [missing information]. tensor;

[0062] Based on the residual enhanced output feature tensor, the channel attention map, and the spatial attention map, a statistical enhanced output feature tensor is obtained. The representation formula of the statistical enhanced output feature tensor includes:

[0063] ;

[0064] in, The statistical enhancement output feature tensor is then used.

[0065] In some embodiments, the step of fusing the statistically enhanced output feature tensor in the decoding path and reconstructing it step by step to obtain a pixel-level probability map includes:

[0066] In the decoding path, the high-level features of the multi-level input feature tensor are upsampled so that the scale of the high-level features is the same as the scale of the low-level features.

[0067] The high-level features and the statistically enhanced output feature tensors of the corresponding scales of the high-level features are concatenated in the channel dimension and then fused through convolutional blocks to obtain fused features;

[0068] The fusion features at each level are reconstructed step by step to obtain the reconstructed features;

[0069] The reconstructed features are mapped to a single channel using two-dimensional convolution to obtain a pixel-level probability map. The representation formula for the pixel-level probability map includes:

[0070] ;

[0071] in, This is the pixel-level probability map. The elements in the matrix are used to characterize the probability that the corresponding pixel belongs to a new blood vessel; For Sigmoid activation operation; for Two-dimensional convolution is used to represent the number of channels that reconstruct the features. Mapped to a single channel; The tensor of the reconstructed features; It is a two-dimensional tensor with H×W elements, consisting of real numbers in the range [0,1].

[0072] In some embodiments, the step of training with a hybrid loss and outputting segmentation results includes:

[0073] The total loss is obtained through Dice loss, Focal loss, and Tversky loss, and the formula for representing the total loss includes:

[0074] ;

[0075] in, Total loss; The Dice loss is used to optimize the overlap. The weights of the Dice loss; The Focal loss is used to focus on segmented samples; The weights of the Focal loss; The Tversky loss is used for asymmetric penalty; The weights of the Tversky loss;

[0076] The formula for characterizing the Focal loss includes:

[0077] ;

[0078] in, The total number of pixels involved in the loss calculation; for The Middle The predicted probability of each pixel; For the segmentation mask, the first The truth label corresponding to each pixel. It can be 0 or 1; Let be the numerical stability constant of the Dice loss. ;

[0079] The formula for characterizing the Focal loss includes:

[0080] ;

[0081] ;

[0082] in, To predict the probability that a sample is a neovascularization for the model; The category balancing weight function; As a focusing factor, ; To prevent logarithmic underflow of tiny constants, ;

[0083] The characterization formula for the Tversky loss includes:

[0084] ;

[0085] in, To control the penalty weight for false positives, ; To control the penalty weight for false negatives, .

[0086] In some embodiments, the pooling operators included in the statistics include global average pooling, global max pooling, and / or local window average pooling.

[0087] Beneficial Effects: Compared with the prior art, the embodiments of this application provide a retinal neovascularization segmentation method based on a dual-path multi-scale attention network, including determining the WF-OCTA image of retinal neovascularization, drawing a segmentation mask on the WF-OCTA image, and obtaining input data; extracting multi-scale features of the input data through an encoder-decoder and establishing skip connections to obtain multi-level input feature tensors; performing dual-path multi-scale feature mining on the input feature tensors, capturing morphological information and contextual information in parallel, and obtaining residual enhanced output feature tensors; performing statistical spatial-spectral attention enhancement on the residual enhanced output feature tensors, performing channel recalibration and statistically based spatial fine localization, and obtaining statistically enhanced output feature tensors; fusing the statistically enhanced output feature tensors in the decoding path and reconstructing them step by step to obtain pixel-level probability maps, and training with a hybrid loss to output segmentation results. This application employs an end-to-end approach combining dual-path multi-scale feature mining with statistical spatial-spectral attention to construct hierarchical features, enhance attention, and optimize hybrid loss in WF-OCTA images, thereby achieving automatic segmentation of neovascularization. This solves the technical problem in existing technologies where neovascularization cannot be accurately distinguished and segmented under conditions of complex artifacts and extremely sparse foregrounds. Attached Figure Description

[0088] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0089] Figure 1 A flowchart illustrating the steps of the retinal neovascularization segmentation method based on a dual-path multi-scale attention network provided in this application embodiment;

[0090] Figure 2 A flowchart illustrating the steps for acquiring input data in the retinal neovascularization segmentation method based on a dual-path multi-scale attention network provided in this application embodiment;

[0091] Figure 3 A flowchart illustrating the steps for obtaining multi-level input feature tensors in the retinal neovascularization segmentation method based on a dual-path multi-scale attention network provided in this application embodiment;

[0092] Figure 4 A flowchart illustrating the steps for obtaining the residual enhancement output feature tensor in the retinal neovascularization segmentation method based on a dual-path multi-scale attention network provided in this application embodiment;

[0093] Figure 5A flowchart illustrating the steps for obtaining the statistical enhancement output feature tensor in the retinal neovascularization segmentation method based on a dual-path multi-scale attention network provided in this application embodiment;

[0094] Figure 6 A flowchart illustrating the steps for obtaining a pixel-level probability map in the retinal neovascularization segmentation method based on a dual-path multi-scale attention network provided in this application embodiment;

[0095] Figure 7 A module connection diagram of a retinal neovascularization segmentation system based on a dual-path multi-scale attention network provided in an embodiment of this application;

[0096] Figure labels: 10, Data acquisition module; 20, Data conversion module; 30, Dual-path feature mining module; 40, Statistical enhancement module; 50, Data output module. Detailed Implementation

[0097] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0098] Retinal neovascularization is a key marker of the progression to the proliferative phase of various retinal vascular diseases, including diabetic retinopathy, retinal vein occlusion, and retinopathy of prematurity. These vessels have thin walls, high permeability, and are often accompanied by fibrous proliferation, easily leading to vitreous hemorrhage and tractional retinal detachment, ultimately causing severe and irreversible visual impairment. Clinical evidence-based studies have shown that timely identification and intervention during the non-proliferative phase or early stages of neovascularization can significantly reduce the risk of vision loss and the burden of treatment. Therefore, early and accurate identification and quantification of neovascularization are of clear clinical necessity.

[0099] In terms of imaging techniques, traditional early assessment mainly relies on fluorescein fundus angiography (FFA). FFA can show leakage and perfusion dynamics, making it an important tool for assessing lesion activity; however, it is an invasive examination with the risk of allergies, and leakage and hemorrhage may obscure small lesions. It also has limited coverage of the peripheral retina, and the accessibility of objective quantification and repeated follow-up is limited. Complementing this, optical coherence tomography (OCTA) has the advantages of being non-invasive, high-resolution, and capable of layered imaging. It can clearly show the microvascular network and support objective quantitative indicators such as lesion area, skeleton length, number of branches / rings, tortuosity, and fractal dimension, facilitating follow-up management. Building on this, wide-angle optical coherence tomography (WF-OCTA) further expands the field of view to the peripheral retina. Through large-area scanning or multi-field stitching, it significantly improves the detection capability of peripheral RNVs and alleviates the blind spots in central visual field imaging. However, wide-angle scenes are more susceptible to motion and projection artifacts, segmentation faults, low signal-to-noise ratios in the surrounding area, and cross-device differences, which increases the workload and inconsistencies of manual interpretation and makes stable quantization more difficult.

[0100] Therefore, achieving automated and precise segmentation of neovascularization in WF-OCTA not only helps to detect small and weak-contrast lesions early and objectively, but also assesses lesion activity and progression risk based on area and morphological indicators, assisting in individualized treatment decisions and efficacy monitoring; at the same time, it can improve consistency across time points and across devices, reduce manual burden, and support large-scale clinical applications and standardized follow-up.

[0101] In recent years, deep learning has developed rapidly as a research hotspot in the field of artificial intelligence, especially in image data processing. Meanwhile, some deep learning-based methods have been proposed for blood vessel segmentation. However, most methods remain at the level of "vessel / background" binary classification, extracting the overall vascular network from the background, lacking fine-grained differentiation between pathological and normal blood vessels. Especially in the WF-OCTA scenario, research on lesion-level identification and fine segmentation of RNVs is relatively scarce, making it difficult to meet the clinical need for early and precise quantification.

[0102] In view of this, embodiments of this application provide a method for retinal neovascularization segmentation based on a dual-path multi-scale attention network, including determining a WF-OCTA image of retinal neovascularization, drawing a segmentation mask on the WF-OCTA image, and obtaining input data; extracting multi-scale features of the input data through an encoder-decoder and establishing skip connections to obtain multi-level input feature tensors; performing dual-path multi-scale feature mining on the input feature tensors, capturing morphological and contextual information in parallel, and obtaining residual-enhanced output feature tensors; performing statistical spatial-spectral attention enhancement on the residual-enhanced output feature tensors, performing channel recalibration and statistically-based spatial fine localization, and obtaining statistically enhanced output feature tensors; fusing the statistically enhanced output feature tensors in the decoding path and reconstructing them step by step to obtain pixel-level probability maps, and training with a hybrid loss to output segmentation results. This application employs an end-to-end approach combining dual-path multi-scale feature mining with statistical spatial-spectral attention to construct hierarchical features, enhance attention, and optimize hybrid loss in WF-OCTA images, thereby achieving automatic segmentation of neovascularization. This solves the technical problem in existing technologies where neovascularization cannot be accurately distinguished and segmented under conditions of complex artifacts and extremely sparse foregrounds, offering the following advantages:

[0103] (1) Dual-path multi-scale feature mining to decouple morphologically relevant and context-relevant scales in parallel: one branch targets fine-grained geometry and texture, while the other branch extends the effective receptive field to capture long-range cues, improving the stability of cross-scale representations without sacrificing resolution.

[0104] (2) The spectral-spatial attention mechanism is applied to suppress redundant channels and highlight the regions to be highlighted. The second-order statistic (standard deviation) is combined in the spatial branch to capture the variability, which reduces the instability of relying only on the first-order statistic under noise.

[0105] (3) A mixed loss of Dice loss, Focal loss and Tversky loss is adopted to achieve an adjustable balance between sensitivity and specificity under class imbalance conditions, effectively reducing false negatives and false positives and improving overall consistency.

[0106] In some embodiments, please refer to Figure 1 , Figure 1 This is a flowchart illustrating the steps of the retinal neovascularization segmentation method based on a dual-path multi-scale attention network provided in this application. The retinal neovascularization segmentation method based on a dual-path multi-scale attention network in this application is specifically implemented through steps 100 to 500:

[0107] Step 100: Determine the WF-OCTA image of retinal neovascularization, draw a segmentation mask on the WF-OCTA image, and obtain input data.

[0108] In some embodiments, the representation formula for WF-OCTA images is:

[0109] ;

[0110] in, This is the input tensor used to characterize the WF-OCTA image; Constructed from real numbers, possessing One channel, height is Width is The tensor.

[0111] In some embodiments, the characterization formula for the segmentation mask includes:

[0112] ;

[0113] in, For segmentation mask; 255 represents a black pixel; 255 represents a white pixel.

[0114] In some embodiments, please refer to Figure 2 , Figure 2 This is a flowchart illustrating the steps for obtaining input data in the retinal neovascularization segmentation method based on a dual-path multi-scale attention network provided in this application embodiment. The method for obtaining input data in this application is specifically implemented through steps 110 to 120:

[0115] Step 110: Perform spatial consistency resampling on the WF-OCTA image and segmentation mask to unify the size of the WF-OCTA image and segmentation mask.

[0116] Step 120: Scale the intensity of the WF-OCTA image to a preset numerical range to obtain input data.

[0117] In some embodiments, the representation formula for the input data includes:

[0118] ;

[0119] in, The input data is used to characterize the WF-OCTA image after spatial consistency resampling and intensity scaling.

[0120] Understandably, the WF-OCTA image is represented as an input tensor. The mask marked by the doctor is represented as ,in, H represents the number of input channels (at least 1, used for compatibility with single-channel or multi-channel representations generated by preprocessing, without limiting the specific structure), and W represents the unified spatial dimensions. This indicates that M has both black and white attributes. Consistent spatial resampling is performed on X and M to obtain a uniform size. X is intensity-scaled to a preset numerical range while preserving the binary attributes of M. During the training phase, pairwise geometric enhancement and image-only photometric enhancement can be used; no enhancement is applied during the validation and testing phases. This process yields an input that meets the uniform specification. This serves as the direct input for step 200.

[0121] Step 200: Extract multi-scale features of the input data through encoder-decoder and establish skip connections to obtain multi-level input feature tensors.

[0122] In some embodiments, please refer to Figure 3 , Figure 3 This is a flowchart illustrating the steps involved in obtaining multi-level input feature tensors in a retinal neovascularization segmentation method based on a dual-path multi-scale attention network provided in this application. Specifically, the method for obtaining multi-level input feature tensors in this application is implemented through steps 210 to 230:

[0123] Step 210: The input data is downsampled step by step by the encoder to obtain downsampled data.

[0124] Step 220: Upsample the downsampled data step by step using the decoder, and scale-align the high-level features with the low-level features.

[0125] Step 230: Perform skip connections between high-level features and corresponding scale-based encoding end features in the channel dimension, and fuse them through convolutional blocks to obtain multi-level input feature tensors.

[0126] In some embodiments, the representation formula for the multi-level input feature tensor includes:

[0127] ;

[0128] ;

[0129] in, It is a set of feature tensors containing S+1 levels, used to represent multi-level input feature tensors; Let s be the input feature tensor of level s; The number of channels for the s-th level input feature tensor; It is the set of real numbers; and Let be the resolution of the s-th level input feature tensor. and It decreases as s increases to accommodate hierarchical representations from fine-grained to high-semantic levels. In subsequent steps, each level of feature will be processed in the same way.

[0130] Understandably, in order to preserve details and alleviate the semantic gap during the decoding process, high-level features are upsampled (denoted as Up(·), used to expand spatial resolution) and aligned to the same scale as the low-level features. Then, they are spliced ​​together with the corresponding scale of the encoded end features in the channel dimension (denoted as Concat[·]) and fused by convolutional blocks to restore spatial details step by step and provide aligned input for the scale-symmetric embedding in steps 300 and 400.

[0131] Step 300: Perform dual-path multi-scale feature mining on the input feature tensor, capture morphological information and contextual information in parallel, and obtain residual-enhanced output feature tensor.

[0132] In some embodiments, please refer to Figure 4 , Figure 4 This is a flowchart illustrating the steps for obtaining the residual enhancement output feature tensor in the retinal neovascularization segmentation method based on a dual-path multi-scale attention network provided in this application embodiment. The method for obtaining the residual enhancement output feature tensor in this application is specifically implemented through steps 310 to 360:

[0133] Step 310: Map the number of channels of the input feature tensor to the number of intermediate channels based on the channel compression function.

[0134] Understandably, in order to form a lightweight and efficient multi-branch bottleneck, the number of s-th level channels is... Through channel compression function Mapped to the number of intermediate channels This aims to reduce the number of parameters and computational overhead while maintaining representational capabilities.

[0135] Step 320: Based on the number of intermediate channels, apply a two-dimensional convolution to the input feature tensor to obtain the morphological branch output feature tensor.

[0136] In some embodiments, the representation formula for the morphological branch output tensor includes:

[0137] ;

[0138] in, Output tensors for morphological branches; for Two-dimensional convolution; The number of channels in the input feature tensor; This refers to the number of intermediate channels; For the first Level input feature tensor; Given a finite set of convolution kernel sizes, odd-numbered kernels are preferred in this application.

[0139] Understandably, in order to capture geometric shapes and textural details at different scales, Apply convolution kernel size sets separately Two-dimensional convolution in the model yields the morphological branch output feature tensor. .

[0140] Step 330: Apply dilated convolution to the input feature tensor based on the number of intermediate channels to obtain the context branch output feature tensor.

[0141] In some embodiments, the representation formula for the context branch output feature tensor includes:

[0142] ;

[0143] in, Output the feature tensor for the context branch; The void ratio is of Two-dimensional dilated convolution; Let be a finite set of positive integers representing the void ratio.

[0144] Understandably, in parallel with morphological branching, to expand the effective receptive field and model long-range dependencies without downsampling, for Apply void ratio set The dilated convolution yields the context branch output feature tensor. .

[0145] Step 340: Concatenate the morphological branch output feature tensor and the context branch output feature tensor along the channel dimension to obtain the multi-branch aggregated feature tensor.

[0146] In some embodiments, the representation formula for the multi-branch aggregate feature tensor includes:

[0147] ;

[0148] in, For multi-branch aggregated feature tensors; This is for splicing operations.

[0149] Step 350: Perform channel fusion on the multi-branch aggregated feature tensor through two-dimensional convolution to obtain the channel fused feature tensor.

[0150] In some embodiments, the representation formula for the channel fusion feature tensor includes:

[0151] ;

[0152] in, For channel fusion feature tensors; To change the number of channels from Mapped to of Two-dimensional convolution, This represents the number of channels in the multi-branch aggregated feature tensor.

[0153] Understandably, in order to fall back to the original number of channels at level s and facilitate the establishment of residual connections with the input, for Apply Two-dimensional convolution is used to perform channel fusion, resulting in a channel fusion feature tensor. .

[0154] Step 360: Obtain the residual enhancement output feature tensor based on the channel fusion feature tensor and the input feature tensor.

[0155] In some embodiments, the representation formula for the residual enhancement output feature tensor includes:

[0156] ;

[0157] in, This enhances the output feature tensor of the residual.

[0158] Understandably, to stabilize the gradient and preserve the information pathway, the channel fusion result is added to the input identity to obtain the residual-enhanced output feature tensor. .

[0159] Furthermore, a dual-path multi-scale feature mining module is applied to the input feature tensors at each level to capture morphological and contextual information in parallel and output residual enhancement features as direct input to the attention module. These features then participate in cross-scale fusion and progressive reconstruction at the decoding end through skip connections.

[0160] Step 400: Perform statistical spatial-spectral attention enhancement on the residual enhancement output feature tensor, perform channel recalibration and spatial fine localization based on statistics, and obtain the statistical enhancement output feature tensor.

[0161] In some embodiments, please refer to Figure 5 , Figure 5 This is a flowchart illustrating the steps for obtaining the statistical enhancement output feature tensor in the retinal neovascularization segmentation method based on a dual-path multi-scale attention network provided in this application embodiment. The method for obtaining the statistical enhancement output feature tensor in this application is specifically implemented through steps 410 to 470:

[0162] Step 410: Apply statistical pooling operator to the residual enhancement output feature tensor, and calculate the average value after nonlinear mapping through a two-layer perceptron to obtain the channel attention map.

[0163] In some embodiments, the representation formula for the channel attention map includes:

[0164] ;

[0165] in, Channel attention map; For Sigmoid activation operation; The set of statistical pooling operators used. for Cardinality; for The index variable in the set is used to represent each specific pooling operator in the set of statistical pooling operators; It is a two-layer perceptron; This refers to the channel reduction ratio; For statistical pooling operators, statistical pooling operators include global average pooling, global max pooling, and / or local window average pooling; To enhance the output feature tensor of the residual; Composed of real numbers, having Each channel, resolution is The tensor.

[0166] Understandably, in order to suppress redundant channels and highlight channel responses more relevant to angiogenesis detection, [the following is likely a separate, unrelated sentence:] ... Apply the statistical pooling operator (denoted as This includes global average pooling, global max pooling, and local window average pooling, and is achieved through two layers of perceptron (denoted as...). ,in After performing a nonlinear mapping (to the channel reduction ratio), the average is calculated and then activated by a Sigmoid function (denoted as ). ), to obtain the channel attention map .

[0167] Step 420: Obtain the channel recalibration feature tensor based on the residual enhancement output feature tensor and the channel attention map.

[0168] In some embodiments, the representation formula for the channel recalibration feature tensor includes:

[0169] ;

[0170] in, Recalibrate the feature tensor for the channel; This is element-wise multiplication.

[0171] Understandably, in order to suppress redundancy and highlight key channels based on channel attention, for and Perform element-wise multiplication (denoted as ") "(broadcast on demand in the channel dimension) to obtain the channel recalibration feature tensor" .

[0172] Step 430: Perform average pooling on the channel recalibrated feature tensor to obtain the channel mean map.

[0173] In some embodiments, the formula for representing the channel mean plot includes:

[0174] ;

[0175] in, This is a channel mean plot; This is for average pooling operations.

[0176] Step 440: Perform max pooling on the channel recalibrated feature tensor to obtain the channel maximum value map.

[0177] In some embodiments, the representation formula for the channel maximum value map includes:

[0178] ;

[0179] in, This is a chart showing the maximum values ​​of the channels; This is a max pooling operation.

[0180] Understandably, to describe the average and peak responses at each spatial location, respectively... Average pooling along the channel dimension With max pooling The channel mean plot is obtained. With channel maximum value chart .

[0181] Step 450: Obtain the spatial location mean and standard deviation map based on the channel recalibration feature tensor.

[0182] In some embodiments, the formula for representing the spatial location mean and standard deviation plot includes:

[0183] ;

[0184] ;

[0185] in, This is the average value of spatial location; for In the passage Spatial location The value at that location; For standard deviation plots; It is a tiny constant. This is used to ensure numerical stability.

[0186] Understandably, to characterize the intensity of local fluctuations that differ from those in healthy main blood vessels, the spatial mean is calculated. Plot with standard deviation .

[0187] Step 460: Concatenate the channel mean map, channel maximum map, and standard deviation map in the channel dimension, and obtain the spatial attention map after two-dimensional convolution processing.

[0188] In some embodiments, the representation formula for the spatial attention map includes:

[0189] ;

[0190] in, Spatial attention map; for Two-dimensional convolution is used to represent the mapping of three input channels to one output channel; For splicing operations; This is a channel mean plot; This is a chart showing the maximum values ​​of the channels; It consists of real numbers, has 1 channel, and has a resolution of [missing information]. The tensor.

[0191] Understandably, to locate structural anomalies at the pixel level, , and After the channel is spliced, Two-dimensional convolution (denoted as) (This indicates that three input channels are mapped to one output channel) and sigmoid activation to obtain a spatial attention map. .

[0192] Step 470: Obtain the statistically enhanced output feature tensor based on the residual enhanced output feature tensor, channel attention map, and spatial attention map.

[0193] In some embodiments, the representation formula for the statistically enhanced output feature tensor includes:

[0194] ;

[0195] in, To statistically enhance the output feature tensor.

[0196] Understandably, in order to comprehensively apply the channel-space recalibration to the features, for sequentially with , Perform element-wise multiplication to obtain the statistically enhanced output feature tensor. .

[0197] Understandably, in step 400, the encoder output... Together with the output features from other levels, they are input into the decoding path for cross-scale fusion and step-by-step reconstruction, forming the direct input for step 500.

[0198] Step 500: In the decoding path, fuse the statistically enhanced output feature tensor and reconstruct it step by step to obtain a pixel-level probability map, and train it with a hybrid loss to output the segmentation result.

[0199] In some embodiments, please refer to Figure 6 , Figure 6 This is a flowchart illustrating the steps for obtaining a pixel-level probability map in the retinal neovascularization segmentation method based on a dual-path multi-scale attention network provided in this application embodiment. The method for obtaining the pixel-level probability map in this application is specifically implemented through steps 510 to 540:

[0200] Step 510: Upsample the high-level features of the multi-level input feature tensor in the decoding path so that the scale of the high-level features is the same as that of the low-level features.

[0201] Step 520: Concatenate high-level features and statistically enhanced output feature tensors of the corresponding scales of the high-level features in the channel dimension, and fuse them through convolutional blocks to obtain fused features.

[0202] Understandably, to restore spatial resolution and integrate contextual and detailed information, high-level features are upsampled to the same scale as low-level features (using...). After that, with the corresponding s-scale splicing is performed in the channel dimension (using...) The convolutional blocks are then fused to reconstruct spatial details step by step, and the fused reconstructed features are denoted as follows: Its channel number is denoted as .

[0203] Step 530: Reconstruct the fusion features of each level step by step to obtain the reconstructed features.

[0204] Step 540: Map the reconstructed features to a single channel using two-dimensional convolution to obtain a pixel-level probability map.

[0205] In some embodiments, the representation formula for the pixel-level probability map includes:

[0206] ;

[0207] in, A pixel-level probability map. The elements in the matrix are used to represent the probability that the corresponding pixel belongs to a new blood vessel. During the inference phase, this can be used to... Thresholding is performed to obtain a binary segmentation mask; For Sigmoid activation operation; for Two-dimensional convolution is used to represent the number of channels that will reconstruct the features. Mapped to a single channel; Tensors for reconstructing features; The tensor is a two-dimensional tensor with H×W elements, consisting of real numbers ranging from [0,1]. [0,1] represents the numerical range, meaning that each element takes a value between 0 and 1 (inclusive). This is because after passing through the Sigmoid activation function σ, the output value is compressed to the [0,1] interval. H×W represents the spatial dimension of the tensor, where H is the height of the image and W is the width of the image. The complete representation is a two-dimensional tensor of size H×W. The value of each element in the tensor is between [0,1]. Each element corresponds to a pixel position in the image, and its value represents the probability that the pixel belongs to a new blood vessel.

[0208] In some embodiments, the total loss is obtained through Dice loss, Focal loss, and Tversky loss, and the formula for representing the total loss includes:

[0209] ;

[0210] in, Total loss; The Dice loss is used to optimize the overlap. The weights for the Dice loss; Focal loss is used to focus and segment samples; The weights for Focal loss; Tversky loss is used for asymmetric penalty; The weights for the Tversky loss;

[0211] The formulas for characterizing Focal loss include:

[0212] ;

[0213] in, The total number of pixels involved in the loss calculation; for The Middle The predicted probability of each pixel; For the segmentation mask, the first The truth label corresponding to each pixel. It can be 0 or 1; Let be the numerical stability constant of the Dice loss. ;

[0214] The formulas for characterizing Focal loss include:

[0215] ;

[0216] ;

[0217] in, To predict the probability that a sample is a neovascularization for the model; The category balancing weight function; As a focusing factor, ; To prevent logarithmic underflow of tiny constants, ;

[0218] The characterization formulas for Tversky loss include:

[0219] ;

[0220] in, To control the penalty weight for false positives, ; To control the penalty weight for false negatives, .

[0221] Understandably, in order to establish an adjustable tradeoff between sensitivity and specificity, a total loss consisting of a linear combination of Dice loss, Focal loss, and Tversky loss is used. As an optimization target.

[0222] Understandably, embodiments of this application provide a method for retinal neovascularization segmentation based on a dual-path multi-scale attention network, including determining a WF-OCTA image of retinal neovascularization, drawing a segmentation mask on the WF-OCTA image, and obtaining input data; extracting multi-scale features of the input data through an encoder-decoder and establishing skip connections to obtain multi-level input feature tensors; performing dual-path multi-scale feature mining on the input feature tensors, capturing morphological and contextual information in parallel, and obtaining residual-enhanced output feature tensors; performing statistical spatial-spectral attention enhancement on the residual-enhanced output feature tensors, performing channel recalibration and statistically-based spatial fine localization, and obtaining statistically enhanced output feature tensors; fusing the statistically enhanced output feature tensors in the decoding path and reconstructing them level by level to obtain pixel-level probability maps, and training with a hybrid loss to output segmentation results. This application employs an end-to-end approach combining dual-path multi-scale feature mining with statistical spatial-spectral attention to construct hierarchical features, enhance attention, and optimize hybrid loss in WF-OCTA images, thereby achieving automatic segmentation of neovascularization. This solves the technical problem in existing technologies where neovascularization cannot be accurately distinguished and segmented under conditions of complex artifacts and extremely sparse foregrounds.

[0223] In some embodiments, please refer to Figure 7 , Figure 7This is a module connection diagram of the retinal neovascularization segmentation system based on a dual-path multi-scale attention network provided in this application embodiment. The retinal neovascularization segmentation system based on a dual-path multi-scale attention network in this application includes:

[0224] Data acquisition module 10 is used to determine the WF-OCTA image of retinal neovascularization, draw a segmentation mask on the WF-OCTA image, and acquire input data;

[0225] Data conversion module 20 is used to extract multi-scale features of input data through encoder-decoder and establish skip connections to obtain multi-level input feature tensors;

[0226] Dual-path feature mining module 30 is used to perform dual-path multi-scale feature mining on the input feature tensor, capture morphological information and contextual information in parallel, and obtain residual-enhanced output feature tensor.

[0227] The statistical enhancement module 40 is used to perform statistical spatial-spectral attention enhancement on the residual enhancement output feature tensor, perform channel recalibration and spatial fine localization based on statistics, and obtain the statistical enhancement output feature tensor.

[0228] The data output module 50 is used to fuse statistically enhanced output feature tensors in the decoding path and reconstruct them step by step to obtain pixel-level probability maps, and train them with hybrid loss to output segmentation results.

[0229] This application has provided a detailed description of a method and system for retinal neovascularization segmentation based on a dual-path multi-scale attention network, as provided in the embodiments of this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and its core ideas. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for retinal neovascularization segmentation based on a dual-path multi-scale attention network, characterized in that, include: Determine the WF-OCTA image of retinal neovascularization, draw a segmentation mask on the WF-OCTA image, and obtain input data; The multi-scale features of the input data are extracted by an encoder-decoder and skip connections are established to obtain a multi-level input feature tensor. The input feature tensor is subjected to dual-path multi-scale feature mining, capturing morphological and contextual information in parallel, to obtain a residual-enhanced output feature tensor: The number of channels in the input feature tensor is mapped to the number of intermediate channels based on the channel compression function; Based on the number of intermediate channels, a two-dimensional convolution is applied to the input feature tensor to obtain the morphological branch output feature tensor. The representation formula of the morphological branch output tensor includes: ； in, Output a tensor for the morphological branch; for Two-dimensional convolution; The number of channels of the input feature tensor; This refers to the number of intermediate channels; For the first Level input feature tensor; It is a finite set of convolution kernel sizes; Dilated convolution is applied to the input feature tensor based on the number of intermediate channels to obtain the context branch output feature tensor, the representation formula of the context branch output feature tensor includes: ； in, Output a feature tensor for the context branch; The void ratio is of Two-dimensional dilated convolution; Let be a finite set of positive integers representing the void ratio; By concatenating the morphological branch output feature tensor and the context branch output feature tensor along the channel dimension, a multi-branch aggregated feature tensor is obtained. The representation formula of the multi-branch aggregated feature tensor includes: ； in, For the multi-branch aggregation feature tensor; For splicing operations; The multi-branch aggregated feature tensor is fused using two-dimensional convolution to obtain a channel-fused feature tensor. The representation formula of the channel-fused feature tensor includes: ； in, The channel fusion feature tensor; To change the number of channels from Mapped to of Two-dimensional convolution, The number of channels of the multi-branch aggregated feature tensor; The residual enhanced output feature tensor is obtained based on the channel fusion feature tensor and the input feature tensor. The representation formula of the residual enhanced output feature tensor includes: ； in, The residual enhancement output feature tensor; The residual enhanced output feature tensor is subjected to statistical spatial-spectral attention enhancement, channel recalibration and spatial fine localization based on statistics to obtain the statistically enhanced output feature tensor; The statistically enhanced output feature tensor is fused in the decoding path and reconstructed step by step to obtain a pixel-level probability map. The map is then trained with a hybrid loss and the segmentation result is output.

2. The retinal neovascularization segmentation method based on a dual-path multi-scale attention network according to claim 1, characterized in that, The representation formula for the WF-OCTA image is: ； in, This is the input tensor used to characterize the WF-OCTA image; Constructed from real numbers, possessing One channel, height is Width is tensor; The characterization formula for the segmentation mask includes: ； in, The segmentation mask; 255 represents a black pixel; 255 represents a white pixel.

3. The retinal neovascularization segmentation method based on a dual-path multi-scale attention network according to claim 2, characterized in that, The steps for obtaining input data include: Spatial consistency resampling is performed on the WF-OCTA image and the segmentation mask to unify their sizes; The intensity of the WF-OCTA image is scaled to a preset numerical range to obtain the input data, the representation formula of the input data including: ； in, The input data is used to characterize the WF-OCTA image after spatial consistency resampling and intensity scaling.

4. The retinal neovascularization segmentation method based on a dual-path multi-scale attention network according to claim 1, characterized in that, The steps of extracting multi-scale features from the input data through an encoder-decoder and establishing skip connections to obtain multi-level input feature tensors include: The input data is downsampled step by step by an encoder to obtain downsampled data. The downsampled data is upsampled step by step by the decoder, and the high-level features are scale-aligned with the low-level features. The high-level features and the corresponding scale-based encoding features are skip-connected in the channel dimension and fused through convolutional blocks to obtain the multi-level input feature tensor. The representation formula of the multi-level input feature tensor includes: ； ； in, It is a set of feature tensors containing S+1 levels, used to characterize the multi-level input feature tensors; The input feature tensor is the s-th level; The number of channels for the s-th level input feature tensor; It is the set of real numbers; and Let be the resolution of the s-th level input feature tensor. and It decreases as s increases, in order to carry hierarchical representations from fine-grained to high semantic levels.

5. The retinal neovascularization segmentation method based on a dual-path multi-scale attention network according to claim 1, characterized in that, The steps of performing statistical spatial-spectral attention enhancement on the residual enhanced output feature tensor, performing channel recalibration and spatial fine localization based on statistical quantities, and obtaining the statistically enhanced output feature tensor include: A statistical pooling operator is applied to the residual enhancement output feature tensor, and the average value is calculated after nonlinear mapping through a two-layer perceptron to obtain the channel attention map. The representation formula of the channel attention map includes: ； in, This is the channel attention map; For Sigmoid activation operation; The set of statistical pooling operators used. for Cardinality; for The index variable in the set is used to characterize each specific pooling operator in the set of statistical pooling operators; For the two-layer sensor; This refers to the channel reduction ratio; For statistical pooling operators; The residual enhancement output feature tensor; Constructed from real numbers, possessing Each channel, resolution is tensor; Based on the residual enhancement output feature tensor and the channel attention map, a channel recalibration feature tensor is obtained. The representation formula of the channel recalibration feature tensor includes: ； in, Recalibrate the feature tensor for the channel; This is element-wise multiplication; The channel recalibrated feature tensor is subjected to average pooling to obtain the channel mean map; Max pooling is performed on the channel recalibration feature tensor to obtain the channel maximum value map; Based on the channel recalibration feature tensor, a spatial location mean and standard deviation map are obtained. The representation formulas for the spatial location mean and the standard deviation map include: ； ； in, The average value of the spatial location; for In the passage Spatial location The value at that location; For the standard deviation plot; It is a tiny constant. This is used to ensure numerical stability; The channel mean map, the channel maximum map, and the standard deviation map are concatenated along the channel dimension, and then subjected to two-dimensional convolution to obtain a spatial attention map. The representation formula of the spatial attention map includes: ； in, This is the spatial attention map; for Two-dimensional convolution is used to represent the mapping of three input channels to one output channel; For splicing operations; This is the mean value plot of the channel; This is a graph showing the maximum value of the channel; It consists of real numbers, has 1 channel, and has a resolution of [missing information]. tensor; Based on the residual enhanced output feature tensor, the channel attention map, and the spatial attention map, a statistical enhanced output feature tensor is obtained. The representation formula of the statistical enhanced output feature tensor includes: ； in, The statistical enhancement output feature tensor is then used.

6. The retinal neovascularization segmentation method based on a dual-path multi-scale attention network according to claim 1, characterized in that, The step of fusing the statistically enhanced output feature tensor in the decoding path and reconstructing it step by step to obtain a pixel-level probability map includes: In the decoding path, the high-level features of the multi-level input feature tensor are upsampled so that the scale of the high-level features is the same as the scale of the low-level features. The high-level features and the statistically enhanced output feature tensors of the corresponding scales of the high-level features are concatenated in the channel dimension and then fused through convolutional blocks to obtain fused features; The fusion features at each level are reconstructed step by step to obtain the reconstructed features; The reconstructed features are mapped to a single channel using two-dimensional convolution to obtain a pixel-level probability map. The representation formula for the pixel-level probability map includes: ； in, This is the pixel-level probability map. The elements in the matrix are used to characterize the probability that the corresponding pixel belongs to a new blood vessel; For Sigmoid activation operation; for Two-dimensional convolution is used to represent the number of channels that reconstruct the features. Mapped to a single channel; The tensor of the reconstructed features; It is a two-dimensional tensor with H×W elements, consisting of real numbers in the range [0,1].

7. The retinal neovascularization segmentation method based on a dual-path multi-scale attention network according to claim 6, characterized in that, The steps for training with a hybrid loss and outputting segmentation results include: The total loss is obtained through Dice loss, Focal loss, and Tversky loss, and the formula for representing the total loss includes: ； in, Total loss; The Dice loss is used to optimize the overlap. The weights of the Dice loss; The Focal loss is used to focus on segmented samples; The weights of the Focal loss; The Tversky loss is used for asymmetric penalty; The weights of the Tversky loss; The formula for characterizing the Focal loss includes: ； in, The total number of pixels involved in the loss calculation; for The Middle The predicted probability of each pixel; For the segmentation mask, the first The truth label corresponding to each pixel. It can be 0 or 1; Let be the numerical stability constant of the Dice loss. ; The formula for characterizing the Focal loss includes: ； ； in, To predict the probability that a sample is a neovascularization for the model; The category balancing weight function; As a focusing factor, ; To prevent logarithmic underflow of tiny constants, ; The characterization formula for the Tversky loss includes: ； in, To control the penalty weight for false positives, ; To control the penalty weight for false negatives, .

8. The retinal neovascularization segmentation method based on a dual-path multi-scale attention network according to claim 5, characterized in that, The pooling operators included in the statistics are global average pooling, global max pooling, and / or local window average pooling.

Images