A macular microvessel image segmentation method based on multi-scale multi-feature fusion
Patent Information
- Application Number
- CN202611091112.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-07-22
- Publication Date
- 2026-11-03
Smart Images
Figure CN123032601A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of image processing technology, and in particular to a method for macular microvessel image segmentation based on multi-scale and multi-feature fusion. Background Technology
[0002] The macula is a crucial region of the retina responsible for central vision, and the integrity and functional state of its microvascular network directly affect visual health. Optical coherence tomography (OCTA), as a non-invasive imaging technique, enables three-dimensional visualization of the retinal microvascular system by detecting blood flow signals without the need for exogenous contrast agents. Compared to traditional fluorescein fundus angiography, OCTA offers significant advantages such as being non-invasive, rapid, and capable of layered imaging, and has been widely used in the clinical diagnosis and follow-up assessment of ischemic eye diseases such as diabetic retinopathy, age-related macular degeneration, and retinal vein occlusion. However, accurate segmentation of microvessels in OCTA images is the foundation and prerequisite for these clinical applications; its accuracy directly affects the reliability of subsequent quantitative analyses such as vascular density calculation, non-perfusion area identification, and disease severity grading.
[0003] Traditional image processing methods, such as matched filtering and multi-scale Hessian matrix analysis, rely on manually designed features, resulting in poor sensitivity and specificity for small capillaries and limited generalization ability. While deep learning-based U-shaped network architectures have made significant progress, they still suffer from the following shortcomings: encoders typically use standard square convolutional kernels, whose fixed sampling grids struggle to adapt to the highly tortuous and randomly oriented morphology of microvessels, leading to weak feature responses to tortuous capillaries and a tendency for fragmentation or missed detections. Multi-scale feature fusion mechanisms are still imperfect. The semantic gap between low-level detailed features and high-level semantic features is significant; simple addition or concatenation is insufficient for adequate fusion, and the upsampling process easily introduces jagged edges, affecting segmentation boundary accuracy. Decoders lack explicit constraints on the spatial distribution of blood vessels, leading to misjudgments in complex regions such as bifurcation and terminal segments, and the topological continuity of the segmentation mask is insufficient. Most existing methods only output pixel-level segmentation results, lacking the ability for further structured analysis and quantitative evaluation of the vascular network. Specifically, the current segmentation method fails to accurately identify non-perfusion areas based on the segmentation results, fails to determine the dormant state of vessels within non-perfusion areas by combining structural imaging modalities, and fails to predict the recanalization potential of dormant vessels. These deficiencies mean that the current segmentation method remains at the morphological description level, making it difficult to meet the clinical needs for assessing collateral circulation and predicting prognosis in ischemic retinal diseases. Summary of the Invention
[0004] This application provides a macular microvessel image segmentation method based on multi-scale and multi-feature fusion, which solves the problems in the prior art such as the difficulty of standard convolution kernels to adapt to tortuous vessel morphology, insufficient multi-scale fusion, lack of spatial morphological constraints in decoders, and lack of structured clinical quantitative assessment. It realizes intelligent analysis of the entire process, including high-precision vessel segmentation, accurate identification of non-perfusion areas, determination of dormant vessels and prediction of recanalization potential, and assessment of collateral circulation.
[0005] This application provides a method for macular microvessel image segmentation based on multi-scale and multi-feature fusion, including: S1: Input the preprocessed OCTA image into the strip convolution feature enhancement module. The strip convolution feature enhancement module includes square convolution, orthogonal direction strip convolution and dilated convolution branches. The three outputs are spliced together and then fused by convolution and connected by residuals to output multi-scale initial feature maps. S2: Input the multi-scale initial feature map into the encoder. The encoder contains four downsampling stages and outputs four feature maps at different scales. Input the feature maps of each layer into the spatial adaptive feature enhancement module. The spatial adaptive feature enhancement module extracts global context and local texture features in orthogonal directions through a three-way parallel structure. S3: Input the enhanced feature map into the multi-scale fusion attention enhancement module, perform channel and spatial attention calibration in sequence, and fuse the features of each layer from top to bottom to obtain multi-scale fusion features; S4: Input the multi-scale fusion features into the mask attention transformer decoder. The decoder consists of a progressive upsampling branch and a global mask attention branch. The outputs of the two branches are concatenated and then processed by convolution and normalized exponential function to generate a blood vessel probability map. The output is then divided into a binary segmentation mask using thresholding.
[0006] Furthermore, the preprocessing includes: dividing the image into several sub-blocks, filtering the signal region and background region based on the gray-scale mean, calculating the signal-to-noise ratio and removing unqualified images; performing median filtering and Gaussian filtering on qualified images in sequence for noise reduction; normalizing all image sizes to 512×512 pixels through bilinear interpolation; and normalizing the gray-scale values of all images to the [0,1] range through linear stretching.
[0007] Furthermore, each stage of the encoder includes two 3×3 convolutional blocks, with a deformable convolutional module embedded between the two blocks; the deformable convolutional module uses a 3×3 deformable convolutional kernel, and the sampling points are adaptively distributed along the blood vessel direction through a learnable spatial offset; each stage sets a feedforward network module after the two 3×3 convolutional blocks, the feedforward network module includes two serial fully connected layers, and is added to the input through residual connections; each stage ends with a 2×2 max pooling layer to halve the feature map spatial size.
[0008] Furthermore, the spatial adaptive feature enhancement module constructs a three-way parallel feature extraction and attention calibration structure: The first branch generates a spatial attention weight map through 1×1 convolution and Softmax normalization, aggregating global contextual information; the second and third branches use 1×5 horizontal strip convolution and 5×1 vertical strip convolution, respectively, to extract local texture features in orthogonal directions; spatial-channel collaborative attention weights are generated through adaptive average pooling, multilayer perceptron and sigmoid normalization, and the global spatial enhancement features, directional local texture features and original input features are dynamically weighted and fused.
[0009] Furthermore, the multi-scale fusion attention enhancement module sequentially performs channel attention calibration and spatial attention enhancement on each layer of features, and outputs preliminary enhanced features through residual connections; The initial enhanced features are input into a multilayer perceptron for nonlinear mapping and feature refinement, and then the enhanced features of the output layer are output through residual connections. The enhanced features at each level are fused in a top-down order, and then processed by bilinear interpolation upsampling, 1×1 convolution to adjust the channels, element-wise addition and 3×3 convolution smoothing to obtain multi-scale fused features.
[0010] Furthermore, the mask attention transformer decoder includes a decoding branch and a global mask attention branch; The decoding branch adopts a progressive upsampling structure, which includes four upsampling stages. Each stage doubles the feature map size through transposed convolution, and then concatenates the upsampled feature map with the enhanced feature map of the corresponding stage of the encoder in the channel dimension. Finally, it is refined by two 3×3 convolutional layers. The global mask attention branch generates a learnable spatial mask matrix, which is multiplied pixel-by-pixel with the multi-scale fusion features to obtain mask enhancement features; The outputs of the decoding branch and the global mask attention branch are concatenated along the channel dimension, then fused by convolution. A blood vessel probability map is generated through 1×1 convolution and the Softmax function, and a binarized segmentation mask is output by threshold segmentation.
[0011] Furthermore, the method also includes performing morphological refinement processing on the binarized segmentation mask to obtain a single-pixel-wide vascular centerline skeleton map; based on the skeleton map, according to the number and connection pattern of skeleton pixels in the eight-neighborhood of the pixel, the vascular endpoints, vascular bifurcation points and points inside ordinary vascular segments are marked, and a vascular topology map is constructed with vascular endpoints and bifurcation points as nodes and continuous skeleton line segments as edges.
[0012] Furthermore, the method also includes identifying non-perfusion regions based on a binarized segmentation mask: The binarized segmentation mask is divided into regular grids, and the local vascular density of each grid is calculated. Grids with a density lower than the first density threshold are marked as candidate grids without perfusion. The original OCTA blood flow signal intensity map is introduced, and candidate grids with an average signal intensity higher than the signal intensity threshold are removed. Connectivity analysis is performed on the remaining candidate grids, and regions with an area smaller than the minimum connected area threshold are removed. Morphological closing operations are performed on the remaining connected regions to obtain the binary mask of the non-perfusion region.
[0013] Furthermore, the method also includes: acquiring en-face OCT structural images synchronously acquired with OCTA images; for each vascular segment within a non-perfusion region, extracting grayscale profile lines along the normal direction of its skeleton line on the en-face OCT structural image, and calculating the peak signal-to-noise ratio: , in, Peak signal-to-noise ratio, and These represent the maximum and minimum grayscale values on the profile line, respectively. The standard deviation of the grayscale values on the profile line; The proportion of sampling points on a blood vessel segment that meet the condition of having a peak signal-to-noise ratio higher than a preset signal-to-noise ratio threshold is recorded as the vessel wall integrity ratio. If a blood vessel segment simultaneously meets the condition of having a vessel wall integrity ratio not lower than a preset integrity threshold and an average signal intensity on the OCTA blood flow signal intensity map not greater than a preset signal intensity threshold, it is marked as a dormant vessel, and its structural integrity score is calculated. , in, Score the structural integrity. This represents the total number of sampling points on the blood vessel segment.
[0014] Furthermore, the method also includes: taking the vascular topology graph as input, constructing feature vectors for each node and each edge, and inputting them into a graph neural network; the graph neural network adopts a structure of three stacked graph convolutional layers, and updates the hidden feature vectors of nodes and edges through message passing; For an edge marked as a dormant vessel, the hidden feature vectors of its two endpoints are concatenated with the hidden feature vector of the edge and then input into the classification head to output the recanalization probability of the dormant vessel. The recanalization probability is mapped to a color code and superimposed on the original OCTA image to generate a collateral circulation potential map, and the collateral circulation potential index is calculated. , in, As a potential index for collateral circulation, A collection of all dormant blood vessels. For dormant blood vessels The repass probability, The length of the blood vessel segment. The total area of all NP regions; Generate a visualization report that includes a segmentation mask, a contour map of the non-perfusion area, a map of collateral circulation potential, a list of recanalization probabilities of dormant vessels, and collateral circulation potential index values.
[0015] One or more technical solutions provided in this application have at least the following technical effects or advantages: 1. Identify the core defects of existing technologies and their causes. Existing deep learning-based methods for macular microvessel segmentation suffer from several drawbacks. The fixed sampling grid of standard convolutional kernels struggles to adapt to the highly tortuous morphology of blood vessels, resulting in weak responses and susceptibility to breakage in small vessels. Furthermore, a semantic gap exists between low-level details and high-level semantics during multi-scale feature fusion, making simple overlay or splicing insufficient for adequate integration. Upsampling also tends to produce jagged edges, affecting boundary accuracy. The decoder lacks explicit constraints on the spatial distribution of blood vessels, leading to misclassification in bifurcation and terminal regions, and insufficient topological continuity in segmentation. Moreover, existing methods often focus on pixel-level segmentation, lacking structured and quantitative assessments of vascular topology, non-perfusion zone identification, dormant vessel determination, and recanalization potential, thus failing to meet the clinical needs for assessing collateral circulation and predicting prognosis in ischemic eye diseases.
[0016] 2. Propose new technological approaches This application designs a three-way parallel strip convolutional enhancement module to extract multi-directional vascular features and long-range contextual information; embeds deformable convolution in the encoder to adaptively distribute sampling points along the vascular direction; constructs a global-local collaborative spatial adaptive enhancement module and a multi-scale fusion attention enhancement module to achieve full fusion of multi-level features and aliasing elimination; employs a decoder with a global mask attention branch to explicitly label vascular regions with a learnable mask matrix and guide segmentation recovery. A vascular topology map is constructed through morphological refinement, and non-perfusion areas are identified by combining blood flow signal verification and connected component analysis. Dormant vessels are determined using synchronously acquired structural images for vessel wall profile analysis, and recanalization probability is predicted using a graph neural network, generating a collateral circulation potential map and quantification index.
[0017] 3. Overcoming technical barriers to new pathways By employing directional design of strip convolution and end-to-end offset prediction of deformable convolution, the problem of insufficient capture of tortuous vessel features by standard convolution is addressed. Spatial-channel collaborative attention, top-down hierarchical fusion, and smoothing processes resolve the issues of difficulty in coordinating global semantics and local details, as well as the aliasing effect of upsampling. Explicit spatial prior guidance through masked attention branches addresses the problem of misjudgment in complex regions due to a lack of morphological constraints in the decoder. Multi-level screening using local density thresholding, signal strength verification, and connected region area constraints solves the problem of high false positives in the identification of non-perfusion areas. Peak signal-to-noise ratio analysis of vessel wall profiles using structural images, combined with blood flow signal intensity, solves the problem that relying solely on blood flow signals cannot distinguish between structurally intact but blood-deficient vessels and structurally collapsed, non-functional vessels. Graph neural networks, through message passing on the topology graph, enable individualized quantitative assessment of the recanalization potential of dormant vessels.
[0018] 4. Provide the optimal solution. This application employs strip convolution enhancement and deformable convolution to ensure accurate extraction of tortuous blood vessels, and utilizes multi-scale fusion attention and mask attention decoders to achieve a highly continuous and accurate blood vessel segmentation mask. Furthermore, through topology graph construction, multi-level screening for non-perfusion zone identification, structural image-assisted dormant vessel determination, and graph neural network-driven recanalization probability prediction, it generates a visualized report containing the segmentation mask, non-perfusion zone contours, collateral circulation potential maps, recanalization probability lists, and collateral circulation potential indices. This overcomes the limitations of traditional methods that only output pixel-level segmentation, achieving intelligent analysis across the entire process from high-precision segmentation to blood vessel topology modeling, pathological region identification, recanalization potential prediction, and collateral circulation assessment, effectively improving segmentation accuracy, topological continuity, and clinical decision support capabilities. Attached Figure Description
[0019] Figure 1 This is a flowchart of a macular microvessel image segmentation method based on multi-scale and multi-feature fusion in an embodiment of the present invention. Detailed Implementation
[0020] To facilitate understanding of the present invention, a more complete description of this application will be given below with reference to the accompanying drawings, which illustrate preferred embodiments of the invention. However, the invention can be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided to enable a more thorough and complete understanding of the disclosure of the present invention.
[0021] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains; the terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to limit the invention; the term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.
[0022] Example 1: As Figure 1 As shown, a method for segmenting macular microvessel images based on multi-scale and multi-feature fusion is presented.
[0023] S1: Input the preprocessed OCTA image into the strip convolution feature enhancement module. The strip convolution feature enhancement module includes square convolution, orthogonal direction strip convolution and dilated convolution branches. The three outputs are spliced together and then fused by convolution and connected by residuals to output multi-scale initial feature maps. Specifically, an OCTA device was used to scan the macula of the subject, with the scanning area centered on the fovea of the macula and measuring 6mm × 6mm. The device output a frontal blood flow image of the superficial capillary plexus (SCP) as the image to be segmented.
[0024] After image acquisition, a quality screening process is performed, dividing the image into several 32×32 pixel sub-blocks. The grayscale mean of each sub-block is calculated, and the top 10% of sub-blocks with the highest grayscale mean are selected as the signal region, while the bottom 10% of sub-blocks with the lowest grayscale mean are selected as the background region. The ratio of the grayscale mean of the signal region to the standard deviation of the grayscale of the background region is the signal-to-noise ratio (SNR). If the SNR is below 10dB, the image quality is deemed unacceptable, discarded, and a re-acquisition is requested. The image sharpness index is calculated by taking the gradient magnitude and averaging the values. If the average value is below a preset lower threshold, the image is also deemed unacceptable. The lower threshold is calibrated by statistically analyzing normal image quality indicators.
[0025] Denoising was performed on images of acceptable quality using a 3×3 window median filter. The median gray value of each pixel was taken as its new value to remove common salt-and-pepper noise and isolated bright spots in OCTA images. Gaussian filtering with a 5×5 kernel size and a standard deviation σ=0.8 was then applied to smooth the median-filtered image to suppress Gaussian noise. After denoising, all images were normalized, and bilinear interpolation was used to uniformly scale the images to a resolution of 512×512 pixels. Bilinear interpolation used a weighted average of the gray values of the four nearest neighbors of the target pixel, with the weights determined by distance. The normalized images were then gray-level normalized using linear stretching to map the gray values to the [0,1] interval. The mapping formula was: , in, Grayscale value The original grayscale value. This represents the minimum grayscale value of the entire image. This represents the maximum grayscale value of the entire image.
[0026] The preprocessed 512×512×1 grayscale image is input into the strip convolutional feature enhancement module (SCFE). The SCFE module contains three parallel dilated convolution branches, each specifically enhancing different morphological features of macular microvessels. The first branch uses 3×3 square convolutions for basic local feature modeling, with 32 kernels, a stride of 1, and same padding to maintain the spatial dimensions of the feature map. This branch extracts basic local spatial features, including grayscale contrast variations between vessels and the background, and vessel edge information, serving as a baseline reference for other branches. The second branch uses asymmetric strip convolution kernels. First, 1×1 convolutions are used for channel dimensionality reduction, compressing the number of channels to half the input number to reduce computation. Then, two sets of orthogonal strip convolutions are concatenated: first, a 1×5 horizontal strip convolution is used to extract horizontally oriented vessel structure information, and then a 5×1 vertical strip convolution is used to extract vertically oriented vessel structure information. The horizontal strip convolution kernel has an asymmetric structure extending horizontally, exhibiting stronger sensitivity to horizontally oriented strip-like structures in the image. Similarly, the vertical strip convolution kernel has an asymmetric structure extending vertically, exhibiting stronger sensitivity to vertically oriented strip-like structures in the image. By concatenating these two directions, this branch can capture complex vascular patterns with both horizontal and vertical components. This branch has 32 kernels, a stride of 1, and same padding. The third branch introduces a 3×3 dilated convolution with a dilation rate of 5 to expand the receptive field and enhance the perception of long-range vascular topology. Dilated convolution expands the receptive field by inserting holes between kernel elements, enabling each output pixel to perceive a larger input region without increasing the number of parameters or reducing the feature map spatial resolution. This branch is used to capture a longer range of vascular topological context information, helping the network understand the extension direction and connectivity of blood vessels at the receptive field level. This branch also has 32 kernels, a stride of 1, and same padding.
[0027] The output feature maps of all three branches have a spatial size of 512×512 and 32 channels. The outputs of the three branches are concatenated along the channel dimension to obtain a 512×512×96 tensor. A 1×1 convolution with 64 kernels and a stride of 1 is used to perform channel fusion and dimensionality reduction on the concatenated tensor. Different types of features extracted by the three branches are fused through pixel-wise linear combination of channels. The fused feature map is then added element-wise to the input feature map of the SCFE module via residual connections. This allows grayscale information from the original image to bypass the convolutional layers and be directly passed to subsequent networks, preserving low-level detail features and mitigating gradient vanishing. The SCFE module ultimately outputs a 512×512×64 multi-scale initial feature map.
[0028] S2: Input the multi-scale initial feature map into the encoder. The encoder contains four downsampling stages and outputs four feature maps at different scales. Input the feature maps of each layer into the spatial adaptive feature enhancement module. The spatial adaptive feature enhancement module extracts global context and local texture features in orthogonal directions through a three-way parallel structure. Each stage of the encoder contains two 3×3 convolutional blocks, with a deformable convolutional module embedded between them. The deformable convolutional module uses a 3×3 deformable convolutional kernel and adaptively distributes the sampling points along the blood vessel direction through a learnable spatial offset. After the two 3×3 convolutional blocks, each stage sets a feedforward network module, which contains two serial fully connected layers and is added to the input through residual connections. Finally, each stage uses a 2×2 max pooling layer to halve the spatial size of the feature map.
[0029] Specifically, the multi-scale initial feature map output by the SCFE module is input into the encoder. The encoder adopts a hierarchical structure, consisting of four downsampling stages connected in series. Each stage contains a convolutional block, a DCNv4 module, and a feedforward network module.
[0030] Each stage transforms the feature map using two 3×3 convolutional blocks without altering its spatial dimensions. Each convolutional block contains a 3×3 convolutional layer, a batch normalization layer, and a ReLU activation layer with a stride of 1 and same padding. A DCNv4 module is embedded between the two convolutional blocks. The DCNv4 module employs a 3×3 deformable convolutional kernel, introducing a learnable spatial offset for each sampling point on top of the fixed sampling grid of standard convolution. The offset is predicted from the current feature map through a separate lightweight convolutional layer, allowing sampling points to adaptively distribute along the tortuous direction of the blood vessel, rather than being confined to a regular square grid position. For highly tortuous capillaries in OCTA images, this module enables the network to aggregate sampling points along the vessel direction during sampling, thereby extracting the feature responses of vascular pixels more fully while reducing invalid sampling of non-vascular background areas. Offset prediction and convolution operations are performed end-to-end on the same feature map, requiring no additional supervision signals. Two 3×3 convolutional blocks are followed by a feedforward network module. This module contains two sequential fully connected layers, with ReLU activation in the middle layer and the output layer maintaining the same input dimension. It performs non-linear mapping and refinement of features through pixel-wise inter-channel linear transformations, further enhancing the non-linear expressive power of the features and enabling the network to learn more complex vascular morphological features. The output and input of the feedforward network module are also summed using residual connections. Each stage concludes with a 2×2 max-pooling layer (stride of 2) to halve the feature map spatial size. As the resolution decreases, the receptive field of the feature map gradually expands, and the number of channels increases progressively. The feature maps output from the four stages are denoted as F1, F2, F3, and F4, with spatial sizes of 1 / 2, 1 / 4, 1 / 8, and 1 / 16 of the input image, respectively, and channel numbers of 64, 128, 256, and 512. F1 and F2 are low-level features that preserve the edges, textures, and spatial details of blood vessels; F3 and F4 are high-level features that encode the overall distribution pattern of blood vessels, the location of the avascular area in the fovea of the macula, and the structural semantics of the background region.
[0031] S3: Input the enhanced feature map into the multi-scale fusion attention enhancement module, perform channel and spatial attention calibration in sequence, and fuse the features of each layer from top to bottom to obtain multi-scale fusion features; Specifically, the F1, F2, F3, and F4 outputs from each stage of the encoder are input into the Spatial Adaptive Feature Enhancement (SAFE) module and the Multi-Scale Fusion Attention Enhancement (MFAE) module, respectively, for feature enhancement. The SAFE module independently performs global context modeling and directional texture enhancement on each layer of features, and outputs the enhanced features. The enhanced multi-layer features are input into the multi-scale fusion attention enhancement module (MFAE). The MFAE fuses high-level semantic features and low-level detail features step by step through a dual-path input mode, and outputs the final multi-scale fused features.
[0032] The spatial adaptive feature enhancement module constructs a three-way parallel feature extraction and attention calibration structure: Specifically, the SAFE module uses the input feature map as the processing object and constructs a three-way parallel feature extraction and attention calibration structure. The first branch maps the input features to the channel dimension through 1×1 convolution, generates a spatial attention weight map after Softmax normalization, and aggregates global context information through matrix multiplication, enabling each pixel to perceive information from all other locations in the entire image. The second and third branches use orthogonal strip convolutions to extract local texture and edge information in the horizontal and vertical directions, respectively. The second branch uses a 1×5 horizontal strip convolution, and the third branch uses a 5×1 vertical strip convolution. The two branches jointly extract local texture features of the two main directions of microvessels in the macular region. The SAFE module compresses the output features of the three branches into channel-dimensional feature vectors through adaptive average pooling, and generates spatial-channel co-attention weights through multilayer perceptron and sigmoid normalization. These weights reflect the importance of features from different branches at different spatial locations and in different channel dimensions. The generated collaborative attention weights are multiplied pixel-wise and channel-wise with the global spatial enhancement features, directional local texture features, and original input features, respectively, to achieve dynamic weighted fusion of the three types of features. Finally, the weighted features are added element-wise to output the enhanced blood vessel feature map. .
[0033] The MFAE module employs a dual-input feature model, receiving both high-level and low-level features enhanced by the SAFE module as inputs. One input is a high-level semantic feature with global representation capabilities, while the other is a low-level detail feature that preserves edge and texture information. The MFAE module unifies the dimensionality of the two input features through learnable 1×1 convolutions, ensuring that the number of channels is consistent across both inputs. After dimensionality unification, the MFAE module sequentially performs the following steps on the input multi-level features F1, F2, F3, and F4, outputting the enhanced multi-level features. , , and .
[0034] With any feature map For example, the size is .right Perform global average pooling to reduce the spatial dimension Compression yields the length. The channel description vector; the vector is processed by two fully connected layers, the first layer will... Dimensions dropped to The second layer restores the dimension to its original value using the ReLU activation function for nonlinear transformation; The channel weighting coefficients in the [0,1] interval are then normalized using the Sigmoid function. ;Will and Channel-by-channel multiplication yields channel enhancement features. .right Dimensionality reduction is performed using 1×1 convolution, compressing the number of channels to 1, and generating a spatial response map with a spatial size of [missing value]. The spatial weight matrix is obtained by normalizing the spatial response map using the Sigmoid function. ;Will and Pixel-wise multiplication yields spatially enhanced features. Using residual connections to... With the original input Add to output enhanced features This will initially enhance the features. The input is processed by a multilayer perceptron for nonlinear mapping and feature refinement, and the refined features are then connected to the network via residual connections. The features are summed to output the enhanced features for that level. This process is repeated sequentially for F1 to F4 to obtain the enhanced multi-level features. , , and .
[0035] Will , , and Merge the layers sequentially from top to bottom. (The highest layer...) Upsampled to bilinear interpolation The size, due to and With different channel numbers, a 1×1 convolution is used to... The number of channels was adjusted to match Consistent. After channel adjustment. and Element-wise addition is performed to obtain a preliminary fusion result. This is then smoothed using a 3×3 convolution to eliminate the jagged edges and feature discontinuities introduced by upsampling, resulting in the third-level fusion feature M3. M3 is then upsampled using bilinear interpolation to... The spatial dimensions, after adjusting the number of channels using a 1×1 convolution, are compared with... Element-wise summation, followed by 3×3 convolution for smoothing, yields the second-level fused feature M2. M2 is then upsampled using bilinear interpolation. The spatial dimensions, after channel adjustment, are consistent with... The elements are added together and then smoothed by 3×3 convolution to obtain the multi-scale fusion feature M1.
[0036] S4: Input the multi-scale fusion features into the mask attention transformer decoder. The decoder consists of a progressive upsampling branch and a global mask attention branch. The outputs of the two branches are concatenated and then processed by convolution and normalized exponential function to generate a blood vessel probability map. The output is then divided into a binary segmentation mask using thresholding.
[0037] The mask attention transformer decoder includes a decoding branch and a global mask attention branch; Specifically, the multi-scale fused feature M1 output from the MFAE module is input into the mask attention Transformer decoder. The decoder consists of a decoding branch and a global mask attention branch. The two branches process M1 in parallel and then collaboratively output the segmentation result. The decoding branch adopts a progressive upsampling structure, containing four upsampling stages, each corresponding to one of the four downsampling stages of the encoder. Each upsampling stage uses a transposed convolution (4×4 kernel size, stride 2) to double the spatial size of the feature map. The upsampled feature map is then concatenated with the enhanced feature map of the corresponding stage of the encoder along the channel dimension to achieve skip connections. Two 3×3 convolutional layers refine the concatenated features, eliminating artifacts introduced by upsampling and fusing the high-resolution spatial details retained by the encoder. The four stages sequentially restore the feature map from 32×32 to 512×512.
[0038] A global mask attention branch applies a mask attention mechanism to the multi-scale fused feature M1. The mask attention mechanism first generates a learnable spatial mask matrix with the same spatial dimensions as M1, predicted from M1 through independent convolutional layers. The mask matrix is then multiplied pixel-by-pixel with M1 to obtain the mask-enhanced feature. This mask matrix is used to explicitly label candidate locations of blood vessel regions and exclusion locations of background regions, serving as auxiliary guidance information for the decoding branch.
[0039] The outputs of the decoding branch and the global mask attention branch are concatenated along the channel dimension and fused through a 3×3 convolutional layer, providing structural prior guidance from the global mask for pixel-level detail recovery. Finally, a 1×1 convolutional layer maps the fused feature channels to 2, outputting an unnormalized score for each pixel indicating whether it belongs to the background or a blood vessel. The scores are normalized to probability values using a Softmax function, resulting in a probability map of each pixel belonging to a blood vessel. A classification threshold of 0.5 is used: pixels with a probability value greater than or equal to 0.5 are classified as blood vessels and assigned a value of 1 in the segmentation mask; pixels with a probability value less than 0.5 are classified as background and assigned a value of 0 in the segmentation mask. The output is a binarized macular microvessel segmentation mask with the same size as the input image.
[0040] After obtaining the binarized macular microvessel segmentation mask, the vessel density (VD) of the superficial capillary plexus (SCP) and deep capillary plexus (DCP) is calculated based on this mask. The calculation region for vessel density is divided with the fovea as the center: the area within 1 mm in diameter of the fovea is defined as the foveal region, the annular region within 1 mm to 3 mm in diameter is defined as the parafoveal region, and the annular region within 3 mm to 6 mm in diameter is defined as the peripheral foveal region. The parafoveal and peripheral foveal regions are divided into four quadrants with equal areas: temporal, superior, nasal, and inferior. Within each region, the vessel density is defined as the proportion of the number of vessel pixels to the total number of pixels in that region. , in, Blood vessel density, To segment the number of pixels in the mask that belong to blood vessels, This represents the total number of pixels in the region. This allows for the output of SCP and DCP vascular density values in each region and quadrant, for subsequent quantitative analysis and clinical evaluation.
[0041] The technical solutions described in the embodiments of this application have at least the following technical effects or advantages: This application extracts basic local features, horizontal and vertical vascular structure features, and long-range topological context information through three parallel branches of the strip convolution feature enhancement module. After splicing, fusion, and residual connection, the original low-level details are effectively preserved. The encoder embeds a DCNv4 module, which uses learnable spatial offsets to adaptively distribute sampling points along the tortuous direction of the blood vessel, breaking through the limitations of the fixed sampling grid of standard convolution. The spatial adaptive feature enhancement module adopts a three-way parallel structure of global context aggregation, orthogonal direction strip convolution extraction, and spatial-channel collaborative attention weighting, realizing the dynamic fusion of global spatial information and local texture information. The multi-scale fusion attention enhancement module performs channel attention calibration, spatial attention enhancement, and residual refinement on each level of features in sequence, and then eliminates the jagged effect and feature discontinuity through top-down stepwise fusion and convolution smoothing. The mask attention decoder works in parallel with the global mask attention branch through pixel-level progressive upsampling branch, using a learnable spatial mask matrix to mark vascular candidate regions and guide the segmentation and recovery process, realizing high-precision localization and boundary refinement of vascular regions.
[0042] Example 2: Example 1 achieved high-precision segmentation of macular microvessel images, but it still has the shortcomings of only outputting the segmentation mask and lacking quantitative analysis and clinical evaluation of vascular network topology, non-perfusion area identification, dormant vessel determination, and their recanalization potential. This example further supplements and explains the content of Example 1.
[0043] The method further includes performing morphological refinement on the binarized segmentation mask to obtain a single-pixel-wide vascular centerline skeleton map; based on the skeleton map, according to the number and connection pattern of skeleton pixels in the eight-neighborhood of the pixel, the vascular endpoints, vascular bifurcation points and points inside ordinary vascular segments are marked, and a vascular topology map is constructed with vascular endpoints and bifurcation points as nodes and continuous skeleton line segments as edges.
[0044] Specifically, the binary segmentation mask undergoes morphological thinning to obtain a single-pixel-wide vessel centerline skeleton. The thinning process employs an iterative morphological algorithm, repeatedly peeling pixels from the vessel boundaries while strictly maintaining the topological connectivity of the vessels until all vessel segments are reduced to lines one pixel wide. During the thinning process, the algorithm ensures no breakpoints are generated and no spurious connections are introduced. After thinning, a binary skeleton image of the same size as the original image is obtained, where the vessel centerline pixel is assigned a value of 1, and the remaining pixels are assigned a value of 0.
[0045] Based on the skeleton graph, topology extraction is performed. Each pixel in the skeleton graph is traversed, and the distribution of skeleton pixels within its eight neighbors is analyzed. According to the number and connection pattern of skeleton pixels in the neighborhood: if a pixel has one skeleton pixel connected to it in its eight neighbors, the pixel is marked as a vessel endpoint, representing the tip of the vessel. If a pixel has two or more skeleton pixels connected to it in its eight neighbors, the pixel is marked as a vessel bifurcation point, representing the confluence of branches of the vessel. The remaining skeleton pixels are marked as points inside ordinary vessel segments, belonging to the middle positions of continuous vessel segments. Using all vessel endpoints and bifurcation points as graph nodes, and continuous skeleton line segments connecting two adjacent nodes as graph edges, an undirected graph structure is constructed. Starting from each graph node, pixel-by-pixel traversal is performed along the direction of the skeleton line until the next graph node is encountered; the sequence of skeleton pixels traversed constitutes an edge. Each edge corresponds to a complete vessel segment, and the segment length, average width, principal direction angle, and connected component number are recorded. The connected component number is the identifier of the connected component to which the vessel segment belongs, used to distinguish different vascular network subgraphs. If the length of an edge is less than a preset length threshold, such as 5 pixels, and one end of it is a blood vessel endpoint, then the edge is considered a spur caused by noise or segmentation error and is deleted. After the deletion operation, the node connected to the other end of the deleted edge is updated. If that node becomes a new endpoint as a result, the process is recursively repeated. This pruning step does not affect the main blood vessel topology. Finally, a complete blood vessel topology map is obtained. ,in, It is a set of nodes, containing all blood vessel endpoints and bifurcation points; Let be the set of edges, containing all pruned vessel segments.
[0046] The method also includes identification of non-perfusion regions based on a binarized segmentation mask: Specifically, non-perfusion regions are identified based on a binary segmentation mask and a vascular topology map. Non-perfusion regions are areas lacking OCTA blood flow signals and exhibiting significantly sparse or completely absent vascular structures, representing a key pathological feature of ischemic retinal diseases. The binary segmentation mask is divided into a regular grid of 32×32 pixels. Within each grid, the proportion of vascular pixels is calculated and denoted as the local vascular density. Set the first density threshold. If a certain grid If the grid is not properly defined, then the grid is marked as a non-inflation (NP) candidate grid. After traversing all grids, a binary candidate image with the same grid division as the original image is obtained, where the NP candidate grid is assigned a value of 1 and the remaining grids are assigned a value of 0.
[0047] To eliminate false positive candidate regions caused by image noise or segmentation errors, the original OCTA blood flow signal intensity map is introduced as an auxiliary verification basis. The original blood flow signal intensity map corresponding to the OCTA image is obtained, i.e., the original OCTA image without grayscale normalization. The average signal intensity of all pixels within each NP candidate grid is calculated. Set signal strength threshold. The signal intensity threshold is obtained by calibrating the statistical distribution of signal intensity in normal macular OCTA images, taking the lower limit of the mean signal intensity in the normal region minus three standard deviations. If a certain NP candidate grid... If the grid is not correctly identified as NP due to segmentation error, it is considered to be incorrectly classified as NP and is therefore discarded; otherwise, if... If the signal strength is negative, the grid is retained as a valid NP candidate grid. Finally, a set of NP candidate grids verified by signal strength is obtained.
[0048] Connectivity analysis is performed on the obtained NP candidate mesh set. The four-neighbor connectivity criterion is used to merge adjacent NP candidate meshes into connected NP regions. A minimum connectivity area threshold is set. A square pixel corresponds to approximately 0.14 square millimeters of actual retinal area, calculated based on image resolution and scanning range. If the area of a connected region is smaller than... If a region is found to be isolated due to noise, it is removed. For the remaining connected regions, morphological closing operations are performed, using a circular kernel with a radius of 3 pixels as the structuring element. The closing operation first dilates and then erodes to fill any small holes that may exist within the NP region, such as small dots mistakenly segmented as blood vessels, and to smooth the boundaries of the NP region to better conform to realistic physiological contours. After the closing operation, a final binary mask of the NP region is obtained, where pixels inside the NP region are assigned a value of 1, and pixels in other regions are assigned a value of 0.
[0049] For the binary mask of the NP region, the Canny edge detection algorithm is used to extract the boundary contour of each NP region. The dual threshold of the Canny algorithm is calibrated by statistically analyzing the gradient magnitude distribution of NP region boundary pixels in a large number of normal OCTA images. The 10th percentile is used as the low threshold (0.1), and the 90th percentile as the high threshold (0.3), to detect true boundaries and suppress false edges. The extracted boundary contours are stored in the form of closed polygon chains, recording the vertex coordinate sequence of each contour. Simultaneously, the NP region area, NP region perimeter, NP region equivalent diameter, and NP region center coordinates are recorded for each NP region. The equivalent diameter of the NP region is the diameter of a circle with the same area as the region. , The area is the region.
[0050] The method further includes: acquiring en-face OCT structural images synchronously acquired with OCTA images; for each vascular segment within a non-perfusion region, extracting grayscale profile lines along the normal direction of its skeleton line on the en-face OCT structural image, and calculating the peak signal-to-noise ratio; Specifically, en-face OCT structural images from the same subject and at the same scanning time point as the OCTA images were acquired. These images were acquired synchronously using the retinal structural imaging function built into the OCT device, and their scanning range and spatial resolution were consistent with the OCTA images, both being a 6mm × 6mm area centered on the fovea of the macula and a resolution of 512 × 512 pixels.
[0051] For each NP region, traverse all vascular segments within it and perform the following operations: Along the skeletal line direction of the vascular segment, take several sampling points at equal intervals (e.g., every 2 pixels). At each sampling point, along the normal direction of the blood vessel at that point, extract a grayscale profile line of length 21 pixels on the en-face OCT structural image, taking 10 pixels on each side of the sampling point as the center. For each grayscale profile line, calculate its peak signal-to-noise ratio (PSNR). Use the difference between the maximum and minimum grayscale values on the profile line as the signal amplitude, and the standard deviation of the grayscale values on the profile line as the noise amplitude. The PSNR is then: , in, Peak signal-to-noise ratio, and These represent the maximum and minimum grayscale values on the profile line, respectively. This represents the standard deviation of the grayscale values along the profile line.
[0052] If at a certain sampling point Higher than the preset signal-to-noise ratio threshold If a clear vessel wall structure is found at a sampling point, it is considered that a clear vessel wall structure exists at that sampling point. The signal-to-noise ratio (SNR) threshold is obtained by calibrating the statistical distribution of the grayscale profile of the vessel wall in a normal vessel segment, taking the 5th percentile of the peak SNR of the normal vessel segment. The threshold is calculated based on the statistical distribution of the grayscale profile of the vessel wall in all sampling points on that vessel segment. The proportion of sampling points is denoted as the pipe wall integrity proportion. Calculate the average signal intensity of this vessel segment on the original OCTA blood flow signal intensity map. .
[0053] If a certain blood vessel segment simultaneously meets the vessel wall integrity ratio And the average signal intensity of the vessel segment on the OCTA blood flow signal intensity map If a segment is marked as a dormant vessel, its structural integrity score is calculated. , in, The structural integrity score is the arithmetic mean of the peak signal-to-noise ratio at all sampling points along the vessel segment. A higher value indicates a clearer and more complete blood vessel wall structure. This represents the total number of sampling points on the blood vessel segment.
[0054] Invert the binary mask for the NP region to obtain the mask for the non-NP region. Vascular networks within the non-NP region are considered functional vascular networks with normal blood perfusion. In the vascular topology graph, all nodes and edges located within the non-NP region are labeled as normal vascular network nodes and edges. For each dormant vessel, the graph edge is... Get its two endpoints and Judge separately and Whether an endpoint belongs to a normal vascular network node (i.e., located outside the NP region) is determined. If an endpoint is itself a normal network node, the distance from that endpoint to the normal network is 0. If an endpoint is inside the NP region, a breadth-first search (BFS) is performed on the vascular topology graph starting from that endpoint to find the nearest normal vascular network node. The BFS expands layer by layer along the graph edges, expanding one hop at a time, until the first normal network node is encountered. The shortest path length from that endpoint to the nearest normal network node is recorded, denoted as . and Dormant blood vessels The topological connectivity score is: , in, Score the topological connectivity. This is the smaller of the shortest path lengths from each endpoint to the normal network. If one end of the dormant blood vessel is directly connected to the normal network, i.e. ,but This indicates that the dormant vessel has the highest topological connectivity. If neither end of the dormant vessel is connected to the normal network, that is... and If all are infinite, then =0 indicates that the dormant blood vessel is completely isolated. The value ranges from [0,1]. A higher value indicates a tighter connection between the dormant vessel and the functional blood supply network, and a better anatomical basis for recanalization through collateral circulation. The topological connectivity score for each dormant vessel is calculated. Record the attributes of this blood vessel segment.
[0055] The method further includes: taking a blood vessel topology graph as input, constructing a feature vector for each node and each edge, and inputting it into a graph neural network; the graph neural network adopts a structure of three stacked graph convolutional layers, and updates the hidden feature vectors of nodes and edges through message passing; Specifically, using vascular topology maps The graph structure serves as the input to the graph neural network. For each node... and each edge Construct feature vectors. Node features include node degree, vessel density at the node's location, whether the node is an NP region boundary point, and the average grayscale value of the en-face OCT structure image at the node's location. Node degree is represented using one-hot encoding with a dimension of 3, corresponding to vessel endpoints, vessel bifurcation points, and ordinary nodes, respectively. Edge features include edge length, average width, direction angle, whether the edge is located within an NP region, whether the edge is marked as a dormant vessel, structural integrity score, and topological connectivity score. The identifiers for whether an edge is located within an NP region and whether it is marked as a dormant vessel are Boolean values: 1 for yes and 0 for no.
[0056] The graph neural network employs a stacked structure of three graph convolutional neural network (GCN) layers. Each GCN layer has a hidden layer dimension of 64, meaning that the feature vectors of each node and edge are mapped to a 64-dimensional hidden feature vector after passing through the GCN layer. The message passing mechanism of the GCN layer is as follows: each node aggregates the features of its neighboring nodes and the features of the connecting edges, performs a linear transformation using a learnable weight matrix, and then performs a non-linear mapping using the ReLU activation function. The three GCN layers sequentially pass and update the hidden feature vectors of nodes and edges, enabling each node to aggregate topological information within a maximum of three hops. After the last GCN layer, for each edge marked as a dormant vessel, the hidden feature vectors of its two endpoints are concatenated with the hidden feature vector of the edge itself, and input into a classification head consisting of two fully connected layers. The first layer of the classification head maps the concatenated vector to 32 dimensions and activates it using ReLU; the second layer maps the 32-dimensional vector to 1 dimension, normalizes it using the Sigmoid function, and outputs the recanalization probability p∈[0,1] of the dormant vessel.
[0057] During training, a set of OCTA data with known treatment outcomes was used as the training set. Based on OCTA images retrieved 3 to 6 months post-treatment, the true recanalization status of each dormant vessel was determined: if the retrieved image showed new blood flow signals in the vessel segment, it was marked as a positive sample with a label of 1; if the retrieved image still showed no blood flow signals, it was marked as a negative sample with a label of 0. The training set consisted of at least 200 patients, and the ratio of positive to negative samples was adjusted to 1:1 through undersampling or oversampling. The binary cross-entropy loss function was used, and the Adam optimizer was employed. The initial learning rate was set to 0.001, and the number of training epochs was 200. After each training epoch, the AUC (area under the curve) was calculated on the validation set, and the model parameter with the highest AUC on the validation set was saved as the final model.
[0058] For any new OCTA image, the constructed vascular topology map and its features are input into a trained graph neural network model. The model outputs the recanalization probability p for each dormant vessel. The closer the probability value is to 1, the greater the likelihood that the dormant vessel will achieve blood flow recanalization under future treatment intervention.
[0059] The recanalization probability p of each dormant vessel is mapped to a color code and overlaid on the original OCTA image. Dormant vessels with p ≥ 0.7 are shown in green, indicating high recanalization potential; dormant vessels with p ≤ 0.4 < 0.7 are shown in yellow, indicating moderate recanalization potential; and dormant vessels with p < 0.4 are shown in red, indicating low recanalization potential. Non-dormant vessels are shown in gray semi-transparent, and the boundaries of NP regions are drawn with white dashed lines. This process ultimately generates a collateral circulation potential map.
[0060] The collateral circulation potential index of the entire macular region is calculated to assess the collateral circulation reserve capacity of the entire macular region: , in, As a potential index for collateral circulation, A collection of all dormant blood vessels. For dormant blood vessels The repass probability, This is the length of the blood vessel segment. The total area of all NP regions. A higher value indicates that the eye has a better collateral circulation reserve. When the NP region area is 0, the CPI is 0.
[0061] The final result is a visualization report that includes a binarized macular microvessel segmentation mask, an NP region contour map, a collateral circulation potential map, a list of recanalization probabilities for each dormant vessel, and a collateral circulation potential index (CPI) value.
[0062] There is a certain correlation between changes in macular microvascular structure and the imaging severity of cerebral small vessel disease (CSVD). This correlation has been preliminarily verified in several OCTA-based clinical studies. Common clinical types of CSVD include symptomatic lacunar infarction and white matter hyperintensity. To demonstrate the potential application value of the segmentation method of this invention in the derivation of clinical research data, this application example uses the high-precision macular microvascular segmentation mask obtained in Examples 1 and 2 as a basis to exemplarily construct a framework for extracting and analyzing quantitative parameters of microvascular changes related to CSVD. It should be noted that this application example is only a demonstration scenario for the downstream data use of the segmentation method. Its output results are exploratory quantitative references and should not replace gold standard examinations such as cranial MRI / CT, nor do they constitute a diagnostic or predictive claim for any specific disease (including symptomatic lacunar infarction).
[0063] Based on a segmentation mask, the vessel density of the superficial capillary plexus (SCP) and deep capillary plexus (DCP) is calculated separately. The calculation is performed in three quadrants: the fovea centralis (within 1 mm in diameter), the parafovea centralis (1–3 mm ring area), and the perifovea centralis (3–6 mm ring area). The parafovea centralis and perifovea centralis are further calculated in four quadrants: temporal, superior, nasal, and inferior. The overall vessel density of the SCP is a core parameter reflecting the overall microvascular perfusion status, and the reliability of its calculation depends on the complete capture of fine capillaries by the segmentation mask. The precision advantage of this invention in segmenting tortuous and peripheral vessels provides a fundamental guarantee for the stable extraction of such parameters.
[0064] Based on a segmentation mask, the boundary of the foveal avascular zone (FAZ) is automatically identified, and its area, perimeter, and non-circularity index are calculated. The expansion and irregularity of the FAZ are important indirect signs of microvascular damage, and accurate delineation of its boundary depends on the segmentation mask's precise differentiation of microvessels from the background in the FAZ edge region. The multi-scale fusion attention enhancement module and mask attention decoder of this invention provide more refined multi-level feature data for accurate FAZ boundary localization, effectively reducing FAZ boundary offset caused by missegmentation of blood vessels.
[0065] Based on the aforementioned segmentation-derived parameters, combined with the subjects' routine clinical indicators (such as age, systolic blood pressure, diastolic blood pressure, fasting blood glucose, total cholesterol, and past vascular disease history), a multidimensional quantitative feature set for CSVD studies can be formed. This set objectively records comprehensive information on the subjects' ocular microvascular status and systemic vascular risk factors, and can be used for baseline quantitative description in retrospective data analysis or prospective cohort studies.
[0066] On retrospective study datasets with complete CSVD imaging annotations (referencing cranial MRI), an exploratory CSVD risk propensity scoring model can be fitted based on a multidimensional quantitative feature set, using conventional statistical modeling methods (such as binary logistic regression) or machine learning models (such as random forests and gradient boosting trees). The model output score (ranging from 0 to 1) serves only as an observational reference indicator, reflecting the statistical correlation trend of subjects under the current ocular microvascular quantitative feature data. A high or low score does not represent an accurate judgment of an individual's disease status, nor does it constitute a basis for clinical decision-making.
[0067] The technical solutions described in the embodiments of this application have at least the following technical effects or advantages: This application employs morphological refinement and topology map construction to achieve precise extraction and structured characterization of the vascular network skeleton, providing an accurate topological basis for subsequent pathological analysis. Through a multi-level screening strategy involving local vessel density threshold screening, original blood flow signal intensity verification, connected region area constraints, and morphological closing operations, high-precision identification and contour extraction of non-perfusion areas are achieved, effectively eliminating false positive regions caused by noise and segmentation errors. Using synchronously acquired en-face OCT structural images, signal-to-noise ratio analysis of vessel wall profiles within the NP region is performed, and combined with blood flow signal intensity, objective determination of dormant vessels is achieved. Quantitative assessment of structural integrity: Based on vascular topology maps and graph neural networks, through multidimensional feature encoding of nodes and edges and message passing of three-layer graph convolution, accurate prediction of the recanalization probability of dormant vessels is achieved. Furthermore, a collateral circulation potential map and collateral circulation potential index are generated, providing quantitative imaging indicators for the assessment of collateral circulation in ischemic retinal diseases. The final generated visualization report integrates segmentation mask, NP region contour, recanalization probability distribution, and CPI value, assisting clinicians in intuitively assessing the pathological status and recanalization prognostic potential of microvessels in the macular region, realizing intelligent analysis of the entire process from image segmentation to clinical decision support.
[0068] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. For those skilled in the art, the present invention can have various modifications and variations. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.
Claims
1. A method for segmenting macular microvessel images based on multi-scale and multi-feature fusion, characterized in that, include: S1: Input the preprocessed OCTA image into the strip convolution feature enhancement module. The strip convolution feature enhancement module includes square convolution, orthogonal direction strip convolution and dilated convolution branches. The three outputs are spliced together and then fused by convolution and connected by residuals to output multi-scale initial feature maps. S2: Input the multi-scale initial feature map into the encoder. The encoder contains four downsampling stages and outputs four feature maps at different scales. Input the feature maps of each layer into the spatial adaptive feature enhancement module. The spatial adaptive feature enhancement module extracts global context and local texture features in orthogonal directions through a three-way parallel structure. S3: Input the enhanced feature map into the multi-scale fusion attention enhancement module, perform channel and spatial attention calibration in sequence, and fuse the features of each layer from top to bottom to obtain multi-scale fused features; S4: Input the multi-scale fusion features into the mask attention transformer decoder. The decoder consists of a progressive upsampling branch and a global mask attention branch. The outputs of the two branches are concatenated and then processed by convolution and normalized exponential function to generate a blood vessel probability map. The output is then divided into a binary segmentation mask using thresholding.
2. The method for segmenting macular microvessel images based on multi-scale and multi-feature fusion according to claim 1, characterized in that, The preprocessing includes: dividing the image into several sub-blocks, filtering signal regions and background regions based on grayscale mean, calculating the signal-to-noise ratio and removing unqualified images; performing median filtering and Gaussian filtering on qualified images in sequence for noise reduction; normalizing all image sizes to 512×512 pixels through bilinear interpolation; and normalizing the grayscale values of all images to the [0,1] range through linear stretching.
3. The method for segmenting macular microvessel images based on multi-scale and multi-feature fusion according to claim 1, characterized in that, Each stage of the encoder contains two 3×3 convolutional blocks, with a deformable convolutional module embedded between them. The deformable convolutional module uses a 3×3 deformable convolutional kernel and adaptively distributes the sampling points along the blood vessel direction through a learnable spatial offset. After the two 3×3 convolutional blocks, each stage sets a feedforward network module, which contains two serial fully connected layers and is added to the input through residual connections. Finally, each stage uses a 2×2 max pooling layer to halve the spatial size of the feature map.
4. The method for segmenting macular microvessel images based on multi-scale and multi-feature fusion according to claim 1, characterized in that, The spatial adaptive feature enhancement module constructs a three-way parallel feature extraction and attention calibration structure: The first branch generates a spatial attention weight map through 1×1 convolution and Softmax normalization, aggregating global contextual information; the second and third branches use 1×5 horizontal strip convolution and 5×1 vertical strip convolution, respectively, to extract local texture features in orthogonal directions; spatial-channel collaborative attention weights are generated through adaptive average pooling, multilayer perceptron and sigmoid normalization, and the global spatial enhancement features, directional local texture features and original input features are dynamically weighted and fused.
5. The method for segmenting macular microvessel images based on multi-scale and multi-feature fusion according to claim 1, characterized in that, The multi-scale fusion attention enhancement module sequentially performs channel attention calibration and spatial attention enhancement on each layer of features, and outputs preliminary enhanced features through residual connections; The initial enhanced features are input into a multilayer perceptron for nonlinear mapping and feature refinement, and then the enhanced features of the output layer are output through residual connections. The enhanced features at each level are fused in a top-down order, and then processed by bilinear interpolation upsampling, 1×1 convolution to adjust the channels, element-wise addition and 3×3 convolution smoothing to obtain multi-scale fused features.
6. The method for segmenting macular microvessel images based on multi-scale and multi-feature fusion according to claim 1, characterized in that, The mask attention transformer decoder includes a decoding branch and a global mask attention branch; The decoding branch adopts a progressive upsampling structure, which includes four upsampling stages. Each stage doubles the feature map size through transposed convolution, and then concatenates the upsampled feature map with the enhanced feature map of the corresponding stage of the encoder in the channel dimension. Finally, it is refined by two 3×3 convolutional layers. The global mask attention branch generates a learnable spatial mask matrix, which is multiplied pixel-by-pixel with the multi-scale fusion features to obtain mask enhancement features; The outputs of the decoding branch and the global mask attention branch are concatenated along the channel dimension, then fused by convolution. A blood vessel probability map is generated through 1×1 convolution and the Softmax function, and a binarized segmentation mask is output by threshold segmentation.
7. The method for segmenting macular microvessel images based on multi-scale and multi-feature fusion according to claim 1, characterized in that, The method further includes performing morphological refinement on the binarized segmentation mask to obtain a single-pixel-wide vascular centerline skeleton map; based on the skeleton map, according to the number and connection pattern of skeleton pixels in the eight-neighborhood of the pixel, the vascular endpoints, vascular bifurcation points and points inside ordinary vascular segments are marked, and a vascular topology map is constructed with vascular endpoints and bifurcation points as nodes and continuous skeleton line segments as edges.
8. The method for segmenting macular microvessel images based on multi-scale and multi-feature fusion according to claim 7, characterized in that, The method also includes identification of non-perfusion regions based on a binarized segmentation mask: The binarized segmentation mask is divided into regular grids, and the local vascular density of each grid is calculated. Grids with a density lower than the first density threshold are marked as candidate grids without perfusion. The original OCTA blood flow signal intensity map is introduced, and candidate grids with an average signal intensity higher than the signal intensity threshold are removed. Connectivity analysis is performed on the remaining candidate grids, and regions with an area smaller than the minimum connected area threshold are removed. Morphological closing operations are performed on the remaining connected regions to obtain the binary mask of the non-perfusion region.
9. The method for segmenting macular microvessel images based on multi-scale and multi-feature fusion according to claim 8, characterized in that, The method further includes: acquiring en-face OCT structural images synchronously acquired with OCTA images; for each vascular segment within a non-perfusion region, extracting grayscale profile lines along the normal direction of its skeleton line on the en-face OCT structural image, and calculating the peak signal-to-noise ratio: ， in, Peak signal-to-noise ratio, and These represent the maximum and minimum grayscale values on the profile line, respectively. The standard deviation of the grayscale values on the profile line; The proportion of sampling points on a blood vessel segment that meet the condition of having a peak signal-to-noise ratio higher than a preset signal-to-noise ratio threshold is recorded as the vessel wall integrity ratio. If a blood vessel segment simultaneously meets the condition of having a vessel wall integrity ratio not lower than a preset integrity threshold and an average signal intensity on the OCTA blood flow signal intensity map not greater than a preset signal intensity threshold, it is marked as a dormant vessel, and its structural integrity score is calculated. ， in, Score the structural integrity. This represents the total number of sampling points on the blood vessel segment.
10. The method for segmenting macular microvessel images based on multi-scale and multi-feature fusion according to claim 9, characterized in that, The method further includes: taking a blood vessel topology graph as input, constructing a feature vector for each node and each edge, and inputting it into a graph neural network; the graph neural network adopts a structure of three stacked graph convolutional layers, and updates the hidden feature vectors of nodes and edges through message passing; For an edge marked as a dormant vessel, the hidden feature vectors of its two endpoints are concatenated with the hidden feature vector of the edge and then input into the classification head to output the recanalization probability of the dormant vessel. The recanalization probability is mapped to a color code and superimposed on the original OCTA image to generate a collateral circulation potential map, and the collateral circulation potential index is calculated. ， in, As a potential index for collateral circulation, A collection of all dormant blood vessels. For dormant blood vessels The repass probability, The length of the blood vessel segment. The total area of all NP regions; Generate a visualization report that includes a segmentation mask, a contour map of the non-perfusion area, a map of collateral circulation potential, a list of recanalization probabilities of dormant vessels, and collateral circulation potential index values.