Retinal vessel segmentation method based on direction field guidance and graph neural network optimization

Abstract

This invention discloses a retinal vessel segmentation method based on orientation field guidance and graph neural network optimization, belonging to the field of image analysis and medical image processing technology. The invention first acquires retinal fundus images and their corresponding vessel annotation maps, and preprocesses the raw retinal fundus images. Then, it constructs a retinal vessel segmentation model based on orientation field guidance and graph neural network optimization. This model includes a backbone segmentation model based on a UNet++ network architecture and a post-processing refinement module based on a visual graph neural network. The preprocessed image is input into the backbone segmentation model to obtain a preliminary vessel segmentation probability map, which is then processed by the post-processing refinement module to obtain the final vessel segmentation result. Experiments show that this invention effectively addresses the shortcomings of existing retinal vessel segmentation methods in terms of small vessel extraction, structural connectivity preservation, and background noise suppression.

Description

Technical Field

[0001] This invention belongs to the field of image analysis and medical image processing technology, specifically relating to a retinal vessel segmentation method based on orientation field guidance and graph neural network optimization. Background Technology

[0002] Retinal vessel segmentation refers to the process of automatically extracting the vascular network structure from fundus images, which can provide crucial information for the auxiliary diagnosis, assessment, and treatment of ophthalmic diseases. This task is of great significance in practical applications, but the segmentation process is quite challenging due to factors such as low background and foreground contrast in fundus images, the long and complex morphology of blood vessels, and uneven illumination.

[0003] Early retinal vessel segmentation relied heavily on manual delineation or traditional image processing methods. Traditional image processing methods achieved segmentation through rule-driven strategies such as thresholding and region growing. With the development of machine learning methods, researchers began to transform the problem of vessel pixel classification into a feature learning and classification task, constructing features such as texture, orientation, and gradient and using a classifier to complete the segmentation. In recent years, the introduction of deep learning has significantly improved the performance of retinal vessel segmentation, especially the encoder-decoder structure represented by U-Net. UNet++ achieves good semantic segmentation results of vessels by fusing multi-scale feature extraction and skip connections. Based on this, further research has proposed various improved networks, such as AttU-Net with an attention mechanism, R2U-Net with a recurrent / residual structure, and ResUNet, and these models have formed commonly used benchmarks on public datasets.

[0004] Meanwhile, considering the characteristics of blood vessels being long, thin, and with variable orientations, some studies have attempted to incorporate directional information and graph structure modeling. For example, some studies have proposed a method based on optimal directional gradient flux filtering, using filters to obtain blood vessel response maps and assist in segmentation; other studies have used graph convolutional networks to locally refine high-uncertainty regions in coarse segmentation results, thereby improving the connectivity and integrity of the segmentation results. In addition, dynamically deformable convolution, due to its adaptive ability to irregular blood vessel morphologies, has been gradually applied to blood vessel segmentation tasks, enhancing the detection accuracy of curved and intersecting blood vessels by adjusting the convolution sampling position.

[0005] While the aforementioned traditional image processing methods have achieved vessel segmentation to some extent, they still have certain limitations. Traditional image processing methods often rely excessively on manually designed rules, which are not only significantly affected by subjective factors and time-consuming, but also lack robustness in the face of complex backgrounds, vessels of different scales, noise, and uneven lighting, especially in adapting to small vessels and low-contrast regions. Although machine learning-based methods transform the segmentation problem into a classification task, they still heavily rely on feature engineering and lack generalization ability when facing different devices, different populations, or lesion types, and have limited ability to express weak contrast and small structures. Although deep learning-based network models have achieved significant improvements in segmentation accuracy, existing models may still exhibit phenomena such as missegmentation of main vessel edges, breakage of small branches, missed segmentation, and false detection of background in complex backgrounds and small branch regions. Specifically, although some studies have used directional gradient flux filters or orientation field information, most existing methods treat orientation information as a preprocessing tool or auxiliary feature. The failure to fully integrate the orientation field into the internal feature learning and multi-scale fusion process of the encoder-decoder architecture results in insufficient ability of the model to aggregate slender structural features along the main direction of blood vessels. Existing graph neural network research is mostly limited to local refinement of the region of interest in the coarse segmentation results, failing to achieve global vascular topology modeling and optimization across the entire graph, and has limited ability to maintain vascular connectivity and recover small branches. Existing applications of dynamic deformable convolution often replace the original convolution with a single branch, lacking collaborative design of standard convolution, deformable convolution, and orientation-aware convolution, and failing to fully leverage the complementary advantages of different convolution mechanisms in basic feature extraction, irregular shape adaptation, and slender structure aggregation. Existing methods usually use simple concatenation or addition operations when fusing features at multiple scales, lacking an adaptive weighted fusion mechanism based on orientation information, making it difficult to effectively fuse shallow detail features and deep semantic features.

[0006] In summary, existing retinal vessel segmentation methods still have significant shortcomings in terms of small vessel extraction, structural connectivity preservation, and background noise suppression, especially in the lack of orientation priors and adaptive selection mechanisms during cross-scale fusion and topology restoration. Therefore, this invention provides a retinal vessel segmentation method based on orientation field guidance and graph neural network optimization, aiming to improve the accuracy and connectivity of vessel segmentation, reduce missed detections and breaks in small vessels, and optimize computational efficiency and model deployment feasibility. Summary of the Invention

[0007] The purpose of this invention is to provide a retinal vessel segmentation method based on orientation field guidance and graph neural network optimization, which can effectively solve the problems of missed segmentation, breakage, blurred boundaries, and incomplete recovery of the topological structure of complex bifurcation regions that are common in existing retinal vessel segmentation methods during the extraction of small vessels. This improves the ability to identify small vessels, the continuity of the vascular network, and the generalization ability across datasets.

[0008] To achieve the above objectives, the technical solution adopted by this invention is: a retinal vessel segmentation method based on orientation field guidance and graph neural network optimization, comprising the following steps: S1. Obtain retinal fundus images and their corresponding vascular annotation maps, and preprocess the raw retinal fundus images; S2. Construct a retinal vessel segmentation model based on orientation field guidance and graph neural network optimization. The retinal vessel segmentation model includes a backbone segmentation model based on the UNet++ network architecture and a post-processing refinement module based on a visual graph neural network. The trunk segmentation model includes an encoder, a decoder, and nested dense skip connections. The shallow layer of the encoder is equipped with a dynamic strip convolution and adaptive morphology perception module to extract multi-scale features and generate a direction field representing the local orientation information of blood vessels. The decoder is equipped with a direction field-guided multi-scale fusion module to receive the direction field, as well as the shallow high-resolution features from the corresponding layer of the encoder and the upsampled deep semantic features, and perform feature fusion. The preprocessed retinal fundus image is input into the trunk segmentation model to obtain a preliminary blood vessel segmentation probability map; the preliminary blood vessel segmentation probability map is input into the post-processing refinement module, and global context information and blood vessel topology relationship are extracted through graph convolution operation to output the refined final blood vessel segmentation result; S3. A two-stage training method is used to train the trunk segmentation model and the post-processing refinement module in step S2. In the first stage, the trunk segmentation model is trained with the blood vessel annotation map as the supervision signal. In the second stage, the parameters of the trunk segmentation model are fixed, and the preliminary blood vessel segmentation probability map output by the model is used as the input. The post-processing refinement module is trained with the blood vessel annotation map as the supervision signal. S4. Use the trained retinal vessel segmentation model to segment the vessels in the retinal fundus image to obtain the final segmentation result.

[0009] Furthermore, the preprocessing in step S1 specifically involves: firstly, standardizing the original retinal fundus image to obtain a training image; then, performing online random data augmentation on the training image to obtain an enhanced retinal fundus image; finally, generating fixed-size image blocks from the enhanced retinal fundus image using a random cropping method, and combining this with a directional sampling strategy to ensure that the cropped image blocks contain a blood vessel region of no less than a predetermined proportion.

[0010] Furthermore, the dynamic strip convolution and adaptive morphology perception module includes three parallel feature extraction branches: a standard convolution branch, a deformable convolution branch, and a dynamic strip convolution branch. The standard convolution branch uses a 3×3 convolutional layer to extract basic local contextual features. The deformable convolution branch learns the spatial offset field of the input features and adjusts the convolution sampling position to enhance its adaptability to blood vessel cross-sections and irregular morphological regions. The dynamic strip convolution branch predicts the local blood vessel direction based on the input features and constructs a strip-shaped sampling structure along the main direction of the blood vessel, enabling the convolution operation to perform feature aggregation along the extension direction of the slender blood vessel.

[0011] Furthermore, the dynamic strip convolution and adaptive morphology perception module also includes a local-macro attention fusion module for obtaining the enhanced encoder output features, the specific steps of which are as follows: The input for constructing the attention mechanism is as follows: the original input features of the module are expanded by a magnification factor λ to obtain an expanded feature map, which is then used as the query vector in the attention mechanism. The output features of the three branches—standard convolution, deformable convolution, and dynamic strip convolution—are concatenated along the channel dimension, and the concatenated features are used as key vectors. Sum value vector ; Calculate the query vector using the attention formula. With key vector The correlation between them is used to generate attention weights, and the expression of the attention formula is: In the formula, The dimension of the key vector. Represents the key vector Transpose of; The weighted summation output is reconstructed into a two-dimensional feature map to obtain the fused enhanced features, i.e., the enhanced encoder output features.

[0012] Furthermore, the decoder of the backbone segmentation model introduces a direction field-guided multi-scale fusion module. The input of the direction field-guided multi-scale fusion module comes from the shallow high-resolution features of the corresponding layer of the encoder, the upsampled deep semantic features, and the direction field information output by the dynamic strip convolution and adaptive morphology perception module and passed to the current layer.

[0013] Furthermore, the orientation field-guided multi-scale fusion module participates in the fusion of horizontal and vertical strip features through an orientation-aware gating mechanism. Specifically, semantic features and orientation fields are concatenated along the channel dimension and input into the gating network. The original weights of the two channels at each spatial location are output. After applying a Softmax operation along the channel dimension to the original weights of the two channels, normalized gating weights corresponding to the horizontal and vertical directions are obtained, respectively. The gating weights are then used to adaptively weight and fuse the output features of the horizontal and vertical strip convolutional branches to obtain orientation-weighted strip features.

[0014] Furthermore, the post-processing refinement module based on the visual graph neural network is specifically used for: dividing the preliminary blood vessel segmentation probability map into blocks through a block embedding layer, dividing the two-dimensional probability map into multiple non-overlapping image blocks, and mapping each image block to a corresponding feature vector to form node features in the graph structure; dynamically searching for the nearest neighbor node for each node in the node feature space, and constructing a k-nearest neighbor graph based on node feature similarity; inputting the constructed graph structure into multiple sequentially stacked visual graph neural network blocks for feature inference and updating, each visual graph neural network block including a graph construction and processing module and a feedforward network; reshaping the updated node features into a two-dimensional feature map, and restoring it to the same spatial resolution as the original retinal fundus image through upsampling, and generating the refined final blood vessel segmentation result through a convolutional output layer.

[0015] Furthermore, during the training process of the backbone segmentation model, a composite loss function combining pixel-level segmentation constraints and structural connectivity constraints is adopted. The composite loss function includes a main loss term, a Dice loss term, and a connectivity constraint term.

[0016] Furthermore, in step S4, when performing vascular segmentation on the retinal fundus image to be tested, the entire retinal fundus image is divided into blocks for prediction using a sliding window method, and the prediction results of each window are stitched together to obtain a complete retinal vascular segmentation result.

[0017] The beneficial effects of the above technical solution are as follows: This invention introduces a dynamic strip convolution and an adaptive morphology perception module into the shallow layer of the encoder, achieving adaptive adjustment of the convolutional receptive field along the local orientation of blood vessels and explicit extraction of vascular orientation field information. Specifically, this module sets up three feature extraction branches in parallel: standard convolution, deformable convolution, and dynamic strip convolution. The dynamic strip convolution branch predicts the local vascular orientation based on input features and constructs a strip-shaped sampling structure along the main vascular direction, enabling the convolution operation to perform feature aggregation along the extension direction of slender blood vessels. The deformable convolution branch adjusts the convolution sampling position by learning the spatial offset field, enhancing its adaptability to vascular cross-sections and irregular morphological regions. The standard convolution branch is responsible for extracting basic local contextual features. The outputs of the three branches are adaptively weighted and fused by a local-macro attention fusion module, thereby enhancing the representation ability of small, tortuous, and oriented vascular structures.

[0018] This invention achieves adaptive fusion of shallow detail features and deep semantic features driven by orientation prior by setting up a multi-scale fusion module guided by orientation field in the decoding stage. Specifically, the module's input comes from shallow high-resolution features of the corresponding layer of the encoder, upsampled deep semantic features, and orientation field information output by the dynamic strip convolution and adaptive morphology perception module and passed to the current layer. Through an orientation-aware gating mechanism, the semantic features and orientation field are concatenated along the channel dimension and input into the gating network. The output is the original weights of the two channels corresponding to the horizontal and vertical directions for each spatial position. After Softmax normalization, the gating weights are obtained. These weights are used to adaptively weight and fuse the output features of the horizontal strip convolution branch and the vertical strip convolution branch to obtain orientation-weighted strip features. This achieves accurate feature fusion guided by orientation prior, effectively improving the ability to restore blood vessel boundaries and the continuity of small blood vessels.

[0019] 3. This invention introduces a post-processing refinement module based on visual graph neural networks to achieve global topological relationship modeling and structural integrity optimization of the initial segmentation probability map. Specifically, this module performs block mapping on the initial blood vessel segmentation probability map through a block embedding layer, dividing the two-dimensional probability map into multiple non-overlapping image blocks, and mapping each image block to a corresponding feature vector to form node features in the graph structure; dynamically searches for the nearest neighbor node for each node in the node feature space, and constructs a k-nearest neighbor graph based on node feature similarity; inputs the constructed graph structure into multiple sequentially stacked visual graph neural network blocks for feature inference and updating, each visual graph neural network block including a graph construction and processing module and a feedforward network; reshapes the updated node features into a two-dimensional feature map, and restores it to the same spatial resolution as the original retinal fundus image through upsampling, and generates the refined final blood vessel segmentation result through a convolutional output layer, thereby effectively repairing small blood vessel fractures, improving the structural integrity of intersecting and bifurcerating regions, and reducing isolated noise points. Attached Figure Description

[0020] Figure 1 This is a flowchart of the method of the present invention; Figure 2 This is a schematic diagram of the overall model structure of the present invention; Figure 3 This is a schematic diagram of the dynamic strip convolution and adaptive shape perception module. Figure 4 A schematic diagram of a multi-scale fusion module structure guided by the directional field; Figure 5 This is a schematic diagram of the post-processing refinement module structure based on a visual graph neural network. Figure 6 This is a comparison chart of the segmentation results of the method of this invention and the benchmark model. Detailed Implementation

[0021] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0022] It should be noted that, unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains.

[0023] To address the shortcomings of existing retinal vessel segmentation methods in terms of small vessel extraction, structural connectivity preservation, and background noise suppression, this invention designs a retinal vessel segmentation method based on orientation field guidance and graph neural network optimization, such as... Figure 1 As shown, the steps include: S1. Obtain the retinal fundus image and its corresponding vascular annotation map, and preprocess the raw retinal fundus image.

[0024] Raw retinal fundus images and their corresponding vascular annotation maps are used as basic training samples to train and validate the retinal vessel segmentation model. This embodiment uses publicly available retinal fundus image datasets DRIVE, STARE, and CHASEDB1 as experimental data sources.

[0025] First, the original retinal fundus images are standardized to ensure that the pixel value distribution meets preset mean and variance requirements, resulting in training images. This reduces the impact of brightness and contrast differences between different images on the training of the retinal vessel segmentation model. Next, online random data augmentation is performed on the training images to obtain enhanced retinal fundus images, increasing the diversity of training samples and enhancing the model's generalization ability. The data augmentation methods include random horizontal flipping, random vertical flipping, random rotation at multiples of 90 degrees, random brightness and contrast perturbation, and random Gaussian blurring. Finally, the enhanced retinal fundus images are randomly cropped to generate fixed-size image patches. Combined with a directional sampling strategy, this ensures that the cropped image patches contain at least a predetermined proportion of vascular regions, alleviating the imbalance between foreground and background vascular pixel counts in the retinal vessel segmentation task.

[0026] To balance local detail feature extraction and computational efficiency, this embodiment sets the size of the cropped image blocks to a uniform 512×512 pixels, thereby ensuring consistent network input scale and improving the retinal vessel segmentation model's ability to learn small vessel regions.

[0027] S2. Construct a retinal vessel segmentation model based on orientation field guidance and graph neural network optimization.

[0028] The retinal vessel segmentation model based on orientation field guidance and graph neural network optimization uses the UNet++ network with an encoder-decoder structure as its basic architecture, and integrates three core functional modules on this basis: Dynamic Strip Convolution and Adaptive Morphology Perception (DSCAMP) module, Direction Field-guided Multi-scale Fusion (DFMF) module, and Vision Graph Neural Network (ViG) post-processing refinement module. Its overall structure is as follows: Figure 2 As shown.

[0029] The backbone segmentation model based on the UNet++ network includes an encoder, a decoder, and nested dense skip connections. The DSCAMP module, located within the encoder, extracts enhanced features and generates a direction field. The DFMF module, located within the decoder, uses the direction field to weightedly fuse features from each level to obtain a preliminary vessel segmentation probability map. This preliminary probability map is then input into a post-processing refinement module based on a visual graph neural network. Graph convolution operations are used to extract global contextual information and vessel topological relationships, outputting the refined final vessel segmentation result.

[0030] (1) Dynamic strip convolution and adaptive shape perception module To improve the encoder's ability to perceive small, curved, and directionally changing retinal vascular structures, a DSCAMP module is introduced in the shallow feature extraction stage of the backbone network encoder. Its structure is as follows: Figure 3 As shown, the DSCAMP module is located at multiple shallow convolutional positions in front of the encoder, which is used to enhance the directional sensitivity and morphological adaptation to strip-shaped vascular structures while preserving local contextual information.

[0031] The DSCAMP module includes three feature extraction branches set in parallel: a standard convolution branch, a deformable convolution branch, and a dynamic strip convolution branch.

[0032] The standard convolution branch uses a 3×3 convolutional layer to extract basic local contextual features; the deformable convolution branch learns the spatial offset field of the input features and adjusts the convolution sampling position to enhance its adaptability to blood vessel cross sections and irregular shaped regions; the dynamic strip convolution branch is used to predict the local blood vessel direction based on the input features and construct a strip-shaped sampling structure along the main direction of the blood vessel, so that the convolution operation can perform feature aggregation along the extension direction of the slender blood vessel.

[0033] By modeling in parallel through the above three branches, vascular texture information, structural information, and directional information can be extracted from different angles.

[0034] Furthermore, the dynamic strip convolution branch estimates the local orientation angle corresponding to each spatial location in the input feature map through a lightweight orientation prediction sub-network, and generates a strip-shaped orientation sampling structure based on the local orientation angle, so that the convolution receptive field can better fit the spatial distribution morphology of small blood vessels, tortuous blood vessels, and bifurcated blood vessels.

[0035] Compared with traditional fixed square convolution kernels, this method can improve the orientation perception of slender structures and reduce feature response bias caused by the mismatch between the receptive field and the actual blood vessel orientation.

[0036] The features extracted by the three parallel branches are further input into a local-macro attention fusion module for adaptive weighted fusion to obtain the enhanced encoder output features. The specific implementation process of the local-macro attention fusion module is as follows: First, the input to the attention mechanism is constructed. The original input features of the module are expanded by a magnification factor λ to obtain an expanded feature map, which is then used as the query vector in the attention mechanism. In this embodiment, λ is set to 3. Simultaneously, the output features of the standard convolution, deformable convolution, and dynamic strip convolution branches are concatenated along the channel dimension, and the concatenated features are used as key vectors. Sum value vector Query vector Derived from input features that extend the receptive field, it carries an understanding of the macroscopic morphological context surrounding the current location; while the key vector... Sum value vector It contains local detailed features from three branches.

[0037] Next, attention weights are calculated and features are fused. The query vector is calculated using the attention formula. With key vector The correlation between them is analyzed, and attention weights are generated. The expression of the attention formula is: In the formula, The dimension of the key vector. Represents the key vector The transpose of .

[0038] This weight dynamically reflects the contribution of different branch features to the current geometry within the expanded receptive field. Subsequently, this weight is used to adjust the value vector. Weighted summation is performed to achieve adaptive feature fusion, wherein the different branch features include at least local texture, morphological adaptation and orientation information.

[0039] Finally, the weighted summation output is reconstructed into a two-dimensional feature map, yielding the fused enhanced features, i.e., the enhanced encoder output features. This attention mechanism, through extended context awareness, enables the network to assign appropriate convolutional branch weights to local structures of different morphologies, resulting in enhanced features that take into account both local vascular details and overall morphology, thereby significantly improving the retinal vessel segmentation model's ability to represent and perceive heterogeneous vascular morphologies.

[0040] The key to this embodiment lies in the fact that the dynamic strip convolution branch generates a direction field representing the local orientation information of blood vessels during the calculation process, and this direction field is explicitly retained as part of the module output. This direction field does not participate in the feature fusion calculation of the local-macro attention fusion module, but is directly passed to the corresponding layer of the decoder as geometric prior information for subsequent orientation-guided multi-scale fusion. In this way, effective transmission of blood vessel orientation information from the encoding stage to the decoding stage can be achieved, providing clear structural guidance for the subsequent feature recovery process.

[0041] (2) Multi-scale fusion module guided by directional field To fully utilize the orientation field information explicitly extracted by the DSCAMP module during the encoder stage and improve the fusion effect of shallow detail features and deep semantic features during skip connections, an orientation field-guided multi-scale fusion module is set up in the decoder feature recovery stage, with the structure as follows: Figure 4 As shown.

[0042] The input to the DFMF module consists of three parts: shallow high-resolution features from the corresponding layer of the encoder, deep semantic features after upsampling, and orientation field information output by the DSCAMP module and passed to the current layer. The module first decomposes the input tensor, where the last two channels are the orientation field and the remaining channels are semantic features; subsequently, the semantic features undergo multi-branch parallel processing.

[0043] In one specific implementation, the multi-branch structure includes at least the following four branches: One is a 1×1 convolution branch, used to implement channel alignment and linear mapping; The second is a 3×3 convolutional branch, used to extract regular local context information; The third is the horizontal strip convolution branch, which consists of a 1×5 convolutional layer, a batch normalization layer, and a ReLU activation function, used to enhance the horizontal strip structure features; The fourth is the vertical strip convolution branch, which consists of a 5×1 convolutional layer, a batch normalization layer, and a ReLU activation function, used to enhance the vertical strip structure features.

[0044] The multi-branch structure described above can simultaneously handle both general local feature modeling and direction-related strip feature extraction.

[0045] Furthermore, the orientation field participates in the fusion of horizontal and vertical stripe features through an orientation-aware gating mechanism. Specifically, semantic features and the orientation field are concatenated along the channel dimension and input into a gating network consisting of two convolutional layers. The gating network first maps the concatenated features to an intermediate feature space through a 1×1 convolutional layer, then processes them through a batch normalization layer and a ReLU activation function, and finally outputs the original weights of the two channels for each spatial location through another 1×1 convolutional layer. After applying a Softmax operation along the channel dimension to the original weights of the two channels, normalized gating weights corresponding to the horizontal and vertical directions are obtained, respectively. and .

[0046] Subsequently, the gating weights are used to output features from the horizontal strip convolution branches. Output features of vertical strip convolution branches Adaptive weighted fusion is performed to obtain directional weighted strip features. In one specific embodiment, the fusion method can be expressed as follows: in, This indicates element-wise multiplication.

[0047] This approach enhances the directional feature response consistent with the local blood vessel orientation and suppresses interfering features from irrelevant or orthogonal directions, thereby improving the expression ability for small and tortuous blood vessels.

[0048] Finally, the strip features after direction-weighted fusion are... The output features of the 1×1 convolutional branch and the output features of the 3×3 convolutional branch are concatenated in the channel dimension, and the channel fusion and dimensionality reduction are completed through the 1×1 convolutional layer. The optimized fused features are then output to the next stage of the decoder.

[0049] In subsequent layers of the decoder, progressive upsampling and convolutional recovery are performed, and a preliminary blood vessel segmentation probability map corresponding to the original retinal fundus image is generated through the output layer. The preliminary blood vessel segmentation probability map represents the probability distribution of each pixel belonging to the blood vessel region, providing input for the subsequent topology optimization of the post-processing refinement module based on visual graph neural network.

[0050] (3) Post-processing refinement module based on visual graph neural network To further improve the output of the backbone segmentation network, issues such as broken small blood vessels, incomplete connections in bifurcation regions, and uneven boundaries are addressed by adding a post-processing refinement module based on a visual graph neural network after the backbone segmentation model. Its structure is as follows: Figure 5 As shown, the post-processing refinement module based on the visual graph neural network is used to perform topology optimization and refinement on the preliminary blood vessel segmentation probability map output by the trunk segmentation model, so as to improve the connectivity and structural integrity of the final segmentation result.

[0051] The post-processing refinement module first performs block mapping on the input preliminary segmentation probability map through a block embedding layer. Specifically, the input two-dimensional probability map is divided into multiple non-overlapping image blocks, and each image block is mapped to a corresponding feature vector through a convolutional layer to form the node features in the subsequent graph structure.

[0052] In one specific implementation, the block embedding layer uses a convolutional layer with a kernel size and stride of 16 to achieve image block segmentation and feature projection, and combines a batch normalization layer and an activation function to normalize and nonlinearly transform the projected node features.

[0053] In this way, the initial segmentation results can be converted from a regular pixel grid representation to a node representation suitable for graph structure modeling.

[0054] After obtaining node features, the post-processing refinement module dynamically searches for the nearest neighbor nodes for each node in the node feature space and constructs a k-nearest neighbor graph based on node feature similarity. The value of k can be set to a fixed value according to the network level or gradually increased with the network depth to expand the receptive field and enhance the ability to model long-distance dependencies.

[0055] The graph construction method described above allows for the connection of structurally related but geographically distant vascular segments in the pixel space, providing a foundation for subsequent topology repair.

[0056] The constructed graph structure is then fed into multiple stacked visual graph neural network blocks for feature inference and updates. Each visual graph neural network block consists of a graph construction and processing module and a feedforward network.

[0057] The graph construction and processing module performs message passing between nodes on the graph structure and aggregates neighborhood node features through graph convolution operations to model local structural relationships and global topological relationships. The feedforward network performs node-by-node nonlinear transformations on the updated node features to enhance feature representation capabilities. In one specific implementation, the graph construction and processing module uses a maximum relative graph convolution strategy to aggregate neighborhood features, thereby highlighting key structural differences and suppressing noise responses.

[0058] By cascading multiple visual graph neural network blocks, more consistent structural associations can be established for broken blood vessel segments in the feature space, and the interference of isolated noise points on the segmentation results can be further reduced.

[0059] After updating the features on the graph structure, the node features are reshaped into a two-dimensional feature map, and upsampling is used to restore the spatial resolution to be consistent with the original retinal fundus image. Subsequently, a refined retinal vessel segmentation probability map is generated through a convolutional output layer, which serves as the final vessel segmentation result.

[0060] Compared with the initial segmentation results of the trunk segmentation model, the output results optimized by the post-processing refinement module based on visual graph neural network are improved in terms of small blood vessel connectivity, cross-branch region integrity and boundary smoothness, thereby improving the structural integrity and continuity of the final segmentation results.

[0061] S3. A two-stage training method is used to train the backbone segmentation model and the post-processing refinement module based on the visual graph neural network to achieve local pixel-level segmentation learning and global topology optimization learning.

[0062] The first stage uses the original retinal fundus image as the input image and the blood vessel annotation map as the supervision signal to train the backbone segmentation model. The second stage fixes the parameters of the backbone segmentation model and uses the blood vessel annotation map as the supervision signal to train the post-processing refinement module based on the visual graph neural network.

[0063] During the backbone segmentation model training phase, the input image patch size was set to 512×512 pixels, the batch size to 4, the initial learning rate to 2e-4, and the optimizer to AdamW. The number of training epochs for the backbone segmentation model was set to 100. During training, the network parameters were updated using the backpropagation algorithm, and the learning rate was gradually adjusted using a learning rate annealing strategy to improve training stability and convergence performance. The above process was implemented in code. Additionally, an early stopping mechanism was employed during the backbone segmentation model training to prevent overfitting on the validation set.

[0064] Regarding the loss function setting, this embodiment employs a composite loss function that combines pixel-level segmentation constraints and structural connectivity constraints. Specifically, the main loss term is calculated based on the classification error between the vascular region and the background region. A Dice loss term is introduced to enhance the optimization of the overlap of the foreground vascular region, and a connectivity constraint term is further added to improve the performance of the segmentation result in terms of structural continuity. This composite loss design simultaneously considers pixel accuracy, region overlap, and vascular network connectivity.

[0065] In the training phase of the post-processing refinement module based on a visual graph neural network, the preliminary segmentation probability map output by the backbone segmentation model is input into the post-processing refinement module, and the blood vessel annotation map is used as a supervision signal to train the parameters of the post-processing refinement module. After training, the post-processing refinement module can further optimize the blood vessel network topology based on the preliminary segmentation results, reduce the phenomenon of small blood vessel breakage, and improve local boundary artifacts.

[0066] S4. Use the trained model to segment blood vessels in the retinal fundus image to be tested and output the final segmentation result.

[0067] Specifically, during the validation or testing phase, no random data augmentation is performed on the original retinal fundus images; only standardization processing, consistent with the training phase, is applied. Subsequently, the standardized retinal fundus image is segmented into blocks using a sliding window approach for prediction, and the prediction results from each window are stitched together and input into the trained model to obtain the final retinal vessel segmentation result. This result can be used for retinal vessel morphology analysis, lesion-assisted screening, and related quantitative medical image analysis tasks.

[0068] By using the above methods, the model's adaptability to high-resolution retinal fundus images can be improved while ensuring consistent input scale.

[0069] To visually demonstrate the effectiveness and superiority of this invention, experiments were conducted on three publicly available retinal vessel datasets: DRIVE, STARE, and CHASEDB1. Dice coefficient, crossover ratio (CRO), accuracy, sensitivity, and specificity were used as performance evaluation metrics to quantitatively analyze the segmentation performance of the method. These evaluation metrics allow for a comprehensive assessment of the retinal vessel segmentation model's performance from multiple perspectives, including region overlap, overall classification accuracy, foreground vessel detection capability, and background suppression capability.

[0070] Meanwhile, to verify the effectiveness of each component module of the present invention, ablation experiments and comparative experiments were also conducted in this embodiment. The ablation experiments were based on the benchmark model UNet++, and the DSCAMP module, DFMF module, and visual graph neural network module were introduced sequentially, with the performance improvements brought by each module analyzed. The comparative experiments compared the method of the present invention with U-Net, AttU-Net, ResUNet, Dunet, and the benchmark model UNet++ under the same dataset and evaluation metrics to verify the advantages of the method of the present invention in small blood vessel segmentation, structural continuity preservation, and cross-dataset generalization ability. The results are as follows: Table 1 Ablation Experiment As shown in Table 1, after introducing DSCAMP on the basis of the baseline model UNet++, the Dice coefficient increased from 0.8130 to 0.8195, the crossover ratio increased from 0.6845 to 0.6930, and the sensitivity increased from 0.7810 to 0.8050, indicating that the module can effectively improve the detection capability of microvessels and low-contrast vascular regions.

[0071] After further introducing DFMF, the Dice coefficient increased to 0.8230, the crossover ratio increased to 0.6995, the accuracy increased to 0.9625, and the specificity increased to 0.9820, indicating that the module can effectively suppress background missegmentation while preserving vascular details, thereby improving the stability and accuracy of the segmentation results.

[0072] After integrating DSCAMP, DFMF, and visual graph neural network modules, the present invention achieves a Dice coefficient of 0.8264, an intersection-over-union ratio of 0.7042, an accuracy of 0.9645, a sensitivity of 0.8350, and a specificity of 0.9827. Compared to the benchmark model UNet++, the Dice coefficient is improved by approximately 1.64%, and the sensitivity is improved by approximately 6.91%. This indicates that the present invention has good performance in identifying small blood vessels and maintaining the continuity of vascular networks.

[0073] Table 2 Performance comparison of different algorithms on the DRIVE dataset As shown in Table 2, the method of the present invention achieves Dice coefficients, crossover ratio, accuracy, sensitivity, and specificity of 0.8264, 0.7042, 0.9645, 0.8350, and 0.9827 on the DRIVE dataset, respectively. Overall, it is superior to the comparative method, indicating that the present invention has good effects in the simultaneous segmentation of main blood vessels and small blood vessels, background suppression, and comprehensive accuracy control.

[0074] Table 3 Performance comparison of different algorithms on the STARE dataset As shown in Table 3, on the STARE dataset, the Dice coefficient, crossover ratio, accuracy, sensitivity, and specificity of the method of the present invention reach 0.8115, 0.6828, 0.9669, 0.8122, and 0.9870, respectively; indicating that the present invention can still maintain good segmentation performance on the highly complex STARE dataset.

[0075] Table 4. Performance comparison of different algorithms on the CHASEDB1 dataset. As shown in Table 4, on the CHASEDB1 dataset, the Dice coefficient, crossover ratio, accuracy, sensitivity and specificity of the method of the present invention reached 0.8235, 0.7000, 0.9675, 0.8217 and 0.9787 respectively, indicating that the invention can still effectively extract small blood vessel structures under low contrast image conditions.

[0076] The above results show that the present invention can stably extract the complete retinal vascular network under different resolutions, different pathological complexities and different image quality conditions, and has good cross-dataset adaptability.

[0077] At the same time, according to Figure 6 The comparison between the segmentation results of the present invention and the benchmark model shown in the figure demonstrates that the present invention effectively improves upon the benchmark model's shortcomings in areas such as missed detection of vessel terminals, adhesion in intersecting regions, and local path breaks. This results in segmentation results that more closely resemble the actual labeled results in terms of vessel boundary details, preservation of small branches, and overall connectivity. Therefore, the present invention not only performs well in terms of quantitative evaluation indicators but also demonstrates superior application effects in terms of visual continuity and structural integrity of the segmentation results.

[0078] In summary, this invention achieves high-precision, highly connected retinal vessel segmentation through joint optimization using orientation field guidance and graph neural networks, overcoming the shortcomings of existing methods in extracting small vessels and preserving topology. Experimental results demonstrate that this invention outperforms existing methods on multiple publicly available datasets, exhibiting significant technical advantages and application value.

[0079] Finally, it should be noted that any parts of this invention not described in detail are prior art. Those skilled in the art will understand that the above descriptions are merely preferred embodiments of the invention and are not intended to limit the invention. Although the invention has been described in detail with reference to the foregoing examples, those skilled in the art can still modify the technical solutions described in the foregoing examples or make equivalent substitutions for some of the technical features. All modifications and equivalent substitutions made within the spirit and principles of the invention should be included within the scope of protection of the invention.

Claims

1. A retinal vessel segmentation method based on direction field guidance and graph neural network optimization, characterized in that, Includes the following steps: S1. Obtain retinal fundus images and their corresponding vascular annotation maps, and preprocess the raw retinal fundus images; S2. Construct a retinal vessel segmentation model based on orientation field guidance and graph neural network optimization. The retinal vessel segmentation model includes a backbone segmentation model based on the UNet++ network architecture and a post-processing refinement module based on a visual graph neural network. The trunk segmentation model includes an encoder, a decoder, and nested dense skip connections. The shallow layer of the encoder is equipped with a dynamic strip convolution and adaptive morphology perception module to extract multi-scale features and generate a direction field representing the local orientation information of blood vessels. The decoder is equipped with a direction field-guided multi-scale fusion module to receive the direction field, as well as the shallow high-resolution features from the corresponding layer of the encoder and the upsampled deep semantic features, and perform feature fusion. The preprocessed retinal fundus image is input into the trunk segmentation model to obtain a preliminary blood vessel segmentation probability map; the preliminary blood vessel segmentation probability map is input into the post-processing refinement module, and global context information and blood vessel topology relationship are extracted through graph convolution operation to output the refined final blood vessel segmentation result; S3. A two-stage training method is used to train the trunk segmentation model and the post-processing refinement module in step S2. In the first stage, the trunk segmentation model is trained with the blood vessel annotation map as the supervision signal. In the second stage, the parameters of the trunk segmentation model are fixed, and the preliminary blood vessel segmentation probability map output by the model is used as the input. The post-processing refinement module is trained with the blood vessel annotation map as the supervision signal. S4. Use the trained retinal vessel segmentation model to segment the vessels in the retinal fundus image to obtain the final segmentation result.

2. The retinal blood vessel segmentation method based on direction field guidance and graph neural network optimization according to claim 1, characterized in that, The preprocessing in step S1 is as follows: First, the original retinal fundus image is standardized to obtain a training image; then, the training image is augmented online to obtain an augmented retinal fundus image; finally, the augmented retinal fundus image is cropped randomly to generate image blocks of a fixed size, and a directional sampling strategy is combined to ensure that the cropped image blocks contain a blood vessel region of no less than a predetermined proportion.

3. The retinal blood vessel segmentation method based on direction field guidance and graph neural network optimization according to claim 1, characterized in that, The dynamic strip convolution and adaptive morphology perception module includes three parallel feature extraction branches: a standard convolution branch, a deformable convolution branch, and a dynamic strip convolution branch. The standard convolution branch uses a 3×3 convolutional layer to extract basic local context features. The deformable convolution branch learns the spatial offset field of the input features and adjusts the convolution sampling position to enhance its adaptability to blood vessel cross-sections and irregular morphological regions. The dynamic strip convolution branch predicts the local blood vessel direction based on the input features and constructs a strip-shaped sampling structure along the main direction of the blood vessel, enabling the convolution operation to perform feature aggregation along the extension direction of the slender blood vessel.

4. The retinal blood vessel segmentation method based on direction field guidance and graph neural network optimization according to claim 3, characterized in that, The dynamic strip convolution and adaptive morphology perception module also includes a local-macro attention fusion module for obtaining enhanced encoder output features. The specific steps are as follows: The input for constructing the attention mechanism is as follows: the original input features of the module are expanded by a magnification factor λ to obtain an expanded feature map, which is then used as the query vector in the attention mechanism. The output features of the three branches—standard convolution, deformable convolution, and dynamic strip convolution—are concatenated along the channel dimension, and the concatenated features are used as key vectors. Sum value vector ; Calculate the query vector using the attention formula. With key vector The correlation between them is used to generate attention weights, and the expression of the attention formula is as follows: In the formula, This represents the dimension of the key vector. Represents the key vector transpose; The weighted summation output is reconstructed into a two-dimensional feature map to obtain the fused enhanced features, i.e., the enhanced encoder output features.

5. The retinal vessel segmentation method based on orientation field guidance and graph neural network optimization according to claim 1, characterized in that, The orientation field-guided multi-scale fusion module participates in the fusion of horizontal and vertical strip features through an orientation-aware gating mechanism. Specifically, semantic features and orientation fields are concatenated along the channel dimension and input into the gating network. The original weights of the two channels at each spatial location are output. After applying a Softmax operation along the channel dimension to the original weights of the two channels, normalized gating weights corresponding to the horizontal and vertical directions are obtained, respectively. The gating weights are then used to adaptively weight and fuse the output features of the horizontal and vertical strip convolutional branches to obtain orientation-weighted strip features.

6. The retinal vessel segmentation method based on orientation field guidance and graph neural network optimization according to claim 1, characterized in that, The post-processing refinement module based on visual graph neural networks is specifically used for: dividing the preliminary blood vessel segmentation probability map into blocks through a block embedding layer, dividing the two-dimensional probability map into multiple non-overlapping image blocks, and mapping each image block to a corresponding feature vector to form node features in the graph structure; dynamically searching for the nearest neighbor node for each node in the node feature space, and constructing a k-nearest neighbor graph based on node feature similarity; inputting the constructed graph structure into multiple sequentially stacked visual graph neural network blocks for feature inference and updating, each visual graph neural network block including a graph construction and processing module and a feedforward network; reshaping the updated node features into a two-dimensional feature map, and restoring it to the same spatial resolution as the original retinal fundus image through upsampling, and generating the refined final blood vessel segmentation result through a convolutional output layer.

7. The retinal vessel segmentation method based on orientation field guidance and graph neural network optimization according to claim 1, characterized in that, During the training process of the backbone segmentation model, a composite loss function combining pixel-level segmentation constraints and structural connectivity constraints is adopted. The composite loss function includes a main loss term, a Dice loss term, and a connectivity constraint term.

8. The retinal vessel segmentation method based on orientation field guidance and graph neural network optimization according to claim 1, characterized in that, In step S4, when performing vascular segmentation on the retinal fundus image to be tested, the entire retinal fundus image is divided into blocks for prediction using a sliding window method, and the prediction results of each window are stitched together to obtain a complete retinal vascular segmentation result.

Images