Eye fundus image retinal vessel segmentation method, device and system, and storage medium

Abstract

The invention discloses a fundus image retinal vessel segmentation method, device and system, and a storage medium, and the method comprises the steps: S1, obtaining to-be-segmented fundus image data, and carrying out the preprocessing; s2, constructing a segmentation model based on receptive field attention and cross attention decoding according to the preprocessed fundus image data; performing end-to-end feature extraction and pixel-level classification on the image by using the segmentation model; wherein the segmentation model comprises an RFAConv adaptive encoder, a self-attention bottleneck module, a feature enhancement jump connection module and a decoding module based on cross attention. According to the technical scheme, the defects of the prior art in the aspects of feature extraction flexibility and feature fusion effectiveness are overcome, and the edge precision and robustness of fundus image retinal blood vessel segmentation are improved.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, specifically relating to a method, device, system, and storage medium for retinal vessel segmentation of fundus images based on the fusion of receptive-field attention convolution (RFAConv) and skip connection features. It is suitable for the automatic segmentation and extraction of fine targets such as retinal vessels and their lesion areas. Background Technology

[0002] In recent years, encoder-decoder convolutional neural networks (CNNs), represented by U-Net, have achieved remarkable results in fundus medical image segmentation tasks. However, existing segmentation methods still face significant challenges in handling retinal vessels and small tumors in the fundus.

[0003] First, traditional convolutional operations share kernel parameters across the entire image during feature extraction, meaning the model processes features from different locations in fundus medical images in the same way. However, fundus medical images are often highly non-stationary, with significant differences in texture and shape across different regions; for example, the terminal bifurcation of retinal vessels varies in thickness from the main trunk to the terminal bifurcation. Standard convolution struggles to adaptively adjust receptive field weights based on spatial variations in input features, resulting in insufficient ability to capture fine structures.

[0004] Secondly, the U-Net architecture primarily relies on skip connections to directly concatenate shallow features from the encoder to the decoder. Existing skip connections typically employ simple channel concatenation or element-wise addition. While this approach introduces spatial information, it also introduces background noise and redundant features during the encoding phase. Simple concatenation operations cannot distinguish the importance of foreground and background, leading to edge blurring or artifacts in the decoder when restoring resolution. Furthermore, the upsampling process in existing models often lacks effective semantic guidance, making it difficult to maintain semantic consistency while restoring spatial resolution. Summary of the Invention

[0005] To address the problems existing in the prior art, this invention provides a method, apparatus, system, and storage medium for retinal vessel segmentation in fundus images, which overcomes the shortcomings of the prior art in terms of feature extraction flexibility and feature fusion effectiveness, and improves the edge accuracy and robustness of retinal vessel segmentation in fundus images.

[0006] To achieve the above objectives, the present invention provides the following solution: A method for segmenting retinal vessels in fundus images, comprising: Step S1: Obtain the fundus image data to be segmented and perform preprocessing; Step S2: Based on the preprocessed fundus image data, construct a segmentation model based on receptive field attention and cross-attention decoding; use the segmentation model to perform end-to-end feature extraction and pixel-level classification of the image; wherein, the segmentation model includes: RFAConv adaptive encoder, self-attention bottleneck module, feature enhancement skip connection module and cross-attention-based decoder.

[0007] Preferably, in step S1, the input medical image is set as follows: ,in, For the number of channels, , For height and width; Two downsampled copies are generated using bilinear interpolation during the preprocessing stage. and : ; ; Preferably, in step S2, a joint loss function is used. To optimize the segmentation model, let... If the labels are true, then the total loss is the weighted sum of the losses at the three scales: ; in, For binary cross-entropy loss, To compare the losses, These are the weighting coefficients for each scale. This is a probability prediction graph.

[0008] The present invention also provides a device for segmenting retinal vessels in fundus images, comprising: The first processing module is used to acquire the fundus image data to be segmented and to perform preprocessing; The second processing module constructs a segmentation model based on receptive field attention and cross-attention decoding based on the preprocessed fundus image data; and uses the segmentation model to perform end-to-end feature extraction and pixel-level classification of the image; wherein, the segmentation model includes: an RFAConv adaptive encoder, a self-attention bottleneck module, a feature enhancement skip connection module, and a cross-attention-based decoder.

[0009] Preferably, in the first processing module, the input medical image is set as follows: ,in, For the number of channels, , For height and width; Two downsampled copies are generated using bilinear interpolation during the preprocessing stage. and : ; ; Preferably, the second processing module employs a joint loss function. To optimize the segmentation model, let... If the labels are true, then the total loss is the weighted sum of the losses at the three scales: ; in, For binary cross-entropy loss, To compare the losses, These are the weighting coefficients for each scale. This is a probability prediction graph.

[0010] The present invention also provides a retinal vessel segmentation system for fundus images, comprising: a memory and a processor, wherein the memory stores a computer program executed by the processor, and the computer program performs a retinal vessel segmentation method for fundus images when executed by the processor.

[0011] The present invention also provides a storage medium storing a computer program that executes a retinal vessel segmentation method for fundus images during operation.

[0012] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention achieves adaptive adjustment of the receptive field during the feature extraction stage through RFAConv, solving the problem of low sensitivity of standard convolution to multi-scale targets; it effectively fuses texture and background information and suppresses noise interference through multi-path skip connections; and it significantly improves the boundary clarity and connectivity of the segmentation results in the retinal vessel segmentation task by combining cross-attention decoding. Attached Figure Description

[0013] To more clearly illustrate the technical solution of the present invention, the drawings used in the embodiments are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0014] Figure 1 This is a flowchart of the retinal vessel segmentation method for fundus images according to an embodiment of the present invention; Figure 2 This is a schematic diagram of the network architecture of the retinal vessel segmentation model in fundus images according to an embodiment of the present invention; Figure 3 This is a detailed structural diagram of the SkipConnectModule described in an embodiment of the present invention. Detailed Implementation

[0015] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0016] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0017] Example 1 like Figures 1 to 3 As shown, this invention provides a method for retinal vessel segmentation in fundus images. Through improved convolution operations and a multi-level attention mechanism, it addresses the problems of blurred lesion boundaries and variable vessel shapes in fundus vessels; including: Step S1: Obtain the fundus image data to be segmented and perform preprocessing; Step S2: Based on the preprocessed fundus image data, construct a segmentation model based on receptive field attention and cross-attention decoding; use the segmentation model to perform end-to-end feature extraction and pixel-level classification of the image; wherein, the segmentation model includes: RFAConv adaptive encoder, self-attention bottleneck module, feature enhancement skip connection module, and cross-attention-based decoder.

[0018] In one embodiment of the present invention, in step S1, the input medical image is assumed to be... ,in For the number of channels, , For height and width.

[0019] To enhance the model's ability to perceive targets at different scales, this invention generates two downsampled copies during the preprocessing stage using bilinear interpolation. and : ; ; These three scales of images serve as inputs for different stages of the encoder.

[0020] As one embodiment of the present invention, in the RFAConv adaptive encoder of step S2, the present invention introduces a receptive field attention convolution (RFAConv) module to replace the traditional convolution in the encoder stage. Traditional convolution is limited by a fixed receptive field, while RFAConv achieves dynamic reweighting of features within the receptive field of the convolution kernel through a non-overlapping sliding window and attention weighting mechanism. Specifically, this module first uses average pooling and convolution to generate an attention map for each receptive field region, then uses this attention map to spatially weight the unfolded features, and finally performs feature mapping through grouped convolution. This design enables the model to adaptively strengthen the weights of key regions according to the morphological features of the lesion, significantly improving the feature extraction capability of fine structures such as the ends and edges of blood vessels.

[0021] As one embodiment of the present invention, in the feature enhancement skip connection module of step S2, to address the noise problem introduced by traditional skip connections, the present invention designs a feature fusion module that includes parallel average pooling and max pooling. This module receives shallow features from the encoder, extracts background context information through the average pooling branch, and extracts texture and boundary salient features through the max pooling branch. After concatenating the two feature paths, spatial fusion weights are generated through convolutional layers and a sigmoid activation function, and finally, feature integration is performed again through convolutional layers. This mechanism can effectively "filter" redundant information in shallow features and enhance the response to the target region.

[0022] In one embodiment of the present invention, in the cross-attention-based decoder of step S2, the present invention abandons simple bilinear interpolation and adopts a cross-attention mechanism during the decoding stage. The decoder uses the high-level semantic features of the previous layer as a query and the enhanced features passed in from the skip connections as keys and values, guiding the upsampling process through the attention mechanism. This enables the decoder to fully utilize the high-resolution details of the encoder while maintaining the consistency of global semantics when restoring resolution.

[0023] As one embodiment of the present invention, in the bottom self-attention bottleneck layer (Self-AttentionBlock) of step S2, a self-attention module is introduced at the connection between the encoder and the decoder (bottom bottleneck layer) to capture the deepest global long-range dependencies of the image, thereby further enhancing the model's ability to understand large-scale targets and overall structures.

[0024] Furthermore, step S2 includes: Step 21: Feature Encoding Based on Receptive Field Attention Convolution (RFAConv) In the encoder path, this invention uses an RFAConv adaptive encoder to replace the standard convolution to extract adaptive features.

[0025] For the input feature tensor X 1 Input into the first RFAConv, X 2 Input to the second FRAConv, X 3 Input into the third RFAConv.

[0026] RFAConv first generates attention weights for the receptive field space through grouped convolution and average pooling. The calculation formula is as follows: ; Simultaneously, basic features are extracted through the feature generation branch. : ; Finally, by element-wise superimposing and recombining the basic features with the spatial attention weights, the weighted output features are obtained. : ; This process enables the convolution kernel parameters to be adaptively adjusted based on the feature intensity at different locations in the input image.

[0027] Step 22: Self-Attention Bootleck In the deepest layer of the encoder, by After passing through the final encoder, the final input is fed into the attention bottleneck module. .

[0028] ; The attention bottleneck module captures global dependencies through parallel channel statistics. First, it calculates the average value across the channel dimensions. and maximum value : After concatenating the two along the channel dimension, through Convolution and Activation function generates global attention map : ; This step effectively enhances the model's ability to focus on key regions in deep semantics.

[0029] Step 23: Multi-pooling feature enhancement for SkipConnection To address the noise issue during deep-shallow layer feature fusion, this invention designs a feature enhancement skip connection module. Let the output feature of a certain layer of the encoder be... The processing procedure of the feature enhancement skip connection module is as follows: Background and texture features were extracted using average pooling and max pooling, respectively, and then... Convolution is used for transformation: ; After splicing the two feature paths, through Convolution and Activation generates fusion features : ; Finally, in the backbone network, this skip connection feature is superimposed onto the downsampled features as a residual.

[0030] Step 24: Cross-Attention Decoder The decoder utilizes a cross-attention block to fuse features from the backbone network. and signals from the previous decoder .

[0031] First, regarding the first decoder: ; Obtain feature map .

[0032] Next, the cross-attention module first performs a linear mapping on the input, then... Feature map size enlarged to and Consistent, then cross-attention is applied to enhance semantic features, resulting in... : ; in, , The purpose of the 1x1 convolution operation is to scale the feature map. Alignment.

[0033] Similarly, for and have: ; This mechanism ensures that the decoder retains only shallow details consistent with deep semantics when restoring resolution.

[0034] Step 25: Deep Supervision and Loss Calculation At the three different resolution levels of the decoder Output the prediction results separately. For the first... Features at each scale ,pass Convolution maps to the class space and utilizes... Normalization yields the predicted probability map : ; During training, a joint loss function is used. Optimize the model. Let... If the ground truth is used, the total loss is the weighted sum of the losses at the three scales: ; in, For binary cross-entropy loss, To compare the losses, These are the weighting coefficients for each scale.

[0035] Example 2 The present invention also provides a device for segmenting retinal vessels in fundus images, comprising: The first processing module is used to acquire the fundus image data to be segmented and to perform preprocessing; The second processing module constructs a segmentation model based on receptive field attention and cross-attention decoding based on the preprocessed fundus image data; and uses the segmentation model to perform end-to-end feature extraction and pixel-level classification of the image; wherein, the segmentation model includes: an RFAConv adaptive encoder, a self-attention bottleneck module, a feature enhancement skip connection module, and a cross-attention-based decoder.

[0036] In one embodiment of the present invention, in the first processing module, the input medical image is set as follows: ,in, For the number of channels, , For height and width; Two downsampled copies are generated using bilinear interpolation during the preprocessing stage. and : ; ; As one embodiment of the present invention, the second processing module employs a joint loss function. To optimize the segmentation model, let... If the labels are true, then the total loss is the weighted sum of the losses at the three scales: ; in, For binary cross-entropy loss, To compare the losses, These are the weighting coefficients for each scale. This is a probability prediction graph.

[0037] Example 3 The present invention also provides a retinal vessel segmentation system for fundus images, comprising: a memory and a processor, wherein the memory stores a computer program executed by the processor, and the computer program performs a retinal vessel segmentation method for fundus images when executed by the processor.

[0038] Example 4 The present invention also provides a storage medium storing a computer program that executes a retinal vessel segmentation method for fundus images during operation.

[0039] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Various modifications and improvements made to the technical solutions of the present invention by those skilled in the art without departing from the spirit of the present invention should fall within the protection scope defined by the claims of the present invention.

Claims

1. A method for segmenting retinal vessels in fundus images, characterized in that, include: Step S1: Obtain the fundus image data to be segmented and perform preprocessing; Step S2: Based on the preprocessed fundus image data, construct a segmentation model based on receptive field attention and cross-attention decoding; use the segmentation model to perform end-to-end feature extraction and pixel-level classification of the image; wherein, the segmentation model includes: RFAConv adaptive encoder, self-attention bottleneck module, feature enhancement skip connection module and cross-attention-based decoder.

2. The method for retinal vessel segmentation in fundus images as described in claim 1, characterized in that, In step S1, let the input medical image be... ,in, For the number of channels, , For height and width; Two downsampled copies are generated using bilinear interpolation during the preprocessing stage. and : ； 。 3. The method for retinal vessel segmentation in fundus images as described in claim 2, characterized in that, In step S2, a joint loss function is used. To optimize the segmentation model, let... If the labels are true, then the total loss is the weighted sum of the losses at the three scales: ； in, For binary cross-entropy loss, To compare the losses, These are the weighting coefficients for each scale. This is a probability prediction graph.

4. A device for segmenting retinal vessels in fundus images, characterized in that, include: The first processing module is used to acquire the fundus image data to be segmented and to perform preprocessing; The second processing module constructs a segmentation model based on receptive field attention and cross-attention decoding based on the preprocessed fundus image data; and uses the segmentation model to perform end-to-end feature extraction and pixel-level classification of the image; wherein, the segmentation model includes: an RFAConv adaptive encoder, a self-attention bottleneck module, a feature enhancement skip connection module, and a cross-attention-based decoder.

5. The retinal vessel segmentation device for fundus images as described in claim 4, characterized in that, In the first processing module, let the input medical image be... ,in, For the number of channels, , For height and width; Two downsampled copies are generated using bilinear interpolation during the preprocessing stage. and : ； 。 6. The retinal vessel segmentation device for fundus images as described in claim 5, characterized in that, The second processing module uses a joint loss function. To optimize the segmentation model, let... If the labels are true, then the total loss is the weighted sum of the losses at the three scales: ； in, For binary cross-entropy loss, To compare the losses, These are the weighting coefficients for each scale. This is a probability prediction graph.

7. A retinal vessel segmentation system for fundus images, characterized in that, include: A memory and a processor, wherein the memory stores a computer program executed by the processor, the computer program performing the retinal vessel segmentation method for fundus images as described in any one of claims 1-3 when executed by the processor.

8. A storage medium, characterized in that, The storage medium stores a computer program that, when executed, performs the retinal vessel segmentation method for fundus images as described in any one of claims 1-3.

Images