Retinal vessel segmentation method based on long-range dependency and multi-scale input

Abstract

This specification provides a method, apparatus, device, and medium for retinal vessel segmentation based on long-range dependencies and multi-scale input. The method includes: preprocessing a fundus image dataset, dividing it into a training set, a validation set, and a test set; augmenting the training and validation sets; constructing a retinal vessel segmentation network; extracting features from the training set images to obtain multi-scale semantic information; constructing a Local Information Aware Transformer (LAT) module; adding the LAT to the retinal vessel segmentation network; training the retinal vessel segmentation network model using the training set image data; evaluating the model performance using the validation set images; adjusting the model parameters to obtain the retinal vessel segmentation model; and inputting the preprocessed fundus image to be segmented into the retinal segmentation model for segmentation to obtain a segmented retinal vessel image. This invention enables accurate segmentation of retinal vessels.

Description

Technical Field

[0001] This document relates to the field of image segmentation technology, and in particular to a method, apparatus, device, and medium for retinal vessel segmentation based on long-range dependency and multi-scale input. Background Technology

[0002] Currently, automated retinal vessel segmentation methods can be mainly divided into two categories: unsupervised and supervised methods. Supervised methods directly extract retinal vessels by establishing an algorithmic model, without requiring manual annotation as prior knowledge. Unsupervised methods mainly include morphological and mathematical model methods, matched filtering methods, vessel tracking methods, and variable model methods. Compared to unsupervised methods, supervised methods can better distinguish between large and small vessels, achieving higher segmentation accuracy. Supervised methods can be further subdivided into traditional machine learning-based and deep learning-based methods, with deep learning-based methods being more popular among researchers.

[0003] Currently, deep learning-based retinal vessel segmentation methods mainly rely on convolutional neural networks (CNNs), with U-Net serving as an example. U-Net, a well-known network in medical image segmentation, is highly versatile and frequently used in retinal segmentation tasks. However, due to the inherent locality of CNNs, the range of feature information they extract is limited. While adept at extracting local details, it struggles to capture long-range dependencies and global contextual information. Therefore, the current mainstream CNNs used for retinal vessel segmentation are unsatisfactory in capturing long-range dependencies. Furthermore, downsampling operations are frequently used to extract multi-scale semantic information, inevitably leading to the loss of some semantic information. In addition, the quality of the fundus images to be segmented is often low, frequently containing image noise and uneven illumination, and the low contrast between vessels and background makes it difficult to distinguish vessels. All these problems result in the current mainstream retinal vessel segmentation methods still not achieving ideal segmentation accuracy. Summary of the Invention

[0004] This specification provides one or more embodiments of a retinal vessel segmentation method based on long-range dependencies and multi-scale inputs, including:

[0005] The acquired fundus image dataset is preprocessed to reduce image noise and enhance image contrast. The dataset is divided into a training set, a validation set, and a test set. Data augmentation processing is performed on the training set and the validation set.

[0006] A retinal vessel segmentation network is constructed, a hybrid attention convolution module is designed to extract features from the training set images, a multi-scale input module is designed to obtain multi-scale semantic information, a Local Information Aware Transformer (LAT) module is constructed, and the LAT is added to the retinal vessel segmentation network.

[0007] The training set image data is input into the retinal vessel segmentation network for model training. The model performance is evaluated using validation set images. The model parameters are adjusted based on the evaluation results to obtain the retinal vessel segmentation model.

[0008] The fundus image to be segmented is preprocessed and then input into the retinal segmentation model for segmentation to obtain a retinal vessel segmentation image.

[0009] Furthermore, the preprocessing of the acquired fundus image dataset to reduce image noise and enhance image contrast specifically includes:

[0010] The dataset images are converted to grayscale, and the G channel in the RGB images is extracted. The grayscale conversion formula is shown in Formula 1:

[0011] Gray(x,y)=G(x,y) Formula 1;

[0012] Where G(·) represents the G channel in the RGB image;

[0013] Normalizing an image involves scaling its overall grayscale values. The normalization formula is as follows:

[0014] As shown in Equation 2.

[0015]

[0016] Where, p ij The pixel values ​​of the image. denoted as the standardized image pixel value, μ is the mean of the image pixel value, and δ is the standard deviation of the image pixel value;

[0017] Adaptive histogram equalization is applied to the image to stretch the grayscale histogram of the original image, improve image contrast, and limit the extremely dark and bright parts of the image.

[0018] Gamma correction is applied to the image to suppress uneven illumination and central line reflection in the fundus image;

[0019] Finally, the fundus image data was normalized, reducing the grayscale values ​​from 0 to 255 to between 0 and 1.

[0020] Furthermore, the data augmentation process for the training and validation sets specifically includes:

[0021] The training set and validation set are randomly cropped to form 64×64 image blocks.

[0022] The cropped images are randomly flipped horizontally, randomly flipped vertically, and randomly rotated to obtain the augmented training and validation sets.

[0023] Furthermore, the hybrid attention convolution module consists of 3×3 convolution, batch normalization, ReLU activation function, DropBlock, residual connection and hybrid attention mechanism;

[0024] The hybrid attention mechanism includes a channel attention mechanism and a position attention mechanism. The channel attention mechanism captures the correlation between features of different channels from the perspective of channels and assigns weights to each channel of the feature map. The position attention mechanism improves the network's ability to acquire remote context information based on accurate position information and captures the dependencies between pixels at different locations.

[0025] Furthermore, the design of the multi-scale input module to obtain multi-scale semantic information specifically involves:

[0026] By using a multi-scale input module to perform multiple max pooling operations on the input image to construct an image pyramid, the input images and feature maps of different scales are stitched together and fused to reduce information loss and extract richer multi-scale semantic information.

[0027] Furthermore, the local information-aware Transformer module includes a Transformer structure and a convolutional structure;

[0028] The Transformer structure is used to serialize the input feature map, linearly mapping the two-dimensional image data into one-dimensional serialized data. Spatial location encoding is added to the serialized data to record the position information of pixels. Layer normalization is used to maintain the stability of the data feature distribution and accelerate network convergence.

[0029] The output of the convolutional structure and the output of the Transformer structure are concatenated and merged in the channel dimension, and 1×1 convolution is used to reduce the channel dimension to obtain the LAT output.

[0030] The convolutional structure mainly consists of two 3×3 convolutions and two GELU activation functions.

[0031] Furthermore, the method also includes: testing the retinal vessel segmentation model using test set images, and evaluating the vessel segmentation effect according to preset evaluation indicators, including: accuracy (ACC), sensitivity (SE), specificity (SP), and F1-score.

[0032] This specification provides one or more embodiments of a retinal vessel segmentation system, comprising:

[0033] Data processing module: used to preprocess the acquired fundus image dataset, reduce image noise, enhance image contrast, divide the dataset into training set, validation set and test set, and perform data augmentation processing on the training set and validation set;

[0034] Model building module: used to build the retinal vessel segmentation network DMS-Net, design a hybrid attention convolution module to extract features from the training set images, design a multi-scale input module to obtain multi-scale semantic information, build a local information-aware Transformer module, and add the local information-aware Transformer module to the deepest layer of DMS-Net.

[0035] Model training module: used to input training set image data into the retinal vessel segmentation network DMS-Net for model training, evaluate model performance through validation set images, adjust model parameters according to evaluation results, and obtain retinal vessel segmentation model;

[0036] Segmentation module: This module is used to preprocess the fundus image that needs to be segmented and then input it into the retinal segmentation model for segmentation to obtain a retinal vessel segmentation image.

[0037] This specification provides an electronic device according to one or more embodiments, including: a processor; and a memory arranged to store computer-executable instructions, which, when executed, cause the processor to perform the steps of the retinal vessel segmentation method described above.

[0038] This specification provides one or more embodiments of a storage medium for storing computer-executable instructions that, when executed, implement the steps of the retinal vessel segmentation method described above.

[0039] This invention reduces noise and illumination effects while improving contrast by extracting the green channel from fundus image data, standardizing, applying adaptive histogram equalization, gamma correction, and normalization. It also augments the data to obtain a larger training dataset. A hybrid attention convolutional module replaces the ordinary convolutional module for feature extraction, simultaneously extracting local details and global correlations, resulting in stronger feature extraction capabilities and avoiding overfitting, thus improving the model's segmentation performance and generalization ability. A multi-scale input module is designed to extract richer multi-scale semantic information, improving segmentation performance. Finally, a local information-aware Transformer module is designed to capture long-range dependencies and global contextual semantic information, while also extracting local details through convolutional structures, achieving multi-level and efficient feature representation and further improving the model's segmentation performance, resulting in a superior segmentation model.

[0040] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and in order to make the above description and other objects, features and advantages of the present invention more obvious and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0041] To more clearly illustrate the technical solutions in one or more embodiments of this specification or in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0042] Figure 1 A flowchart illustrating a retinal vessel segmentation method based on long-range dependencies and multi-scale inputs, provided for one or more embodiments of this specification;

[0043] Figure 2 A schematic diagram illustrating the specific process of a retinal vessel segmentation method based on long-range dependency and multi-scale input provided for one or more embodiments of this specification;

[0044] Figure 3 A schematic diagram of the DMS-Net network structure for a retinal vessel segmentation method based on long-range dependency and multi-scale input provided for one or more embodiments of this specification;

[0045] Figure 4 A schematic diagram of a hybrid attention convolutional module structure for a retinal vessel segmentation method based on long-range dependencies and multi-scale inputs, provided for one or more embodiments of this specification;

[0046] Figure 5 A schematic diagram of a hybrid attention mechanism for a retinal vessel segmentation method based on long-range dependency and multi-scale input, provided for one or more embodiments of this specification;

[0047] Figure 6 A schematic diagram of the local information sensing Transformer module structure for a retinal vessel segmentation method based on long-range dependency and multi-scale input provided in one or more embodiments of this specification;

[0048] Figure 7 A schematic diagram illustrating the composition of a retinal vessel segmentation system based on long-range dependency and multi-scale input, provided for one or more embodiments of this specification;

[0049] Figure 8 This is a schematic diagram of the structure of an electronic device provided for one or more embodiments of this specification. Detailed Implementation

[0050] To enable those skilled in the art to better understand the technical solutions in one or more embodiments of this specification, the technical solutions in one or more embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this specification, and not all of the embodiments. Based on one or more embodiments of this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of this document.

[0051] Method Implementation Examples

[0052] According to embodiments of the present invention, a retinal vessel segmentation method based on long-range dependencies and multi-scale inputs is provided. By constructing a retinal vessel segmentation network (DMS-Net) based on long-range dependencies and multi-scale inputs, the segmentation accuracy of retinal vessels is improved. Figure 1 A flowchart illustrating a retinal vessel segmentation method based on long-range dependencies and multi-scale inputs, provided for one or more embodiments of this specification, is shown below. Figure 1 As shown, the retinal vessel segmentation method according to an embodiment of the present invention specifically includes:

[0053] S1. Preprocess the acquired fundus image dataset to reduce image noise and enhance image contrast. Divide the dataset into a training set, a validation set, and a test set. Perform data augmentation processing on the training set and the validation set.

[0054] S2. Construct a retinal vessel segmentation network, design a hybrid attention convolution module to extract features from the training set images, design a multi-scale input module to obtain multi-scale semantic information, construct a Local Information Aware Transformer (LAT) module, and add the LAT to the retinal vessel segmentation network.

[0055] S3. Input the training set image data into the retinal vessel segmentation network for model training, evaluate the model performance using the validation set images, adjust the model parameters according to the evaluation results, and obtain the retinal vessel segmentation model;

[0056] S4. After preprocessing the fundus image to be segmented, input it into the retinal segmentation model for segmentation to obtain a retinal vessel segmentation image.

[0057] Specifically, Figure 2 A flowchart illustrating the retinal vessel segmentation method is shown below. Figure 2 As shown, the specific process of the retinal vessel segmentation method includes:

[0058] Obtain a dataset of fundus images and preprocess it to reduce image noise and improve the contrast between blood vessel pixels and background pixels, thereby enhancing the segmentation accuracy of deep learning models for retinal vessels. Specific methods include:

[0059] First, the dataset images are converted to grayscale. In this embodiment, the grayscale conversion method is to extract the G channel (green channel) from the RGB image because the contrast between the blood vessel outline and the background is the highest and the noise is the least in the green channel. The grayscale conversion formula is shown in Formula 1:

[0060] Gray(x,y)=G(x,y) Formula 1;

[0061] Where G(·) represents the G channel of the RGB image;

[0062] Next, the image is standardized by scaling the overall grayscale values ​​to prevent extreme cases in the image data and to some extent solve the problem of uneven illumination in fundus images. The standardization formula is shown in Formula 2:

[0063]

[0064] Where, p ij The pixel values ​​of the image. The image pixel values ​​are standardized, μ is the mean of the image pixel values, and δ is the standard deviation of the image pixel values. The formulas for calculating μ and δ are shown in Formula 3 and Formula 4.

[0065]

[0066]

[0067] Where W is the number of rows of image pixels and H is the number of columns of image pixels;

[0068] The image is then subjected to adaptive histogram equalization. Adaptive histogram equalization stretches the grayscale histogram of the original image to a certain extent, thereby improving the image contrast and limiting the extremely dark and bright parts of the image.

[0069] Gamma correction is applied to the image to suppress phenomena such as uneven illumination and central line reflection in fundus images. The gamma correction formula used in this invention is shown in Formula 5:

[0070] y = Ax γ Formula 5;

[0071] Where x is the pixel value of the image, and in this embodiment, variable A takes the value of 1 and variable γ takes the value of 0.8;

[0072] Finally, the fundus image data was normalized, reducing the gray levels from 0 to 255 to between 0 and 1.

[0073] The fundus image dataset was divided into training, validation, and test sets.

[0074] After image preprocessing, due to the relatively small amount of data in the fundus image dataset, data augmentation is required for both the training and validation sets to increase their data volume, especially the training set, to facilitate the training of the retinal vessel segmentation model. The data augmentation method in this embodiment includes: randomly cropping the training and validation images into 64×64 image patches; randomly flipping the images horizontally; randomly flipping the images vertically; and randomly rotating the images. After data augmentation, a large number of 64×64 image patches are obtained for training and validating the segmentation model.

[0075] A retinal vessel segmentation network (DMS-Net) based on long-range dependency and multi-scale input was built using the PyTorch deep learning framework. The structure of DMS-Net is as follows. Figure 3 As shown, DMS-Net is an improvement on the classic image segmentation network U-Net. Its overall structure is a U-shaped structure, consisting of an encoder on the left and a decoder on the right. DMS-Net has only 4 layers, one less than U-Net, and the number of feature map channels output by each layer is also half that of U-Net. The overall number of parameters of the model is relatively small.

[0076] A hybrid attention convolutional module is designed to replace the ordinary convolutional module for feature extraction. The structure of the hybrid attention convolutional module is as follows: Figure 4 As shown, compared with ordinary convolutional modules, the hybrid attention convolutional module designed in this embodiment of the invention can extract local detail information and long-range dependencies at the same time, with stronger feature extraction capabilities and can avoid overfitting, thereby improving the segmentation performance and generalization ability of the model.

[0077] The hybrid attention convolution module consists of 3×3 convolution, batch normalization, ReLU activation function, DropBlock, residual connections, and a hybrid attention mechanism. DropBlock performs regularization by randomly discarding some pixel blocks, effectively addressing the model overfitting problem. DropBlock has two main parameters: block_size and γ. block_size represents the size of the block to be deleted, and γ controls the number of activation units to be deleted. The formula for calculating γ is shown in Equation 6.

[0078]

[0079] Here, keep_prob represents the threshold probability, (1-keep_prob) represents the probability of turning off the unit, feat_size represents the size of the feature map, and (feat_size-block_size+1) is the effective seed region. The size of γ depends on keep_prob. Usually, the value of γ is not set directly, but rather the threshold probability is set to calculate γ.

[0080] Residual connections can prevent network degradation during model training and ensure that each module has a positive impact on model performance during training.

[0081] Hybrid attention mechanism structure such as Figure 5 As shown, this attention mechanism adopts a design idea of ​​fusing different attention mechanisms, combining channel attention and position attention in an additive manner. The channel attention mechanism can capture the correlation between features of different channels from the perspective of channels and assign weights to each channel of the feature map; the position attention mechanism can improve the network's ability to obtain remote context information through accurate position information and capture the dependency relationship between pixels at different positions, thereby improving the segmentation performance of the model.

[0082] The formula for calculating channel attention is shown in Formula 7:

[0083] A channel =Conv 1×1 (ReLU(Conv 1×1 (AvgPool(x)))) Formula 7;

[0084] Where x represents the input feature map data, AvgPool(·) represents global average pooling, and Conv 1×1 (·) represents a 1×1 convolution. Two 1×1 convolutions squeeze and expand the channel weight data to learn the relationship between channels. ReLU(·) represents the ReLU activation function.

[0085] Location attention first employs two pooling kernels of size (H, 1) and (1, W) for the input x, performing horizontal and vertical pooling operations on each channel respectively. This aggregates features along two spatial directions and generates a pair of direction-aware attention maps. These two transformations enable the attention module to capture not only long-range dependencies along one spatial direction but also to preserve accurate location information along the other, which helps the network more accurately locate objects of interest. Then, the two attention maps are concatenated, followed by a 1×1 convolution to reduce the dimensionality of the concatenated attention map's channels. The relevant calculation formulas for this operation are shown in Equation 8.

[0086] f = ReLU(Conv) 1×1 Formula 8; ([HorPool(x),VerPool(x)]))

[0087] Where HorPool(·) represents horizontal pooling, VerPool(·) represents vertical pooling, [,] represents concatenation along spatial dimensions, and Conv 1×1 Let f represent a 1×1 convolution, and ReLU(·) represent the ReLU activation function. C / r×(H+W) This represents the intermediate feature maps of spatial information in the horizontal and vertical directions, where r represents the compression ratio parameter. Next, f is split along the spatial dimensions into two independent tensors fi. h ∈R R / r×H and f w ∈R C / r×W Then, two 1×1 convolutions are used to combine the feature map f. h and f w The number of channels is changed to the same as the input x. Finally, the two feature maps are multiplied to obtain the positional attention, as shown in Equation 9:

[0088]

[0089] Where Sigmoid(·) represents the Sigmoid activation function. This represents matrix multiplication. The positional attention and channel attention are added to obtain a hybrid attention weight map, which is then multiplied by the input x, as shown in Equation 10:

[0090] y = x * Sigmoid(A channel +A position) Formula 10;

[0091] Here, * denotes element-wise multiplication, and y represents the final output. It possesses rich semantic information and selectively aggregates semantics based on the attention-weighted graph. Therefore, the network can adaptively select features beneficial for blood vessel segmentation and exhibits strong semantic expressive power.

[0092] Design a multi-scale input module that uses multiple max-pooling operations on the input image to construct an image pyramid. This module then concatenates and fuses input images and feature maps at different scales to reduce information loss. This allows the model to extract richer multi-scale semantic information, thereby improving segmentation performance and reducing the frequency of missegmentation of blood vessels. The structure of the multi-scale input module is as follows: Figure 3 As shown on the left.

[0093] This invention designs a Local Information Aware Transformer Module (LAT). While the Transformer structure naturally possesses excellent modeling capabilities for global features and long-range dependencies of input information, it lacks the inductive bias capability that convolutional neural networks excel at. Therefore, in this embodiment, LAT combines the Transformer and convolutional structures, enabling it to capture long-range dependencies and global contextual semantic information, while also extracting local detail information through convolutional structures. LAT thus obtains multi-level and efficient feature representation, thereby improving the model's segmentation performance. The structure of LAT is as follows: Figure 6 As shown.

[0094] LAT is mainly divided into two parts: the Transformer part and the convolutional part. The Transformer part first performs image serialization on the input feature map, linearly mapping the two-dimensional image data into one-dimensional serialized data. Spatial location encoding is added to the serialized data to record the position information of pixels. Layer normalization is used to maintain the stability of the data feature distribution and accelerate network convergence. The multi-head self-attention mechanism is the core of the Transformer. The serialized data is projected onto three learnable weight matrices to generate query (q), key (k), and value (v) matrices as inputs to the multi-head self-attention mechanism. The multi-head self-attention mechanism can be defined by formulas 11-13:

[0095] MSA(q,k,v)=Concat(head1,…head h ) Formula 11;

[0096] head i =Attention(qW) q kW k ,vW v ) Formula 12;

[0097]

[0098] Where q, k, and v are the inputs to the multi-head self-attention function, Concat(·) represents concatenation and fusion, i∈[1,h], and the projection W q W k W v It is a learnable parameter matrix used to generate q, k, v, d k Let k be the dimension of matrix k.

[0099] The Transformer part incorporates two residual connections, primarily to address the vanishing gradient and weight matrix degradation issues, and to accelerate network convergence. The parameters are then learned using a multilayer perceptron, and the output is transformed into two-dimensional image data through a reshaping operation. The computation process of the Transformer part is shown in Equations 14-17.

[0100] x1 = Linear(x0) + p (Formula 14)

[0101] x2 = MSA(LN(x1)) + x1 (Formula 15)

[0102] x3 = MLP(LN(x2)) + x2 (Formula 16)

[0103] y Tran =Reshape(LN(x3)) Formula 17;

[0104] Where x0 represents the input of LAT, Linear(·) represents linear mapping, p represents spatial location encoding, MLP(·) represents multilayer perceptron, MSA(·) represents multi-head self-attention mechanism, LN(·) represents layer normalization, Reshape(·) represents reshaping, and y Tran This represents the output of the Transformer part.

[0105] The convolutional part of LAT mainly consists of two 3×3 convolutions and two GELU activation functions. The convolutional part can be defined as Equation 18:

[0106] y CNN =GELU(Conv 3×3 (GELU(Conv 3×3 (x0)))) Formula 18;

[0107] Where Conv 3×3 (·) represents a 3×3 convolution, GELU(·) represents the GELU activation function, and y CNN This represents the output of the convolutional part. (The last part, "y", appears to be a typo and can be left as is.) cNN and y TranThe LAT output is obtained by concatenating and fusing the data along the channel dimension, and then using 1×1 convolution to reduce the channel dimension. The calculation formula is shown in Equation 19.

[0108] y = Conv 1×1 ([y CNN ,y Tran ]) Formula 19;

[0109] Where [,] indicates splicing and merging, Conv 1×1 (·) represents a 1×1 convolution, and y represents the output of LAT. Adding LAT to the deepest layer of DMS-Net and stacking this module twice can extract long-range dependencies and integrate global contextual semantic information and local detail information to achieve multi-level feature representation.

[0110] The retinal vessel segmentation model was trained using images from the training set. The loss function used during training was the cross-entropy loss function, as shown in Equation 20:

[0111]

[0112] Where N is the total number of pixels in the image, p i g represents the foreground prediction probability of input pixel i. i is the true label for pixel i, which is 1 (vessel) or 0 (non-vessel) in the retinal vessel segmentation task.

[0113] The trained model can be validated using validation set images. Based on the current model's segmentation performance on the validation set, the model's hyperparameters are tuned. Through multiple adjustments to the hyperparameters, the model's performance is optimized, thus completing the model training.

[0114] After training the retinal vessel segmentation model, the model is tested using test set images. The segmentation effect is evaluated based on evaluation metrics, which include: accuracy (ACC), sensitivity (SE), specificity (SP), and F1-score. The formulas for each metric are shown in formulas 21-26.

[0115]

[0116]

[0117]

[0118]

[0119]

[0120]

[0121] In this algorithm, TP represents a true positive, meaning the actual target is a positive sample, and the algorithm classifies it as a positive sample as well. TN represents a true negative, meaning the actual sample is a negative sample, and the algorithm classifies it as a negative sample as well. FP represents a false positive, meaning the actual sample is negative, but the algorithm incorrectly classifies it as a positive sample. FN represents a false negative, meaning the actual sample is positive, but the algorithm incorrectly classifies it as a negative sample. Precision is the accuracy rate, and Recall is the recall rate. All the above metrics range from 0 to 1; the higher the value, the better the segmentation performance of the algorithm. Accuracy is represented by the percentage of correctly classified pixels out of the total number of pixels in the image. This reflects the overall performance of the algorithm; sensitivity, also known as the true positive rate, represents the percentage of correctly classified blood vessel pixels to real blood vessel pixels, reflecting the algorithm's ability to detect blood vessel pixels; specificity represents the percentage of correctly classified background pixels in the image to the number of real background pixels segmented by experts, reflecting the network's ability to discriminate background information; F1-score comprehensively considers precision and recall. Ideally, both precision and recall should be high, but generally, high precision leads to low recall, and vice versa. Therefore, this embodiment uses a new metric, F1-score, which is a harmonic value that comprehensively considers precision and recall.

[0122] This invention extracts the green channel from fundus image data, performs normalization, adaptive contrast enhancement, gamma correction, reduces noise and illumination effects, and improves contrast. It also enhances and segments the data to obtain a larger training dataset, improving recognition accuracy. A hybrid attention convolution module is designed to replace the ordinary convolution module for feature extraction, simultaneously extracting local details and global correlations, resulting in stronger feature extraction capabilities and avoiding overfitting, thus improving the model's segmentation performance and generalization ability. A multi-scale input module is designed to enable the model to extract richer multi-scale semantic information, improving segmentation performance. Finally, a local information-aware Transformer module is designed to capture long-range dependencies and global contextual semantic information, while also extracting local details through convolutional structures, achieving multi-level and efficient feature representation, thereby improving the model's segmentation performance and resulting in a better segmentation model.

[0123] System Implementation Examples

[0124] According to embodiments of the present invention, a retinal vessel segmentation system based on long-range dependency and multi-scale input is provided. Figure 7A schematic diagram illustrating the composition of a retinal vessel segmentation system based on long-range dependencies and multi-scale inputs, provided for one or more embodiments of this specification, is shown below. Figure 7 As shown, the retinal vessel segmentation system according to an embodiment of the present invention specifically includes:

[0125] Data processing module 70: used to preprocess the acquired fundus image dataset, reduce image noise, enhance image contrast, divide the dataset into training set, validation set and test set, and perform data augmentation processing on the training set and validation set;

[0126] Model building module 72: used to build a retinal vessel segmentation network, design a hybrid attention convolution module to extract features from the training set images, design a multi-scale input module to obtain multi-scale semantic information, build a Local Information Aware Transformer module (LAT), and add the LAT to the retinal vessel segmentation network;

[0127] Model training module 74: used to input training set image data into the retinal vessel segmentation network for model training, evaluate model performance through validation set images, adjust model parameters according to evaluation results, and obtain retinal vessel segmentation model;

[0128] Segmentation module 76: Used to preprocess the fundus image to be segmented and then input it into the retinal segmentation model for segmentation to obtain a retinal vessel segmentation image.

[0129] The embodiments of the present invention are system embodiments corresponding to the above method embodiments. The specific operation of each module can be understood by referring to the description of the method embodiments, and will not be repeated here.

[0130] Device Example 1

[0131] This invention provides an electronic device, such as... Figure 8 As shown, it includes: a memory 80, a processor 82, and a computer program stored in the memory 80 and executable on the processor 82. When the computer program is executed by the processor 82, it performs the following method steps:

[0132] S1. Preprocess the acquired fundus image dataset to reduce image noise and enhance image contrast. Divide the dataset into a training set, a validation set, and a test set. Perform data augmentation processing on the training set and the validation set.

[0133] S2. Construct a retinal vessel segmentation network, design a hybrid attention convolution module to extract features from the training set images, design a multi-scale input module to obtain multi-scale semantic information, construct a Local Information Aware Transformer (LAT) module, and add the LAT to the retinal vessel segmentation network.

[0134] S3. Input the training set image data into the retinal vessel segmentation network for model training, evaluate the model performance using the validation set images, adjust the model parameters according to the evaluation results, and obtain the retinal vessel segmentation model.

[0135] S4. After preprocessing the fundus image to be segmented, input it into the retinal segmentation model for segmentation to obtain a retinal vessel segmentation image.

[0136] Device Example 2

[0137] This invention provides a computer-readable storage medium storing an information transmission implementation program. When executed by a processor 82, the program performs the following method steps:

[0138] S1. Preprocess the acquired fundus image dataset to reduce image noise and enhance image contrast. Divide the dataset into a training set, a validation set, and a test set. Perform data augmentation processing on the training set and the validation set.

[0139] S2. Construct a retinal vessel segmentation network, design a hybrid attention convolution module to extract features from the training set images, design a multi-scale input module to obtain multi-scale semantic information, construct a Local Information Aware Transformer (LAT) module, and add the LAT to the retinal vessel segmentation network.

[0140] S3. Input the training set image data into the retinal vessel segmentation network for model training, evaluate the model performance using the validation set images, adjust the model parameters according to the evaluation results, and obtain the retinal vessel segmentation model.

[0141] S4. After preprocessing the fundus image to be segmented, input it into the retinal segmentation model for segmentation to obtain a retinal vessel segmentation image.

[0142] The computer-readable storage media described in this embodiment include, but are not limited to, ROM, RAM, disk, or optical disk.

[0143] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for retinal vessel segmentation based on long-range dependencies and multiscale input, characterized by, include: The acquired fundus image dataset is preprocessed to reduce image noise and enhance image contrast. The dataset is divided into a training set, a validation set, and a test set. Data augmentation processing is performed on the training set and the validation set. The preprocessing of the acquired fundus image dataset to reduce image noise and enhance image contrast specifically includes: The dataset images are converted to grayscale, and the G channel in the RGB images is extracted. The grayscale conversion formula is shown in Formula 1: Formula 1 ; wherein represents the G channel in the RGB image; The image is standardized by scaling its overall grayscale values. The standardization formula is shown in Formula 2: Formula 2; wherein, is a pixel value of the image, is a normalized pixel value of the image, is a mean value of the pixel values of the image, is a standard deviation of the pixel values of the image; Adaptive histogram equalization is applied to the image to stretch the grayscale histogram of the original image, improve image contrast, and limit the extremely dark and bright parts of the image. Gamma correction is applied to the image to suppress uneven illumination and central line reflection in the fundus image; Finally, the fundus image data was normalized, and the gray values ​​of the fundus images were normalized from 0 to 255 to between 0 and 1. A retinal vessel segmentation network is constructed, a hybrid attention convolution module is designed to extract features from the training set images, a multi-scale input module is designed to obtain multi-scale semantic information, a Local Information Aware Transformer (LAT) module is constructed, and the LAT is added to the retinal vessel segmentation network. The local information-aware Transformer module includes a Transformer structure and a convolutional structure; The Transformer structure is used to serialize the input feature map into an image, linearly mapping the two-dimensional image data into one-dimensional serialized data. Spatial location encoding is added to the serialized data to learn the position information of pixels. Layer normalization is used to maintain the stability of the data feature distribution and accelerate network convergence. The output of the convolutional structure and the output of the Transformer structure are concatenated and merged in the channel dimension, and 1×1 convolution is used to reduce the channel dimension to obtain the LAT output. The convolutional structure mainly consists of two 3×3 convolutions and two GELU activation functions; The training set image data is input into the retinal vessel segmentation network for model training. The model performance is evaluated using validation set images. The model parameters are adjusted based on the evaluation results to obtain the retinal vessel segmentation model. The fundus image to be segmented is preprocessed and then input into the retinal vessel segmentation model for segmentation to obtain a retinal vessel segmentation image.

2. The method of claim 1, wherein, The training and validation sets undergo data augmentation processing, specifically including: The training set and validation set are randomly cropped to form 64×64 image blocks. The cropped images are randomly flipped horizontally, randomly flipped vertically, and randomly rotated to obtain the augmented training and validation sets.

3. The method of claim 1, wherein, The hybrid attention convolution module includes 3×3 convolution, batch normalization, ReLU activation function, DropBlock, residual connection and hybrid attention mechanism; The mixed attention mechanism includes a channel attention mechanism and a position attention mechanism, the channel attention mechanism is used to capture the correlation between different channel features from the perspective of channels, and each channel of the feature map is given a weight; The position attention mechanism is used to improve the ability of the network to obtain remote context information based on accurate position information, and capture the dependency between different position pixels.

4. The method of claim 1, wherein, The design of the multi-scale input module is to obtain multi-scale semantic information, which specifically includes: The multi-scale input module is used to perform multiple maximum pooling operations on the input image to construct an image pyramid, splice and fuse input pictures and feature maps of different scales, reduce information loss, and extract more rich multi-scale semantic information.

5. The method of claim 1, wherein, The method further includes testing the retinal blood vessel segmentation model using test set pictures, and evaluating the blood vessel segmentation effect according to a preset evaluation index, wherein the evaluation index includes accuracy ACC, sensitivity SE, specificity SP and F1-score.

6. A retinal blood vessel segmentation system based on long-range dependencies and multi-scale inputs, characterized in that, It includes: A data processing module is configured to preprocess an acquired fundus image dataset, reduce image noise, enhance image contrast, divide the dataset into a training set, a validation set and a test set, and perform data augmentation processing on the training set and the validation set; The preprocessing of the acquired fundus image dataset, the reduction of image noise and the enhancement of image contrast specifically include: The dataset images are subjected to grayscale processing, and the G channel in the RGB image is extracted, the grayscale formula being as shown in formula 1: Formula 1 ; wherein represents the G channel in the RGB image; The images are subjected to standardization operation, and the overall gray value of the image is scaled, the standardization formula being as shown in formula 2: Official 2; wherein, is the pixel value of the image, is the normalized image pixel value, is the mean of the image pixel values, is the standard deviation of the image pixel values; The images are subjected to adaptive histogram equalization processing, the gray histogram of the original image is stretched, the image contrast is improved, and the extremely dark and extremely bright parts in the image are limited; The images are subjected to gamma correction processing to suppress the uneven illumination and center line reflection phenomenon in the fundus image; Finally, the fundus image data is subjected to normalization processing, and the gray value of the fundus image is normalized from 0 to 255 to 0 to 1; A model construction module is configured to construct a retinal blood vessel segmentation network, design a mixed attention convolution module, extract features from the training set images, design a multi-scale input module, obtain multi-scale semantic information, construct a local information perception Transformer module LAT, and add the LAT to the retinal blood vessel segmentation network; The local information perception Transformer module includes a Transformer structure and a convolution structure; The Transformer structure is used to serialize the input feature map, linearly map the two-dimensional image data into one-dimensional serialized data, add spatial position coding to the serialized data, learn the position information of the pixels, and use layer normalization to keep the stability of the data feature distribution and accelerate the network convergence; The convolution structure part output and the Transformer structure part output are spliced and fused in the channel dimension, and 1×1 convolution is used for channel dimension reduction to obtain the output of the LAT. The convolution structure mainly includes two 3×3 convolutions and two GELU activation functions. The model training module is configured to input training set image data into the retinal blood vessel segmentation network for model training, evaluate the model performance by using verification set images, adjust the model parameters according to the evaluation result, and obtain a retinal blood vessel segmentation model. The segmentation module is configured to input a fundus image to be segmented into the retinal blood vessel segmentation model after preprocessing to obtain a retinal blood vessel segmentation image.

7. An electronic device, comprising: Comprise: a processor; and a memory arranged to store computer-executable instructions that, when executed, cause the processor to implement the steps of the method for retinal blood vessel segmentation based on long-range dependencies and multi-scale input according to any one of claims 1-5.

8. A storage medium, characterized by a computer program product for storing computer-executable instructions that, when executed, implement the steps of the method for retinal blood vessel segmentation based on long-range dependencies and multi-scale input according to any one of claims 1-5.

Images