A retinal vessel segmentation method based on a multi-level full-resolution feature selection network

Abstract

The present invention discloses a retinal blood vessel segmentation method based on a multi-level full-resolution feature selection network. The method first obtains a retinal data set and divides it into a training set and a test set. Secondly, the retinal image data in the retinal data set is preprocessed, and the preprocessed training set is subjected to data expansion and data enhancement. Then, a multi-scale context interaction mechanism is added on the basis of the encoder-decoder network, and a multi-level full-resolution feature selection network is constructed on this basis to output the retinal blood vessel segmentation result. Finally, a binary cross entropy loss function with a logistics function is constructed, and training is performed on the training set. The network weights obtained by training are applied to the test set, and an evaluation index is constructed to observe the segmentation results. The present invention makes the segmentation results more robust, improves the segmentation ability of fine blood vessels and low-contrast blood vessels, and makes the segmentation results more complete and accurate.

Description

Technical Field

[0001] The present invention belongs to the field of medical image processing, and in particular relates to a retinal blood vessel segmentation method based on a multi-level full-resolution feature selection network. Background Art

[0002] Several eye diseases, such as diabetic retinopathy, hypertensive retinopathy, and retinal vascular occlusion, can cause various changes in retinal blood vessels. In the early stages of these conditions, patients may experience symptoms such as microaneurysms, edema, leakage, and hyperplasia of the retinal blood vessels. Early eye examinations, particularly fundus examinations and retinal imaging, can effectively prevent and treat these diseases by observing changes in retinal blood vessel morphology, preventing further deterioration and even blindness. However, due to the increasing number of patients with eye diseases and a limited number of ophthalmologists, relying solely on the experience of ophthalmologists to observe fundus images is time-consuming and labor-intensive, leading to a shortage of hospital resources. Furthermore, retinal blood vessels have low contrast, are numerous and complex, and are significantly affected by the imaging light source. Even professional manual annotation technicians can miss some vessels. Therefore, automatic blood vessel segmentation has become a research hotspot in computer-assisted medical image diagnosis.

[0003] In the early stages of research, unsupervised strategies for retinal vessel segmentation were predominant. These methods employed different filtering and statistical models tailored to the morphological characteristics of retinal vessels to remove noise and extract vascular information. However, these methods were significantly affected by fundus image noise and poorly captured the correlation around blood vessels, resulting in low robustness in practical clinical applications.

[0004] With the advancement of computer hardware performance, deep learning has gained widespread recognition in the field of medical imaging due to its superior contextual feature extraction and representation learning capabilities. Retinal vessel segmentation technology based on convolutional neural networks has become a research hotspot. Different neural network architectures have been designed to address the characteristics of retinal vessels. The most popular is the U-Net network. This network, with its symmetrical encoder-decoder architecture and skip connections, has demonstrated impressive results not only in retinal vessel segmentation but also in a wide range of medical image analysis tasks. During the encoding phase, high-level feature representations are obtained through continuous convolution and downsampling. Convolutional layers and upsampling are then used to restore the original resolution. Skip connections are used to combine original resolution features with upsampled features for mapping and pixel-level supervised learning. By learning mappings at different scales, the U-Net network demonstrates significant advantages in semantic analysis. However, for retinal vessel segmentation, some unresolved issues remain. For example, the segmentation effect of the subtle parts of the retinal blood vessel edges and low-contrast areas is poor. This is mainly because the continuous downsampling in the encoding stage leads to the loss of blood vessel edge details, and the network lacks attention to the correlation between blood vessel pixels, resulting in the blood vessels in low-contrast areas often being treated as noise or background, thereby ignoring the blood vessels in low-contrast areas and affecting the segmentation ability. Summary of the Invention

[0005] To address existing retinal vessel segmentation challenges, this paper proposes a retinal vessel segmentation method based on a multi-level, full-resolution feature selection network. By incorporating a recurrent denoising and feature selection mechanism and a complete edge detail enhancement module on the full-resolution output, this method fully exploits vascular features and constructs long-range relationships between vessel pixels, improving segmentation accuracy. Testing on the internationally available CHASE_DB1 retinal vessel segmentation dataset demonstrates that the proposed method effectively addresses these challenges and reaches state-of-the-art performance in this field.

[0006] In order to achieve the above object, the present invention comprises the following steps:

[0007] Step 1: Obtain the retinal dataset and divide it into training set and test set.

[0008] Step 2: Preprocess the retinal image data in the retina dataset.

[0009] Step 3: Perform data expansion and data enhancement on the preprocessed training set.

[0010] Step 4: Add a multi-scale contextual interaction mechanism on the basis of the encoder-decoder network to obtain a 5-layer full-resolution output, and build a multi-level full-resolution feature selection network based on this to output the retinal vessel segmentation results.

[0011] Step 5: Construct a binary cross entropy loss function with a logistics function, train it on the training set, apply the network weights trained on the training set to the test set, and construct an evaluation index to observe the segmentation results.

[0012] In step 2, the color retinal vascular image in the retinal dataset is grayscaled. The pixel grayscale value is obtained by multiplying the RGB channels by 0.299, 0.587, and 0.114, respectively, and then adding them together. The entire image is grayscaled. The resulting grayscale image is then normalized to obtain a retinal dataset with a mean of 0 and a standard deviation of 1.

[0013] In step 3, the training set is expanded by sliding an n×n patch of image blocks over the original retinal images in the training set with a step size of s to generate new retinal images. During the data augmentation phase, the expanded training set is randomly flipped horizontally, flipped vertically, and rotated, with a probability of γ set to enhance the diversity of the training set.

[0014] In step 4, the multi-level full-resolution feature selection network consists of three parts: a full-resolution feature extraction network, a cyclic denoising and feature selection mechanism, and a complete edge detail enhancement module. After passing through the full-resolution feature extraction network, a multi-level full-resolution output is obtained, and then the cyclic denoising and feature selection mechanism and the complete edge detail enhancement module are respectively performed on them. First, a multi-scale context interaction mechanism is added to the U-Net network to replace the original jump connection. The U-Net network performs horizontal and vertical feature learning through upsampling, downsampling, double-layer convolution, etc., and uses a multi-scale context interaction mechanism to fuse features of adjacent different levels. For the full-resolution output of the first column, the first half of the full-resolution output is selected to perform the cyclic denoising and feature selection mechanism to obtain the vascular trunk segmentation result. For the second half of the full-resolution output, the complete edge detail enhancement module is executed to obtain the vascular fine segmentation result. Finally, the vascular trunk segmentation result and the vascular fine segmentation result are added to obtain the output result. For ease of expression, define F i,j is the internal feature of the U-Net network, where i and j are defined as the rows and columns of the U-Net network respectively.

[0015] The full-resolution feature extraction network structure is as follows:

[0016] The grayscale image block processed in the above steps is input into the full-resolution feature extraction network, and the number of channels is adjusted by performing a convolution operation on it to obtain the first feature F in the first row. 0,0 , and then perform double-layer convolution blocks and downsampling operations on the feature to obtain two hidden layer features of different scales in the second row F 0,1 ,F 1,1, where the feature scale obtained by the double-layer convolution block remains unchanged, and the size of the feature space obtained by downsampling is halved. 0,1 Perform double-layer convolution and downsampling operations (due to F 0,1 It belongs to the first column full resolution feature, so no upsampling is performed (same below) and Two temporary mapping features, where the superscript C represents the temporary mapping feature after double-layer convolution, and the superscript d represents the temporary mapping feature generated by downsampling. 1,1 Perform upsampling, double-layer convolution and downsampling operations to generate and Three temporary mapping features, the superscript u represents the temporary mapping features generated by upsampling, and then the temporary mapping features with the same subscript are subjected to multi-scale context multi-receptive field aggregation operation to fuse the context multi-scale information to obtain the corresponding mapping features. For temporary mapping features without the same subscript, they are directly used as mapping features, thus obtaining the 3 mapping features F in the third row 0,2 , F 1,2 , F 2,2 Then repeat the same operation as the previous row for the features in the third row to obtain the corresponding temporary mapping features. Except for the full-resolution layer features in the first column, all other features are upsampled, and double-layer convolution and downsampling are used to generate temporary mapping features. Then perform multi-scale context multi-receptive field aggregation operation on the temporary mapping features with the same subscript, and the temporary mapping features without the same subscript are directly used as hidden layer nodes, thus obtaining the 4 features F in the fourth row. 0,3 , F 1,3 , F 2,3 , F 3,3 The same operation is performed on the subsequent row features until they reach the bottleneck layer after being downsampled by the previous layer features, that is, the deepest feature F of the network. n,n .

[0017] F n,n The characteristic of the row is F 0,n …F n,n , perform double-layer convolution and downsampling on the full-resolution features of the first column of the row, perform only upsampling on the deepest column features, and perform upsampling, double-layer convolution and downsampling on the remaining column features to obtain the corresponding temporary mapping features. Then, according to the same subscript of the temporary mapping features, perform multi-scale context multi-receptive field aggregation to obtain the features of the next row, i.e. F 0,n+1 …F n-1,n+1 Similar to the previous row, the first column of full-resolution features F 0,n+1 Perform double-layer convolution and downsampling to the deepest column feature F n-1,n+1Only upsampling is performed. The remaining columns of features undergo upsampling, double-layer convolution, and downsampling to obtain corresponding temporary mapping features. These features are then fused according to the subscript to obtain the next column of features. This constructs a complete full-resolution feature extraction network. Finally, a full-resolution feature output is obtained in the last row.

[0018] The multi-scale context multi-receptive field aggregation module is as follows:

[0019] The features with the same subscript are fused after dilated convolution, and then the features of different convolution stages are aggregated to obtain the multi-receptive field capture feature effect, so as to obtain the mapping feature of the position corresponding to the subscript. The specific fusion after dilated convolution is:

[0020]

[0021] in is the temporary mapping feature generated by the downsampling operation, is the temporary mapping feature generated by the upsampling operation, It is a temporary mapping feature generated by the double-layer convolution block operation. Indicates element-wise addition operation, DW r represents a depthwise separable dilated convolution with dilation rate r.

[0022] Get multi-scale fusion F i,j+1 After that, it is subjected to multi-receptive field aggregation, which is specifically expressed as follows:

[0023]

[0024]

[0025]

[0026]

[0027] Conv 3 Represents a convolution with a kernel size of 3×3, ReLU represents the activation function, BN represents batch normalization, Conv 1 represents a convolution with a kernel size of 1×1, and ⊙ represents a cascade operation. Represents features with different receptive fields.

[0028] A cyclic denoising and feature selection mechanism is constructed for the full-resolution output of the first half of the layers. The full-resolution features of the first half are first concatenated and subjected to global average pooling to obtain the weights for each channel. The weights are then sorted from high to low, with the last K weights set to 0, while the rest remain unchanged. The weights between the channel attentions are then reconstructed through a fully connected layer. The resulting attention weights are then assigned element-wise to the concatenated features and summed with the residual blocks constructed in the shallow layers of the first half to obtain the backbone features. This operation is repeated R times, removing the lowest-weighted attention channel and updating the output backbone features, thereby isolating the main vessels and reducing background noise.

[0029] A complete edge detail enhancement module is constructed for the full-resolution output of the second half of the layers. A depthwise separable convolution with a dilation rate of 4 is first performed to expand the connection between each vessel pixel and surrounding pixels, and the outputs are summed. A global spatial attention mechanism is then used to determine the spatial weights of the enhanced details. This weight is multiplied by the corresponding level of resolution features to produce an edge-enhanced output map. Finally, the edge-enhanced output maps are concatenated to produce a deep-level enhanced output, capturing more edge details. Finally, the segmentation result map is obtained by aggregating the backbone features and the enhanced features.

[0030] The main advantages of the retinal vessel segmentation method based on the multi-level full-resolution feature selection network proposed in this invention are as follows:

[0031] 1. The present invention only uses data expansion and data enhancement methods to increase the diversity and breadth of the training set, making the training results more robust.

[0032] 2. The present invention introduces a multi-scale contextual multi-receptive field aggregation module to fuse three different scale features, effectively alleviating the loss of edge details during the downsampling process of ordinary networks and enhancing the mapping ability of overall vascular features in the network. The depthwise separable convolution used reduces the number of model parameters and reduces the risk of overfitting.

[0033] 3. The shallow loop denoising and feature selection mechanism and complete edge detail enhancement module constructed for the five-layer full-resolution output in this invention enhance the processing capabilities of full-resolution features, while removing noise and enhancing the ability to locate major blood vessels. In addition, it also improves the ability to segment fine and low-contrast blood vessels, making the segmentation results more complete and accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 This is the overall process of the present invention;

[0035] Figure 2 Schematic diagram of the multi-level full-resolution feature selection network structure;

[0036] Figure 3Schematic diagram of the multi-scale context multi-receptive field aggregation module;

[0037] Figure 4 This is the result image of retinal blood vessel segmentation achieved by the present invention;

[0038] Figure 5 A comparison chart of the results of different algorithms on the retina dataset. DETAILED DESCRIPTION

[0039] The following specific implementation is shown in conjunction with the accompanying drawings. A retinal blood vessel segmentation method based on a multi-level full-resolution feature selection network is proposed. Based on the UNet network, a multi-scale context multi-receptive field aggregation module is introduced to construct a 5-layer full-resolution output. Based on the 5-layer full-resolution output, a shallow loop denoising and feature selection mechanism and a complete edge detail enhancement module are used to extract more complete blood vessels from the image. The overall process is as follows: Figure 1 shown.

[0040] Step 1: Download the CHASE_DB1 dataset from its official website. This dataset contains 28 retinal images. Use the first 20 as the training set and the last 8 as the test set. Use the first expert's manual annotations as the baseline for our training and as the standard for subsequent supervised model training.

[0041] Step 2: Convert the original color retinal vascular image to grayscale by multiplying the RGB channels by 0.299, 0.587, and 0.114, respectively, and then adding the values ​​to obtain the pixel grayscale value. Finally, convert the entire image to grayscale. The resulting grayscale image is then normalized to obtain a dataset with a mean of 0 and a standard deviation of 1.

[0042] Step 3: Use a template with a patch size of 48×48 to slide on the retinal image training set obtained in step 2 with a step size of 6. The image blocks obtained by sliding are used as a new dataset. A large number of 48×48 images are obtained as a new training set. The obtained training set is subjected to the following data augmentation with a probability of 0.5. If the random number is greater than 0.5, the following data augmentation is performed; otherwise, no augmentation is performed:

[0043] (a) Horizontal flip: flip the image along the vertical center axis.

[0044] (b) Vertical flip: flip the image along the horizontal center axis.

[0045] (c) 90° random rotation: All pixels in the image are randomly rotated by 90°, 180°, and 270° with the image center as the origin.

[0046] For the test set obtained in step 2, no data expansion and enhancement are performed, and the image size is padded to a multiple of 8 in both height and width using a padding operation.

[0047] Step 4: Based on the original encoder-decoder network, the present invention adds a multi-scale context multi-receptive field aggregation module to obtain a 5-layer full-resolution output using a multi-scale interaction mechanism, and builds a multi-level full-resolution feature selection network based on this to output the retinal vessel segmentation results. The network can be generally divided into three main stages: full-resolution feature extraction, main vessel separation, and fine vessel refinement. Figure 2 The details of each stage are as follows:

[0048] 4.1 Full-resolution feature extraction:

[0049] This stage includes a total of 7 layers of structure, each layer structure is as follows:

[0050] First layer: First, pass through the input layer. In this layer, the 48×48×1 grayscale image block obtained in step 3 is input into the network, and the 48×48×32 output F is obtained through the double-layer convolution block. 0,0 , where the double-layer convolution block contains two identical convolution blocks, and one convolution block consists of a convolution layer, a batch normalization layer, a Dropout layer, and a LeakReLU activation function. The convolution layer uses a convolution kernel of 3×3 and a stride of 1, the Dropout layer uses Dropout2d with a dropout rate of 20%, and the activation function ReLU has a negative slope of 0.1.

[0051] Second layer: Then after two levels of hidden layers, the F in the first layer 0,0 The features are subjected to double-layer convolution blocks and downsampling operations to obtain two hidden layer features of different scales in the second layer F 0,1 ,F 1,1 , which contains a convolution layer, batch normalization and LeakReLU activation function in the downsampling module, where the convolution layer parameter convolution kernel is 2, the step size is 2, the input feature size is 48×48×32, and the output feature F 1,1 is 24×24×64. The F obtained by the double-layer convolution block 0,1 Keep 48×48×32 unchanged.

[0052] The third layer: F 0,1 Perform double-layer convolution and downsampling operations (due to F 0,1 It is row 0, so no upsampling is performed, the same below) and Two temporary mapping features, where the downsampling module and the double-layer convolution block are similar to the second layer, are obtained. The size is 48×48×32, d(F 1,2) is 24×24×64, for F 1,1 Perform double-layer convolution, downsampling and upsampling operations to generate and Three temporary mapping features, where the space and channel size remain unchanged after the double convolution layer The size is kept at 24×24×64, and the downsampling is obtained The size is 12×12×128, obtained by upsampling The size is restored to 48×48×32, and then the temporary mapping features with the same subscript are aggregated by multi-scale context and multi-receptive field to fuse the context multi-scale information to obtain the corresponding mapping feature F 0,2 and F 1,2 For temporary mapping features that do not have the same subscript, they are directly used as mapping features F 2,2 , thus obtaining the second column of 3 mapping features F 0,2 , F 1,2 , F 2,2 .

[0053] Fourth layer: Repeat the same operation as the previous layer on the nodes of the third layer to obtain the temporary mapping features of the fourth layer, and 0,2 Perform double convolution and downsampling to obtain and in The size is 48×48×32, The size is 24×24×64, for F 1,2 Perform upsampling, double convolution and downsampling to obtain and Three temporary mapping features, with feature sizes of 48×48×32, 24×24×64, and 12×12×128, for F 2,2 Perform upsampling, double convolution and downsampling to obtain and Three temporary mapping features, the feature sizes are 24×24×64, 12×12×128, 6×6×256, and then the temporary mapping features with the same subscript are aggregated by multi-scale context and multi-receptive field to obtain F 0,3 , F 1,3 , F 2,3 Mapping features, temporary mapping features without the same subscript are directly used as mapping features (F 3,3 ), thus obtaining the fourth layer 4 mapping features F 0,3 , F 1,3 , F 2,3 , F 3,3 , where F 3,3 is the deepest scale feature, which is then only upsampled.

[0054] Fifth layer: Do the same as for the fourth layer, 0,3 Perform double convolution and downsampling to obtain and The feature sizes are 48×48×32 and 24×24×64 respectively. 1,3 Perform upsampling, double-layer convolution and downsampling to obtain and Three temporary mapping features, with feature sizes of 48×48×32, 24×24×64, and 12×12×128, for F 2,3 Perform upsampling and double-layer convolution to obtain Two temporary mapping features, with sizes of 24×24×64 and 12×12×128, for F 3,3 Only upsampling is performed, resulting in The size is 12×12×128. Then, a multi-scale context multi-receptive field aggregation operation is performed on the temporary mapping features with the same subscript to obtain the fifth layer 3 mapping features F 0,4 , F 1,4 , F 2,4 .

[0055] Sixth layer: Then do the same operation on the fifth layer features, and 0,4 Perform double-layer convolution kernel downsampling to obtain and The feature sizes are 48×48×32 and 24×24×64 respectively. 1,4 Perform upsampling and double-layer convolution to obtain Two temporary mapping features, with sizes of 48×48×32 and 24×24×64, for F 2,4 Perform upsampling to obtain The size is 24×24×64, and then the temporary mapping features with the same subscript are aggregated in a multi-scale context and multi-receptive field to obtain the 5th column and 2 nodes F 0,5 , F 1,5 .

[0056] Seventh layer: Repeat the above operation for the sixth layer features, and 0,5 Perform double convolution to get The size is 48×48×32, for F 1,5 Perform upsampling to obtain The size is 48×48×32, and two temporary mapping features are obtained. Then, the temporary mapping features with the same subscript are aggregated by multi-scale context and multi-receptive field to obtain the seventh layer 1 node F 0,6 .

[0057] This can get the full resolution output of 5-level fusion (F 0,2 , F 0,3, F 0,4 F 0,5 , F 0,6 ).like Figure 3 As shown in the figure, the specific formula of the multi-scale context multi-receptive field aggregation module in the multi-scale context interaction mechanism is as follows:

[0058]

[0059] in is the temporary mapping feature generated by the downsampling operation, is the temporary mapping feature generated by the upsampling operation, It is a temporary mapping feature generated by the double-layer convolution block operation. Indicates element-wise addition operation, DW r represents a depthwise separable dilated convolution with dilation rate r.

[0060] After obtaining multi-scale fusion, multi-receptive field aggregation is performed on it, which is specifically expressed as follows:

[0061]

[0062]

[0063]

[0064]

[0065] Conv 3 Represents a convolution with a kernel size of 3×3, ReLU represents the activation function, BN represents batch normalization, Conv 1 represents a convolution with a kernel size of 1×1, and ⊙ represents a cascade operation. Represents features with different receptive fields.

[0066] 4.2 Main vessel separation

[0067] After the multi-scale context interaction mechanism, 5 layers of full-resolution output are obtained. For the first three layers of shallow output, a shallow loop denoising and feature selection mechanism is constructed. The specific implementation is as follows:

[0068] 4.2.1 First adjust the output of the shallow three layers and cascade them to obtain the shallow three layer output graph

[0069] F shallow =Conv 1 (F 0,2 )⊙Conv 1 (F 0,3 )⊙Conv 1 (F 0,4 )

[0070] Among them, F 0,2 F 0,3 F 0,4 They are the full-resolution convolution outputs of the first three layers, ⊙ represents the cascade operation, Conv 1 () indicates a convolution with a convolution kernel of 1×1. The shallow output layer F is obtained by adjusting the number of channels and cascading. shallow .

[0071] 4.2.2 Secondly, through cyclic denoising and feature selection mechanism RDFSM, important channels in the shallow output graph are selected, and the channels with the lowest weight (K) are inactivated. Then, the fully connected layer is used to connect the relationships between channels in each layer to obtain the selected feature weights.

[0072] W ds =FC(St(GAP(F shallow )) K )

[0073]

[0074] F=(F1,F2...,F 2C )

[0075] Among them, GAP(·) represents the global average pooling in the channel dimension, for F with dimensions of H×W×3C shallow Global average pooling obtains the channel weight of 1×1×3C, St(F) K Indicates the denoising and selection of the feature weight F, sets the smallest 3C×K weight values ​​in the feature weight F to 0, and keeps the others unchanged. FC() represents the fully connected layer. The weight obtained after selection is passed through the fully connected layer with half the channel, the ReLU activation function, the fully connected layer that restores the original channel, and the Sigmoid activation function to obtain the selected weight W ds .

[0076] 4.2.3 Multiply the obtained weights by the shallow output graph element by element and then add the residuals to obtain the denoising and feature selection graph, update the original shallow output graph and cycle the RDFSM mechanism to obtain the final output graph

[0077]

[0078] F shallow =F ds

[0079] F ds =RDFSM(F shallow ) i ,for i∈R

[0080] F shallow_out =Conv 1(F ds )

[0081] Where W ds is the denoising and selection weight obtained in the previous step, To multiply by elements, the denoising and feature selection maps are updated with the original F shallow , denoising and feature selection mechanism is executed cyclically, and the number of cycles R is set to 2 in the experiment until the end of Conv 1 () The convolution kernel is 1×1 and the convolution adjustment channel is used to obtain the output F shallow_out .

[0082] 4.3 Extraction of fine blood vessels

[0083] A complete edge detail enhancement module is constructed for the full-resolution output of the fourth and fifth layers. A depth-wise separable dilated convolution with a dilation rate of 4 is used to enhance the long-range representation of vascular pixels. The two layers of output images are fused to obtain spatial enhancement attention weights. The fused spatial attention weights are used to enhance the details in the output images of the last two layers. Finally, the feature maps with enhanced details are aggregated and passed through a double convolution layer to obtain a refined output image. The specific formula is as follows:

[0084]

[0085] W g =(Sigmoid(BN(Conv 1 (W_l+W_h))))

[0086]

[0087]

[0088] F deep =DCB(F_l⊙F_h)

[0089] Among them F 0,5 , F 0,6 The full-resolution output images of the fourth and fifth layers are subjected to depth-separable dilated convolution on the output images of the upper and lower layers. Dilation represents the expansion rate. The W_l and W_h features of the enhanced long-distance representation are obtained through the ReLU() activation function, and then the convolution kernel is 1×1. 1 (), batch normalization BN() and activation function Sigmoid() get the spatial enhanced attention weights, and multiply them by elements to F respectively 0,5 , F 0,6 In the figure, the deep output image F is obtained by adjusting the cascaded upper and lower layer enhancement images through the double-layer convolution block. deep .

[0090] 4.4 Result output: Add the shallow layer output to the deep layer output, and adjust the channel through 1×1 convolution to obtain a probability map with the output channel as 1.

[0091] F out =Conv1(F deep +F shallow_out )

[0092] Among them, Conv 1 () indicates the convolution with kernel size of 1, F shallow_out represents the shallow full-resolution output, F deep This indicates the output after deep micro-vessel extraction.

[0093] Step 5: Construct a binary cross entropy loss function with a logistics function to supervise network training.

[0094] Step 6: Train on the training set obtained in step 3, apply the obtained training weights to the test set, and construct evaluation indicators to observe the network output results.

[0095] During the training process, we use the training set obtained in step 3, and the batch_size of each training is set to 128. The entire experiment is carried out in the pytorch framework, using the Adam algorithm as the optimizer and setting the initial learning rate to 1e -4 , the decay rate is 1e -5 . We performed a total of 50 epochs of training and set the cosine annealing algorithm with a period of 50 as our strategy for adjusting the learning rate. The experiments were all conducted on a single GeForce RTX3080Ti. Finally, the weight file obtained on the training set was saved to obtain a trained retinal vascular segmentation model. We then loaded the test set obtained in step 3 into the trained retinal vascular segmentation model for segmentation. In order to reflect the segmentation performance of the proposed network in several different data sets, we used accuracy (Acc), specificity (Spe), sensitivity (Sen), F1 score (F1) and the area under the receiving characteristic (AUC) to measure the relationship between the segmentation results and the true value. The definitions of ACC, Spe, Sen and F1 are as follows:

[0096]

[0097]

[0098]

[0099]

[0100] Here, true positives (TP) represent the number of correctly segmented blood vessel pixels; true negatives (TN) represent the number of correctly segmented non-vessel pixels; false positives (FP) represent the number of incorrectly segmented blood vessel pixels; and false negatives (FN) represent the number of incorrectly segmented non-vessel pixels. The ROC curve uses the false positive rate and true positive rate as the horizontal axis, reflecting the changes in the false positive rate and true positive rate at different thresholds.

[0101] Finally, the results of the method proposed in this invention on the CHASE_DB1 dataset are shown in the figure below: Figure 4 As shown in the figure, the test results achieved an accuracy of 97.39%, a sensitivity of 88.43%, a specificity of 98%, an AUC of 99.10%, and an F1 score of 81.1%, reaching the state-of-the-art level. Figure 5 As shown in the figure, the proposed method shows more powerful segmentation capabilities on both DRIVE and CHASE_DB1 datasets.

Claims

1. A retinal vessel segmentation method based on a multi-level full-resolution feature selection network, characterized in that: The steps include: Step 1: Obtain the retinal dataset and divide it into training set and test set; Step 2: Preprocess the retinal image data in the retinal dataset; Step 3: Perform data expansion and data enhancement on the preprocessed training set; Step 4: Add a multi-scale contextual interaction mechanism to the encoder-decoder network, and build a multi-level full-resolution feature selection network based on this to output the retinal vessel segmentation results; The multi-level full-resolution feature selection network consists of three parts: a full-resolution feature extraction network, a cyclic denoising and feature selection mechanism, and a complete edge detail enhancement module. After passing through the full-resolution feature extraction network, a multi-level full-resolution output is obtained, and then the cyclic denoising and feature selection mechanism and the complete edge detail enhancement module are respectively applied to it. First, a multi-scale context interaction mechanism is added to the U-Net network to replace the original skip connection. The U-Net network performs horizontal and vertical feature learning and uses the multi-scale context interaction mechanism to fuse features of adjacent different levels. Define F i,j is the internal feature of the U-Net network, where i and j are defined as the rows and columns of the U-Net network respectively; The specific implementation process of the full-resolution feature extraction network structure is as follows: The processed grayscale image block is input into the full-resolution feature extraction network, and the number of channels is adjusted by performing a convolution operation on it to obtain the first feature F in the first row. 0,0 , and then perform double-layer convolution blocks and downsampling operations on the feature to obtain two hidden layer features of different scales in the second row F 0,1 ,F 1,1 , where the feature scale obtained by the double-layer convolution block remains unchanged, and the size of the feature space obtained by downsampling is halved; for F 0,1 Perform double-layer convolution and downsampling operations to generate and Two temporary mapping features, where the superscript C represents the temporary mapping feature after double-layer convolution, and the superscript d represents the temporary mapping feature generated by downsampling. 1,1 Perform upsampling, double-layer convolution and downsampling operations to generate and Three temporary mapping features, the superscript u represents the temporary mapping features generated by upsampling, and then the temporary mapping features with the same subscript are subjected to multi-scale context multi-receptive field aggregation operation to fuse the context multi-scale information to obtain the corresponding mapping features. For temporary mapping features without the same subscript, they are directly used as mapping features, thus obtaining the 3 mapping features F in the third row. 0,2 , F 1,2 , F 2,2 Then repeat the same operation on the features of the third row as on the previous row, and do the same operation on the features of subsequent rows until the bottleneck layer is reached after the features of the previous layer are downsampled, that is, the features of the deepest layer of the network F n,n ; F n,n The characteristic of the row is F 0,n …F n,n , perform double-layer convolution and downsampling on the first column of full-resolution features of the row, and perform double-layer convolution and downsampling on the deepest column feature F n,n Only upsampling is performed, and the remaining column features are subjected to upsampling, double-layer convolution and downsampling to obtain the corresponding temporary mapping features. Then, multi-scale context multi-receptive field aggregation is performed according to the same subscript of the temporary mapping feature to obtain the features of the next row, i.e. F 0,n+1 …F n-1,n+1 Repeat the previous operation to build a complete full-resolution feature extraction network, and finally get a full-resolution feature output in the last row; Step 5: Construct a binary cross entropy loss function with a logistics function, train it on the training set, apply the trained network weights to the test set, and construct an evaluation index to observe the segmentation results.

2. The retinal blood vessel segmentation method using a multi-level full-resolution feature selection network according to claim 1, characterized in that: The preprocessing in step 2 is as follows: grayscale the color retinal vascular image in the retinal dataset by multiplying the RGB channels by 0.299, 0.587, and 0.114 respectively and then adding the pixel grayscale values ​​to grayscale the entire image; The obtained grayscale image is then standardized to obtain a retinal dataset with a mean of 0 and a standard deviation of 1.

3. The retinal blood vessel segmentation method using a multi-level full-resolution feature selection network according to claim 1, characterized in that: The data expansion in step 3 is specifically as follows: first, a new retinal image is obtained by sliding an image block with a patch size of n×n on the original retinal image in the training set with a step size of s; The data enhancement specifically includes randomly performing horizontal flipping, vertical flipping and random rotation on the expanded training set, and the above random probabilities are all set to γ.

4. The retinal blood vessel segmentation method using a multi-level full-resolution feature selection network according to claim 3, characterized in that: The specific implementation process of the cyclic denoising and feature selection mechanism and the complete edge detail enhancement module is as follows: For the full-resolution output of the first half of the layer, a cyclic denoising and feature selection mechanism is constructed. First, the full-resolution features of the first half are spliced ​​together and subjected to global average pooling to obtain the weight of each channel. The weights are sorted from high to low, and the last K weights are set to 0, while the rest remain unchanged. The weights between the channel attentions are then reconstructed through the fully connected layer. The obtained attention weights are then assigned element-wise to the spliced ​​features and added to the residual blocks constructed in the shallow layer of the first half to obtain the backbone features. The above operation is repeated R times to remove the attention channel with the lowest weight and update the output backbone features. For the full-resolution output of the second half of the layer, a complete edge detail enhancement module is constructed. First, depthwise separable convolutions with a dilation rate of 4 are performed on them and added together. Then, the spatial weights of the enhanced details are obtained through the global spatial attention mechanism. This weight is multiplied with the full-resolution features of the corresponding level to obtain the edge enhancement output map. Finally, the edge enhancement output map is spliced ​​to obtain the deep enhancement output. Finally, the segmentation result map is obtained by aggregating the backbone features and the enhanced features.

5. The retinal blood vessel segmentation method using a multi-level full-resolution feature selection network according to claim 4, characterized in that: The specific implementation process of the multi-scale context multi-receptive field aggregation module is as follows: The features with the same subscript are fused after dilated convolution, and then the mapping features of the position corresponding to the subscript are obtained by aggregating the features of different convolution stages; The specific fusion after dilation convolution is: in is the temporary mapping feature generated by the downsampling operation, is the temporary mapping feature generated by the upsampling operation, It is a temporary mapping feature generated by the double-layer convolution block operation. Indicates element-wise addition operation, DW r represents a depth-wise separable dilated convolution with a dilation rate of r; Get multi-scale fusion F i,j+1 After that, it is subjected to multi-receptive field aggregation, which is specifically expressed as follows: Conv 3 Represents a convolution with a kernel size of 3×3, ReLU represents the activation function, BN represents batch normalization, Conv 1 represents a convolution with a kernel size of 1×1, and ⊙ represents a cascade operation.

Images