An eye fundus retinal blood vessel segmentation method, system and electronic device

Abstract

The application discloses an eye fundus retinal blood vessel segmentation method and system and electronic equipment, and relates to the technical field of computer vision image processing. The application obtains an eye fundus color image of a tester as a test eye fundus color image; the test eye fundus color image is input into an eye fundus retinal blood vessel segmentation model to obtain an eye fundus retinal blood vessel segmentation image of the tester; and the eye fundus retinal blood vessel segmentation model is obtained by training an initial eye fundus retinal blood vessel segmentation model by using multiple historical eye fundus color images of different testers. The application can improve the eye fundus retinal blood vessel segmentation precision by constructing an initial eye fundus retinal blood vessel segmentation model comprising a prediction fusion module and a U-shaped symmetrical structure with an encoder and a decoder and training the initial eye fundus retinal blood vessel segmentation model.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision image processing, in particular to an eye fundus retinal vessel segmentation method, system and electronic equipment. BACKGROUND

[0002] The structure and characteristics of the retinal vessels in the fundus image are closely related to diseases such as diabetic retinopathy and hypertension. Therefore, the research on the segmentation of the retinal vessels in the fundus image has clinical significance and practical value. However, the fundus image is easily affected by uneven illumination during imaging, resulting in a small contrast difference between the retinal vessels and the fundus background in the fundus image. Secondly, the collected fundus image usually has certain noise pixels. In addition, the structure of the retinal vessels is usually complex, and it is time-consuming and laborious to manually realize the segmentation of the retinal vessels in the fundus image, and the result often deviates from the true result, which makes it difficult and challenging to accurately segment the retinal vessels manually. Therefore, it is of great significance to accurately segment the retinal vessels by machine.

[0003] The existing methods for segmenting the retinal vessels by machine mainly include non-deep learning methods and deep learning methods. The non-deep learning methods usually need to rely on a large amount of prior knowledge and additional preprocessing strategies to extract the features of the retinal vessels, and the segmentation effect cannot be guaranteed for the fundus image containing noise. In the deep learning method, the mainstream method is the retinal vessel segmentation method based on U-Net, although this method effectively alleviates the interference of noise pixels in the fundus image by focusing on the overall correlation of the retinal vessels, but it ignores the local correlation between the vessels, resulting in that part of the fine vessel features in the fundus image cannot be accurately segmented, and the accurate segmentation of the retinal vessels cannot be realized. Therefore, how to focus on the overall correlation of the retinal vessels while focusing on the local correlation between the vessels to improve the segmentation accuracy of the retinal vessels is a key problem to be solved in the field. SUMMARY

[0004] The purpose of the present application is to provide an eye fundus retinal vessel segmentation method, system and electronic equipment, which can improve the segmentation accuracy of the eye fundus retinal vessels.

[0005] To achieve the above-mentioned purpose, the present application provides the following solutions:

[0006] An eye fundus retinal vessel segmentation method, comprising:

[0007] An eye fundus color image of a tester is acquired as a test eye fundus color image;

[0008] inputting the test fundus color image into a fundus retinal blood vessel segmentation model to obtain a fundus retinal blood vessel segmentation image of the tester, wherein the fundus retinal blood vessel segmentation model is obtained by training an initial fundus retinal blood vessel segmentation model using a plurality of historical fundus color images of different testers.

[0009] Optionally, the initial fundus retinal blood vessel segmentation model comprises a prediction fusion module and a U-shaped symmetric structure, and the U-shaped symmetric structure is nested with an encoder and a decoder.

[0010] The U-shaped symmetric structure comprises a first full convolution module layer, a second effective feature fusion module layer, a third full convolution module layer, a fourth residual U-shaped module layer, a fifth residual U-shaped module layer, a sixth residual U-shaped module layer and a seventh residual U-shaped module layer which are sequentially connected from bottom to top.

[0011] The fourth residual U-shaped module layer, the fifth residual U-shaped module layer, the sixth residual U-shaped module layer and the seventh residual U-shaped module layer are combined with a coordinate attention mechanism.

[0012] The first full convolution module layer comprises a first full convolution module.

[0013] The second effective feature fusion module layer comprises a second effective feature fusion module.

[0014] The third full convolution module layer comprises three first full convolution modules and three second full convolution modules.

[0015] The fourth residual U-shaped module layer comprises four first residual U-shaped modules and four second residual U-shaped modules.

[0016] The fifth residual U-shaped module layer comprises five first residual U-shaped modules and five second residual U-shaped modules.

[0017] The sixth residual U-shaped module layer comprises six first residual U-shaped modules and six second residual U-shaped modules.

[0018] The seventh residual U-shaped module layer comprises seven first residual U-shaped modules and seven second residual U-shaped modules.

[0019] The seven first residual U-shaped modules, the six first residual U-shaped modules, the five first residual U-shaped modules, the four first residual U-shaped modules, the three first full convolution modules, the first full convolution module, the second effective feature fusion module, the three second full convolution modules, the four second residual U-shaped modules, the five second residual U-shaped modules, the six second residual U-shaped modules and the seven second residual U-shaped modules.

[0020] An output end of the three-layer first full convolution module is connected with an input end of the two-layer effective feature fusion module.

[0021] A first splicing module is arranged between the three-layer second full convolution module and the four-layer second residual U-shaped module.

[0022] A second splicing module is arranged between the four-layer second full convolution module and the five-layer second residual U-shaped module.

[0023] A third splicing module is arranged between the five-layer second full convolution module and the six-layer second residual U-shaped module.

[0024] A fourth splicing module is arranged between the six-layer second full convolution module and the seven-layer second residual U-shaped module.

[0025] An output end of the four-layer first residual U-shaped module is connected with an input end of the first splicing module.

[0026] An output end of the five-layer first residual U-shaped module is connected with an input end of the second splicing module.

[0027] An output end of the six-layer first residual U-shaped module is connected with an input end of the third splicing module.

[0028] An output end of the seven-layer first residual U-shaped module is connected with an input end of the fourth splicing module.

[0029] The one-layer full convolution module, the three-layer second full convolution module, the four-layer second residual U-shaped module, the five-layer second residual U-shaped module, the six-layer second residual U-shaped module and the seven-layer second residual U-shaped module are connected with the prediction fusion module.

[0030] The input quantity of the seven-layer first residual U-shaped module is a test fundus color image or a historical fundus color image.

[0031] The output quantity of the prediction fusion module is a test fundus retinal blood vessel segmentation image or a historical fundus retinal blood vessel segmentation image.

[0032] Optionally, the seven-layer first residual U-shaped module, the six-layer first residual U-shaped module, the five-layer first residual U-shaped module, the four-layer first residual U-shaped module, the four-layer second residual U-shaped module, the five-layer second residual U-shaped module, the six-layer second residual U-shaped module and the seven-layer second residual U-shaped module have the same structure.

[0033] Optionally, the seven-layer first residual U-shaped module comprises, in sequence, an input unit, a first convolution kernel, a second convolution kernel, a third convolution kernel, a fourth convolution kernel, a fifth convolution kernel, a sixth convolution kernel, a seventh convolution kernel, an eighth convolution kernel, a coordinate attention mechanism, a ninth convolution kernel, a tenth convolution kernel, an eleventh convolution kernel, a twelfth convolution kernel, a thirteenth convolution kernel, a fourteenth convolution kernel, an addition operation unit, and an output unit.

[0034] An output end of the first convolution kernel is connected with an input end of the addition operation unit.

[0035] An output end of the second convolution kernel is connected with an input end of the fourteenth convolution kernel.

[0036] An output end of the third convolution kernel is connected with an input end of the thirteenth convolution kernel.

[0037] An output end of the fourth convolution kernel is connected with an input end of the twelfth convolution kernel.

[0038] An output end of the fifth convolution kernel is connected with an input end of the eleventh convolution kernel.

[0039] An output end of the sixth convolution kernel is connected with an input end of the tenth convolution kernel.

[0040] An output end of the seventh convolution kernel is connected with an input end of the ninth convolution kernel.

[0041] Optionally, the first convolution kernel, the second convolution kernel, the third convolution kernel, the fourth convolution kernel, the fifth convolution kernel, the sixth convolution kernel, the seventh convolution kernel, the eighth convolution kernel, the ninth convolution kernel, the tenth convolution kernel, the eleventh convolution kernel, the twelfth convolution kernel, the thirteenth convolution kernel, and the fourteenth convolution kernel all have a size of 3x3.

[0042] Optionally, before the eye fundus color image of the tester is acquired as the test eye fundus color image, the method further comprises:

[0043] constructing an initial eye fundus retinal blood vessel segmentation model;

[0044] acquiring a plurality of historical eye fundus color images of different testers;

[0045] performing segmentation processing on each historical eye fundus color image based on the eye fundus retinal blood vessels in the historical eye fundus color image, to obtain a plurality of historical eye fundus retinal blood vessel segmentation images;

[0046] processing the initial eye fundus retinal blood vessel segmentation model, with the historical eye fundus color image as input and the historical eye fundus retinal blood vessel segmentation image as output, to obtain an eye fundus retinal blood vessel segmentation model.

[0047] An eye fundus retinal blood vessel segmentation system comprises:

[0048] An eye fundus color image acquisition module is configured to acquire an eye fundus color image of a testee as a test eye fundus color image.

[0049] An eye fundus retinal blood vessel segmentation module is configured to input the test eye fundus color image into an eye fundus retinal blood vessel segmentation model to obtain an eye fundus retinal blood vessel segmentation image of the testee.

[0050] An electronic device comprises a memory and a processor, the memory is configured to store a computer program, and the processor is configured to run the computer program to make the electronic device execute the method.

[0051] Optionally, the memory is a readable storage medium.

[0052] According to the embodiments of the present application, the following technical effects are provided:

[0053] The eye fundus retinal blood vessel segmentation method, system and electronic device provided by the present application acquire an eye fundus color image of a testee as a test eye fundus color image, input the test eye fundus color image into an eye fundus retinal blood vessel segmentation model to obtain an eye fundus retinal blood vessel segmentation image of the testee, and the eye fundus retinal blood vessel segmentation model is obtained by training an initial eye fundus retinal blood vessel segmentation model using multiple historical eye fundus color images of different testees. The present application can improve the eye fundus retinal blood vessel segmentation precision by constructing and training an initial eye fundus retinal blood vessel segmentation model comprising a prediction fusion module and a U-shaped symmetrical structure with an encoder and a decoder embedded therein. BRIEF DESCRIPTION OF DRAWINGS

[0054] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0055] Figure 1 The eye fundus retinal blood vessel segmentation method flowchart in the embodiment 1 of the present application;

[0056] Figure 2 The initial eye fundus retinal blood vessel segmentation model structure schematic diagram in the embodiment 1 of the present application;

[0057] Figure 3Figure 1 is a schematic diagram of a residual U-shaped module structure in Embodiment 1 of the present application.

[0058] Figure 4 Figure 2 is a schematic diagram of an effective feature fusion module structure in Embodiment 1 of the present application.

[0059] Figure 5 Figure 3 is a schematic diagram of an eye fundus retinal blood vessel segmentation system structure in Embodiment 2 of the present application. DETAILED DESCRIPTION

[0060] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of protection of the present application.

[0061] The purpose of the present application is to provide an eye fundus retinal blood vessel segmentation method, system and electronic device, which can improve the eye fundus retinal blood vessel segmentation accuracy.

[0062] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the present application will be further described in detail below with reference to the drawings and specific embodiments.

[0063] Embodiment 1

[0064] As shown in the figure, the present embodiment provides an eye fundus retinal blood vessel segmentation method, which comprises: Figure 1

[0065] Step 101: obtaining an eye fundus color image of a tester as a test eye fundus color image.

[0066] Step 102: inputting the test eye fundus color image into an eye fundus retinal blood vessel segmentation model to obtain an eye fundus retinal blood vessel segmentation image of the tester. The eye fundus retinal blood vessel segmentation model is obtained by training an initial eye fundus retinal blood vessel segmentation model using multiple historical eye fundus color images of different testers.

[0067] As shown in the figure, the present embodiment provides an eye fundus retinal blood vessel segmentation method, which comprises: Figure 2 ​, the initial fundus retinal blood vessel segmentation model comprises a prediction fusion module and a U-shaped symmetric structure. The U-shaped symmetric structure is nested with an encoder and a decoder. The U-shaped symmetric structure comprises a first full convolution module layer, a second effective feature fusion module layer, a third full convolution module layer, a fourth residual U-shaped module layer, a fifth residual U-shaped module layer, a sixth residual U-shaped module layer and a seventh residual U-shaped module layer which are sequentially connected from bottom to top. The fourth residual U-shaped module layer, the fifth residual U-shaped module layer, the sixth residual U-shaped module layer and the seventh residual U-shaped module layer are all combined with a coordinate attention mechanism. The first full convolution module layer comprises a first full convolution module. The second effective feature fusion module layer comprises a second effective feature fusion module (the structure of which is shown in Figure 4 The third full convolution module layer comprises three first full convolution modules and three second full convolution modules. The fourth residual U-shaped module layer comprises four first residual U-shaped modules and four second residual U-shaped modules. The fifth residual U-shaped module layer comprises five first residual U-shaped modules and five second residual U-shaped modules. The sixth residual U-shaped module layer comprises six first residual U-shaped modules and six second residual U-shaped modules. The seventh residual U-shaped module layer comprises seven first residual U-shaped modules and seven second residual U-shaped modules. Among them, the seven first residual U-shaped modules, the six first residual U-shaped modules, the five first residual U-shaped modules, the four first residual U-shaped modules, the three first full convolution modules, the first full convolution module, the second effective feature fusion module, the three second full convolution modules, the four second residual U-shaped modules, the five second residual U-shaped modules, the six second residual U-shaped modules and the seven second residual U-shaped modules. The output end of the three first full convolution modules is connected with the input end of the second effective feature fusion module. The first splicing module is arranged between the three second full convolution modules and the four second residual U-shaped modules. The second splicing module is arranged between the four second full convolution modules and the five second residual U-shaped modules. The third splicing module is arranged between the five second full convolution modules and the six second residual U-shaped modules. The fourth splicing module is arranged between the six second full convolution modules and the seven second residual U-shaped modules. The output end of the four first residual U-shaped modules is connected with the input end of the first splicing module. The output end of the five first residual U-shaped modules is connected with the input end of the second splicing module. The output end of the six first residual U-shaped modules is connected with the input end of the third splicing module. The output end of the seven first residual U-shaped modules is connected with the input end of the fourth splicing module. The first full convolution module, the three second full convolution modules, the four second residual U-shaped modules, the five second residual U-shaped modules, the six second residual U-shaped modules and the seven second residual U-shaped modules are all connected with the prediction fusion module. The input quantity of the seven first residual U-shaped modules is a test fundus color image or a historical fundus color image. The output quantity of the prediction fusion module is a test fundus retinal blood vessel segmentation image or a historical fundus retinal blood vessel segmentation image.

[0068] Specifically, the structures of the first residual U-shaped module of the seventh layer, the first residual U-shaped module of the sixth layer, the first residual U-shaped module of the fifth layer, the first residual U-shaped module of the fourth layer, the second residual U-shaped module of the fourth layer, the second residual U-shaped module of the fifth layer, the second residual U-shaped module of the sixth layer, and the second residual U-shaped module of the seventh layer are the same.

[0069] like Figure 3 The seven-layer first residual U-shaped module includes, in sequence, an input unit, a first convolutional kernel, a second convolutional kernel, a third convolutional kernel, a fourth convolutional kernel, a fifth convolutional kernel, a sixth convolutional kernel, a seventh convolutional kernel, an eighth convolutional kernel, a coordinate attention mechanism, a ninth convolutional kernel, a tenth convolutional kernel, an eleventh convolutional kernel, a twelfth convolutional kernel, a thirteenth convolutional kernel, a fourteenth convolutional kernel, an addition operation unit, and an output unit. The output of the first convolutional kernel is connected to the input of the addition operation unit. The output of the second convolutional kernel is connected to the input of the fourteenth convolutional kernel. The output of the third convolutional kernel is connected to the input of the thirteenth convolutional kernel. The output of the fourth convolutional kernel is connected to the input of the twelfth convolutional kernel. The output of the fifth convolutional kernel is connected to the input of the eleventh convolutional kernel. The output of the sixth convolutional kernel is connected to the input of the tenth convolutional kernel. The output of the seventh convolutional kernel is connected to the input of the ninth convolutional kernel.

[0070] Specifically, the size of the first, second, third, fourth, fifth, sixth, seventh, eighth, ninth, tenth, eleventh, twelfth, thirteenth and fourteenth convolution kernels is 3×3.

[0071] Before step 101, the following is also included:

[0072] Step 103: Construct an initial retinal vessel segmentation model.

[0073] Step 104: Obtain multiple historical fundus color images from different test subjects.

[0074] Step 105: Based on the retinal vessels in each historical fundus color image, segment each historical fundus color image to obtain multiple historical fundus retinal vessel segmentation images.

[0075] Step 106: Using historical fundus color images as input and historical fundus retinal vessel segmentation images as output, process the initial fundus retinal vessel segmentation model to obtain the fundus retinal vessel segmentation model.

[0076] The method for segmenting retinal vessels in the fundus proposed in this embodiment will be described in detail below:

[0077] Step 1: Image acquisition.

[0078] The fundus color images of patients in different age groups from 20 to 80 years old are collected by using a fundus camera. For example, the DRIVE dataset is collected by using a non-mydriatic 3CCD camera of Canon CR5. After obtaining the fundus color images, the retinal blood vessels in the fundus color images are segmented by medical experts to obtain the corresponding fundus retinal blood vessel images.

[0079] Step 2: data preprocessing.

[0080] Each fundus color image collected and its corresponding retinal blood vessel image are combined into a sample pair to obtain a sample pair set. The sample pair set is divided into a training sample pair set and a test sample pair set.

[0081] Step 3: constructing a fundus retinal blood vessel segmentation model.

[0082] The constructed segmentation model is a symmetric and nested encoder-decoder U-shaped structure, which mainly consists of four parts: residual U-shaped modules with coordinate attention mechanism, full convolution modules, efficient feature fusion modules, and prediction fusion modules.

[0083] Specifically, the constructed symmetric and nested U-shaped structure model is divided into 7 layers from top to bottom. The first 4 layers are composed of 4 groups of residual U-shaped modules with coordinate attention mechanism from deep to shallow and symmetric. The 5th layer is a group of symmetric full convolution modules composed of convolution layers and atrous convolution layers. The 6th layer is an efficient feature fusion module, and the 7th layer is a single full convolution module. The prediction fusion module fuses the prediction results of the 7 different layers.

[0084] Step 4: input the training sample pair set obtained in step 2 into the model in step 3 to train the model. In the segmentation model training, the learning rate of the network is set to 0.005, and the maximum number of iterations is 50.

[0085] Step 4a: the residual U-shaped module as described in step 3 is composed of three parts. Specifically, the first part is composed of a convolution layer with a convolution kernel size of 3x3, which is used to extract local feature information of the retinal blood vessels; the second part is a symmetric U-shaped encoder-decoder structure composed of a convolution layer and a coordinate attention mechanism, which inputs and outputs the feature map extracted by the first part as a whole to obtain the overall correlation information of the retinal blood vessels. Specifically, when the number of layers in the U-shaped module is deeper, larger size feature map information can be extracted to obtain more scale correlation information. Moreover, the coordinate attention mechanism in this part enhances the extraction of feature information in the blood vessel region of the feature map; the third part fuses the local feature information obtained by the first part and the overall correlation information of the blood vessels obtained by the second part through addition operation.

[0086] Step 4b: After the retinal blood vessel image is operated by the residual U-shaped module as described in step 4a, the resolution of the obtained feature map will be reduced, therefore, as described in step 3, the 5th layer and the 7th layer of the segmentation model use a full convolution module composed of convolution layers and atrous convolution layers, replace the convolution layers of the second part described in step 4a with atrous convolution layers, and cancel the use of coordinate attention mechanism to prevent the loss of key information. At this time, different dilution rates are set for different levels of atrous convolution layers, which can still be used to obtain blood vessel feature information of different scales.

[0087] Step 4c: Due to the difference in characteristics of retinal blood vessel features at different levels, low-level features contain more blood vessel edge detail information, but less inter-vascular correlation information, while high-level features contain rich vascular correlation information, but less edge detail information. Therefore, the efficient feature fusion module as described in step 3 is used to fuse blood vessel features at different levels, to strengthen the model's learning of blood vessel edge information while focusing on the correlation between blood vessels.

[0088] Step 4d: The different levels of retinal blood vessel features extracted by the segmentation model are fused by the prediction fusion module as described in step 3, to retain as much detail of the blood vessel features at different levels as possible, and to obtain the final segmentation structure.

[0089] Specifically, the symmetric and nested U-shaped structure model includes:

[0090] 4.1 The residual U-shaped attention module obtains local and global context information, which is divided into three steps. The three steps of the module will be described in detail, and the process is shown in Figure 3 .

[0091] Step 4.1.1: Given the input retinal fundus image P, Conv is a 3x3 normal convolution layer. P is passed through a three-dimensional convolution layer to obtain the output result X. This process can be represented as shown in (1):

[0092] X = Conv(P) (1)

[0093] Step 4.1.2: X output in step one is used as input. Two spatial range pooling kernels (H, 1) and (1, W) are used to encode each channel along the horizontal and vertical coordinates to obtain results y h (h) and y w (w). y h (h) is a feature map with a height of h in the horizontal direction, and y w (w) is a feature map with a width of w in the vertical direction. This process can be represented as shown in (2) and (3):

[0094]

[0095]

[0096] After the above y h (h) and y w (w) are concatenated and F1 operation (dimension reduction using 1x1 convolution kernel) is performed to generate feature map f. The process can be represented as shown in (4):

[0097] f∈δ(F1([y h ·y w ])) (4)

[0098] Where · represents the connection operation in the spatial dimension, and δ(·) represents the activation operation.

[0099] After that, the above obtained f is divided into vertical and horizontal directions along the spatial dimension, and the encoding information in the horizontal and vertical directions is f h and f l respectively. Then, dimension elevation operation is performed using 1x1 convolution kernel, and sigmoid activation function δ is combined to obtain attention weight g h in the horizontal direction and g w in the vertical direction. The process can be represented as shown in (5) and (6):

[0100] g h =δ(F h (f h )) (5)

[0101] g w =δ(F w (f w )) (6)

[0102] After obtaining the attention weights g h and g w in the two directions, the obtained attention weights g h and g w are multiplied by the feature map X to calculate the final weighted feature map U(X). The process can be represented as shown in (7):

[0103] U(X)=X·g h ·g w (7)

[0104] Where · represents the dot product between vectors.

[0105] Step 4.1.3: Fuse X obtained in step one and U(X) obtained in step two by addition operation to obtain the result F(X). The process can be represented as shown in (8):

[0106] F(X) = X + U(X) (8)

[0107] 4.2 The full convolution module obtains blood vessel feature information of different scales

[0108] Step 4.2.1: The feature map F(X) obtained by the residual U-shaped attention module is input into the full convolution module of the fifth layer to obtain a feature map F l (low-level feature map). RSUF is a full convolution operation. The process can be represented as shown in (9):

[0109] F l (X) = RSUF(F(X)) (9)

[0110] Step 4.2.2: The feature map F l (X) obtained in step one is input into the full convolution module of the seventh layer to obtain a feature map F h (high-level feature map), and the process can be represented as shown in (10):

[0111] F h (X) = RSUF(F l (X)) (10)

[0112] 4.3 Processing and weighting of high-level feature maps and low-level feature maps

[0113] The high-level feature maps and the low-level feature maps are processed and weighted through efficient feature fusion, and the process is as shown in (11). Figure 4

[0114] The high-level feature map F h is upsampled to the same size as the low-level feature map F l using upsampling Upsample (bilinear interpolation method), and the upsampled feature map F h is concatenated with F l for a splicing operation Concat, and the fused feature map is input into a 1x1 convolution layer for processing; then the weight a is calculated using global average pooling, batch normalization and sigmoid function, and then Fh and Fl are multiplied by the weight to obtain the weighted feature map. Finally, the weighted feature maps are cascaded and fused to obtain the final output result F out . The process can be represented as shown in (11):

[0115] F out = Concat(Upsample(F h ))*a + Fl*1-a (11)

[0116] 4.4 Fusion of retinal blood vessel features to obtain the final segmentation result​

[0117] The channels of the feature maps in each layer of the model are compressed and upsampled to correspond to the size of the input feature maps. All feature maps are then concatenated and fused, and their channels are compressed to 1 using a 3×3 convolutional layer. Based on this, the segmentation result is obtained.

[0118] Step 5: Using the segmentation model trained in Step 4, input the set of test sample pairs obtained in Step 3 into the trained segmentation model to obtain the retinal vessel image segmented by the model.

[0119] This embodiment constructs a fundus retinal image segmentation model comprising four modules: a residual U-shaped module, a fully convolutional module, an efficient feature fusion module, and a prediction fusion module. After training the fundus retinal image segmentation model, the trained model is used to validate the sample dataset to obtain prediction results. Finally, all prediction results are fused and restored to obtain an accurately segmented fundus retinal vessel image. This method can effectively and accurately segment fundus retinal vessel images, solving the problem of inaccurate segmentation results in existing technologies.

[0120] Example 2

[0121] In order to perform the method corresponding to Embodiment 1 above and achieve the corresponding functions and technical effects, a fundus retinal vessel segmentation system is provided below, including:

[0122] The test fundus color image acquisition module is used to acquire the test subject's fundus color image as the test fundus color image.

[0123] The fundus retinal vessel segmentation module is used to input the test fundus color image into the fundus retinal vessel segmentation model to obtain the segmented fundus retinal vessel image of the test subject. The fundus retinal vessel segmentation model is obtained by training an initial fundus retinal vessel segmentation model using multiple historical fundus color images from different test subjects.

[0124] like Figure 5 The retinal vessel segmentation system provided in this embodiment includes:

[0125] The image acquisition module M1 is used to acquire a color image of the fundus retina and a fundus retinal vessel image, wherein the fundus retinal vessel image is an image containing the retinal vessels separated from the fundus retinal color image.

[0126] The sample pair dataset acquisition module M2 is used to establish a sample pair dataset for the fundus retinal color image and the fundus retinal blood vessel image.

[0127] The image segmentation model establishing module M3 is configured to construct an image segmentation model of fundus retinal blood vessels; the image segmentation model of fundus retinal blood vessels comprises four modules, i.e., a residual U-shaped module with a coordinate attention mechanism, a full convolution module, an efficient feature fusion module and a prediction fusion module.

[0128] The image segmentation model training and verification module M4 is configured to train and verify the image segmentation model of fundus retinal blood vessels by using a sample pair data set composed of the fundus color image and the corresponding image of retinal blood vessels, and input corresponding prediction results.

[0129] The image restoration module M5 is configured to fuse the prediction results to obtain a classification result of the fundus retinal image, and then perform image restoration on the classification result to obtain a final image of fundus retinal blood vessels.

[0130] Embodiment 3

[0131] The embodiment provides an electronic device, comprising a memory and a processor, the memory is configured to store a computer program, and the processor is configured to run the computer program to enable the electronic device to execute the method in embodiment 1. The memory is a readable storage medium.

[0132] In the specification, each embodiment is described in a progressive manner, and each embodiment focuses on the difference from other embodiments, and the same or similar parts of each embodiment can be referred to each other. For the system disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the related parts can be referred to the method part.

[0133] The principles and implementation manners of the present application are described by using specific examples in the present application, and the above embodiment is only used to help understand the method of the present application and its core idea; at the same time, for those skilled in the art, according to the idea of the present application, the specific implementation manner and application range will be changed. In conclusion, the content of the specification should not be understood as a limitation of the present application.

Claims

1. A method for segmenting retinal vessels in the fundus, characterized in that, include: Obtain the color image of the fundus of the test subject as the test fundus color image; The test fundus color image is input into the fundus retinal vessel segmentation model to obtain the fundus retinal vessel segmentation image of the test subject; the fundus retinal vessel segmentation model is obtained by training an initial fundus retinal vessel segmentation model using multiple historical fundus color images of different test subjects. The initial fundus retinal vessel segmentation model includes a prediction fusion module and a U-shaped symmetrical structure; the U-shaped symmetrical structure is nested with an encoder and a decoder; The U-shaped symmetrical structure includes, from bottom to top, a fully convolutional module layer, a second effective feature fusion module layer, a third fully convolutional module layer, a fourth residual U-shaped module layer, a fifth residual U-shaped module layer, a sixth residual U-shaped module layer, and a seventh residual U-shaped module layer, which are connected sequentially. The four-layer residual U-shaped module layer, the five-layer residual U-shaped module layer, the six-layer residual U-shaped module layer, and the seven-layer residual U-shaped module layer are all combined with a coordinate attention mechanism; The layer of fully convolutional modules includes a layer of fully convolutional modules; The two-layer effective feature fusion module layer includes a two-layer effective feature fusion module; The three-layer fully convolutional module layer includes three layers of first fully convolutional modules and three layers of second fully convolutional modules; The four-layer residual U-shaped module layer includes four layers of first residual U-shaped modules and four layers of second residual U-shaped modules; The five-layer residual U-shaped module layer includes five layers of first residual U-shaped modules and five layers of second residual U-shaped modules; The six-layer residual U-shaped module layer includes six layers of first residual U-shaped modules and six layers of second residual U-shaped modules; The seven-layer residual U-shaped module layer includes seven layers of first residual U-shaped modules and seven layers of second residual U-shaped modules; The seven-layer first residual U-shaped module, the six-layer first residual U-shaped module, the five-layer first residual U-shaped module, the four-layer first residual U-shaped module, the three-layer first fully convolutional module, the one-layer fully convolutional module, the two-layer effective feature fusion module, the three-layer second fully convolutional module, the four-layer second residual U-shaped module, the five-layer second residual U-shaped module, the six-layer second residual U-shaped module, and the seven-layer second residual U-shaped module are connected in sequence. The output of the first fully convolutional module of the three layers is connected to the input of the effective feature fusion module of the two layers; A first splicing module is provided between the third-layer second fully convolutional module and the fourth-layer second residual U-shaped module; A second splicing module is provided between the fourth-layer second fully convolutional module and the fifth-layer second residual U-shaped module; A third splicing module is provided between the fifth-layer second fully convolutional module and the sixth-layer second residual U-shaped module; A fourth splicing module is provided between the sixth-layer second fully convolutional module and the seventh-layer second residual U-shaped module; The output end of the four-layer first residual U-shaped module is connected to the input end of the first splicing module; The output of the first residual U-shaped module of the five layers is connected to the input of the second splicing module; The output of the first residual U-shaped module of the sixth layer is connected to the input of the third splicing module; The output end of the first residual U-shaped module of the seventh layer is connected to the input end of the fourth splicing module; The first-layer fully convolutional module, the third-layer second fully convolutional module, the fourth-layer second residual U-shaped module, the fifth-layer second residual U-shaped module, the sixth-layer second residual U-shaped module, and the seventh-layer second residual U-shaped module are all connected to the prediction fusion module; The input to the first residual U-shaped module of the seventh layer is either a test fundus color image or a historical fundus color image; The output of the prediction fusion module is either a test fundus retinal vessel segmentation image or a historical fundus retinal vessel segmentation image. The seven-layer first residual U-shaped module, the six-layer first residual U-shaped module, the five-layer first residual U-shaped module, the four-layer first residual U-shaped module, the four-layer second residual U-shaped module, the five-layer second residual U-shaped module, the six-layer second residual U-shaped module, and the seven-layer second residual U-shaped module have the same structure; The seven-layer first residual U-shaped module includes, in sequence, an input unit, a first convolution kernel, a second convolution kernel, a third convolution kernel, a fourth convolution kernel, a fifth convolution kernel, a sixth convolution kernel, a seventh convolution kernel, an eighth convolution kernel, a coordinate attention mechanism, a ninth convolution kernel, a tenth convolution kernel, an eleventh convolution kernel, a twelfth convolution kernel, a thirteenth convolution kernel, a fourteenth convolution kernel, an addition operation unit, and an output unit; The output of the first convolution kernel is connected to the input of the addition operation unit; The output of the second convolution kernel is connected to the input of the fourteenth convolution kernel; The output of the third convolution kernel is connected to the input of the thirteenth convolution kernel; The output of the fourth convolution kernel is connected to the input of the twelfth convolution kernel; The output of the fifth convolution kernel is connected to the input of the eleventh convolution kernel; The output of the sixth convolution kernel is connected to the input of the tenth convolution kernel; The output of the seventh convolution kernel is connected to the input of the ninth convolution kernel.

2. The method for segmenting retinal vessels according to claim 1, characterized in that, The size of the first convolution kernel, the second convolution kernel, the third convolution kernel, the fourth convolution kernel, the fifth convolution kernel, the sixth convolution kernel, the seventh convolution kernel, the eighth convolution kernel, the ninth convolution kernel, the tenth convolution kernel, the eleventh convolution kernel, the twelfth convolution kernel, the thirteenth convolution kernel, and the fourteenth convolution kernel is 3×3.

3. The method for segmenting retinal vessels according to claim 1, characterized in that, Before obtaining the color image of the test subject's fundus as the test fundus color image, the following steps are also included: Construct an initial retinal vessel segmentation model; Obtain multiple historical color fundus images from different test subjects; Based on the retinal vessels in each historical fundus color image, each historical fundus color image is segmented to obtain multiple historical fundus retinal vessel segmentation images. Using historical fundus color images as input and historical fundus retinal vessel segmentation images as output, the initial fundus retinal vessel segmentation model is processed to obtain a fundus retinal vessel segmentation model.

4. A retinal vessel segmentation system, characterized in that, The fundus retinal vessel segmentation system uses the fundus retinal vessel segmentation method as described in any one of claims 1-3, and the fundus retinal vessel segmentation system comprises: The test fundus color image acquisition module is used to acquire the test fundus color image of the test subject as the test fundus color image. The fundus retinal vessel segmentation module is used to input the test fundus color image into the fundus retinal vessel segmentation model to obtain the fundus retinal vessel segmentation image of the test subject; the fundus retinal vessel segmentation model is obtained by training an initial fundus retinal vessel segmentation model using multiple historical fundus color images of different test subjects.

5. An electronic device, characterized in that, The device includes a memory and a processor, the memory being used to store a computer program, and the processor running the computer program to cause the electronic device to perform the method of any one of claims 1 to 3.

6. An electronic device according to claim 5, characterized in that, The memory is a readable storage medium.

Images