An Automatic Segmentation Method for Fundus Vessels Based on Low-Cost Noise Data

Abstract

The present invention provides a method for automatic segmentation of fundus blood vessels based on low-cost noise data, including: S1: screening the data set, obtaining retinal images and pixel-level blood vessel labels through publicly available data sets; S2: constructing a convolutional neural network to generate quantifiable noise data; S3: segmenting the data set to construct a training set and a test set, and at the same time retaining one clean image data in the training set and applying corresponding noise to other images; S4: setting a loss function and training parameters; S5: alternately inputting the clean image data and the noise data into the network, and updating the network parameters by using a reweighting function; S6: obtaining a retinal image and inputting it into the trained network model to output a segmentation result. The present invention realizes that only a small amount of clean data can correct the bias brought by incorrect noise to the deep model, and improves the accuracy of automatic segmentation of fundus blood vessels when facing low-cost noise data.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision and image processing, and particularly relates to a method for automatically segmenting fundus blood vessels based on low-cost noisy data. Background Art

[0002] Segmenting fundus blood vessel images to obtain the morphology and structure of fundus blood vessels for analysis has important significance in the medical field, and it also has important application prospects in biomedical image analysis and clinical aspects. Medical image segmentation technology also affects the research and development of other related technologies (such as 3D reconstruction, visualization, etc.) in medical image processing.

[0003] Deep learning algorithms are currently the most commonly used methods for automatic medical image processing. Convolutional neural networks (CNNs) are a method for feature extraction in deep learning algorithms in an end-to-end manner, and have been widely used not only in object detection and semantic segmentation, but also in the medical field. As a data-driven segmentation method, the accuracy of deep models is largely limited by the amount of data. However, medical images contain a huge amount of information, which requires extremely high professional levels and experience judgments of annotators. At the same time, pixel-level annotation of regions of interest is a time-consuming and costly task, which leads to an excessive cost of pixel-level label annotation and a limited database size.

[0004] In recent years, the development of crowdsourcing platforms has reduced the annotation cost of some medical images. By outsourcing the segmentation task to non-professional experienced people or medical students, a large amount of relatively cheap manual annotation information can be obtained. Although such labels cannot be fully trusted, they still greatly reduce the annotation cost of medical images. However, considering that retinal blood vessels are dense and complex and are easily affected by the subjective factors of segmenters, the crowdsourced data usually carries a large amount of incorrect noise, which in turn misleads the learning tendency of deep networks.

[0005] An effective way to solve the above problems is to correct the incorrect learning tendency of the network by means of a small amount of trustworthy labels. By increasing the discrimination of training data and increasing the penalty for incorrect information, meta-learning can effectively solve the learning bias problem caused by the increase in noise. Therefore, using meta-learning to correct the learning bias, fully balancing a small amount of clean data and a large amount of noisy data, obtaining accurate and reliable results, solving various complex problems encountered in noisy learning, and making medical diagnosis more scientific, automated, and accurate will also become the main trend in the development of medical image processing.

[0006] Most of the current methods for learning from noisy data focus on classification tasks and only reweigh the losses at the sample level. However, such methods are not applicable to pixel-level object segmentation tasks. At the same time, in the existing technology, the simulation of noise mostly uses traditional dilation and erosion, and such methods cannot reflect the noise situation of real topological objects. Summary of the Invention

[0007] To solve the above problems, the present invention provides a fundus vessel automatic segmentation method based on low-cost noisy data. This method is based on meta-learning and reweighs the losses of noisy data through a fully connected layer, imposing a greater penalty on the errors brought by noise, so as to prevent the model from tilting towards the wrong noise carried by a large amount of data. At the same time, the consideration of losses is no longer limited to the sample level, but specifically to the single-branch losses of blood vessels, enabling the model to perform better in blood vessel segmentation based on noisy data. For the learning bias problem of deep networks caused by relatively low-cost noisy data with annotations, a small amount of clean data is used to correct the network noise bias, while emphasizing the topological structure features of blood vessels, realizing that only a small amount of clean data can correct the bias brought by wrong noise to the deep model.

[0008] The present invention provides a fundus vessel automatic segmentation method based on low-cost noisy data, and the specific technical solutions are as follows:

[0009] S1: Screen the dataset, and obtain retinal images and pixel-level vessel labels through a publicly available dataset;

[0010] S2: Construct a convolutional neural network, and generate quantifiable noisy data according to the obtained retinal images and pixel-level vessel labels, including noisy data in multi-vessel patterns and noisy data in few-vessel patterns;

[0011] S3: According to the noisy data, segment the dataset to construct a training set and a test set. At the same time, retain one clean image data in the training set and impose corresponding noise on other images;

[0012] S4: Set the loss function and training parameters;

[0013] S5: Alternately input the clean image data and the noisy data into the network, and update the network parameters through the losses obtained from the input of the noisy data and the clean data to obtain an updated network model;

[0014] S6: Input the training set data and the test set data into the network model for training and testing, and input the retinal images into the trained network model to output the segmentation results.

[0015] Further, the specific process of generating the noisy data in multi-vessel patterns is as follows:

[0016] A201: Input the retinal image into a convolutional neural network to overfit the blood vessels in the image;

[0017] A202: Decompose the foreground blood vessels in the overfitting result to obtain multiple branches, and quantify them according to the quantity ratio.

[0018] Further, generate the noise data of the multi-vessel pattern, and the specific process is as follows:

[0019] B201: Iteratively erode the retinal image to generate the blood vessel trunk;

[0020] B202: Decompose the part other than the blood vessel trunk into multiple branches, sort the branches by width, and remove the blood vessel branches according to the quantity ratio to obtain the noise data of the few-vessel pattern.

[0021] Further, in step S3, when splitting the dataset, split the dataset into several equal parts.

[0022] Further, the convolutional neural network model includes an encoding path composed of four encoder units and a decoding path composed of four decoder units, and the encoding path to the decoding path is a skip connection structure;

[0023] The skip connection is to connect the corresponding high-resolution features in the encoding path to the upsampled features, and then fuse the two parts of features through convolution.

[0024] Further, each encoder unit is composed of two 3×3 convolutional layers, a ReLU activation layer, a batch normalization layer, and a max pooling layer.

[0025] Further, the skip connection uses a direct connection in the channel dimension.

[0026] Further, the loss function is specifically as follows:

[0027]

[0028] Among them, Ω represents the pixel space of a single branch, {x p ,y p} {p∈Ω} represents the p-th pixel in the Ω space, and F is the segmentation network.

[0029] Further, in step S5, the specific process of updating the network parameters is as follows:

[0030] Input the noise data into the segmentation network, and the loss obtained through the segmentation network is re-weighted by a re-weighting function to update the segmentation network;

[0031] Input the clean data into the segmentation network to obtain a loss update reweighting function;

[0032] After the re-update weighting function is updated, input the noisy data into the segmentation network, and finally update the parameters of the segmentation network according to the obtained loss.

[0033] The beneficial effects of the present invention are as follows:

[0034] The present invention applies an additional reallocation to the loss obtained from the noisy data through a fully connected layer, and the parameters of the fully connected layer are updated with a small amount of clean data, realizing the correction of the noise learning bias of the deep model only by using a small amount of clean data, reducing the training cost of the medical image segmentation model; at the same time, the processing of the noise loss takes into account the vascular branch level and utilizes the topological features of the vascular structure, improving the performance of automatic retinal vessel segmentation. Description of the Drawings

[0035] Figure 1 is a schematic diagram of the overall process of the present invention;

[0036] Figure 2 is a schematic diagram of the convolutional neural network structure of the present invention;

[0037] Figure 3 is a schematic diagram of the reweighting function structure of the present invention;

[0038] Figure 4 is a schematic diagram of the noise learning working framework based on meta-learning of the present invention. Detailed Embodiments

[0039] In the following description, the technical solutions in the embodiments of the present invention are clearly and completely described. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0040] Embodiment 1

[0041] Embodiment 1 of the present invention discloses a method for automatic segmentation of fundus blood vessels based on low-cost noisy data, as Figure 1 shown, the specific step process is as follows:

[0042] S1: Screen the data set, and obtain retinal images and pixel-level blood vessel labels through the publicly available data set;

[0043] In this embodiment, according to the retinal images and pixel-level blood vessel labels of the publicly available data sets DRIVE and STARE, data interfaces required for building a convolutional neural network are written at the same time.

[0044] Among them, DRIVE is a color fundus image library established by the Niemeijer team in 2004 based on the diabetic retinopathy screening work in the Netherlands; its images were taken from 453 different individuals aged 25 to 90. In this embodiment, 40 of them were randomly selected, among which 7 had early diabetic retinopathy and 33 had no diabetic retinopathy. The pixel of each image is 565×584;

[0045] STARE is a project initiated by Michael Goldbaum in 1975, which was first cited and publicly available in a paper by Hoover et al. in 2000. It is a color fundus image database for retinal vessel segmentation, including 20 fundus images, among which 10 have lesions and 10 have no lesions. The image resolution is 605×700, and each image is the result of manual segmentation by 2 corresponding experts. It is one of the most commonly used standard libraries of fundus images.

[0046] S2: Construct a convolutional neural network. According to the obtained retinal image and pixel-level vessel labels, generate quantifiable noise data, including noise data in multi-vessel patterns and noise in few-vessel patterns;

[0047] In this embodiment, the process of generating noise data in multi-vessel patterns is as follows:

[0048] A201: Input the retinal image into the segmentation network to overfit the vessels in the image;

[0049] A202: Decompose the foreground vessels in the overfitting result to obtain multiple branches, and quantify them according to the quantity ratio.

[0050] The labels obtained from the overfitting result have more foreground vessels than the original labels. Such vessels are vascular-like structures in the fundus or unimportant vessels determined not to be labeled; this part of the foreground that is not in the original label is used as multi-vessel noise, which is decomposed into multiple branches and can appear on the label according to the quantity ratio, realizing noise quantification in multi-vessel patterns.

[0051] In this embodiment, the process of generating noise data in few-vessel patterns is as follows:

[0052] B201: Iteratively erode the retinal image to generate the main vessel trunks;

[0053] B202: Decompose the parts other than the main vessel trunks into multiple branches, sort the branches by width, and remove the vessel branches according to the quantity ratio to obtain few-vessel noise data.

[0054] During the process of removing vascular branches according to the quantity ratio, since thin branches are preferentially removed compared to thick branches, removing some branches can obtain less-vascular noise data, and based on this, the effect of simulating actual less-vascular noise can be achieved.

[0055] S3: According to the noise data, segment the data set to construct a training set and a test set. At the same time, retain one clean image data in the training set and apply corresponding noise to other images.

[0056] In this embodiment, when segmenting the data set, the data set is segmented into ten equal parts to implement network training and network testing using the leave-one-out cross-validation method during the subsequent training process.

[0057] S4: Set the loss function and training parameters.

[0058] As Figure 2 shown, the convolutional neural network model includes an encoding path composed of four encoder units and a decoding path composed of four decoder units.

[0059] In order not to lose complete context information during the decoding stage, skip connections are used from the encoding path to the decoding path; the corresponding high-resolution features in the encoding path are connected to the upsampled features. The skip connections use direct connections in the channel dimension, and then the two parts of the features are fused through convolution to generate a more accurate segmentation output.

[0060] After the last decoder unit, the number of channels of the feature map is converted to the number of channels required for the task through a 1×1 convolution.

[0061] Among them, each encoder unit is composed of two 3×3 convolutional layers, a ReLU activation layer, a batch normalization layer, and a max pooling layer. In each pooling operation, the number of channels of the feature map doubles.

[0062] Each decoder unit is composed of a 2×2 transposed convolution layer and two 3×3 convolutional layers. The number of feature channels is halved and the size doubles after each transposed convolution operation.

[0063] The middle part between the encoding path and the decoding path, also known as the bottleneck path, is composed of two 3×3 convolutional layers and a transposed convolution layer.

[0064] In this embodiment, the loss function is specifically as follows:

[0065]

[0066] Among them, Ω represents the pixel space of a single branch, {x p , y p} {p∈Ω} represents the p-th pixel in the Ω space, and F is the segmentation network.

[0067] The network optimizer selects the Adam optimizer with an initial learning rate of 0.0001. The learning rate is decayed using the cosine annealing function. The number of training epochs is set to 1000, the batch size is 40, and the Softmax activation function is used as the activation function.

[0068] S5: Alternately input the clean image data and the noise data into the network. Update the network parameters based on the losses obtained from the input of the noise data and the clean data, and obtain the updated network model.

[0069] As Figure 4 shown in the meta - learning work framework, in this embodiment, the noise data is first input into the segmentation network and the re - weighing function to preliminarily update the segmentation network. On this basis, a small amount of clean data is introduced, and the re - weighing function is updated through the loss obtained from the input into the segmentation network to correct the network bias. Finally, the noise data passes through the segmentation network and the re - weighing function again to complete a full round of learning.

[0070] The specific process of updating the network parameters is as follows:

[0071] Input the noise data into the segmentation network. Through the loss obtained from the segmentation network, perform loss re - ratioing through the re - weighing function to update the segmentation network.

[0072] Input the clean data into the segmentation network to obtain the loss and update the re - weighing function.

[0073] After the re - updated re - weighing function is updated, input the noise data into the segmentation network again, and finally update the parameters of the segmentation network according to the obtained loss.

[0074] The structure of the re - weighing function adopted in the present invention is as Figure 3 shown, an MLP - structured fully - connected layer, including a hidden layer with 100 nodes.

[0075] S6: Input the training set data and the test set data into the network model for training and testing, and input the retinal image into the trained network model to output the segmentation result.

[0076] The present invention is not limited to the foregoing specific embodiments. The present invention extends to any new feature disclosed in this specification or any new combination, as well as any new method or process step disclosed or any new combination.

Claims

1. An automatic fundus vessel segmentation method based on low-cost noise data, characterized in that, Including: S1: Screen the dataset to obtain retinal images and pixel-level vascular labels from publicly available datasets; S2: Construct a convolutional neural network. Based on the obtained retinal images and pixel-level vascular labels, generate quantifiable noise data, including noise data for multi-vascular patterns and noise data for few-vascular patterns; The process of generating the noise data for the multi-vascular pattern is as follows: A201: Input the retinal image into the segmentation network to overfit the blood vessels in the image; A202: Decompose the foreground blood vessels in the overfitting result to obtain multiple branches, and quantify them according to the quantity ratio; The process of generating the noise data for the few-vascular pattern is as follows: B201: Iteratively erode the retinal image to generate the blood vessel trunk; B202: Decompose the part other than the blood vessel trunk into multiple branches, sort the branches by width, and remove the blood vessel branches according to the quantity ratio to obtain the noise data for the few-vascular pattern; S3: According to the noise data, segment the dataset to construct a training set and a test set. At the same time, retain one clean image data in the training set and apply corresponding noise to other images; S4: Set the loss function and training parameters; S5: Alternately input the clean image data and the noise data into the network. Update the network parameters through the losses obtained from the input of the noise data and the clean data to obtain an updated network model; The specific process of updating the network parameters is as follows: Input the noise data into the segmentation network. The loss obtained through the segmentation network is reweighted by the reweighting function to update the segmentation network; Input the clean data into the segmentation network to obtain the loss for updating the reweighting function; After the re-updated reweighting function is updated, input the noise data into the segmentation network again, and finally update the parameters of the segmentation network according to the obtained loss; S6: Input the training set data and the test set data into the network model for training and testing, and input the retinal image into the trained network model to output the segmentation result.

2. The automatic fundus vessel segmentation method according to claim 1, characterized in that, In step S3, when segmenting the dataset, divide the dataset into several equal parts.

3. The automatic fundus vessel segmentation method according to claim 1, characterized in that, The convolutional neural network model includes an encoding path composed of four encoder units and a decoding path composed of four decoder units. The encoding path to the decoding path is a skip connection structure; The skip connection is to connect the corresponding high-resolution features in the encoding path to the upsampled features, and then fuse the two parts of features through convolution.

4. The automatic fundus vessel segmentation method according to claim 3, characterized in that, Each encoder unit is composed of two 3×3 convolutional layers, a ReLU activation layer, a batch normalization layer, and a max pooling layer.

5. The automatic fundus vessel segmentation method according to claim 3, characterized in that, The skip connection uses a direct connection in the channel dimension.

6. The automatic fundus vessel segmentation method according to claim 1, characterized in that, The loss function is specifically as follows: Among them, Ω represents the pixel space of a single branch, {x p , y p} {p∈Ω} represents the p-th pixel in the Ω space, and F is the segmentation network.

Images