A method and system for segmenting fundus retinal blood vessel images

Abstract

The invention relates to a fundus retinal vascular image segmentation method and system, belonging to the field of image segmentation processing. Firstly, a fundus retinal color image and a fundus retinal vascular image are obtained; image blocks and labels are extracted from the fundus retinal color image and the fundus retinal vascular image to establish a sample pair data set; the size of the image block is adjusted, and the image blocks and labels are repeatedly extracted from the fundus retinal color image and the fundus retinal vascular image to obtain a plurality of sample pair data sets of different scales; a fundus retinal vascular image segmentation model is established; the fundus retinal vascular image segmentation model is trained and verified by using the plurality of sample pair data sets of different scales, and corresponding prediction results are output; the prediction results output by each feature extraction module are fused to obtain a fundus retinal vascular image classification result; and the segmented fundus retinal vascular image is obtained after image restoration. The method can improve the segmentation accuracy of the fundus retinal vascular image.

Description

Technical Field

[0001] The present invention relates to the field of image segmentation processing, and in particular to a method and system for fundus retinal vascular image segmentation based on multi-network multi-scale fusion of transfer learning. Background Art

[0002] Retinal blood vessels are the only part of the body's blood vessels that can be observed in a non-invasive way. Therefore, segmenting fundus retinal vascular images is of great significance for accurately obtaining the retinal vascular features in fundus retinal vascular images. However, the imaging of fundus images is easily affected by different lighting, resulting in different clarity and imaging effects of retinal blood vessels. The contrast between capillaries and fundus background is small and easy to be confused, which leads to discrepancies in the results of retinal vascular image segmentation. These reasons make fundus retinal vascular image segmentation very difficult.

[0003] Existing methods for segmenting fundus retinal vascular images usually require the use of machine segmentation algorithms, and the methods for segmentation using machines mainly include two categories, namely deep learning methods and non-deep learning methods. Among them, non-deep learning methods often rely on experience to extract features, so they cannot accurately extract vascular features and cannot meet the requirements for accurate segmentation of fundus retinal vascular images. Among the deep learning methods, the current mainstream method is to use U-Net to segment retinal vascular images, but this method has certain limitations. Since the input of U-Net is an image block segmented from the fundus retinal color image, it does not pay attention to every pixel, which leads to the omission of detailed features, resulting in some capillary features in the image not being segmented out. Therefore, this method cannot achieve accurate segmentation of fundus retinal vascular images. Therefore, how to improve the segmentation accuracy of fundus retinal vascular images is a problem that needs to be solved urgently in this field. Summary of the invention

[0004] The purpose of the present invention is to provide a method and system for segmenting fundus retinal vascular images to improve the segmentation accuracy of fundus retinal vascular images and solve the problems of low segmentation accuracy, inaccurate and unreliable segmentation results of fundus retinal vascular images in the prior art.

[0005] To achieve the above object, the present invention provides the following solutions:

[0006] On the one hand, the present invention proposes a method for segmenting fundus retinal blood vessel images, comprising:

[0007] Acquire a fundus retinal color image and a fundus retinal vascular image; the fundus retinal vascular image is an image containing a retinal vascular portion separated from the fundus retinal color image;

[0008] Extracting image blocks and labels from the fundus retinal color image and the fundus retinal vascular image to establish a sample pair data set;

[0009] Adjusting the size of the image block, and repeatedly extracting image blocks and labels from the fundus retinal color image and the fundus retinal vascular image according to the adjusted image block, to obtain a plurality of sample pair data sets of different scales;

[0010] Establishing a fundus retinal vascular image segmentation model; the fundus retinal vascular image segmentation model includes a plurality of feature extraction modules, each of which includes a plurality of convolutional neural networks of different depths, and the plurality of convolutional neural networks of different depths are used to extract vascular features of different regions and different thicknesses from the fundus retinal vascular image;

[0011] The fundus retinal vascular image segmentation model is trained and verified using the plurality of sample pair data sets of different scales; during the training and verification, each feature extraction module is made to correspond to a unique training set and sample set, and a corresponding prediction result is output;

[0012] The prediction results output by each of the feature extraction modules are integrated to obtain a classification result of the fundus retinal blood vessel image;

[0013] The classification result of the fundus retinal vascular image is restored to obtain a segmented fundus retinal vascular image.

[0014] Optionally, the acquiring of the fundus retinal color image and the fundus retinal vascular image specifically includes:

[0015] Use camera equipment to collect fundus retinal color images of different age groups;

[0016] The retinal blood vessels in the fundus retinal color image are manually separated to obtain the fundus retinal blood vessels image.

[0017] Optionally, extracting image blocks and labels from the fundus retinal color image and the fundus retinal vascular image to establish a sample pair data set specifically includes:

[0018] Extracting an image block of size p×p according to pixel points from the fundus retinal color image;

[0019] Extracting a label corresponding to the image block from the fundus retinal vascular image; the label is used to mark the value of the pixel point of the image block corresponding to the label in the fundus retinal vascular image;

[0020] Taking each image block and a label corresponding to the image block as a sample pair, traversing the fundus retinal color image and the fundus retinal vascular image to obtain a plurality of sample pairs;

[0021] The sample pair data set is constructed based on the multiple sample pairs.

[0022] Optionally, adjusting the size of the image block, and repeatedly extracting image blocks and labels from the fundus retinal color image and the fundus retinal vascular image according to the adjusted image block to obtain a plurality of sample pair data sets of different scales specifically includes:

[0023] Select different p values, and according to the selected p values, repeat the loop steps of "extracting image blocks of size p×p from the fundus retinal color image according to pixel points; extracting labels corresponding to the image blocks from the fundus retinal vascular image; taking each image block and the label corresponding to the image block as a sample pair, traversing the fundus retinal color image and the fundus retinal vascular image to obtain multiple sample pairs; constructing the sample pair data set based on the multiple sample pairs", to obtain multiple sample pair data sets of different scales.

[0024] Optionally, the establishing of the fundus retinal blood vessel image segmentation model specifically includes:

[0025] A fundus retinal vascular image segmentation model is established; and a transfer learning method is adopted to initialize the convolutional layer of the fundus retinal vascular image segmentation model using the pre-trained VGG16 network parameters in the existing ImageNet dataset.

[0026] Optionally, the fundus retinal vascular image segmentation model includes 7 feature extraction modules, and each feature extraction module includes 3 convolutional neural networks of different depths.

[0027] Optionally, the using the plurality of sample pair data sets of different scales to train and verify the fundus retinal vascular image segmentation model specifically includes:

[0028] The data set is divided into samples of multiple different scales to obtain training sets of multiple different scales and test sets of multiple different scales; in the training sets and test sets, each feature extraction module has a corresponding training set and test set of a unique scale;

[0029] Inputting the training sets of each scale into the corresponding feature extraction module in the fundus retinal vascular image segmentation model for training to obtain a trained fundus retinal vascular image segmentation model;

[0030] The test sets of each scale are input into the corresponding feature extraction modules in the trained fundus retinal vascular image segmentation model for testing, and each feature extraction module outputs a prediction result.

[0031] Optionally, the prediction results output by each of the feature extraction modules are fused to obtain a classification result of the fundus retinal vascular image, specifically including:

[0032] Based on the principle of majority voting, the pixel point category of each image block is judged according to the prediction results outputted by each feature extraction module, and the judgment results are integrated to obtain the fundus retinal vascular image classification result; the pixel point category includes vascular pixel points and image background pixel points;

[0033] When the classification result of the fundus retinal blood vessel image is 1, it indicates that the pixel point category of the image block is a blood vessel pixel point; when the classification result of the fundus retinal blood vessel image is 0, it indicates that the pixel point category of the image block is an image background pixel point.

[0034] Optionally, performing image restoration on the classification result of the fundus retinal vascular image to obtain a segmented fundus retinal vascular image specifically includes:

[0035] According to each image block and its pixel point category in all fundus retinal vascular image classification results, all image blocks are restored into a complete fundus retinal vascular image to obtain the segmented fundus retinal vascular image.

[0036] On the other hand, the present invention also proposes a fundus retinal blood vessel image segmentation system, comprising:

[0037] An image acquisition module, used for acquiring a fundus retinal color image and a fundus retinal vascular image; the fundus retinal vascular image is an image containing a retinal vascular portion separated from the fundus retinal color image;

[0038] A single sample pair data set acquisition module is used to extract image blocks and labels from the fundus retinal color image and the fundus retinal vascular image to establish a sample pair data set;

[0039] A plurality of sample pair data set acquisition modules are used to adjust the size of the image block, and repeatedly extract image blocks and labels from the fundus retinal color image and the fundus retinal vascular image according to the adjusted image block to obtain a plurality of sample pair data sets of different scales;

[0040] An image segmentation model building module is used to build a fundus retinal vascular image segmentation model; the fundus retinal vascular image segmentation model includes a plurality of feature extraction modules, each of which includes a plurality of convolutional neural networks of different depths, and the plurality of convolutional neural networks of different depths are used to extract vascular features of different regions and different thicknesses from the fundus retinal vascular image;

[0041] An image segmentation model training and verification module is used to train and verify the fundus retinal vascular image segmentation model using the plurality of sample pair data sets of different scales; during training and verification, each feature extraction module is made to correspond to a unique training set and sample set, and a corresponding prediction result is output;

[0042] A prediction result fusion module, used to fuse the prediction results output by each of the feature extraction modules to obtain a fundus retinal vascular image classification result;

[0043] The image restoration module is used to restore the image of the fundus retinal blood vessel image classification result to obtain a segmented fundus retinal blood vessel image.

[0044] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0045] The present invention proposes a pixel-level retinal vascular image segmentation method based on multi-scale multi-network fusion of transfer learning, which can effectively and accurately segment fundus retinal vascular images. The fundus retinal vascular image segmentation model includes multiple feature extraction modules, and the vascular features of different regions and thicknesses in the fundus retinal vascular image are extracted by convolutional neural networks of different depths in the feature extraction modules. After the fundus retinal vascular image segmentation model is trained, the image blocks and labels in the sample pair data set are verified using the trained fundus retinal vascular image segmentation model, so as to obtain the prediction results of the pixel point category representing each image block in the fundus retinal vascular image, and then all the prediction results are fused and the image is restored, so as to obtain an accurate and reliable segmented fundus retinal vascular image. In the model training stage, the fundus retinal vascular image segmentation model can learn relatively complete fundus retinal vascular image knowledge, so that the fundus retinal vascular image segmentation model has better fitting ability, which can effectively improve the accuracy of image segmentation, and solve the problems of low segmentation accuracy, inaccurate and unreliable segmentation results of fundus retinal vascular images in the prior art. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. The following drawings are not deliberately scaled according to the actual size, and the focus is on illustrating the main purpose of the present invention.

[0047] Figure 1 A flowchart of a method for segmenting fundus retinal blood vessel images provided in Example 1 of the present invention;

[0048] Figure 2 The fundus retinal color image provided by Example 1 of the present invention;

[0049] Figure 3 The fundus retinal blood vessel image provided by Example 1 of the present invention;

[0050] Figure 4 A schematic diagram of the structure of a fundus retinal blood vessel image segmentation model provided in Example 1 of the present invention;

[0051] Figure 5 A schematic diagram of a method for segmenting fundus retinal blood vessel images provided in Example 1 of the present invention;

[0052] Figure 6 A structural block diagram of a fundus retinal blood vessel image segmentation system is provided for Example 2 of the present invention. DETAILED DESCRIPTION

[0053] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0054] As shown in the present invention and claims, unless the context clearly indicates an exception, the words "a", "an", "an" and / or "the" do not refer to the singular, but also include the plural. Generally speaking, the terms "include" and "comprise" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list, and the method or device may also include other steps or elements.

[0055] Although the present invention has made various references to certain modules in the system according to an embodiment of the present invention, any number of different modules can be used and run on a user terminal and / or server. The modules are only illustrative, and different aspects of the system and method can use different modules.

[0056] The present invention uses a flow chart to illustrate the operations performed by the system according to an embodiment of the present invention. It should be understood that the preceding or following operations are not necessarily performed precisely in order. On the contrary, various steps may be processed in reverse order or simultaneously as required. At the same time, other operations may also be added to these processes, or one or more operations may be removed from these processes.

[0057] The purpose of the present invention is to provide a method and system for segmenting fundus retinal vascular images to improve the segmentation accuracy of fundus retinal vascular images and solve the problems of low segmentation accuracy, inaccurate and unreliable segmentation results of fundus retinal vascular images in the prior art.

[0058] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0059] Example 1

[0060] like Figure 1 As shown, this embodiment provides a method for segmenting fundus retinal blood vessel images, and the method specifically includes the following steps:

[0061] Step S1, obtaining a fundus retinal color image and a fundus retinal vascular image; the fundus retinal vascular image is an image containing retinal vascular parts separated from the fundus retinal color image. Specifically comprising:

[0062] Step S1.1, using a camera to collect fundus retinal color images of different age groups.

[0063] This embodiment collects fundus retinal color images of various ages between 20 and 80 years old. Taking the DRIVE dataset as an example, the DRIVE dataset uses a Canon CR5 non-mydriatic 3CCD camera for image acquisition to obtain fundus retinal color images. It is easy to understand that the selected different age groups and the types and models of specific camera equipment are not fixed and unique, and can be set according to actual conditions.

[0064] Step S1.2, manually separating the retinal blood vessels in the fundus retinal color image to obtain the fundus retinal blood vessel image.

[0065] In this embodiment, after obtaining the fundus retinal color image, since the fundus retinal color image is an image of the entire fundus retina, such as Figure 2 Therefore, medical experts usually manually segment the retinal blood vessels in the fundus retinal color image using a computer to obtain the corresponding fundus retinal blood vessel image, such as Figure 3 shown.

[0066] Step S2: extract image blocks and labels from the fundus retinal color image and the fundus retinal vascular image to establish a sample pair data set. Specifically, it includes:

[0067] Step S2.1: extracting image blocks of size p×p from the fundus retinal color image according to pixel points.

[0068] Step S2.2: extract a label corresponding to the image block from the fundus retinal vascular image; the label is used to mark the value of the pixel point of the image block corresponding to the label in the fundus retinal vascular image.

[0069] Step S2.3, each image block and the label corresponding to the image block are regarded as a sample pair, that is, a sample pair includes an image block and its corresponding label. Then, according to the method of extracting image blocks and labels and forming sample pairs in steps S2.1 and S2.2, all fundus retinal color images and fundus retinal vascular images are traversed, so that multiple sample pairs can be obtained.

[0070] Step S2.4: construct the sample pair data set based on the multiple sample pairs.

[0071] Step S3, adjusting the size of the image block, and repeatedly extracting image blocks and labels from the fundus retinal color image and the fundus retinal vascular image according to the adjusted image block, to obtain a plurality of sample pair data sets of different scales. Specifically comprising:

[0072] Select different p values, and according to the selected p values, repeat the loop steps of "extracting image blocks of size p×p from the fundus retinal color image according to pixel points; extracting labels corresponding to the image blocks from the fundus retinal vascular image; taking each image block and the label corresponding to the image block as a sample pair, traversing the fundus retinal color image and the fundus retinal vascular image to obtain multiple sample pairs; constructing the sample pair data set based on the multiple sample pairs", to obtain multiple sample pair data sets of different scales.

[0073] It should be noted that the fundus retinal vascular image segmentation model in this embodiment includes 7 feature extraction modules, and each feature extraction module includes 3 convolutional neural networks of different depths. Therefore, in step S3, 7 sample pair data sets of different scales are obtained, and 7 different p values ​​need to be selected. Repeat step S2 for a total of 7 times to extract 7 image blocks and corresponding labels, so as to obtain 7 sample pair data sets of different scales. In this embodiment, different p values ​​are selected to extract image blocks of 7 different scales, and the scales corresponding to each p value are 17×17, 19×19, 21×21, 23×23, 25×25, 27×27, and 29×29. It is easy to understand that the p value can also select other values, which can be determined according to the actual situation.

[0074] In the present invention, during data preprocessing, it is assumed that the retinal vascular dataset is D, and the fundus retinal color image in the dataset is denoted as d 1 , d 2 , ..., d n , the corresponding fundus retinal vascular image is recorded as l 1 , l 2 ,……,l n .d n The length and width of are denoted by h and w respectively. First, fill the fundus retinal color image d n , the filled fundus retinal color image is denoted by d' n Indicates. d' n The length and width of are represented by H and W respectively. The size of the extracted image block is p×p, and the filled fundus retinal color image is shown in formula (1):

[0075] d' n (H,W) =d n (h+p / 2,w+p / 2) (1)

[0076] Secondly, starting from the first pixel of the fundus retinal vascular image, traverse the t-th fundus retinal color image and fundus retinal vascular image to obtain an image block of size p×p. The set of all image blocks extracted from the t-th fundus retinal color image and fundus retinal vascular image is expressed as m represents the number of pixels in the t-th fundus retinal color image and fundus retinal vascular image, i represents the current pixel, and the attribute of each central pixel is described by all elements in the image block. The corresponding label All labels in the tth fundus retinal vascular image are recorded as Each image block and the corresponding label form a sample pair, denoted as {xt i ,y t i}. Through the above steps, the sample pair set is extracted Where n represents the number of fundus retinal vascular images. According to the above steps of processing data, 7 sample pairs of different scales are extracted from the retinal vascular dataset.

[0077] Step S4, establishing a fundus retinal vascular image segmentation model; the fundus retinal vascular image segmentation model includes multiple feature extraction modules, each feature extraction module includes multiple convolutional neural networks of different depths, and the multiple convolutional neural networks of different depths are used to extract vascular features of different regions and different thicknesses from the fundus retinal vascular image. Specifically including:

[0078] First, a fundus retinal vascular image segmentation model is established. In the fundus retinal vascular image segmentation model, each feature extraction module includes a plurality of convolutional neural networks of different depths, and the plurality of convolutional neural networks of different depths are used to extract retinal vascular features of corresponding scales from sample pair data sets of different scales, respectively.

[0079] In this embodiment, the fundus retinal vascular image segmentation model includes 7 feature extraction modules, each of which includes 3 convolutional neural networks with different depths. The model structure is as follows: Figure 4 shown.

[0080] Furthermore, in order to reduce the training time, the present invention adopts a transfer learning method and uses the pre-trained VGG16 network parameters in the existing ImageNet data set to initialize the convolutional layer of the fundus retinal vascular image segmentation model.

[0081] The present invention is based on the top 2, 4, and 7 convolutional layers in the VGG16 network, and adds 5 fully connected layers after the convolutional layers to construct a new convolutional neural network, which are called VGG-16-2, VGG-16-4, and VGG-16-7 respectively. These three convolutional neural networks form a feature extraction module, and there are a total of 7 feature extraction modules in the fundus retinal vascular image segmentation model. The size of the image blocks input in the same feature extraction module is the same, and the size of the image blocks input between different feature extraction modules is different.

[0082] The present invention inputs sample pairs of different scales of the extracted training set into different feature extraction modules in the fundus retinal vascular image segmentation model, and three convolutional neural networks with different depths in each feature extraction module are used to extract vascular features of different depths, and the vascular features extracted in each feature extraction module are fused. Sample pairs of different scales contain vascular features of different thicknesses, and sample pairs of 7 scales are correspondingly input into 7 feature extraction modules, each feature extraction module can be used to extract vascular features of different thicknesses, and the features of the 7 feature extraction modules are fused, so that the fundus retinal vascular image segmentation model can learn vascular features of different regions and thicknesses, thereby obtaining better vascular feature segmentation capabilities, and improving the prediction accuracy of the model and the image segmentation accuracy.

[0083] It should be noted that the fundus retinal vascular image segmentation model set in this embodiment includes 7 feature extraction modules, and each feature extraction module includes 3 convolutional neural networks of different depths. This is only to illustrate the preferred values ​​of the setting, and other values ​​can also be selected. The specific number of feature extraction modules and convolutional neural networks can be set according to actual conditions.

[0084] Step S5, using the plurality of sample pair data sets of different scales to train and verify the fundus retinal vascular image segmentation model; during training and verification, each feature extraction module corresponds to a unique training set and sample set, and the feature extraction module, training set and sample set are marked with numbers 1-7 by numbers, and finally the corresponding prediction results are output. Specifically including:

[0085] Step S5.1, divide the data set into samples of multiple different scales to obtain training sets of multiple different scales and test sets of multiple different scales; in the training sets and test sets, each feature extraction module has a corresponding training set and test set of a unique scale.

[0086] In this embodiment, after obtaining 7 sample pair data sets of different scales, the 7 sample pair data sets of different scales are divided into 7 training sets and 7 test sets according to the ratio of 7:3 between the training set and the test set, and are respectively marked as training set 1, test set 1; training set 2, test set 2; ...; training set 7, test set 7. Correspondingly, the 7 feature extraction modules are also marked as feature extraction module 1, feature extraction module 2, ..., feature extraction module 7.

[0087] Step S5.2, input the training sets of each scale into the corresponding feature extraction module in the fundus retinal vascular image segmentation model for training. In this embodiment, the learning rate during training is set to 0.001, and the maximum number of iterations is set to 100. After the training is completed, a trained fundus retinal vascular image segmentation model is obtained.

[0088] When inputting into the fundus retinal vascular image segmentation model in this embodiment, there is a one-to-one correspondence between the training set and the feature extraction module, that is, training set 1 is input into feature extraction module 1, training set 2 is input into feature extraction module 2, and so on, and the final training set 7 is input into feature extraction module 7.

[0089] In the present invention, convolutional neural networks of different depths in each feature extraction module are used to extract vascular features of different areas of the fundus retinal vascular image. According to the order of convolutional neural networks of different depths from shallow to deep, the extracted vascular features progress from the edge of the image to the center of the image, that is, the shallower convolutional neural network extracts edge features, and the deeper convolutional neural network extracts high-level features of the image center.

[0090] In this embodiment, each feature extraction module can obtain the vascular features corresponding to the input training set, and different feature extraction modules can obtain vascular features of different training sets. Since the scales of training sets 1 to 7 are different, the thickness of the blood vessels contained in training sets 1 to 7 is different. Then, different feature extraction modules can obtain vascular features of different thicknesses. Therefore, the fundus retinal vascular image segmentation model can learn vascular features of different regions and thicknesses during the training stage, thereby effectively improving the prediction accuracy of the fundus retinal vascular image segmentation model, thereby improving the accuracy of the final image segmentation.

[0091] Step S5.3: input the test sets of each scale into the corresponding feature extraction modules in the trained fundus retinal vascular image segmentation model for testing, and each feature extraction module outputs a prediction result.

[0092] Similarly, in this embodiment, test set 1 is input into feature extraction module 1, test set 2 is input into feature extraction module 2, and so on, test set 7 is input into feature extraction module 7, each feature extraction module will obtain a prediction result, a total of 7 prediction results, and these 7 prediction results are saved.

[0093] like Figure 5As shown, the principle of the fundus retinal vascular image segmentation method of the present invention is: first, when obtaining the training data and test data of the fundus retinal vascular image segmentation model, the fundus retinal vascular image is preprocessed based on the pixels, that is, the image blocks and labels are extracted from the fundus retinal color image and the fundus retinal vascular image according to the pixels, and a plurality of sample pair data sets of different scales are established, so as to construct a fundus retinal vascular image segmentation model of multiple multi-scale networks based on transfer learning. After completing the training of the fundus retinal vascular image segmentation model, the trained fundus retinal vascular image segmentation model is used to predict the pixel category of each image block, and the pixel category includes two types of vascular pixels and image background pixels, so as to judge whether each image block belongs to a vascular pixel or an image background pixel, so as to determine the regional position of the image block in the fundus retinal vascular image according to the corresponding vascular features of different thicknesses in the image block, and then after the prediction results are fused and the image is restored, each image block is restored into a complete fundus retinal vascular image to generate a predicted fundus retinal vascular image.

[0094] Step S6, fusing the prediction results output by each feature extraction module to obtain a classification result of the fundus retinal vascular image. Specifically comprising:

[0095] Based on the principle of majority voting, that is, the minority obeys the majority principle when voting, the pixel point category of each image block is judged according to the prediction results output by each feature extraction module and the judgment results are fused to obtain the fundus retinal vascular image classification result.

[0096] For the same image block, the seven feature extraction modules will output seven prediction results. Among them, some feature extraction modules may determine that the pixel category of the image block is the image background pixel, and another feature extraction module may determine that the pixel category of the image block is a blood vessel pixel. At this time, since the seven prediction results have a total of seven votes, according to the principle of majority voting, it is easy to determine the final pixel category of the image block, with higher accuracy and more reliable results.

[0097] In the present invention, the final classification result of the fundus retinal vascular image is obtained by fusing the results of multiple feature extraction modules, which is expressed as formula (2):

[0098]

[0099] Among them, R represents the final classification result of fundus retinal vascular image, R arepresents the classification result of each feature extraction module, and c represents the number of feature extraction modules. When the sum of the classification results output by multiple feature extraction modules is greater than or equal to c / 2, the final classification result of the fundus retinal vascular image is 1, indicating that the pixel point category of the image block is a vascular pixel point. When the sum of the classification results output by multiple feature extraction modules is less than c / 2, the final classification result of the fundus retinal vascular image is 0, indicating that the pixel point category of the image block is an image background pixel point.

[0100] Step S7, performing image restoration on the classification result of the fundus retinal vascular image to obtain a segmented fundus retinal vascular image. Specifically comprising:

[0101] According to each image block and its pixel point category in all fundus retinal vascular image classification results, all image blocks are restored into a complete fundus retinal vascular image to obtain the segmented fundus retinal vascular image.

[0102] The present invention provides a retinal vascular image segmentation method based on multiple multi-scale fusions of transfer learning. The method extracts a number of image blocks of preset sizes according to pixel points from each fundus retinal color image. The category of each image block is the category of its central pixel point, thereby converting the image-based segmentation problem of the present invention into an image block-based classification problem.

[0103] First, the fundus retinal color image and fundus retinal vascular image are preprocessed, that is, image blocks of size p×p and their corresponding labels are extracted from the fundus retinal color image and its corresponding fundus retinal vascular image by pixel point to form a set of sample pairs. The above two images are extracted multiple times to obtain multiple sets of sample pairs of different sizes, and the sample pair sets are divided into training sets and test sets. A fundus retinal vascular image segmentation model with multiple feature extraction modules is constructed, each feature extraction module has three convolutional neural networks with different depths, the parameters of the convolutional layer in the fundus retinal vascular image segmentation model are initialized using the parameters of the VGG16 network pre-trained on the ImageNet data set, the sample pair set of the acquired training set is input into the fundus retinal vascular image segmentation model, each feature extraction module in the fundus retinal vascular image segmentation model inputs a sample pair set of the same scale, the fundus retinal vascular image segmentation model is trained, the convolutional neural networks of different depths in each feature extraction module are used to extract vascular features in different regions, and different feature extraction modules input vascular features of different scales to extract vascular features of different thicknesses, so that the fundus retinal vascular image segmentation model extracts as many vascular features as possible, so that the fundus retinal vascular image segmentation model has better fitting ability, can accurately segment fundus retinal vascular images, and improve image segmentation accuracy and segmentation effect. The present invention can accurately segment retinal vascular images at the pixel level and has a good segmentation effect. In addition, the present invention is tested on existing retinal image datasets such as the DRIVE dataset and the ImageNet dataset, proving the superiority of the method proposed in the present invention in retinal vascular image segmentation.

[0104] Example 2

[0105] like Figure 6 As shown, this embodiment provides a fundus retinal blood vessel image segmentation system, the system comprising:

[0106] An image acquisition module M1 is used to acquire a fundus retinal color image and a fundus retinal vascular image; the fundus retinal vascular image is an image containing a retinal vascular portion separated from the fundus retinal color image;

[0107] A single sample pair data set acquisition module M2 is used to extract image blocks and labels from the fundus retinal color image and the fundus retinal vascular image to establish a sample pair data set;

[0108] A plurality of sample pair data set acquisition modules M3 are used to adjust the size of the image block, and repeatedly extract image blocks and labels from the fundus retinal color image and the fundus retinal vascular image according to the adjusted image block to obtain a plurality of sample pair data sets of different scales;

[0109] An image segmentation model building module M4 is used to build a fundus retinal vascular image segmentation model; the fundus retinal vascular image segmentation model includes a plurality of feature extraction modules, each of which includes a plurality of convolutional neural networks of different depths, and the plurality of convolutional neural networks of different depths are used to extract vascular features of different regions and different thicknesses from the fundus retinal vascular image respectively;

[0110] The image segmentation model training and verification module M5 is used to train and verify the fundus retinal blood vessel image segmentation model using the plurality of sample pair data sets of different scales; during the training and verification, each feature extraction module is made to correspond to a unique training set and sample set, and the corresponding prediction results are output;

[0111] A prediction result fusion module M6 is used to fuse the prediction results output by each of the feature extraction modules to obtain a classification result of the fundus retinal blood vessel image;

[0112] The image restoration module M7 is used to restore the image of the fundus retinal blood vessel image classification result to obtain a segmented fundus retinal blood vessel image.

[0113] The above is an explanation of the present invention and should not be considered as a limitation thereof. Although several exemplary embodiments of the present invention have been described, it will be readily appreciated by those skilled in the art that many modifications may be made to the exemplary embodiments without departing from the novel teachings and advantages of the present invention. Therefore, all such modifications are intended to be included within the scope of the present invention as defined in the claims. It should be understood that the above is an explanation of the present invention and should not be considered as being limited to the specific embodiments disclosed, and modifications to the disclosed embodiments and other embodiments are intended to be included within the scope of the appended claims. The present invention is defined by the claims and their equivalents.

Claims

1. A method for segmenting fundus retinal blood vessel images, characterized in that: include: Acquire a fundus retinal color image and a fundus retinal vascular image; the fundus retinal vascular image is an image containing a retinal vascular portion separated from the fundus retinal color image; Extracting image blocks and labels from the fundus retinal color image and the fundus retinal vascular image to establish a sample pair data set specifically includes: Extracting image blocks of size p×p from the fundus retinal color image according to pixel points; extracting labels corresponding to the image blocks from the fundus retinal vascular image; the labels are used to mark the values ​​of the pixel points of the image blocks corresponding to the labels in the fundus retinal vascular image; taking each image block and the label corresponding to the image block as a sample pair, traversing the fundus retinal color image and the fundus retinal vascular image to obtain multiple sample pairs; constructing the sample pair data set according to the multiple sample pairs; The size of the image block is adjusted, and image blocks and labels are repeatedly extracted from the fundus retinal color image and the fundus retinal vascular image according to the adjusted image block to obtain a plurality of sample pair data sets of different scales, specifically including: Select different p values, and according to the selected p values, repeat the loop steps of "extracting image blocks of size p×p from the fundus retinal color image according to pixel points; extracting labels corresponding to the image blocks from the fundus retinal vascular image; taking each image block and the label corresponding to the image block as a sample pair, traversing the fundus retinal color image and the fundus retinal vascular image to obtain multiple sample pairs; constructing the sample pair data set according to the multiple sample pairs", to obtain multiple sample pair data sets of different scales; Establishing a fundus retinal vascular image segmentation model; the fundus retinal vascular image segmentation model includes a plurality of feature extraction modules, each of which includes a plurality of convolutional neural networks of different depths, and the plurality of convolutional neural networks of different depths are used to extract vascular features of different regions and different thicknesses from the fundus retinal vascular image; The fundus retinal vascular image segmentation model is trained and verified using the sample pair data sets of the multiple different scales; during the training and verification, each feature extraction module corresponds to a unique training set and sample set, and outputs a corresponding prediction result; wherein, by dividing the sample pair data sets of the multiple different scales, a plurality of training sets of different scales and a plurality of test sets of different scales are obtained; in the training set and the test set, each feature extraction module has a corresponding training set and test set of a unique scale; The prediction results output by each of the feature extraction modules are integrated to obtain a classification result of the fundus retinal blood vessel image; The classification result of the fundus retinal vascular image is restored to obtain a segmented fundus retinal vascular image.

2. The method for segmenting fundus retinal blood vessel images according to claim 1, characterized in that: The obtaining of the fundus retinal color image and the fundus retinal vascular image specifically includes: Use camera equipment to collect fundus retinal color images of different age groups; The retinal blood vessels in the fundus retinal color image are manually separated to obtain the fundus retinal blood vessels image.

3. The method for segmenting fundus retinal blood vessel images according to claim 1, characterized in that: The establishing of the fundus retinal blood vessel image segmentation model specifically includes: A fundus retinal vascular image segmentation model is established; and the convolutional layer of the fundus retinal vascular image segmentation model is initialized by using the VGG16 network parameters pre-trained in the existing ImageNet data set in a transfer learning manner; in the fundus retinal vascular image segmentation model, each of the feature extraction modules includes a plurality of convolutional neural networks of different depths, and the plurality of convolutional neural networks of different depths are used to extract retinal vascular features of corresponding scales from sample pair data sets of different scales, respectively.

4. The method for segmenting fundus retinal blood vessel images according to claim 1 or 3, characterized in that: The fundus retinal vascular image segmentation model includes 7 feature extraction modules, and each feature extraction module includes 3 convolutional neural networks of different depths.

5. The method for segmenting fundus retinal blood vessel images according to claim 1, characterized in that: The training and verification of the fundus retinal vascular image segmentation model using the plurality of sample pair data sets of different scales specifically includes: Inputting the training sets of each scale into the corresponding feature extraction module in the fundus retinal vascular image segmentation model for training to obtain a trained fundus retinal vascular image segmentation model; The test sets of each scale are input into the corresponding feature extraction modules in the trained fundus retinal vascular image segmentation model for testing, and each feature extraction module outputs a prediction result.

6. The method for segmenting fundus retinal blood vessel images according to claim 1, characterized in that: The prediction results output by each feature extraction module are integrated to obtain the classification result of the fundus retinal vascular image, which specifically includes: Based on the principle of majority voting, the pixel point category of each image block is judged according to the prediction results outputted by each feature extraction module, and the judgment results are integrated to obtain the fundus retinal vascular image classification result; the pixel point category includes vascular pixel points and image background pixel points; When the classification result of the fundus retinal blood vessel image is 1, it indicates that the pixel point category of the image block is a blood vessel pixel point; when the classification result of the fundus retinal blood vessel image is 0, it indicates that the pixel point category of the image block is an image background pixel point.

7. The method for segmenting fundus retinal blood vessel images according to claim 6, characterized in that: The image restoration of the classification result of the fundus retinal vascular image to obtain a segmented fundus retinal vascular image specifically includes: According to each image block and its pixel point category in all fundus retinal vascular image classification results, all image blocks are restored into a complete fundus retinal vascular image to obtain the segmented fundus retinal vascular image.

8. A fundus retinal blood vessel image segmentation system, characterized in that: include: An image acquisition module, used for acquiring a fundus retinal color image and a fundus retinal vascular image; the fundus retinal vascular image is an image containing a retinal vascular portion separated from the fundus retinal color image; A single sample pair data set acquisition module is used to extract image blocks and labels from the fundus retinal color image and the fundus retinal vascular image to establish a sample pair data set, specifically including: Extracting image blocks of size p×p from the fundus retinal color image according to pixel points; extracting labels corresponding to the image blocks from the fundus retinal vascular image; the labels are used to mark the values ​​of the pixel points of the image blocks corresponding to the labels in the fundus retinal vascular image; taking each image block and the label corresponding to the image block as a sample pair, traversing the fundus retinal color image and the fundus retinal vascular image to obtain multiple sample pairs; constructing the sample pair data set according to the multiple sample pairs; A plurality of sample pair data set acquisition modules are used to adjust the size of the image block, and repeatedly extract image blocks and labels from the fundus retinal color image and the fundus retinal vascular image according to the adjusted image block to obtain a plurality of sample pair data sets of different scales, specifically including: Select different p values, and according to the selected p values, repeat the loop steps of "extracting image blocks of size p×p from the fundus retinal color image according to pixel points; extracting labels corresponding to the image blocks from the fundus retinal vascular image; taking each image block and the label corresponding to the image block as a sample pair, traversing the fundus retinal color image and the fundus retinal vascular image to obtain multiple sample pairs; constructing the sample pair data set according to the multiple sample pairs", to obtain multiple sample pair data sets of different scales; An image segmentation model building module is used to build a fundus retinal vascular image segmentation model; the fundus retinal vascular image segmentation model includes a plurality of feature extraction modules, each of which includes a plurality of convolutional neural networks of different depths, and the plurality of convolutional neural networks of different depths are used to extract vascular features of different regions and different thicknesses from the fundus retinal vascular image; An image segmentation model training and verification module is used to train and verify the fundus retinal vascular image segmentation model using the sample pair data sets of multiple different scales; during training and verification, each feature extraction module corresponds to a unique training set and sample set, and outputs corresponding prediction results; wherein, by dividing the sample pair data sets of multiple different scales, a plurality of training sets of different scales and a plurality of test sets of different scales are obtained; in the training set and the test set, each feature extraction module has a corresponding training set and test set of a unique scale; A prediction result fusion module, used to fuse the prediction results output by each of the feature extraction modules to obtain a classification result of the fundus retinal blood vessel image; The image restoration module is used to restore the image of the fundus retinal blood vessel image classification result to obtain a segmented fundus retinal blood vessel image.

Images