Method, device, electronic device and storage medium for identifying blood vessels in fundus images

Abstract

The present invention relates to a method, device, electronic device and storage medium for identifying blood vessels in fundus images. The method comprises: obtaining a fundus image to be tested; extracting a first feature image and a second feature image of the fundus image to be tested based on a detection operator; extracting a spatial shape feature image of the fundus image to be tested based on a semantic segmentation model; reconstructing the fundus image to be tested according to the first feature image, the second feature image and the spatial shape feature image; inputting the reconstructed fundus image to be tested into a blood vessel segmentation model to obtain a blood vessel segmentation image of the fundus image to be tested; wherein the semantic segmentation model is obtained by training based on a fundus image training set; and the blood vessel segmentation model is obtained by training based on a reconstructed fundus image training set. The present invention improves the image clarity and makes the graphic features more obvious by reconstructing the fundus image to be tested, and obtains a blood vessel segmentation image with higher segmentation accuracy by inputting the reconstructed fundus image into the blood vessel segmentation model for blood vessel identification.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a method, device, electronic equipment and storage medium for identifying blood vessels in fundus images. Background Art

[0002] Fundus image analysis is an important topic in medical image analysis. Due to the objectivity, repeatability, accuracy and large-scale requirements of fundus screening results, effective methods are used to automatically extract vascular structures in retinal fundus images, which helps in the early diagnosis and treatment tracking of fundus diseases and has important application value in clinical practice.

[0003] Image feature extraction is an essential step in image processing and computer vision related fields. When training a model, the selection and extraction of image features will directly affect the learning effect of the model, thereby affecting the accuracy and learning rate of the final classification. At present, the accuracy of blood vessel extraction in the field of medical images needs to be improved, especially for blurred fundus images, where the edges of fundus blood vessels are difficult to accurately segment, making the extraction effect unsatisfactory. Summary of the invention

[0004] The present invention provides a method, a device, an electronic device and a storage medium for identifying blood vessels in fundus images, so as to solve the defect of low accuracy in identifying blood vessels in fundus images in the prior art.

[0005] In a first aspect, the present invention provides a method for identifying blood vessels in fundus images, comprising:

[0006] Acquire the fundus image to be tested;

[0007] Extracting a first characteristic image and a second characteristic image of the fundus image to be tested based on a detection operator;

[0008] Extracting a spatial shape feature image of the fundus image to be measured based on a semantic segmentation model;

[0009] Reconstructing the fundus image to be measured according to the first characteristic image, the second characteristic image and the spatial shape characteristic image;

[0010] Inputting the reconstructed fundus image to be tested into a blood vessel segmentation model to obtain a blood vessel segmentation image of the fundus image to be tested;

[0011] The semantic segmentation model is obtained by training based on a fundus image training set; and the blood vessel segmentation model is obtained by training based on a reconstructed fundus image training set.

[0012] According to a method for identifying blood vessels in fundus images provided by the present invention, the first feature image and the second feature image of the fundus image to be tested are extracted based on a detection operator, further comprising:

[0013] Extracting multiple types of feature images of the fundus image to be tested based on the detection operator;

[0014] For the characteristic images of multiple types of the fundus image to be measured, respectively, calculating the information entropy;

[0015] Take the feature image corresponding to the maximum information entropy as the first feature image;

[0016] A feature image whose information entropy is second only to the maximum information entropy is taken as the second feature image.

[0017] According to a method for identifying blood vessels in fundus images provided by the present invention, the plurality of types of feature images are at least two of grayscale feature images, texture feature images, edge feature images, and gradient feature images;

[0018] Correspondingly, the extracting of multiple types of feature images of the fundus image to be tested based on the detection operator further includes:

[0019] Performing grayscale conversion on the fundus image to be tested to obtain a grayscale feature image;

[0020] Histogram equalization is performed on the grayscale feature image, and based on a detection operator, a texture feature image, an edge feature image or a gradient feature image of the fundus image to be tested is extracted.

[0021] According to a method for identifying blood vessels in fundus images provided by the present invention, the method of reconstructing the fundus image to be measured according to the first characteristic image, the second characteristic image and the spatial shape characteristic image further comprises:

[0022] The first characteristic image, the second characteristic image and the spatial shape characteristic image are respectively used as a component of the RGB three-channel of the fundus image to be measured, and image synthesis is performed to obtain a reconstructed fundus image to be measured.

[0023] According to a method for identifying blood vessels in a fundus image provided by the present invention, after inputting the reconstructed fundus image to be tested into a blood vessel segmentation model to obtain a blood vessel segmentation image of the fundus image to be tested, the method further comprises:

[0024] The blood vessel segmentation image of the fundus image to be tested is input into a random field model to obtain a denoised blood vessel segmentation image.

[0025] According to a method for identifying blood vessels in fundus images provided by the present invention, before acquiring the fundus image to be tested, the method further includes:

[0026] Obtain a fundus image training set;

[0027] According to the fundus image training set, the semantic segmentation model is trained;

[0028] Based on the detection operator, extracting a first feature image set and a second feature image set of the fundus image training set;

[0029] Reconstructing the fundus image training set according to the first feature image set, the second feature image set of the fundus image training set and the spatial shape feature image set corresponding to the fundus image training set;

[0030] The blood vessel segmentation model is obtained by training according to the reconstructed fundus image training set.

[0031] According to a method for identifying blood vessels in fundus images provided by the present invention, before obtaining a fundus image training set, the method further includes:

[0032] Obtaining a training set of original fundus images;

[0033] Performing image preprocessing on the original fundus image training set to obtain a first sample set; the image preprocessing includes: at least one of rotation and flipping;

[0034] Adding the first sample set shown to the original fundus image sample set to obtain a second sample set;

[0035] The second sample set is normalized to a uniform size to obtain the fundus image training set.

[0036] In a second aspect, the present invention further provides a device for identifying blood vessels in fundus images, comprising:

[0037] An acquisition module, used for acquiring the fundus image to be tested;

[0038] A first extraction module, used for extracting a first characteristic image and a second characteristic image of the fundus image to be tested based on a detection operator;

[0039] A second extraction module is used to extract a spatial shape feature image of the fundus image to be measured based on a semantic segmentation model;

[0040] A reconstruction module, used for reconstructing the fundus image to be measured according to the first characteristic image, the second characteristic image and the spatial shape characteristic image;

[0041] A generating module, used for inputting the reconstructed fundus image to be tested into a blood vessel segmentation model to obtain a blood vessel segmentation image of the fundus image to be tested;

[0042] The semantic segmentation model is obtained by training based on a fundus image training set; and the blood vessel segmentation model is obtained by training based on a reconstructed fundus image training set.

[0043] In a third aspect, the present invention further provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of the method for identifying blood vessels in fundus images as described in any one of the first aspects above are implemented.

[0044] In a fourth aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any one of the methods for identifying blood vessels in fundus images as described in the first aspect above.

[0045] The present invention provides a method, device, electronic device and storage medium for identifying blood vessels in fundus images. The first feature image and the second feature image of the fundus image to be tested are extracted by a detection operator, the spatial shape feature image of the fundus image to be tested is extracted by a semantic segmentation model, and the fundus image to be tested is reconstructed according to the three types of feature images, thereby improving the image clarity and making the graphic features more obvious. By inputting the reconstructed fundus image into a blood vessel segmentation model for blood vessel identification, a blood vessel segmentation image with higher segmentation accuracy is obtained. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0047] Figure 1 It is a flow chart of a method for identifying blood vessels in fundus images provided by the present invention;

[0048] Figure 2 It is a schematic diagram of a semantic segmentation model and a vascular segmentation model acquisition process of a method for identifying blood vessels in fundus images provided by the present invention;

[0049] Figure 3 It is a schematic diagram of the structure of a device for identifying blood vessels in fundus images provided by the present invention;

[0050] Figure 4 is a schematic diagram of the structure of the first extraction module 32 in the present invention;

[0051] Figure 5 It is a schematic structural diagram of another device for identifying blood vessels in fundus images provided by the present invention;

[0052] Figure 6 It is a structural schematic diagram of an electronic device provided by the present invention;

[0053] Figure 7is an example image of the grayscale feature image of the fundus image in the present invention;

[0054] Figure 8 is an example image of a texture feature image of a fundus image in the present invention;

[0055] Fig. 9 is an example diagram of edge feature images of fundus images in the present invention;

[0056] Fig.10 This is an example diagram after histogram equalization is performed on the grayscale feature image in the present invention. DETAILED DESCRIPTION

[0057] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0058] The effect of the blood vessel segmentation model trained by the prior art is closely related to the image quality of the training samples, that is, the classification results of the model trained with the training samples with poor image quality are poor, so that the classification results obtained by the model trained with some fuzzy pathological images with poor blood vessel features are not fine enough, and the model is not sensitive to the details in the images with poor quality, and the edge accuracy is not high. Based on this, the present invention is proposed.

[0059] Combine the following Figure 1-Figure 10 The present invention is described.

[0060] In a first aspect, the present invention provides a method for identifying blood vessels in fundus images, such as Figure 1 As shown, the present invention provides a method for identifying blood vessels in fundus images, comprising:

[0061] S11, obtaining a fundus image to be tested;

[0062] The fundus image to be tested is the target object of the blood vessel recognition of this method, and is used to extract the vascular tissue image for reference and analysis by doctors. The fundus is a medical concept, which refers to the tissue in the back of the eyeball.

[0063] S12, extracting a first characteristic image and a second characteristic image of the fundus image to be measured based on a detection operator;

[0064] Among them, the detection operator is a tool for extracting image features. It can be understood that different detection operators are required to extract different image features. For example, if it is an edge feature image that needs to be extracted, the detection operator can be a Canny operator, a Sobel operator, a Lapace operator, a Prewitt operator, a Roberts operator, etc. If it is a texture feature image that needs to be extracted, the detection operator can be an LBP operator, a GLCM operator, a GLDS operator, a Gabort operator, etc. There is no restriction on the specific selection of the detection operator, as long as it meets the requirements for extracting the target feature image. Based on the detection operator, feature extraction is performed on the fundus image to be tested to obtain a first feature image and a second feature image, which are used for subsequent reconstruction of the fundus image to be tested, wherein the first feature image and the second feature image represent two different types of features of the image to be tested.

[0065] S13, extracting a spatial shape feature image of the fundus image to be measured based on a semantic segmentation model;

[0066] The semantic segmentation model is an encoder-decoder network, in which the encoder downsamples the input, the decoder upsamples the output of the encoder, restores the resolution, and semantically projects the discriminable features learned by the encoder to the pixel space to obtain dense classification. The semantic segmentation model can be FCN, U-Net, SegNet, DeepLab, E-Net, Mask R-CNN, PSPNet, RefineNet, G-FRNet model, etc., and no specific restrictions are made here. The semantic segmentation model is pre-trained based on the fundus image training set and is used to extract the spatial shape feature image of the fundus image to be tested, wherein the spatial shape feature refers to the spatial position or relative direction relationship between multiple blood vessels extracted from the fundus image to be tested, and these relationships can also be divided into connection / adjacency relationships, overlap / overlapping relationships, etc. The pre-trained semantic segmentation model is used to extract the spatial shape feature image of the fundus image to be tested, which is used for subsequent reconstruction of the fundus image to be tested.

[0067] S14, reconstructing the fundus image to be measured according to the first characteristic image, the second characteristic image and the spatial shape characteristic image;

[0068] The first characteristic image, the second characteristic image and the spatial shape characteristic image reflect different types of vascular features of the image to be tested. The fundus image to be tested is reconstructed based on these features to make the image clearer and the image features more obvious, which is convenient for vascular identification.

[0069] S15, inputting the reconstructed fundus image to be measured into a blood vessel segmentation model to obtain a blood vessel segmentation image of the fundus image to be measured;

[0070] The blood vessel segmentation model is obtained by training the fundus image training samples reconstructed from the fundus image training samples, and the specific structure of the blood vessel segmentation model can be FCN, U-net, RNN model and its deformation, etc. The trained blood vessel segmentation model is used to perform blood vessel recognition on the fundus image to be tested, and a blood vessel segmentation image of the image to be tested is obtained.

[0071] This embodiment extracts the first feature image and the second feature image of the fundus image to be tested through the detection operator, extracts the spatial shape feature image of the fundus image to be tested through the semantic segmentation model, and reconstructs the fundus image to be tested based on the three types of feature images, thereby improving the image clarity and making the graphic features more obvious. By inputting the reconstructed fundus image into the vascular segmentation model for vascular identification, a vascular segmentation image with higher segmentation accuracy is obtained.

[0072] In one embodiment, the blood vessel segmentation model is obtained by training a U-net model based on the reconstructed fundus training sample to be tested.

[0073] Since medical images have small data volume, relatively fixed structure, unclear boundaries, and relatively difficult to obtain training data, the U-Net model uses a skip connection structure to connect the downsampling layer with the upsampling layer, so that the features (semantic information) extracted by the downsampling layer can be directly transferred to the upsampling layer, fusing features of different scales, making the pixel positioning of the U-net model more accurate and the segmentation accuracy higher. And because of the combination of underlying features, it is not easy to overfit even for small samples, which improves the efficiency of model training.

[0074] In one embodiment, the extracting of the first characteristic image and the second characteristic image of the fundus image to be measured based on the detection operator further includes: extracting multiple types of characteristic images of the fundus image to be measured based on the detection operator; calculating the information entropy of the multiple types of characteristic images of the fundus image to be measured respectively; taking the characteristic image corresponding to the maximum information entropy as the first characteristic image; taking the characteristic image whose information entropy is second only to the maximum information entropy as the second characteristic image.

[0075] The information entropy of an image is a statistical form that reflects the characteristics of the average amount of information in an image. It represents the aggregation characteristics of the grayscale distribution in an image. Image entropy is expressed as the average value of the grayscale set in the image, and it also describes the average amount of information in the image. In other words, the greater the information entropy of an image, the more information it contains, which means that the more information this image has, the better its quality.

[0076] In this embodiment, the information entropy of different types of extracted feature images is calculated, and by comparing the information entropy values, feature images with better quality and clearer fundus image information to be tested are screened out for use in reconstructing the image to be tested, which can make the image to be tested clearer and the features more obvious.

[0077] In one embodiment, information entropy is calculated by the following formula:

[0078]

[0079] Where H is information entropy, p i It represents the proportion of pixels with gray value i in the image. i is an integer from 0 to n, and n is the maximum gray value.

[0080] In one embodiment, the plurality of types of feature images are at least two of grayscale feature images, texture feature images, edge feature images, and gradient feature images;

[0081] Correspondingly, the extracting of multiple types of feature images of the fundus image to be tested based on the detection operator further includes:

[0082] Performing grayscale conversion on the fundus image to be tested to obtain a grayscale feature image;

[0083] Histogram equalization is performed on the grayscale feature image, and based on a detection operator, a texture feature image, an edge feature image or a gradient feature image of the fundus image to be tested is extracted.

[0084] Specifically, Figure 7 This is an example of a grayscale feature image. Grayscale uses black tones to represent objects, that is, black is used as the base color, and different saturations of black are used to display images; Figure 8 This is an example of a texture feature image. Texture is a visual feature that reflects the homogeneity of an image. It reflects the slowly changing or periodically changing surface structure and arrangement properties of an object. Fig. 9 This is an example of an edge feature image. An edge refers to a significant change in the local grayscale of an image, which exists between the target and the background, between targets and targets, and between regions; a gradient refers to the rate of change of a certain pixel in the image in the x and y directions. Grayscale feature images, texture feature images, edge feature images, and gradient feature images describe the characteristics of the fundus image to be tested from different angles. The multiple types of feature images include at least two of the above in order to ensure a certain screening range, so as to extract the first feature image and the second feature image with clearer features.

[0085] Since the grayscale feature is a relatively basic feature, in this embodiment, the graphic is first converted into grayscale to obtain a grayscale feature image, and then the grayscale feature image is histogram equalized. Based on the detection operator, feature extraction is performed on the grayscale feature image after histogram equalization to obtain a texture feature image, an edge feature image or a gradient feature image of the fundus image to be tested.

[0086] Fig.10 This is an example of an image after grayscale feature image is grayscale converted. In the process of taking fundus images, pathological changes in the vitreous body in the eyeball cause changes in light transmittance, errors in fundus cameras, uneven lighting, complex backgrounds and other factors, all of which will cause the low contrast of fundus images to decrease and cover up some vascular details. Therefore, when extracting features based on grayscale images, using histogram equalization to enhance grayscale images can highlight the vascular vein information in the fundus image to be tested.

[0087] In this embodiment, the first feature image and the second feature image are selected from at least two types of specific features to obtain the first feature image and the second feature image with clearer features. By performing histogram equalization on the grayscale feature image, the vascular venous information in the fundus image to be tested is highlighted.

[0088] In one embodiment, reconstructing the fundus image to be measured based on the first characteristic image, the second characteristic image and the spatial shape characteristic image further includes: using the first characteristic image, the second characteristic image and the spatial shape characteristic image as a component of the RGB three channels of the fundus image to be measured, respectively, performing image synthesis, and obtaining a reconstructed fundus image to be measured.

[0089] RGB is a color standard that obtains a variety of colors by changing the three color channels of red (R), green (G), and blue (B) and superimposing them on each other. In this embodiment, the first characteristic image, the second characteristic image, and the spatial shape characteristic image are respectively used as a component of the three RGB channels to perform image synthesis to obtain a reconstructed fundus image to be tested. It can be understood that the above three types of characteristic images are not fixed to the input R, G, and B channels, and the combination relationship can be adjusted.

[0090] Similarly, based on different color spaces, such as CMY, HSV, and HSI color spaces, the first characteristic image, the second characteristic image, and the spatial shape characteristic image can also be used as a component of the CMY, HSV, or HSI three-channel of the fundus image to be tested, respectively, to perform image synthesis and obtain a reconstructed fundus image to be tested.

[0091] In this embodiment, the first characteristic image, the second characteristic image and the spatial shape characteristic image are respectively used as a component of the RGB three-channel to perform image synthesis, thereby obtaining a fundus image to be tested with higher definition and more obvious features.

[0092] In one embodiment, after inputting the reconstructed fundus image to be tested into a vascular segmentation model to obtain the vascular segmentation image of the fundus image to be tested, it also includes: inputting the vascular segmentation image of the fundus image to be tested into a random field model to obtain a denoised vascular segmentation image.

[0093] The random field model is used to consider the association information between different pixels in the pixel point space, thereby suppressing the noise pixels of segmentation errors and enhancing the segmentation accuracy of blood vessels. The random field model can be a conditional model, a Markov random field model, a Gibbs random field model, etc.

[0094] In one embodiment, the random field model is specifically a denseCRF model.

[0095] After the vascular segmentation model outputs the vascular segmentation image, the pixel points of the vascular segmentation image are used as the input of denseCRF, and the average approximate field method is used to solve and calculate the final vascular segmentation image. Since denseCRF has global information that takes into account the spatial context of the pixel points, it can suppress the noise pixels of segmentation errors, thereby enhancing the segmentation accuracy of the blood vessels and obtaining a denoised vascular segmentation image.

[0096] like Figure 2 As shown, in one embodiment, before acquiring the fundus image to be measured, the method further includes:

[0097] S21, obtaining a fundus image training set;

[0098] Among them, the fundus image training set is a fundus image sample data set that has been labeled. The data labeling can be completed by a doctor expert or by a data labeling model.

[0099] S22, training the semantic segmentation model according to the fundus image training set;

[0100] The semantic segmentation model is an encoder-decoder network, in which the encoder downsamples the input, the decoder upsamples the output of the encoder, restores the resolution, and semantically projects the discriminable features learned by the encoder to the pixel space to obtain dense classification. The semantic segmentation model can be FCN, U-Net, SegNet, DeepLab, E-Net, Mask R-CNN, PSPNet, RefineNet, G-FRNet model, etc., and no specific restrictions are made here. The above model is trained according to the fundus image training set, and the trained semantic segmentation model is used to extract the spatial shape feature image of the fundus image to be tested, wherein the spatial shape feature refers to the spatial position or relative direction relationship between multiple blood vessels extracted from the fundus image to be tested, and these relationships can also be divided into connection / adjacency relationships, overlap / overlapping relationships, etc.

[0101] S23, extracting a first feature image set and a second feature image set from the fundus image training set based on a detection operator;

[0102] Among them, the detection operator is a tool for extracting image features. It can be understood that different detection operators are required to extract different image features. For example, if it is necessary to extract an edge feature image, the detection operator can be a Canny operator, a Sobel operator, a Lapace operator, a Prewitt operator, a Roberts operator, etc. If it is necessary to extract a texture feature image, the detection operator can be an LBP operator, a GLCM operator, a GLDS operator, a Gabort operator, etc. There is no restriction on the specific selection of the detection operator, as long as it meets the requirements for extracting the target feature image. Based on the detection operator, feature extraction is performed on the fundus image samples in the fundus image training set to obtain a first feature image set and a second feature image set, which are used for subsequent reconstruction of the fundus image training set, wherein the images in the first feature image set and the second feature image set represent two different types of features of the fundus image samples.

[0103] S24, reconstructing the fundus image training set according to the first feature image set, the second feature image set of the fundus image training set, and the spatial shape feature image set corresponding to the fundus image training set;

[0104] The feature images in the spatial shape feature image set corresponding to the first feature image set, the second feature image set and the fundus image training set reflect the different types of vascular features of the fundus image samples. The fundus image training set is reconstructed based on these feature sets, so that the fundus image samples have higher clarity and more obvious image features, which facilitates vascular identification.

[0105] S25. According to the reconstructed fundus image training set, the blood vessel segmentation model is trained and obtained.

[0106] The specific structure of the blood vessel segmentation model can be FCN, U-net, RNN model and its deformation, etc. The above model is trained according to the reconstructed fundus image training set to obtain a blood vessel segmentation model for blood vessel recognition in the image to be tested.

[0107] In this embodiment, a semantic segmentation model is trained according to a fundus image training set to extract the spatial shape feature image of the fundus image to be tested; a first feature image set and a second feature image set of the fundus image training set are extracted by a detection operator, and the fundus image training set is reconstructed according to the first feature image set, the second feature image set and the spatial shape feature image set corresponding to the fundus image training set, thereby improving the image clarity and making the graphic features more obvious; a vascular segmentation model is obtained by training the reconstructed fundus image training set, thereby improving the accuracy of vascular segmentation.

[0108] In one embodiment, before obtaining the fundus image training set, it also includes: obtaining an original fundus image sample set; performing image preprocessing on the original fundus image sample set to obtain a first sample set; the image preprocessing includes: at least one of rotation and flipping; adding the first sample set to the original fundus image sample set to obtain a second sample set; normalizing the second sample set to a uniform size to obtain the fundus image training set.

[0109] Since it is difficult to obtain medical image training data, the original fundus image training set is preprocessed by rotating and flipping, and adding the original image to expand the training set data and improve the accuracy of the model. By normalizing the second sample set, a fundus image training set of uniform size is obtained, which facilitates the training and learning of the semantic segmentation model.

[0110] In a second aspect, the present invention further provides a fundus image blood vessel recognition device. The fundus image blood vessel recognition device described below and the fundus image blood vessel recognition method described above can refer to each other.

[0111] like Figure 3 As shown, the present invention provides a device for identifying blood vessels in fundus images, including: an acquisition module 31 , a first extraction module 32 , a second extraction module 33 , a reconstruction module 34 , and a generation module 35 .

[0112] Among them, the acquisition module 31 is used to acquire the fundus image to be tested; the first extraction module 32 is used to extract the first feature image and the second feature image of the fundus image to be tested based on the detection operator; the second extraction module 33 is used to extract the spatial shape feature image of the fundus image to be tested based on the semantic segmentation model; the reconstruction module 34 is used to reconstruct the fundus image to be tested according to the first feature image, the second feature image and the spatial shape feature image; the generation module 35 is used to input the reconstructed fundus image to be tested into the vascular segmentation model to obtain the vascular segmentation image of the fundus image to be tested; wherein the semantic segmentation model is obtained by training according to the fundus image training set; the vascular segmentation model is obtained by training according to the fundus image training set reconstructed from the fundus image training set.

[0113] The fundus image blood vessel recognition device provided in this embodiment extracts the first feature image and the second feature image of the fundus image to be tested through a detection operator, extracts the spatial shape feature image of the fundus image to be tested through a semantic segmentation model, and reconstructs the fundus image to be tested based on the three types of feature images, thereby improving the image clarity and making the graphic features more obvious. By inputting the reconstructed fundus image into the blood vessel segmentation model for blood vessel recognition, a blood vessel segmentation image with higher segmentation accuracy is obtained.

[0114] like Figure 4 As shown, in one embodiment, the first extraction module 32 further includes: a multi-feature extraction module 321, an information entropy calculation module 322, and a feature determination module 323.

[0115] Among them, the multi-feature extraction module 321 is used to extract multiple types of feature images of the fundus image to be measured based on the detection operator; the information entropy calculation module 322 is used to calculate the information entropy of the multiple types of feature images of the fundus image to be measured respectively; the feature determination module 323 is used to determine the feature image corresponding to the maximum information entropy as the first feature image, and determine the feature image whose information entropy is second only to the maximum information entropy as the second feature image.

[0116] In this embodiment, the information entropy of different types of extracted feature images is calculated, and by comparing the information entropy values, feature images with better quality and clearer fundus image information to be tested are screened out for use in reconstructing the image to be tested, which can make the image to be tested clearer and the features more obvious.

[0117] like Figure 5 As shown, in one embodiment, the present invention provides a device for identifying blood vessels in fundus images, further comprising a noise reduction module 36 for inputting the blood vessel segmentation image of the fundus image to be tested into a random field model to obtain a noise-reduced blood vessel segmentation image.

[0118] This embodiment uses a random field model to consider the association information between different pixels in the pixel point space, thereby suppressing noise pixels with segmentation errors and enhancing the segmentation accuracy of blood vessels.

[0119] Figure 6 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 6 As shown, the electronic device may include: a processor 610, a communication interface 620, a memory 630 and a communication bus 640, wherein the processor 610, the communication interface 620 and the memory 630 communicate with each other through the communication bus 640. The processor 610 may call the logic instructions in the memory 630 to execute a method for recognizing blood vessels in a fundus image according to the first aspect, the method comprising: obtaining a fundus image to be tested; extracting a first feature image and a second feature image of the fundus image to be tested based on a detection operator; extracting a spatial shape feature image of the fundus image to be tested based on a semantic segmentation model; reconstructing the fundus image to be tested according to the first feature image, the second feature image and the spatial shape feature image; inputting the reconstructed fundus image to be tested into a blood vessel segmentation model to obtain a blood vessel segmentation image of the fundus image to be tested; wherein the semantic segmentation model is obtained by training according to fundus image training samples; and the blood vessel segmentation model is obtained by training according to fundus image training samples reconstructed from the fundus image training samples.

[0120] In addition, the logic instructions in the above-mentioned memory 630 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0121] In a third aspect, the present invention also provides a computer program product, comprising a computer program stored on a non-transitory computer-readable storage medium, wherein the computer program comprises program instructions. When the program instructions are executed by a computer, the computer is able to execute the method for identifying blood vessels in fundus images provided in the first aspect above.

[0122] In a fourth aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the method for identifying blood vessels in fundus images provided in the first aspect above.

[0123] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0124] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0125] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for identifying blood vessels in fundus images, characterized in that: include: Acquire the fundus image to be tested; Extracting a first characteristic image and a second characteristic image of the fundus image to be tested based on a detection operator; Extracting a spatial shape feature image of the fundus image to be measured based on a semantic segmentation model; Reconstructing the fundus image to be measured according to the first characteristic image, the second characteristic image and the spatial shape characteristic image; Inputting the reconstructed fundus image to be tested into a blood vessel segmentation model to obtain a blood vessel segmentation image of the fundus image to be tested; The semantic segmentation model is obtained by training based on a training set of fundus images; the vascular segmentation model is obtained by training based on a training set of reconstructed fundus images. The fundus image training set is obtained by: obtaining an original fundus image sample set; performing image preprocessing on the original fundus image sample set to obtain a first sample set; the image preprocessing includes: at least one of rotation and flipping; adding the first sample set to the original fundus image sample set to obtain a second sample set; normalizing the second sample set to a uniform size to obtain the fundus image training set. The method further comprises: after inputting the reconstructed fundus image to be tested into the blood vessel segmentation model to obtain the blood vessel segmentation image of the fundus image to be tested, inputting the blood vessel segmentation image of the fundus image to be tested into a random field model to obtain a denoised blood vessel segmentation image, The blood vessel segmentation model is a U-Net model, and the random field model is a denseCRF model.

2. The method for identifying blood vessels in fundus images according to claim 1, characterized in that: The extracting the first characteristic image and the second characteristic image of the fundus image to be tested based on the detection operator further includes: Extracting multiple types of feature images of the fundus image to be tested based on the detection operator; For the characteristic images of multiple types of the fundus image to be measured, respectively, calculating the information entropy; Take the feature image corresponding to the maximum information entropy as the first feature image; A feature image whose information entropy is second only to the maximum information entropy is taken as the second feature image.

3. The method for identifying blood vessels in fundus images according to claim 2, characterized in that: The plurality of types of feature images are at least two of grayscale feature images, texture feature images, edge feature images, and gradient feature images; Correspondingly, the extracting of multiple types of feature images of the fundus image to be tested based on the detection operator further includes: Performing grayscale conversion on the fundus image to be tested to obtain a grayscale feature image; Histogram equalization is performed on the grayscale feature image, and based on a detection operator, a texture feature image, an edge feature image or a gradient feature image of the fundus image to be tested is extracted.

4. The method for identifying blood vessels in fundus images according to claim 1, characterized in that: The reconstructing the fundus image to be measured according to the first characteristic image, the second characteristic image and the spatial shape characteristic image further comprises: The first characteristic image, the second characteristic image and the spatial shape characteristic image are respectively used as a component of the RGB three-channel of the fundus image to be measured, and image synthesis is performed to obtain a reconstructed fundus image to be measured.

5. The method for identifying blood vessels in fundus images according to claim 1, characterized in that: Before acquiring the fundus image to be measured, the method further comprises: Obtain a fundus image training set; According to the fundus image training set, the semantic segmentation model is trained; Based on the detection operator, extracting a first feature image set and a second feature image set of the fundus image training set; Reconstructing the fundus image training set according to the first feature image set, the second feature image set of the fundus image training set and the spatial shape feature image set corresponding to the fundus image training set; The blood vessel segmentation model is obtained by training according to the reconstructed fundus image training set.

6. A device for identifying blood vessels in fundus images, characterized in that: include: An acquisition module, used for acquiring the fundus image to be tested; A first extraction module, used for extracting a first characteristic image and a second characteristic image of the fundus image to be tested based on a detection operator; A second extraction module is used to extract a spatial shape feature image of the fundus image to be measured based on a semantic segmentation model; A reconstruction module, used for reconstructing the fundus image to be measured according to the first characteristic image, the second characteristic image and the spatial shape characteristic image; A generating module, used for inputting the reconstructed fundus image to be tested into a blood vessel segmentation model to obtain a blood vessel segmentation image of the fundus image to be tested; The semantic segmentation model is obtained by training based on a training set of fundus images; the vascular segmentation model is obtained by training based on a training set of reconstructed fundus images. The fundus image training set is obtained by: obtaining an original fundus image sample set; performing image preprocessing on the original fundus image sample set to obtain a first sample set; the image preprocessing includes: at least one of rotation and flipping; adding the first sample set to the original fundus image sample set to obtain a second sample set; normalizing the second sample set to a uniform size to obtain the fundus image training set. The generation module further inputs the reconstructed fundus image to be tested into the blood vessel segmentation model to obtain the blood vessel segmentation image of the fundus image to be tested, and then inputs the blood vessel segmentation image of the fundus image to be tested into the random field model to obtain the denoised blood vessel segmentation image. The blood vessel segmentation model is a U-Net model, and the random field model is a denseCRF model.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the method for identifying blood vessels in fundus images according to any one of claims 1 to 5 are implemented.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method for identifying blood vessels in fundus images according to any one of claims 1 to 5 are implemented.

Images