Eye ground hard exudate identification method and device and medium
By combining pre-trained convolutional neural network and support vector machine model, combined with technologies such as denoising, contrast enhancement and color space conversion, the fundus hard exudates were identified, which solved the problem of low recognition accuracy in the existing technology, and achieved the More efficient feature extraction and classification of hard exudate in the fundus.
Patent Information
- Application Number
- CN202411149693.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-21
- Publication Date
- 2025-01-07
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
In the prior art, when identifying fundus images, the recognition accuracy of fundus hard exudates is not high, and is greatly affected by external factors, making it difficult to effectively detect and segment.
The pre-trained fundus hard exudate convolutional neural network model is used to extract the fundus image to be measured, and it is used to identify it in combination with the support vector machine model. Color and texture are extracted through pre-processing steps such as denoising, contrast enhancement and color space conversion. and morphological characteristics, target candidate regions are determined using region growth algorithms and non-maximum suppression algorithms.
It improves the recognition accuracy of hard exudates in the fundus, enhances the extraction and classification capabilities of hard exudates, and reduces the noise impact during the identification process.
Smart Images
Figure CN119274225A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of medical image processing, and in particular to a method, device and medium for identifying fundus hard exudates. Background Art
[0002] Diabetic retinopathy is one of the most important complications of diabetes. Effective detection and segmentation of hard exudates (HE) in fundus images can help with early screening of diabetic retinopathy. Hard exudates are the result of lipid deposition in the retina, usually appearing as yellow or white dot-like lesions.
[0003] At present, when identifying fundus images, the accuracy of identifying hard exudates may be reduced due to the influence of some external factors (such as the quality of the collected image); therefore, it is necessary to provide a method for identifying fundus hard exudates to improve the accuracy of identifying hard exudates. Summary of the invention
[0004] In view of the problems existing in the prior art, the embodiments of the present invention provide a method, device and medium for identifying hard fundus exudates, so as to solve or partially solve the technical problem of low recognition accuracy of hard fundus exudates when identifying fundus images in the prior art.
[0005] A first aspect of the present invention provides a method for identifying fundus hard exudates, the method comprising:
[0006] Inputting the fundus image to be tested containing hard exudates into a pre-trained convolutional neural network model of fundus hard exudates to obtain a corresponding feature vector of the hard exudates to be tested; the convolutional neural network model of fundus hard exudates is trained based on a fundus image data set containing hard exudates;
[0007] The feature vector of the hard exudate to be tested is identified using a pre-trained support vector machine model to obtain the hard exudate area of the fundus image to be tested; the support vector machine model is trained based on a hard exudate feature vector data set, and the hard exudate feature vector data set is extracted from the fundus image data set using the fundus hard exudate convolutional neural network model.
[0008] In the above scheme, before inputting the fundus image to be tested containing hard exudates into the pre-trained fundus hard exudate convolutional neural network model, the method further includes:
[0009] Performing denoising on each fundus image data in the fundus image data set to obtain a first fundus image;
[0010] Performing contrast enhancement processing on the denoised fundus image to obtain a second fundus image;
[0011] Perform color space conversion on the second fundus image to obtain a third fundus image.
[0012] In the above solution, performing color space conversion on the second fundus image to obtain a third fundus image includes:
[0013] Extracting R color value, G color value and B color value of the second fundus image;
[0014] Performing color space conversion on the second fundus image using the formula V=max(R, G, B) to obtain the brightness V of the third fundus image;
[0015] Performing color space conversion on the second fundus image using the formula S=(V-min(R, G, B)) / V to obtain the saturation S of the third fundus image;
[0016] When it is determined that V=R, the color space conversion is performed on the second fundus image using the formula H=60*(GB) / (V-min(R,G,B)) to obtain the hue H of the third fundus image;
[0017] When it is determined that V=G, the color space conversion is performed on the second fundus image using the formula H=120+60*(BR) / (V-min(R,G,B)) to obtain the hue H of the third fundus image;
[0018] When it is determined that V=B, the color space conversion is performed on the second fundus image using the formula H=240+60*(RG) / (V-min(R,G,B)) to obtain the hue H of the third fundus image.
[0019] In the above scheme, before inputting the fundus image to be tested containing hard exudates into the pre-trained fundus hard exudate convolutional neural network model, the method further includes:
[0020] The pre-built convolutional neural network for fundus hard exudates is trained using the labeled fundus image dataset, including:
[0021] The pre-built convolutional neural network for fundus hard exudates is trained using the labeled fundus image dataset, including:
[0022] Using the fundus hard exudate convolutional neural network, image feature extraction is performed on each region of each fundus image in the training set to obtain color features, hard texture features and morphological features of each region;
[0023] Determine a target candidate region in the fundus image in the training set according to the color feature, texture feature and morphological feature of each region;
[0024] If it is determined that the target candidate area meets the loss function convergence condition of the fundus hard exudate convolutional neural network, it is determined that the fundus hard exudate convolutional neural network training is qualified, and the fundus hard exudate convolutional neural network model is output.
[0025] In the above scheme, determining the target candidate region in the fundus image in the training set according to the color feature, texture feature and morphological feature of each region includes:
[0026] Acquire a hard exudate texture feature corresponding to the hard exudate region, compare the hard exudate texture feature with the texture features of each region, and determine a region where the comparison result is greater than a preset threshold as a first candidate region where the hard exudate is located;
[0027] Screening the contours of all first candidate regions according to the preset hard exudate contour constraint conditions to determine the second candidate region that meets the hard exudate contour constraint conditions;
[0028] Acquire color features of hard exudate, and determine a segmentation threshold according to the color features of the hard exudate;
[0029] According to the segmentation threshold, the pixel points in the second candidate area whose pixel values are greater than the segmentation threshold are set to white, and the pixel points whose pixel values are less than or equal to the segmentation threshold are set to black; thereby obtaining a plurality of white areas;
[0030] Determine each of the white areas as a corresponding third candidate area;
[0031] Expanding each of the third candidate regions using a region growing algorithm to obtain a corresponding fourth candidate region;
[0032] Each of the fourth candidate regions is processed using a non-maximum suppression algorithm, and overlapping regions are deleted to obtain the target candidate region.
[0033] In the above solution, the method of using a region growing algorithm to expand each of the third candidate regions to obtain a corresponding fourth candidate region includes:
[0034] For any of the third candidate regions, one or more pixels are selected in the third candidate region as a seed point; based on the seed point, if it is determined that the similarity between the neighboring pixel points of the seed point and the seed point is greater than a similarity threshold, the neighboring pixel points and the seed point are merged into the same region;
[0035] The neighborhood pixel point is then used as a new seed point, and the above process is repeated until the growth stop condition is met, and the growth is stopped to obtain the fourth candidate region.
[0036] In the above solution, the non-maximum suppression algorithm is used to process each of the fourth candidate regions, and the overlapping regions are deleted to obtain the target candidate region, including:
[0037] Performing confidence scoring processing on each of the fourth candidate regions using the non-maximum suppression algorithm to obtain a confidence score of each fourth candidate region;
[0038] Determine the fourth candidate area with the highest confidence score as the reference area;
[0039] Each remaining fourth candidate region is traversed, and if it is determined that the overlapping area between the remaining fourth candidate region and the reference region is greater than a threshold, the remaining fourth candidate region is deleted. After the traversal is completed, the target candidate region is obtained.
[0040] In the above scheme, after determining the target candidate area in the preprocessed fundus image according to the hard exudate characteristics, the method further includes:
[0041] Performing boundary smoothing processing on each of the target candidate regions;
[0042] Determine the area and shape of each target candidate region after smoothing;
[0043] For any current target candidate region, if the area of the target candidate region is smaller than a preset area threshold, or it is determined that the shape of the target candidate region is an irregular shape, the current target candidate region is deleted from the multiple target candidate regions.
[0044] A second aspect of the present invention provides a device for identifying fundus hard exudates, the device comprising:
[0045] A first input unit is used to input the fundus image to be tested containing hard exudates into a pre-trained fundus hard exudate convolutional neural network model to obtain a corresponding hard exudate feature vector to be tested; the fundus hard exudate convolutional neural network model is trained based on a fundus image data set containing hard exudates;
[0046] The recognition unit is used to use a pre-trained support vector machine model to identify the hard exudate feature vector to be tested, so as to obtain the hard exudate area of the fundus image to be tested; the support vector machine model is trained based on a hard exudate feature vector data set, and the hard exudate feature vector data set is extracted from the fundus image data set using the fundus hard exudate convolutional neural network model.
[0047] According to a third aspect of the present invention, a computer-readable storage medium is disclosed, on which a computer program is stored, and when the program is executed by a processor, the steps of the above method are implemented.
[0048] According to a fourth aspect of the present invention, a computer device is disclosed, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the program.
[0049] The present invention provides a method, device and medium for identifying fundus hard exudates, comprising: inputting a fundus image to be tested containing hard exudates into a pre-trained convolutional neural network model of fundus hard exudates to obtain a corresponding feature vector of the hard exudates to be tested; the convolutional neural network model of fundus hard exudates is trained based on a fundus image data set containing hard exudates; using a pre-trained support vector machine model to identify the feature vector of the hard exudates to be tested, and obtaining a hard exudate area of the fundus image to be tested; the support vector machine model is trained based on the hard exudates. The hard exudate feature vector data set is obtained by training, and the hard exudate feature vector data set is extracted from the fundus image data set using the fundus hard exudate convolutional neural network model; in this way, the pre-trained fundus hard exudate convolutional neural network model can be used to extract the features of the fundus image to be tested, and then the support vector machine model can be used to classify the features of the fundus image to be tested. Since the fundus hard exudate convolutional neural network is good at extracting image features, the support vector machine model can accurately classify image features, and thus the recognition accuracy of fundus hard exudates can be ultimately improved. BRIEF DESCRIPTION OF THE DRAWINGS
[0050] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present invention. Moreover, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:
[0051] Figure 1 A schematic diagram of a method for identifying fundus hard exudates according to an embodiment of the present invention is shown;
[0052] Figure 2 A schematic diagram of the structure of a device for identifying fundus hard exudates according to an embodiment of the present invention is shown. DETAILED DESCRIPTION
[0053] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.
[0054] The present invention provides a method for identifying fundus hard exudates, such as Figure 1 As shown, the method mainly includes the following steps:
[0055] S110, inputting the fundus image to be tested containing hard exudates into a pre-trained fundus hard exudate convolutional neural network model to obtain a corresponding hard exudate feature vector to be tested; the fundus hard exudate convolutional neural network model is trained based on a fundus image data set containing hard exudates.
[0056] Before inputting the fundus image to be tested containing hard exudates into the pre-trained fundus hard exudate convolutional neural network model, the fundus hard exudate convolutional neural network needs to be trained first. Therefore, a fundus image data set containing hard exudates must be obtained first, and the fundus images in the fundus image data set must be preprocessed to obtain the preprocessed fundus images; and the hard exudates in the preprocessed fundus images must be marked to form a marked fundus image data set.
[0057] Specifically, it is necessary to collect fundus images containing hard exudates to form a fundus image dataset. In order to improve the recognition accuracy of fundus images, it is necessary to preprocess the fundus images in the fundus image dataset to reduce the noise in the fundus images, highlight the characteristics of hard exudates, and reduce the impact of noise on subsequent recognition.
[0058] The present invention mainly pre-processes the fundus image by performing denoising, contrast enhancement, and color space conversion on the fundus image. Then, in one embodiment, before inputting the fundus image to be tested containing hard exudates into the pre-trained fundus hard exudate convolutional neural network model, the method also includes:
[0059] Performing denoising on the fundus image data to obtain a first fundus image;
[0060] Performing contrast enhancement processing on the denoised fundus image to obtain a second fundus image;
[0061] Performing color space conversion on the second fundus image to obtain a third fundus image.
[0062] Specifically, the fundus image can be denoised by conventional methods such as mean filtering or Gaussian filtering to obtain a first fundus image (denoised image), and then the first fundus image can be contrast enhanced by a histogram equalization algorithm or a grayscale transformation method to obtain a second fundus image. Since the histogram equalization algorithm and the grayscale transformation method are conventional processing methods, they will not be described in detail here.
[0063] It is understandable that, since hard exudates are usually yellow or white, hard exudates may have a higher contrast in a specific color. Therefore, the present invention can perform color space conversion on the second fundus image to obtain a third fundus image, so as to better highlight the characteristics of the hard exudate. Among them, the color conversion of the second fundus image can be performed by using a color conversion method of RGB to HSV or a color conversion method of RGB to Lab. The present invention mainly uses a color conversion method of RGB to HSV to perform color space conversion on the second fundus image, which is implemented as follows:
[0064] Extracting R color value, G color value and B color value of the second fundus image;
[0065] The second fundus image is converted into a color space using the formula V=max(R, G, B) to obtain the brightness V of the third fundus image;
[0066] The second fundus image is converted into a color space using the formula S=(V-min(R, G, B)) / V to obtain a saturation S of the third fundus image;
[0067] When V=R is determined, the color space conversion is performed on the second fundus image using the formula H=60*(GB) / (V-min(R,G,B)) to obtain the hue H of the third fundus image;
[0068] When V=G is determined, the color space conversion is performed on the second fundus image using the formula H=120+60*(BR) / (V-min(R,G,B)) to obtain the hue H of the third fundus image;
[0069] When it is determined that V=B, the color space conversion is performed on the second fundus image using the formula H=240+60*(RG) / (V-min(R,G,B)) to obtain the hue H of the third fundus image.
[0070] In this way, the fundus image in RGB format is preprocessed and converted into the third fundus image in HSV format.
[0071] Then, the hard exudate regions in each third fundus image (preprocessed fundus image) are marked to obtain a marked fundus image data set.
[0072] The labeled fundus image dataset can then be used to train a pre-constructed fundus hard exudate convolutional neural network.
[0073] The convolutional neural network for fundus hard exudates pre-constructed in the present invention includes: an input layer, a convolutional layer, a pooling layer, a first fully connected layer, and a second fully connected layer. The first fully connected layer is used to splice the hard exudate features output by the pooling layer to form a complete feature vector; the second fully connected layer is mainly used to classify the complete feature vector to obtain the background area and the hard exudate area.
[0074] The present invention needs to use the labeled fundus image data set to train the pre-constructed fundus hard exudate convolutional neural network until the fundus hard exudate convolutional neural network is qualified.
[0075] In one embodiment, before inputting the fundus image to be tested containing hard exudates into the pre-trained fundus hard exudate convolutional neural network model, the method further includes:
[0076] The pre-constructed fundus hard exudate convolutional neural network is trained using the labeled fundus image data set, including: using the fundus hard exudate convolutional neural network to extract image features from each region of each fundus image in the training set to obtain color features, hard texture features, and morphological features of each region;
[0077] Determine the target candidate region in the fundus image in the training set according to the color feature, texture feature and morphological feature of each region;
[0078] If it is determined that the target candidate area meets the loss function convergence condition of the fundus hard exudate convolutional neural network, then the fundus hard exudate convolutional neural network training is determined to be qualified, and the fundus hard exudate convolutional neural network model is output.
[0079] Specifically, the labeled fundus image dataset can be divided into a training set and a validation set, so that the training set can be used to train the fundus hard exudate convolutional neural network, and the test set can be used to verify the training effect.
[0080] When the convolutional neural network for fundus hard exudates is trained using the training set, the image features of each region of each fundus image in the training set are extracted by the convolutional neural network for fundus hard exudates, and then the target candidate region (hard exudate prediction region) is determined in multiple regions according to the color features, texture features and morphological features of each region and the color features, texture features and morphological features of the pre-set hard exudate region. If the loss value between the hard exudate prediction region and the pre-marked hard exudate region in the marked image data set is less than or equal to the preset loss threshold, it is determined that the target candidate region meets the loss function convergence condition of the convolutional neural network for fundus hard exudates, and it is determined that the convolutional neural network for fundus hard exudates is qualified.
[0081] Among them, according to the formula Determine the loss value LOSS between the target candidate area and the pre-marked hard exudate area; N is the number of training samples in the training set, n is the training sample sequence number in the training set, A p_n is the pixel value of the target candidate area corresponding to the nth training sample, B t_n is the true pixel value of the hard exudate area marked in the nth training sample.
[0082] In the present invention, the color features of objects in each area of the fundus image of the training set can be extracted by means of a color histogram, as follows:
[0083] The preprocessed image is an HSV image, which includes an H channel image, an S channel image, and a V channel image. The color histograms of the three channels are determined in the same way. Here we take the H channel as an example:
[0084] For the H channel image, the H channel image can be divided into multiple color intervals (bins), usually 256 intervals. For each pixel in the H channel image, the pixel value of each pixel is obtained, and then the color interval described by the pixel value is determined. In this way, the number of pixels contained in each color interval can be counted, and the color histogram of the H channel can be drawn according to the number of pixels. The color histogram is normalized to eliminate the influence of the image scale; the normalized color histogram is spliced into a feature vector as the feature representation of the H channel image.
[0085] Furthermore, the gray level co-occurrence matrix can be used to extract the texture features of each region of the fundus image of the training set, as follows:
[0086] Gray-Level Co-occurrence Matrix (GLCM) is a commonly used image texture feature extraction technology. It is a matrix that describes the relationship between the gray levels of pixels in fundus images. It extracts the texture features of fundus images by counting the frequency of occurrence and spatial relationship of the gray values of adjacent pixels.
[0087] First, the fundus images in the training set are converted into grayscale images, and then the grayscale co-occurrence matrix parameters (including the number of grayscale levels, the distance and direction of the grayscale co-occurrence matrix) are set. The number of grayscale levels indicates how many levels the grayscale value is divided into, the distance indicates the distance between pixel pairs when calculating the grayscale co-occurrence matrix, and the direction indicates the direction between pixel pairs when calculating the grayscale co-occurrence matrix. Finally, according to the window defined by the selected direction and distance, slide on the fundus image, and for each current pixel in each window, determine the probability of the grayscale combination formed by the current pixel and the adjacent pixel points, and these probabilities are saved in the grayscale co-occurrence matrix. Based on the grayscale co-occurrence matrix, call the extraction function in OpenCV to extract various texture features of the fundus images in the training set, such as contrast, energy, correlation, and uniformity.
[0088] Then, various morphological features in the fundus images can be extracted by performing opening or closing operations on each region in the training set fundus images; the opening operation is to first perform an erosion operation on each region of the fundus image and then perform an expansion operation; the closing operation is to first perform an expansion operation on each region of the fundus image and then perform an erosion operation. The closing operation is used as an example for explanation, as follows:
[0089] The fundus images in the training set are subjected to dilation operations, and the preset structural elements and the fundus images in the training set are subjected to bitwise operations to eliminate edges and small connected areas. The dilation operation is used to merge adjacent pixels into a connected area, which helps to expand the area of the area.
[0090] The erosion operation also requires a structural element to be preset, and the structural element is used to perform an “AND” operation with the fundus image, so as to fill holes in the image, smooth edges, and connect broken areas.
[0091] After the texture features, morphological features and color features of each region are extracted, the hard exudate region can be determined from multiple regions according to the texture features, morphological features and color features of each region, specifically including:
[0092] Acquire a hard exudate texture feature corresponding to the hard exudate region, compare the hard exudate texture feature with the texture features of each region, and determine a region where the comparison result is greater than a preset threshold as a first candidate region where the hard exudate is located;
[0093] Screening the contours of all first candidate regions according to the preset hard exudate contour constraint conditions to determine the second candidate region that meets the hard exudate contour constraint conditions;
[0094] Acquire color features of hard exudate, and determine a segmentation threshold according to the color features of the hard exudate;
[0095] According to the segmentation threshold, the pixel points in the second candidate area whose pixel values are greater than the segmentation threshold are set to white, and the pixel points whose pixel values are less than or equal to the segmentation threshold are set to black; thereby obtaining a plurality of white areas;
[0096] Determine each of the white areas as a corresponding third candidate area;
[0097] Expanding each of the third candidate regions using a region growing algorithm to obtain a corresponding fourth candidate region;
[0098] Each of the fourth candidate regions is processed using a non-maximum suppression algorithm, and overlapping regions are deleted to obtain the target candidate region.
[0099] Generally speaking, the texture features corresponding to the hard exudate area are more complex, so the hard exudate texture features corresponding to the hard exudate area can be obtained in advance, and the hard exudate texture features are compared with the texture features of each area, and the area with a comparison result greater than a preset threshold is first determined as the first candidate area where the hard exudate is located. Similarly, since the contour shape of the hard exudate area is also special compared to the contour of the ordinary area, the contours of all the first candidate areas can be screened according to the preset hard exudate contour constraint conditions to determine the second candidate area that meets the hard exudate contour constraint conditions.
[0100] Finally, the second candidate area is further screened according to the color features, and the hard exudate area is determined from the second candidate area. Since hard exudates are all white or yellow, the preset hard exudate color features can be obtained first. The texture features, hard exudate contour constraints, and hard exudate color features corresponding to the hard exudate area are extracted in advance based on the marked hard exudate area in the fundus image data set.
[0101] Determine the segmentation threshold according to the color characteristics of the hard exudate;
[0102] According to the segmentation threshold, the pixel points in the second candidate area whose pixel values are greater than the segmentation threshold are set to white, and the pixel points whose pixel values are less than or equal to the segmentation threshold are set to black; thereby obtaining multiple white areas;
[0103] Determine each white area as a corresponding third candidate area;
[0104] Using a region growing algorithm to expand each third candidate region to obtain a corresponding fourth candidate region;
[0105] Each fourth candidate region is processed using a non-maximum suppression algorithm, and overlapping regions are deleted to obtain the target candidate region.
[0106] Since in most cases, the target object and the background have different colors, it is more intuitive to define the white area as the candidate area. And the white area is often regarded as part of the foreground or the target. Therefore, using the white area as the candidate area also helps to extract the target of interest. The advantage of doing so is that possible candidate areas can be quickly locked, the amount of calculation for subsequent processing can be reduced, and the efficiency of determining the target candidate area can be improved.
[0107] In one implementation, each third candidate region is expanded using a region growing algorithm to obtain a corresponding fourth candidate region, including:
[0108] For any third candidate region, one or more pixels are selected in the third candidate region as seed points; based on the seed point, if it is determined that the similarity between the neighboring pixel points of the seed point and the seed point is greater than a similarity threshold, the neighboring pixel points and the seed point are merged into the same region;
[0109] The neighboring pixel point is used as a new seed point, and the above process is repeated until the growth stop condition is met, and the growth stops to obtain the fourth candidate region. The growth stop condition may include reaching a preset growth number or reaching a preset growth area.
[0110] In one implementation, each fourth candidate region is processed using a non-maximum suppression algorithm, and overlapping regions are deleted to obtain a target candidate region, including:
[0111] Perform confidence scoring processing on each fourth candidate region using a non-maximum suppression algorithm to obtain a confidence score for each fourth candidate region;
[0112] Determine the fourth candidate area with the highest confidence score as the reference area;
[0113] Each remaining fourth candidate region is traversed, and if it is determined that the overlapping area between the remaining fourth candidate region and the reference region is greater than a threshold, the remaining fourth candidate region is deleted. After the traversal is completed, the target candidate region is obtained.
[0114] In one embodiment, after determining the target candidate region in the preprocessed fundus image according to the hard exudate feature, the method further includes:
[0115] Perform boundary smoothing on each target candidate area;
[0116] Determine the area and shape of each target candidate region after smoothing;
[0117] For any current target candidate region, if the area of the target candidate region is smaller than a preset area threshold, or the shape of the target candidate region is determined to be an irregular shape, the current target candidate region is deleted from the multiple target candidate regions.
[0118] Then, the second fully connected layer in the fundus hard exudate convolutional neural network is used to classify multiple target candidate areas to obtain background areas and hard exudate prediction areas. As described above, if the loss value between the hard exudate prediction area and the pre-marked hard exudate area in the marked image data set is less than or equal to the preset loss threshold, it is determined that the target candidate area meets the loss function convergence condition of the fundus hard exudate convolutional neural network, and the fundus hard exudate convolutional neural network training is determined to be qualified.
[0119] If the loss value between the target hard exudate prediction area and the pre-marked hard exudate area is greater than the preset loss threshold, it is necessary to optimize the weight parameters in the convolutional layer of the fundus hard exudate convolutional neural network until it is determined that the fundus hard exudate convolutional neural network training is qualified and the fundus hard exudate convolutional neural network model is output.
[0120] Finally, the fundus image to be tested containing hard exudates can be input into the fundus hard exudate convolutional neural network model, and the corresponding hard exudate feature vector to be tested can be extracted from the first fully connected layer of the fundus hard exudate convolutional neural network model.
[0121] S111, using a pre-trained support vector machine model to identify the feature vector of the hard exudate to be tested, and obtain the hard exudate area of the fundus image to be tested; the support vector machine model is trained based on the hard exudate feature vector data set, and the hard exudate feature vector data set is extracted from the fundus image data set using the fundus hard exudate convolutional neural network model.
[0122] In order to improve the classification accuracy of hard exudate feature vectors, the present invention does not directly use the fundus hard exudate convolutional neural network to identify the hard exudate feature vectors, but uses a pre-built support vector machine model to identify and classify the hard exudate feature vectors.
[0123] Then, similarly, it is also necessary to use multiple hard exudate feature vectors as feature vector training sets of the support vector machine model, and set parameters such as the kernel function and penalty coefficient of the support vector machine model to realize the training of the support vector machine model. The support vector machine model is trained based on the hard exudate feature vector data set, and the hard exudate feature vector data set is extracted from the fundus image data set using the fundus hard exudate convolutional neural network model.
[0124] Specifically, when it is determined that the fundus hard exudate convolutional neural network training is qualified, the weight parameters in the convolutional layer of the fundus hard exudate convolutional neural network are retained. Since the first fully connected layer of the fundus hard exudate convolutional neural network outputs a hard exudate feature vector, the fundus hard exudate convolutional neural network model is used again to identify the fundus image data set containing hard exudates. Since there are multiple training samples in the fundus image data set, the first fully connected layer can output multiple hard exudate feature vectors, and the corresponding hard exudate feature vectors are obtained to form a hard exudate feature vector data set.
[0125] Similarly, the hard exudate feature vector data set is divided into a training set and a test set, and the support vector machine model is trained using the hard exudate feature vector training set until it is determined that the support vector machine model training is qualified.
[0126] After determining that the support vector machine model training is qualified, further, the above test set is input into the convolutional neural network of fundus hard exudates to obtain the corresponding test hard exudate feature vector, and form a feature vector test set. The support vector machine model is tested using the feature vector test set, and the performance of the support vector machine model is evaluated by calculating the classification accuracy, recall rate and other indicators of the support vector machine model for the feature vector test set. If it is determined that the performance of the support vector machine model does not meet expectations, the support vector machine model can be adjusted by adjusting parameters such as the penalty coefficient and the kernel function, and then the adjusted support vector machine model is trained again until the training accuracy and test accuracy meet expectations.
[0127] The trained support vector machine model can be used to identify the fundus images to be tested, including:
[0128] The feature vector of the hard exudate to be tested is input into the support vector machine model to obtain the hard exudate area and the noise area.
[0129] In this way, by using the fundus image data set containing hard exudates to train the fundus hard exudate convolutional neural network + support vector machine model, after the training is qualified, the fundus hard exudate convolutional neural network can be used to extract the features of the fundus image to be tested, and then the support vector machine model can be used to classify the features of the fundus image to be tested. Since the fundus hard exudate convolutional neural network is good at extracting image features and the support vector machine model can accurately classify image features, it can ultimately improve the recognition accuracy of fundus hard exudates.
[0130] Based on the same inventive concept as in the above-mentioned embodiment, this embodiment also provides a method and device for identifying fundus hard exudates, such as Figure 2 As shown, the device comprises:
[0131] The first input unit 21 is used to input the fundus image to be tested containing hard exudates into a pre-trained convolutional neural network model of fundus hard exudates to obtain a corresponding feature vector of the hard exudates to be tested; the convolutional neural network model of fundus hard exudates is trained based on a fundus image data set containing hard exudates;
[0132] The identification unit 22 is used to use a pre-trained support vector machine model to identify the hard exudate feature vector to be tested, and obtain the hard exudate area of the fundus image to be tested; the support vector machine model is trained based on the hard exudate feature vector data set, and the hard exudate feature vector data set is extracted from the fundus image data set using the fundus hard exudate convolutional neural network model.
[0133] Since the device introduced in the embodiment of the present invention is a device used to implement the method for identifying fundus hard exudates in the embodiment of the present invention, based on the method introduced in the embodiment of the present invention, the person skilled in the art can understand the specific structure and deformation of the device, so it is not repeated here. All devices used in the method of the embodiment of the present invention belong to the scope of protection of the present invention.
[0134] Through one or more embodiments of the present invention, the present invention has the following beneficial effects or advantages:
[0135] The present invention provides a method, device and medium for identifying fundus hard exudates, comprising: inputting a fundus image to be tested containing hard exudates into a pre-trained fundus hard exudate convolutional neural network model to obtain a corresponding feature vector of the hard exudate to be tested; the fundus hard exudate convolutional neural network model is trained based on a fundus image data set containing hard exudates; using a pre-trained support vector machine model to identify the feature vector of the hard exudate to be tested to obtain a hard exudate area of the fundus image to be tested; the support vector machine model is trained based on the hard exudate feature vector data set, and the hard exudate feature vector is trained based on the hard exudate feature vector data set. The vector data set is obtained by extracting the fundus image data set using the fundus hard exudate convolutional neural network model; in this way, the fundus hard exudate convolutional neural network + support vector machine model is trained by using the fundus image data set containing hard exudates. After qualified training, the fundus hard exudate convolutional neural network can be used to extract the features of the fundus image to be tested, and then the support vector machine model can be used to classify the features of the fundus image to be tested. Since the fundus hard exudate convolutional neural network is good at extracting image features, the support vector machine model can accurately classify image features, so it can be achieved that the recognition accuracy of fundus hard exudates can be ultimately improved.
[0136] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.
[0137] The above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.
Claims
1. A method for identifying fundus hard exudates, characterized in that: The method comprises: Inputting the fundus image to be tested containing hard exudates into a pre-trained convolutional neural network model of fundus hard exudates to obtain a corresponding feature vector of the hard exudates to be tested; the convolutional neural network model of fundus hard exudates is trained based on a fundus image data set containing hard exudates; The feature vector of the hard exudate to be tested is identified using a pre-trained support vector machine model to obtain the hard exudate area of the fundus image to be tested; the support vector machine model is trained based on a hard exudate feature vector data set, and the hard exudate feature vector data set is extracted from the fundus image data set using the fundus hard exudate convolutional neural network model.
2. The method according to claim 1, characterized in that Before inputting the fundus image to be tested containing hard exudates into the pre-trained fundus hard exudate convolutional neural network model, the method further includes: Performing denoising on each fundus image data in the fundus image data set to obtain a first fundus image; Performing contrast enhancement processing on the denoised fundus image to obtain a second fundus image; Perform color space conversion on the second fundus image to obtain a third fundus image.
3. The method according to claim 2, characterized in that The performing color space conversion on the second fundus image to obtain a third fundus image includes: Extracting R color value, G color value and B color value of the second fundus image; Performing color space conversion on the second fundus image using the formula V=max(R, G, B) to obtain the brightness V of the third fundus image; Performing color space conversion on the second fundus image using the formula S=(V-min(R, G, B)) / V to obtain the saturation S of the third fundus image; When it is determined that V=R, the color space conversion is performed on the second fundus image using the formula H=60*(GB) / (V-min(R,G,B)) to obtain the hue H of the third fundus image; When it is determined that V=G, the color space conversion is performed on the second fundus image using the formula H=120+60*(BR) / (V-min(R,G,B)) to obtain the hue H of the third fundus image; When it is determined that V=B, the color space conversion is performed on the second fundus image using the formula H=240+60*(RG) / (V-min(R,G,B)) to obtain the hue H of the third fundus image.
4. The method according to claim 1, characterized in that Before inputting the fundus image to be tested containing hard exudates into the pre-trained fundus hard exudate convolutional neural network model, the method further includes: The pre-built convolutional neural network for fundus hard exudates is trained using the labeled fundus image dataset, including: The fundus hard exudate convolutional neural network is used to extract image features from each region of each fundus image in the training set to obtain color features, hard texture features and morphological features of each region; Determine a target candidate region in the fundus image in the training set according to the color feature, texture feature and morphological feature of each region; If it is determined that the target candidate area meets the loss function convergence condition of the fundus hard exudate convolutional neural network, it is determined that the fundus hard exudate convolutional neural network training is qualified, and the fundus hard exudate convolutional neural network model is output.
5. The method according to claim 4, characterized in that The step of determining the target candidate region in the fundus image of the training set according to the color feature, texture feature and morphological feature of each region includes: Acquire a hard exudate texture feature corresponding to the hard exudate region, compare the hard exudate texture feature with the texture features of each region, and determine a region where the comparison result is greater than a preset threshold as a first candidate region where the hard exudate is located; Screening the contours of all first candidate regions according to the preset hard exudate contour constraint conditions to determine the second candidate region that meets the hard exudate contour constraint conditions; Acquire color features of hard exudate, and determine a segmentation threshold according to the color features of the hard exudate; According to the segmentation threshold, the pixel points in the second candidate area whose pixel values are greater than the segmentation threshold are set to white, and the pixel points whose pixel values are less than or equal to the segmentation threshold are set to black; thereby obtaining a plurality of white areas; Determine each of the white areas as a corresponding third candidate area; Expanding each of the third candidate regions using a region growing algorithm to obtain a corresponding fourth candidate region; Each of the fourth candidate regions is processed using a non-maximum suppression algorithm, and overlapping regions are deleted to obtain the target candidate region.
6. The method according to claim 5, characterized in that The method of expanding each of the third candidate regions using a region growing algorithm to obtain a corresponding fourth candidate region includes: For any of the third candidate regions, one or more pixels are selected in the third candidate region as a seed point; based on the seed point, if it is determined that the similarity between the neighboring pixel points of the seed point and the seed point is greater than a similarity threshold, the neighboring pixel points and the seed point are merged into the same region; The neighborhood pixel point is then used as a new seed point, and the above process is repeated until the growth stop condition is met, and the growth is stopped to obtain the fourth candidate region.
7. The method according to claim 5, characterized in that The method of processing each of the fourth candidate regions by using a non-maximum suppression algorithm, deleting overlapping regions, and obtaining a target candidate region includes: Performing confidence scoring processing on each of the fourth candidate regions using the non-maximum suppression algorithm to obtain a confidence score of each fourth candidate region; Determine the fourth candidate area with the highest confidence score as the reference area; Each remaining fourth candidate region is traversed, and if it is determined that the overlapping area between the remaining fourth candidate region and the reference region is greater than a threshold, the remaining fourth candidate region is deleted. After the traversal is completed, the target candidate region is obtained.
8. The method according to claim 4, characterized in that After determining the target candidate area in the preprocessed fundus image according to the hard exudate feature, the method further includes: Performing boundary smoothing processing on each of the target candidate regions; Determine the area and shape of each target candidate region after smoothing; For any current target candidate region, if the area of the target candidate region is smaller than a preset area threshold, or it is determined that the shape of the target candidate region is an irregular shape, the current target candidate region is deleted from the multiple target candidate regions.
9. A device for identifying fundus hard exudates, characterized in that: The device comprises: A first input unit is used to input the fundus image to be tested containing hard exudates into a pre-trained fundus hard exudate convolutional neural network model to obtain a corresponding hard exudate feature vector to be tested; the fundus hard exudate convolutional neural network model is trained based on a fundus image data set containing hard exudates; The recognition unit is used to use a pre-trained support vector machine model to identify the hard exudate feature vector to be tested, so as to obtain the hard exudate area of the fundus image to be tested; the support vector machine model is trained based on a hard exudate feature vector data set, and the hard exudate feature vector data set is extracted from the fundus image data set using the fundus hard exudate convolutional neural network model.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.