Diabetic retinopathy hard exudate segmentation method and system

Abstract

The invention provides a diabetic retinopathy hard exudate segmentation method and system, and relates to the technical field of image segmentation, and the method comprises the steps: determining the image complexity of a color fundus image; segmenting the color eye fundus image into a plurality of exudation areas of the hard exudate according to the distribution characteristics of the hard exudate in the diabetic retina; determining the edge contrast of each exudation area, and determining the lesion heterogeneity index after the diabetic retinopathy according to all the edge contrast and the image complexity; determining a convolution fusion boundary of all the exudation regions through lesion heterogeneity indexes and boundary characteristics of all the exudation regions, and performing convolution fusion segmentation on all the exudation regions of the hard exudate based on the convolution fusion boundary to obtain a plurality of segmented regions of the hard exudate in the color fundus image, according to the method, the convolution fusion segmentation of the diabetic retinopathy hard exudate can be realized, so that the segmentation precision of the hard exudate after the diabetic retinopathy can be improved.

Description

Technical Field

[0001] The present application relates to the technical field of image segmentation, and more specifically, to a method and system for segmenting hard exudates of diabetic retinopathy. Background Art

[0002] Diabetic retinopathy is an eye disease caused by diabetes. Hard exudates are one of the main manifestations of DR. They are formed by fat and protein deposits in the retina and mark the progression of the disease. Therefore, timely and accurate detection and segmentation of hard exudates are crucial for disease assessment and treatment plan formulation. In recent years, with the rapid development of medical imaging technology and deep learning algorithms, hard exudate detection methods based on image segmentation have been widely studied. Image segmentation technology can accurately segment different areas in medical images and extract information about the lesion area, which helps doctors quickly identify and diagnose diseases.

[0003] Traditional convolutional neural networks have limitations in extracting complex edge and detail information, especially in the segmentation of hard exudates, where subtle visual features are easily overlooked. The convolution fusion method combines multiple layers of features to enable the network to integrate information at different levels and extract key features more accurately. At the same time, diabetic retinal images often contain noise or complex backgrounds, which will interfere with the detection of hard exudates. Convolution fusion segmentation helps to enhance the separation of signals and noise and improve the stability of segmentation results in complex scenes through multi-channel feature fusion. Therefore, how to achieve convolution fusion segmentation of hard exudates in diabetic retinopathy and thus improve the segmentation accuracy of hard exudates after diabetic retinopathy is a difficult problem faced by the industry. Summary of the invention

[0004] The present application provides a method and system for segmenting hard exudates of diabetic retinopathy, which can realize convolution fusion segmentation of hard exudates of diabetic retinopathy, thereby improving the segmentation accuracy of hard exudates after diabetic retinopathy.

[0005] In a first aspect, the present application provides a diabetic retinopathy hard exudate segmentation method, the segmentation method comprising the following steps:

[0006] Acquiring a color fundus image after diabetic retinopathy, and then determining the image complexity of the color fundus image;

[0007] Segmenting the color fundus image into multiple exudate areas of hard exudate according to the distribution characteristics of hard exudate in diabetic retina;

[0008] The edge contrast of each exudate area is determined based on the attention mechanism of the convolutional neural network combined with the grayscale gradient between the hard exudate and the background area in the color fundus image, and the pathological variation heterogeneity index after diabetic retinopathy is determined through all edge contrasts and the image complexity;

[0009] The convolution fusion boundary of all exudate areas is determined by the pathological variation heterogeneity index and the boundary characteristics of each exudate area, and all exudate areas of hard exudate are segmented by convolution fusion based on the convolution fusion boundary to obtain multiple segmented areas of hard exudate in the color fundus image.

[0010] In this embodiment, determining the image complexity of the color fundus image specifically includes:

[0011] Performing grayscale conversion on the color fundus image to obtain a grayscale image;

[0012] Converting the grayscale image into a gray-level co-occurrence matrix;

[0013] The image complexity of the color fundus image is determined by the gray level co-occurrence matrix.

[0014] In this embodiment, segmenting the color fundus image into multiple exudate regions of hard exudate according to the distribution characteristics of hard exudate in diabetic retina specifically includes:

[0015] Identify all exudation locations of the color fundus image based on a data annotation mechanism;

[0016] All hard exudate locations were screened out from all exudate locations according to their distribution characteristics in the diabetic retina;

[0017] Multiple exudate areas of hard exudate are determined based on all hard exudate locations.

[0018] In this embodiment, the edge contrast of each exudate area is determined based on the attention mechanism of the convolutional neural network combined with the grayscale gradient between the hard exudate and the background area in the color fundus image, specifically including:

[0019] determining a grayscale gradient between a hard exudate and a background area in the color fundus image;

[0020] For each seepage area, obtaining a grayscale difference value between the seepage area and the background area from the grayscale gradient;

[0021] The influence weight of grayscale difference in the exudation area is determined based on the attention mechanism of convolutional neural network;

[0022] The edge contrast of the seepage area is determined by the influence weight and the grayscale difference value, and then the edge contrast of each seepage area is determined.

[0023] In this embodiment, determining the pathological heterogeneity index after diabetic retinopathy through all edge contrasts and the image complexity specifically includes:

[0024] For each exudative area, a lesion statistical model based on support vector machine is initialized;

[0025] Determine a statistical function of the lesion statistical model according to the image complexity and the edge contrast of the exudation area;

[0026] The lesion statistical model is used to perform fusion statistics on the exudative area after diabetic retinopathy to obtain the lesion variation heterogeneity of the exudative area, and then the lesion variation heterogeneity of each exudative area is obtained;

[0027] The heterogeneity index of lesion after diabetic retinopathy was determined by all lesion heterogeneity.

[0028] In this embodiment, the hard exudate is a spot material formed by the deposition of fat and protein in the retina.

[0029] In this embodiment, the convolution fusion boundaries of all exudative regions are determined by using the pathological variation heterogeneity index and the boundary characteristics of each exudative region, specifically including:

[0030] For each seepage area, the boundary characteristics of the seepage area are obtained, and then the fusion weight of the seepage area is determined;

[0031] Determining the boundary fusion value of the seepage area by using the fusion weight and the boundary characteristic, and then obtaining the boundary fusion value of each seepage area;

[0032] Perform convolution operation on all boundary fusion values ​​to obtain the convolution fusion boundaries of all leakage areas.

[0033] In this embodiment, all exudate regions of hard exudate are segmented by convolution fusion based on the convolution fusion boundary, and the multiple segmented regions of hard exudate in the color fundus image are obtained, which specifically include:

[0034] Convolution kernels of various scales are determined according to the shape characteristics of hard exudates;

[0035] Determining the fusion weights of convolution kernels of various scales through the convolution fusion boundary;

[0036] All the exudate regions are fused into a plurality of segmented regions of hard exudate in the color fundus image according to all the fusion weights.

[0037] In this embodiment, a high-resolution fundus camera is used to collect color fundus images after diabetic retinopathy.

[0038] In a second aspect, the present application provides a diabetic retinopathy hard exudate segmentation system for executing a diabetic retinopathy hard exudate segmentation method, the segmentation system comprising:

[0039] An image acquisition module, used for acquiring a color fundus image after diabetic retinopathy, and then determining the image complexity of the color fundus image;

[0040] An initial segmentation module, configured to segment the color fundus image into a plurality of exudate regions of hard exudate according to the distribution characteristics of hard exudate in diabetic retina;

[0041] A heterogeneity assessment module, for determining the edge contrast of each exudate area based on the attention mechanism of a convolutional neural network combined with the grayscale gradient between the hard exudate and the background area in the color fundus image, and determining the heterogeneity index of the lesion after diabetic retinopathy through all edge contrasts and the image complexity;

[0042] A fusion segmentation module is used to determine the convolution fusion boundary of all exudate areas according to the pathological variation heterogeneity index and the boundary characteristics of each exudate area, and perform convolution fusion segmentation on all exudate areas of hard exudate based on the convolution fusion boundary to obtain multiple segmented areas of hard exudate in the color fundus image.

[0043] The technical solution provided by the embodiments disclosed in this application has the following beneficial effects:

[0044] The color fundus image after diabetic retinopathy is collected, and the image complexity of the color fundus image is determined; the color fundus image is segmented into multiple exudate areas of hard exudate according to the distribution characteristics of hard exudates in the diabetic retina; the edge contrast of each exudate area is determined based on the attention mechanism of the convolutional neural network combined with the grayscale gradient between the hard exudate and the background area in the color fundus image, and the lesion variation heterogeneity index after diabetic retinopathy is determined through all edge contrasts and the image complexity; the convolution fusion boundary of all exudate areas is determined through the lesion variation heterogeneity index and the boundary characteristics of each exudate area, and all exudate areas of the hard exudate are convolutionally fused and segmented based on the convolution fusion boundary to obtain multiple segmented areas of the hard exudate in the color fundus image.

[0045] It can be seen that in the present application, the convolution fusion boundaries of all exudate areas are determined by the pathological heterogeneity index and the boundary characteristics of each exudate area, and all exudate areas of hard exudate are convolutionally fused and segmented based on the convolution fusion boundary to obtain multiple segmented areas of hard exudate in the color fundus image; firstly, the areas with hard exudate characteristics can be accurately extracted through data annotation and feature-based screening, thereby avoiding unnecessary noise interference. Among them, combining the image complexity with the boundary characteristics of the exudate area can make the segmented area more targeted and reliable, and can provide accurate input data for subsequent convolution fusion segmentation, so that the convolution network can focus more on relevant areas during processing, which helps to reduce The misclassification and omission phenomena are eliminated to ensure that the convolution fusion segmentation algorithm can perform a refined analysis of hard exudates, thereby improving the segmentation accuracy and detection effectiveness; then, the attention mechanism and grayscale gradient analysis based on convolutional neural networks are used to help evaluate the edge characteristics of different exudate areas and reveal the differences in edge characteristics in color fundus images. The lesion heterogeneity index reflects the changing characteristics of hard exudates in different regions, and provides a weight and regional characteristic basis for the convolution fusion operation. When the convolution fusion segmentation is performed in combination with the lesion heterogeneity index, it is helpful to perform dynamic adjustment and fine fusion according to the actual complexity of the lesion area, optimize the accuracy of the fusion result, and thus improve the accuracy of hard exudate segmentation after diabetic retinopathy.

[0046] In summary, the technical solution adopted in the present application can realize the convolution fusion segmentation of hard exudates of diabetic retinopathy, thereby improving the segmentation accuracy of hard exudates after diabetic retinopathy. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0048] Figure 1 is a flow chart of a method for segmenting hard exudates in diabetic retinopathy provided in the present application;

[0049] Figure 2 is an exemplary flow chart for determining edge contrast according to the present application;

[0050] Figure 3 is an exemplary flow chart for determining a convolution fusion bound according to the present application;

[0051] Figure 4 This is a module structure diagram of the diabetic retinopathy hard exudate segmentation system provided in this application. DETAILED DESCRIPTION

[0052] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0053] The embodiment of the present application provides a method and system for segmenting hard exudates in diabetic retinopathy, the core of which is to collect color fundus images after diabetic retinopathy, and then determine the image complexity of the color fundus image; segment the color fundus image into multiple exudate areas of hard exudate according to the distribution characteristics of hard exudate in the diabetic retina; determine the edge contrast of each exudate area based on the attention mechanism of the convolutional neural network combined with the grayscale gradient between the hard exudate and the background area in the color fundus image, and determine the lesion variation heterogeneity index after diabetic retinopathy through all edge contrasts and the image complexity; determine the convolution fusion boundary of all exudate areas through the lesion variation heterogeneity index and the boundary characteristics of each exudate area, and perform convolution fusion segmentation on all exudate areas of hard exudate based on the convolution fusion boundary to obtain multiple segmented areas of hard exudate in the color fundus image. The above scheme can realize the convolution fusion segmentation of hard exudates in diabetic retinopathy, thereby improving the segmentation accuracy of hard exudates after diabetic retinopathy.

[0054] Embodiment 1

[0055] In order to better understand the above technical solution, the above technical solution will be described in detail below in conjunction with the accompanying drawings and specific implementation methods. Figure 1 As shown in FIG. 1 , this figure is an exemplary flow chart of a diabetic retinopathy hard exudate segmentation method according to this embodiment of the present application, and the segmentation method includes the following steps:

[0056] In step S1, a color fundus image after diabetic retinopathy is acquired, and then the image complexity of the color fundus image is determined.

[0057] It should be noted that the color fundus image is a retinal image used to examine the structure and health of the eye; in specific implementation, a high-resolution fundus camera can be used to capture color fundus images after diabetic retinopathy.

[0058] In this embodiment, the image complexity of the color fundus image may be determined in the following manner:

[0059] Performing grayscale conversion on the color fundus image to obtain a grayscale image;

[0060] Converting the grayscale image into a gray-level co-occurrence matrix;

[0061] The image complexity of the color fundus image is determined by the gray level co-occurrence matrix.

[0062] It should be noted that image complexity is an indicator to measure the richness of image details and texture information in color fundus images; the grayscale co-occurrence matrix represents the matrix of spatial relationship between pixel gray levels in color fundus images; and the grayscale image is a single-channel image of brightness values ​​in color fundus images.

[0063] In the specific implementation, first, the image processing library (such as cv2.cvtColor() in OpenCV) can be used to complete the grayscale conversion of the color fundus image, and the result of the grayscale conversion is used as the grayscale image; then, the co-occurrence frequency of the grayscale levels of pixels in the grayscale image at a specific direction and distance is counted, and a fixed offset parameter (such as horizontal, vertical or diagonal direction) is set through historical experience. A matrix is ​​generated using a statistical method and normalized to ensure that the sum of the elements of the matrix is ​​1. The matrix can be used as a grayscale co-occurrence matrix; finally, the information entropy feature in the grayscale co-occurrence matrix can be used as the image complexity of the color fundus image. The information entropy feature reflects the texture and structural characteristics of the image. High complexity means that the image has rich details and information, while low complexity means that the image texture is relatively simple.

[0064] In step S2, the color fundus image is segmented into a plurality of exudate regions of hard exudate according to the distribution characteristics of hard exudate in diabetic retina.

[0065] In this embodiment, the color fundus image is segmented into multiple exudate regions of hard exudate according to the distribution characteristics of hard exudate in diabetic retina, specifically in the following manner, namely:

[0066] Identify all exudation locations of the color fundus image based on a data annotation mechanism;

[0067] All hard exudate locations were screened out from all exudate locations according to their distribution characteristics in the diabetic retina;

[0068] Multiple exudate areas of hard exudate are determined based on all hard exudate locations.

[0069] It should be noted that in the present application, the exudate area is an independent area formed by the aggregation of multiple hard exudates, and the hard exudate is a spot material formed by fat and protein deposition in the retina; in specific implementation, first, a labeling tool (for example: Labelbox) is used for semi-automatic labeling to mark the exudate area in the color fundus image. In other embodiments, expert experience and pre-trained deep learning models can be combined to improve the accuracy of labeling, which is not limited here; then, hard exudates after diabetic retinopathy usually have specific colors, shapes and distribution patterns, such as white or yellow spots or plaques, and are usually concentrated in specific areas of the retina. Therefore, a set of specific colors, shapes and distribution patterns can be used as the distribution characteristics of hard exudates in the diabetic retina in the target detection algorithm, so that all hard exudate positions can be screened out from all exudate positions using the target detection algorithm; finally, the region growing algorithm can be used to aggregate adjacent hard exudate positions into a complete exudate area to obtain multiple exudate areas of hard exudate.

[0070] In step S3, the edge contrast of each exudate area is determined based on the attention mechanism of the convolutional neural network combined with the grayscale gradient between the hard exudate and the background area in the color fundus image, and the pathological heterogeneity index after diabetic retinopathy is determined by all edge contrasts and the image complexity.

[0071] Preferably, in this embodiment, reference Figure 2 As shown in FIG. 1 , this figure is an exemplary flow chart of determining edge contrast in an embodiment of the present application. In this embodiment, the edge contrast of each exudate area is determined based on the attention mechanism of the convolutional neural network combined with the grayscale gradient between the hard exudate and the background area in the color fundus image, which can be specifically implemented by the following steps:

[0072] First, in step S31, the grayscale gradient between the hard exudate and the background area in the color fundus image is determined;

[0073] Then, in step S32, for each seepage area, a grayscale difference value between the seepage area and the background area is obtained from the grayscale gradient;

[0074] Then, in step S33, the influence weight of the grayscale difference in the seepage area is determined based on the attention mechanism of the convolutional neural network;

[0075] Finally, in step S34, the edge contrast of the seepage area is determined by the influence weight and the grayscale difference value, and then the edge contrast of each seepage area is determined.

[0076] It should be noted that in the present application, edge contrast represents the degree of grayscale difference between two exudation areas in the color fundus image; the grayscale gradient represents the degree of spatial variation of the grayscale value of each pixel in the color fundus image; the grayscale difference value is a quantitative value used to measure the contrast between the exudation area and the background area; the influence weight reflects the importance of different exudation areas in the color fundus image in convolution fusion.

[0077] In the specific implementation, first, by performing grayscale conversion on the color fundus image and applying an edge detection algorithm (for example, the Sobel operator), all values ​​of the grayscale gradient between the hard exudate and the background area can be determined; secondly, for each exudate area, the grayscale gradient between the hard exudate and the background area in the exudate area is obtained from the grayscale gradient as the grayscale difference value between the exudate area and the background area; then, based on the attention mechanism in the convolutional neural network, the grayscale difference in the exudate area is weighted to determine the influence weight of the grayscale difference in the exudate area. The attention mechanism enables the model to automatically adjust the weight, highlight the important areas in the image, and optimize the analysis results; finally, the product of the influence weight and the grayscale difference value can be used as the edge contrast of the exudate area. The edge contrast of each exudate area can be obtained in the above manner.

[0078] In this embodiment, the following method can be used to determine the pathological heterogeneity index after diabetic retinopathy by using all edge contrasts and the image complexity, namely:

[0079] For each exudative area, a lesion statistical model based on support vector machine is initialized;

[0080] Determine a statistical function of the lesion statistical model according to the image complexity and the edge contrast of the exudation area;

[0081] The lesion statistical model is used to perform fusion statistics on the exudative area after diabetic retinopathy to obtain the lesion variation heterogeneity of the exudative area, and then the lesion variation heterogeneity of each exudative area is obtained;

[0082] The heterogeneity index of lesion after diabetic retinopathy was determined by all lesion heterogeneity.

[0083] It should be noted that, in the present application, the lesion variation heterogeneity index is a characteristic value that quantifies the severity of the overall lesion; in specific implementation, first, for each exudation area, a lesion statistical model can be initialized using a support vector mechanism, and the lesion statistical model is a deep learning model for evaluating the lesion characteristics in the exudation area. The lesion statistical model can comprehensively consider the characteristics in the infiltration area and use the classifier or regressor of the support vector machine to generate a fusion statistical value; secondly, the statistical function of the lesion statistical model is trained using the image complexity and the edge contrast of the exudation area, and the statistical function represents the relationship between different pixels in the exudation area and the lesion characteristics; then, the lesion statistical model is used to perform fusion statistics on the exudation area after diabetic retinopathy, and the quantified value of the fusion statistics can be used as the lesion variation heterogeneity of the exudation area. The lesion variation heterogeneity of each exudation area can be obtained by the above method, and the lesion variation heterogeneity represents the degree of change and complexity of different parts in the lesion area; finally, the standard deviation of all lesion variation heterogeneities can be used as the lesion variation heterogeneity index after diabetic retinopathy.

[0084] In step S4, the convolution fusion boundary of all exudate areas is determined by the pathological variation heterogeneity index and the boundary characteristics of each exudate area, and all exudate areas of hard exudate are segmented by convolution fusion based on the convolution fusion boundary to obtain multiple segmented areas of hard exudate in the color fundus image.

[0085] Preferably, in this embodiment, reference Figure 3 As shown in FIG. 1 , this figure is an exemplary flow chart for determining the convolution fusion boundary in an embodiment of the present application. In this embodiment, the convolution fusion boundary of all exudative regions is determined by the pathological variation heterogeneity index and the boundary characteristics of each exudative region, which can be specifically implemented by the following steps:

[0086] First, in step S41, for each seepage area, the boundary characteristics of the seepage area are obtained, and then the fusion weight of the seepage area is determined;

[0087] Then, in step S42, the boundary fusion value of the seepage area is determined by the fusion weight and the boundary characteristic, thereby obtaining the boundary fusion value of each seepage area;

[0088] Finally, in step S43, a convolution operation is performed on all boundary fusion values ​​to obtain the convolution fusion boundaries of all seepage areas.

[0089] It should be noted that in this application, the convolution fusion boundary is the boundary that reflects the characteristics of the overall lesion area; the fusion weight is the weight value used to indicate the importance of the boundary characteristics in the fusion calculation; the boundary characteristics represent the geometric texture characteristics of the edge of the exudation area; and the boundary fusion value refers to the characteristic value of the boundary of the exudation area.

[0090] In the specific implementation, first, for each exudation area, an edge detection algorithm (such as the Canny operator) can be used to identify the boundary information of the hard exudate in the color fundus image as the boundary characteristic of the exudation area, and the fusion weight of the exudation area can be preset through historical experience; then, an edge fusion model is initialized, the fusion weight is used as the weight parameter in the edge fusion model, and the boundary characteristic is used as the fusion feature in the edge fusion model. The edge fusion model is used to quantitatively evaluate the boundary influence of the exudation area, and the result of the quantitative evaluation can be used as the boundary fusion value of the exudation area. The boundary fusion value of each exudation area can be obtained in the above manner; finally, a standard two-dimensional convolution method can be used to perform a convolution operation on all boundary fusion values, and the result of the convolution operation can be used as the convolution fusion boundary of all exudation areas.

[0091] In this embodiment, all exudate regions of hard exudate are segmented by convolution fusion based on the convolution fusion boundary to obtain multiple segmented regions of hard exudate in the color fundus image, which can be specifically obtained in the following manner, namely:

[0092] Convolution kernels of various scales are determined according to the shape characteristics of hard exudates;

[0093] Determining the fusion weights of convolution kernels of various scales through the convolution fusion boundary;

[0094] All the exudate regions are fused into a plurality of segmented regions of hard exudate in the color fundus image according to all the fusion weights.

[0095] In the specific implementation, first, the shape features of hard exudates can be used to train standard convolution kernels to obtain training results of various scales. All training results are used as convolution kernels of corresponding scales to obtain convolution kernels of multiple scales. Then, the weight distribution in the convolution fusion boundary is used as the weight calculation rule for edge strength and regional distribution. The calculation rule is used to adjust the weights of convolution kernels of different scales. The adjusted result is used as the fusion weight of the convolution kernel of the corresponding scale to obtain the fusion weight of convolution kernels of various scales. Finally, a convolution fusion model is initialized, each convolution kernel is used as the convolution kernel in the convolution fusion model, and the fusion weight is used as the scale weight in the convolution fusion model. The convolution fusion model is used to perform multi-scale fusion on all exudation areas. The results of multi-scale fusion can be used as all segmented areas to obtain multiple segmented areas of hard exudates in the color fundus image.

[0096] It can be seen that in the present application, the convolution fusion boundaries of all exudate areas are determined by the pathological heterogeneity index and the boundary characteristics of each exudate area, and all exudate areas of hard exudate are convolutionally fused and segmented based on the convolution fusion boundary to obtain multiple segmented areas of hard exudate in the color fundus image; firstly, the areas with hard exudate characteristics can be accurately extracted through data annotation and feature-based screening, thereby avoiding unnecessary noise interference. Among them, combining the image complexity with the boundary characteristics of the exudate area can make the segmented area more targeted and reliable, and can provide accurate input data for subsequent convolution fusion segmentation, so that the convolution network can focus more on relevant areas during processing, which helps to reduce The misclassification and omission phenomena are eliminated to ensure that the convolution fusion segmentation algorithm can perform a refined analysis of hard exudates, thereby improving the segmentation accuracy and detection effectiveness; then, the attention mechanism and grayscale gradient analysis based on convolutional neural networks are used to help evaluate the edge characteristics of different exudate areas and reveal the differences in edge characteristics in color fundus images. The lesion heterogeneity index reflects the changing characteristics of hard exudates in different regions, and provides a weight and regional characteristic basis for the convolution fusion operation. When the convolution fusion segmentation is performed in combination with the lesion heterogeneity index, it is helpful to perform dynamic adjustment and fine fusion according to the actual complexity of the lesion area, optimize the accuracy of the fusion result, and thus improve the accuracy of hard exudate segmentation after diabetic retinopathy.

[0097] In summary, the technical solution adopted in the present application can realize the convolution fusion segmentation of hard exudates of diabetic retinopathy, thereby improving the segmentation accuracy of hard exudates after diabetic retinopathy.

[0098] Embodiment 2

[0099] The present application provides a diabetic retinopathy hard exudate segmentation system, referring to Figure 4 As shown, this figure is a schematic diagram of a segmentation system according to this embodiment of the present application, and the segmentation system includes:

[0100] An image acquisition module 100 is used to acquire a color fundus image after diabetic retinopathy, and then determine the image complexity of the color fundus image;

[0101] An initial segmentation module 200, for segmenting the color fundus image into a plurality of exudate regions of hard exudate according to the distribution characteristics of hard exudate in diabetic retina;

[0102] A heterogeneity assessment module 300, for determining the edge contrast of each exudate area based on the attention mechanism of a convolutional neural network combined with the grayscale gradient between the hard exudate and the background area in the color fundus image, and determining the heterogeneity index of the lesion after diabetic retinopathy through all edge contrasts and the image complexity;

[0103] The fusion segmentation module 400 is used to determine the convolution fusion boundary of all exudate areas according to the pathological variation heterogeneity index and the boundary characteristics of each exudate area, and perform convolution fusion segmentation on all exudate areas of hard exudate based on the convolution fusion boundary to obtain multiple segmented areas of hard exudate in the color fundus image.

[0104] The present application is described with reference to flowcharts and / or block diagrams of methods, devices (systems) and computer program products according to embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A system that specifies the functions of a box or multiple boxes.

[0105] Those skilled in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, the storage medium including a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable programmable read-only memory (EEPROM), a compact disc (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.

[0106] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

Claims

1. A method for segmenting hard exudates of diabetic retinopathy, characterized in that: The segmentation method comprises the following steps: Acquiring a color fundus image after diabetic retinopathy, and then determining the image complexity of the color fundus image; Segmenting the color fundus image into multiple exudate areas of hard exudate according to the distribution characteristics of hard exudate in diabetic retina; The edge contrast of each exudate area is determined based on the attention mechanism of the convolutional neural network combined with the grayscale gradient between the hard exudate and the background area in the color fundus image, and the pathological variation heterogeneity index after diabetic retinopathy is determined through all edge contrasts and the image complexity; The convolution fusion boundary of all exudate areas is determined by the pathological variation heterogeneity index and the boundary characteristics of each exudate area, and all exudate areas of hard exudate are segmented by convolution fusion based on the convolution fusion boundary to obtain multiple segmented areas of hard exudate in the color fundus image.

2. A diabetic retinopathy hard exudate segmentation method according to claim 1, characterized in that: Determining the image complexity of the color fundus image specifically includes: Performing grayscale conversion on the color fundus image to obtain a grayscale image; Converting the grayscale image into a gray-level co-occurrence matrix; The image complexity of the color fundus image is determined by the gray level co-occurrence matrix.

3. A diabetic retinopathy hard exudate segmentation method as claimed in claim 1, characterized in that: Segmenting the color fundus image into multiple exudate regions of hard exudate according to the distribution characteristics of hard exudate in diabetic retina specifically includes: Identify all exudation locations of the color fundus image based on a data annotation mechanism; All hard exudate locations were screened out from all exudate locations according to their distribution characteristics in the diabetic retina; Multiple exudate areas of hard exudate are determined based on all hard exudate locations.

4. The method for segmenting hard exudates of diabetic retinopathy according to claim 1, characterized in that: The edge contrast of each exudate area is determined based on the attention mechanism of the convolutional neural network in combination with the grayscale gradient between the hard exudate and the background area in the color fundus image, specifically including: determining a grayscale gradient between a hard exudate and a background area in the color fundus image; For each seepage area, obtaining a grayscale difference value between the seepage area and the background area from the grayscale gradient; The influence weight of grayscale difference in the exudation area is determined based on the attention mechanism of convolutional neural network; The edge contrast of the seepage area is determined by the influence weight and the grayscale difference value, and then the edge contrast of each seepage area is determined.

5. The method for segmenting hard exudates of diabetic retinopathy according to claim 1, characterized in that: The pathological heterogeneity index after diabetic retinopathy is determined by all edge contrasts and the image complexity, specifically including: For each exudative area, a lesion statistical model based on support vector machine is initialized; Determine a statistical function of the lesion statistical model according to the image complexity and the edge contrast of the exudation area; The lesion statistical model is used to perform fusion statistics on the exudative area after diabetic retinopathy to obtain the lesion variation heterogeneity of the exudative area, and then the lesion variation heterogeneity of each exudative area is obtained; The heterogeneity index of lesion after diabetic retinopathy was determined by all lesion heterogeneity.

6. The method for segmenting hard exudates of diabetic retinopathy according to claim 1, characterized in that: The hard exudates are flecks of material formed by deposits of fat and protein within the retina.

7. The method for segmenting hard exudates of diabetic retinopathy according to claim 1, characterized in that: The convolution fusion boundaries of all exudative regions are determined by the pathological heterogeneity index and the boundary characteristics of each exudative region, specifically including: For each seepage area, the boundary characteristics of the seepage area are obtained, and then the fusion weight of the seepage area is determined; Determining the boundary fusion value of the seepage area by using the fusion weight and the boundary characteristic, and then obtaining the boundary fusion value of each seepage area; Perform convolution operation on all boundary fusion values ​​to obtain the convolution fusion boundaries of all leakage areas.

8. The method for segmenting hard exudates of diabetic retinopathy according to claim 1, characterized in that: Based on the convolution fusion boundary, all exudate regions of the hard exudate are segmented by convolution fusion, and the multiple segmented regions of the hard exudate in the color fundus image are obtained, which specifically include: Convolution kernels of various scales are determined according to the shape characteristics of hard exudates; Determining the fusion weights of convolution kernels of various scales through the convolution fusion boundary; All the exudate regions are fused into a plurality of segmented regions of hard exudate in the color fundus image according to all the fusion weights.

9. The method for segmenting hard exudates of diabetic retinopathy according to claim 1, characterized in that: Color fundus images of diabetic retinopathy were acquired using a high-resolution fundus camera.

10. A diabetic retinopathy hard exudate segmentation system, used to execute a diabetic retinopathy hard exudate segmentation method according to any one of claims 1 to 9, characterized in that: The segmentation system comprises: An image acquisition module, used for acquiring a color fundus image after diabetic retinopathy, and then determining the image complexity of the color fundus image; An initial segmentation module, used for segmenting the color fundus image into a plurality of exudate regions of hard exudate according to the distribution characteristics of hard exudate in diabetic retina; A heterogeneity assessment module, for determining the edge contrast of each exudate area based on the attention mechanism of a convolutional neural network combined with the grayscale gradient between the hard exudate and the background area in the color fundus image, and determining the heterogeneity index of the lesion after diabetic retinopathy through all edge contrasts and the image complexity; A fusion segmentation module is used to determine the convolution fusion boundary of all exudate areas according to the pathological variation heterogeneity index and the boundary characteristics of each exudate area, and perform convolution fusion segmentation on all exudate areas of hard exudate based on the convolution fusion boundary to obtain multiple segmented areas of hard exudate in the color fundus image.

Images