Fundus lesion recognition method and device

Abstract

The disclosure provides an eye fundus lesion identification method and device, including obtaining color information of an eye fundus lesion in a target eye image, classifying the eye fundus lesion through a classification model based on the color information of the eye fundus lesion, and obtaining the category of the eye fundus lesion in the target eye image, wherein the category includes red eye fundus lesions and / or white eye fundus lesions; for different categories of eye fundus lesions, based on a pre-trained eye fundus lesion segmentation model, a segmentation result of the eye fundus lesion is determined; and the segmentation result is classified by features to obtain the subcategory of the eye fundus lesion corresponding to each segmentation result of the different categories of eye fundus lesions. By segmenting and identifying the lesions under different categories, the segmentation and identification accuracy is improved. Furthermore, the related art problem of high difficulty in identifying and classifying lesions by artificial, and the situation of misidentification or misclassification is solved.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to a method and apparatus for identifying fundus lesions. Background Technology

[0002] In related technologies, conditions such as hypertension, diabetes, vascular occlusion, and trauma can lead to eye damage and various ocular lesions. However, the identification and classification of these lesions usually rely on manual judgment by doctors and other personnel. This not only results in a heavy workload for doctors but also presents significant challenges in lesion classification and identification, making it prone to misidentification or misclassification due to human factors.

[0003] The information disclosed in the background section of this application is intended only to enhance the understanding of the general background of this application and should not be construed as an admission or in any way implying that the information constitutes prior art known to those skilled in the art. Summary of the Invention

[0004] This disclosure provides a method and apparatus for identifying fundus lesions.

[0005] A first aspect of this disclosure provides a method for identifying fundus lesions, comprising classifying fundus lesions contained in a target eye image using a classification model to obtain the category of the fundus lesions, wherein the classification model determines the category of the fundus lesions by extracting and identifying color features of the target eye image; the categories of fundus lesions include red fundus lesions and / or white fundus lesions; for different categories of fundus lesions, determining the segmentation result of the fundus lesions based on a pre-trained fundus lesion segmentation model; and performing feature classification on the segmentation result to obtain the sub-category of the fundus lesions corresponding to the segmentation result.

[0006] Optionally, before classifying the target eye image, the method further includes: removing the background region of the eye image to be processed to obtain a first eye image; and normalizing the first eye image to obtain the target eye image.

[0007] Optionally, before classifying the fundus lesions based on their color information using a classification model, the method further includes: detecting the initial image using a preset detection model to determine whether the initial image contains first feature information; if it does not contain the first feature information, then removing the regions containing second feature information.

[0008] Optionally, performing feature classification on the segmentation result to obtain the sub-category of the fundus lesion corresponding to the segmentation result includes: inputting the segmentation result into a pre-trained lesion recognition model and outputting the sub-category of the fundus lesion corresponding to the segmentation result.

[0009] Optionally, inputting the segmentation result into a pre-established lesion identification model and outputting sub-category information of the fundus lesions corresponding to the segmentation result includes: inputting the segmentation result into a pre-established lesion identification model and outputting the fundus microaneurysm lesion category and fundus hemorrhage category corresponding to each segmentation result under the red fundus lesion; and / or, inputting the segmentation result into a pre-established lesion identification model and outputting the fundus exudative lesion category and fundus drusen lesion category under the white fundus lesion.

[0010] A second aspect of this disclosure provides a fundus lesion identification device, comprising: a first classification module configured to classify fundus lesions contained in a target eye image using a classification model to obtain a category of fundus lesions, wherein the classification model determines the category of fundus lesions by extracting and identifying color features of the target eye image; the categories of fundus lesions include red fundus lesions and / or white fundus lesions; a segmentation module configured to determine a segmentation result of fundus lesions based on a pre-trained fundus lesion segmentation model for different categories of fundus lesions; and a second classification module configured to perform feature classification on the segmentation result to obtain a sub-category of fundus lesions corresponding to each segmentation result under the different categories of fundus lesions.

[0011] Optionally, the device is further configured to: remove the background region of the eye image to be processed to obtain a first eye image; and normalize the first eye image to obtain the target eye image.

[0012] Optionally, the device further includes a detection unit configured to detect the initial image using a preset detection model to determine whether the initial image contains first feature information; if it does not contain the first feature information, the region containing second feature information is removed.

[0013] A third aspect of this disclosure provides a fundus lesion identification device, comprising: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to invoke the instructions stored in the memory to execute the above-described method.

[0014] A fourth aspect of this disclosure provides a computer-readable storage medium having computer program instructions stored thereon, wherein the computer program instructions, when executed by a processor, implement the above-described method.

[0015] This embodiment of the method and apparatus for identifying fundus lesions includes classifying fundus lesions contained in a target eye image using a classification model to obtain the category of the fundus lesions. The classification model determines the category of fundus lesions by extracting and recognizing color features of the target eye image; the categories of fundus lesions include red fundus lesions and / or white fundus lesions. For different categories of fundus lesions, a pre-trained fundus lesion segmentation model is used to determine the segmentation result of the fundus lesions. The segmentation result is then subjected to feature classification to obtain the sub-category of the fundus lesions corresponding to each segmentation result under different categories of fundus lesions. By first determining the category information of fundus lesions in the target eye image and then determining the segmentation result based on the category information, the accuracy of the segmentation result is improved. Further determining the sub-category information of the fundus lesions based on the segmentation result improves the accuracy of lesion identification and classification. This solves the problem in related technologies where manual identification and classification of lesions is difficult and prone to misidentification or misclassification. Attached Figure Description

[0016] Figure 1 A schematic flowchart of the fundus lesion identification method according to an embodiment of the present disclosure is shown as an example;

[0017] Figure 2 A schematic diagram of an eye image, exemplarily illustrating an embodiment of the present disclosure, is shown.

[0018] Figure 3 A block diagram of a fundus lesion identification device according to an embodiment of the present disclosure is shown as an example;

[0019] Figure 4 This is a block diagram illustrating a fundus lesion identification device according to an exemplary embodiment. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0021] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in this disclosure, claims, and accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such use of data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein.

[0022] It should be understood that in the various embodiments of this disclosure, the sequence number of each process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this disclosure.

[0023] It should be understood that in this disclosure, "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, for example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units that are explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to such process, method, product, or device.

[0024] It should be understood that in this disclosure, "multiple" refers to two or more. "And / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, "and / or B" can represent: A existing alone, A and B existing simultaneously, or B existing alone. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "Contains A, B, and C", "Contains A, B, and C" means that all three A, B, and C are contained; "Contains A, B, or C" means that one of A, B, and C is contained; "Contains A, B, and / or C" means that any one, two, or three of A, B, and C are contained.

[0025] It should be understood that in this disclosure, "B corresponding to A", "B corresponding to A", "A corresponds to B", or "B corresponds to A" means that B is associated with A, and B can be determined based on A. Determining B based on A does not mean determining B solely based on A; B can also be determined based on A and / or other information. Matching A and B is defined as a similarity between A and B that is greater than or equal to a preset threshold.

[0026] Depending on the context, "if" as used here can be interpreted as "when," "when," "in response to determination," or "in response to detection."

[0027] The technical solutions of this disclosure will be described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0028] Figure 1 An exemplary flowchart of a method for identifying fundus lesions according to an embodiment of this disclosure is shown, such as... Figure 1 As shown, the method includes:

[0029] Step S101: Classify the fundus lesions contained in the target eye image using a classification model to obtain the category of the fundus lesions. The classification model determines the category of fundus lesions by extracting and recognizing the color features of the target eye image. The categories of fundus lesions include red fundus lesions and / or white fundus lesions.

[0030] In this embodiment, the target eye image can be automatically classified using a classification model based on the color information in the target eye image to obtain the category of fundus lesions in the target eye image. A fundus image (i.e., the target eye image) may include one or more types of fundus lesions. For example, a fundus image may include a type of red fundus lesion, a type of white fundus lesion, or both red and white fundus lesions.

[0031] For example, when acquiring the color information of fundus lesions in the target eye image, color features can be extracted using a deep learning neural network. Then, the color features can be identified and classified based on a classification model to determine the category of the fundus lesion, such as whether it belongs to the red or white fundus lesion type. It should be understood that other methods can also be used to determine the color information, such as obtaining the color information of the fundus lesion through pixel detection; this is merely an example.

[0032] Furthermore, in determining the category information of fundus lesions, regression analysis, Bayesian models, and other methods can also be used. This disclosure does not restrict the specific methods for determining category information.

[0033] As an optional implementation of this embodiment, before classifying the fundus lesions based on the color information of the fundus lesions using a classification model, the method further includes: detecting the initial image using a preset detection model to determine whether the initial image contains first feature information; if it does not contain the first feature information, then removing the region containing the second feature information.

[0034] In this optional implementation, the first feature information may include feature information indicating large-area hemorrhage. The symptoms causing the large-area hemorrhage may be caused by severe diabetic retinopathy, venous obstruction, etc. If the initial image is detected to contain features of large-area hemorrhage, further lesion extraction of the image containing this feature is meaningless and wastes computational resources. Therefore, the image can be left unprocessed, or the information that the initial image belongs to a large-area hemorrhage lesion can be output.

[0035] Furthermore, if the region does not contain feature information indicating large-area hemorrhage, then the region containing feature information that causes segmentation interference can be removed, that is, the region containing the second feature information can be removed. For example, the second feature information may include, but is not limited to, feature information that can be used to represent the optic disc, feature information that can be used to represent blood vessels, etc.

[0036] Furthermore, the fundus must contain objects such as the optic disc and blood vessels. Therefore, the initial image must contain the optic disc region, blood vessel region, etc., that is, there must be regions containing the second feature information. Therefore, before removing these regions, these regions can be identified to accurately identify their positions, thereby achieving accurate segmentation.

[0037] In this optional implementation, if the initial image does not contain feature information indicating large-area hemorrhage, the area indicated by the second feature information can be directly removed, thus avoiding interference with subsequent segmentation steps and preventing inaccurate lesion segmentation.

[0038] Furthermore, image quality can be assessed before detection to determine if there are issues such as blurring or out of focus. If such issues exist, detection can be skipped to avoid wasting computational resources.

[0039] Furthermore, medical examination of the eye can yield an initial image, which can be obtained through methods such as fundus camera, fundus fluorescein angiography, or optical coherence tomography. However, this initial image may contain noise due to lens contamination. Therefore, after removing regions containing the second feature information to obtain an intermediate image, denoising can be performed on the intermediate image. The denoising method is not limited here and may include, but is not limited to, median filtering.

[0040] As an optional implementation of this embodiment, before classifying the target eye image, the method further includes: removing the background region of the eye image to be processed to obtain a first eye image; and normalizing the first eye image to obtain the target eye image.

[0041] In this optional implementation, Figure 2 A schematic diagram of an eye image, exemplarily illustrating an embodiment of the present disclosure, is shown below. Figure 2 As shown, due to factors such as the shape of the lens, a portion of the background area may exist in the eye image to be processed, for example, Figure 2 The black background area in the image lacks information about the eye and may interfere with model calculations. Furthermore, due to the varying resolutions of lenses, the size and resolution of the eye images to be processed may also differ. All of these factors can negatively impact the processing of eye images and the identification and classification of lesions.

[0042] To mitigate the aforementioned adverse effects, the background region of the eye image to be processed can be removed, and the first eye image after background removal can be scaled to a preset size to obtain the target eye image. This processing reduces interference from the background region and ensures that the target eye images processed by the model are of consistent size, thus improving the model's robustness in processing, as well as its robustness in lesion identification and classification.

[0043] In the example, a ROI (region of interest), i.e., the non-background region, can be determined in the eye image to be processed. This non-background region is then cropped to obtain the first eye image. The first eye image can then be normalized to obtain a target eye image of uniform size. Normalization can include, but is not limited to, translation, rotation, and scaling, such as scaling to a size of 512×512.

[0044] Step S102: For different types of fundus lesions, determine the segmentation results of fundus lesions based on the pre-trained fundus lesion segmentation model.

[0045] In this embodiment, a pre-trained fundus lesion segmentation model can be used to segment different types of fundus lesions, resulting in detailed segmented images of the fundus lesions. Each segmented image can include different segmentation boundaries, and different segmentation boundaries can enclose different segmented regions.

[0046] The segmentation result of fundus lesions can be determined by using a fundus lesion segmentation model. The segmentation result can be a binary image, that is, a binary image with the same size as the target eye image. In this binary image, the pixel value of the area where the lesion is located (i.e., within the segmentation boundary) is 1, and the pixel value of other areas is 0.

[0047] According to embodiments of this disclosure, the lesion segmentation model can be a deep learning neural network model, such as a U-NET type neural network model. This disclosure does not limit the type and structure of the lesion segmentation model.

[0048] According to embodiments of this disclosure, the above processing can improve the segmentation accuracy of fundus lesions by the fundus lesion segmentation model, thereby obtaining a segmentation boundary with higher accuracy.

[0049] Step S103: Perform feature classification on the segmentation results to obtain the sub-category of fundus lesions corresponding to the segmentation results.

[0050] According to embodiments of this disclosure, the boundary features of fundus lesions with different segmentation results may be different from each other. Therefore, detailed segmentation boundaries of fundus lesions can be obtained, thereby determining the corresponding sub-category of fundus lesions based on the segmentation boundaries.

[0051] Furthermore, in this embodiment, under the red fundus lesion type, the subcategories obtained based on the segmentation results may include, but are not limited to, microaneurysms and fundus hemorrhage; under the white fundus lesion type, the subcategories obtained based on the segmentation results may include, but are not limited to, fundus exudates and drusen.

[0052] As an optional implementation of this embodiment, the segmentation result is classified by features to obtain the sub-category of fundus lesions corresponding to the segmentation result, including: inputting the segmentation result into a pre-trained lesion recognition model to identify the features of the segmentation result; and outputting the sub-category of fundus lesions corresponding to the segmentation result.

[0053] In this optional implementation, after obtaining the segmentation result, the segmentation boundary features and the lesion region surrounded by the segmentation boundary can be obtained. The shape of the lesion region and the feature information such as the segmentation boundary are input into the pre-trained lesion recognition model, and the sub-category corresponding to the segmentation result is output.

[0054] Furthermore, the lesion recognition model can be pre-trained. During training, the lesion subcategories in the fundus image can be pre-labeled, and then the labeled training samples can be used as input to the lesion recognition model, with the corresponding subcategories of the lesions as output to train the lesion recognition model.

[0055] As an optional implementation of this embodiment, inputting the segmentation result into a pre-established lesion identification model and outputting the sub-category information of the fundus lesions corresponding to the segmentation result includes: inputting the segmentation result into a pre-established lesion identification model and outputting the fundus microaneurysm lesion category and fundus hemorrhage category corresponding to each segmentation result under the red fundus lesion; and / or, inputting the segmentation result into a pre-established lesion identification model and outputting the fundus exudative lesion category and fundus drusen lesion category under the white fundus lesion.

[0056] In this optional implementation, the recognition model can identify different subcategories under different types of fundus lesions.

[0057] Further, in step S102, for different categories of fundus lesions, the segmentation results of the fundus lesions are determined based on the pre-trained fundus lesion segmentation model. This can be achieved in the following way:

[0058] Different types of fundus lesions can be detected in advance using a pre-trained lesion detection model to obtain lesion regions of different subcategories selected by the detection box. Then, the image within the selected region is segmented to obtain the first image. This method can achieve the segmentation and extraction of different lesion regions.

[0059] Furthermore, to further improve segmentation accuracy, a preset computer vision algorithm can be used to perform boundary detection on different types of fundus lesion images to extract different sub-categories of lesion regions and obtain a second image. By performing intersection processing on the first and second images, a more accurate fundus lesion segmentation result can be obtained.

[0060] It should be understood that lesion regions of different subcategories have different characteristics. Therefore, in order to achieve accurate segmentation of lesion regions of different subcategories, the above segmentation method can be further refined. As described below, for the red fundus lesion category, it may include subcategories such as microaneurysms or hemorrhages. If the subcategory of the lesion is determined to be microaneurysms or hemorrhages based on the segmentation results, the following method can be used when segmenting images of lesions of different subcategories:

[0061] (1) Using a category recognition model, the type of fundus hemorrhage lesion in the target eye image is identified to obtain a type recognition result; if the type recognition result indicates that the fundus hemorrhage lesion belongs to the first category, the target eye image is detected using a pre-trained fundus hemorrhage lesion detection model to obtain a first initial image of the fundus hemorrhage lesion region; the first initial image is segmented using a pre-trained lesion segmentation model to obtain a first image of the fundus hemorrhage lesion; based on a preset image processing algorithm, the fundus hemorrhage lesion region is extracted from the target eye image to obtain a second image of the fundus hemorrhage lesion; the intersection of the first initial image, the first image, and the second image is performed to obtain the fundus hemorrhage lesion segmentation result corresponding to the target eye image. For example, the preset image processing algorithm may include, but is not limited to, traditional computer vision algorithms.

[0062] If the type identification result indicates that the fundus hemorrhage lesion belongs to the second category, the target eye image is detected by a pre-trained fundus hemorrhage lesion detection model to obtain a second initial image of the fundus hemorrhage lesion region.

[0063] By pre-identifying the type of retinal hemorrhage lesions and then processing them differently for different types, the segmentation accuracy of hemorrhage lesions is improved. A second image obtained through computer vision methods is used to correct the first image obtained from the retinal hemorrhage lesion segmentation model, further improving the accuracy of the segmentation results. This addresses the problem that in related technologies, the identification and classification of hemorrhage lesions often rely on manual judgment by doctors and other professionals, which not only leads to a heavy workload for doctors but also presents significant challenges due to the difficulty of lesion classification and identification, and is prone to misidentification or misclassification caused by human factors. The first type of hemorrhage includes minor categories such as "diabetic retinopathy, hypertension, and unexplained hemorrhage"; the second type includes major categories such as "venous obstruction hemorrhage, vitreous hemorrhage, and optic disc edema hemorrhage." This method can segment hemorrhage lesions specifically.

[0064] Furthermore, after obtaining the segmentation results of the retinal hemorrhage lesion, the specific category of the hemorrhage lesion can be determined based on its location feature information. This location feature information can include the positional relationship between the segmented hemorrhage lesion and other preset objects, which may include, but are not limited to, retinal veins and central retinal arteries. For example, if the location feature information of the segmented hemorrhage lesion indicates that it is close to a vein, then the hemorrhage lesion can be further identified as a venous occlusion hemorrhage lesion.

[0065] (2) The eye image to be segmented is divided into multiple eye image blocks; each eye image block is detected by a pre-trained lesion detection model to obtain an initial eye image block marked with a fundus microaneurysm lesion detection box; the initial eye image blocks are combined to form a target eye image; the region in each fundus microaneurysm lesion detection box in the target eye image is segmented by a pre-trained fundus microaneurysm lesion segmentation model to obtain a first image of fundus microaneurysm lesions; the fundus microaneurysm lesion region is extracted from the eye image to be segmented based on a preset image processing algorithm to obtain a second image; the intersection of the first image and the second image is performed to obtain the segmentation result corresponding to the eye image to be segmented. By pre-segmenting the eye image into multiple eye image patches, detecting each patch, and merging them into a target eye image, and then performing intersection processing on the images obtained by the fundus microaneurysm lesion segmentation model and image processing algorithms, accurate segmentation results of fundus microaneurysms can be obtained. Based on this accurate segmentation result, it is helpful to quickly determine the type of fundus microaneurysm lesion. This solves the problems of high difficulty and high false identification rate caused by manual identification of fundus microaneurysms in related technologies. This method can segment ocular aneurysm lesions.

[0066] The above segmentation steps can be used to assist in quickly determining the type of lesion based on the segmentation results, thereby improving the efficiency of medical services.

[0067] For fundus lesions classified as white lesions, which may include subcategories such as exudates and drusen, image segmentation of different subcategories of lesions can be performed using the following methods:

[0068] (1) An initial image of the fundus exudative lesion region is obtained by detecting the target eye image using a pre-trained fundus exudative lesion detection model; the initial image is segmented using a pre-trained lesion segmentation model to obtain a first image of the fundus exudative lesion; the fundus exudative lesion region is extracted from the target eye image based on a preset image processing algorithm to obtain a second image of the fundus exudative lesion region; the intersection of the first image and the second image is performed to obtain the segmentation result corresponding to the fundus exudative lesion region. By performing intersection processing on the images obtained by the lesion segmentation model and the image processing algorithm respectively, accurate segmentation results of fundus exudative lesions can be obtained, and based on this accurate segmentation result, it is helpful to quickly determine the type of exudative lesion. This solves the problem that in related technologies, the identification and classification of hard and soft exudative lesions usually rely on manual judgment by doctors and other professionals, which not only leads to a large workload for doctors, but also makes the classification and identification of lesions difficult and prone to misidentification or misclassification due to human factors. This method can be used to segment retinal exudative lesions.

[0069] (2)(2) A method and apparatus for extracting drusen lesions, comprising: performing feature classification processing on a target eye image to determine whether a drusen lesion exists in the target eye image; if the drusen lesion exists in the target eye image, detecting the target eye image using a pre-trained drusen lesion detection model to obtain an initial image of the drusen lesion region; segmenting the initial image using a pre-trained drusen lesion segmentation model to obtain a first image of the drusen lesion; extracting the drusen lesion region from the target eye image based on a preset image processing algorithm to obtain a second image of the drusen lesion region; and performing intersection processing on the first image of the drusen lesion and the second image of the drusen lesion region to obtain the segmentation result corresponding to the drusen lesion region. The classification and segmentation steps improve the accuracy of drusen lesion segmentation. This method can be used to segment drusen lesions.

[0070] The above segmentation steps can be used to assist in quickly determining the type of lesion based on the segmentation results, thereby improving the efficiency of medical services.

[0071] This embodiment first determines the category information of fundus lesions in the target eye image, and then determines the segmentation result based on the category information, thereby improving the accuracy of the segmentation result. Based on the segmentation result, it further determines the sub-category information of the fundus lesions, thus improving the accuracy of lesion identification and classification. This solves the problem in related technologies where manual identification and classification of lesions is difficult and prone to misidentification or misclassification.

[0072] Figure 3 A block diagram of a fundus lesion identification device according to an embodiment of the present disclosure is shown as an example. Figure 3 As shown, the device may include: a first classification module, configured to classify fundus lesions contained in a target eye image using a classification model to obtain the category of the fundus lesions, wherein the classification model determines the category of the fundus lesions by extracting and recognizing the color features of the target eye image; the categories of fundus lesions include red fundus lesions and / or white fundus lesions; a segmentation module, configured to determine the segmentation result of fundus lesions based on a pre-trained fundus lesion segmentation model for different categories of fundus lesions; and a second classification module, configured to perform feature classification on the segmentation result to obtain the sub-category of fundus lesions corresponding to each segmentation result under the different categories of fundus lesions.

[0073] As an optional implementation of this embodiment, the device is further configured to remove the background region of the eye image to be processed to obtain a first eye image; and to perform normalization processing on the first eye image to obtain the target eye image.

[0074] As an optional implementation of this embodiment, the device further includes: a detection unit configured to detect the initial image using a preset detection model to determine whether the initial image contains first feature information; if it does not contain preset feature information, the region containing second feature information is removed.

[0075] As an optional implementation of this embodiment, the segmentation result is classified by features to obtain the sub-category of fundus lesions corresponding to the segmentation result, including: inputting the segmentation result into a pre-trained lesion recognition model to identify the features of the segmentation result; and outputting the sub-category of fundus lesions corresponding to the segmentation result.

[0076] As an optional implementation of this embodiment, inputting the segmentation result into a pre-established lesion identification model and outputting the sub-category information of the fundus lesions corresponding to the segmentation result includes: inputting the segmentation result into a pre-established lesion identification model and outputting the fundus microaneurysm lesion category and fundus hemorrhage category corresponding to each segmentation result under the red fundus lesion; and / or, inputting the segmentation result into a pre-established lesion identification model and outputting the fundus exudative lesion category and fundus drusen lesion category under the white fundus lesion.

[0077] Figure 4 This is a block diagram illustrating a fundus lesion identification device according to an exemplary embodiment. For example, the device 1600 may be provided as a terminal or server. The device 1600 includes a processing component 1602 and memory resources represented by a memory 1603 for storing instructions executable by the processing component 1602, such as application programs. The application programs stored in the memory 1603 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processing component 1602 is configured to execute instructions to perform the methods described above.

[0078] Device 1600 may also include a power supply component 1606 configured to perform power management of device 1600, a wired or wireless network interface 1605 configured to connect device 1600 to a network, and an input / output (I / O) interface 1608. Device 1600 can operate on an operating system stored in memory 1603, such as Windows Server™, MacOS X™, Unix™, Linux™, FreeBSD™, or similar.

[0079] This invention can be a method, apparatus, system, and / or computer program product. The computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for performing various aspects of the invention.

[0080] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example—but not limited to—electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination thereof. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0081] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or border servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0082] The computer program instructions used to perform the operations of this invention may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing state information from the computer-readable program instructions. This electronic circuitry can execute the computer-readable program instructions to implement various aspects of the invention.

[0083] Various aspects of the present invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0084] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processing unit of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0085] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions that execute on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0086] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction, which contains one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0087] Note that, unless otherwise explicitly stated, all features disclosed in this specification (including any appended claims, abstract, and drawings) may be replaced by alternative features for achieving the same, equivalent, or similar purpose. Therefore, unless explicitly stated otherwise, each disclosed feature is merely one example of a set of equivalent or similar features. Where used, "further," "preferably," "even further," and "more preferably" are simple starting points for describing another embodiment based on the foregoing embodiments, the combination of which with the foregoing embodiments constitutes the complete configuration of another embodiment. Any combination of several "further," "preferably," "even further," or "more preferably" settings following the same embodiment constitutes yet another embodiment.

[0088] Those skilled in the art should understand that the embodiments of the present invention described above and shown in the accompanying drawings are merely examples and do not limit the present invention. The objectives of the present invention have been fully and effectively achieved. The functions and structural principles of the present invention have been demonstrated and explained in the embodiments, and any variations or modifications may be made to the implementation of the present invention without departing from the stated principles.

[0089] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this disclosure, and are not intended to limit them. Although this disclosure has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this disclosure.

Claims

1. A method for identifying fundus lesions, characterized in that, include: The retinal lesions contained in the target eye image are classified using a classification model to obtain the category of the retinal lesions. The classification model determines the category of retinal lesions by extracting and recognizing the color features of the target eye image. The categories of retinal lesions include red retinal lesions and white retinal lesions. The red lesion category includes subcategories such as microaneurysms and hemorrhages. The white lesion category includes subcategories such as exudates and drusen. For different types of fundus lesions, the segmentation results of fundus lesions are determined based on the corresponding pre-trained fundus lesion segmentation models; the pre-trained fundus lesion segmentation models include fundus hemorrhage lesion detection models, fundus microaneurysm lesion segmentation models, fundus exudative lesion detection models, and drusen lesion detection models. The segmentation results are classified by features to obtain the subcategories of fundus lesions corresponding to the segmentation results; Image segmentation of lesions of different subcategories was performed using a pre-trained fundus lesion segmentation model: A category recognition model is used to identify the type of fundus hemorrhage lesions in the target eye image, resulting in a category recognition result. If the category recognition result indicates that the fundus hemorrhage lesion belongs to the first category, a pre-trained fundus hemorrhage lesion detection model is used to detect the target eye image, resulting in a first initial image of the fundus hemorrhage lesion region. A pre-trained lesion segmentation model is used to segment the first initial image, resulting in a first image of the fundus hemorrhage lesion. Based on a preset image processing algorithm, the fundus hemorrhage lesion region is extracted from the target eye image, resulting in a second image of the fundus hemorrhage lesion. The intersection of the first initial image, the first image, and the second image is then performed to obtain the fundus hemorrhage lesion segmentation result corresponding to the target eye image.

2. The method for identifying fundus lesions according to claim 1, characterized in that, Before classifying the target eye image, the method further includes: Remove the background area from the eye image to be processed to obtain the first eye image; The first eye image is normalized to obtain the target eye image.

3. The method for identifying fundus lesions according to claim 1, characterized in that, Before classifying fundus lesions contained in the target eye image using a classification model, the method further includes: The initial image is detected using a preset detection model to determine whether the initial image contains first feature information; If the region does not contain the first feature information, then the region containing the second feature information is removed.

4. The method for identifying fundus lesions according to claim 3, characterized in that, The segmentation results are classified by feature to obtain the subcategories of fundus lesions corresponding to the segmentation results, including: The segmentation results are input into a pre-trained lesion recognition model to identify the features of the segmentation results; Output the subcategories of fundus lesions corresponding to the segmentation results.

5. The method for identifying fundus lesions according to claim 3, characterized in that, The segmentation results are input into a pre-established lesion identification model, and the sub-category information of the fundus lesions corresponding to the segmentation results is output, including: The segmentation results are input into a pre-established lesion identification model, and the output is the category of fundus microaneurysm lesion and fundus hemorrhage corresponding to each segmentation result under the red fundus lesion. And / or, the segmentation results are input into a pre-established lesion identification model, and the categories of fundus exudative lesions and fundus drusen lesions under the white fundus lesions are output.

6. A device for identifying fundus lesions, characterized in that, include: The first category module is configured to classify fundus lesions contained in a target eye image using a classification model to obtain the category of the fundus lesions. The classification model determines the category of fundus lesions by extracting and recognizing the color features of the target eye image. The categories of fundus lesions include red fundus lesions and white fundus lesions. The red lesion category includes subcategories such as microaneurysms and hemorrhages; the white lesion category includes subcategories such as exudates and drusen. The segmentation module is configured to determine the segmentation result of fundus lesions based on the corresponding pre-trained fundus lesion segmentation model for different categories of fundus lesions; the pre-trained fundus lesion segmentation model includes a fundus hemorrhage lesion detection model, a fundus microaneurysm lesion segmentation model, a fundus exudative lesion detection model, and a drusen lesion detection model. The second category module is configured to perform feature classification on the segmentation results to obtain subcategories of fundus lesions corresponding to each segmentation result under different categories of fundus lesions; Image segmentation of lesions of different subcategories was performed using a pre-trained fundus lesion segmentation model: A category recognition model is used to identify the type of fundus hemorrhage lesions in the target eye image, resulting in a category recognition result. If the category recognition result indicates that the fundus hemorrhage lesion belongs to the first category, a pre-trained fundus hemorrhage lesion detection model is used to detect the target eye image, resulting in a first initial image of the fundus hemorrhage lesion region. A pre-trained lesion segmentation model is used to segment the first initial image, resulting in a first image of the fundus hemorrhage lesion. Based on a preset image processing algorithm, the fundus hemorrhage lesion region is extracted from the target eye image, resulting in a second image of the fundus hemorrhage lesion. The intersection of the first initial image, the first image, and the second image is then performed to obtain the fundus hemorrhage lesion segmentation result corresponding to the target eye image.

7. The fundus lesion identification device according to claim 6, characterized in that, The device is further configured to: remove the background region of the eye image to be processed to obtain a first eye image; and normalize the first eye image to obtain the target eye image.

8. The fundus lesion identification device according to claim 6, characterized in that, The device also includes: The detection unit is configured to detect the initial image using a preset detection model to determine whether the initial image contains first feature information; if it does not contain first feature information, the region containing second feature information is removed.

9. A device for identifying fundus lesions, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to invoke instructions stored in the memory to execute the method according to any one of claims 1 to 5.

10. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When the computer program instructions are executed by the processor, they implement the method described in any one of claims 1 to 5.

Images