Method and device for classifying fundus multi-disease in macular area and storage medium

Abstract

The application discloses a macular region fundus multiple disease classification method and device and a storage medium, and belongs to the field of image processing. The method comprises the following steps: classifying the macular region image of the fundus photo through a multiple disease classification task of transfer learning to obtain a classification result; performing image preprocessing on the disease spot picture to obtain a segmented picture; generating disease feature description for the segmented picture based on a picture explainable algorithm to obtain pixel-level clues; performing picture enhancement based on a Gan network according to the pixel-level clues to obtain a synthetic picture; training based on the segmented picture and the synthetic picture to obtain a pixel-level-based macular region multiple disease classification system; identifying the disease spot of the fundus photo through the pixel-level-based macular region multiple disease classification system to obtain a disease spot identification result; and generating a disease feature heat map based on the disease spot identification result through the picture explainable algorithm. The application can help ophthalmologists better understand the principle and basis of the prediction model.

Description

Technical Field

[0001] This invention relates to the field of image processing, and in particular to a method, apparatus, and storage medium for classifying multiple diseases of the fundus in the macular region. Background Technology

[0002] In recent years, with the continuous improvement of medical imaging acquisition equipment and the ongoing development of disciplines such as image processing, pattern recognition, and machine learning, the interdisciplinary field of medical image processing and analysis has achieved fruitful results. These achievements are of great significance in assisting doctors to make rapid and accurate diagnoses.

[0003] While machine learning models can assist doctors in classifying and diagnosing diseases in the macular region of the retina, several challenges remain. For example, the amount of data available for learning is limited, especially for rare retinal diseases, and real-world data is difficult to obtain. Furthermore, accurately labeling large amounts of data is challenging. Labeling in the medical field, particularly pixel-based labeling, is particularly difficult and requires highly skilled labelers. Finally, simply outputting classification results does not allow doctors to understand the model's reasoning, thus hindering their evaluation and adoption of the model's classification results. Summary of the Invention

[0004] This invention aims to address at least one of the technical problems existing in the prior art. To this end, this invention proposes a method, device, and storage medium for classifying multiple fundus diseases in the macular region, which can help ophthalmologists better understand the principles and basis of predictive models.

[0005] This invention provides a method for classifying multiple diseases in the macular region of the fundus, comprising the following steps: classifying macular region images from fundus photographs using a multi-disease classification task based on transfer learning to obtain classification results; the fundus photographs include OCT fundus photographs and UWF fundus photographs; the classification results include lesion images; preprocessing the lesion images to obtain segmented images; generating disease feature descriptions from the segmented images based on an image interpretability algorithm to obtain pixel-level clues; performing image enhancement based on a Gaining Network (GAN) based on the pixel-level clues to obtain synthesized images; training the system based on the segmented images and the synthesized images to obtain a pixel-level macular region multi-disease classification system; identifying lesions in fundus photographs using the pixel-level macular region multi-disease classification system to obtain lesion identification results; and generating a disease feature heatmap based on the lesion identification results using an image interpretability algorithm.

[0006] According to some embodiments of the present invention, the method further includes: detecting the optic disc center and fovea of ​​the fundus photograph based on an image segmentation algorithm to obtain predicted values ​​and cross-entropy; obtaining a detection result based on the predicted values; the detection result includes the fovea of ​​the fovea; wherein the image segmentation algorithm includes, but is not limited to, FCN, Unet, or an image segmentation algorithm based on the positional relationship of the optic disc center.

[0007] According to some embodiments of the present invention, the image preprocessing of the lesion image to obtain a segmented image includes at least one of the following: correcting left and right eyes, correcting inverted images, image normalization, coarse segmentation based on the fovea, and image segmentation based on the FOV segmentation model; the coarse segmentation based on the fovea includes: dividing the OCT fundus image with the fovea as the center and a segmentation boundary of 10mm in diameter; and dividing the UWF fundus image with the fovea as the center and the midpoint connecting the fovea and the center of the test disc as the boundary.

[0008] According to some embodiments of the present invention, the method further includes: acquiring the background image after the coarse segmentation and marking the pixels, training it based on a precise FOV extraction algorithm to obtain the FOV segmentation model; the precise FOV extraction algorithm includes, but is not limited to, FCN, U-net, YOLO, and Inception.

[0009] According to some embodiments of the present invention, the image enhancement based on the Gann network to obtain the synthesized image includes: based on the Gann network, according to given pathological features and binary blood vessel segmentation, synthesizing fundus images with controllable lesion location and number to obtain the synthesized image.

[0010] According to some embodiments of the present invention, the segmented image and the synthesized image are referred to as lesion segmentation images. The step of training based on the segmented image and the synthesized image to obtain a pixel-level classification system for multiple macular diseases includes: pixel labeling of the lesion segmentation images using pixel numerical values; transfer learning of the lesion segmentation images based on a classification backbone model algorithm to obtain pixel-level classification cues; wherein, all layers of lesion segmentation images of type OCT fundus photography are unfrozen and trained, with training weights equal to initial weights; classification layers of lesion segmentation images other than those of type OCT fundus photography are unfrozen and trained; and an image-level classification model is obtained based on the pixel-level classification cues.

[0011] The image-level classification model is as follows:

[0012]

[0013] If the lesion is binary, then σ is the sigmoid activation function; otherwise, σ is the MSE squared loss function with a threshold set; where m represents the m-th lesion, M represents the number of lesions, and w m,n The initial values ​​are the neuron weights of the classification model, where m and M are positive integers.

[0014] According to some embodiments of the present invention, the classification of macular region images of fundus photographs through a multi-disease classification task via transfer learning includes: employing one or a variation of one of the following algorithms: AlexNet, VGG, GoogleNet, EfficientNet, ResNet, SENet, NiN, Wide ResNet, ResNext, DenseNet, FractalNet, MobileNets, NASNet.

[0015] Another embodiment of the present invention provides a macular region fundus disease classification device, comprising: a first module for classifying macular region images from fundus photographs using a multi-disease classification task based on transfer learning, and obtaining a classification result; the fundus photographs include OCT fundus photographs and UWF fundus photographs; the classification result includes lesion images; a second module for preprocessing the lesion images to obtain segmented images; a third module for generating disease feature descriptions from the segmented images based on an image interpretability algorithm, and obtaining pixel-level clues; a fourth module for performing image enhancement based on a Gaining Network (GAN) based on the pixel-level clues, and obtaining a synthesized image; a fifth module for training based on the segmented images and the synthesized images to obtain a pixel-level macular region multi-disease classification system; a sixth module for identifying lesions from fundus photographs using the pixel-level macular region multi-disease classification system, and obtaining lesion identification results; and a seventh module for generating a disease feature heatmap based on the lesion identification results using an image interpretability algorithm.

[0016] Another aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described above.

[0017] The embodiments of the present invention achieve at least the following beneficial effects: the embodiments of the present invention improve the fault tolerance of the classification model by inputting multiple types of fundus photos; generate disease feature heatmaps through image interpretability algorithms to provide ophthalmologists with real-world causal evidence for research, helping ophthalmologists to better understand the principles and basis of the prediction model, i.e. the features of disease identification; and achieve accurate classification of lesion types in fundus photos through deep learning.

[0018] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0019] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which:

[0020] Figure 1 This is a flowchart illustrating the method according to an embodiment of the present invention;

[0021] Figure 2 This is a schematic diagram of the training process of the macular region multi-disease classification system and FOV segmentation model according to an embodiment of the present invention;

[0022] Figure 3 This is a schematic block diagram of the device according to an embodiment of the present invention. Detailed Implementation

[0023] Embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0024] In the description of this invention, "several" means one or more, "multiple" means two or more, "greater than," "less than," and "exceeding" are understood to exclude the stated number, while "above," "below," and "within" are understood to include the stated number. The use of "first" and "second" in the description is merely for distinguishing technical features and should not be construed as indicating or implying relative importance, or implicitly indicating the number of indicated technical features, or implicitly indicating the order of the indicated technical features.

[0025] Definitions:

[0026] OCT: OCT angiography is a non-contact, high-resolution fundus imaging system. Its advantages include the application of split-spectrum amplitude-deeorrelation angiography (SSADA) algorithms, enabling rapid, high-resolution fundus vascular imaging without the need for contrast agents, and high repeatability. Furthermore, it employs an En-face mode combining traditional transverse (B-scan) and coronal (C-scan) scanning to acquire three-dimensional images, allowing direct observation of the extent and depth of ocular lesions, as well as changes in the length, diameter, and area of ​​neovascularization, providing significant clinical guidance.

[0027] UWF: Ultra-wide-angle fundus imaging. The advent of ultra-wide-angle fundus imaging (UWF) has made it possible to see almost the entire fundus through a small pupil, no longer science fiction. UWF is easy to use, allowing visualization of 82.5% of the retina without pupil dilation. The ultra-wide-angle retinal imaging reaches 200°, providing a near-panoramic fundus image. UWF can not only acquire ultra-wide-angle pseudo-color fundus images, but also capture ultra-wide-angle fundus autofluorescence (FAF) images, fluorescein fundus angiography (FFA), and choroidal angiography (ICGA) images. The application of this new ultra-wide-angle fundus examination and diagnostic technology has given us new insights into the screening, classification, and treatment of some common clinical diseases.

[0028] The optic disc and fovea: The optic disc is the origin of the optic nerve and blood vessels, appearing as a bright white, disc-shaped organ approximately 1.5 mm in diameter. The macula is located 3.5 mm temporally and slightly below the optic disc, and is the part with the sharpest vision. The macula and optic disc are important features in fundus imaging and play a crucial role in fundus image analysis.

[0029] FOV segmentation: The field of view, which is the effective area of ​​focus for the target.

[0030] Interpretable algorithms: Many machine learning models can make very good predictions, but they cannot explain how they make those predictions. Many data scientists find it difficult to understand why the algorithm arrives at such a result. For the classification and diagnosis of multiple retinal diseases, existing methods generally directly output the classification results, leaving doctors unable to understand the basis for these results. Interpretable algorithms, on the other hand, can provide a basis for judgment.

[0031] GAN (Generative Adversarial Network) consists of two models: a generative model and a discriminative model. The generative model's task is to generate instances that appear natural and realistic, similar to the original data. The discriminative model's task is to determine whether a given instance appears natural and realistic or is artificially created (real instances come from the dataset, while fake instances come from the generative model).

[0032] Disease feature heatmap: Based on the weight values ​​of the last layer of the neural network, pixels with high weights are displayed, thus forming feature visualization. This step simply explains to medical staff the principle behind the model's prediction: the prediction result is obtained because these displayed pixels are more important. These pixels with high weights are very likely the reason why the patient is judged to be positive, that is, the location of their disease feature values. This knowledge serves as a visualization tool to assist medical staff in understanding and diagnosis.

[0033] Pixel marking: Marking software is used to plot points and connect them along the segmented areas of the image. The marked areas will assign a special value to each pixel, usually fluorescent or white.

[0034] Reference Figure 1 The method of this invention includes the following steps:

[0035] S100. The macular region image of the fundus photograph is classified using a multi-disease classification task based on transfer learning to obtain the classification result. The fundus photographs include, but are not limited to, OCT fundus photographs and UWF fundus photographs. The classification result includes lesion images. This embodiment uses multiple data sources, including but not limited to OCT fundus photographs and UWF fundus photographs, to achieve data collection and input from multiple data sources, including OCT / UWF fundus photographs. This embodiment uses multiple data sources to improve the model's fault tolerance, because clinical data may be one of these categories. If only OCT or fundus photographs are set, the model cannot be used if the data source is different in actual clinical practice. Just like fundus photographs are different, there are 45° angles, 98° angles, and 200° angles. Different angles may lead to different diagnostic results.

[0036] S200. Perform image preprocessing on the lesion images to obtain segmented images. The original fundus images need to undergo processing including left and right eye correction, image inversion correction, image normalization, and lesion segmentation to meet the needs of subsequent model training and lesion classification. This preprocessing step improves the accuracy of lesion classification.

[0037] S300. Based on image interpretability algorithms, generate disease feature descriptions for segmented images to obtain pixel-level clues. Although the fundus photograph has been classified, it is unclear in this embodiment which pixels in the fundus photograph are used to determine the classification. This step uses image interpretability algorithms to mark feature regions and generate disease feature descriptions, thereby obtaining pixel-level clues based on the original fundus photograph. It is understood that the interpretable algorithms in this embodiment include, but are not limited to, multiple image interpretability algorithms such as CAM, GAP, and Grad-CAM.

[0038] S400. Based on pixel-level clues, image enhancement is performed using a GA network to obtain a synthesized image. For situations with limited data, and even fewer fundus images of rare fundus diseases, this embodiment uses a generative adversarial network (GAN) for data enhancement. It is understood that this embodiment generates images based on given pathological features and binary vessel segmentation, and the location and number of lesions are completely controllable. Generating images based on binary vessels in this embodiment avoids obscuring blood vessels in the generated image. The image enhancement principle of this embodiment is also performed within the aforementioned macular region.

[0039] S500 is trained based on segmented and synthesized images to obtain a pixel-level classification system for multiple diseases in the macular region.

[0040] S600 uses a pixel-level macular region multi-disease classification system to identify lesions in fundus images and obtains lesion identification results.

[0041] S700. Based on the lesion recognition results, a symptom feature heatmap is generated using an image-interpretable algorithm. In this embodiment, the feature heatmap obtained in this step is based on the weight values ​​of the last layer of the neural network. Pixels with high weights are displayed, thus forming feature visualization. This step can explain to medical staff the principle behind the model's prediction: the prediction result is obtained because these displayed pixels are more important. These pixels with high weights are very likely the reason why the patient is judged to be positive, that is, the location of their symptom feature values.

[0042] Understandably, before proceeding to step S100, fovea detection needs to be performed on the original fundus image. This specifically includes: detecting the optic disc center and fovea of ​​the fovea based on an image segmentation algorithm, obtaining predicted values ​​and cross-entropy; obtaining the detection results based on the predicted values; and the detection results including the fovea of ​​the fovea. The predicted values ​​are... The cross-entropy loss function is Where δ I,n Let F represent the true value, F represent the image number, and N represent the number of categories. It should be understood that image segmentation algorithms include, but are not limited to, FCN, Unet, or image segmentation algorithms based on the positional relationship of the optic disc center. After locating the fovea and the center of the optic disc, the macular region can be divided, and lesion identification can be performed within this region, rather than the entire image.

[0043] In some embodiments, the lesion image is preprocessed to obtain a segmented image that includes at least one of the following: correcting left and right eyes, correcting inverted images, image normalization, coarse segmentation based on the fovea, and image segmentation based on the FOV segmentation model; wherein, the coarse segmentation based on the fovea includes: segmenting the OCT fundus image with the fovea as the center and a segmentation boundary of 10mm as the diameter; and segmenting the UWF fundus image with the fovea as the center and the midpoint connecting the fovea and the center of the test disc as the boundary.

[0044] In some embodiments, the method of this invention further includes: acquiring a background image after coarse segmentation and marking its pixels, training it based on a precise FOV extraction algorithm to obtain the FOV segmentation model; the precise FOV extraction algorithm includes, but is not limited to, FCN, U-net, YOLO, and Inception. Specific training steps are described below. Figure 2 After segmenting the lesions, the background image is pixel-labeled and marked as 0. OCT images differ significantly from other fundus images; therefore, in the transfer learning of the fundus image model based on OCT classification, all layers are unfrozen for learning, with training weights equal to initial weights. For other fundus images, the classification layer is directly unfrozen for transfer learning. The predicted values ​​from the FOV algorithm are accurately extracted. The cross-entropy loss function is Accuracy fov .

[0045] Reference Figure 2 In some embodiments, the segmented image and the synthesized image are referred to as lesion segmentation images. The training process of the pixel-level macular region multi-disease classification system includes:

[0046] Prepare the data, such as Figure 2 As shown, the input images consist of K datasets and M target lesions. For example: OCT input data (number of lesions M); fundus photograph input data 1 (number of lesions a); fundus photograph input data 1 (number of lesions b); fundus photograph input data 1 (number of lesions c)..., where a>b>c>...

[0047] Then, a coarse segmentation of multiple data sources (FOV image segmentation) is implemented.

[0048] Pixel labeling is performed on the lesion segmentation image, denoted by pixel values; wherein, the pixel labeling includes: lesion data labeled as n, Background data is marked as 0; and pixel labels are unique. In this embodiment, a point-marking method is used with marking software to draw lines along the image segmentation area. The marked area will mark the pixel with a special value, usually fluorescent or white.

[0049] The classification backbone model algorithm is used to perform transfer learning on lesion segmentation images to obtain pixel-level classification and recognition clues; among them, the classification backbone model algorithm... The model employs, but is not limited to, FCN, U-net, YOLO, and Inception. Specifically, for lesion segmentation images of type OCT fundus photography, all layers are unfrozen for learning, with training weights equal to the initial weights; for lesion segmentation images other than OCT fundus photography, the classification layer is unfrozen for learning; based on the pixel-level classification cues, an image-level classification model is obtained. Lesion prediction value: Lesion cross-entropy loss function: Accuracy c .

[0050] In this embodiment, the image level classification model is:

[0051]

[0052] If the condition is binary, then σ is the sigmoid activation function; otherwise, σ is the MSE squared loss function. And set a threshold; where m represents the m-th lesion, M represents the number of lesions, and w m,n The initial values ​​are the neuron weights of the classification model. Accuracy is... m .

[0053] This embodiment realizes pixel-level lesion recognition and image-level lesion recognition through a classification backbone model algorithm. Image-level lesion recognition is a fusion of pixel-level classification recognition clues.

[0054] In some embodiments, classifying macular region images from fundus photographs using a multi-disease classification task via transfer learning includes employing one or a variation of one of the following algorithms: AlexNet, VGG, GoogleNet, EfficientNet, ResNet, SENet, NiN, Wide ResNet, ResNext, DenseNet, FractalNet, MobileNets, and NASNet.

[0055] In some embodiments, the evaluation criteria for the pixel-level macular region multi-disease classification system of the present invention are as follows:

[0056] 1. Loss function: w2>w1>w3; If the FOV-based precise image segmentation model is not used, then ζ primary_fov =0.

[0057] 2. Accuracy: If FOV is not used to accurately extract image segmentation models, then ζ primary_fov =0.

[0058] Reference Figure 3 Another embodiment of the present invention provides a macular region fundus disease classification device, comprising: a first module for classifying macular region images from fundus photographs using a multi-disease classification task based on transfer learning, and obtaining classification results; the fundus photographs include OCT fundus photographs and UWF fundus photographs; the classification results include lesion images; a second module for preprocessing the lesion images to obtain segmented images; a third module for generating disease feature descriptions from the segmented images based on an image interpretability algorithm, and obtaining pixel-level clues; a fourth module for image enhancement based on a Gaining Network (GAN) based on the pixel-level clues, and obtaining synthesized images; a fifth module for training based on the segmented images and synthesized images to obtain a pixel-level macular region multi-disease classification system; a sixth module for identifying lesions from fundus photographs using the pixel-level macular region multi-disease classification system, and obtaining lesion identification results; and a seventh module for generating a disease feature heatmap based on the lesion identification results using an image interpretability algorithm.

[0059] Another aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described above.

[0060] Although specific embodiments are described herein, those skilled in the art will recognize that many other modifications or alternative embodiments are also within the scope of this disclosure. For example, any of the functions and / or processing capabilities described in connection with a particular device or component can be performed by any other device or component. Furthermore, while various exemplary embodiments and architectures have been described according to embodiments of this disclosure, those skilled in the art will recognize that many other modifications to the exemplary embodiments and architectures described herein are also within the scope of this disclosure.

[0061] The foregoing description, with reference to block diagrams and flowcharts of systems, methods, systems, and / or computer program products according to exemplary embodiments, has described certain aspects of this disclosure. It should be understood that one or more blocks in the block diagrams and flowcharts, as well as combinations of blocks in the block diagrams and flowcharts, can be implemented by executing computer-executable program instructions, respectively. Similarly, according to some embodiments, some blocks in the block diagrams and flowcharts may not need to be executed in the order shown, or may not all need to be executed. Furthermore, additional components and / or operations beyond those shown in the blocks in the block diagrams and flowcharts may exist in some embodiments.

[0062] Therefore, blocks in block diagrams and flowcharts support combinations of means for performing a specified function, combinations of elements or steps for performing a specified function, and program instruction means for performing a specified function. It should also be understood that each block in a block diagram and flowchart, and combinations of blocks in block diagrams and flowcharts, can be implemented by a dedicated hardware computer system or a combination of dedicated hardware and computer instructions that performs a specific function, element, or step.

[0063] The program modules, applications, etc., described herein may include one or more software components, including, for example, software objects, methods, data structures, etc. Each such software component may include computer-executable instructions that, in response to execution, cause at least a portion of the functionality described herein (e.g., one or more operations of the exemplary methods described herein) to be performed.

[0064] Software components can be coded using any of a variety of programming languages. An exemplary programming language could be a low-level programming language, such as assembly language associated with a specific hardware architecture and / or operating system platform. Software components including assembly language instructions may need to be converted into executable machine code by an assembler before being executed by the hardware architecture and / or platform. Another exemplary programming language could be a higher-level programming language that is portable across multiple architectures. Software components including higher-level programming languages ​​may need to be converted into an intermediate representation by an interpreter or compiler before execution. Other examples of programming languages ​​include, but are not limited to, macro languages, shell or command languages, job control languages, scripting languages, database query or search languages, or report writing languages. In one or more exemplary embodiments, a software component containing instructions from one of the above-described programming language examples can be executed directly by the operating system or other software components without first being converted into another form.

[0065] Software components can be stored as files or other data storage structures. Software components of similar type or related function can be stored together in a specific directory, folder, or library. Software components can be static (e.g., pre-defined or fixed) or dynamic (e.g., created or modified at runtime).

[0066] The embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the present invention is not limited to the above embodiments. Within the scope of knowledge possessed by those skilled in the art, various changes can be made without departing from the spirit of the present invention.

Claims

1. A method for classifying multiple fundus diseases in the macular region, characterized in that, Includes the following steps: Based on image segmentation algorithms, the optic disc center and fovea of ​​the macula are detected in fundus images to obtain predicted values ​​and cross-entropy. The detection results are obtained based on the predicted values; the detection results include the fovea of ​​the macula. The image segmentation algorithm mentioned includes, but is not limited to, FCN, Unet, or image segmentation algorithms based on the optic disc center position relationship, with the predicted value being... The cross-entropy loss function is ,in Represents the actual value. Indicates which photo, Indicates the number of categories; The macular region images from fundus photographs are classified using a multi-disease classification task based on transfer learning to obtain classification results; the fundus photographs include OCT fundus photographs and UWF fundus photographs; the classification results include lesion images; The lesion images are preprocessed to obtain segmented images; Based on an image interpretability algorithm, a disease feature description is generated from the segmented image to obtain pixel-level clues; Based on the pixel-level clues, image enhancement is performed using a Gann network to obtain a synthesized image. A pixel-level classification system for multiple macular diseases is obtained by training the segmented images and the synthesized images, including: pixel labeling of the lesion segmentation images with pixel values, wherein the segmented images and the synthesized images are denoted as lesion segmentation images; transfer learning of the lesion segmentation images based on a classification backbone model algorithm to obtain pixel-level classification and recognition cues; wherein, all layers of lesion segmentation images of type OCT fundus photography are unfrozen and trained, with training weights equal to initial weights; classification layers of lesion segmentation images other than type OCT fundus photography are unfrozen and trained; and an image-level classification model is obtained based on the pixel-level classification and recognition cues; wherein the image-level classification model is: If the symptoms are classified into two categories, then If it is a sigmoid activation function, otherwise Set the MSE squared loss function and set the threshold; where, Indicates the first A lesion, Indicates the number of lesions. The initial values ​​are the neuron weights of the classification model. and It is a positive integer; The lesion identification results are obtained by using the pixel-level macular region multi-disease classification system to identify lesions in fundus images; Based on the lesion identification results, a heatmap of lesion features is generated using an image interpretability algorithm.

2. The method for classifying multiple macular fundus diseases according to claim 1, characterized in that, The image preprocessing of the lesion image to obtain a segmented image includes at least one of the following: Correction of left and right eyes, correction of inverted images, image normalization, coarse segmentation based on fovea and image segmentation based on FOV segmentation model; The coarse segmentation based on the fovea of ​​the macula includes: The OCT fundus images were divided with the fovea of ​​the macular region as the center and a diameter of 10mm as the dividing boundary; and The UWF fundus image was divided with the fovea of ​​the macula as the center and the midpoint connecting the fovea of ​​the macula and the center of the test disc as the boundary.

3. The method for classifying multiple fundus diseases in the macular region according to claim 2, characterized in that, The method further includes: The background image after the coarse segmentation is obtained and pixel-marked. The image is then trained based on the precise FOV extraction algorithm to obtain the FOV segmentation model.

4. The method for classifying multiple macular fundus diseases according to claim 3, characterized in that, The precise FOV extraction algorithm includes, but is not limited to, FCN, U net, Yolo, Inception.

5. The method for classifying multiple macular fundus diseases according to claim 1, characterized in that, The image enhancement based on the Gann network to obtain the synthesized image includes: Based on the GA network, fundus images with controllable lesion location and number are synthesized according to given pathological features and binary blood vessel segmentation to obtain synthesized images.

6. The method for classifying multiple macular fundus diseases according to claim 1, characterized in that, The classification of macular region images from fundus photographs using a multi-disease classification task based on transfer learning includes: The following algorithms or variations thereof are used: AlexNet, VGG, GoogleNet, EfficientNet, ResNet, SENet, NiN, Wide ResNet, ResNext, DenseNet, FractalNet, MobileNets, NASNet.

7. A device for classifying multiple diseases of the fundus in the macular region, characterized in that, include: The first module is used to classify the macular region images of fundus photographs through a multi-disease classification task using transfer learning, and obtain the classification results; The fundus photographs include OCT fundus photographs and UWF fundus photographs; the classification results include lesion images; The second module is used to preprocess the lesion image to obtain a segmented image; The third module is used to generate a description of disease features from the segmented image based on an image interpretability algorithm, thereby obtaining pixel-level clues. The fourth module is used to perform image enhancement based on the pixel-level clues using a Gann network to obtain a synthesized image; The fifth module is used to train the system based on the segmented image and the synthesized image to obtain a pixel-level classification system for multiple macular diseases. The sixth module is used to identify lesions in fundus photographs using the pixel-level macular region multi-disease classification system to obtain lesion identification results; The seventh module is used to generate a heatmap of disease features based on the lesion recognition results using an image interpretable algorithm; The macular region fundus multiple disease classification device is used to perform the macular region fundus multiple disease classification method as described in any one of claims 1 to 6.

8. A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method of any one of claims 1 to 6.

Images