A Domain Adaptation Method Based on an Asymmetric Two-Classifier Model

Abstract

This invention provides a domain adaptation method based on an asymmetric dual classifier model. The method includes: acquiring a training set, which includes multiple source domain samples, multiple target domain samples, and labels indicating the fundus category to which each source domain sample belongs, wherein both the source domain samples and the target domain samples are fundus images; and acquiring a dual classifier model, which includes: performing multiple adversarial trainings on a feature extractor and two classifiers using the training set based on forward distribution compensation values ​​and backward distribution compensation values ​​to obtain a trained dual classification model.

Description

Technical Field

[0001] This invention relates to the field of neural network technology, specifically to the field of medical image grading based on neural networks, and more specifically, to a domain adaptation method based on an asymmetric dual classifier model. Background Technology

[0002] With the rapid growth of medical image data and the advancement of medical technology, using computer vision and machine learning techniques for medical image analysis has become an important tool. This method can help doctors quickly and accurately assess the severity of diseases, enabling them to provide targeted medical care to patients.

[0003] However, image-level supervised medical grading tasks face some difficulties and challenges. Taking diabetic retinopathy (DR) grading as an example, according to the International Clinical Classification of Diabetic Retinopathy (ICSC), diabetic retinopathy (DR0) is divided into five levels: no retinopathy, mild non-proliferative diabetic retinopathy (DR1), moderate non-proliferative diabetic retinopathy (DR2), severe non-proliferative diabetic retinopathy (DR3), and proliferative diabetic retinopathy (DR4). Through analysis, for example, ... Figure 1 As shown, the four main problems that make fundus image grading tasks (or fundus classification tasks that divide fundus images into multiple categories) difficult are as follows:

[0004] First, there's the issue of domain differences between the source and target domains. In practical applications, when using models for DR (Digital Retention) classification, differences in equipment, acquisition scenarios (such as environment, lighting, and the relative posture of personnel and equipment), and acquisition regions (where differences may exist in the fundus examination of personnel) lead to various discrepancies between the source domain data used to train the model and the target domain data required for the model's actual prediction. In other words, there are often significant differences between training data and test data. For example... Figure 1 The fundus images obtained from different locations in three real hospitals, as shown in figure 'a', exhibit visual domain differences (also referred to as inter-domain shifts in some literature). These domain differences may degrade classification performance. Domain differences across centers and across device vendors are defined as covariate shifts in the data. Covariates refer to the input variables of the model. Covariate shifts indicate that the input variables in the training and test sets (or the actual input data used in the application) have different data distributions; that is, the input distribution changes and no longer conforms to the independent and identically distributed assumption in traditional machine learning.

[0005] Second, there is the problem of insignificant differences in lesion characteristics between adjacent grades; such as... Figure 1As shown in b, an original image and its corresponding annotated image (i.e., the lesion location is marked on the original image) are provided. It can be seen that the lesion in the fundus image is small. This results in insignificant differences in lesion characteristics between adjacent grades, further increasing the possibility of misclassification, making it easier for a few grades to be classified as the majority of adjacent grades. The clinical sign of mild DR (DR1) is the presence of small aneurysms in the fundus, which usually manifests as subtle color changes of a few pixels in the fundus image. Meanwhile, the proportion of DR1 in the population is extremely low. It has been proven that instances of DR1 are frequently misclassified as DR0 or DR2.

[0006] Third, the uneven distribution of sample sizes across different fundus categories leads to label shift issues. For example... Figure 1 As shown in Figure c, the distribution of the number of each level in the three diabetic retinopathy datasets (DDR, Messidor, and EyePACS) exhibits a high degree of imbalance, regardless of whether it's a publicly available dataset or a dataset actually used in hospitals. This imbalanced data distribution naturally biases a general empirical risk-minimizing classifier towards these majority classes. The label distribution of DR levels is severely skewed. Furthermore, there is a difference in label distribution between the source and target domains, i.e., label shift.

[0007] Fourth, pixel-level lesion annotation is expensive, and even with image-level annotation, it still requires annotation within the target domain; pixel-level annotation of lesions on retinal images (such as...) Figure 1 (d) is expensive, and there are few publicly available samples of data. Summary of the Invention

[0008] Therefore, the purpose of this invention is to overcome the shortcomings of the prior art and provide a domain adaptation method based on an asymmetric dual classifier model.

[0009] The objective of this invention is achieved through the following technical solution:

[0010] According to a first aspect of the present invention, a domain adaptation method based on an asymmetric dual classifier model is provided, comprising: acquiring a training set including multiple source domain samples, multiple target domain samples, and labels indicating the fundus category to which each source domain sample belongs, wherein both the source domain samples and the target domain samples are fundus images; acquiring a dual classifier model including: a feature extractor for extracting fundus features from the input fundus images, a conventional classifier for classifying based on a first Logit vector determined by the fundus features, and a Bayesian classifier for classifying based on a second Logit vector determined by the fundus features, wherein both the first Logit vector and the second Logit vector include the Logit value corresponding to each fundus category in the fundus lesion grading system, and the conventional classifier is constructed based on empirical risk minimization; and performing multiple adversarial trainings on the feature extractor and the two classifiers using the training set to obtain a trained dual classification model, wherein each adversarial training... The training includes: inputting each source domain sample into the current dual classifier model to obtain the first Logit vector and the second Logit vector corresponding to each source domain sample; inputting each target domain sample into the current dual classifier model to obtain the first Logit vector and the second Logit vector corresponding to each target domain sample; with the parameters of the feature extractor fixed, updating the parameters of the two classifiers based on the first sub-loss of each source domain sample on the conventional classifier and the second sub-loss on the Bayesian classifier. The first sub-loss is determined by the difference between the Logit value of each fundus category in the first Logit vector corresponding to each source domain sample and the positive distribution compensation value that is positively correlated with the proportion of source domain samples of that fundus category, and the label. The second sub-loss is determined by the difference between the Logit value of each fundus category in the second Logit vector corresponding to each source domain sample and the negative distribution compensation value that is negatively correlated with the proportion of source domain samples of that fundus category, and the label.

[0011] Optionally, the positive distribution compensation value corresponding to a fundus category is the logarithm of the proportion of source domain samples of that fundus category in the training set, or the positive distribution compensation value corresponding to a fundus category is the product of the logarithm of the proportion of source domain samples of that fundus category in the training set and a preset hyperparameter for measuring the degree of distribution compensation.

[0012] Optionally, the inverse distribution compensation value corresponding to a fundus category is the logarithm of the inverse of the proportion of the source domain samples of that fundus category in the training set, or the inverse distribution compensation value corresponding to a fundus category is the product of the logarithm of the inverse of the proportion of the source domain samples of that fundus category in the training set and a preset hyperparameter for measuring the degree of distribution compensation.

[0013] Optionally, with the parameters of the feature extractor fixed, the loss used to update the parameters of the two classifiers is determined as follows:

[0014]

[0015] in, This represents the mean of the first classifier loss for each source domain sample used in the current adversarial training. Let α represent the mean of the second classifier loss for each source domain sample used in the current adversarial training. The weight, β represents The weight.

[0016] Optionally, with the parameters of the two classifiers fixed, the loss used to update the parameters of the feature extractor is determined as follows:

[0017]

[0018] in, This indicates that the current adversarial training uses samples x from each source domain based on the feature extractor F(·) using a conventional classifier. s The softmax value of the first Logit vector determined by the extracted fundus features is calculated and then compared with the mean of the classifier loss calculated from the labels. This indicates that the Bayesian classifier, based on the feature extractor F(·), performs the analysis of each source domain sample x used in the current adversarial training. s The softmax value of the second Logit vector determined by the extracted fundus features is calculated, and then the mean of the classifier loss calculated from the labels is taken, π. s This represents the label distribution vector, which is composed of the proportion of source domain samples in each fundus category in the training set. γ1 represents the mean difference between the two classifiers in classifying each target domain sample during the current adversarial training. The weight, γ2 represents The weight, γ3 represents The weight.

[0019] Optionally, each adversarial training session may further include updating the parameters of the two classifiers and the parameters of the feature extractor based on the classifier loss of each source domain sample.

[0020] Optionally, in the step of updating the parameters of the two classifiers and the feature extractor based on the classification loss of each source domain sample by the two classifiers, the loss used to update the parameters of the two classifiers and the feature extractor is determined in the following manner:

[0021]

[0022] Among them, among them, This indicates that the current adversarial training uses samples x from each source domain based on the feature extractor F(·) using a conventional classifier. s The softmax value of the first Logit vector determined by the extracted fundus features is calculated and then compared with the mean of the classifier loss calculated from the labels. This indicates that the Bayesian classifier, based on the feature extractor F(·), performs the analysis of each source domain sample x used in the current adversarial training. s The softmax value of the second Logit vector determined by the extracted fundus features is calculated, and then the mean of the classifier loss calculated from the labels is used. γ1 represents... The weight, γ2 represents The weight.

[0023] Optionally, the set of tags includes no retinopathy, mild nonproliferative diabetic retinopathy, moderate nonproliferative diabetic retinopathy, severe nonproliferative diabetic retinopathy, and proliferative diabetic retinopathy. Alternatively,

[0024] Optionally, the set of tags includes normal, early stage of age-related macular degeneration, and late stage of age-related macular degeneration.

[0025] Optionally, the set of tags includes normal, grade I hypertensive retinopathy, grade II hypertensive retinopathy, grade III hypertensive retinopathy, and grade IV hypertensive retinopathy.

[0026] According to a second aspect of the present invention, a method for fundus type identification based on fundus images is provided, the method comprising: acquiring a fundus image to be identified from a user; acquiring at least one classifier among a feature extractor, a conventional classifier, and a Bayesian classifier in a trained dual classifier model obtained according to the method of the first aspect; and identifying the fundus category to which the fundus image to be identified belongs using the feature extractor of the trained dual classifier model and the at least one classifier.

[0027] Optionally, the process of identifying the fundus category to which the fundus image to be identified belongs using the feature extractor of the trained dual classifier model and the at least one classifier includes: extracting fundus features corresponding to the fundus image to be identified from the fundus image to be identified using the feature extractor of the trained dual classifier model; and determining the confidence level of the fundus image to be identified belonging to each fundus category in the fundus disease grading based on the fundus features corresponding to the fundus image to be identified using at least one classifier of the trained dual classifier model.

[0028] According to a third aspect of the invention, a computer-readable storage medium is provided having a computer program stored thereon, the computer program being executable by a processor to implement the steps of the method described in any one of the first and / or second aspects.

[0029] According to a fourth aspect of the present invention, an electronic device is provided, comprising: one or more processors; and a memory for storing executable instructions; wherein the one or more processors are configured to implement the steps of the method described in any one of the first and / or second aspects by executing the executable instructions. Attached Figure Description

[0030] The embodiments of the present invention will be further described below with reference to the accompanying drawings, wherein:

[0031] Figure 1 A schematic diagram illustrating the problems existing in the prior art;

[0032] Figure 2 This is a schematic diagram illustrating the structural principle of a dual classifier model according to an embodiment of the present invention;

[0033] Figure 3 This is a schematic diagram illustrating the principle of adversarial training according to an embodiment of the present invention;

[0034] Figure 4 This is a schematic diagram illustrating the principle of classifying differences between two classifiers for a single target domain sample according to an embodiment of the present invention. Detailed Implementation

[0035] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative and are not intended to limit the invention.

[0036] As mentioned in the background section, in the task of grading and identifying fundus lesions based on fundus images, four main factors affect the accuracy of grading. By formulating the problem with these four factors as a grading imbalance domain adaptation problem, the purpose of this invention is to propose a domain adaptation method based on an asymmetric dual-classifier model (or a grading imbalance domain adaptation method based on minimizing the difference between asymmetric dual-classifiers, or an adversarial training method based on minimizing the difference between asymmetric dual-classifiers, or an adversarial training method based on an asymmetric dual-classifier model), aiming to solve the fundus classification task with grading imbalance and domain shift in practical problems.

[0037] To address the issue of domain discrepancies between the source and target domains, this invention solves the problem by involving both source and target domain samples in adversarial training. To address the issue of insignificant differences in lesion features between adjacent grades, this invention enhances the feature extractor's ability to extract grade-related fundus features through adversarial training of a feature extractor and a dual classifier—namely, classifiers based on both Bayesian optimality and empirical risk minimization strategies. To address the label shift caused by uneven distribution of samples across different fundus categories (i.e., grades), this invention compensates for this with forward and reverse distribution compensation values. These forward and reverse compensation values ​​can effectively mitigate the label shift caused by uneven distribution of samples across different fundus categories (i.e., grades). For some easily confused samples, the feature extractor is forced to capture essential lesion features between levels, and the distance between the convex hulls of features of adjacent levels is expanded in the latent space to discover essential difference features between levels, thereby solving the label offset problem. Addressing the issue of expensive pixel-level lesion annotation and the need for annotation in the target domain even with image-level annotation, this invention uses image-level annotation for adversarial training, requiring only labels for source domain samples and no labels for target domain samples. Supervision of the target domain is achieved through the prediction difference between the two classifiers (i.e., the distance sub-loss between the first and second Logit vectors of each target domain sample). This reduces annotation costs while improving prediction performance in the target domain.

[0038] To facilitate intuitive understanding, let's first simplify the overall application process:

[0039] K1. Obtain set A, which includes multiple source domain samples and labels indicating the fundus category to which each source domain sample belongs (e.g., a set A of multiple source domain samples and their labels extracted from an existing dataset of fundus image-annotated diabetic retinopathy grading labels).

[0040] K2. Obtain set B, which includes multiple target domain samples (e.g., multiple target domain samples are set B composed of fundus images collected from multiple patients. Due to the aforementioned four issues, the classification model obtained by training a traditional feature extractor + conventional classifier using only set A may have poor classification performance on the target domain corresponding to set B).

[0041] K3. Obtain the initial dual classifier model. Based on the adversarial training technique of this invention, use set A and set B to perform multiple iterative adversarial training on its feature extractor and two classifiers to obtain the trained dual classification model.

[0042] K4. Based on a trained dual-classification model, identify the fundus type of fundus images in the target domain.

[0043] To further illustrate the technical solution of this invention, the following descriptions will cover training data, model structure, training process, training system, and application scenarios.

[0044] I. Training Data

[0045] Unlike traditional training scenarios, considering that labeling target domain samples still incurs costs, the target domain samples in the training data of this invention do not require labels; only the source domain samples are labeled accordingly.

[0046] The source of the samples can be selected according to the implementer's needs. For example, the samples can come from datasets corresponding to fundus disease grading, such as diabetic retinopathy grading, age-related macular degeneration grading, and hypertension-related fundus disease grading. According to one embodiment of the present invention, multiple fundus images and their labels extracted from the dataset corresponding to the preset fundus classification task are used as multiple source domain samples and their corresponding labels in the training set, where the label is the fundus category to which each source domain sample belongs. Taking the dataset for diabetic retinopathy grading as an example, it can be at least one dataset or a combination of the IDRid dataset, DDR dataset, Messidor dataset, and EyePACS dataset. The target domain samples in the training set come from fundus images of users in the scenario to be predicted. Of course, it should be understood that in this field, experiments are often required to verify the feasibility and effectiveness of the technical solution, and the target domain must have labels to observe its effect. Therefore, in experiments, an existing first dataset (such as the DDR dataset or the Messidor dataset) can be considered as the source of the source domain samples, and another existing second dataset (such as the EyePACS dataset) can be considered as the source of the target domain samples. Multiple fundus images (without labels) from the second dataset can be used as multiple target domain samples, and the remaining fundus images and their labels can be used as the test set to test the performance of the trained model in target domain recognition. Alternatively, within the same dataset, source domain samples and target domain samples can be divided according to different populations (or different users and / or different acquisition devices) of the fundus images. Moreover, the number of source domain samples for different fundus categories in the training set can be imbalanced. However, it should be understood that even if the number of source domain samples for different fundus categories in the training set is balanced, the principle of training with this invention is still feasible.

[0047] The type of fundus indicated by the label can be set according to the implementer's needs. According to one embodiment of the present invention, when the preset fundus classification task is the grading of diabetic retinopathy, the set of labels (also referred to in some literature as the label space, i.e., all fundus categories, i.e., the set of fundus categories in the preset fundus classification task) includes no retinopathy, mild non-proliferative diabetic retinopathy, moderate non-proliferative diabetic retinopathy, severe non-proliferative diabetic retinopathy, and proliferative diabetic retinopathy. It should be understood that the set of labels can also be set according to the implementer's needs. For example, it may differ in some other classification standards. For instance, when diabetic retinopathy is graded into two levels, the set of labels includes the non-proliferative and proliferative phases; or, when diabetic retinopathy is graded into six stages, according to the 1985 grading standard of the Chinese Medical Association Fundus Disease Group, the set of labels includes stages I, II, III, IV, V, and VI. The definitions of each stage are existing and will not be elaborated here. Other preset fundus classification tasks are also feasible. For example, when the preset fundus classification task is age-related macular degeneration (AMD), the corresponding label set would be: normal (no AMD), early-stage AMD, and late-stage AMD; or when divided into two categories, the corresponding label set would be: normal (no AMD) and AMD. Another example: when the preset fundus classification task is the grading of hypertensive retinopathy, it can be divided into five grades based on the condition, with the corresponding label set being: normal (no hypertensive retinopathy), grade 1 hypertensive retinopathy (retinal vasoconstriction and narrowing, arterial thinning), grade 2 hypertensive retinopathy (generalized retinal artery stenosis, arteriovenous crossing compression), grade 3 hypertensive retinopathy (vascular narrowing, crossing compression, and retinal hemorrhage and exudation), and grade 4 hypertensive retinopathy (grade 3 with papilledema); or when divided into two grades, the corresponding label set would be: normal (no hypertensive retinopathy) and hypertensive retinopathy.

[0048] The final training set includes multiple source domain samples, multiple target domain samples, and labels indicating the fundus category to which each source domain sample belongs. Each source domain sample is a fundus image, and each target domain sample is a fundus image.

[0049] It should be understood that some source domain samples may come from the same or different users, and some target domain samples may also come from the same or different users; source domain samples and target domain samples may come from different users, be collected by different devices, etc.; the present invention does not impose any limitations on this.

[0050] In addition, if there are few fundus images in the dataset, image enhancement techniques (such as angle rotation, flipping, contrast adjustment, noise addition, etc.) can be used to enhance the fundus images to obtain more source domain samples or target domain samples.

[0051] II. Model Structure

[0052] Let me first explain why this invention uses a dual classifier:

[0053] To address the class imbalance problem, improved class-balancing classifiers are a common approach. These classifiers reduce the impact of class imbalance on the model by introducing class rebalancing schemes. For example, oversampling methods increase the number of samples from the minority class to balance the proportion of classes; or cost-sensitive learning incorporates inter-class cost information into the loss function, causing the model to focus more on the minority class during prediction. However, such methods can introduce another challenging complexity: especially when the label distribution of the target domain closely matches the imbalanced label distribution of the source domain, the improved classifier may increase the average accuracy of each class but decrease the overall accuracy because the majority class, with its absolute numerical advantage, will perform worse.

[0054] The goal of training a neural network by minimizing empirical risk is to approximate the latent conditional probability distribution p(y|x), representing the probability that a known sample x belongs to class y. In improved class-balanced classifier methods, the goal is to obtain a Bayesian optimality that better approximates the balanced error, i.e., p(y|x) / p(y), representing p(y|x) divided by the proportion p(y) of samples belonging to class y.

[0055] From a classifier design perspective, balancing the average accuracy of each class with the overall accuracy is a challenging trade-off. Nevertheless, by training a more discriminative feature extractor, it's possible to extract the inherent features z (corresponding to fundus features) that distinguish different classes from the samples. This allows for better differentiation of samples from different classes, ultimately achieving both higher overall accuracy and higher intra-class average accuracy.

[0056] Equivalently, ideally, features extracted by a feature extractor F*, when input into two different classifiers based on different classification criteria, should produce the same prediction results. However, when the feature extractor's discriminative power is weak, a large number of features processed by an imperfect feature extractor are concentrated near the decision boundaries of adjacent classes. Different classifiers then produce drastically different prediction results. Therefore, the difference between the prediction results of two different classifiers can be used to evaluate the feature extractor's performance (or, in other words, to assess whether the feature extractor has good discriminative power).

[0057] Based on two different classification strategies, an asymmetric dual-classifier structure is used to obtain two different decision boundaries. For samples highly similar to the source domain, classification differences usually do not occur due to the differences in classifiers. However, for samples far from the source support (referring to the data distribution in the source domain, i.e., the sample distribution in the source domain) or close to the original decision boundary, significant differences are likely to occur between the two classifiers.

[0058] Therefore, the difference between the prediction results of different classifiers can be used to measure the discriminative power of the original feature extractor (especially between adjacent categories with similar features), forcing the feature extractor to find the essential features between levels, so that the prediction results of the two classifiers tend to be consistent.

[0059] Based on the defined problem, this invention introduces two asymmetric classifiers (corresponding to a bi-classifier) ​​based on different classification criteria, namely, the conventional classifier C. ERM and Bayesian classifier C BO C ERM Based on the principle of minimizing empirical risk, C BO Based on Bayesian optimal balance risk minimization.

[0060] According to one embodiment of the present invention, see Figure 2 During training, a dual-classifier model is used, which includes a feature extractor, a regular classifier, and a Bayesian classifier connected to the feature extractor. Specifically:

[0061] According to one embodiment of the present invention, the feature extractor is configured to extract fundus features z from the input fundus image. The feature extractor may be, for example, a ConvNeXt-Tiny model, a ResNet model (such as ResNet18, ResNet34, or ResNet50), an AlexNet model, a VGG model (such as VGG16, VGG19, etc.), the backbone network of a Transformer network, or a custom feature extractor defined by the implementer.

[0062] Indicatively, a feature extractor can be represented as F(·). For example, for an input fundus image x, fundus features z are extracted by the feature extractor, i.e., z = F(x). The extracted fundus features z are used as input to two classifiers.

[0063] According to one embodiment of the present invention, a conventional classifier is configured to determine a first confidence vector based on the confidence scores of the corresponding fundus image belonging to each fundus category in a preset fundus classification task, according to the fundus features. The conventional classifier is constructed based on empirical risk minimization. According to another embodiment of the present invention, the conventional classifier may include at least one (single or multiple) fully connected layer for extracting a first Logit vector (i.e., Logits, or first classification feature) from the fundus features and a Softmax classification layer for determining a first confidence vector based on the first Logit vector.

[0064] An illustrative, conventional classifier is represented as follows: Where F(·) represents the feature extractor, C ERM This represents the fully connected layer of a conventional classifier. For function combination operators, Indicate C ERM The first Logit vector is extracted based on the fundus features extracted by F(·), and softmax(·) represents the Softmax classification layer. The calculated value is the first confidence vector.

[0065] According to one embodiment of the present invention, a Bayesian classifier is configured to determine a second confidence vector based on the confidence scores of the corresponding fundus image belonging to each fundus category in a preset fundus classification task, according to the fundus features. The Bayesian classifier is constructed based on balanced Bayesian optimality. According to another embodiment of the present invention, the Bayesian classifier may include at least one (single or multiple) fully connected layer for extracting a second Logit vector (i.e., Logits value, or second classification feature) from the fundus features and a Softmax classification layer for determining a second confidence vector based on the second Logit vector.

[0066] Indicative of a Bayesian classifier C BO It is configured to optimize Bayesian balance error. For a regular classifier, its classification strategy utilizes the model output. Entropy optimization is performed to optimize P(y|x). For a Bayesian classifier, to eliminate the influence of the distribution frequency P(y), i.e., the proportion of source domain samples of fundus category y in the training set, on the result, the optimal Bayesian risk balance optimization objective should be P(y|x) / P(y). Let's assume we combine the known source domain label distribution... The optimization objective is:

[0067]

[0068] in, and P(yi All of these indicate that they belong to the fundus category y. i The proportion of source domain samples, This represents the logit value corresponding to fundus category i. This represents the logit value corresponding to fundus category j.

[0069] For source domain samples, the Bayesian classifier can be adjusted by modifying the output logits, i.e., y. out +logπ s To achieve this; therefore, combining the known label distribution π of the source domain samples. s The Bayesian classifier directly adds logπ to logits. s Adjust the offset.

[0070] Therefore, the Bayesian classifier can be represented schematically as follows:

[0071]

[0072] Where F(·) represents the feature extractor of the Bayesian classifier, and C BO This represents the fully connected layer of a Bayesian classifier. For function combination operators, Indicate C BO The second Logit vector extracted from the fundus features extracted by F(·), π s represents the label distribution vector consisting of the proportion of source domain samples in each fundus category in the training set, and softmax(·) represents the Softmax classification layer of the Bayesian classifier. The calculated value is the second confidence vector.

[0073] III. Training Process

[0074] According to one embodiment of the present invention, the feature extractor and two classifiers are subjected to multiple adversarial training sessions using the training set to obtain a trained dual-classification model. Each adversarial training session includes steps A1, A2, A3, and A4. Each step is described below.

[0075] Step A1: Input each source domain sample into the current dual classifier model to obtain the first Logit vector and the second Logit vector corresponding to each source domain sample.

[0076] According to one embodiment of the present invention, in step A1, a feature extractor is used to extract fundus features from the input source domain sample; a conventional classifier is used to determine the first Logit vector of the source domain sample based on the fundus features, and classification is performed based on the first Logit vector; a Bayesian classifier is used to determine the second Logit vector of the source domain sample based on the fundus features, and classification is performed based on the second Logit vector. The classification process is described in the model structure section and will not be repeated here.

[0077] Step A2: Input each target domain sample into the current dual classifier model to obtain the first Logit vector and the second Logit vector corresponding to each target domain sample.

[0078] According to one embodiment of the present invention, in step A2, a feature extractor is used to extract fundus features from the input target domain sample; a conventional classifier is used to determine the first Logit vector of the target domain sample based on the fundus features, and classification is performed based on the first Logit vector; a Bayesian classifier is used to determine the second Logit vector of the target domain sample based on the fundus features, and classification is performed based on the second Logit vector. The classification process is described in the model structure section and will not be repeated here.

[0079] Step A3: With the parameters of the feature extractor fixed, update the parameters of the two classifiers based on the first sub-loss on the conventional classifier and the second sub-loss on the Bayesian classifier for each source domain sample. The first sub-loss is determined by the difference between the Logit value of each fundus category in the first Logit vector corresponding to each source domain sample and the positive distribution compensation value that is positively correlated with the proportion of source domain samples of that fundus category, and the label. The second sub-loss is determined by the difference between the Logit value of each fundus category in the second Logit vector corresponding to each source domain sample and the negative distribution compensation value that is negatively correlated with the proportion of source domain samples of that fundus category, and the label.

[0080] According to one embodiment of the present invention, with the parameters of the feature extractor fixed, the loss used to update the parameters of the two classifiers is determined as follows:

[0081]

[0082] in, This represents the mean of the first classifier loss for each source domain sample used in the current adversarial training. Let α represent the mean of the second classifier loss for each source domain sample used in the current adversarial training. The weight, β represents The weights are α and β. For illustration, α and β can be set to 0.25, but can be adjusted according to the implementer's needs.

[0083] According to one embodiment of the present invention, the positive distribution compensation value corresponding to a fundus category is the logarithm of the proportion of source domain samples of that fundus category in the training set. Correspondingly, Determined in the following manner:

[0084]

[0085] Where, x s Represents a source domain sample. Represents the source domain sample x s The corresponding first Logit vector, Represents the positive distribution compensation value, y s Represents the source domain sample x s The corresponding label, E{·}, indicates the calculation of the mean. The whole within the brackets {·} represents the sum of the first classifier losses of each source domain sample. This represents the proportion (π) of source domain samples of this fundus category in the training set. s .or The proportion of source domain samples of this fundus category in the training set plus the sum of random noise ( Where σ represents random noise.

[0086] According to another embodiment of the present invention, the positive distribution compensation value corresponding to a fundus category is the product of the logarithm of the proportion of source domain samples of that fundus category in the training set and a preset hyperparameter for measuring the degree of distribution compensation. By using the hyperparameter for measuring the degree of distribution compensation, the implementer can easily adjust the compensation level according to the training situation during application to achieve a better training effect. Correspondingly, Determined in the following manner:

[0087]

[0088] Where, x s Represents a source domain sample. Represents the source domain sample x s The corresponding first Logit vector, μ represents the preset hyperparameter for measuring the degree of distribution compensation, y s Represents the source domain sample x s The corresponding label, E{·}, indicates the mean value. The entire value within the brackets {·} represents the sum of the second-class sub-losses of all source domain samples. μ is a positive value and can be set according to the implementer's needs, such as 0.5, 0.4, 0.6, etc.

[0089] According to one embodiment of the present invention, the inverse distribution compensation value corresponding to a fundus category is the logarithm of the reciprocal of the proportion of source domain samples of that fundus category in the training set. Correspondingly, Determined in the following manner:

[0090]

[0091] Where, π s The vector represents the proportion of source domain samples of each fundus category in the training set. This represents a vector formed by the reciprocal of the proportion of source domain samples of each fundus category in the training set.

[0092] According to another embodiment of the present invention, the inverse distribution compensation value corresponding to a fundus category is the product of the logarithm of the reciprocal of the proportion of source domain samples of that fundus category in the training set and a preset hyperparameter for measuring the degree of distribution compensation. Correspondingly, Determined in the following manner:

[0093]

[0094] Where μ represents a preset hyperparameter that measures the degree of distribution compensation.

[0095] Step A4: With the parameters of the two classifiers fixed, update the parameters of the feature extractor based on the classifier loss of each source domain sample by the two classifiers and the classification difference between the two classifiers for each target domain sample.

[0096] According to one embodiment of the present invention, with the parameters of two classifiers fixed, the loss used to update the parameters of the feature extractor is determined as follows:

[0097]

[0098] in, This indicates that the current adversarial training uses samples x from each source domain based on the feature extractor F(·) using a conventional classifier. s The softmax value of the first Logit vector determined by the extracted fundus features is calculated and then compared with the mean of the classifier loss calculated from the labels. This indicates that the Bayesian classifier, based on the feature extractor F(·), performs the analysis of each source domain sample x used in the current adversarial training. s The softmax value of the second Logit vector determined by the extracted fundus features is calculated, and then the mean of the classifier loss calculated from the labels is taken, π. s This represents the label distribution vector, which is composed of the proportion of source domain samples in each fundus category in the training set. γ1 represents the mean difference between the two classifiers in classifying each target domain sample during the current adversarial training. The weight, γ2 represents The weight, Y3 represents The weights are γ1 and γ2. For illustration, γ1 and γ2 can be set to 0.5, and γ3 can be set to 1. Of course, these can be adjusted according to the implementer's needs. Optionally, the classification difference of the target domain samples can be the L1 distance (or L2 distance) between the first Logit vector and the second Logit vector. Alternatively, the classification difference can be the L1 distance (or L2 distance) between the first confidence vector and the second confidence vector.

[0099] Adversarial training through steps A1-A4 described above can improve the model's classification performance. For illustration, see [link to documentation]. Figure 3 The existing methods for training feature extractors produce feature convex hulls (the space formed by fundus features of fundus images of a certain type of fundus) that overlap, and the classification boundaries between fundus categories are not clear enough. Therefore, based on step A3, after amplifying the differences between classifiers using forward and backward distribution compensation values, it is equivalent to widening the classification boundaries between fundus categories by forward and backward compensation of fundus features of easily confused source domain samples. Subsequently, through step A4, by fixing the classifier and optimizing only the feature extractor parameters, the feature extractor can be forced to improve its ability to extract classification-related features (i.e., force the feature extractor to capture more essential lesion features between levels) and expand the distance between the feature convex hulls of fundus features corresponding to adjacent levels in the latent feature space (i.e., expand the distance between fundus features of samples of different fundus categories in the feature space), so as to discover the essential difference features between levels, enabling it to generate discriminative fundus features that make the two classifiers classify them in a consistent manner. Furthermore, due to the imbalance in the number of samples from different fundus categories in the dataset, the classification results (or original pseudo-labels) obtained on the target domain samples are biased and unreliable. Unlike traditional adversarial methods for two classifiers, step A3 no longer utilizes the pseudo-label differences of the two classifiers on the target domain samples to maximize the classifier difference. Instead, it amplifies the standard difference of the asymmetric classifiers on the absolutely accurate source domain labels. On the one hand, adversarial training of asymmetric two classifiers does not require maximizing the difference to generate the difference between the two classifiers. This difference stems from the adjustment logic based on the imbalanced source domain, resulting in a natural "segmentation gully" at the classification boundary between levels. This enhances the clarity and separability of the latent spatial features of different levels after training. On the other hand, the positive and negative distribution compensation mechanism can help identify targets that exhibit greater ambiguity near the classification boundary and make the model more adaptable to a wider range of label distribution shifts centered on the source domain prior label distribution, thereby improving model performance through adversarial training.

[0100] For step A4 The function here is through Figure 4 This can be illustrated using a simplified binary classification task. Suppose there are three target domain samples A, B, and C. This indicates that the conventional classifier performs well on the target domain sample x. t The first confidence vector obtained from the prediction, This indicates that the Bayesian classifier evaluates the target domain sample x. t The predicted second confidence vector; Indicates in Figure 4 The L1 distance between the first and second confidence vectors of a single target domain sample is represented by the vector. Taking target domain sample A as an example, the L1 distance between its corresponding first confidence vector [0.7, 0.3] and second confidence vector [0.4, 0.6] is 0.3; the rest are similar and will not be elaborated further. It can be seen that the closer a target domain sample is to the classification boundary, the higher its confidence level. The larger the value, the more it forces the feature extractor to push its fundus features toward the convex hull of its fundus category during training, thereby improving the performance of the feature extractor.

[0101] To avoid the cumulative impact of the distribution compensation mechanism on the classifier and further improve model performance, according to an embodiment of the present invention, each adversarial training session further includes, after steps A1-A4, step A5: updating the parameters of the two classifiers and the parameters of the feature extractor based on the sub-loss of each source domain sample by the two classifiers respectively.

[0102] According to one embodiment of the present invention, the loss used to update the parameters of the two classifiers and the parameter extractor based on the classification loss of the two classifiers for each source domain sample is determined as follows:

[0103]

[0104] in, This indicates that the current adversarial training uses samples x from each source domain based on the feature extractor F(·) using a conventional classifier. s The softmax value of the first Logit vector determined by the extracted fundus features is calculated and then compared with the mean of the classifier loss calculated from the labels. This indicates that the Bayesian classifier, based on the feature extractor F(·), performs the analysis of each source domain sample x used in the current adversarial training. s The softmax value of the second Logit vector determined by the extracted fundus features is calculated, and then the mean of the classifier loss calculated from the labels is used. γ1 represents... The weight, Y2 represents The weight.

[0105] The order of step A5 does not necessarily have to be the last step in each adversarial training session. For example, it can be adjusted to be between steps A2 and A3 to obtain other implementation methods.

[0106] IV. Training System

[0107] The methods described in the foregoing embodiments of the present invention can be implemented by a corresponding system.

[0108] According to one embodiment of the present invention, an adversarial training system based on minimizing the difference between asymmetric dual classifiers is provided, comprising: a module for acquiring a training set input, a model building module for acquiring a dual classifier model, and a training module for adversarially training the feature extractor and the two classifiers of the dual classifier model based on the training set. Implementation details of this system can be found in corresponding embodiments of the adversarial training method based on minimizing the difference between asymmetric dual classifiers, and will not be elaborated here.

[0109] V. Application Scenarios

[0110] According to an embodiment of the present invention, a method for fundus type identification based on fundus images is provided, the method comprising:

[0111] B1. Obtain the fundus image of the user to be identified;

[0112] B2. Obtain the feature extractor in the trained dual classifier model obtained according to the adversarial training method based on the asymmetric dual classifier difference minimization according to the foregoing embodiment, and at least one classifier among the conventional classifier and the Bayesian classifier.

[0113] B3. Using the feature extractor of the trained dual classifier model and the at least one classifier, identify the fundus category to which the fundus image to be identified belongs.

[0114] Because adversarial training enhances the capabilities of each classifier, enabling them to perform classification independently or together, the dual-classifier model can be configured according to the implementer's needs during deployment. Below are three illustrative examples:

[0115] Prediction can be performed using only the regular classifier from the dual classifier. According to one embodiment of the invention, B3 includes:

[0116] B31. Use the feature extractor of the trained dual classifier model to extract fundus features corresponding to the fundus image to be identified from the fundus image to be identified;

[0117] B32. Using the conventional classifier of the trained dual classifier model, the confidence level of the fundus image to be identified belonging to each fundus category in the preset fundus classification task is determined based on the fundus features corresponding to the fundus image to be identified.

[0118] Prediction can be performed using only the Bayesian classifier from the two classifiers. According to another embodiment of the invention, B3 includes:

[0119] B31' Utilize the feature extractor of the trained dual classifier model to extract fundus features corresponding to the fundus image to be identified from the fundus image to be identified;

[0120] B32' Utilizing the Bayesian classifier of the trained dual classifier model, based on the fundus features corresponding to the fundus image to be identified, determine the confidence level of the fundus image to be identified belonging to each fundus category in the preset fundus classification task.

[0121] During prediction, both classifiers in a dual classifier can be used together. According to yet another embodiment of the invention, B3 includes:

[0122] B31”, using the feature extractor of the trained dual classifier model to extract fundus features corresponding to the fundus image to be identified;

[0123] B32 uses the conventional classifier of the trained dual classifier model to determine the confidence level of the fundus image to be identified as belonging to each fundus category in the preset fundus classification task based on the fundus features corresponding to the fundus image to be identified.

[0124] B33" uses the Bayesian classifier of the trained dual classifier model to determine the confidence level of the fundus image to be identified as belonging to each fundus category in the preset fundus classification task based on the fundus features corresponding to the fundus image to be identified.

[0125] B34 calculates the average of the confidence scores for each fundus category determined by the conventional classifier and the confidence scores for each fundus category determined by the Bayesian classifier to obtain the final confidence score for each fundus category.

[0126] The confidence level of each fundus category identified above can be used as the result of fundus identification. Alternatively, the fundus category with the highest confidence level identified above can be used as the result of fundus identification.

[0127] It should be noted that the results of fundus identification in this invention are not the final diagnosis of the disease, but only serve as an auxiliary diagnostic indicator for medical personnel to refer to.

[0128] According to one embodiment of the present invention, a system for fundus type identification based on fundus images is also provided. The system includes a feature extractor in a trained dual classifier model, and at least one classifier selected from a conventional classifier and a Bayesian classifier. The system is configured to: acquire a fundus image to be identified from a user; and identify the fundus category to which the fundus image to be identified belongs using the feature extractor of the trained dual classifier model and the at least one classifier.

[0129] To verify the effectiveness of the technical solution of this invention, the inventors conducted fair comparisons and extensive analyses on the publicly available Indian Diabetic Retinopathy Image Dataset (IDRiD), demonstrating the effectiveness of the proposed asymmetric dual classifier difference minimization hierarchical imbalance domain adaptive method. In the experiments, using a partitioning scheme provided by the IDRiD dataset, 413 fundus images belonging to one population were classified as source domain samples, and 103 fundus images belonging to another population were classified as target domain samples (no labels were used during training; labels were used to evaluate performance during prediction after training). All comparison methods used the same backbone network, ConvNeXt-Tiny, as the feature extractor, with a batch size of 32 fundus images (if the last batch had fewer than 32 images, the images in that batch could be discarded or augmented using enhancement techniques), and continuous training for 70 epochs.

[0130] To measure the model's performance, the evaluation metrics used in the experiment included: overall accuracy, level-average accuracy, and second-weighted Kappa, which are frequently used to evaluate the effectiveness of medical image grading.

[0131] The calculation method for double-weighted kappa is as follows:

[0132]

[0133] Where O(i,j) represents the number of classes i classified as class j, and E(i,j) represents the average prediction accuracy if the values ​​are randomly distributed, with the weighted scheme w(i,j) = (ij). 2 / (V-1) 2 V represents the number of categories. As shown in Table 1, on public datasets, the method of this invention outperforms several existing state-of-the-art methods across all three evaluation dimensions.

[0134] After the experiment, it was observed that the prediction results of the two classifiers of this invention tended to be consistent after convergence. Therefore, the last row of Table 1 shows the experimental results corresponding to the prediction results of the conventional classifier.

[0135] Table 1 shows the results of comparing the classification accuracy of the method of the present invention with other methods.

[0136]

[0137] Note:

[0138] Method 1: A method for minimizing the source domain empirical loss, reference:

[0139] [1] V. Vapnik, "An overview of statistical learning theory," IEEETrans.Neural Networks, 1999.10, 988-999.

[0140] Method 2: A domain adversarial neural network method, reference:

[0141] [2] Y.Ganin, E.Ustinova, H.Ajakan, P.Germain, H.Larochelle, FDomain-adversarial training of neural networks. "J.Mach.Learn, Res. 2016.17.

[0142] Method 3: A maximum classifier difference method, reference:

[0143] [3] K.Saito, K.Watanabe, Y.Ushiku, T.Harada, "Maximum classiferdiscrepancy for unsupervised domain adaptation," in Proc.CVPR, 2018.pp.3723-3732.

[0144] Method 4: A deterministic maximization method for a dual classifier, reference:

[0145] [4] S. Li, F. Lv, B. Xie, C. Liu, J. Liang, C. Qin, "Bi-classifier determinacymaximization for unsupervised domain adaptation," Proc.AAAl, 2021.35, 8455-8464.

[0146] Method 5: A selective entropy optimization method, reference:

[0147] [5] V.Prabhu, S.Khare, D.Kartik, J.Hoffman, "Sentry: Selective entropyoptimization via committee consistency for unsupervised domain adaptation," inProc.ICCV, 2021.pp.8558-8567.

[0148] Method 6: A Logits tuning method, reference:

[0149] [6] A.Menon, S.Jayasumana, A.Rawat, H.Jain, A.Veit, S.Kumar, "Longtail learning via logit adjustment," in Proc.ICLR, 2021.

[0150] In summary, the method of the present invention has the following beneficial effects:

[0151] This invention proposes a novel dual-classifier model framework. This framework connects asymmetric dual classifiers based on different classification strategies after the feature extractor, obtaining empirically minimized risk predictions and balanced Bayesian optimal predictions, respectively. The difference between these two predictions reflects the skewness of the imbalanced dataset. Adversarial training between the classifier and the feature extractor, based on the Bayesian optimal and empirically minimized risk classification strategies, forces the feature extractor to generate features that minimize the difference. This improves the feature extractor's ability to extract classification-related fundus features from easily confused samples at the decision boundary, thereby enhancing classification performance.

[0152] It should be noted that although the steps are described in a specific order above, it does not mean that the steps must be executed in the above specific order. In fact, some of these steps can be executed concurrently, or even in a different order, as long as the required function can be achieved.

[0153] This invention can be a system, method, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of the invention.

[0154] Computer-readable storage media can be tangible devices that hold and store instructions for use by an instruction execution device. Computer-readable storage media can include, for example, but not limited to, electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination thereof.

[0155] The various embodiments of the present invention have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or technical improvements to the embodiments in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. An asymmetric-based dual classifier model field adaptation method, comprising: obtaining a training set comprising a plurality of source domain samples, a plurality of target domain samples, and labels indicating the category of fundus to which each source domain sample belongs, wherein the source domain samples and the target domain samples are fundus images; obtaining a dual classifier model comprising: a feature extractor configured to extract fundus features from an input fundus image, a regular classifier configured to classify based on a first Logit vector determined from the fundus features, and a Bayesian classifier configured to classify based on a second Logit vector determined from the fundus features, wherein the first Logit vector and the second Logit vector each comprise a Logit value corresponding to the probability that the corresponding fundus image belongs to each fundus category in a preset fundus classification task, and the regular classifier is constructed based on empirical risk minimization; performing multiple adversarial training of the feature extractor and the two classifiers using the training set, to obtain a trained dual classifier model, each adversarial training comprising: inputting each source domain sample into the current dual classifier model to obtain a corresponding first Logit vector and a corresponding second Logit vector for each source domain sample; inputting each target domain sample into the current dual classifier model to obtain a corresponding first Logit vector and a corresponding second Logit vector for each target domain sample; updating the parameters of the two classifiers based on a first classification sub-loss of each source domain sample on the regular classifier and a second classification sub-loss of each source domain sample on the Bayesian classifier, while fixing the parameters of the feature extractor, wherein the first classification sub-loss is determined based on the difference between the Logit value of each fundus category in the first Logit vector corresponding to each source domain sample and a positive distribution compensation value positively correlated with the proportion of source domain samples of the fundus category, and the label, and the second classification sub-loss is determined based on the difference between the Logit value of each fundus category in the second Logit vector corresponding to each source domain sample and a negative distribution compensation value negatively correlated with the proportion of source domain samples of the fundus category, and the label, wherein the positive distribution compensation value corresponding to one fundus category is the logarithm of the proportion of source domain samples of the fundus category in the training set, or the product of the logarithm of the proportion of source domain samples of the fundus category in the training set and a preset hyperparameter measuring the degree of distribution compensation, and the negative distribution compensation value corresponding to one fundus category is the logarithm of the inverse of the proportion of source domain samples of the fundus category in the training set, or the product of the logarithm of the inverse of the proportion of source domain samples of the fundus category in the training set and a preset hyperparameter measuring the degree of distribution compensation; updating the parameters of the feature extractor based on the classification sub-loss of each source domain sample on the two classifiers and the classification difference between the two classifiers for each target domain sample, while fixing the parameters of the two classifiers.

2. The method of claim 1, wherein, In the case of fixing the parameters of the feature extractor, the loss used to update the parameters of the two classifiers is determined as follows: wherein, denotes a mean value of the first classification sub-loss of each source domain sample employed in the current adversarial training, denotes a mean value of the second classification sub-loss of each source domain sample employed in the current adversarial training, denotes a weight of denotes a weight of denotes a weight of​ 3. The method of claim 2, wherein, In the case of fixing the parameters of the two classifiers, the loss for updating the parameters of the feature extractor is determined in the following manner: wherein, represents the classification sub-loss calculated by the Bayes classifier based on the feature extractor each source domain sample adopted in the current adversarial training the mean value of the classification sub-loss calculated after the first Logit vector determined by the extracted fundus features is taken the Softmax value and the label, represents the classification sub-loss calculated by the Bayes classifier based on the feature extractor each source domain sample adopted in the current adversarial training the mean value of the classification sub-loss calculated after the second Logit vector determined by the extracted fundus features is taken the Softmax value and the label, represents the label distribution vector composed of the proportion of each fundus category in the source domain samples in the training set, represents the mean value of the classification difference between the two classifiers for each target domain sample in the current adversarial training, represents the weight of represents the weight of represents the weight of ​​​ 4. The method of claim 3, wherein, The each-time adversarial training further includes: The parameters of the two classifiers and the parameters of the feature extractor are updated according to the classification sub-losses of the two classifiers for each source domain sample.

5. The method of claim 4, wherein, In the updating of the parameters of the two classifiers and the parameters of the feature extractor according to the classification sub-losses of the two classifiers for each source domain sample, the loss for updating the parameters of the two classifiers and the parameters of the feature extractor is determined in the following manner: wherein, wherein, denotes the classification sub-loss calculated based on the feature extractor each source domain sample employed in the current adversarial training the mean of the classification sub-loss calculated after taking the Softmax value of the first Logit vector determined by the extracted fundus features, ) denotes the classification sub-loss calculated based on the feature extractor each source domain sample employed in the current adversarial training the mean of the classification sub-loss calculated after taking the Softmax value of the second Logit vector determined by the extracted fundus features, denotes the weight of denotes the weight of denotes the weight of denotes the weight of 6. The method according to one of claims 1 to 5, characterized in that The set of labels includes no retinopathy, mild non-proliferative diabetic retinopathy, moderate non-proliferative diabetic retinopathy, severe non-proliferative diabetic retinopathy, and proliferative diabetic retinopathy; or, The set of labels includes normal, early stage of senile macular lesions, and late stage of senile macular lesions; or, The set of labels includes normal, first grade of hypertensive fundus lesions, second grade of hypertensive fundus lesions, third grade of hypertensive fundus lesions, and fourth grade of hypertensive fundus lesions.

7. A method of identifying a fundus type from a fundus image, characterized by, The method includes: obtaining an eye fundus image to be identified of a user; obtaining a feature extractor and at least one classifier of a trained double-classifier model obtained by the method according to any one of claims 1-6; identifying an eye fundus category to which the eye fundus image to be identified belongs by using the feature extractor and the at least one classifier of the trained double-classifier model.

8. The method of claim 7, wherein, The identifying an eye fundus category to which the eye fundus image to be identified belongs by using the feature extractor and the at least one classifier of the trained double-classifier model includes: extracting eye fundus features corresponding to the eye fundus image to be identified from the eye fundus image to be identified by using the feature extractor of the trained double-classifier model; determining a confidence degree of the eye fundus image to be identified belonging to each eye fundus category in a preset eye fundus classification task according to the eye fundus features corresponding to the eye fundus image to be identified by using the at least one classifier of the trained double-classifier model.

9. A computer-readable storage medium, characterized in that, A computer program is stored thereon, and the computer program is executable by a processor to implement the steps of the method of any one of claims 1-8.

10. An electronic device, comprising: comprise: one or more processors; and a memory, wherein the memory is configured to store executable instructions; the one or more processors are configured to implement the steps of the method of any one of claims 1-8 by executing the executable instructions.

Images