OCT Image Classification Method and System for Rare Fundus Diseases Based on Distilled Meta-Learning

Abstract

The present invention relates to the technical field of image classification, and in particular to a method and system for classifying OCT images of rare fundus diseases based on distilled meta-learning. A teacher network and a student network for classifying retinal OCT images are respectively built based on a residual module and a fully connected layer; a training set of common fundus diseases is constructed to perform meta-training on the teacher network; a small-sample training task of common diseases is randomly constructed, and based on the distilled meta-learning training strategy, the teacher network and the student network are jointly trained. An attention-guided distillation module is introduced, and a temperature coefficient adaptive method is adopted to train the student to adaptively learn the features of the teacher in pixels and channels with higher attention values; a training set of rare fundus diseases is constructed to fine-tune the student network to obtain a small-sample classification network model applicable to rare diseases; the OCT image is input into the small-sample classification network model of rare diseases to obtain the classification result of the lesion type.

Description

Technical Field

[0001] The present invention relates to the technical field of image classification, and particularly to a method and system for classifying OCT images of rare fundus diseases based on distilled meta-learning. Background Art

[0002] The retina is an important part of the human eye, and the health of the retina is crucial for the human visual system. Any damage, disease or abnormality of the retina may lead to eye problems or visual impairment. Optical Coherence Tomography (OCT) technology is widely used in the field of ophthalmology for the diagnosis of retinal diseases. The automatic classification of retinal OCT images can effectively assist doctors in screening, diagnosing and following up patients. Currently, convolutional neural networks have been widely applied in the field of medical images, providing assistance for the automatic screening of some common diseases. However, for rare diseases with a small number of cases, traditional deep learning methods cannot achieve good results.

[0003] Few Shot Learning (FSL) has attracted wide attention in many fields such as image classification, object detection and semantic segmentation. One of the very important reasons is that the training of deep learning relies on a large amount of labeled data. In some practical applications, such as rare diseases, animals, plants and other fields, it is very difficult to obtain a sufficient number of samples. The goal of few shot learning is to identify new categories when only using a small number of labeled samples. In fact, the emergence of few shot learning is inspired by the human learning mechanism. When facing new tasks or new categories, humans can usually quickly learn and generalize knowledge from limited samples. This ability has inspired researchers to conduct research on few shot learning so that machine learning methods can simulate the human learning method.

[0004] Meta-learning method is an important idea for dealing with few shot learning problems. Its core idea is to enable the model to learn the ability to analyze and solve problems from past knowledge and experience, so as to guide the learning process of new tasks. In this way, the model can quickly adapt to and solve problems when facing new tasks with only a small amount of sample data. Model-Agnostic Meta-Learning (MAML) is one of the most classical methods of meta-learning. The core of this method lies in maximizing the sensitivity of the loss function to the initial weights of new tasks. Regardless of the model used, the weights are optimized along the direction of the sum of the gradient vectors of each task, and the optimal weights of the model are estimated for new tasks to achieve good results on new tasks. Since then, many scholars have carried out various optimizations and improvements on MAML. Summary of the Invention

[0005] In view of the problems existing in the above-mentioned prior art, the present invention is proposed.

[0006] Since traditional deep learning cannot effectively classify rare retinal diseases, the present invention proposes a method for classifying OCT images of rare fundus diseases based on distillation meta-learning. The aim is to train a model with low complexity, strong generalization ability, and relatively good classification performance for classifying OCT images of rare retinal diseases by using only a small number of rare-disease retinal OCT images on the basis of training with common-disease data. The present invention proposes a two-stage distillation meta-learning method to train a network model with lower complexity through knowledge distillation technology. The student network is encouraged to focus on the information in the key areas, so as to better guide the model for classification. In addition, in order to improve the adaptability of the method under different tasks, the parameters are automatically optimized during the training process, enabling the method to automatically adjust the parameters according to different task requirements, and thus optimizing its performance. The present invention can effectively improve the performance of classifying OCT images of rare fundus diseases and shows superiority compared with other existing few-shot learning methods.

[0007] To solve the above technical problems, the present invention provides the following technical solution. A method for classifying OCT images of rare fundus diseases based on distillation meta-learning includes: building two retinal OCT image classification networks based on convolutional layers, residual connections, and fully connected layers;

[0008] Based on the training set of common fundus diseases, randomly construct small-sample training tasks for common diseases, and perform meta-training on the teacher network to obtain a small-sample classification teacher network model applicable to different combinations of common diseases;

[0009] Based on the training set of common fundus diseases, randomly construct small-sample training tasks for common diseases, and based on the distillation meta-learning training strategy, combine attention-guided distillation and temperature coefficient adaptive module to jointly train the teacher network and the student network to obtain a small-sample classification student network model applicable to different combinations of common diseases;

[0010] Construct a training set of rare fundus diseases, and based on the training samples, fine-tune the student network model to obtain a small-sample classification network model applicable to rare diseases;

[0011] Input the OCT image into the small-sample classification network model applicable to rare diseases to obtain the classification result of the lesion type.

[0012] As a preferred scheme of the method for classifying OCT images of rare fundus diseases based on distillation meta-learning according to the present invention, among them: the two retinal OCT image classification networks are respectively a teacher network and a student network, and both networks take OCT images as input and image lesion category labels as output.

[0013] As a preferred solution of the OCT image classification method for rare fundus diseases based on distilled meta - learning according to the present invention, wherein: the randomly constructed small - sample training tasks for common diseases include constructing a training set for common fundus diseases containing at least 6 categories, randomly sampling, and constructing a small - sample training task set for common fundus diseases ={ }, where the i - th task is , and each task contains categories in the common - disease dataset. For each category, samples are taken as the support - set data , and M samples are taken as the query - set data .

[0014] As a preferred solution of the OCT image classification method for rare fundus diseases based on distilled meta - learning according to the present invention, wherein: the meta - training includes, based on the small - sample training task set for common fundus diseases, using the model - agnostic meta - learning method to perform meta - training on the teacher network;

[0015] The model - agnostic meta - learning method includes an inner loop and an outer loop. The inner loop is used to train a small - sample classification network model applicable to a specific category combination, and the outer loop is used to train a small - sample classification network model applicable to different lesion combinations, obtaining a set of teacher network model parameters adapted to new tasks with different lesion combinations .

[0016] As a preferred solution of the OCT image classification method for rare fundus diseases based on distilled meta - learning according to the present invention, wherein: the distilled meta - learning training strategy is expressed as,

[0017] Let the student model parameters be a temporary copy of the student model, initialized to be equal to . Update it on the support set of a batch of tasks:

[0018] ,

[0019] wherein, is the current teacher model parameter, represents taking the gradient of , is the learning rate of the student network;

[0020] The loss function consists of three parts:

[0021] ,

[0022] wherein,​ is the KL divergence between the soft labels output by the teacher network and the soft labels output by the student network, is the cross-entropy loss between the hard labels output by the student network and the true labels, is the attention-guided distillation loss;

[0023] Let the final output vector of the teacher model be , be the number of classes, and let the th item of be the th item of , and after being processed by the softmax function with the distillation temperature coefficient , the output label is obtained, which is expressed as:

[0024] ,

[0025] Let the final output vector of the student model be , and let the k-th item of be the j-th item of , and after being processed by the softmax function with the temperature parameter , the output label is obtained,

[0026] ,

[0027] The soft label KL divergence loss is calculated as:

[0028] ,

[0029] Calculate the hard label output by the student network:

[0030] ,

[0031] The hard label cross-entropy loss is calculated as:

[0032] ,

[0033] where, is the value of the th class in the true label of the sample;

[0034] Let be the features extracted by the backbone network, where Represent the number of channels, height, and width respectively. Use channel attention and spatial attention mechanisms to generate teacher channel feature maps for the features extracted by the teacher network and student network ( ) and student channel feature maps ( ) and teacher spatial feature maps and student spatial feature maps . Sum the attention maps of the obtained teacher and student models, and obtain the spatial attention mask and channel attention mask used in attention-guided distillation through the softmax function. The formula is expressed as:

[0035] ,

[0036] ,

[0037] Calculate the attention-guided distillation loss , including attention transfer loss and feature distillation loss . Use to encourage the student model to imitate the spatial and channel attention of the teacher model. The formula is expressed as:

[0038] ,

[0039] where is the Euclidean distance between two features;

[0040] Use to encourage the student model to imitate the features of the teacher model, and by adjusting the weights on different pixels and channels in the feature map, enable the student model to identify key pixels and important channels in the image. The formula is expressed as:

[0041] ,

[0042] where is the value of at the position, is the value of at the position, is the th value of ;

[0043] Weightedly sum and to obtain the final attention-guided distillation loss :

[0044] ,

[0045] Among them, and are weight hyperparameters;

[0046] Set the initial temperature coefficient , and adaptively adjust the temperature according to the output of the teacher model :

[0047] ,

[0048] ,

[0049] Among them, is the maximum value in the predicted probability distribution of the sample by the teacher model.

[0050] As a preferred scheme of the OCT image classification method for rare fundus diseases based on distillation meta-learning described in the present invention, wherein: the distillation meta-learning training strategy is also expressed as

[0051] After a batch of tasks are trained, update the parameters of the teacher model based on the loss function on the query set : :

[0052] ,

[0053] Among them, represents taking the gradient of , represents the i-th task, is the learning rate of the teacher network;

[0054] Discard and use the updated to update the real student model on the support set samples , which is expressed as:

[0055] ,

[0056] Among them, represents taking the gradient of , represents the i-th task, is the learning rate of the student network.

[0057] As a preferred scheme of the OCT image classification method for rare fundus diseases based on distillation meta-learning described in the present invention, wherein: the fine-tuning of the student network model includes constructing a training set for rare fundus diseases, and the number of categories is , and each type of rare disease uses a sample, and take the same values as in meta-training, train based on the sample, adapt to the new task, and use the model parameters finally output by the student network as the initial value, perform gradient descent optimization on the small sample training set of rare fundus diseases to obtain a small sample classification network model suitable for rare diseases.

[0058] As a preferred solution of the rare fundus disease OCT image classification system based on distillation meta-learning described in the present invention, it includes: a network construction module, a meta-training module, a joint training module, an adjustment module, and an image classification output module;

[0059] The network construction module includes building two retinal OCT image classification networks based on convolutional layers, residual connections, and fully connected layers;

[0060] The meta-training module includes randomly constructing small sample training tasks for common fundus diseases based on the common fundus disease training set, performing meta-training on the teacher network to obtain a small sample classification teacher network model suitable for different combinations of common diseases;

[0061] The joint training module includes randomly constructing small sample training tasks for common fundus diseases based on the common fundus disease training set, and based on the distillation meta-learning training strategy, combining attention-guided distillation and the temperature coefficient adaptive module to jointly train the teacher network and the student network to obtain a small sample classification student network model suitable for different combinations of common diseases;

[0062] The adjustment module includes constructing a rare fundus disease training set, and fine-tuning the student network model based on the training sample to obtain a small sample classification network model suitable for rare diseases;

[0063] The image classification output module includes inputting the OCT image into the small sample classification network model suitable for rare diseases to obtain the classification result of the lesion type.

[0064] A computer device includes a memory and a processor, the memory stores a computer program, and is characterized in that when the processor executes the computer program, it implements the steps of any one of the methods of the rare fundus disease OCT image classification method based on distillation meta-learning.

[0065] A computer-readable storage medium stores a computer program thereon, and is characterized in that when the computer program is executed by a processor, it implements the steps of any one of the methods of the rare fundus disease OCT image classification method based on distillation meta-learning.

[0066] Advantages of the present invention: The present invention proposes an OCT image classification method for rare fundus diseases based on distillation meta-learning. This method combines knowledge distillation with meta-learning. Through the knowledge transfer from the teacher model to the student model, the student model can benefit from the rich knowledge of the teacher model. This helps to improve the generalization ability of the student model and enables it to perform better on unseen tasks and data. In addition, an attention-guided distillation method is designed to guide the student model to focus on the information of key lesion areas, thereby helping the model to classify more accurately. To enable the method to adapt to different tasks, a temperature coefficient adaptive mechanism is introduced, and the temperature coefficient is used as an adaptive parameter to be automatically optimized during the training process, accelerating the model training and deployment process. Experimental results show that on the constructed small-sample retinal OCT dataset, the performance of this method is significantly improved compared with other distillation learning and small-sample learning methods in most cases. Description of the Drawings

[0067] To more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0068] Figure 1 It is the overall framework diagram of the two-stage distillation meta-learning method for the OCT image classification method of rare fundus diseases based on distillation meta-learning provided by an embodiment of the present invention.

[0069] Figure 2 It is the network structure diagram of the OCT image classification method for rare fundus diseases based on distillation meta-learning provided by an embodiment of the present invention, where (a) is the teacher network Resnet32 and (b) is the student network Resnet16.

[0070] Figure 3 It is the attention-guided distillation module diagram of the OCT image classification method for rare fundus diseases based on distillation meta-learning provided by an embodiment of the present invention.

[0071] Figure 4 It is the example diagram of various types of images in the OCT dataset of the OCT image classification method for rare fundus diseases based on distillation meta-learning provided by an embodiment of the present invention. Detailed Embodiments

[0072] To make the above objects, features, and advantages of the present invention more apparent and understandable, the following provides a detailed description of the specific embodiments of the present invention in conjunction with the accompanying drawings of the specification. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the scope of protection of the present invention.

[0073] Embodiment 1

[0074] Referring to Figures 1-3 , which is the first embodiment of the present invention. This embodiment provides a method for classifying OCT images of rare fundus diseases based on distillation meta-learning, including:

[0075] The present invention designs a distillation meta-learning method to achieve the classification of OCT images of rare fundus diseases. As shown in the appendix Figure 1 , the training process includes three stages. The first stage is meta-training. In the first stage, a teacher network is trained using MAML, and the teacher model with the obtained initial parameters is used to guide the training of the student network. The second stage is distillation meta-learning. In the second stage, the teacher model and the student model learn from each other. On the one hand, the student model feeds back the results of learning from the teacher model to improve the ability of the teacher. On the other hand, the student model learns the knowledge of the teacher model to obtain a higher classification accuracy. At the same time, in order to make the network pay more attention to the information in the key regions, an Attention Guided Distillation (AGD) module is introduced to train the student to learn the features of the teacher in the pixels and channels with higher attention values. In addition, an Adaptive Temperature Coefficient (ATC) method is proposed to automatically adjust the temperature coefficient to the optimal value during training to help the training of the model. In the third stage, the student network is further fine-tuned to be applicable to the classification of specific rare diseases.

[0076] The present invention proposes a method for classifying OCT images of rare fundus diseases based on distillation meta-learning, which is applicable to the classification of OCT image lesions under small sample conditions. (1) Design a two-stage training method consisting of meta-training and distillation meta-learning. (2) Design an attention-guided distillation module, which uses the attention mechanism to identify the key pixels and channels in the feature map, so that the student model can focus on the important feature values, avoiding inaccurate feature recognition and waste of resources. (3) Design an adaptive temperature coefficient mechanism to balance the ability of the student model in dealing with difficult-to-separate samples and easy-to-separate samples, and promote the student model to better learn the probability distribution knowledge from the teacher model, thereby improving the generalization ability and performance of the model.

[0077] A method for classifying OCT images of rare fundus diseases based on distilled meta - learning, characterized by the following steps:

[0078] S1. Build two retinal OCT image classification networks based on convolutional layers, residual connections, and fully - connected layers, serving as the teacher network and the student network respectively. Both networks take OCT images as input and the image lesion category labels as output.

[0079] The structures of the constructed teacher network and student network are as shown in the appendix Figure 2 as follows.

[0080] For the teacher network, the first 31 layers of the network are 3×3 convolutional layers, and the 32nd layer is a fully - connected layer. The number of convolutional channels in layers 1 - 7 is 64, in layers 8 - 15 is 128, in layers 16 - 25 is 256, and in layers 26 - 31 is 512. The outputs of the 1st and 3rd layers, the 3rd and 5th layers, the 5th and 7th layers, the 9th and 11th layers, the 11th and 13th layers, the 13th and 15th layers, the 17th and 19th layers, the 19th and 21st layers, the 21st and 23rd layers, the 23rd and 25th layers, the 27th and 29th layers, and the 29th and 31st layers are directly added through residual connections and then input into the next layer. The outputs of the 7th and 9th layers, the 15th and 17th layers, and the 25th and 27th layers are added through residual connections with 1×1 convolutions to change the number of channels and then input into the next layer.

[0081] For the student network, the first 15 layers of the network are 3×3 convolutional layers, and the 16th layer is a fully - connected layer. The number of convolutional channels in layers 1 - 3 is 64, in layers 4 - 7 is 128, in layers 8 - 11 is 256, and in layers 12 - 15 is 512. The outputs of the 1st and 3rd layers, the 5th and 7th layers, the 9th and 11th layers, and the 13th and 15th layers are directly added through residual connections and then input into the next layer. The outputs of the 3rd and 5th layers, the 7th and 9th layers, and the 11th and 13th layers are added through residual connections with 1×1 convolutions to change the number of channels and then input into the next layer.

[0082] Both the teacher network and the student network take OCT images as input and the predicted probabilities of the image belonging to each type of lesion as output. The maximum value is taken among the probabilities of each category, and the corresponding category is the final image classification result.

[0083] S2. Based on the training set of common fundus diseases, randomly construct small - sample training tasks for common diseases and conduct meta - training on the teacher network to obtain a small - sample classification teacher network model that can be widely applicable to different combinations of common diseases.

[0084] Construct a training set for common fundus diseases with at least 6 categories, randomly sample from it, and construct a small-sample training task set for common fundus diseases ={ }. The i-th task is . Each task contains K categories in the common disease dataset. For each category, N samples are taken as the support set data , and M samples are taken as the query set data . K is a positive integer between 3 and 6. N is a positive integer less than 11, and M is an integer between 10 and 15

[0085] S3. Based on the training set of common fundus diseases, randomly construct small-sample training tasks for common diseases. Based on the distillation meta-learning training strategy, combined with attention-guided distillation and temperature coefficient adaptive module, jointly train the teacher network and the student network to obtain a small-sample classification student network model that can be widely applied to different combinations of common diseases

[0086] Based on the small-sample training task set of common fundus diseases, use the model-agnostic meta-learning (MAML) method to perform meta-training on the teacher network. The MAML method includes an inner loop and an outer loop. The inner loop is used to train a small-sample classification network model suitable for a specific category combination, and the outer loop is used to train a small-sample classification network model that can be widely applied to different lesion combinations. Finally, a set of teacher network model parameters that can quickly adapt to new tasks with different lesion combinations is obtained .

[0087] Based on the small-sample training task set of common fundus diseases, train the teacher network parameters and the student network parameters . Finally, obtain the student network model parameters that can quickly adapt to new tasks with different lesion combinations. The teacher model parameters are initialized as , and the student model parameters are randomly initialized

[0088] Before starting the training, a temporary copy of the student model needs to be created , initialized to be equal to . The distillation meta-learning includes a total of three steps

[0089] The first step

[0090] Update on the support set of a batch of tasks

[0091] ,

[0092] where is the current teacher model parameter Indicates to calculate the gradient, which is the learning rate of the student network, initially set to 0.001, and updated to 0.5 times every 10 epochs. The loss function consists of three parts:

[0093] ,

[0094] Among them, is the KL divergence between the soft labels output by the teacher network and the soft labels output by the student network, is the cross-entropy loss between the hard labels output by the student network and the true labels, is the attention-guided distillation loss. Summing up these three losses gives the total loss function of this method.

[0095] Soft label KL divergence loss

[0096] Let the final output vector of the teacher model be , where is the number of classes. Let the -th item of be , and the -th item of is . After being processed by the softmax function with the distillation temperature coefficient , the output label

[0097] is obtained, which is expressed as:

[0098] Let the final output vector of the student model be , and let the -th item of be , and the -th item of is . After being processed by the softmax function with the temperature parameter

[0099] ,

[0100] The soft label KL divergence loss is calculated as:

[0101] ,

[0102] Hard label cross-entropy loss:

[0103] Calculate the hard label output by the student network:

[0104] ,

[0105] The hard label cross-entropy loss is calculated as follows:

[0106] ,

[0107] where, is the value of the -th class in the true label of the sample;

[0108] (3) Attention-guided distillation loss: As shown in the attached Figure 3 , let be the features extracted by the backbone network, where C, H, and W represent the number of channels, height, and width, respectively. The channel attention and spatial attention mechanisms are used to generate the teacher channel feature map ( ), the student channel feature map ( ), the teacher spatial feature map , and the student spatial feature map . Then, the attention maps of the teacher and student models obtained from the above steps are summed, and the spatial attention mask and the channel attention mask used in attention-guided distillation are obtained through the softmax function. The formula is:

[0109] ,

[0110] ,

[0111] Then, the attention-guided distillation loss is calculated. It consists of two parts: the attention transfer loss and the feature distillation loss . Using encourages the student model to imitate the spatial and channel attention of the teacher model, which can be expressed as:

[0112] ,

[0113] where, is the Euclidean distance between the two features.

[0114] Using encourages the student to imitate the features of the teacher model and adjusts the weights on different pixels and channels in the feature map so that it can identify the key pixels and important channels in the image, which can be expressed as:

[0115] ,

[0116] Among them, is the value at the position, is the value at the position, is the th value;

[0117] Perform a weighted sum of and to obtain the final attention-guided distillation loss :

[0118] ,

[0119] Among them, and are weight hyperparameters, which are set to 0.1 and 0.3 respectively in the experiment.

[0120] Temperature coefficient adaptation: Set the initial temperature coefficient , and adaptively adjust the temperature according to the output of the teacher model

[0121] ,

[0122] ,

[0123] Among them, is the maximum value in the predicted probability distribution of the sample by the teacher model. In the experiment, the initial temperature coefficient is set to 4.

[0124] When the teacher model outputs a probability prediction vector for a certain sample, if the predicted probability of a certain category in the vector is relatively high, it indicates that the predicted probability distribution of the sample is uneven and the confidence is relatively high. In this case, set a smaller temperature coefficient T to reduce the entropy of the output distribution, so that the network can learn the knowledge of the classification probability distribution more confidently. On the contrary, when the output probability is uniform and the confidence is low , set a higher temperature coefficient T to increase the entropy of the output distribution, thereby enhancing the learning of the sample class during training and enabling the student model to learn the knowledge of the teacher model more diversely.

[0125] The second step:

[0126] After a batch of tasks are trained and completed through the above steps, update the parameters of the teacher model based on the loss function on the query set :

[0127] ,

[0128] Among them, denotes taking the gradient of and represents the i-th task. is the learning rate of the teacher network, initially set to 0.005, and updated to 0.5 times every 10 rounds.

[0129] The third step:

[0130] Discard and use the updated to update the true student model on the support set samples . This step immediately applies the updated teacher network to the update of the student network, thus aligning the training process:

[0131] ,

[0132] Among them, denotes taking the gradient of and represents the i-th task. is the learning rate of the student network, initially set to 0.001, and updated to 0.5 times every 10 rounds.

[0133] S4. Construct a training set for rare fundus diseases, and based on the training samples, fine-tune the student network model to obtain a small-sample classification network model suitable for rare diseases;

[0134] Construct a training set for rare fundus diseases with the number of categories being K. Each type of rare disease uses N samples. The values of K and N are the same as those in meta-training. Based on the training of these samples, adapt to new tasks. Use the model parameters output by the student network finally as the initial value, and perform gradient descent optimization on the small-sample training set of rare fundus diseases, so that the model can adapt to new tasks and obtain a small-sample classification network model suitable for rare diseases.

[0135] S5. Input the OCT image into the small-sample classification network model suitable for rare diseases to obtain the classification result of the lesion type.

[0136] Input the test OCT image into the trained small-sample classification student network model for rare diseases obtained in step S5 to obtain the classification result.

[0137] Example 2

[0138] Refer to Figure 4 , which is the second embodiment of the present invention, provides a method for classifying OCT images of rare fundus diseases based on distillation meta-learning. In order to verify the beneficial effects of the present invention, scientific demonstration is carried out through experiments.

[0139] It should be noted that the data includes OCT images obtained by an Optovue scanner. Common diseases include Age-related Macular Degeneration (AMD), Diabetic Retinopathy (DR), Choroidal Neovascularization (CNV), Central Serous Chorioretinopathy (CSC), and Retinal Vein Occlusion (RVO). The common disease dataset also includes the "normal" category, for a total of 6 categories. Rare diseases include Retinal Detachment (RD), Retinal Hemorrhage (RH), Optic Atrophy (OA), Epiretinal Membrane (ERM), Central Retinal Artery Occlusion (CRAO), retinoschisis (RS), and Retinal Detachment (RD), for a total of 7 categories. Examples of various images are shown in the appendix Figure 4 as follows

[0140] As shown in Table 1 in the appendix, the OCT dataset is divided into three parts for meta-training, distilled meta-learning, fine-tuning, and testing. The datasets for meta-training and distilled meta-learning are the same, including normal and five common fundus diseases, for a total of 6 categories, with 400 OCT images in each category. A task set with a class number K = 4, a support set number of N, and a query set number of M = 10 is randomly generated for network parameter optimization

[0141] In the fine-tuning stage, a 7-class rare disease dataset is used, with 60 OCT images in each class; in the testing stage, a 7-class rare disease dataset is used, with 120 OCT images in each class. In the fine-tuning and testing stages, a task with a class number K = 4, a training set number of N, and a test set number of M = 10 is randomly generated. And it is strictly ensured that the test set and the training set have no intersection to accurately evaluate the performance of the model on new samples

[0142] Table 1 in the appendix. Composition of the OCT dataset

[0143] ,

[0144] Our invention uses accuracy as the evaluation index for the small-sample learning method in classification tasks

[0145] ,

[0146] Each fine-tuning and test experiment contains 200 tasks. In the test phase, the test experiment is repeated 5 times to obtain a classification experiment of 1000 tasks. Finally, the average accuracy rate with a 95% confidence interval is reported.

[0147] The test tasks are 4 types of N-shot tasks. The value of N in meta-distillation training is the same as that in fine-tuning, and the test results are calculated when N is 1, 5, and 10 respectively.

[0148] 2) Experimental setup

[0149] The model is trained and tested based on the public platform PyTorch and a GeForce GTX 2080Ti GPU with 11GB of memory. In the meta-training phase, the Stochastic Gradient Descent Algorithm (SGD) is used to update the model weights. A total of 50 epochs are executed. The learning rates of both the inner loop and the outer loop are initialized to 0.05, the initial optimization momentum is 0.9, and the weight decay coefficient is 0.0005. In the distillation meta-learning phase, the Adam optimizer is used to optimize the model parameters for both the teacher network and the student network, and a total of 50 epochs are trained. The initial learning rate of the teacher network is set to 0.005, and the learning rate is updated to 0.5 times every 10 epochs. The initial learning rate of the student network is set to 0.001, and the learning rate is updated to 0.5 times every 10 epochs.

[0150] 3) Results

[0151] Table 2 shows the results of the ablation experiment, Table 3 shows the results of the comparative experiment with other few-shot learning methods, and Table 4 shows the results of the comparative experiment with other distillation learning methods. In the ablation experiment, distillation meta-learning is used as the basic method (Baseline), and the two-stage training strategy, attention-guided distillation module, and temperature coefficient adaptation designed in the present invention are added to the basic method (Baseline). The classification accuracy on the test samples under various settings has been significantly improved. In the comparative experiment, the method proposed in the present invention outperforms other classical methods in most cases, indicating that the two-stage distillation meta-learning method proposed in the present invention can effectively solve the problem of model performance degradation caused by insufficient sample size of rare fundus diseases.

[0152] Table 2, Results of the ablation experiment

[0153] ,

[0154] Table 3, Results of the comparative experiment with other few-shot learning methods

[0155] ,

[0156] Appendix 4. Comparative Experiments with Other Distillation Learning Methods

[0157] ,

[0158] Example 3

[0159] The third embodiment of the present invention is different from the previous two embodiments in that:

[0160] If the above-mentioned functions are implemented in the form of software function units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such understanding, the technical solution of the present invention, in essence, or the part that makes contributions to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs, etc., which can store program codes of various kinds.

[0161] The logic and / or steps represented in the flowchart or described in other ways herein, for example, can be considered as a definite sequence list of executable instructions for implementing logical functions, and can be specifically implemented in any computer-readable medium for use by an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processor, or other systems that can fetch instructions from the instruction execution system, apparatus, or device and execute the instructions), or used in combination with these instruction execution systems, apparatuses, or devices. For the purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device.

[0162] More specific examples (non-exhaustive list) of computer-readable media include the following: electrical connections (electronic devices) having one or more wirings, portable computer disk cartridges (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber devices, and portable compact disc read-only memory (CDROM). Additionally, the computer-readable media can even be paper or other suitable media on which the program can be printed, because the program can be obtained electronically, for example, by optically scanning the paper or other media, followed by editing, interpretation, or otherwise processing it in a suitable manner if necessary, and then storing it in a computer memory.

[0163] It should be understood that the various parts of the present invention can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one or a combination of the following techniques well known in the art can be used: discrete logic circuits having logic gate circuits for implementing logical functions on data signals, application-specific integrated circuits having suitable combinational logic gate circuits, programmable gate arrays (PGA), field programmable gate arrays (FPGA), etc.

[0164] Example 4

[0165] This is the fourth embodiment of the present invention. This embodiment provides a rare fundus disease OCT image classification system based on distilled meta-learning, which is characterized in that it includes a network construction module, a meta-training module, a joint training module, an adjustment module, and an image classification output module;

[0166] The network construction module includes building two retinal OCT image classification networks based on convolutional layers, residual connections, and fully connected layers;

[0167] The meta-training module includes randomly constructing small-sample training tasks for common fundus diseases based on a common fundus disease training set, and performing meta-training on the teacher network to obtain a small-sample classification teacher network model applicable to different combinations of common fundus diseases;

[0168] The joint training module includes randomly constructing small-sample training tasks for common fundus diseases based on a common fundus disease training set, and based on the distilled meta-learning training strategy, combining attention-guided distillation and a temperature coefficient adaptive module to jointly train the teacher network and the student network to obtain a small-sample classification student network model applicable to different combinations of common fundus diseases;

[0169] The adjustment module includes constructing a training set for rare fundus diseases, and based on the training samples, fine-tuning the student network model to obtain a small-sample classification network model applicable to rare diseases.

[0170] The image classification output module includes inputting the OCT image into the small-sample classification network model applicable to rare diseases to obtain the classification result of the lesion type.

[0171] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not restrictive. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the technical solutions of the present invention, and they should all be covered by the scope of the claims of the present invention.

Claims

1. A method for classifying OCT images of rare fundus diseases based on distilled meta-learning, characterized in that: including, building two retinal OCT image classification networks based on convolutional layers, residual connections, and fully connected layers; randomly constructing small-sample training tasks for common eye diseases based on a common fundus disease training set, and performing meta-training on the teacher network to obtain a small-sample classification teacher network model applicable to different combinations of common eye diseases; randomly constructing small-sample training tasks for common eye diseases based on a common fundus disease training set, and jointly training the teacher network and the student network based on the distillation meta-learning training strategy, combined with attention-guided distillation and temperature coefficient adaptive module, to obtain a small-sample classification student network model applicable to different combinations of common eye diseases; constructing a rare fundus disease training set, and fine-tuning the student network model based on the training set to obtain a small-sample classification network model applicable to rare diseases; inputting the OCT image into the small-sample classification network model applicable to rare diseases to obtain the classification result of the lesion type; the two retinal OCT image classification networks are respectively a teacher network and a student network, both networks take OCT images as input and the image lesion category label as output; the distillation meta-learning training strategy is expressed as Let the student network parameters be θ S , θ' S be a temporary copy of the student network, and θ' S is initialized to be equal to θ S . Update θ' S on the support set S of a batch task tr as follows: where, θ T is the current teacher network parameter, denotes the gradient with respect to θ′ S and λ is the learning rate of the student network; Loss function L S consists of three parts: L S = L soft + L hard + L AGD Among them, L soft is the KL divergence between the soft label output by the teacher network and the soft label output by the student network, and L hard is the cross-entropy loss between the hard label output by the student network and the true label, and L AGD is the attention-guided distillation loss; Let the output vector of the teacher network at the end be \(z\) T \(\in\mathbb{R}\) K×1 Let the \(k\)-th term of \(z\) T be \(z\) T and the \(j\)-th term of After being processed by the softmax function containing the distillation temperature coefficient \(T\), the output label \(t\) is obtained k which is expressed as: Let the output vector of the student network be \(z\) S ∈R K×1 Let the \(k\)-th term of \(z\) S be z S and the \(j\)-th term of After being processed by the softmax function with temperature parameter \(T\), the output label \(s\) is obtained k , the soft-label KL divergence loss is calculated as: Calculate the hard label q of the student network output k : the hard-label cross-entropy loss is calculated as: where p k is the value of the k-th class in the true label of the sample; Let F t ∈ R C,H,W be the features extracted by the teacher network, and F s ∈ R C,H,W be the features extracted by the student network, where C, H, and W represent the number of channels, height, and width respectively. The channel attention and spatial attention mechanisms are used to generate the teacher channel feature map A ca (F t ), the student channel feature map A ca (F s ), the teacher spatial feature map A sa (F t ), and the student spatial feature map A sa (F s ). The attention maps of the teacher and student networks obtained are summed up, and the spatial attention mask M sa and the channel attention mask M ca used in attention-guided distillation are obtained through the softmax function. The formula is expressed as: M sa = H·W·softmax((A ca (F s ) + A ca (F t )) / T) M ca = C · softmax((A ca (F s ) + A ca (F t )) / T) Calculate the attention-guided distillation loss L AGD , L AGD includes the attention transfer loss L AT and the feature distillation loss L FD , using L AT to encourage the student network to imitate the spatial and channel attention of the teacher network, which is expressed by the formula: L AT = L2(A sa (F s ), A sa (F t )) + L2(A ca (F s ), A ca (F t )) where L2 is the Euclidean distance between two features; Using L FD Encourage students' networks to imitate the features of the teacher's network, and by adjusting the weights on different pixels and channels in the feature map, enable the students' networks to identify key pixels and important channels in the image. It is expressed by the formula as follows: Among them, is F t the value at the (c, a, b) position, is M sa the value at the (a, b) position, is M ca the c-th value of, is F s the value at the (c, a, b) position; Sum L AT and L FD with weights to obtain the final attention-guided distillation loss L AGD : L AGD = αL AT + βL FD where α and β are weight hyperparameters; Set the initial temperature coefficient T N , and adaptively adjust the temperature T according to the output of the teacher network, expressed as: where p t is the maximum value in the predicted probability distribution of the teacher network for the samples; the distillation meta-learning training strategy is also expressed as After a batch of tasks are trained and completed, based on the query set S ts update the parameters θ of the teacher network with the loss function on T : Among them, represents the gradient with respect to θ T , T i represents the i-th task, and μ is the learning rate of the teacher network; Discard θ′ S and use the updated θ T Update the true student network θ on the support set samples S , expressed as: Among them, represents the gradient with respect to θ S , T i represents the i-th task, and λ is the learning rate of the student network.

2. The OCT image classification method for rare fundus diseases based on distilled meta-learning according to claim 1, wherein: The random structure common disease small sample training task includes constructing a small sample training task set D for common fundus diseases tr ={T1,T2,...,T n}}, where the i-th task is Each task contains K categories in the common disease dataset. In each category, N samples are taken as the support set data and M samples are taken as the query set data 3. The method for classifying OCT images of rare fundus diseases based on distilled meta - learning according to claim 2, wherein: the meta-training includes performing meta-training on the teacher network using the model-agnostic meta-learning method based on the common fundus disease small-sample training task set; The model-agnostic meta-learning method includes an inner loop and an outer loop. The inner loop is used to train a few-shot classification network model applicable to a specific category combination, and the outer loop is used to train a few-shot classification network model applicable to different lesion combinations, obtaining a set of teacher network model parameters for new tasks adapted to different lesion combinations 4. The method for classifying OCT images of rare fundus diseases based on distilled meta-learning according to claim 3, wherein: The fine-tuning of the student network model includes constructing a training set for rare fundus diseases with K categories, where each category of rare disease uses N samples. The values of K and N are the same as those in meta-training. Based on sample training, adaptation to new tasks is carried out, and the model parameters θ finally output by the student network are used. S Using this as the initial value, gradient descent optimization is performed on the small sample training set of rare fundus diseases to obtain a small sample classification network model applicable to rare diseases.

5. A system for a method of classifying OCT images of rare fundus diseases based on distilled meta-learning according to any one of claims 1-4, characterized in that: including a network construction module, a meta-training module, a joint training module, an adjustment module, and an image classification output module; the network construction module includes building two retinal OCT image classification networks based on convolutional layers, residual connections, and fully connected layers; the meta-training module includes randomly constructing small-sample training tasks for common eye diseases based on a common fundus disease training set, and performing meta-training on the teacher network to obtain a small-sample classification teacher network model applicable to different combinations of common eye diseases; the joint training module includes randomly constructing small-sample training tasks for common eye diseases based on a common fundus disease training set, and jointly training the teacher network and the student network based on the distillation meta-learning training strategy, combined with attention-guided distillation and temperature coefficient adaptive module, to obtain a small-sample classification student network model applicable to different combinations of common eye diseases; the adjustment module includes constructing a rare fundus disease training set, and fine-tuning the student network model based on the training samples to obtain a small-sample classification network model applicable to rare diseases; the image classification output module includes inputting the OCT image into the small-sample classification network model applicable to rare diseases to obtain the classification result of the lesion type.

6. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 4.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the method according to any one of claims 1 to 4.

Images