Rare fundus disease OCT image classification method and system based on distillation element learning

Abstract

The invention relates to the technical field of image classification, in particular to a rare fundus disease OCT image classification method and system based on distillation element learning. Respectively constructing a teacher network and a student network for retina OCT image classification on the basis of the residual module and the full connection layer; constructing a common fundus disease training set, and performing meta-training on the teacher network; the method comprises the following steps: randomly constructing a common disease small sample training task, carrying out joint training on a teacher network and a student network based on a distillation element learning training strategy, introducing an attention guidance distillation module, and training students to adaptively learn characteristics of teachers in pixels and channels with relatively high attention values by adopting a temperature coefficient adaptive method; constructing a rare fundus disease training set, and finely adjusting the student network to obtain a small sample classification network model suitable for rare diseases; and inputting the OCT image into the rare disease small sample classification network model to obtain a classification result of the lesion type.

Description

Technical Field

[0001] The present invention relates to the technical field of image classification, and in particular to a method and system for classifying rare fundus disease OCT images based on distillation meta-learning. Background Art

[0002] The retina is an important part of the human eye, and the health of the retina is crucial to the human visual system. Any damage, disease or abnormality of the retina may lead to eye problems or visual impairment. Optical Coherence Tomography (OCT) technology is widely used in the diagnosis of retinal diseases in the field of ophthalmology. Automatic classification of retinal OCT images can effectively help doctors screen, diagnose and follow up patients. At present, convolutional neural networks have been widely used in the field of medical imaging, providing assistance for the automatic screening of some common diseases. However, for rare diseases with a small number of cases, traditional deep learning methods cannot achieve good results.

[0003] Few Shot Learning (FSL) has attracted widespread attention in many fields such as image classification, object detection, and semantic segmentation. One of the important reasons is that the training of deep learning relies on a large amount of labeled data. In some practical applications, such as rare diseases, animals, plants, etc., it is very difficult to obtain a sufficient number of samples. The goal of few-shot learning is to identify new categories using only a small number of labeled samples. In fact, the emergence of few-shot learning is inspired by the human learning mechanism. When faced with new tasks or new categories, humans are usually able to quickly learn and generalize knowledge from limited samples. This ability has inspired researchers to conduct research on few-shot learning to enable machine learning methods to simulate the way humans learn.

[0004] The meta-learning method is an important approach to deal with small sample learning problems. Its core idea is to let the model learn the ability to analyze and solve problems from past knowledge and experience, thereby guiding the learning process of new tasks. In this way, the model can quickly adapt and solve problems when faced with new tasks using only a small amount of sample data. Model-Agnostic Meta-Learning (MAML) is one of the most classic methods of meta-learning. The core of this method is to maximize the sensitivity of the loss function to the weights initialized for the new task. Regardless of the model used, the weights are optimized along the direction of the sum of the gradient vectors of each task, and the optimal weights of the model are estimated for the new task, so that it can achieve good results on the new task. Since then, many scholars have optimized and improved MAML in many aspects. Summary of the invention

[0005] In view of the above problems existing in the prior art, the present invention is proposed.

[0006] Because traditional deep learning cannot effectively classify rare retinal diseases. The present invention proposes a rare fundus disease OCT image classification method based on distillation meta-learning. The purpose is to train a model with low complexity, strong generalization and better classification performance for the classification of rare retinal disease OCT images based on common disease data training using only a small number of rare disease retinal OCT images. The present invention proposes a two-stage distillation meta-learning method, which trains a network model with low complexity through knowledge distillation technology. The student network is encouraged to pay attention to the information in key areas, so as to better guide the model for classification. In addition, in order to improve the adaptability of the method under different tasks, the parameters are automatically optimized during the training process, so that the method can automatically adjust the parameters according to different task requirements, thereby optimizing its performance. The present invention can effectively improve the performance of rare fundus disease OCT image classification, and shows superiority compared with other existing small sample learning methods.

[0007] In order to solve the above technical problems, the present invention provides the following technical solutions: a rare fundus disease OCT image classification method based on distillation meta-learning, comprising: building two retinal OCT image classification networks based on convolutional layers, residual connections, and fully connected layers; Based on the common fundus disease training set, we randomly constructed small sample training tasks for common diseases, meta-trained the teacher network, and obtained a small sample classification teacher network model suitable for different common disease combinations. Based on the common fundus disease training set, a small sample training task of common diseases is randomly constructed. Based on the distillation meta-learning training strategy, combined with attention-guided distillation and temperature coefficient adaptive modules, the teacher network and the student network are jointly trained to obtain a small sample classification student network model suitable for different common disease combinations. Construct a training set for rare fundus diseases, and fine-tune the student network model based on the training samples to obtain a small sample classification network model suitable for rare diseases; The OCT images are input into a small sample classification network model suitable for rare diseases to obtain the classification results of the lesion type.

[0008] As a preferred solution of the rare fundus disease OCT image classification method based on distillation meta-learning described in the present invention, the two retinal OCT image classification networks are respectively a teacher network and a student network, both networks take OCT images as input and image lesion category labels as output.

[0009] As a preferred solution of the rare fundus disease OCT image classification method based on distillation meta-learning described in the present invention, wherein: the random construction of common disease small sample training tasks includes constructing a common fundus disease training set containing more than or equal to 6 categories, randomly sampling, and constructing a common fundus disease small sample training task set ={ }, where the i-th task is , each task contains categories, in each category, take The samples are support set data , M samples are query set data .

[0010] As a preferred solution of the rare fundus disease OCT image classification method based on distillation meta-learning described in the present invention, wherein: the meta-training includes, based on a small sample training task set of common fundus diseases, using a model-independent meta-learning method to meta-train the teacher network; The model-independent meta-learning method includes an inner loop and an outer loop, wherein the inner loop is used to train a small sample classification network model suitable for a specific category combination, and the outer loop is used to train a small sample classification network model suitable for different lesion combinations, thereby obtaining a set of teacher network model parameters suitable for new tasks with different lesion combinations. .

[0011] As a preferred solution of the rare fundus disease OCT image classification method based on distillation meta-learning described in the present invention, wherein: the distillation meta-learning training strategy is expressed as: Assume that the parameters of the student model are A temporary copy of the student model, Initialized with Equal, yes The support set in a batch task Update on: , in, is the current teacher model parameter, Express Find the gradient, is the learning rate of the student network; Loss Function It consists of three parts: , in, is the KL divergence between the soft labels output by the teacher network and the soft labels output by the student network, is the cross entropy loss between the hard labels and the true labels output by the student network, To note the guided distillation losses; Suppose the final output vector of the teacher model is , is the number of categories, let No. Item is , No. Item is , including distillation temperature coefficient The output label is obtained after the softmax function is processed , expressed as: , Suppose the final output vector of the student model is ,set up The kth item of , The jth term of , including temperature parameters The output label is obtained after the softmax function is processed , , The soft label KL divergence loss is calculated as: , Compute hard labels for the student network output : , The hard label cross entropy loss is calculated as: , in, is the true label of the sample The value of the class; set up is the feature extracted by the backbone network, where Represent the number of channels, height and width respectively, and use the channel attention and spatial attention mechanisms to generate the teacher channel feature map for the features extracted from the teacher network and the student network respectively ( ), Student channel characteristic diagram ( ) and the teacher space feature map , Student Space Feature Map , the attention maps of the teacher and student models are summed, and the spatial attention mask used in attention-guided distillation is obtained through the softmax function and channel attention mask , the formula is: , , Compute the attention-guided distillation loss , Including attention shift loss and feature distillation loss ,use The student model is encouraged to imitate the spatial and channel attention of the teacher model, which is formulated as: , in, is the Euclidean distance between two features; use The student model is encouraged to imitate the characteristics of the teacher model, and by adjusting the weights on different pixels and channels in the feature map, the student model can identify the key pixels and important channels in the image. The formula is expressed as: , in, for exist The value of the position, for exist The value of the position, for No. Values; Will and Perform weighted summation to obtain the final attention-guided distillation loss : , in, and is the weight hyperparameter; Set the initial temperature coefficient , and adaptively adjust the temperature according to the output of the teacher model : , , in, It is the maximum value in the probability distribution of the teacher model’s prediction of the sample.

[0012] As a preferred solution of the rare fundus disease OCT image classification method based on distillation meta-learning described in the present invention, wherein: the distillation meta-learning training strategy is also expressed as: After a batch of tasks are trained, based on the query set The loss function on updates the parameters of the teacher model : , in, Express Find the gradient, represents the i-th task, is the learning rate of the teacher network; throw away And use the updated Update the true student model on the support set samples , expressed as: , in, Express Find the gradient, represents the i-th task, Learning rate for the student network.

[0013] As a preferred solution of the rare fundus disease OCT image classification method based on distillation meta-learning described in the present invention, wherein: the fine-tuning of the student network model includes constructing a rare fundus disease training set with a number of categories of , each type of rare disease uses samples, and The value of is the same as that in meta-training. Based on sample training, adapt to new tasks and use the model parameters finally output by the student network. As the initial value, gradient descent optimization was performed on a small sample training set of rare fundus diseases to obtain a small sample classification network model suitable for rare diseases.

[0014] As a preferred solution of the rare fundus disease OCT image classification system based on distillation meta-learning described in the present invention, it includes: a network building module, a meta-training module, a joint training module, an adjustment module, and an image classification output module; The network building module includes building two retinal OCT image classification networks based on convolutional layers, residual connections, and fully connected layers; The meta-training module includes randomly constructing small sample training tasks for common diseases based on the common fundus disease training set, meta-training the teacher network, and obtaining a small sample classification teacher network model suitable for different common disease combinations; The joint training module includes randomly constructing small sample training tasks for common diseases based on the common fundus disease training set, and jointly training the teacher network and the student network based on the distillation meta-learning training strategy, combined with the attention-guided distillation and temperature coefficient adaptive modules, to obtain a small sample classification student network model suitable for different common disease combinations; The adjustment module includes constructing a rare fundus disease training set, fine-tuning the student network model based on the training samples, and obtaining a small sample classification network model suitable for rare diseases; The image classification output module includes inputting the OCT image into a small sample classification network model suitable for rare diseases to obtain the classification result of the lesion type.

[0015] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and wherein when the processor executes the computer program, the steps of any one of the methods for classifying rare fundus disease OCT images based on distillation meta-learning are implemented.

[0016] A computer-readable storage medium having a computer program stored thereon, characterized in that when the computer program is executed by a processor, the steps of any one of the methods for classifying rare fundus disease OCT images based on distillation meta-learning are implemented.

[0017] Beneficial effects of the present invention: The present invention proposes a classification method for OCT images of rare fundus diseases based on distillation meta-learning. The method combines knowledge distillation with meta-learning, and enables the student model to benefit from the rich knowledge of the teacher model through knowledge transfer from the teacher model to the student model. This helps to improve the generalization ability of the student model, so that it performs better on unseen tasks and data. In addition, an attention-guided distillation method is designed to guide the student model to focus on the information of key lesion areas, thereby helping the model to classify more accurately. In order for the method to adapt to different tasks, a temperature coefficient adaptive mechanism is introduced, and the temperature coefficient is automatically optimized during the training process as an adaptive parameter, which accelerates the process of model training and deployment. Experimental results show that on the constructed small-sample retinal OCT dataset, the performance of this method is significantly improved compared with other distillation learning and small-sample learning methods in most cases. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0019] Figure 1 An overall framework diagram of a two-stage distillation meta-learning method for a rare fundus disease OCT image classification method based on distillation meta-learning provided in one embodiment of the present invention.

[0020] Figure 2 A network structure diagram of a rare fundus disease OCT image classification method based on distillation meta-learning provided for an embodiment of the present invention, wherein: (a) the teacher network Resnet32, and (b) the student network Resnet16.

[0021] Figure 3 An attention-guided distillation module diagram of a rare fundus disease OCT image classification method based on distillation meta-learning provided for an embodiment of the present invention.

[0022] Figure 4 This is an example diagram of various types of images in an OCT dataset of a rare fundus disease OCT image classification method based on distillation meta-learning provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0023] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the drawings of the specification. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in the art without creative work should fall within the scope of protection of the present invention.

[0024] Example 1 Reference Figure 1-Figure 3 , which is the first embodiment of the present invention, and provides a rare fundus disease OCT image classification method based on distillation meta-learning, comprising: This paper designs a distillation meta-learning method to achieve rare fundus disease OCT image classification, as shown in the attached Figure 1 As shown in the figure, the training process includes three stages. The first stage is meta-training. In the first stage, a teacher network is trained with MAML, and the teacher model with initialization parameters is used to guide the training of the student network; the second stage is distillation meta-learning. In the second stage, the teacher model and the student model learn from each other. On the one hand, the student model feeds back the results learned from the teacher model to improve the teacher's ability, and on the other hand, the student model learns the knowledge of the teacher model to obtain higher classification accuracy. At the same time, in order to make the network pay more attention to the information in the key areas, the Attention Guided Distillation (AGD) module is introduced to train students to learn the characteristics of the teacher in pixels and channels with higher attention values. In addition, an Adaptive Temperature Coefficient (ATC) method is proposed to automatically adjust the temperature coefficient to the optimal value during training to help the training of the model. In the third stage, the student network is further fine-tuned to make it suitable for the classification of specific rare diseases.

[0025] The present invention proposes a classification method for rare fundus disease OCT images based on distillation meta-learning, which is suitable for classification of OCT image lesions under small sample conditions. (1) A two-stage training method consisting of meta-training and distillation meta-learning is designed. (2) An attention-guided distillation module is designed, which uses the attention mechanism to identify key pixels and channels in the feature map, thereby helping the student model to focus on important feature values ​​and avoid inaccurate feature recognition and waste of resources. (3) A temperature coefficient adaptive mechanism is designed to balance the ability of the student model in processing difficult-to-classify samples and easy-to-classify samples, and promote the student model to better learn probability distribution knowledge from the teacher model, thereby improving the generalization ability and performance of the model.

[0026] A rare fundus disease OCT image classification method based on distillation meta-learning is characterized by comprising the following steps: S1. Build two retinal OCT image classification networks based on convolutional layers, residual connections, and fully connected layers, as teacher networks and student networks respectively. Both networks take OCT images as input and image lesion category labels as output.

[0027] The teacher network and student network structures are shown in the attached figure. Figure 2 shown.

[0028] For the teacher network, the first 31 layers are 3×3 convolutional layers, and the 32nd layer is a fully connected layer. The number of convolutional channels in layers 1-7 is 64, the number of convolutional channels in layers 8-15 is 128, the number of channels in layers 16-25 is 256, and the number of channels in layers 26-31 is 512. The outputs of the 1st and 3rd layers, the 3rd and 5th layers, the 5th and 7th layers, the 9th and 11th layers, the 11th and 13th layers, the 13th and 15th layers, the 17th and 19th layers, the 19th and 21st layers, the 21st and 23rd layers, the 23rd and 25th layers, the 27th and 29th layers, and the 29th and 31st layers are directly added through residual connections and input into the next layer. The outputs of the 7th and 9th layers, the 15th and 17th layers, and the 25th and 27th layers are respectively added through residual connections containing 1×1 convolution and changing the number of channels and then input into the next layer.

[0029] For the student network, the first 15 layers of the network are 3×3 convolutional layers, and the 16th layer is a fully connected layer. The number of convolutional channels in layers 1-3 is 64, the number of convolutional channels in layers 4-7 is 128, the number of channels in layers 8-11 is 256, and the number of channels in layers 12-15 is 512. The outputs of the 1st and 3rd layers, the outputs of the 5th and 7th layers, the outputs of the 9th and 11th layers, and the outputs of the 13th and 15th layers are directly added through residual connections and then input into the next layer. The outputs of the 3rd and 5th layers, the outputs of the 7th and 9th layers, and the outputs of the 11th and 13th layers are added through residual connections containing 1×1 convolutions and changing the number of channels and then input into the next layer.

[0030] Both the teacher network and the student network take OCT images as input and the predicted probability that the image belongs to each type of lesion as output. The maximum value among the probabilities of each category is taken, and the corresponding category is the final image classification result.

[0031] S2. Based on the common fundus disease training set, small sample training tasks for common diseases are randomly constructed, and the teacher network is meta-trained to obtain a small sample classification teacher network model that can be widely applied to different common disease combinations.

[0032] Construct a common fundus disease training set containing more than or equal to 6 categories, randomly sample from it, and construct a small sample training task set for common fundus diseases ={ }. The i-th task is Each task contains K categories in the common disease dataset, and N samples are taken as support set data in each category. , M samples are query set data K is a positive integer between 3 and 6. N is a positive integer less than 11, and M is an integer between 10 and 15. S3. Based on the common fundus disease training set, small sample training tasks for common diseases are randomly constructed. Based on the distillation meta-learning training strategy, combined with attention-guided distillation and temperature coefficient adaptive modules, the teacher network and the student network are jointly trained to obtain a small sample classification student network model that can be widely applied to different common disease combinations.

[0033] Based on the small sample training task set of common fundus diseases, the model-agnostic meta-learning (MAML) method is used to meta-train the teacher network. The MAML method includes an inner loop and an outer loop. The inner loop is used to train a small sample classification network model suitable for a specific category combination, and the outer loop is used to train a small sample classification network model that can be widely applied to different lesion combinations. Finally, a set of teacher network model parameters that can quickly adapt to new tasks with different lesion combinations is obtained. .

[0034] Based on a small sample training task set of common fundus diseases, the parameters of the teacher network are adjusted. and student network parameters After training, the student network model parameters are finally obtained, which can quickly adapt to new tasks with different lesion combinations. The teacher model parameters are initialized as , the student model parameters are randomly initialized.

[0035] Before training begins, you need to create a student model A temporary copy of , Initialized with Equal. Distillation meta-learning consists of three steps in total: first step: right The support set in a batch task Update on: , in, is the current teacher model parameter, Express Find the gradient, is the learning rate of the student network, which is initially set to 0.001 and is updated to 0.5 times every 10 rounds. It consists of three parts: , in, is the KL divergence between the soft labels output by the teacher network and the soft labels output by the student network, is the cross entropy loss between the hard labels and the true labels output by the student network, The three losses are summed to obtain the total loss function of the method.

[0036] Soft label KL divergence loss Suppose the final output vector of the teacher model is , is the number of categories, let No. Item is , No. Item is , including distillation temperature coefficient The output label is obtained after the softmax function is processed , expressed as: , Suppose the final output vector of the student model is ,set up The kth item of , The jth term of , including temperature parameters The output label is obtained after the softmax function is processed , , The soft label KL divergence loss is calculated as: , Hard label cross entropy loss: Compute hard labels for the student network output : , The hard label cross entropy loss is calculated as: , in, is the true label of the sample The value of the class; (3) Attention guided distillation loss: Attention guided distillation module as shown in the attached Figure 3 As shown, The features extracted by the backbone network are C, H, and W, respectively, representing the number of channels, height, and width. The channel attention and spatial attention mechanisms are used to generate the teacher channel feature map for the features extracted by the teacher network and the student network. ( ), Student channel characteristic diagram ( ) and the teacher space feature map , Student Space Feature Map Next, the attention maps of the teacher and student models obtained in the above steps are summed and the spatial attention mask used in attention-guided distillation is obtained through the softmax function. and channel attention mask , the formula is: , , Then calculate the attention guided distillation loss , It consists of two parts: attention transfer loss and feature distillation loss . use The student model is encouraged to imitate the spatial and channel attention of the teacher model, which can be expressed as: , in, is the Euclidean distance between two features.

[0037] use To encourage students to imitate the characteristics of the teacher model, and by adjusting the weights on different pixels and channels in the feature map, it can identify the key pixels and important channels in the image, which can be expressed as: , in, for exist The value of the position, for exist The value of the position, for No. Values; Will and Perform weighted summation to obtain the final attention-guided distillation loss : , in, and are weight hyperparameters, which are set to 0.1 and 0.3 respectively in the experiments.

[0038] Temperature coefficient adaptation: set the initial temperature coefficient , and adaptively adjust the temperature according to the output of the teacher model : , , in, is the maximum value in the probability distribution of the teacher model’s prediction of the sample. The initial temperature coefficient in the experiment Set to 4.

[0039] When the teacher model outputs a probability prediction vector for a certain sample, if the prediction probability for a certain category in the vector is high, it indicates that the prediction probability distribution for the sample is uneven, and the confidence In this case, a smaller temperature coefficient T is set to reduce the entropy of the output distribution, so that the network can learn the knowledge of the classification probability distribution more confidently. On the contrary, when the output probability is uniform, the confidence is low. When , a higher temperature coefficient T is set to increase the entropy of the output distribution, thereby enhancing the learning of this sample class during training and making the student model learn the knowledge of the teacher model more diversely.

[0040] Step 2: After a batch of tasks have been trained through the above steps, based on the query set The loss function on updates the parameters of the teacher model : , in, Express Find the gradient, represents the i-th task. is the teacher network learning rate, which is initially set to 0.005 and updated to 0.5 times every 10 rounds.

[0041] Step 3: throw away And use the updated Update the true student model on the support set samples This step applies the updated teacher network to the student network immediately, thus aligning the training process: , in, Express Find the gradient, represents the i-th task. is the learning rate of the student network, which is initially set to 0.001 and updated to 0.5 times every 10 rounds.

[0042] S4. Construct a training set for rare fundus diseases, and fine-tune the student network model based on the training samples to obtain a small sample classification network model suitable for rare diseases; Construct a training set for rare fundus diseases, with K categories. Use N samples for each rare disease category. The values ​​of K and N are the same as those in meta-training. Based on these sample training, adapt to the new task. Use the model parameters finally output by the student network As the initial value, gradient descent optimization is performed on a small sample training set of rare fundus diseases to enable the model to adapt to new tasks and obtain a small sample classification network model suitable for rare diseases.

[0043] S5. Input the OCT image into a small sample classification network model suitable for rare diseases to obtain the classification result of the lesion type.

[0044] The test OCT image is input into the trained rare disease small sample classification student network model obtained in step S5 to obtain the classification result.

[0045] Example 2 refer to Figure 4 , which is the second embodiment of the present invention, provides a rare fundus disease OCT image classification method based on distillation meta-learning. In order to verify the beneficial effects of the present invention, scientific demonstration is carried out through experiments.

[0046] It should be noted that the data contains OCT images acquired by the Optovue scanner, among which common diseases include age-related macular degeneration (AMD), diabetic retinopathy (DR), choroidal neovascularization (CNV), central serous chorioretinopathy (CSC), retinal vein occlusion (RVO), and the common disease data set also includes the "normal" category, a total of 6 categories. Rare diseases include retinal detachment (RD), retinal hemorrhage (RH), optic atrophy (OA), epiretinal membrane (ERM), central retinal artery occlusion (CRAO), retinoschisis (RS), and retinal detachment (RD), a total of 7 categories. Examples of various images are shown in the attached Figure 4 shown.

[0047] As shown in Appendix 1, the OCT dataset is divided into three parts, which are used for meta-training and distillation meta-learning, fine-tuning, and testing. The datasets for meta-training and distillation meta-learning are the same, including normal and five common fundus diseases, a total of 6 categories, and 400 OCT images per category. A randomly generated task set with K=4 categories, N support sets, and M=10 query sets is used for network parameter optimization.

[0048] In the fine-tuning stage, 7 rare disease datasets were used, each containing 60 OCT images; in the testing stage, 7 rare disease datasets were used, each containing 120 OCT images. In the fine-tuning and testing stages, tasks with K=4 categories, N training sets, and M=10 test sets were randomly generated. It was strictly ensured that the test set had no intersection with the training set to accurately evaluate the performance of the model on new samples.

[0049] Appendix 1. Composition of OCT dataset , Our invention uses accuracy as the evaluation indicator of small sample learning methods for classification tasks.

[0050] , Each fine-tuning and testing experiment contains 200 tasks. In the testing phase, the testing experiment is repeated 5 times, resulting in 1000 classification experiments on the task. Finally, the average accuracy with 95% confidence interval is reported.

[0051] The test task is a 4-type N-shot task. The N value is the same in meta-distillation training and fine-tuning, and the test results are calculated when N is 1, 5, and 10 respectively.

[0052] 2) Experimental setup The model is trained and tested based on the public platform PyTorch and a GeForce GTX 2080Ti GPU with 11GB memory. In the meta-training stage, the Stochastic Gradient Descent Algorithm (SGD) method is used to update the model weights, and a total of 50 rounds are performed. The inner and outer loop learning rates are initialized to 0.05, the initial optimization momentum is 0.9, and the weight decay coefficient is 0.0005. In the distillation meta-learning stage, the Adam optimizer is used to optimize the model parameters for both the teacher network and the student network, and a total of 50 rounds are trained. The initial learning rate of the teacher network is set to 0.005, and the learning rate is updated to 0.5 times every 10 rounds. The initial learning rate of the student network is set to 0.001, and the learning rate is updated to 0.5 times every 10 rounds.

[0053] 3) Results Appendix 2 shows the results of ablation experiments, Appendix 3 shows the results of comparative experiments with other small sample learning methods, and Appendix 4 shows the results of comparative experiments with other distillation learning methods. In the ablation experiment, distillation meta-learning is used as the basic method Baseline, and the two-stage training strategy, attention-guided distillation module and temperature coefficient adaptation designed by the present invention are added to the basic method Baseline, which significantly improves the classification accuracy on test samples under various settings. In the comparative experiment, the performance of the method proposed in the present invention surpasses other classical methods in most cases, indicating that the two-stage distillation meta-learning method proposed in the present invention can effectively solve the problem of model performance degradation caused by insufficient sample size of rare fundus diseases.

[0054] Appendix 2. Ablation experiment results , Appendix 3. Comparative experimental results with other small sample learning methods , Appendix 4. Comparative experiments with other distillation learning methods ,

[0055] Example 3 The third embodiment of the present invention is different from the first two embodiments in that: If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program codes.

[0056] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0057] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.

[0058] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0059] Example 4 This is the fourth embodiment of the present invention, which provides a rare fundus disease OCT image classification system based on distillation meta-learning, characterized by: including a network building module, a meta-training module, a joint training module, an adjustment module, and an image classification output module; The network building module includes building two retinal OCT image classification networks based on convolutional layers, residual connections, and fully connected layers; The meta-training module includes randomly constructing small sample training tasks for common diseases based on the common fundus disease training set, meta-training the teacher network, and obtaining a small sample classification teacher network model suitable for different common disease combinations; The joint training module includes randomly constructing small sample training tasks for common diseases based on the common fundus disease training set, and jointly training the teacher network and the student network based on the distillation meta-learning training strategy, combined with the attention-guided distillation and temperature coefficient adaptive modules, to obtain a small sample classification student network model suitable for different common disease combinations; The adjustment module includes constructing a rare fundus disease training set, fine-tuning the student network model based on the training samples, and obtaining a small sample classification network model suitable for rare diseases; The image classification output module includes inputting the OCT image into a small sample classification network model suitable for rare diseases to obtain the classification result of the lesion type.

[0060] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A rare fundus disease OCT image classification method based on distillation meta-learning, characterized by: include, Two retinal OCT image classification networks were built based on convolutional layers, residual connections, and fully connected layers; Based on the common fundus disease training set, we randomly constructed small sample training tasks for common diseases, meta-trained the teacher network, and obtained a small sample classification teacher network model suitable for different common disease combinations. Based on the common fundus disease training set, a small sample training task of common diseases is randomly constructed. Based on the distillation meta-learning training strategy, combined with attention-guided distillation and temperature coefficient adaptive modules, the teacher network and the student network are jointly trained to obtain a small sample classification student network model suitable for different common disease combinations. Construct a training set for rare fundus diseases, and fine-tune the student network model based on the training samples to obtain a small sample classification network model suitable for rare diseases; The OCT images are input into a small sample classification network model suitable for rare diseases to obtain the classification results of the lesion type.

2. The rare fundus disease OCT image classification method based on distillation meta-learning as claimed in claim 1, characterized in that: The two retinal OCT image classification networks are respectively a teacher network and a student network. Both networks take OCT images as input and image lesion category labels as output.

3. The rare fundus disease OCT image classification method based on distillation meta-learning as claimed in claim 2, characterized in that: The randomly constructed common disease small sample training tasks include constructing a common fundus disease small sample training task set ={ }, among which, The tasks are , each task contains categories, in each category, take The samples are support set data , The samples are query set data .

4. The rare fundus disease OCT image classification method based on distillation meta-learning as claimed in claim 3, characterized in that: The meta-training includes, based on a small sample training task set of common fundus diseases, using a model-independent meta-learning method to meta-train a teacher network; The model-independent meta-learning method includes an inner loop and an outer loop, wherein the inner loop is used to train a small sample classification network model suitable for a specific category combination, and the outer loop is used to train a small sample classification network model suitable for different lesion combinations, thereby obtaining a set of teacher network model parameters suitable for new tasks with different lesion combinations. .

5. The rare fundus disease OCT image classification method based on distillation meta-learning as claimed in claim 4, characterized in that: The distillation meta-learning training strategy is expressed as, Assume that the parameters of the student model are A temporary copy of the student model, Initialized with Equal, yes The support set in a batch task Update on: ， in, is the current teacher model parameter, Express Find the gradient, is the learning rate of the student network; Loss Function It consists of three parts: ， in, is the KL divergence between the soft labels output by the teacher network and the soft labels output by the student network, is the cross entropy loss between the hard labels and the true labels output by the student network, To note the guided distillation losses; Suppose the final output vector of the teacher model is ,set up No. Item is , No. Item is , including distillation temperature coefficient The output label is obtained after the softmax function is processed , expressed as: ， Suppose the final output vector of the student model is ,set up The kth item of , The jth term of , including temperature parameters The output label is obtained after the softmax function is processed , ， The soft label KL divergence loss is calculated as: ， Compute hard labels for the student network output : ， The hard label cross entropy loss is calculated as: ， in, is the true label of the sample The value of the class; set up The features extracted for the teacher network, Features extracted for the student network, where Represent the number of channels, height and width respectively, and use the channel attention and spatial attention mechanisms to generate the teacher channel feature map for the features extracted from the teacher network and the student network respectively ( ), Student channel characteristic diagram ( ) and the teacher space feature map , Student Space Feature Map , the attention maps of the teacher and student models are summed, and the spatial attention mask used in attention-guided distillation is obtained through the softmax function and channel attention mask , the formula is: ， ， Compute the attention-guided distillation loss , Including attention shift loss and feature distillation loss ,use The student model is encouraged to imitate the spatial and channel attention of the teacher model, which is formulated as: ， in, is the Euclidean distance between two features; use The student model is encouraged to imitate the characteristics of the teacher model, and by adjusting the weights on different pixels and channels in the feature map, the student model can identify the key pixels and important channels in the image. The formula is expressed as: ， in, for exist The value of the position, for exist The value of the position, for No. values, for exist The value of the position; Will and Perform weighted summation to obtain the final attention-guided distillation loss : ， in, and is the weight hyperparameter; Set the initial temperature coefficient , and adaptively adjust the temperature according to the output of the teacher model , expressed as: ， ， in, It is the maximum value in the probability distribution of the teacher model’s prediction of the sample.

6. The rare fundus disease OCT image classification method based on distillation meta-learning as claimed in claim 5, characterized in that: The distillation meta-learning training strategy is also expressed as, After a batch of tasks are trained, based on the query set The loss function on updates the parameters of the teacher model : ， in, Express Find the gradient, Indicates tasks, is the learning rate of the teacher network; throw away And use the updated Update the true student model on the support set samples , expressed as: ， in, Express Find the gradient, Indicates tasks, is the learning rate of the student network.

7. The rare fundus disease OCT image classification method based on distillation meta-learning as claimed in claim 6, characterized in that: The fine-tuning of the student network model includes constructing a rare fundus disease training set with the number of categories being , each type of rare disease uses samples, and The value of is the same as that in meta-training. Based on sample training, adapt to new tasks and use the model parameters finally output by the student network. As the initial value, gradient descent optimization was performed on a small sample training set of rare fundus diseases to obtain a small sample classification network model suitable for rare diseases.

8. A system based on the rare fundus disease OCT image classification method based on distillation meta-learning according to any one of claims 1 to 7, characterized in that: Including network building module, meta-training module, joint training module, adjustment module, image classification output module; The network building module includes building two retinal OCT image classification networks based on convolutional layers, residual connections, and fully connected layers; The meta-training module includes randomly constructing small sample training tasks for common diseases based on the common fundus disease training set, meta-training the teacher network, and obtaining a small sample classification teacher network model suitable for different common disease combinations; The joint training module includes randomly constructing small sample training tasks for common diseases based on the common fundus disease training set, and jointly training the teacher network and the student network based on the distillation meta-learning training strategy, combined with the attention-guided distillation and temperature coefficient adaptive modules, to obtain a small sample classification student network model suitable for different common disease combinations; The adjustment module includes constructing a rare fundus disease training set, fine-tuning the student network model based on the training samples, and obtaining a small sample classification network model suitable for rare diseases; The image classification output module includes inputting the OCT image into a small sample classification network model suitable for rare diseases to obtain the classification result of the lesion type.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images