Eye fundus image classification method and device based on multi-task curriculum type learning, equipment and medium

Abstract

The invention discloses an eye fundus image classification method and device based on multi-task curriculum type learning, equipment and a medium. The method comprises the following steps: training a teacher network through employing an eye fundus image sample, and then generating an evidence graph of the eye fundus image sample; designing a multi-task student network comprising two branches of evidence graph prediction and glaucoma prediction; designing a sample prior weighting coefficient and a sample feedback loss coefficient according to glaucoma classification labels and prediction results of the teacher network and the student network, and designing a loss function of the student network based on the two coefficients; taking the eye fundus image sample as input of the two prediction branches of the student network, taking the classification label as output of the glaucoma prediction branch, taking a label evidence graph as the output of the evidence graph prediction branch, and training the student network based on the loss function; and using the trained student network to generate the glaucoma classification label and the evidence graph of a to-be-classified eye fundus image. According to the invention, the classification accuracy is improved and the evidence graph of a classification decision is generated.

Description

technical field

[0001] The invention belongs to the field of image information processing, and in particular relates to a glaucoma diagnosis method, device, equipment and method based on multi-task course learning. Background technique

[0002] Glaucoma has become the second leading cause of blindness in the world, threatening the visual health of more than 65 million people. Glaucoma develops slowly and its symptoms are mild in the early stages, making it easy for patients to ignore the condition, which can cause irreversible damage to their vision. Therefore, early screening and treatment are crucial for the prevention and treatment of glaucoma. The common clinical diagnosis method of glaucoma is the optic nerve head (ONH) examination, which refers to the comprehensive analysis of the pathological phenomenon and physiological structure of glaucoma in fundus images by ophthalmologists. The main pathological changes are neural retinal edge erosion, optic cup dilation, reti...

Examples

Embodiment 1

[0066] Embodiment 1 provides a fundus image classification method based on multi-task curriculum learning, such as figure 1 shown, proceed as follows:

[0067] Step A, design a teacher network based on the self-attention mechanism; take the fundus image sample as input and the glaucoma classification label as output, and supervise the training of the teacher network;

[0068] 1) Design a teacher network based on the self-attention mechanism

[0069] The structure of the teacher network is as follows figure 2As shown, it includes in turn: the ResNet-34 backbone structure with the fully connected layer removed, the convolutional layer, the GC self-attention mechanism module, the global average pooling layer, and the fully connected layer. In the ResNet-34 backbone structure with the fully connected layer removed, a set of feature maps after each pooling layer is named as the 1st to 5th set of feature maps in sequence, and the 2nd to 5th set of feature maps are reduced to Sam...

Embodiment 2

[0120] This embodiment provides a fundus image classification device based on multi-task curriculum learning, including: a teacher network module and a multi-task student network module; wherein,

[0121] The structure of the teacher network module is designed based on a self-attention mechanism, and the fundus image sample is used as an input, and the glaucoma classification label is used as an output to perform supervised training, and after the training is completed, it is used to generate a label evidence map corresponding to each fundus image sample;

[0122] The structure of the multi-task student network module includes an evidence map prediction branch and a glaucoma prediction branch; and the loss function for training the multi-task student network is designed according to the sample prior weighting coefficient θ and the sample feedback loss coefficient α of the fundus image sample Obtained; among them, the sample prior weighting coefficient θ is designed according to...

Embodiment 3

[0127] This embodiment provides an electronic device, including a memory and a processor, where a computer program is stored in the memory, and when the computer program is executed by the processor, the processor implements the method described in Embodiment 1.

technical field

[0001] The invention belongs to the field of image information processing, and in particular relates to a glaucoma diagnosis method, device, equipment and method based on multi-task course learning. Background technique

[0002] Glaucoma has become the second leading cause of blindness in the world, threatening the visual health of more than 65 million people. Glaucoma develops slowly and its symptoms are mild in the early stages, making it easy for patients to ignore the condition, which can cause irreversible damage to their vision. Therefore, early screening and treatment are crucial for the prevention and treatment of glaucoma. The common clinical diagnosis method of glaucoma is the optic nerve head (ONH) examination, which refers to the comprehensive analysis of the pathological phenomenon and physiological structure of glaucoma in fundus images by ophthalmologists. The main pathological changes are neural retinal edge erosion, optic cup dilation, reti...

Examples

Embodiment 1

[0066] Embodiment 1 provides a fundus image classification method based on multi-task curriculum learning, such as figure 1 shown, proceed as follows:

[0067] Step A, design a teacher network based on the self-attention mechanism; take the fundus image sample as input and the glaucoma classification label as output, and supervise the training of the teacher network;

[0068] 1) Design a teacher network based on the self-attention mechanism

[0069] The structure of the teacher network is as follows figure 2As shown, it includes in turn: the ResNet-34 backbone structure with the fully connected layer removed, the convolutional layer, the GC self-attention mechanism module, the global average pooling layer, and the fully connected layer. In the ResNet-34 backbone structure with the fully connected layer removed, a set of feature maps after each pooling layer is named as the 1st to 5th set of feature maps in sequence, and the 2nd to 5th set of feature maps are reduced to Sam...

Embodiment 2

[0120] This embodiment provides a fundus image classification device based on multi-task curriculum learning, including: a teacher network module and a multi-task student network module; wherein,

[0121] The structure of the teacher network module is designed based on a self-attention mechanism, and the fundus image sample is used as an input, and the glaucoma classification label is used as an output to perform supervised training, and after the training is completed, it is used to generate a label evidence map corresponding to each fundus image sample;

[0122] The structure of the multi-task student network module includes an evidence map prediction branch and a glaucoma prediction branch; and the loss function for training the multi-task student network is designed according to the sample prior weighting coefficient θ and the sample feedback loss coefficient α of the fundus image sample Obtained; among them, the sample prior weighting coefficient θ is designed according to...

Embodiment 3

[0127] This embodiment provides an electronic device, including a memory and a processor, where a computer program is stored in the memory, and when the computer program is executed by the processor, the processor implements the method described in Embodiment 1.

Images