A method, system, device and medium for classifying high myopia traction lesions

Abstract

The present invention discloses a method, system, device, and medium for classifying high myopia tractional lesions. These methods belong to the field of medical image processing technology and aim to address the lack of high-quality and diverse sample images of high myopia tractional lesions, as well as the low accuracy of high myopia tractional lesion classification caused by the difficulty in capturing the differential features between levels. The method first uses labeled data for training and obtains four trained models. A new model is then constructed, comprising five basic models and a distributor, and a multi-head attention mechanism is incorporated. Unlabeled fundus color image samples are then input into each of the four trained models, and the unlabeled fundus color image samples and the outputs of the four trained models are used as inputs to the new model, which is then semi-supervisedly trained. Finally, the fundus color image to be classified is obtained and input into the trained new model, which then outputs the classification result.

Description

Technical Field

[0001] The present invention belongs to the technical field of medical image processing, and relates to the classification of tractional lesions of high myopia, and in particular to a method, system, device and medium for classifying tractional lesions of high myopia. Background Art

[0002] High myopia traction maculopathy is a relatively serious ophthalmic disease with an incidence rate of 1% to 4.7% worldwide, and in East Asia, this rate is as high as 2.6% to 5.5%. This disease often occurs in patients with high myopia, manifesting as lesions such as macular holes, macular detachment, interlaminar macular holes, and full-thickness macular holes. According to studies, up to 35.0% of patients with high myopia suffer from myopic traction maculopathy (MTM), and the incidence rate is highest in East Asia. In addition, high myopia traction maculopathy is one of a series of complications of pathological myopia (PM). PM is characterized by excessive elongation of the eye axis, which leads to deformation of the posterior segment of the eye and visual impairment. It is expected that the prevalence of PM will continue to rise in the next few years, making PM-related eye diseases one of the main causes of irreversible vision loss.

[0003] In recent years, with the development of artificial intelligence technology, its application in disease screening and diagnosis has become increasingly widespread. In the field of ophthalmic disease screening, artificial intelligence-assisted diagnosis and treatment technology can automatically identify, classify, and predict the risk of high myopia fundus lesions using readily available fundus images, even without the need for experienced ophthalmologists or specialized high myopia detection equipment. Regular fundus monitoring and risk prediction for high myopia can be achieved through simple examinations, enabling early detection, intervention, and treatment, which is of great significance for myopia prevention and control.

[0004] The invention patent application with application number 202011429632.1 discloses a device for recognizing fundus lesions in the image of high myopia. Its fundus lesion risk prediction method includes the following steps: obtaining fundus images and randomly dividing the data into training sets and test sets; preprocessing the training set to achieve the purpose of data augmentation, and performing normalization preprocessing on the test set; using the knowledge distillation method to train the classification network model, and sending the training data to the teacher network pre-trained on ImageNet and the student network to be trained respectively; using the soft label value and the true label value output by the teacher network as supervision information, and calculating KL Loss and Focal Loss respectively with the predicted value output by the student network; weighted summing the two different Loss values ​​as the final loss function, which is used for updating the parameters of the student network; the trained student network is used as a classification network model to perform three-category prediction of normal-low-risk high myopia fundus lesions-high-risk high myopia fundus lesions on the fundus image test set, thereby completing the risk prediction of high myopia fundus lesions.

[0005] Patent application number 201911347569.4 also discloses a fundus image multi-disease detection system based on a regional feature set neural network. The system includes a trained multi-disease detection network model, which includes a feature extraction network, a semantic segmentation subnetwork, and multiple classifiers. The feature extraction network is used to extract fundus image features to obtain a final feature map, which includes a two-dimensional convolutional layer with a convolution kernel size of 7×7, a BatchNorm layer, a rectified linear unit, a ReLU layer, a maximum pooling layer, and five feature modules. The semantic segmentation subnetwork is used to segment the optic disc, optic cup and macular areas, and extract the corresponding areas from the final feature map of the fundus photo according to the segmentation results. It is composed of five deconvolution modules, wherein the deconvolution module is composed of sequentially connected Conv3×3, Batch Norm, ReLU, and deconvolution layers; the deconvolution layer is used to upsample the size of the feature map to twice the original size. After the final feature map is upsampled 5 times, the size is restored to the original fundus image size, and then the output is mapped to the probability space through the Softmax function, and the semantic segmentation result is obtained through binarization; the multiple classifiers are used to receive feature combinations of different areas to realize fundus lesion identification. After receiving different feature combinations, they first pass through the global mean pooling layer to reduce their size to 1×1, and then pass through two fully connected layers to obtain an output vector of length k. After passing through the Softmax function, the vector can obtain the probability of each category, thereby realizing fundus disease identification.

[0006] As in the above-mentioned patent application, the existing technology requires a large number of sample images of high myopia tractional lesions in the process of identifying and classifying high myopia tractional lesions, but lacks high-quality and diverse sample images of high myopia tractional lesions for training network models. In addition, the classification of high myopia tractional lesions is actually a six-category task, which requires the lesions to be divided into six different levels, from level 0 to level 5. Each level corresponds to a different type and severity of lesions, which requires the model to understand and distinguish the subtle differences between the levels. However, it is difficult for existing models to understand and distinguish the differences between the levels, making it difficult for the model to capture these features and correctly associate them with the corresponding levels, and the classification accuracy of high myopia tractional lesions is low. Summary of the Invention

[0007] The purpose of the present invention is to provide a method, system, device and medium for classifying tractional lesions of high myopia in order to solve the technical problems in the prior art of lack of high-quality and diversified sample images of tractional lesions of high myopia and low accuracy of classification of tractional lesions of high myopia due to difficulty in capturing the difference characteristics between different levels.

[0008] In order to achieve the above-mentioned purpose, the present invention specifically adopts the following technical solutions:

[0009] A method for classifying high myopia traction lesions comprises the following steps:

[0010] Step S1, obtaining a trained model;

[0011] The labeled fundus color image samples and their label data were input into four network models, namely MobilenetV3, Resnet50, Resnet152 and InceptionV3, and the four network models were trained to obtain four trained models.

[0012] Step S2, building a new model;

[0013] Build a new model, which includes five basic models: MobilenetV2, MobilenetV3, Resnet50, Resnet152, and InceptionV3. Each of the five basic models is connected to a classifier, and the output of the basic model is used as the input of the corresponding classifier. The tail of the sixth residual block of the MobilenetV2 basic model is attached with the first multi-head attention module, and the tail of the 12th residual block is attached with the second multi-head attention module. The tail of the sixth residual block of the MobilenetV3 basic model is attached with the first multi-head attention module, and the tail of the 13th residual block is attached with the second multi-head attention module. Multi-head attention module, the first multi-head attention module is attached to the tail of the third layer of the Resnet50 base model, the first multi-head attention module is attached to the tail of the fifth residual block of the third stage of the Resnet152 base model, and the second multi-head attention module is attached to the tail of the tenth residual block of the fourth stage; the classifier includes a first linear layer, a first dropout layer, a second linear layer, a second dropout layer, a feature fusion layer, a batch normalization layer, a softmax layer and an output layer arranged in sequence; the first multi-head attention module is connected to the first linear layer through a residual, and the second multi-head attention module is connected to the second linear layer through a residual;

[0014] Step S3, new model training;

[0015] The unlabeled fundus color image samples are input into the four trained models respectively, and the unlabeled fundus color image samples and the output of the four trained models are used as the input of the new model to perform semi-supervised training on the new model;

[0016] Step S4, real-time classification;

[0017] Obtain the fundus color image to be classified and input it into the trained new model, which then outputs the classification result.

[0018] Furthermore, in step S1, when four network models are trained to obtain four trained models, the training method is:

[0019] The number of training rounds is 100 and the batch size is 64; the initial learning rate is 3e-5, and the cosine annealing decay is used to adjust the learning rate after each round of learning, and the AdamW optimizer is used to automatically adjust the learning rate.

[0020] Furthermore, in the classifier of step S2, the dropout rates of the first dropout layer and the second dropout layer are both set to 0.25.

[0021] Furthermore, in step S2, the batch normalization layer is expressed as:

[0022]

[0023] The Softmax layer is represented as:

[0024]

[0025] in, Represents the input data in a batch, represents the mean of the same batch of data, represents the variance of the same batch of data, Represents a very small constant, used to ensure that the denominator is greater than 0; represents the i-th element of the input vector, Indicates the dimension of the input vector.

[0026] Furthermore, in step S3, the specific method of semi-supervised training is:

[0027] Step S3-1: inputting unlabeled fundus color image samples into four trained models respectively, and the four trained models respectively output preliminary prediction results;

[0028] Step S3-2: Compare the preliminary prediction results output by the four trained models with the preset prediction result thresholds. If the preliminary prediction result is greater than the prediction result threshold, a record label is added to the corresponding preliminary prediction result. If the preliminary prediction result is less than the prediction result threshold, the preliminary prediction result is discarded.

[0029] In step S3-3, the preliminary prediction results with record labels and their record labels are input into the new model, and the new model outputs the corresponding re-prediction results; if the accuracy of the re-prediction results is higher than the preset accuracy threshold, the semi-supervised training ends; if the accuracy of the re-prediction results is lower than the accuracy threshold, the process returns to step S3-1, and the new model re-predicts the corresponding preliminary prediction results until the number of iterations reaches the iteration threshold, and the semi-supervised training ends.

[0030] Furthermore, in step S3, during semi-supervised training, the number of training rounds is set to 100 and the batch size is set to 16; the initial learning rate is set to 1e-2, and the step learning rate scheduler is used to adjust the learning rate after each round of learning, and the AdamW optimizer is used to automatically adjust the learning rate;

[0031] The step size of the step learning rate scheduler is set to the number of steps per epoch, and the decay coefficient is set to 0.5.

[0032] A classification system for traction lesions in high myopia includes:

[0033] The trained model acquisition module is used to input labeled fundus color image samples and their label data into four network models, namely MobilenetV3, Resnet50, Resnet152, and InceptionV3, and train the four network models to obtain four trained models;

[0034] The new model construction module is used to build new models. The new models include five basic models: MobilenetV2, MobilenetV3, Resnet50, Resnet152, and InceptionV3. Each of the five basic models is connected to a classifier, and the output of the basic model is used as the input of the corresponding classifier; the tail of the 6th residual block of the MobilenetV2 basic model is attached with the first multi-head attention module, and the tail of the 12th residual block is attached with the second multi-head attention module. The tail of the 6th residual block of the MobilenetV3 basic model is attached with the first multi-head attention module, and the tail of the 13th residual block is attached with the second multi-head attention module. The second multi-head attention module is attached to the tail of the third layer of the Resnet50 base model, the first multi-head attention module is attached to the tail of the fifth residual block of the third stage of the Resnet152 base model, and the second multi-head attention module is attached to the tail of the tenth residual block of the fourth stage; the classifier includes a first linear layer, a first dropout layer, a second linear layer, a second dropout layer, a feature fusion layer, a batch normalization layer, a Softmax layer and an output layer arranged in sequence; the first multi-head attention module is connected to the first linear layer through a residual, and the second multi-head attention module is connected to the second linear layer through a residual;

[0035] A new model training module is used to input unlabeled fundus color image samples into the four trained models respectively, and use the unlabeled fundus color image samples and the outputs of the four trained models as inputs of the new model to perform semi-supervised training on the new model;

[0036] The real-time classification module is used to obtain the color fundus image to be classified and input it into the trained new model, which then outputs the classification result.

[0037] A computer device includes a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the above method.

[0038] A computer-readable storage medium stores a computer program, which, when executed by a processor, causes the processor to perform the steps of the above method.

[0039] The beneficial effects of the present invention are as follows:

[0040] 1. In the present invention, the existing model is first trained with labeled samples to obtain a trained model, and then the unlabeled samples are input into the trained model, and the output of the trained model and the unlabeled samples are input into the new model together for semi-supervised training. Only a few high-quality and diverse sample images are required, which effectively solves the problem of lack of high-quality and diverse sample images of high myopia tractional lesions; the trained model and the new model work together to capture the difference characteristics between each level, which can greatly improve the classification accuracy of the new model; and combined with the multi-head attention mechanism, the classification accuracy of the new model is further improved, and ultimately the classification accuracy of high myopia tractional lesions is greatly improved, effectively solving the problem of low accuracy in the classification of high myopia tractional lesions in the prior art.

[0041] 2. In the present invention, the accuracy of the trained model is about 95.2%. After the trained model is combined with the new model and subjected to four rounds of data refresh in a semi-supervised training method, the highest accuracy of a single model reaches 97.7%, and the accuracy after integration can reach up to 98.4%; before the multi-head attention mechanism is added to the new model, the accuracy of each model of the new model is between 95.7% and 96.5%. After the multi-head attention mechanism is added, the performance is generally improved by 0.9% to 1.2%; this can further verify that this method has a high classification accuracy for tractional lesions of high myopia. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 It is a schematic flow diagram of the present invention;

[0043] Figure 2 is a flow chart of the present invention;

[0044] Figure 3 It is a flow chart of the new model in the present invention. DETAILED DESCRIPTION

[0045] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments.

[0046] Therefore, based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative work shall fall within the scope of protection of the present invention.

[0047] Example 1

[0048] This embodiment provides a method for classifying high myopia tractional lesions, which is used to classify fundus color images related to high myopia tractional lesions. Figure 1 、 2 As shown, it includes the following steps:

[0049] Step S1, obtaining a trained model;

[0050] The labeled fundus color image samples and their label data were input into four network models, namely MobilenetV3, Resnet50, Resnet152 and InceptionV3, and the four network models were trained respectively to obtain four trained models.

[0051] First, we construct four blank, untrained network models: MobilenetV3, Resnet50, Resnet152, and InceptionV3. These four network models all use network models from existing technologies, and their structures can refer to existing technologies. There is no need to spend creative labor to innovate or improve the model structure.

[0052] Then, obtain fundus color image samples. The fundus color image samples in this embodiment are obtained from two channels. One is from the public data set in the "PaddlePaddle Learning Competition: PALM Pathological Myopia Prediction", which contains 800 fundus color images and is re-labeled by doctors from the Zhongshan Eye Center of Sun Yat-sen University (referred to as "Zhongshan Eye Center") to form labeled data to ensure the accuracy and consistency of the data, which can be used as a training set for the model; the other is from the actual clinical data of Zhongshan Eye Center. Part of these actual clinical data are labeled by doctors from Zhongshan Eye Center and form labeled data, and the other part is not labeled.

[0053] When training the four network models MobilenetV3, Resnet50, Resnet152 and InceptionV3, labeled fundus color image samples and their label data were used to train the four network models separately to obtain four trained models.

[0054] Conventional training methods were used, with 100 epochs and a batch size of 64. The number of categories was set to 6, which determines the number of categories the model needs to distinguish. The initial learning rate was 3e-5, and after each epoch, the learning rate was adjusted using cosine annealing decay. The AdamW optimizer automatically adjusted the learning rate to achieve faster convergence during training. Furthermore, the loss function used the existing cross-entropy loss function (CrossEntropyLoss) to calculate the difference between the model's predicted results and the actual results.

[0055] Step S2, building a new model;

[0056] Create new models. These include five base models: MobilenetV2, MobilenetV3, Resnet50, Resnet152, and InceptionV3. These five base models are also existing network models. You can directly use existing, blank, and untrained MobilenetV2, MobilenetV3, Resnet50, Resnet152, and InceptionV3 network models. Connect a classifier to each of these five base models, and use the output of the base model as the input to the corresponding classifier.

[0057] like Figure 3 As shown, the first multi-head attention module is attached to the tail of the 6th residual block of the MobilenetV2 basic model, and the second multi-head attention module is attached to the tail of the 12th residual block. The first multi-head attention module is attached to the tail of the 6th residual block of the MobilenetV3 basic model, and the second multi-head attention module is attached to the tail of the 13th residual block. The first multi-head attention module is attached to the tail of the 3rd layer of the Resnet50 basic model, and the first multi-head attention module is attached to the tail of the 5th residual block of the third stage of the Resnet152 basic model, and the second multi-head attention module is attached to the tail of the 10th residual block of the fourth stage.

[0058] The classifier includes a first linear layer, a first dropout layer, a second linear layer, a second dropout layer, a feature fusion layer, a batch normalization layer, a Softmax layer and an output layer arranged in sequence; the first multi-head attention module is connected to the first linear layer through a residual, and the second multi-head attention module is connected to the second linear layer through a residual.

[0059] This embodiment designs a classifier to improve model performance and robustness. The first multi-head attention mechanism within the pre-trained model is implemented through residual connections. A first linear layer is then added. This linear layer transforms and extracts features, mapping the input features to a new feature space, helping the model better understand and utilize them. Next, the classifier adds a first dropout layer with a dropout rate set to 0.25. Dropout is a regularization technique whose primary purpose is to prevent model overfitting. During training, the first dropout layer randomly sets the outputs of some neurons to 0, making the model more robust and less dependent on specific neurons. A second multi-head attention mechanism within the pre-trained model is then implemented through residual connections. A second linear layer and a second dropout layer are then added, also with a dropout rate set to 0.25. The classifier then performs feature fusion, averaging the outputs of the first two multi-head attention mechanisms with the output of the second dropout layer. Feature fusion combines information from multiple features to obtain a more comprehensive and accurate feature representation. Finally, batch normalization, a softmax activation function, and an output layer are added.

[0060] Batch normalization is a technique used to accelerate the training of deep networks while also providing a certain degree of regularization to help prevent overfitting. It is expressed as:

[0061]

[0062] The Softmax function is a commonly used activation function that can map any real number to between (0,1), and the sum of all output values ​​is 1, so that the network can handle multi-classification problems. It is expressed as:

[0063]

[0064] in, Represents the input data in a batch, represents the mean of the same batch of data, represents the variance of the same batch of data, Represents a very small constant, used to ensure that the denominator is greater than 0; represents the i-th element of the input vector, Indicates the dimension of the input vector.

[0065] Step S3, new model training;

[0066] The unlabeled fundus color image samples are input into the four trained models respectively, and the unlabeled fundus color image samples and the outputs of the four trained models are used as the input of the new model, and the new model is semi-supervised trained.

[0067] The specific method of semi-supervised training is:

[0068] Step S3-1: inputting unlabeled fundus color image samples into four trained models respectively, and the four trained models respectively output preliminary prediction results;

[0069] Step S3-2: Compare the preliminary prediction results output by the four trained models with the preset prediction result threshold. If the preliminary prediction result is greater than the prediction result threshold (the prediction result threshold is 95%), a record label is added to the corresponding preliminary prediction result. If the preliminary prediction result is less than the prediction result threshold, the preliminary prediction result is discarded.

[0070] In step S3-3, the preliminary prediction results with record labels and their record labels are input into the new model, and the new model outputs the corresponding re-prediction results; if the accuracy of the re-prediction results is higher than the preset accuracy threshold (the accuracy threshold is 99%), the semi-supervised training ends; if the accuracy of the re-prediction results is lower than the accuracy threshold, it returns to step S3-1, and the new model will re-predict the corresponding preliminary prediction results until the number of iterations reaches the iteration threshold (the iteration threshold is 10 times), and the semi-supervised training ends.

[0071] In step S3-3, since the new model includes five basic models, each basic model will output a re-prediction result during training, and the five re-prediction results are processed using a voting ensemble method, that is, taking the mode, and taking the category with the highest proportion among the five re-prediction results.

[0072] During semi-supervised training, the number of training rounds is set to 100 and the batch size is set to 16; the initial learning rate is set to 1e-2, and the step learning rate scheduler is used to adjust the learning rate after each round of learning, and the AdamW optimizer is used to automatically adjust the learning rate; the loss function uses the cross-entropy loss function (CrossEntropyLoss) to calculate the difference between the model's predicted results and the actual results.

[0073] The step size of the step learning rate scheduler is set to the number of steps per epoch, and the decay coefficient is set to 0.5.

[0074] Step S4, real-time classification;

[0075] Obtain a color fundus image to be classified and input it into the trained new model. The new model outputs a classification result, that is, which specific category of tractional lesions the color fundus image to be classified belongs to.

[0076] Example 2

[0077] This embodiment provides a classification system for high myopia traction lesions, which includes:

[0078] The trained model acquisition module is used to input labeled fundus color image samples and their label data into four network models, namely MobilenetV3, Resnet50, Resnet152 and InceptionV3, and train the four network models respectively to obtain four trained models.

[0079] First, we construct four blank, untrained network models: MobilenetV3, Resnet50, Resnet152, and InceptionV3. These four network models all use network models from existing technologies, and their structures can refer to existing technologies. There is no need to spend creative labor to innovate or improve the model structure.

[0080] Then, obtain fundus color image samples. The fundus color image samples in this embodiment are obtained from two channels. One is from the public data set in the "PaddlePaddle Learning Competition: PALM Pathological Myopia Prediction", which contains 800 fundus color images and is re-labeled by doctors from the Zhongshan Eye Center of Sun Yat-sen University (referred to as "Zhongshan Eye Center") to form labeled data to ensure the accuracy and consistency of the data, which can be used as a training set for the model; the other is from the actual clinical data of Zhongshan Eye Center. Part of these actual clinical data are labeled by doctors from Zhongshan Eye Center and form labeled data, and the other part is not labeled.

[0081] When training the four network models MobilenetV3, Resnet50, Resnet152 and InceptionV3, labeled fundus color image samples and their label data were used to train the four network models separately to obtain four trained models.

[0082] Conventional training methods were used, with 100 epochs and a batch size of 64. The number of categories was set to 6, which determines the number of categories the model needs to distinguish. The initial learning rate was 3e-5, and after each epoch, the learning rate was adjusted using cosine annealing decay. The AdamW optimizer automatically adjusted the learning rate to achieve faster convergence during training. Furthermore, the loss function used the existing cross-entropy loss function (CrossEntropyLoss) to calculate the difference between the model's predicted results and the actual results.

[0083] The new model construction module is used to build new models. The new models include five basic models: MobilenetV2, MobilenetV3, Resnet50, Resnet152, and InceptionV3. Each of the five basic models is connected to a classifier, and the output of the basic model serves as the input of the corresponding classifier.

[0084] The first multi-head attention module is attached to the tail of the 6th residual block of the MobilenetV2 basic model, and the second multi-head attention module is attached to the tail of the 12th residual block. The first multi-head attention module is attached to the tail of the 6th residual block of the MobilenetV3 basic model, and the second multi-head attention module is attached to the tail of the 13th residual block. The first multi-head attention module is attached to the tail of the 3rd layer of the Resnet50 basic model. The first multi-head attention module is attached to the tail of the 5th residual block of the third stage of the Resnet152 basic model, and the second multi-head attention module is attached to the tail of the 10th residual block of the fourth stage.

[0085] The classifier includes a first linear layer, a first dropout layer, a second linear layer, a second dropout layer, a feature fusion layer, a batch normalization layer, a Softmax layer and an output layer arranged in sequence; the first multi-head attention module is connected to the first linear layer through a residual, and the second multi-head attention module is connected to the second linear layer through a residual.

[0086] This embodiment designs a classifier to improve model performance and robustness. The first multi-head attention mechanism within the pre-trained model is implemented through residual connections. A first linear layer is then added. This linear layer transforms and extracts features, mapping the input features to a new feature space, helping the model better understand and utilize them. Next, the classifier adds a first dropout layer with a dropout rate set to 0.25. Dropout is a regularization technique whose primary purpose is to prevent model overfitting. During training, the first dropout layer randomly sets the outputs of some neurons to 0, making the model more robust and less dependent on specific neurons. A second multi-head attention mechanism within the pre-trained model is then implemented through residual connections. A second linear layer and a second dropout layer are then added, also with a dropout rate set to 0.25. The classifier then performs feature fusion, averaging the outputs of the first two multi-head attention mechanisms with the output of the second dropout layer. Feature fusion combines information from multiple features to obtain a more comprehensive and accurate feature representation. Finally, batch normalization, a softmax activation function, and an output layer are added.

[0087] Batch normalization is a technique used to accelerate the training of deep networks while also providing a certain degree of regularization to help prevent overfitting. It is expressed as:

[0088]

[0089] The Softmax function is a commonly used activation function that can map any real number to between (0,1), and the sum of all output values ​​is 1, so that the network can handle multi-classification problems. It is expressed as:

[0090]

[0091] in, Represents the input data in a batch, represents the mean of the same batch of data, represents the variance of the same batch of data, Represents a very small constant, used to ensure that the denominator is greater than 0; represents the i-th element of the input vector, Indicates the dimension of the input vector.

[0092] The new model training module is used to input unlabeled fundus color photo image samples into four trained models respectively, and use the unlabeled fundus color photo image samples and the outputs of the four trained models as the input of the new model to perform semi-supervised training on the new model.

[0093] The specific method of semi-supervised training is:

[0094] Step S3-1: inputting unlabeled fundus color image samples into four trained models respectively, and the four trained models respectively output preliminary prediction results;

[0095] Step S3-2: Compare the preliminary prediction results output by the four trained models with the preset prediction result threshold. If the preliminary prediction result is greater than the prediction result threshold (the prediction result threshold is 95%), a record label is added to the corresponding preliminary prediction result. If the preliminary prediction result is less than the prediction result threshold, the preliminary prediction result is discarded.

[0096] In step S3-3, the preliminary prediction results with record labels and their record labels are input into the new model, and the new model outputs the corresponding re-prediction results; if the accuracy of the re-prediction results is higher than the preset accuracy threshold (the accuracy threshold is 99%), the semi-supervised training ends; if the accuracy of the re-prediction results is lower than the accuracy threshold, it returns to step S3-1, and the new model will re-predict the corresponding preliminary prediction results until the number of iterations reaches the iteration threshold (the iteration threshold is 10 times), and the semi-supervised training ends.

[0097] In step S3-3, since the new model includes five basic models, each basic model will output a re-prediction result during training, and the five re-prediction results are processed using a voting ensemble method, that is, taking the mode, and taking the category with the highest proportion among the five re-prediction results.

[0098] During semi-supervised training, the number of training rounds is set to 100 and the batch size is set to 16; the initial learning rate is set to 1e-2, and the step learning rate scheduler is used to adjust the learning rate after each round of learning, and the AdamW optimizer is used to automatically adjust the learning rate; the loss function uses the cross-entropy loss function (CrossEntropyLoss) to calculate the difference between the model's predicted results and the actual results.

[0099] The step size of the step learning rate scheduler is set to the number of steps per epoch, and the decay coefficient is set to 0.5.

[0100] The real-time classification module is used to obtain the fundus color image to be classified and input it into the trained new model. The new model outputs the classification result, that is, which specific category of tractional lesions the fundus color image to be classified belongs to.

[0101] Example 3

[0102] A computer device includes a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of a method for classifying high myopia tractional lesions.

[0103] The computer device may be a desktop computer, a notebook computer, a PDA, a cloud server, etc. The computer device may interact with the user via a keyboard, a mouse, a remote control, a touchpad, or a voice control device.

[0104] The memory includes at least one type of readable storage medium, including flash memory, hard disk, multimedia card, card-type memory (e.g., SD or D interface display memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory can be an internal storage unit of the computer device, such as the hard disk or memory of the computer device. In other embodiments, the memory can also be an external storage device of the computer device, such as a plug-in hard disk equipped with the computer device, a smart memory card (SMC), a secure digital (SD) card, a flash card, etc. Of course, the memory can also include both the internal storage unit of the computer device and its external storage device. In this embodiment, the memory is often used to store the operating system and various application software installed on the computer device, such as the program code of the high myopia traction lesion classification method. In addition, the memory can also be used to temporarily store various types of data that have been output or are about to be output.

[0105] In some embodiments, the processor may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip. The processor is typically used to control the overall operation of the computer device. In this embodiment, the processor is used to execute program code stored in the memory or process data, such as executing the program code for the method for classifying high myopia tractional lesions.

[0106] Example 4

[0107] A computer-readable storage medium stores a computer program. When the computer program is executed by a processor, the processor executes the steps of a method for classifying high myopia tractional lesions.

[0108] The computer-readable storage medium stores an interface display program, and the interface display program can be executed by at least one processor to enable the at least one processor to perform the steps of the above-mentioned method for classifying high myopia tractional lesions.

[0109] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment method can be implemented by means of software plus the necessary general hardware platform, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server or network device, etc.) to execute the high myopia traction lesion classification method described in the embodiment of the present application.

Claims

1. A method for classifying tractional lesions of high myopia, characterized in that: The following steps are involved: Step S1, obtaining a trained model; The labeled fundus color image samples and their label data were input into four network models, namely MobilenetV3, Resnet50, Resnet152 and InceptionV3, and the four network models were trained to obtain four trained models. Step S2, building a new model; Build a new model, which includes five basic models: MobilenetV2, MobilenetV3, Resnet50, Resnet152, and InceptionV3. Each of the five basic models is connected to a classifier, and the output of the basic model is used as the input of the corresponding classifier. The tail of the sixth residual block of the MobilenetV2 basic model is attached with the first multi-head attention module, and the tail of the 12th residual block is attached with the second multi-head attention module. The tail of the sixth residual block of the MobilenetV3 basic model is attached with the first multi-head attention module, and the tail of the 13th residual block is attached with the second multi-head attention module. Multi-head attention module, the first multi-head attention module is attached to the tail of the third layer of the Resnet50 base model, the first multi-head attention module is attached to the tail of the fifth residual block of the third stage of the Resnet152 base model, and the second multi-head attention module is attached to the tail of the tenth residual block of the fourth stage; the classifier includes a first linear layer, a first dropout layer, a second linear layer, a second dropout layer, a feature fusion layer, a batch normalization layer, a softmax layer and an output layer arranged in sequence; the first multi-head attention module is connected to the first linear layer through a residual, and the second multi-head attention module is connected to the second linear layer through a residual; Step S3, new model training; The unlabeled fundus color image samples are input into the four trained models respectively, and the unlabeled fundus color image samples and the output of the four trained models are used as the input of the new model to perform semi-supervised training on the new model; The specific method of semi-supervised training is: Step S3-1: inputting unlabeled fundus color image samples into four trained models respectively, and the four trained models respectively output preliminary prediction results; Step S3-2: Compare the preliminary prediction results output by the four trained models with the preset prediction result thresholds. If the preliminary prediction result is greater than the prediction result threshold, a record label is added to the corresponding preliminary prediction result. If the preliminary prediction result is less than the prediction result threshold, the preliminary prediction result is discarded. In step S3-3, the preliminary prediction results with the record labels added and their record labels are input into the new model, and the new model outputs the corresponding re-prediction results. If the accuracy of the re-prediction results is higher than the preset accuracy threshold, the semi-supervised training ends. If the accuracy of the re-prediction results is lower than the accuracy threshold, the process returns to step S3-1, and the new model re-predicts the corresponding preliminary prediction results until the number of iterations reaches the iteration threshold, at which point the semi-supervised training ends. Step S4, real-time classification; Obtain the fundus color image to be classified and input it into the trained new model, which then outputs the classification result.

2. The method for classifying tractional lesions of high myopia according to claim 1, wherein: In step S1, when four network models are trained to obtain four trained models, the training method is: The number of training rounds is 100 and the batch size is 64; the initial learning rate is 3e-5, and the cosine annealing decay is used to adjust the learning rate after each round of learning, and the AdamW optimizer is used to automatically adjust the learning rate.

3. The method for classifying tractional lesions of high myopia according to claim 1, wherein: In the classifier of step S2, the dropout rates of the first dropout layer and the second dropout layer are both set to 0.

25.

4. The method for classifying tractional lesions of high myopia according to claim 3, wherein: In step S2, the batch normalization layer is expressed as: The Softmax layer is represented as: in, Represents the input data in a batch, represents the mean of the same batch of data, represents the variance of the same batch of data, represents a constant; represents the i-th element of the input vector, Indicates the dimension of the input vector.

5. The method for classifying tractional lesions of high myopia according to claim 1, wherein: In step S3, during semi-supervised training, the number of training rounds is set to 100 and the batch size is set to 16; the initial learning rate is set to 1e-2, and the step learning rate scheduler is used to adjust the learning rate after each round of learning, and the AdamW optimizer is used to automatically adjust the learning rate; The step size of the step learning rate scheduler is set to the number of steps per epoch, and the decay coefficient is set to 0.

5.

6. A classification system for tractional lesions in high myopia, characterized by: include: The trained model acquisition module is used to input labeled fundus color image samples and their label data into four network models, namely MobilenetV3, Resnet50, Resnet152, and InceptionV3, and train the four network models to obtain four trained models; The new model construction module is used to build new models. The new models include five basic models: MobilenetV2, MobilenetV3, Resnet50, Resnet152, and InceptionV3. Each of the five basic models is connected to a classifier, and the output of the basic model is used as the input of the corresponding classifier; the tail of the 6th residual block of the MobilenetV2 basic model is attached with the first multi-head attention module, and the tail of the 12th residual block is attached with the second multi-head attention module. The tail of the 6th residual block of the MobilenetV3 basic model is attached with the first multi-head attention module, and the tail of the 13th residual block is attached with the second multi-head attention module. The second multi-head attention module is attached to the tail of the third layer of the Resnet50 base model, the first multi-head attention module is attached to the tail of the fifth residual block of the third stage of the Resnet152 base model, and the second multi-head attention module is attached to the tail of the tenth residual block of the fourth stage; the classifier includes a first linear layer, a first dropout layer, a second linear layer, a second dropout layer, a feature fusion layer, a batch normalization layer, a Softmax layer and an output layer arranged in sequence; the first multi-head attention module is connected to the first linear layer through a residual, and the second multi-head attention module is connected to the second linear layer through a residual; A new model training module is used to input unlabeled fundus color image samples into the four trained models respectively, and use the unlabeled fundus color image samples and the outputs of the four trained models as inputs of the new model to perform semi-supervised training on the new model; The specific method of semi-supervised training is: Step S3-1: inputting unlabeled fundus color image samples into four trained models respectively, and the four trained models respectively output preliminary prediction results; Step S3-2: Compare the preliminary prediction results output by the four trained models with the preset prediction result thresholds. If the preliminary prediction result is greater than the prediction result threshold, a record label is added to the corresponding preliminary prediction result. If the preliminary prediction result is less than the prediction result threshold, the preliminary prediction result is discarded. In step S3-3, the preliminary prediction results with the record labels added and their record labels are input into the new model, and the new model outputs the corresponding re-prediction results. If the accuracy of the re-prediction results is higher than the preset accuracy threshold, the semi-supervised training ends. If the accuracy of the re-prediction results is lower than the accuracy threshold, the process returns to step S3-1, and the new model re-predicts the corresponding preliminary prediction results until the number of iterations reaches the iteration threshold, at which point the semi-supervised training ends. The real-time classification module is used to obtain the color fundus image to be classified and input it into the trained new model, which then outputs the classification result.

7. A computer device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that: A computer program is stored, and when the computer program is executed by a processor, the processor is caused to perform the steps of the method according to any one of claims 1 to 5.

Images