A corneal ulcer classification method based on multi-scale information fusion network

Abstract

The present invention discloses a corneal ulcer classification method based on a multiscale information fusion network, which belongs to the field of slit lamp image classification. The method comprises preprocessing images in an image data set, constructing a multiscale information fusion network model, adding a multiscale information fuser in the network model, introducing a label smoothing strategy, training and testing the multiscale information fusion network model, and finally using the multiscale information fusion network model to classify a two-dimensional slit lamp fluorescent staining target image. The method combines image preprocessing, construction and training of a deep neural network model, and testing, so that subsequent research on corneal ulcer diseases is greatly helped, a better judgment can be made on the two-dimensional slit lamp fluorescent staining image, the classification and detection of the two-dimensional slit lamp fluorescent staining image is facilitated, the screening efficiency of the two-dimensional slit lamp fluorescent staining image is improved, and the accuracy of classifying different types of corneal ulcers is improved.

Description

Technical Field

[0001] The invention belongs to the field of slit lamp image classification, and in particular, relates to a corneal ulcer classification method based on a multi-scale information fusion network. Background Art

[0002] Corneal ulcer is a serious blinding eye disease, which is particularly prominent in developing countries. It is one of the main causes of corneal blindness. It is an inflammatory or more serious infectious corneal disease, including damage to the epithelium and involvement of the corneal stroma, which can cause great pain to patients. Failure to treat it in time or improper treatment may cause irreversible damage to the eye or even blindness. Standardized screening, timely diagnosis and early treatment are effective ways to reduce the blindness rate caused by corneal ulcers.

[0003] Due to its fluorescent properties and high visibility at low concentrations, fluorescein has become the most widely used method in optometry and ophthalmic diagnosis. Fluorescent staining is often used to observe the integrity of the ocular surface, especially the integrity of the cornea. Therefore, fluorescent staining technology has become a common tool to assist ophthalmologists in diagnosing corneal ulcers, providing great convenience for the diagnosis and treatment of corneal ulcers. The use of slit lamp microscope combined with fluorescent staining technology to perform ocular surface examinations on high-risk populations and the use of fluorescent staining images to identify corneal ulcers play an important role in formulating treatment plans.

[0004] The current prevention and treatment model for corneal ulcers is that ophthalmologists use the general pattern and type grade (TG) standard of ulcers. The general pattern of ulcers is based on the shape and distribution characteristics of corneal ulcers, which can be divided into three categories: punctate corneal ulcers, punctate-lamellar mixed corneal ulcers, and lamellar corneal ulcers. The TG grading method divides corneal ulcers into 5 categories (0-4 types) according to their specific types.

[0005] However, accurate classification of corneal ulcers is very challenging, mainly because of differences in subjective experience and professional knowledge among ophthalmologists. Doctors have certain differences in identifying corneal ulcers based on fluorescent staining images, which can easily lead to some patients with severe corneal ulcers not receiving timely treatment. Some patients may relapse after treatment, and the differences in doctors' choice of treatment time and treatment methods for recurrent corneal ulcers may lead to treatment uncertainty. At the same time, corneal ulcers have complex pathological characteristics and are very susceptible to noise interference. Different types of corneal ulcers are similar in pathological morphology and distribution, resulting in a high degree of similarity in fluorescent staining images of different types of corneal ulcers, which ultimately greatly reduces the accuracy of classification of different types of corneal ulcers. Summary of the invention

[0006] Problem to be solved

[0007] In view of the problem that there are certain differences in the existing identification of corneal ulcers by doctors based on fluorescent staining images, and that different types of corneal ulcers have similarities in pathological morphology and distribution, resulting in very high similarity of fluorescent staining images, which ultimately greatly reduces the accuracy of corneal ulcer classification. The present invention provides a corneal ulcer classification method based on a multi-scale information fusion network.

[0008] Technical Solution

[0009] To solve the above problems, the present invention adopts the following technical solutions.

[0010] A corneal ulcer classification method based on a multi-scale information fusion network adopts the following steps:

[0011] Step 1: Obtain a two-dimensional slit lamp fluorescence staining image dataset and preprocess the image;

[0012] Step 2: Construct a multi-scale information fusion network model;

[0013] Step 3: Train and test the multi-scale information fusion network model;

[0014] Step 4: Use the multi-scale information fusion network model to classify the two-dimensional slit lamp fluorescence staining target image.

[0015] Preferably, the image preprocessing in step 1 is to downsample all images in the data set using a bilinear interpolation method, and then perform normalization processing, while performing an online data augmentation operation on the image data.

[0016] Furthermore, the online data augmentation operation includes randomly rotating the image by 30 degrees, randomly flipping the image horizontally, and randomly flipping the image vertically.

[0017] Preferably, the multi-scale information fusion network model constructed in step 2 is to first establish a backbone network, then design and add a multi-scale information fuser based on the backbone network, and finally introduce a label smoothing strategy to optimize the network model.

[0018] Furthermore, the multi-scale information fusion network model is based on a two-dimensional convolutional neural network model as the backbone network. The backbone network is provided with multiple two-dimensional convolution kernels, four densely connected layers and three conversion layers. The size of the first two-dimensional convolution kernel is 7*7 and the step size is 2. The sizes of the other two-dimensional convolution kernels are 3*3 and 1*1 respectively, and the step size of the convolution kernel is 1 or 2.

[0019] Furthermore, the multi-scale information fuser is composed of two adaptive average pooling layers, two fully connected layers and an adder. The two adaptive average pooling layers are respectively connected to the second conversion layer and the fourth densely connected layer in the backbone network. The output features of the second conversion layer and the fourth densely connected layer are respectively compressed in spatial dimensions through two adaptive average pooling layers. The results are then respectively passed through two fully connected layers and converted into a predicted probability distribution with a result dimension of N*1, where N represents the number of categories of corneal ulcers. The two predicted probability distributions are finally added and fused through the adder to obtain a fused N*1 predicted probability distribution.

[0020] Furthermore, the label smoothing strategy is implemented by adding noise to the label. The label smoothing strategy formula is as follows:

[0021]

[0022] Where t and t' represent the one-hot labels of label smoothing, ε is a random number between 0.1 and 0.2, I is a matrix with the same dimension as t, the element value of I is 1, and N is the total number of categories.

[0023] Furthermore, the label smoothing strategy adopts the cross entropy loss function, and the formula is as follows:

[0024]

[0025] Where m is the number of samples in each mini-batch, K is the total number of input image category labels, and x i represents the input corneal ulcer image, t i is the category label of the input image, f(·) is the indicator function, if t i If k is equal to k, it is 1, otherwise it is 0.

[0026] Preferably, in step 3, the multi-scale information fusion network model is trained by minimizing the cost function through the optimizer Adam, the basic learning rate and weight decay are both set to 0.0001, the batch size is set to 16, and the number of iterations is set to 50.

[0027] Preferably, in step 3, three classification evaluation indicators need to be set for testing the multi-scale information fusion network model, namely, accuracy, weighted average F1 score and weighted average area under the curve, wherein the definition formulas of accuracy and weighted average F1 score are as follows:

[0028]

[0029]

[0030] Among them, Accuracy is the accuracy, W_F1-score is the weighted average F1 score, TP, FP, TN and FN are true positive, false positive, true negative and false negative respectively, W_P and W_R represent the weighted average precision and weighted average recall respectively.

[0031] A corneal ulcer classification method based on a multiscale information fusion network is proposed. The images in the image data set are preprocessed to construct a multiscale information fusion network model. A multiscale information fuser is added to the network model, a label smoothing strategy is introduced, and the multiscale information fusion network model is trained and tested. Finally, the multiscale information fusion network model is used to classify the two-dimensional slit lamp fluorescence staining target image. Combined with image preprocessing, the construction and training of a deep neural network model, and testing, the method is of great help to subsequent research on corneal ulcer diseases, such as lesion area segmentation and automatic diagnosis research. It can make better judgments on two-dimensional slit lamp fluorescence staining images, help the classification and detection of two-dimensional slit lamp fluorescence staining images, improve the screening efficiency of two-dimensional slit lamp fluorescence staining images, and improve the accuracy of classification of different types of corneal ulcers.

[0032] Beneficial Effects

[0033] Compared with the prior art, the present invention has the following beneficial effects:

[0034] (1) The present invention preprocesses the collected two-dimensional slit lamp fluorescence staining image data set, performs a bilinear interpolation downsampling method and normalization processing on it, and performs online data amplification. The downsampling method can prevent processing memory overflow and reduce the computational cost of the learning task. The normalization processing can improve the difference. The online data amplification can improve the diversity and anti-interference of the data.

[0035] (2) The network model constructed by the present invention is an improvement on the convolutional neural network. A multi-scale information fusion device is designed on the basis of the convolutional neural network, which can combine shallow local information with deep global information, avoid the loss of low-resolution feature information related to the category in the shallow layer, make full use of the shallow information, and fuse the deep and shallow prediction information to enhance the robustness of the prediction results;

[0036] (3) The present invention introduces a label smoothing strategy in the optimization process of the network model. By adding noise, the category weight corresponding to the true label can be reduced and the risk of overfitting can be reduced. The cross entropy loss function is adopted based on the label smoothing strategy, which can alleviate the problems of inter-class similarity and intra-class diversity. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the technical solutions in the embodiments or exemplary embodiments of the present application, the drawings required for use in the embodiments or exemplary descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application and should not be regarded as limiting the scope. For ordinary technicians in this field, other drawings can be obtained according to the drawings without paying creative work.

[0038] Figure 1 It is a schematic diagram of the steps of the present invention;

[0039] Figure 2 It is a schematic diagram of the process of the present invention;

[0040] Figure 3 It is a schematic diagram of the conversion layer structure in the network model of the present invention;

[0041] Figure 4 It is a schematic diagram of the dense module structure in the network model of the present invention;

[0042] Figure 5 A schematic diagram of the classification results of corneal ulcers according to the general model of the present invention;

[0043] Figure 6 It is a schematic diagram of the classification results of corneal ulcers in the specific mode of the present invention. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Generally, the components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations.

[0045] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for protection, but merely represents selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0046] Example 1

[0047] like Figure 1 As shown in the figure, a corneal ulcer classification method based on a multi-scale information fusion network is shown in the figure. The specific process is as follows:

[0048] A two-dimensional slit lamp fluorescence staining image dataset was obtained and preprocessed. All images in the dataset were downsampled using the bilinear interpolation method and then normalized. At the same time, the image data was augmented online, including random rotation of the image by 30 degrees, random horizontal flipping, and random vertical flipping.

[0049] To construct a multi-scale information fusion network model, we first set up the backbone network, then designed and added a multi-scale information fuser based on the backbone network, and finally introduced a label smoothing strategy to optimize the network model.

[0050] The multi-scale information fusion network model uses a two-dimensional convolutional neural network model as the backbone network. The backbone network is provided with multiple two-dimensional convolution kernels, four densely connected layers and three conversion layers. The size of the first two-dimensional convolution kernel is 7*7 and the step size is 2. The sizes of the other two-dimensional convolution kernels are 3*3 and 1*1 respectively, and the step size of the convolution kernel is 1 or 2.

[0051] The multi-scale information fusion device is composed of two adaptive average pooling layers, two fully connected layers and an adder. The two adaptive average pooling layers are connected to the second conversion layer and the fourth densely connected layer in the backbone network respectively. The output features of the second conversion layer and the fourth densely connected layer are compressed in spatial dimensions through two adaptive average pooling layers respectively. The obtained results are then respectively passed through two fully connected layers and converted into a prediction probability distribution with a result dimension of N*1, where N represents the number of categories of corneal ulcers. The two prediction probability distributions are finally added and fused through the adder to obtain the fused N*1 prediction probability distribution.

[0052] The label smoothing strategy is achieved by adding noise to the label. The label smoothing strategy formula is as follows:

[0053]

[0054] Where t and t' represent the one-hot labels of label smoothing, ε is a random number between 0.1 and 0.2, I is a matrix with the same dimension as t, the element value of I is 1, and N is the total number of categories;

[0055] The label smoothing strategy uses the cross entropy loss function, the formula is as follows:

[0056]

[0057] Where m is the number of samples in each mini-batch, K is the total number of input image category labels, and x i represents the input corneal ulcer image, t i is the category label of the input image, f(·) is the indicator function, if t iIf k is equal to k, it is 1, otherwise it is 0.

[0058] The multi-scale information fusion network model is trained and tested. The training is to minimize the cost function through the optimizer Adam. The basic learning rate and weight decay are both set to 0.0001, the batch size is set to 16, and the number of iterations is set to 50. The test requires setting three classification evaluation indicators, namely accuracy, weighted average F1 score and weighted average area under the curve. The definition formulas of accuracy and weighted average F1 score are as follows:

[0059]

[0060]

[0061] Among them, Accuracy is the accuracy, W_F1-score is the weighted average F1 score, TP, FP, TN and FN are true positive, false positive, true negative and false negative respectively, W_P and W_R represent the weighted average precision and weighted average recall respectively.

[0062] Finally, a multi-scale information fusion network model is used to classify the two-dimensional slit lamp fluorescence staining target images.

[0063] From the above description, it can be seen that in this example, by preprocessing the images in the image data set, building a multiscale information fusion network model, adding a multiscale information fuser to the network model, introducing a label smoothing strategy, training and testing the multiscale information fusion network model, and finally using the multiscale information fusion network model to classify the two-dimensional slit lamp fluorescence staining target image, it can make a better judgment on the two-dimensional slit lamp fluorescence staining image, which is helpful for the classification and detection of two-dimensional retinal fundus color images, and improves the screening efficiency of two-dimensional retinal fundus color images. Combined with image preprocessing, the construction and training of the deep neural network model, and the testing, it will be of great help to the subsequent research on corneal ulcer diseases, such as lesion area segmentation and automatic diagnosis research.

[0064] Example 2

[0065] Firstly, a two-dimensional slit lamp fluorescence staining image dataset was obtained and the images were preprocessed. The original two-dimensional slit lamp fluorescence staining images in the image dataset were mostly 2592×1728 in size. The general patterns of corneal ulcers can be divided into punctate corneal ulcers, punctate and sheet-like mixed corneal ulcers and sheet-like corneal ulcers. The specific patterns of corneal ulcers can be divided into types 0 to 4. In order to reduce the computational cost of the learning task, all two-dimensional slit lamp images were downsampled using bilinear interpolation to make the image size 320×320, and then normalized to improve the difference. In order to prevent the model from overfitting and enhance the generalization ability of the model, online data augmentation operations were performed on the image data, including random rotation of the image by 30 degrees, random horizontal flipping and random vertical flipping.

[0066] To construct a multi-scale information fusion network model, a two-dimensional convolutional neural network model (DenseNet121) is needed as the backbone network. Aiming at the complex pathological characteristics of different corneal ulcers, a multi-scale information fuser is designed and added on the basis of the two-dimensional convolutional neural network model (DenseNet121). Taking into account the similarities in pathological morphology and distribution of different types of corneal ulcers, a label smoothing strategy is introduced in the network optimization process.

[0067] The original two-dimensional convolutional neural network model (DenseNet121) is composed of conversion layers and densely connected layers, including two-dimensional convolutional layers, two-dimensional maximum pooling layers, two-dimensional batch normalization layers, average pooling layers, global average pooling layers, fully connected layers and softmax output layers. Among them, the densely connected layers are composed of dense modules, and the four densely connected layers are composed of 6, 12, 24 and 16 dense modules respectively.

[0068] The backbone network of the multi-scale information fusion network model is equipped with multiple two-dimensional convolution kernels, four densely connected layers and three conversion layers. The size of the first two-dimensional convolution kernel is 7*7 and the step size is 2. The sizes of the other two-dimensional convolution kernels are 3*3 and 1*1 respectively. The step size of the convolution kernel is 1 or 2. The kernel size of the maximum pooling layer and the average pooling layer is 3*3 and the step size is 2. The purpose is to not merge the depth information prematurely, but also to reduce the number of network parameters and enhance the robustness of the network.

[0069] The multi-scale information fuser is composed of two adaptive average pooling layers, two fully connected layers and an adder. The two adaptive average pooling layers are connected to the second conversion layer and the fourth densely connected layer in the backbone network respectively. The output features of the second conversion layer and the fourth densely connected layer are compressed in spatial dimensions through two adaptive average pooling layers respectively. The resulting dimensions are 256*1*1 and 1024*1*1 respectively, indicating the numerical distribution of the corresponding layer 256 and 1024 feature maps. The calculation results are then converted into a prediction probability distribution with a result dimension of N*1 through two fully connected layers, where N represents the number of categories of corneal ulcers. The two prediction probability distributions are finally added and fused through the adder to obtain the fused N*1 prediction probability distribution.

[0070] The multi-scale information fuser can combine shallow local information with deep global information. On the one hand, it makes full use of shallow information and avoids the loss of low-resolution feature information related to categories in the shallow layer. On the other hand, the combination of shallow edge information and deep semantic information can increase the robustness of the prediction results.

[0071] The label smoothing strategy is achieved by adding noise to the label. Assume that D∈(x i ,y i )(i=1,2,…,M) is a classification data set with M samples, where x i and i Represent the input image and the corresponding category label respectively. A standard multi-classification problem is to predict the input image x i The probability of belonging to category k (y i =k), where category k is encoded as a vector t = (0,0,…,0,1,0,…,0) through a one-hot label, where the value of the kth position is 1 and the rest are all 0. This form of label encoding encourages the model to learn in the direction with the largest difference between the correct label and the wrong label, which means that only the loss of the correct label position is calculated during the optimization of the model. However, when the inter-class similarity and intra-class difference are relatively large, it may cause the network to overfit.

[0072] In order to solve the above problems, the present invention introduces label smoothing, which is a regularization strategy that mainly reduces the weight of the true label category by adding noise and slightly increases the penalty of the wrong label category in model training. The label smoothing strategy formula is as follows:

[0073]

[0074] Where t and t' represent the one-hot labels of label smoothing, ε is a random number between 0.1 and 0.2, I is a matrix with the same dimension as t, the element value of I is 1, and N is the total number of categories;

[0075] The label smoothing strategy uses the cross entropy loss function, the formula is as follows:

[0076]

[0077] Where m is the number of samples in each mini-batch, K is the total number of input image category labels, and x i represents the input corneal ulcer image, t i and k represent the category labels of the input images, respectively, and f(·) is the indicator function. If t i If k is equal to k, it is 1, otherwise it is 0.

[0078] 712 two-dimensional slit lamp fluorescence staining images were used as a data set, and 5-fold cross validation was used on the entire data set to evaluate the performance of the present invention. In order to reduce the computational cost of the learning task, all two-dimensional slit lamp fluorescence staining images were downsampled to 320×320 using a bilinear interpolation method and then normalized to improve the distinction.

[0079] In order to prevent overfitting of the model and enhance the generalization ability of the model, the data is augmented online during the training process to increase the diversity of the data, including random rotation of 30 degrees, random horizontal flipping, and random vertical flipping.

[0080] The model was trained and tested based on the Pytorch integrated environment and the NVIDIA GTX Titan X GPU with 12GB storage space. The model was trained by minimizing the loss function through the back propagation algorithm, and the optimizer Adam was used to minimize the cost function. The basic learning rate and weight decay were both set to 0.0001. The batch size was set to 16 and the number of iterations was set to 50.

[0081] In order to quantitatively evaluate the performance of the present invention, the test needs to set three classification evaluation indicators, namely, accuracy, weighted average F1 score and weighted average area under the curve, where the definition formulas of accuracy and weighted average F1 score are as follows:

[0082]

[0083]

[0084] Among them, Accuracy is the accuracy, W_F1-score is the weighted average F1 score, TP, FP, TN and FN are true positive, false positive, true negative and false negative respectively, W_P and W_R represent the weighted average precision and weighted average recall respectively.

[0085] In order to demonstrate the effectiveness of the multi-scale fuser and label smoothing strategy, a series of ablation experiments were conducted. The experimental results are shown in Figure 5 and Figure 6 As shown:

[0086] “Backbone network” represents the original 2D convolutional neural network model (DenseNet121);

[0087] “Backbone network + multi-scale information fuser” means adding a multi-scale information fuser to the original two-dimensional convolutional neural network model (DenseNet121);

[0088] “Backbone network + label smoothing” means using the label smoothing strategy in the original 2D convolutional neural network model (DenseNet121);

[0089] “LmNet” indicates the method published on July 3, 2021 in the journal “IEEE ACCESS” with the journal number (Digital Object Identifier, DOI) 10.1109 / ACCESS.2021.3093308;

[0090] “Proposed method” refers to the method proposed in the present invention, namely “backbone network + multi-scale information fuser + label smoothing”.

[0091] The general pattern of corneal ulcers is classified by Figure 5 It can be seen that the classification accuracy of the original two-dimensional convolutional neural network model (DenseNet121) is 84.39%, the classification accuracy of LmNet is 85.52%, and the classification accuracy (ACC) of the improved method, that is, the method of the present invention, can reach 87.07%. The weighted average F1 score (W_F1) and the weighted average area under the curve (W_AUC) of the present invention are 86.82% and 92.20%, respectively, which are 3.05% and 1.63% higher than the original DenseNet121.

[0092] from Figure 5 It can be seen that the multi-scale information fusion device and label smoothing strategy designed in the present invention can effectively improve the classification accuracy of the original two-dimensional convolutional neural network model (DenseNet121), and the classification accuracy is higher than that of LmNet.

[0093] For specific patterns of corneal ulcers, the Figure 6 It can be seen that the classification accuracy of the original two-dimensional convolutional neural network model (DenseNet121) is 81.45%, and the classification accuracy of LmNet is 82.42%. The improved classification accuracy (ACC) of the present invention can reach 83.84%, which is 2.93% and 1.72% higher than that of the original two-dimensional convolutional neural network model (DenseNet121) and LmNet, respectively. The weighted average F1 score (W_F1) and weighted average area under the curve (W_AUC) of the present invention are 80.52% and 91.11%, respectively, which are 4.19% and 0.62% higher than that of the original DenseNet121.

[0094] from Figure 6 It can be seen that the multi-scale information fuser and label smoothing strategy designed in the present invention can effectively improve the classification accuracy of the original two-dimensional convolutional neural network model (DenseNet121).

[0095] The present invention proposes a multi-scale information fusion device and a referenced label smoothing strategy to ensure the accuracy of corneal ulcer classification and identification.

[0096] So far, a corneal ulcer classification method for slit lamp fluorescence staining images has been implemented and verified. Its performance in the experiment is better than the original two-dimensional convolutional neural network model (DenseNet121). The present invention can make better judgments on two-dimensional slit lamp fluorescence staining images. On the other hand, the multi-scale information fusion device designed in the present invention is not complicated and can be embedded in any other convolutional neural network, making the feature extraction ability of the network model stronger, thereby improving the overall performance of the network model, which is helpful for the classification and detection of two-dimensional slit lamp fluorescence staining images, and greatly improving the screening efficiency of two-dimensional slit lamp fluorescence staining images. The present invention combines image preprocessing, the construction and training of deep neural network models, and testing, which is of great help to the subsequent research on corneal ulcer diseases, such as lesion area segmentation and automatic diagnosis research.

[0097] The above-mentioned embodiments only express the preferred implementation modes of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present invention. It should be pointed out that, for those skilled in the art, several modifications, improvements and substitutions can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention.

Claims

1. A corneal ulcer classification method based on multi-scale information fusion network, It is characterized in that Use the following steps: Step 1: Obtain a two-dimensional slit lamp fluorescence staining image dataset and preprocess the image; Step 2: Construct a multi-scale information fusion network model; Step 3: Train and test the multi-scale information fusion network model; Step 4: Use a multi-scale information fusion network model to classify the two-dimensional slit lamp fluorescence staining target image; The multi-scale information fusion network model constructed in step 2 is to first establish a backbone network, then design and add a multi-scale information fuser based on the backbone network, and finally introduce a label smoothing strategy to optimize the network model; The multi-scale information fusion network model is based on a two-dimensional convolutional neural network model as the backbone network. The backbone network is provided with multiple two-dimensional convolution kernels, four densely connected layers and three conversion layers. The size of the first two-dimensional convolution kernel is 7*7 and the step size is 2. The sizes of the other two-dimensional convolution kernels are 3*3 and 1*1 respectively, and the step size of the convolution kernel is one of 1 and 2.

2. A corneal ulcer classification method based on a multi-scale information fusion network according to claim 1, Features: The image preprocessing in step 1 is to downsample all images in the data set using a bilinear interpolation method, and then perform normalization processing, while performing online data augmentation operations on the image data.

3. A corneal ulcer classification method based on a multi-scale information fusion network according to claim 2, Features: The online data augmentation operation includes randomly rotating the image by 30 degrees, randomly flipping the image horizontally, and randomly flipping the image vertically.

4. The corneal ulcer classification method based on a multi-scale information fusion network according to claim 1, Features: The multi-scale information fuser is composed of two adaptive average pooling layers, two fully connected layers and an adder. The two adaptive average pooling layers are respectively connected to the second conversion layer and the fourth densely connected layer in the backbone network. The output features of the second conversion layer and the fourth densely connected layer are respectively compressed in spatial dimensions through two adaptive average pooling layers. The obtained results are then respectively passed through two fully connected layers to be converted into a prediction probability distribution with a result dimension of N*1, wherein N represents the number of categories of corneal ulcers. The two prediction probability distributions are finally added and fused through the adder to obtain a fused N*1 prediction probability distribution.

5. A corneal ulcer classification method based on a multi-scale information fusion network according to claim 4, Features: The label smoothing strategy is achieved by adding noise to the label. The label smoothing strategy formula is as follows: Where t and t' represent the one-hot labels of label smoothing, ε is a random number between 0.1 and 0.2, I is a matrix with the same dimension as t, the element value of I is 1, and N is the total number of categories.

6. A corneal ulcer classification method based on a multi-scale information fusion network according to claim 5, Features: The label smoothing strategy adopts the cross entropy loss function, and the formula is as follows: Where m is the number of samples in each mini-batch, K is the total number of input image category labels, and x i represents the input corneal ulcer image, t i is the category label of the input image, f(·) is the indicator function, if t i If k is equal to k, it is 1, otherwise it is 0.

7. A corneal ulcer classification method based on a multi-scale information fusion network according to claim 1, Features: In step 3, the multi-scale information fusion network model is trained by minimizing the cost function through the optimizer, the basic learning rate and weight decay are both set to 0.0001, the batch size is set to 16, and the number of iterations is set to 50.

8. The corneal ulcer classification method based on multi-scale information fusion network according to claim 1, Features: In step 3, three classification evaluation indicators need to be set for testing the multi-scale information fusion network model, namely, accuracy, weighted average F1 score and weighted average area under the curve, where the definition formulas of accuracy and weighted average F1 score are as follows: Among them, Accuracy is the accuracy, W_F1-score is the weighted average F1 score, TP, FP, TN and FN are true positive, false positive, true negative and false negative respectively, W_P and W_R represent the weighted average precision and weighted average recall respectively.

Images