A corneal ulcer classification and identification method based on a Mobile ViT network

Abstract

The application discloses a corneal ulcer classification and identification method based on a Mobile ViT network, and belongs to the technical field of corneal ulcer classification and identification, and comprises the following steps: S1, sample pretreatment; S2, adding an attention mechanism; S3, feature splicing; S4, network building; S5, network training; and S6, classification and identification.The application adds an SE attention module in the original Mobile ViT network, enhances the ability of the network to extract important features, and suppresses unnecessary features; local feature maps are extracted from the original Mobile ViT network, and the local feature maps are fused with global feature maps to extract more effective features, and meanwhile, information exchange between various features is increased; various comparative experiments prove the effectiveness of the improvement, and successfully increase the accuracy from 88.7% to 91.5%.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of corneal ulcer classification, and in particular to a corneal ulcer classification and identification method based on Mobile ViT. BACKGROUND

[0002] Corneal ulcer, also known as keratitis, is a common eye disease that can cause irreversible damage to the eye if not promptly detected and treated. This condition is often caused by bacterial, viral, fungal or parasitic infections, eye trauma, dry eye syndrome, keratitis, immune system diseases, improper use of contact lenses or eye drops, and corneal cell defects. According to statistics released by the National Health Commission, the number of new cases of corneal ulcer in China has exceeded 100,000, which is one of the main causes of blindness. Timely and effective detection of corneal ulcers is crucial for proper treatment and prevention of vision impairment. The development of new technologies such as fluorescent staining technology has completely changed the diagnosis and treatment of corneal ulcers.

[0003] Fluorescent staining technology is a widely used diagnostic method for assessing ocular surface integrity, particularly corneal ulcers. This technology involves the use of fluorescein dye to stain the cornea and its defects. After staining, the area and type of ulcer can be accurately determined and assessed, allowing for appropriate treatment. The use of a slit lamp microscope in conjunction with fluorescent staining technology allows researchers to more accurately diagnose and analyze corneal ulcers while reducing patient discomfort and pain during testing.

[0004] The rapid development of neural networks has led to an increasing number of network models being applied in various fields of medicine. Through understanding of relevant literature and diagnostic and treatment techniques, we found that traditional diagnostic and treatment methods not only slow down the diagnosis speed, but also cause pain to patients during the diagnosis process. Combining deep learning methods to assist in diagnosis and treatment can not only speed up the diagnosis, but also alleviate the pain of patients. In this invention, we combine deep learning methods to improve a new neural network model. By extracting the features of the ulcer area on the corneal ulcer image, we use these features to learn a new network model to automatically identify the type of corneal ulcer. After our long-term research and repeated experiments, the modified model has been recognized by us and has achieved good results. Our model not only has high accuracy, but also is very accurate from the final visual results.

[0005] In summary, with the increasing use of electronic devices in our daily life and work, the number of people suffering from corneal ulcers is also increasing year by year. Therefore, there is an urgent need to develop new and effective diagnostic methods. This invention will combine deep learning and fluorescent staining technology to improve a new method for diagnosing corneal ulcers to improve the speed and accuracy of diagnosis. SUMMARY

[0006] The technical problems to be solved by the present application are: how to quickly and accurately identify the type of corneal ulcer, reduce the pain of patients, quickly and accurately identify, realize automatic classification and identification by machine, reduce labor costs, and provide a corneal ulcer classification and identification method based on a Mobile ViT network to improve the speed and accuracy of identification.

[0007] The present application solves the above technical problems by the following technical solutions, and comprises the following steps:

[0008] S1: sample pretreatment

[0009] The image samples of the corneal ulcer data set are pretreated;

[0010] S2: increase attention mechanism

[0011] An SE attention module is fused in the MV2 module of the original Mobile ViT network to obtain an improved MV2 module, i.e., the MV2-SE module;

[0012] S3: feature splicing

[0013] The local representation in the Mobile ViT module of the original Mobile ViT network is extracted, fused with the global representation, and then subjected to a Concat splicing operation with the original input to obtain an improved Mobile ViT module, i.e., the Mobile ViT X module;

[0014] S4: network building

[0015] A new Mobile ViT network is built using the MV2-SE module and the Mobile ViT X module based on the structure of the original Mobile ViT network;

[0016] S5: network training

[0017] The new Mobile ViT network in step S4 is trained using the training set, and a corneal ulcer classification and identification model is obtained after training, and the corneal ulcer classification and identification model is tested using an independent test set, and the Grad-CAM method is used to visualize and verify the output of the corneal ulcer classification and identification model;

[0018] S6: classification and identification

[0019] The image to be detected is input into the corneal ulcer classification and identification model for classification and identification to obtain the classification and identification result.

[0020] Further, in step S1, the corneal ulcer data set is divided into a training set and a test set according to a set proportion, and the image size in the training set is adjusted to a set size.

[0021] Further, in the step S1, the label types of the image samples include punctate corneal ulcer, punctate patchy mixed corneal ulcer and patchy corneal ulcer.

[0022] Further, in the step S2, the MV2 module of the original Mobile ViT network comprises a first convolutional layer, a depth separable convolutional layer and a second convolutional layer connected in sequence, and the three convolutional layers are all used for extracting features.

[0023] Further, in the step S2, the SE attention module comprises a compression layer and an excitation layer; the excitation layer comprises two fully connected layers connected in sequence, the compression layer compresses a feature map into a 1*1*C vector through global pooling, the first fully connected layer of the excitation layer has C*SE-Radio neurons, the input of which is 1*1*C and the output of which is 1*1*C*SE-Radio, the second fully connected layer has C neurons, the input of which is 1*1*C*SE-Radio and the output of which is 1*1*C, wherein SE-Radio is a scaling parameter; the calculation formulas of the compression layer and the excitation layer are as follows:

[0024]

[0025] wherein, F sq represents a Squeeze operation, Z c is the output after the Squeeze operation, U c is the input; H and W respectively represent the length and width of the feature map, i and j respectively represent the length and width of the picture pixel point, and the specific position is represented by i and j;

[0026] s=F ex (z,W)=σ(g(z,W))=σ(W2δ(W1z))

[0027] wherein, the output of the Squeeze operation is named Z, W1 and W2 are respectively the weights of the first and second fully connected operations, a ReLU activation function is used after the first fully connected operation, a Sigmoid function is connected after the second fully connected layer, and finally the output weight S is obtained.

[0028] Further, in the step S2, the SE attention module is fused in the MV2 module of the original Mobile ViT network to obtain an improved MV2 module, i.e., an MV2-SE module, wherein the SE attention module is inserted between the depth separable convolutional layer and the second convolutional layer in the MV2 module.

[0029] Further, in the step S3, the Mobile ViT X module comprises a local feature layer, a global feature layer and a fusion layer, the local feature output by the local feature layer is extracted, and the global feature output by the global feature layer is added through the fusion layer to improve the feature extraction amount and fusion capacity of the fusion part.

[0030] Further, in the step S5, the specific training process of the corneal ulcer classification and recognition model mainly comprises the following steps:

[0031] S501: After the test picture is preprocessed, the size is adjusted to 224*224;

[0032] S502: After a 3*3 convolution layer, the feature map size is 128*128;

[0033] S503: The feature map is sequentially processed by three MV2-SE modules, and the feature map size is down-sampled to 64*64, wherein the input and output sizes of the first MV2-SE module and the third MV2-SE module are unchanged, and the second MV2-SE module is down-sampled by two times;

[0034] S504: Then the feature map sequentially passes through three mixed modules, and the output feature map size is 8*8, wherein each mixed module comprises an MV2-SE module and a Mobile ViT X module connected in sequence;

[0035] S505: The feature map is output after passing through a 1*1 convolution layer and a global pooling layer, and type judgment is performed.

[0036] Further, in the step S5, the corneal ulcer classification and recognition model is tested, the Grad-CAM method is used to visualize and verify the output of the corneal ulcer classification and recognition model, the weight matrix is obtained by backward propagation of the extracted features, and the important features to the secondary features are displayed by red to blue gradient display through the method of heat map.

[0037] Compared with the prior art, the present application has the following advantages:

[0038] (1) In the MV2 module, the SE attention module is added to improve the feature extraction capability. The SE attention module can effectively learn the correlation of feature channels, thereby improving the performance of the model. By adjusting the weight of the feature channel, the SE attention module can enhance the expression of specific features, so that the model can focus more on important features. Secondly, the SE attention module can be easily combined with various deep neural network structures, and has strong pluggability. Finally, the SE attention can be realized by using global pooling and a small group of fully connected layers, which has low computational cost, so the model performance can be enhanced without increasing the computational burden. After adding the SE attention module in the model, not only the accuracy is improved from 88.7% to 90.8%, but also through Grad-CAM visualization, it can be seen that the feature map of the extracted features has a great similarity with the ulcer area.

[0039] (2) The local representation and the global representation are fused, which can improve the accuracy of the model. Local features can help the model identify local structures and texture information in the image, while global features can provide overall context information of the image. Fusing these two kinds of information can improve the accuracy and robustness of the model. It can also enhance the generalization ability of the model. Local features can only represent local information in the image, while global features can provide overall structure and semantic information of the image. Fusing the two features together can make the model better understand the image, thereby improving its generalization ability. In addition, it can also reduce the risk of overfitting: fusing local features and global features can reduce the risk of overfitting of the model. Through this improvement, the original accuracy of 90.8% is improved to 91.5%. After visualization, it can be seen that the extracted feature map area is basically consistent with the ulcer area, so it can be seen that the extracted features are correct. BRIEF DESCRIPTION OF DRAWINGS

[0040] Figure 1 is a flowchart of the corneal ulcer classification and recognition method based on the Mobile ViT network in the embodiment of the application;

[0041] Figure 2 is a structural diagram of the MV2 combined with the SE module (i.e. MV2-SE module) in the embodiment of the application;

[0042] Figure 3 is a structural diagram of the improved Mobile ViT block (i.e. Mobile ViT X module) in the embodiment of the application;

[0043] Figure 4 is a structural diagram of the improved Mobile ViT network in the embodiment of the application;

[0044] Figure 5 (a) is an example diagram of a point-shaped corneal ulcer in an embodiment of the present application;

[0045] Figure 5 (b) is an example diagram of a point-flap mixed corneal ulcer in an embodiment of the present application;

[0046] Figure 5 (c) is an example diagram of a flap-shaped corneal ulcer in an embodiment of the present application;

[0047] Figure 6 is a visualization effect diagram after adding different modules in an embodiment of the present application, wherein (a)-(d) are original pictures, (e)-(h) are effect diagrams without any improvement, (i)-(l) are effect diagrams after adding SE attention modules, and (m)-(p) are finally improved effect diagrams;

[0048] Figure 7 is a visualization effect diagram after adding different attention modules in an embodiment of the present application, wherein (a)-(d) are original pictures, (e)-(h) are effect diagrams after adding SE attention modules, (i)-(l) are effect diagrams after adding CBAM attention modules, and (m)-(p) are effect diagrams after adding CA attention modules;

[0049] Figure 8 is a structural schematic diagram of a module improved according to Mobile ViT V3 in an embodiment of the present application. DETAILED DESCRIPTION

[0050] The embodiments of the present application will be described in detail below, and the embodiments are implemented on the premise of the technical solutions of the present application, and detailed implementation manners and specific operation processes are given, but the protection scope of the present application is not limited to the following embodiments.

[0051] The present embodiment provides a technical solution: a corneal ulcer classification and recognition method based on a Mobile ViT network, comprising the following steps:

[0052] S1: Preprocessing the corneal ulcer dataset.

[0053] S2: Fusing an SE attention module in the MV2 module of the original Mobile ViT network to improve the feature extraction capability of the model and increase the information exchange of the model in different dimensions.

[0054] S3: Extracting the local representation in the Mobile ViT module of the original Mobile ViT network, fusing it with the global representation, and then performing a Concat splicing operation with the original input to improve the feature extraction capability and information exchange capability.

[0055] S4: Using the original Mobile ViT network structure as a prototype, a new network is built using improved modules.

[0056] S5: Test the results on the test set and use the Grad-CAM method to visualize and verify the results.

[0057] In this embodiment, the step S1 includes the following sub-steps:

[0058] S11: The data set used in the present application is the corneal ulcer automatic classification data set released by Southern University of Science and Technology-Sun Yat-sen University, which contains 712 eye staining images. In addition, each image is provided with a three-class label. The focus of the present application is the classification and identification of three common ulcer patterns. They include punctate corneal ulcer, point and sheet mixed corneal ulcer, and sheet corneal ulcer. In this work, we will allocate 80% of the data for the training set and 20% for the test set. And ensure that the data used for the test set does not exist in the training set and does not participate in any model training process. Table 1 shows the number and proportion of each type in the entire data set.

[0059] Table 1 Number and proportion of various types of corneal ulcer data

[0060] Identification Type Number Proportion 0 Punctate corneal ulcer 358 50.28％ 1 Punctate and patchy mixed corneal ulcer 263 36.94％ 2 Patchy corneal ulcer 91 12.78％

[0061] S12: Adjust the image size in the training set to 224x224.

[0062] Further, the specific process of the step S2 is:

[0063] S21: SE (Squeeze and Excitation) attention module is a module for convolutional neural network (CNN), which aims to improve the performance of the model by enhancing important feature maps in the network. The core idea of SE attention module is that, for a given input feature map, first compress the channel dimension of the feature map through a global pooling layer (usually average pooling), thereby generating a global descriptor. Then convert this descriptor into a vector through some fully connected layers, which can be used to weight each channel. These weights are used to scale the activation values of each channel in the feature map, thereby enhancing important feature maps and suppressing unimportant feature maps; the SE attention module includes a compression layer and an excitation layer; the excitation layer includes two fully connected layers connected in turn, the compression layer compresses the feature map into a 1*1*C vector through global pooling, the first fully connected layer of the excitation layer has C*SE-Radio neurons, the input is 1*1*C and the output is 1*1*C*SE-Radio, the second fully connected layer has C neurons, the input is 1*1*C*SE-Radio and the output is 1*1*C, wherein SE-Radio is a scaling parameter; the compression layer and the excitation layer have the following calculation formulas,

[0064]

[0065] wherein F sq represents the Squeeze operation, Z c is the output after the Squeeze operation, U c is the input; H and W represent the length and width of the feature map respectively, i and j represent the length and width of the pixel point respectively, and the specific position is represented by i and j;

[0066] s = F ex (z, W) = σ (g (z, W)) = σ (W2δ (W1z))

[0067] wherein the output of the Squeeze operation is named Z, W1 and W2 are the weights of the first and second fully connected operations respectively, a ReLU activation function is used after the first fully connected operation, and a Sigmoid function is connected after the second fully connected layer to obtain the output weight S.

[0068] S22: The MV2 module mainly includes three parts, a first convolutional layer, a depth separable convolutional layer and a second convolutional layer connected in turn, and the three convolutional layers are all used to extract features, wherein the depth separable convolutional layer can not only extract features, but also has the largest difference from ordinary convolutional layers, that is, there is no large weight redundancy, which can greatly reduce the parameter amount.

[0069] S23: The feature extraction capability of the MV2 module for the picture is not perfect. Both the MV2 module and the MobileViT module have spatial induction bias when extracting features, but lack information exchange between channels. Therefore, an SE attention module is added to the MV2 module to improve information exchange between channels, and also to amplify features and increase the extraction capability of important features.

[0070] Further, the step S3 is specifically divided into the following parts:

[0071] S31: The Mobile ViT block (MobileViT module) is a core component in the MobileViT network, which is a basic component unit in the MobileViT network and has high reusability. The MobileViT block is internally composed of multiple layers, including a local representation layer, a global representation layer, and a fusion layer;

[0072] The structure of the MobileViT network is similar to that of the ViT network, but some improvements are introduced to adapt to the limitations on mobile devices; a lightweight attention mechanism is used in the MobileViT network instead of the original multi-head attention mechanism, which realizes the core mechanism in the ViT network with less computational complexity. The overall structure of the MobileViT network consists of an encoder similar to the ViT network and a classification head; the encoder is composed of multiple lightweight ViT modules, each module containing multiple attention blocks based on attention mechanism and an MLP (Multilayer Perceptron) module implemented using depth separable convolution; the classification head maps the feature vector output by the encoder to the class score for classification tasks. Compared with traditional convolutional neural networks, the MobileViT network has greatly optimized the number of parameters and computational complexity, and is suitable for real-time computer vision tasks on mobile devices.

[0073] S32: The original MobileViT network outputs the local representation and takes the output as the input of the global representation; we believe that this may cause the loss of features extracted by the local representation, which is also confirmed in subsequent tests; therefore, we extract the output of the local representation layer and add it to the global representation to improve the feature extraction and fusion capability of the fusion part.

[0074] Further, the specific process of step S5 is as follows: first, the training picture is preprocessed and adjusted to a size of 224x224. Then, the picture is subjected to a 3x3 convolution for downsampling by two, and the output size is 128x128. Then, the picture is processed by three MV2-SE modules, wherein the middle module is subjected to downsampling by two, and the input and output sizes of the remaining modules remain unchanged, so that the picture size is downsampled to 64x64. Next, the picture is subjected to three mixed structures composed of MV2-SE modules and Mobile ViT X modules in an alternating manner. Among them, the MV2-SE modules of the three mixed structures are subjected to downsampling operations, and the output picture size is 8x8. Finally, the picture is subjected to a 1x1 convolution and global pooling operation for output, and type judgment is performed; finally, Grad-CAM is used for visual verification of the results.

[0075] In the present embodiment, Table 2 represents the influence of the experimental results after adjusting different modules, wherein +SE represents the influence of the experimental results after adding the SE attention module, and it can be seen that the accuracy is increased from 88.7% to 90.8 after adding the SE attention module, +SE+X1 represents the influence of the experimental results after adding the SE attention module and the Mobile ViT X1 structure in the Mobile ViT network, and the last one is the experimental result of the network model improved finally by the present application, which is 91.5%. Figure 8 Figure 6 is the visualization effect diagram after adding different modules, wherein (a)-(d) are original pictures, (e)-(h) are effect diagrams without any improvement, (i)-(l) are effect diagrams after adding the SE attention, and (m)-(p) are effect diagrams of the final improved version, and it can be seen that the extracted feature positions after our improvement are almost completely corresponding to the ulcer areas of the original pictures.

[0076] Table 2 Comparison of experimental results after adding different modules

[0077] Method Accuracy Total parameters Parameter memory Total memory usage Mobile ViT 88.7％ 951987 3.63MB 3440.32MB +SE Block 90.8％ 969807 3.70MB 3651.96MB +SE+X1 90.1％ 969,807 3.70MB 3651.96MB Ours 91.5％(↑2.8) 969807 3.70MB 3651.96MB

[0078] Table 3 represents the comparison of experimental results of different attention mechanisms, and through comparison and verification experiments, it is found that the SE attention module is more suitable for our network and the used data set, mainly because the SE attention module is relatively simple. After comparing the CBAM attention module and the CA attention module, it is considered that the SE attention module is easier to integrate with the original Mobile ViT network. In addition, the data set has fewer images, and the similarity of image data is high. The SE attention module has a good regularization effect on the network, which can effectively prevent overfitting. Figure 7 ​These are visualizations of the effects of adding different attention modules. (a)-(d) are the original images, (e)-(h) are the effects of adding the SE attention module, (i)-(l) are the effects of adding the CBAM attention module, and (m)-(p) are the effects of adding the CA attention module. It can be seen that the feature locations extracted by adding the SE attention module are closest to the ulcer area in the original image.

[0079] Table 3 Comparison of experimental results for different attention mechanisms

[0080]

[0081]

[0082] Table 4 shows the experimental results of different networks on the corneal ulcer dataset. Modified VGG is the classification result obtained by Tang Ningbjao et al. after improving the VGG network. Another network, Shuffle ViT, was developed as a combination of Shuf-fleNet and Mobile ViT. This novel network is an attempt to improve upon ShuffleNet, and it has shown significant improvement compared to other networks besides Mobile ViT. However, in terms of performance, it still lags slightly behind Mobile ViT. Other networks compared include the classic ResNet, the lightweight ShuffleNet, ConvNext, the currently popular Swin Transformer, and the classic VisionTransformer. After a thorough comparison of all results, it can be seen that the improved Mobile ViT network model not only outperforms the original model but also surpasses the best result by 1.4%.

[0083] Table 4. Experimental results of different networks on the corneal ulcer dataset.

[0084] Model Accuracy Modified VGG 88.89％ ResNet 83.0％ ShuffleNet 83.5％ Vision Transformer 73.0％ Swin Transformer 90.1％ ConvNext 85.1％ Shuffle+ViT 85.3％ Mobile ViT 88.7％ Ours 91.5％

[0085] In summary, the corneal ulcer classification and identification method based on the Mobile ViT network of the above-mentioned embodiments enhances the ability of the network to extract important features by adding an SE attention module to the original Mobile ViT network, while suppressing unnecessary features; for example, the fluorescent solution remains on the corneal surface and the upper and lower eyelids, and these residual fluorescent solutions are very similar to ulcers and are easily confused; an optimization scheme is proposed to significantly reduce these mixed features by adding an SE attention module; in the present application, the connection method of the Mobile ViT network has also been improved, a part is added to extract local feature maps from the original model and fuse them with global feature maps to extract more effective features, while increasing the exchange of information between various features; finally, the effectiveness of the improvement is also demonstrated through various comparative experiments, and the accuracy is successfully improved from 88.7% to 91.5%. The improved network model is also compared with other methods, proving that it has the best accuracy and the number of currently available parameters; at the same time, the heat map of some output feature maps is visualized using Grad-CAM, and it can be seen that the extracted features almost overlap with the ulcer area, so it can be seen that the extracted features are correct.

[0086] Although the embodiments of the present application have been shown and described above, it will be understood that the above-mentioned embodiments are exemplary and cannot be construed as limiting the present application, and those of ordinary skill in the art can make changes, modifications, replacements and variations to the above-mentioned embodiments within the scope of the present application.

Claims

1. A method for classifying and identifying corneal ulcers based on a Mobile ViT network, characterized in that, Includes the following steps: S1: Sample Preprocessing Preprocessing of image samples from the corneal ulcer dataset; S2: Increased attention mechanisms By fusing the SE attention module into the original Mobile Vit network's MV2 module, an improved MV2 module, namely the MV2-SE module, is obtained. S3: Feature splicing The local representations in the Mobile ViT module of the original Mobile ViT network are extracted, fused with the global representations, and then concatenated with the original input to obtain the improved Mobile ViT module, namely the Mobile ViT X module. S4: Network Setup Based on the original Mobile ViT network structure, a new Mobile ViT network was built using the MV2-SE module and the Mobile ViT X module; S5: Network Training The new Mobile ViT network in step S4 is trained using the training set to obtain a corneal ulcer classification and recognition model. The corneal ulcer classification and recognition model is then tested using an independent test set. The output of the corneal ulcer classification and recognition model is visualized and verified using the Grad-CAM method. S6: Classification and Recognition The image to be detected is input into the corneal ulcer classification and recognition model for classification and recognition, and the classification and recognition results are obtained. In step S2, the MV2 module of the original Mobile ViT network includes a first convolutional layer, a depthwise separable convolutional layer, and a second convolutional layer connected in sequence. All three convolutional layers are used to extract features. In step S2, the SE attention module includes a compression layer and an activation layer. The activation layer includes two sequentially connected fully connected layers. The compression layer compresses the feature map into a 1*1*C vector using global pooling. The first fully connected layer of the activation layer has C*SE-Radio neurons, with an input of 1*1*C and an output of 1*1*C*SE-Radio. The second fully connected layer has C neurons, with an input of 1*1*C*SE-Radio and an output of 1*1*C, where SE-Radio is used as a scaling parameter. The calculation formulas for the compression layer and the activation layer are as follows: Among them, F sq Z represents the Squeeze operation. c It is the output after the Squeeze operation, U c The input is H and W, which represent the length and width of the feature map, respectively, and i and j represent the length and width of the image pixels, respectively. The specific location is represented by i and j. s=F ex (z,W)=σ(g(z,W))=σ(W2δ(W1z)) The output of the Squeeze operation is named Z, W1 and W2 are the weights of the first and second fully connected operations, respectively. After the first fully connected operation, a ReLU activation function is applied, and the second fully connected layer is followed by a Sigmoid function to finally obtain the output weight S. In step S2, the SE attention module is fused into the MV2 module of the original Mobile ViT network to obtain the improved MV2 module, namely the MV2-SE module. The SE attention module is inserted between the depth-separable convolutional layer and the second convolutional layer in the MV2 module.

2. The method for corneal ulcer classification and identification based on Mobile ViT network according to claim 1, characterized in that: In step S1, the corneal ulcer dataset is divided into a training set and a test set according to a set ratio, and the image size in the training set is adjusted to a set size.

3. The method for corneal ulcer classification and identification based on Mobile ViT network according to claim 2, characterized in that: In step S1, the label types of the image samples include punctate corneal ulcers, punctate-pattern mixed corneal ulcers, and patchy corneal ulcers.

4. The method for corneal ulcer classification and identification based on Mobile ViT network according to claim 1, characterized in that: In step S3, the Mobile ViT X module includes a local representation layer, a global representation layer, and a fusion layer. The local representation output by the local representation layer is extracted and added to the global representation output by the global representation layer through the fusion layer to improve the feature extraction amount and fusion capability of the fused part.

5. The method for corneal ulcer classification and identification based on Mobile ViT network according to claim 4, characterized in that: In step S5, the specific training process of the corneal ulcer classification and recognition model mainly includes the following steps: S501: The test image was resized to 224×224 after preprocessing; S502: After passing through a 3×3 convolutional layer and undergoing double downsampling, the output feature map size is 128×128; S503: The feature map is processed sequentially through three MV2-SE modules to downsample the feature map size to 64×64. The input and output sizes of the first and third MV2-SE modules remain unchanged, while the second MV2-SE module performs a double downsampling. S504: The feature map then passes through three mixing modules in sequence, and the output feature map size is 8×8. Each mixing module includes an MV2-SE module and a Mobile ViT X module connected in sequence. S505: The feature map is output after passing through a 1×1 convolutional layer and a global pooling layer, and the type is determined.

6. The method for corneal ulcer classification and identification based on Mobile ViT network according to claim 5, characterized in that: In step S5, the corneal ulcer classification and recognition model is tested. The output of the corneal ulcer classification and recognition model is visualized and verified using the Grad-CAM method. The extracted features are backpropagated to obtain the weight matrix, which is then displayed on the original image using a heatmap method. The important features are displayed in a gradient from red to blue to the secondary features.

Images