Ophthalmology image classification method and device based on global local feature extraction, equipment and medium

Abstract

The invention discloses an ophthalmology image classification method and device based on global local feature extraction, equipment and a medium, and relates to the technical field of medical image analysis, and the method comprises the steps: obtaining a to-be-classified high-resolution cornea image; inputting the to-be-classified high-resolution cornea image into a preset keratitis image classification model so as to perform fungal image classification on the to-be-classified high-resolution cornea image through the preset keratitis image classification model, and outputting a corresponding aspergillus keratitis image classification result or fusarium keratitis image classification result; wherein the preset keratitis image classification model is an image classification model which learns multi-scale fungus feature extraction capability, fuses image global features and image local features and performs image classification. And fine classification of the two types of fungi is realized through the model.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image analysis, and in particular to an ophthalmic image classification method, device, equipment and medium based on global and local feature extraction. Background Art

[0002] Fungal keratitis is a serious corneal disease caused by direct fungal infection of the cornea, leading to blindness and even loss of the eyeball in severe cases. It is one of the leading causes of blindness. The incidence of fungal keratitis has increased significantly, particularly in the past decade. If fungal keratitis is not diagnosed early, the fungus is likely to invade deep into the cornea, impairing the penetration of therapeutic drugs. Soleimani et al. proposed a multi-network combination classification algorithm that combines multiple neural network models to leverage the characteristics of different neural networks for more efficient classification of keratitis images. However, this multi-network combination classification algorithm requires a large amount of data and computing resources during training, resulting in low stability. Furthermore, the network structure is complex, and the classification performance is significantly affected by parameter values. Jiang et al. proposed an object localization method based on a cost-sensitive deep attention convolutional neural network model for the automated diagnosis of keratitis. They used a single-shot multi-frame detector algorithm to automatically localize the conjunctival and corneal regions on raw slit-lamp images. A cost-sensitive deep attention convolutional network was then used to classify the localized regions to distinguish between keratitis, other corneal abnormalities, and normal corneas. However, the single-minded approach of the detection layer within their network architectures results in inadequate use of feature information, resulting in less robust target detection and difficulty effectively extracting hyphae features. Both technologies primarily utilize slit-lamp microscope images, which are primarily suitable for observing the surface and superficial structures of the eye and are unable to fully capture the complex mycelial structure of corneal lesions. Zeng Qingyan et al.'s "A Method and System for Detecting Fungal Keratitis Based on Convolutional Neural Networks" leverages the power of deep learning in image classification to train a convolutional neural network on pre-processed images to classify images with and without hyphae. However, this method is limited to classifying positive fungal keratitis and lacks detailed classification of fungal keratitis.

[0003] In summary, how to accurately classify fungal keratitis based on the image features of high-resolution corneal images taken by high-efficiency confocal microscopes is a technical problem that needs to be solved in this field. Summary of the Invention

[0004] In view of this, the present invention aims to provide an ophthalmic image classification method, apparatus, device, and medium based on global and local feature extraction, which can accurately classify fungal keratitis based on the image features of high-resolution corneal images captured by an efficient confocal microscope. The specific scheme is as follows: In a first aspect, the present application discloses an ophthalmic image classification method based on global and local feature extraction, which is applied to a computer device and comprises: Acquire a high-resolution corneal image to be classified; the high-resolution corneal image to be classified is a corneal image containing a microscopic structure of hyphae; The high-resolution corneal image to be classified is input into a preset keratitis image classification model so that the high-resolution corneal image to be classified is classified as a fungal image by the preset keratitis image classification model, and the corresponding Aspergillus keratitis image classification result or the Fusarium keratitis image classification result is output; wherein, the preset keratitis image classification model is an image classification model that has learned multi-scale fungal feature extraction capabilities, integrated image global features and image local features, and performed image classification.

[0005] Optionally, the preset keratitis image classification model includes a preset convolutional neural network and a preset local feature extraction module. Accordingly, performing fungal image classification on the high-resolution corneal image to be classified by using the preset keratitis image classification model includes: Extracting global semantic information of fungal lesion areas at different levels in the high-resolution corneal image to be classified through the preset convolutional neural network of the preset keratitis image classification model to obtain global image features; Extracting local features of the global features of the image by the preset local feature extraction module connected to the preset convolutional neural network to obtain multi-scale local features of the image including hypha branching angles and hypha terminal morphology; Performing orthogonal fusion processing on the global features of the image and the local features of the multi-scale image by using the preset keratitis image classification model to obtain multi-scale joint features, and inputting the multi-scale joint features into a classifier so as to map the multi-scale joint features to a Fusarium image feature category space and an Aspergillus image feature category space, respectively, by the classifier; The probability values of the multi-scale joint features in the Fusarium image feature category space and the Aspergillus image feature category space are calculated respectively, and the maximum probability value is output.

[0006] Optionally, the preset convolutional neural network includes several convolution modules, each of which is connected to a corresponding preset local feature extraction module. Accordingly, the preset convolutional neural network of the preset keratitis image classification model extracts global semantic information of fungal lesion areas at different levels in the high-resolution corneal image to be classified to obtain global image features; the preset local feature extraction module connected to the preset convolutional neural network extracts local features of the global image features to obtain multi-scale image local features including hyphae branching angles and hyphae terminal morphology, including: Extracting edge and texture features of the high-resolution corneal image to be classified using the first convolution module of the preset convolutional neural network, and locating the initial lesion area on the corneal surface to obtain shallow global semantic information; wherein the edge and texture features include hyphae outline, hyphae edge direction, and texture distribution; Extracting the local morphology of mycelium from the shallow global semantic information through a first preset local feature extraction module connected to the first convolution module to obtain shallow image local features; Performing orthogonal fusion processing on the shallow global semantic information and the shallow image local features through the preset keratitis image classification model to obtain shallow multi-scale joint features; Inputting the shallow multi-scale joint features into a second convolution module connected to the first preset local feature extraction module, so as to extract the hyphae distribution pattern in the shallow multi-scale joint features through the second convolution module, and extract the geometric shape and spatial extent of the lesion area to obtain mid-level global semantic information; wherein the hyphae distribution pattern includes hyphae density and hyphae arrangement direction; extracting the spatial arrangement regularity of hyphae in the middle-layer global semantic information through a second preset local feature extraction module connected to the second convolution module to obtain local features of the middle-layer image; Performing orthogonal fusion processing on the middle-layer global semantic information and the middle-layer image local features through the preset keratitis image classification model to obtain middle-layer multi-scale joint features; The middle-level multi-scale joint features are input into a third convolution module connected to the second preset local feature extraction module, so as to extract the hyphae network structure, lesion area semantic information and global context information in the middle-level multi-scale joint features through the third convolution module to obtain high-level global semantic information; wherein the hyphae network structure is used to describe the branching pattern and connection relationship of the hyphae; the lesion area semantic information is semantic category information including hyphae infection area, inflammatory response area, and normal tissue area; and the global context information is used to describe the spatial relationship between the lesion area and surrounding tissues; extracting the three-dimensional spatial relationship of hyphae in the high-level global semantic information through a third preset local feature extraction module connected to the third convolution module to obtain high-level image local features; The high-level global semantic information and the high-level image local features are orthogonally fused using the preset keratitis image classification model to obtain target multi-scale joint features.

[0007] Optionally, the preset local feature extraction module includes a dilated convolution layer and a self-attention mechanism layer. Accordingly, the preset local feature extraction module connected to the preset convolutional neural network extracts local features of the image global features to obtain multi-scale image local features including hyphae branching angles and hyphae terminal morphology, including: Extracting multi-scale local lesion features on the global features of the image that match the current hierarchical convolution module through the dilated convolution layer according to a preset expansion rate; The self-attention mechanism layer is used to dynamically assign weights to the multi-scale local lesion features, and obtain multi-scale image local features including hyphae branching angles and hyphae terminal morphology.

[0008] Optionally, the using the self-attention mechanism layer to dynamically assign weights to the multi-scale local lesion features and obtain multi-scale image local features including hypha branching angles and hypha terminal morphologies includes: Performing convolution processing on the multi-scale local lesion features to obtain corresponding query matrix, key matrix and value matrix; Calculating an attention weight matrix based on the query matrix, the key matrix, and the value matrix to obtain a weighted feature tensor based on the attention weight matrix and the value matrix; The weighted feature tensor and the multi-scale local lesion feature are added together to generate a multi-scale image local feature including hypha branching angle and hypha terminal morphology.

[0009] Optionally, before inputting the high-resolution corneal image to be classified into a preset keratitis image classification model, the method further includes: Constructing a data training set based on several historical high-resolution corneal images carrying real keratitis labels; wherein the real keratitis labels are Fusarium keratitis labels or Aspergillus keratitis labels; Using the data training set, an initial keratitis image classification model including an initial convolutional neural network and an initial local feature extraction module is trained, global features are extracted by the initial convolutional neural network, and multi-scale local features are extracted by the initial local feature extraction module, and the global features and the multi-scale local features are orthogonally fused to obtain fused features; Input the fusion features into the classifier and output the prediction result; A cross entropy loss value between the prediction result and the true keratitis label is calculated, and model parameters of the initial keratitis image classification model are reversely optimized based on the cross entropy loss value to obtain a preset keratitis image classification model.

[0010] Optionally, before constructing a data training set based on a plurality of historical high-resolution corneal images carrying real keratitis labels, the method further includes: Acquiring a fungal keratitis image that meets a preset high-resolution condition and is captured by a confocal microscope, and performing standardized preprocessing and data enhancement processing on the fungal keratitis image that meets the preset high-resolution condition to obtain an enhanced high-resolution fungal keratitis image; the data enhancement processing includes any one or more of random horizontal flipping, rotation, and brightness adjustment; The enhanced high-resolution fungal keratitis image is labeled with corresponding Fusarium keratitis or Aspergillus keratitis to obtain a historical high-resolution corneal image carrying a real keratitis label.

[0011] In a second aspect, the present application discloses an ophthalmic image classification device based on global and local feature extraction, comprising: An image acquisition module is used to acquire a high-resolution corneal image to be classified; the high-resolution corneal image to be classified is a corneal image containing a microscopic structure of hyphae; An image classification module is used to input the high-resolution corneal image to be classified into a preset keratitis image classification model, so as to perform fungal image classification on the high-resolution corneal image to be classified through the preset keratitis image classification model, and output the corresponding Aspergillus keratitis image classification result or Fusarium keratitis image classification result; wherein, the preset keratitis image classification model is an image classification model that has learned multi-scale fungal feature extraction capabilities, integrated image global features and image local features, and performed image classification.

[0012] In a third aspect, the present application discloses an electronic device, comprising: Memory, used to store computer programs; A processor is used to execute the computer program to implement the steps of the aforementioned disclosed method for ophthalmic image classification based on global-local feature extraction.

[0013] In a fourth aspect, the present application discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, the steps of the aforementioned ophthalmic image classification method based on global-local feature extraction are implemented.

[0014] As can be seen, the present application discloses an ophthalmic image classification method based on global and local feature extraction, which is applied to a computer device and comprises: obtaining a high-resolution corneal image to be classified; the high-resolution corneal image to be classified is a corneal image containing the microstructure of hyphae; inputting the high-resolution corneal image to be classified into a preset keratitis image classification model, so that the preset keratitis image classification model classifies the high-resolution corneal image to be classified as a fungal image and outputs a corresponding Aspergillus keratitis image classification result or a Fusarium keratitis image classification result; wherein the preset keratitis image classification model is an image classification model that has learned multi-scale fungal feature extraction capabilities and fuses global and local image features to perform image classification. As can be seen, by using the high-resolution corneal image to be classified, the microstructure of hyphae (such as branching morphology and terminal features) can be clearly captured, improving the input data quality of the classification model. The classification model combines the high-resolution confocal image with the multi-scale feature fusion to automatically analyze the corneal image, reduce manual subjective judgment errors, and improve the accuracy of the sub-classification of the two types of fungi. The model data results can be used to distinguish Aspergillus (slender, sharp-angled branches) from Fusarium (thick, straight, and sparse branches), reducing the misdiagnosis rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0016] Figure 1 This is a flow chart of an ophthalmic image classification method based on global and local feature extraction disclosed in this application; Figure 2 A flow chart of classifying a fungal keratitis image disclosed in this application using a preset keratitis image classification model; Figure 3 This is a flowchart of a specific ophthalmic image classification method based on global and local feature extraction disclosed in this application; Figure 4 This is a schematic diagram of the structure of an ophthalmic image classification device based on global and local feature extraction disclosed in this application; Figure 5 This is a structural diagram of an electronic device disclosed in this application. DETAILED DESCRIPTION

[0017] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0018] Fungal keratitis is a serious corneal disease caused by direct fungal infection of the cornea, potentially leading to blindness and even loss of the eye. The incidence of fungal keratitis has increased significantly over the past decade. If fungal keratitis is not diagnosed promptly, the fungus is likely to invade deep into the cornea, resulting in poor penetration of therapeutic drugs. Therefore, early diagnosis is crucial to avoiding serious complications and minimizing the need for surgery. Recently, Soleimani et al. proposed a multi-network combination classification algorithm. By combining multiple neural network models, they leverage the characteristics of different neural networks to achieve more efficient classification of keratitis images. Zeng Qingyan et al. proposed a "Fungal Keratitis Detection Method and System Based on Convolutional Neural Networks," leveraging the advantages of deep learning in image classification. A convolutional neural network trained on pre-processed images can classify images with and without hyphae. Jiang et al. proposed an object localization method based on a cost-sensitive deep attention convolutional neural network model for the automated diagnosis of keratitis. A single-shot multi-frame detector algorithm is used to automatically locate conjunctival and corneal regions on raw slit-lamp images, and a cost-sensitive deep attention convolutional network is used to classify the located regions to identify keratitis, other corneal abnormalities, and normal cornea.

[0019] Soleimani et al. proposed a multi-network combination classification algorithm. However, this algorithm requires a large amount of data and computing resources during the training phase, resulting in low stability, a complex network structure, and significant impact of parameter values on classification performance. Its classification performance lags behind the latest image classification methods. Jiang et al. proposed an object localization method based on a cost-sensitive deep attention convolutional neural network model. However, the single detection layer in their network structure results in inadequate utilization of feature information, resulting in less robust target detection and difficulty in effectively extracting hyphae features. Furthermore, both technologies primarily utilize images captured using a slit lamp microscope. These images are primarily suitable for observing the surface and shallow structures of the eye and are unable to fully capture the complex mycelial structure of corneal lesions, significantly different from images captured using a confocal microscope. Zeng Qingyan et al. and Liu et al. used a deep learning method using convolutional networks to classify confocal microscope images. However, this method was limited to the classification of positive fungal keratitis and did not subclassify fungal keratitis. This is because within the broad category of fungal keratitis, Fusarium keratitis and Aspergillus keratitis are the most common and difficult to distinguish, which is a difficulty that urgently needs to be overcome.

[0020] To this end, this paper provides an ophthalmic image classification scheme based on global local feature extraction, which can accurately classify fungal keratitis based on the image features of high-resolution corneal images taken by an efficient confocal microscope.

[0021] Reference Figure 1 As shown, the embodiment of the present application discloses an ophthalmic image classification method based on global and local feature extraction, which is applied to a computer device, including: Step S11: obtaining a high-resolution corneal image to be classified; the high-resolution corneal image to be classified is a corneal image containing a microscopic structure of hyphae.

[0022] In this example, a high-resolution corneal image to be classified was first acquired using a confocal microscope. It is important to note that confocal microscopy provides micron-level high-resolution imaging, clearly demonstrating the microscopic structure of hyphae, such as hyphal diameter, branching angles, terminal morphology, and arrangement density. Furthermore, confocal microscopy can capture three-dimensional information. By scanning the cornea layer by layer through optical sectioning, it can penetrate all corneal layers (epithelium, stroma, and endothelium), revealing the penetration depth and spatial distribution of hyphae within each layer. Therefore, the high-resolution corneal image to be classified acquired using a confocal microscope contains high-contrast, low-noise microscopic features, such as the clear outlines of hyphal cell walls and the sharp angles of bifurcation points. These microscopic features facilitate the distinction between Fusarium (thick, straight, and sparsely branched) and Aspergillus (slender, densely branched), providing rich multi-scale information suitable for extracting discriminative features using deep learning models.

[0023] Step S12: Input the high-resolution corneal image to be classified into a preset keratitis image classification model, so that the high-resolution corneal image to be classified is classified as a fungal image by the preset keratitis image classification model, and the corresponding Aspergillus keratitis image classification result or the Fusarium keratitis image classification result is output; wherein the preset keratitis image classification model is an image classification model that has learned multi-scale fungal feature extraction capabilities, integrated image global features and image local features, and performed image classification.

[0024] In this embodiment, before inputting the corneal image to be resolved into the preset keratitis image classification model for classification, a preset keratitis image classification model is pre-built and trained. Specifically: A data training set is constructed based on several historical high-resolution corneal images carrying real keratitis labels; wherein the real keratitis label is a Fusarium keratitis label or an Aspergillus keratitis label; the data training set is used to train an initial keratitis image classification model comprising an initial convolutional neural network and an initial local feature extraction module, wherein global features are extracted by the initial convolutional neural network and multi-scale local features are extracted by the initial local feature extraction module, and the global features and the multi-scale local features are orthogonally fused to obtain fused features; the fused features are input into a classifier and a prediction result is output; a cross-entropy loss value between the prediction result and the real keratitis label is calculated, and the model parameters of the initial keratitis image classification model are reversely optimized based on the cross-entropy loss value to obtain a preset keratitis image classification model. It can be understood that historical high-resolution corneal images carrying corresponding keratitis labels are first obtained to construct a data training set, so that the initial keratitis image classification model can be trained with the data training set. The initial keratitis image classification model is constructed with the ConvNext V2 (Convolutional Neural Networks Next Version 2) network as the initial convolutional network, the initial void convolution layer and the initial self-attention mechanism layer as the initial local feature extraction module. Then, the data training set is input into the initial keratitis image classification model, and the global features and multi-scale local features of the image are extracted in turn through the initial convolutional network and the initial local feature extraction module, and the extracted multi-scale local features are projected into the orthogonal space corresponding to the global features to obtain the projected feature information. Through the orthogonal projection operation, local details that are complementary to the global features are retained to avoid redundancy. Then, the projected feature information is spliced according to the channel dimension to obtain fused features (multi-scale joint feature representation). In this way, the initial convolutional network is used as the backbone network, and the initial local feature extraction module is used as the local network. The features extracted by the two parts of the network (global information, local information) are fused to obtain a feature representation containing discriminative lesion information at different scales. The mycelium features are distinguished by rich and effective lesion information, thereby improving the classification accuracy.The fused features are then flattened and input into the fully connected layer, mapped to the category space (Fusarium category space vs. Aspergillus category space), and two categories of probability distributions are output through the Softmax layer. The maximum probability is selected as the prediction result for output. Furthermore, the cross-entropy loss value between the prediction result and the true keratitis label is calculated, and the model parameters of the initial keratitis image classification model are reversely optimized based on the cross-entropy loss value. When the cross-entropy loss value between the prediction result output by the model and the true keratitis label meets the model training stop condition (the loss value is less than the preset threshold), the training is stopped, and the model under the current model parameters is used as the preset keratitis image classification model, wherein the preset keratitis image classification model includes a preset convolutional neural network, a preset local feature extraction module, a fusion layer, a classifier, and a Softmax layer.

[0025] In this embodiment, the specific steps for constructing a data training set are as follows: obtaining fungal keratitis images that meet preset high-resolution conditions, taken through a confocal microscope, and performing standardization preprocessing and data enhancement processing on the fungal keratitis images that meet the preset high-resolution conditions to obtain enhanced high-resolution fungal keratitis images; the data enhancement processing includes any one or more of random horizontal flipping, rotation, and brightness adjustment; the enhanced high-resolution fungal keratitis images are labeled with corresponding Fusarium keratitis or Aspergillus keratitis to obtain historical high-resolution corneal images carrying real keratitis labels. It is understandable that in order for the data training set to better train the initial model, it is necessary to perform standardization preprocessing and data enhancement processing on each fungal keratitis image in the data training set, wherein the standardization preprocessing steps are as follows: The original corneal images captured by the confocal microscope were uniformly scaled to a fixed size (e.g., 224×224 pixels) to meet the input requirements of the convolutional neural network while preserving the integrity of the key hyphae regions. The image pixel values were normalized from the integer range of 0-255 to the floating-point range [0,1] or Z-score normalization (i.e., subtracting the mean and dividing by the standard deviation) was performed to eliminate the impact of illumination differences on the model. If the original image was a three-channel RGB image, its color information was retained; if it was a grayscale image, it was expanded to a single-channel input. To address possible speckle noise or motion artifacts in the original image, a Gaussian filter or median filter was used for smoothing, preserving hyphae edge details while reducing background interference to obtain a standardized preprocessed high-resolution fungal keratitis image. Furthermore, the preprocessed high-resolution fungal keratitis image was enhanced. The specific steps are as follows: The pre-processed high-resolution fungal keratitis images were mirror-flipped (random horizontal / vertical flipping) with a probability of 50% to simulate different shooting angles to obtain enhanced high-resolution fungal keratitis images.

[0026] And / or, the pre-processed high-resolution fungal keratitis image is rotated (randomly rotated) within the range of ±15° to enhance the robustness of the model to changes in hyphae direction, so as to obtain an enhanced high-resolution fungal keratitis image.

[0027] And / or, sub-regions (e.g., 200×200 pixels) are randomly cut out from the pre-processed high-resolution fungal keratitis image and padded to a fixed size to simulate local lesion focusing to obtain an enhanced high-resolution fungal keratitis image.

[0028] And / or, the overall brightness of the pre-processed high-resolution fungal keratitis image is adjusted within a range of ±20% to simulate shooting effects under different lighting conditions, so as to obtain an enhanced high-resolution fungal keratitis image.

[0029] And / or, the contrast difference between hyphae and background in the pre-processed high-resolution fungal keratitis image is enhanced by linear transformation to highlight the microstructure features, so as to obtain an enhanced high-resolution fungal keratitis image.

[0030] And / or, a slight non-rigid deformation (such as mesh distortion) is applied to the preprocessed high-resolution fungal keratitis image to simulate the natural deformation of biological tissue and enhance the model's adaptability to the diversity of hyphae morphology to obtain an enhanced high-resolution fungal keratitis image.

[0031] It should be noted that the above-mentioned image enhancement means can be used in combination or individually, and there is no specific limitation on this.

[0032] Through the above steps, the standardization and diversity of input data are ensured, providing high-quality training samples for the model.

[0033] In this embodiment, the preset keratitis image classification model includes a preset convolutional neural network and a preset local feature extraction module. Accordingly, the fungal image classification of the high-resolution corneal image to be classified by the preset keratitis image classification model includes: extracting global semantic information of fungal lesion areas at different levels in the high-resolution corneal image to be classified by the preset convolutional neural network of the preset keratitis image classification model to obtain image global features; extracting local features of the image global features by the preset local feature extraction module connected to the preset convolutional neural network to obtain multi-scale image local features including hypha branching angles and hypha terminal morphology; performing orthogonal fusion processing on the image global features and the multi-scale image local features by the preset keratitis image classification model to obtain multi-scale joint features, and inputting the multi-scale joint features into a classifier so that the classifier can map the multi-scale joint features to a Fusarium image feature category space and an Aspergillus image feature category space, respectively; calculating the probability values of the multi-scale joint features in the Fusarium image feature category space and the Aspergillus image feature category space, respectively, and outputting the maximum probability value. It is understood that during the classification and prediction process, the preset keratitis image classification model first extracts global semantic information from the fungal lesion area at various levels and extracts local features at different levels from the high-resolution corneal image to be classified. Specifically, the preset convolutional neural network performs hierarchical global semantic information extraction on the high-resolution corneal image to obtain global image features. Preset local feature extraction modules connected to each preset convolutional neural network further extract local features from the global image features. Because the preset convolutional neural network gradually compresses the spatial resolution of the feature map through hierarchical downsampling (such as pooling operations), microscopic details (such as hyphae terminal morphology) are blurred or lost. Furthermore, the receptive field of the preset convolutional neural network covers a large area and focuses more on global context (such as hyphae network structure) rather than local microscopic features. Therefore, the detailed information in the image is accurate, and the receptive field is further expanded without reducing the resolution by presetting the local feature extraction module, and multi-scale local features (such as hyphae branching angles and bifurcation points) are captured. In this way, feature extraction of the high-resolution corneal image to be classified is achieved from different scales, and more comprehensive feature information is obtained. Then, through orthogonal fusion processing such as in the model training process, the fused multi-scale joint features are obtained, and they are input into the two different category spaces mentioned during training, and their respective probability values are calculated, and the maximum probability value is output.

[0034] In this embodiment, the preset convolutional neural network includes several convolution modules, each of which is connected to a corresponding preset local feature extraction module. Accordingly, the preset convolutional neural network of the preset keratitis image classification model extracts global semantic information of fungal lesion areas at different levels in the high-resolution corneal image to be classified to obtain global image features; the local features of the image global features are extracted by the preset local feature extraction module connected to the preset convolutional neural network to obtain multi-scale image local features including hyphae branching angles and hyphae terminal morphology, including: extracting edge and texture features of the high-resolution corneal image to be classified by the first convolution module of the preset convolutional neural network, and Locate the initial lesion area on the corneal surface to obtain shallow global semantic information; wherein the edge and texture features include: hyphae contour, hyphae edge direction, and texture distribution; extract the hyphae local morphology of the shallow global semantic information through a first preset local feature extraction module connected to the first convolution module to obtain shallow image local features; perform orthogonal fusion processing on the shallow global semantic information and the shallow image local features through the preset keratitis image classification model to obtain shallow multi-scale joint features; input the shallow multi-scale joint features into a second convolution module connected to the first preset local feature extraction module to extract the hyphae distribution pattern in the shallow multi-scale joint features through the second convolution module , and extract the geometric shape and spatial range of the lesion area to obtain middle-level global semantic information; wherein, the hyphae distribution pattern includes hyphae density and hyphae arrangement direction; the spatial arrangement pattern of hyphae in the middle-level global semantic information is extracted by a second preset local feature extraction module connected to the second convolution module to obtain middle-level image local features; the middle-level global semantic information and the middle-level image local features are orthogonally fused by the preset keratitis image classification model to obtain middle-level multi-scale joint features; the middle-level multi-scale joint features are input into a third convolution module connected to the second preset local feature extraction module to extract the hyphae network in the middle-level multi-scale joint features by the third convolution module structure, lesion area semantic information and global context information to obtain high-level global semantic information; wherein, the hyphae network structure is used to describe the branching pattern and connection relationship of the hyphae; the lesion area semantic information is semantic category information including hyphae infection area, inflammatory response area and normal tissue area; the global context information is used to describe the spatial relationship between the lesion area and the surrounding tissue; the three-dimensional spatial relationship of the hyphae in the high-level global semantic information is extracted by a third preset local feature extraction module connected to the third convolution module to obtain high-level image local features; the high-level global semantic information and the high-level image local features are orthogonally fused by the preset keratitis image classification model to obtain the target multi-scale joint features.

[0035] like Figure 2 As shown, the preset convolutional neural network includes several convolution modules, which are generally divided into three levels of convolution modules, including: a first convolution module, a second convolution module and a third convolution module, wherein the first convolution module is used to extract the hyphae edge and texture features of the shallow layer in the high-resolution corneal image to be classified, and locate the initial lesion area on the surface of the corneal image, such as: hyphae aggregation area, inflammatory reaction area; the second convolution module is used to receive the shallow multi-scale joint features extracted and fused by the first convolution module and the first preset local feature extraction module, and then execute the global feature extraction process of the middle stage, at which time the hyphae distribution pattern and lesion are further extracted. The third convolutional module receives the fused mid-level multi-scale joint features extracted by the second convolutional module and the second preset local feature extraction module. It then performs the deep-level global feature extraction process, extracting the hyphal network structure (describing the branching pattern, connectivity, and overall network structure of the hyphae (e.g., thick straight branches of Fusarium vs. sharp-angled branches of Aspergillus)), semantic information about the lesion region (hyphae infection area, inflammatory response area, and normal tissue area), and global contextual information (combining full-thickness corneal information to describe the spatial relationship between the lesion region and surrounding tissue (e.g., hyphal penetration depth and lesion spread range)), i.e., deep-level multi-scale joint features. This process gradually extracts lesion information from the shallow to the deep layers, from local to global, forming a hierarchical feature representation. It is important to note that each convolutional module can contain multiple ConvNext V2 networks, without specific restrictions.

[0036] In this embodiment, the preset local feature extraction module includes a dilated convolution layer and a self-attention mechanism layer. Accordingly, the preset local feature extraction module connected to the preset convolutional neural network extracts local features of the global image features to obtain multi-scale local image features including hyphae branching angles and hyphae terminal morphology, including: extracting multi-scale local lesion features on the global image features that match the current level convolution module according to a preset dilation rate through the dilated convolution layer; and dynamically assigning weights to the multi-scale local lesion features using the self-attention mechanism layer to obtain multi-scale local image features including hyphae branching angles and hyphae terminal morphology. It is understandable that since the preset local feature extraction module corresponds to the preset convolutional neural network, from shallow to deep layers, they are also the first preset local feature extraction module, the second preset local feature extraction module, and the third preset local feature extraction module, each of which includes a dilated convolution layer and a self-attention mechanism layer. Accordingly, the local feature extraction process of the first preset local feature extraction module, the second preset local feature extraction module, and the third preset local feature extraction module is as follows: like Figure 2As shown, the first preset local feature extraction module receives the global feature tensor output by the first convolution module (such as the 1st to 3rd layers), which contains low-level visual information (such as hyphae edges, textures, and local lesion contours). The atrous convolution layer uses a convolution kernel with a small dilation rate (such as r = 1, 2) to expand the receptive field to cover the local lesion area in the shallow global semantic information, but avoids excessive blurring of details. It outputs multiple sets of feature maps (first multi-scale local lesion features) to capture the local morphology of hyphae (such as the starting point of the branch and the end morphology). Through the self-attention mechanism layer, it focuses on the high-contrast areas of the first multi-scale local lesion features (such as the junction between hyphae and normal tissue) in the spatial dimension, suppresses background noise (such as corneal epithelial cell fragments), and outputs the local features of the shallow image.

[0037] like Figure 2 As shown, the second preset local feature extraction module receives the global feature tensor output by the second convolutional module (e.g., layers 4-6), which contains mid-level semantic information (e.g., hyphae distribution density and lesion area morphology). The dilated convolution layer uses a kernel with a moderate dilation rate (e.g., r = 2 or 3) to expand the receptive field to cover the hyphae cluster area and extract a wider range of local context (e.g., branching connectivity patterns). The output feature map (the second multi-scale local lesion feature) describes the spatial arrangement of hyphae (e.g., parallel arrangement or random distribution). Furthermore, the self-attention mechanism layer dynamically assigns weights to hyphae bifurcation points and dense regions in the second multi-scale local lesion feature, enhancing the ability to distinguish between Fusarium (sparse branches) and Aspergillus (dense, sharp branches). Spatial attention is used to suppress non-lesion areas (e.g., normal fiber structures in the corneal stroma).

[0038] like Figure 2 As shown, the third preset local feature extraction module receives the global feature tensor output by the third convolutional module (e.g., layers 7-9), which contains high-level semantic information (e.g., hyphal network structure and global lesion context). The dilated convolutional layer uses a kernel with a large dilation rate (e.g., r = 3 or 4) to cover the entire structure of the hyphal network and extract long-range dependencies (e.g., hyphal penetration depth and cross-layer distribution). The output feature map (the third multi-scale local lesion feature) describes the three-dimensional spatial relationships of the hyphae (e.g., hyphal diffusion paths within the deep matrix). The self-attention mechanism layer focuses on key regions of the third multi-scale local lesion feature (e.g., hyphal infection centers and deep penetration points) and combines channel attention to enhance discriminative features (e.g., the sharp-angle branching pattern of Aspergillus).

[0039] In this way, the dilated convolution layer uses convolution kernels with different expansion rates to expand the receptive field without reducing the resolution, capturing multi-scale local details (such as hyphae branching angles and bifurcation points). The output local feature map retains the sharp angles and terminal details of the hyphae branches. The working steps of the self-attention mechanism layer are as follows: The multi-scale local lesion features are convolved to obtain corresponding query matrices, key matrices, and value matrices; an attention weight matrix is calculated based on the query matrix, the key matrix, and the value matrix to obtain a weighted feature tensor based on the attention weight matrix and the value matrix; the weighted feature tensor is added to the multi-scale local lesion features to generate multi-scale image local features including hyphae branching angles and hyphae terminal morphologies. It can be understood that through three independent 1×1 convolutional layers, the multi-scale local lesion features are mapped into a query matrix (Query, Q), a key matrix (Key, K) and a value matrix (Value, V). Furthermore, based on the dot product between the query matrix and the key matrix, a similarity matrix is obtained to measure the correlation between different positions in the multi-scale local lesion features. The similarity matrix is normalized by the Softmax function to generate an attention weight matrix. The attention weight matrix is multiplied by the value matrix to generate a dynamically weighted feature representation (weighted feature tensor). In this way, the features of key areas (such as hyphae bifurcation points and terminal morphology) are enhanced and background noise is suppressed. Finally, the weighted feature tensor is added to the multi-scale local lesion features to retain the original detail information and generate multi-scale image local features.

[0040] As can be seen, the self-attention mechanism layer focuses on key regions of hyphae, clearly distinguishing the different fungal features of the two fungi. Specifically, the high-weighted regions highlight the blunt-ended morphology and sparse branching of Fusarium's thick, straight hyphae. It also emphasizes the local features of Aspergillus' sharp branch points and densely interwoven networks. Simultaneously, it reduces the weight of noisy regions such as corneal epithelial cell debris and reflections. The attention region is automatically adjusted for different input images, improving the model's generalization ability.

[0041] like Figure 3 As shown, the present invention discloses a specific ophthalmic image classification method based on global and local feature extraction. Specifically, confocal microscope corneal images that conform to the distribution of real clinical process data are collected to construct a training dataset. The images are from patient corneal images taken during the clinical diagnosis process. The image data is labeled and divided on a patient-by-patient basis, combined with the patient's actual clinical diagnosis, to construct a training dataset for training and testing. The preprocessed corneal images in the training dataset are then input into a ConvNeXt V2 network for feature extraction. The specific steps are as follows: A local feature extraction module is added after each layer of the ConvNeXt V2 network, and the extracted features are overlapped to obtain the local feature tensor , and use the features obtained in the last layer of the network as global features . Among them, the local feature extraction module consists of a hole convolution layer and a self-attention mechanism layer. The hole convolution layer compensates for the information loss caused by downsampling by increasing the receptive field of the convolution kernel of the same size using the expansion rate parameter. The self-attention mechanism layer here mainly simulates the importance of each feature map from the local branch. The hole convolution layer module consists of three expanded convolution layers to obtain feature maps with different spatial receptive fields and global average pooling branches. These features are concatenated and then processed by a 1x1 convolution layer. The output feature map is fed into the self-attention module to further establish the importance model of each local feature point. Specifically, its input is first processed using a 1x1 convolution layer, and then the attention map generated by the 1x1 convolution layer is normalized and adjusted according to subsequent functions. Orthogonal fusion is used to aggregate the global feature tensor and local feature tensor obtained by the local branch. Using global and local features as input, each local feature point is Calculate globally Feature Points The calculation formula is as follows: ; in, Represents the local feature tensor output from the local feature extraction module In spatial position The eigenvector at represents the global feature tensor, Represents the enhanced local feature vector generated by orthogonal fusion, located at the spatial position Finally, the global features and local features are combined through orthogonal fusion to obtain a final feature map which is input into the classifier for classification and loss calculation.

[0042] In this way, since it is difficult to extract rich lesion information with simple feature extraction, especially for fungal keratitis, the hyphae structure is complex and the features are similar, it is necessary to mine deep multi-scale feature information in order to achieve a better classification task. Therefore, the local feature extraction method is added to the ConvNeXt V2 network to focus on the detail features ignored by the global feature extraction. After fusing the global features and local features, the deep and shallow features and the global and local features of the lesion area in the image can be fully mined. Compared with the existing methods, the number of misdiagnoses and missed diagnoses is significantly reduced, which is more conducive to improving the detection accuracy at the patient level. In summary, the present invention uses a better image feature extraction network to distinguish the hyphae structure, so that the classification accuracy of Fusarium keratitis and Aspergillus keratitis is significantly improved.

[0043] As can be seen, the present application discloses an ophthalmic image classification method based on global and local feature extraction, which is applied to a computer device and comprises: obtaining a high-resolution corneal image to be classified; the high-resolution corneal image to be classified is a corneal image containing the microstructure of hyphae; inputting the high-resolution corneal image to be classified into a preset keratitis image classification model, so that the preset keratitis image classification model classifies the high-resolution corneal image to be classified as a fungal image and outputs a corresponding Aspergillus keratitis image classification result or a Fusarium keratitis image classification result; wherein the preset keratitis image classification model is an image classification model that has learned multi-scale fungal feature extraction capabilities and fuses global and local image features to perform image classification. As can be seen, by using the high-resolution corneal image to be classified, the microstructure of hyphae (such as branching morphology and terminal features) can be clearly captured, improving the input data quality of the classification model. The classification model combines the high-resolution confocal image with the multi-scale feature fusion to automatically analyze the corneal image, reduce manual subjective judgment errors, and improve the accuracy of the sub-classification of the two types of fungi. The model data results can be used to distinguish Aspergillus (slender, sharp-angled branches) from Fusarium (thick, straight, and sparse branches), reducing the misdiagnosis rate.

[0044] Reference Figure 4 As shown, the present invention also discloses an ophthalmic image classification device based on global and local feature extraction, comprising: An image acquisition module 11 is configured to acquire a high-resolution corneal image to be classified; the high-resolution corneal image to be classified is a corneal image containing a microscopic structure of hyphae; The image classification module 12 is used to input the high-resolution corneal image to be classified into a preset keratitis image classification model, so as to perform fungal image classification on the high-resolution corneal image to be classified through the preset keratitis image classification model, and output the corresponding Aspergillus keratitis image classification result or Fusarium keratitis image classification result; wherein, the preset keratitis image classification model is an image classification model that has learned multi-scale fungal feature extraction capabilities, integrated image global features and image local features, and performed image classification.

[0045] As can be seen, the present application discloses a method for obtaining a high-resolution corneal image to be classified; the high-resolution corneal image to be classified is an image of a corneal image containing the microstructure of hyphae; and inputting the high-resolution corneal image to be classified into a preset keratitis image classification model, so that the preset keratitis image classification model performs fungal image classification on the high-resolution corneal image to be classified and outputs a corresponding Aspergillus keratitis image classification result or a Fusarium keratitis image classification result. The preset keratitis image classification model is an image classification model that has learned multi-scale fungal feature extraction capabilities and fuses global and local image features to perform image classification. Thus, by using the high-resolution corneal image to be classified, the microstructure of hyphae (such as branching morphology and terminal features) can be clearly captured, improving the input data quality of the classification model. The classification model combines high-resolution confocal images with multi-scale feature fusion to automatically analyze the corneal image, reducing manual subjective judgment errors and improving the accuracy of the detailed classification of the two fungi. The model data results can distinguish Aspergillus (slender, sharp-angled branches) from Fusarium (coarse, straight, and sparse branches), reducing the misdiagnosis rate.

[0046] Furthermore, the embodiment of the present application also discloses an electronic device, Figure 5 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content in the diagram should not be considered as any limitation to the scope of application of the present application.

[0047] Figure 5 This is a schematic diagram of the structure of an electronic device 20 provided in an embodiment of the present application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 is used to store a computer program, which is loaded and executed by the processor 21 to implement the relevant steps of the ophthalmic image classification method based on global-local feature extraction disclosed in any of the aforementioned embodiments. Furthermore, the electronic device 20 in this embodiment may specifically be an electronic computer.

[0048] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and the external device. The communication protocol it follows is any communication protocol that can be applied to the technical solution of this application and is not specifically limited here; the input and output interface 25 is used to obtain external input data or output data to the outside world. Its specific interface type can be selected according to specific application needs and is not specifically limited here.

[0049] Among them, the processor 21 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 21 can be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 21 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 21 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 21 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0050] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or CD, etc. The resources stored thereon can include an operating system 221, a computer program 222, etc., and the storage method can be temporary storage or permanent storage.

[0051] The operating system 221 is used to manage and control the hardware devices and computer program 222 on the electronic device 20, enabling the processor 21 to calculate and process the massive amount of data 223 in the memory 22. It can be Windows Server, NetWare, Unix, Linux, etc. In addition to including computer programs capable of implementing the ophthalmic image classification method based on global and local feature extraction performed by the electronic device 20 as disclosed in any of the aforementioned embodiments, the computer program 222 may further include computer programs capable of performing other specific tasks. Data 223 may include data received by the electronic device from external devices as well as data collected by its own input and output interface 25.

[0052] Furthermore, this application discloses a computer-readable storage medium for storing a computer program; wherein, when executed by a processor, the computer program implements the aforementioned method for ophthalmic image classification based on global-local feature extraction. The specific steps of this method can be found in the corresponding contents disclosed in the aforementioned embodiments and will not be further described here.

[0053] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from the other embodiments. Reference can be made to the descriptions of the identical or similar parts between the various embodiments. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple, and the relevant parts can be referred to the descriptions of the methods.

[0054] Professionals may further appreciate that the units and algorithmic steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application. The steps of the method or algorithm described in conjunction with the embodiments disclosed herein can be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module can be placed in a random access memory RAM (Random Access Memory), memory, read-only memory ROM (Read Only Memory), electrically programmable EPROM (Electrically Programmable Read Only Memory), electrically erasable programmable EEPROM (Electric Erasable Programmable Read Only Memory), registers, hard disk, removable disk, CD-ROM (Compact Disc-Read Only Memory), or any other form of storage medium known in the technical field.

[0055] Finally, it should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or device comprising the element.

[0056] The above is a detailed introduction to the solution provided by the present invention. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core ideas. At the same time, for those skilled in the art, according to the ideas of the present invention, there may be changes in the specific implementation methods and application scopes. In summary, the contents of this specification should not be understood as limiting the present invention.

Claims

1. A method for ophthalmic image classification based on global and local feature extraction, characterized in that: Applicable to computer devices, including: Acquire a high-resolution corneal image to be classified; the high-resolution corneal image to be classified is a corneal image containing a microscopic structure of hyphae; The high-resolution corneal image to be classified is input into a preset keratitis image classification model so that the high-resolution corneal image to be classified is classified as a fungal image by the preset keratitis image classification model, and the corresponding Aspergillus keratitis image classification result or the Fusarium keratitis image classification result is output; wherein, the preset keratitis image classification model is an image classification model that has learned multi-scale fungal feature extraction capabilities, integrated image global features and image local features, and performed image classification.

2. The ophthalmic image classification method based on global and local feature extraction according to claim 1, characterized in that: The preset keratitis image classification model includes a preset convolutional neural network and a preset local feature extraction module. Accordingly, the fungus image classification of the high-resolution corneal image to be classified by the preset keratitis image classification model includes: Extracting global semantic information of fungal lesion areas at different levels in the high-resolution corneal image to be classified through the preset convolutional neural network of the preset keratitis image classification model to obtain global image features; Extracting local features of the global features of the image by the preset local feature extraction module connected to the preset convolutional neural network to obtain multi-scale local features of the image including hypha branching angles and hypha terminal morphology; Performing orthogonal fusion processing on the global features of the image and the local features of the multi-scale image by using the preset keratitis image classification model to obtain multi-scale joint features, and inputting the multi-scale joint features into a classifier so as to map the multi-scale joint features to a Fusarium image feature category space and an Aspergillus image feature category space, respectively, by the classifier; The probability values of the multi-scale joint features in the Fusarium image feature category space and the Aspergillus image feature category space are calculated respectively, and the maximum probability value is output.

3. The ophthalmic image classification method based on global and local feature extraction according to claim 2, characterized in that: The preset convolutional neural network includes several convolution modules, each of which is connected to a corresponding preset local feature extraction module. Accordingly, the preset convolutional neural network of the preset keratitis image classification model extracts global semantic information of fungal lesion areas at different levels in the high-resolution corneal image to be classified to obtain global image features; the preset local feature extraction module connected to the preset convolutional neural network extracts local features of the global image features to obtain multi-scale image local features including hyphae branching angles and hyphae terminal morphology, including: Extracting edge and texture features of the high-resolution corneal image to be classified using the first convolution module of the preset convolutional neural network, and locating the initial lesion area on the corneal surface to obtain shallow global semantic information; wherein the edge and texture features include hyphae outline, hyphae edge direction, and texture distribution; Extracting the local morphology of mycelium from the shallow global semantic information through a first preset local feature extraction module connected to the first convolution module to obtain shallow image local features; Performing orthogonal fusion processing on the shallow global semantic information and the shallow image local features through the preset keratitis image classification model to obtain shallow multi-scale joint features; Inputting the shallow multi-scale joint features into a second convolution module connected to the first preset local feature extraction module, so as to extract the hyphae distribution pattern in the shallow multi-scale joint features through the second convolution module, and extract the geometric shape and spatial extent of the lesion area to obtain mid-level global semantic information; wherein the hyphae distribution pattern includes hyphae density and hyphae arrangement direction; extracting the spatial arrangement regularity of hyphae in the middle-layer global semantic information through a second preset local feature extraction module connected to the second convolution module to obtain local features of the middle-layer image; Performing orthogonal fusion processing on the middle-layer global semantic information and the middle-layer image local features through the preset keratitis image classification model to obtain middle-layer multi-scale joint features; The middle-level multi-scale joint features are input into a third convolution module connected to the second preset local feature extraction module, so as to extract the hyphae network structure, lesion area semantic information and global context information in the middle-level multi-scale joint features through the third convolution module to obtain high-level global semantic information; wherein the hyphae network structure is used to describe the branching pattern and connection relationship of the hyphae; the lesion area semantic information is semantic category information including hyphae infection area, inflammatory response area, and normal tissue area; and the global context information is used to describe the spatial relationship between the lesion area and surrounding tissues; extracting the three-dimensional spatial relationship of hyphae in the high-level global semantic information through a third preset local feature extraction module connected to the third convolution module to obtain high-level image local features; The high-level global semantic information and the high-level image local features are orthogonally fused using the preset keratitis image classification model to obtain target multi-scale joint features.

4. The ophthalmic image classification method based on global and local feature extraction according to claim 2, characterized in that: The preset local feature extraction module includes a hole convolution layer and a self-attention mechanism layer. Accordingly, the preset local feature extraction module connected to the preset convolutional neural network extracts local features of the global image features to obtain multi-scale image local features including hypha branching angles and hypha terminal morphology, including: Extracting multi-scale local lesion features on the global features of the image that match the current hierarchical convolution module through the dilated convolution layer according to a preset expansion rate; The self-attention mechanism layer is used to dynamically assign weights to the multi-scale local lesion features, and obtain multi-scale image local features including hyphae branching angles and hyphae terminal morphology.

5. The ophthalmic image classification method based on global and local feature extraction according to claim 4, characterized in that: The method of dynamically assigning weights to the multi-scale local lesion features using the self-attention mechanism layer and obtaining multi-scale image local features including hyphae branching angles and hyphae terminal morphologies includes: Performing convolution processing on the multi-scale local lesion features to obtain corresponding query matrix, key matrix and value matrix; Calculating an attention weight matrix based on the query matrix, the key matrix, and the value matrix to obtain a weighted feature tensor based on the attention weight matrix and the value matrix; The weighted feature tensor and the multi-scale local lesion feature are added together to generate a multi-scale image local feature including hypha branching angle and hypha terminal morphology.

6. The ophthalmic image classification method based on global-local feature extraction according to any one of claims 1 to 5, characterized in that: Before inputting the high-resolution corneal image to be classified into a preset keratitis image classification model, the method further includes: Constructing a data training set based on several historical high-resolution corneal images carrying real keratitis labels; wherein the real keratitis labels are Fusarium keratitis labels or Aspergillus keratitis labels; Using the data training set, an initial keratitis image classification model including an initial convolutional neural network and an initial local feature extraction module is trained, global features are extracted by the initial convolutional neural network, and multi-scale local features are extracted by the initial local feature extraction module, and the global features and the multi-scale local features are orthogonally fused to obtain fused features; Input the fusion features into the classifier and output the prediction result; A cross entropy loss value between the prediction result and the true keratitis label is calculated, and model parameters of the initial keratitis image classification model are reversely optimized based on the cross entropy loss value to obtain a preset keratitis image classification model.

7. The ophthalmic image classification method based on global and local feature extraction according to claim 6, characterized in that: Before constructing a data training set based on a plurality of historical high-resolution corneal images carrying real keratitis labels, the method further includes: Acquiring a fungal keratitis image that meets a preset high-resolution condition and is captured by a confocal microscope, and performing standardized preprocessing and data enhancement processing on the fungal keratitis image that meets the preset high-resolution condition to obtain an enhanced high-resolution fungal keratitis image; the data enhancement processing includes any one or more of random horizontal flipping, rotation, and brightness adjustment; The enhanced high-resolution fungal keratitis image is labeled with corresponding Fusarium keratitis or Aspergillus keratitis to obtain a historical high-resolution corneal image carrying a real keratitis label.

8. An ophthalmic image classification device based on global and local feature extraction, characterized in that: include: An image acquisition module is used to acquire a high-resolution corneal image to be classified; the high-resolution corneal image to be classified is a corneal image containing a microscopic structure of hyphae; An image classification module is used to input the high-resolution corneal image to be classified into a preset keratitis image classification model, so as to perform fungal image classification on the high-resolution corneal image to be classified through the preset keratitis image classification model, and output the corresponding Aspergillus keratitis image classification result or Fusarium keratitis image classification result; wherein, the preset keratitis image classification model is an image classification model that has learned multi-scale fungal feature extraction capabilities, integrated image global features and image local features, and performed image classification.

9. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor, configured to execute the computer program to implement the steps of the ophthalmic image classification method based on global-local feature extraction as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that Used to store computer programs; wherein, when the computer program is executed by a processor, the steps of the ophthalmic image classification method based on global-local feature extraction as described in any one of claims 1 to 7 are implemented.

Images