Glaucoma fundus image classification method and imaging method

Abstract

The application discloses a glaucoma fundus image classification method, which comprises the following steps: acquiring existing glaucoma fundus image data information, performing classification labeling and preprocessing to obtain a training data set; constructing a glaucoma fundus image classification initial model based on a ConvNeXt model and training to obtain a glaucoma fundus image classification model; and using the obtained glaucoma fundus image classification model to classify actual glaucoma fundus images. The application also discloses an imaging method comprising the glaucoma fundus image classification method. The application constructs a glaucoma fundus image classification network with a feature fusion process through a ConvNeXt model, so that the application can not only realize the classification of glaucoma fundus images, but also be suitable for the classification of single-eye images and double-eye images, and has high reliability, good accuracy, wide application range, and is more objective and scientific.

Description

Technical Field

[0001] The present invention belongs to the field of image processing, and in particular relates to a glaucoma fundus image classification method and an imaging method. Background Art

[0002] With the development of economy and technology and the improvement of people's living standards, people are paying more and more attention to their health. Glaucoma is a common eye disease; therefore, medical research on glaucoma has always been one of the research hotspots.

[0003] Fundus image classification for glaucoma is crucial both in clinical and laboratory research. Traditional glaucoma fundus image classification methods rely on manual classification, where medical professionals draw on their medical knowledge, expertise, and experience to manually classify glaucoma fundus images. However, this manual classification approach is not only time-consuming and labor-intensive, but also inefficient and unreliable.

[0004] With the rapid development of neural network technology, researchers have applied it to the classification of glaucoma fundus images. These approaches are generally categorized as machine learning and deep learning. However, traditional machine learning approaches rely on artificial feature extraction, which is susceptible to human interference. While deep learning approaches have achieved some success, their generalization capabilities are poor, and they still suffer from poor reliability and accuracy in specific applications. Summary of the Invention

[0005] One of the purposes of the present invention is to provide a glaucoma fundus image classification method that is highly reliable, accurate, objective and scientific.

[0006] A second object of the present invention is to provide an imaging method including the glaucoma fundus image classification method.

[0007] The glaucoma fundus image classification method provided by the present invention comprises the following steps:

[0008] S1. Obtain existing glaucoma fundus image data information and classify and annotate it;

[0009] S2. Preprocess the data information obtained in step S1 to obtain a training data set;

[0010] S3. Build an initial model for glaucoma fundus image classification based on the ConvNeXt model;

[0011] S4. Using the training data set obtained in step S2, the initial glaucoma fundus image classification model constructed in step S3 is trained to obtain a glaucoma fundus image classification model;

[0012] S5. Using the glaucoma fundus image classification model obtained in step S4, the actual glaucoma fundus image is classified.

[0013] The data information obtained in step S1 is preprocessed to obtain a training data set, specifically including the following steps:

[0014] For the fundus image data obtained in step S1, it is detected whether there is an optic disc region:

[0015] If there is no optic disc region, the detection fails; the corresponding fundus image is directly deleted;

[0016] If there is an optic disc region, the detection passes; the square boundary region of the optic disc region is calculated, the corresponding fundus image is cropped to a set size, and the corresponding optic disc region is extracted;

[0017] All fundus images that pass the detection and are cropped are judged and classified:

[0018] All monocular images are formed into a monocular image data training set;

[0019] All binocular images are paired to form a binocular image data training set.

[0020] The pre-trained YOLOv5 model is used to detect whether there is an optic disc region; for the fundus image with an optic disc region, the pre-trained YOLOv5 model is used to extract the optic disc region to obtain the optic disc region image.

[0021] The initial model for classifying glaucoma fundus images based on the ConvNeXt model in step S3 includes the following steps:

[0022] Based on the ConvNeXt model and the pooling layer, an initial model for classifying glaucoma fundus images of monocular images is constructed;

[0023] Based on the ConvNeXt model, the pooling layer, the linear layer and the dropout layer, an initial model for classifying glaucoma fundus images of binocular images is constructed.

[0024] The initial model for classifying glaucoma fundus images of monocular images based on the ConvNeXt model and the pooling layer includes the following steps:

[0025] The processing process of the initial model for classifying glaucoma fundus images of monocular images includes the following steps:

[0026] The input images of the initial model for classifying monocular glaucoma fundus images are the global fundus image and the corresponding local optic disc image of the monocular image; the global fundus image is the monocular image, and the local optic disc image is the optic disc area image corresponding to the monocular image;

[0027] The global fundus image is input into the ConvNeXt model for processing, and the global image feature f is obtained through the activation function global ; Global image feature f global Expressed as

[0028] f global =φ(ConvNeXt(x global θ global ))

[0029] Where x global is the global fundus image; θ global are the parameters of the ConvNeXt model corresponding to the global fundus image; ConvNeXt() represents the processing process of the ConvNeXt model corresponding to the global fundus image; φ() represents the processing process of the activation function corresponding to the global fundus image;

[0030] The local optic disc image is input into the ConvNeXt model for processing, and the local image feature f is obtained through the activation function local ; Local image feature f local Expressed as

[0031] f local =φ(ConvNeXt'(x local θ local ))

[0032] Where x local is the local optic disc image; θ local are the parameters of the ConvNeXt model corresponding to the local optic disc image; ConvNeXt'() is the processing process of the ConvNeXt model corresponding to the local optic disc image; φ'() is the processing process of the activation function corresponding to the local optic disc image;

[0033] The obtained global image feature f global Through adaptive average pooling, it is converted into a global one-dimensional vector f g ' lobal f g ' lobal =AdaptiveAvgPool(f global ); AdaptiveAvgPool() represents the adaptive average pooling process;

[0034] The obtained local image feature f localThrough adaptive average pooling, it is converted into a local one-dimensional vector f l ' ocal f l ' ocal =AdaptiveAvgPool(f local );

[0035] The global one-dimensional vector f g ' lobal and the local one-dimensional vector f l ' ocal Connect and get the classification feature f cat f cat =Concat(f g ' lobal ,f l ' ocal ); Concat() represents the connection process;

[0036] The classification feature f cat After passing through the adaptive average pooling and fully connected layers in sequence, the classification result of the input monocular image is obtained.

[0037] The aforementioned method of constructing an initial model for classifying glaucoma fundus images of binocular images based on the ConvNeXt model, pooling layer, linear layer, and dropout layer specifically includes the following steps:

[0038] The processing of the initial model for classifying glaucoma fundus images of binocular images includes the following steps:

[0039] The input images of the initial model for glaucoma fundus image classification of binocular images are the left eye global fundus image and the left eye local optic disc image of the left eye image, and the right eye global fundus image and the right eye local optic disc image of the right eye image;

[0040] The left eye feature extraction network is constructed based on the ConvNeXt model, and the left eye global fundus image and the left eye local optic disc image are input into the left eye feature extraction network for processing to obtain the left eye preliminary feature f l ' eft ;

[0041] Based on the ConvNeXt model, a right eye feature extraction network is constructed, and the right eye global fundus image and the right eye local optic disc image are input into the right eye feature extraction network for processing to obtain the right eye preliminary feature f r ' ight ;

[0042] The obtained left eye preliminary feature f l ' eft Through adaptive average pooling processing, the left eye feature f is obtained left ;

[0043] The obtained preliminary feature f of the right eye r ' ight Through adaptive average pooling processing, the right eye feature f is obtained right ;

[0044] In the order of left eye features and right eye features, the left eye features f left and right eye feature f right Connect and get the left eye classification feature f left-cat f left-cat =Concat(f left ,f right );

[0045] In the order of right eye features and left eye features, the right eye features f right and left eye feature f left Connect and get the right eye classification feature f right-cat f right-cat =Concat(f right ,f left );

[0046] The left eye classification network is constructed based on the pooling layer, linear layer and dropout layer, and the left eye classification feature f left-cat Input into the left eye classification network for processing to obtain the final left eye image classification result;

[0047] The right eye classification network is constructed based on the pooling layer, linear layer and dropout layer, and the right eye classification feature f right-cat Input into the right eye classification network for processing to obtain the final right eye image classification result.

[0048] The left eye feature extraction network is constructed based on the ConvNeXt model, which specifically includes the following steps:

[0049] The left eye global fundus image is input into the ConvNeXt model for processing, and the left eye global image features are obtained through the activation function Left eye global image features Expressed as

[0050]

[0051] In the formula is the global fundus image of the left eye; are the parameters of the ConvNeXt model corresponding to the global fundus image of the left eye; Represents the processing process of the ConvNeXt model corresponding to the global fundus image of the left eye; The processing process of the activation function corresponding to the global fundus image of the left eye;

[0052] The left eye local optic disc image is input into the ConvNeXt model for processing, and the left eye local image features are obtained through the activation function Left eye local image features Expressed as

[0053]

[0054] In the formula It is the local optic disc image of the left eye; are the parameters of the ConvNeXt model corresponding to the local optic disc image of the left eye; The processing process of the ConvNeXt model corresponding to the local optic disc image of the left eye; The processing process of the activation function corresponding to the local optic disc image of the left eye;

[0055] The obtained left eye global image features Through adaptive average pooling, it is converted into a global one-dimensional vector for the left eye for

[0056] The obtained left eye local image features Converted into a local one-dimensional vector for the left eye through adaptive average pooling for

[0057] The left eye global one-dimensional vector and the left eye local one-dimensional vector Connect and get the preliminary feature f of the left eye l ' eft for

[0058] The right eye feature extraction network is constructed based on the ConvNeXt model, which specifically includes the following steps:

[0059] The right eye global fundus image is input into the ConvNeXt model for processing, and the right eye global image features are obtained through the activation function Right eye global image features Expressed as

[0060]

[0061] In the formula is the global fundus image of the right eye; are the parameters of the ConvNeXt model corresponding to the global fundus image of the right eye; Represents the processing process of the ConvNeXt model corresponding to the global fundus image of the right eye; The processing process of the activation function corresponding to the global fundus image of the right eye;

[0062] The right eye local optic disc image is input into the ConvNeXt model for processing, and the right eye local image features are obtained through the activation function Right eye local image features Expressed as

[0063]

[0064] In the formula This is the partial optic disc image of the right eye; are the parameters of the ConvNeXt model corresponding to the local optic disc image of the right eye; The processing process of the ConvNeXt model corresponding to the local optic disc image of the right eye; The processing process of the activation function corresponding to the local optic disc image of the right eye;

[0065] The obtained right eye global image features Converted into a global one-dimensional vector for the right eye through adaptive average pooling for

[0066] The obtained right eye local image features Converted into a local one-dimensional vector for the right eye through adaptive average pooling for

[0067] The right eye global one-dimensional vector and the right eye local one-dimensional vector Connect and get the preliminary feature f of the right eye r ' ight for

[0068] The left-eye classification network is constructed based on the pooling layer, the linear layer, and the dropout layer, and specifically includes the following steps:

[0069] The left eye classification feature f left-cat After feature extraction through the linear layer, processing through the maximum pooling layer, activation through the Relu function, and finally processing through the dropout layer, the first feature f of the left eye classification is obtained. left-cat-1 ;

[0070] Classify the left eye with the first feature f left-cat-1 The feature extraction is performed again through the linear layer, processed through the maximum pooling layer, activated by the Relu function, and finally processed through the dropout layer to obtain the second feature f of the left eye classification. left-cat-2 ;

[0071] The left eye classification second feature f left-cat-2 The final left eye image classification result is obtained by processing through the linear layer.

[0072] The right eye classification network is constructed based on the pooling layer, the linear layer and the dropout layer, and specifically includes the following steps:

[0073] The right eye classification feature f right-cat The right eye classification first feature f right-cat-1 is obtained by performing feature extraction through the linear layer, processing through the max pooling layer, activating through the Relu function, and processing through the dropout layer.

[0074] The right eye classification second feature f right-cat-1 The right eye classification second feature f right-cat-2 is obtained by performing feature extraction through the linear layer again, processing through the max pooling layer, activating through the Relu function, and processing through the dropout layer.

[0075] The right eye classification second feature f right-cat-2 The final right eye image classification result is obtained by processing through the linear layer.

[0076] The training in step S4 specifically includes the following steps:

[0077] For the initial model of glaucoma fundus image classification of monocular images, the following formula is used as the loss function L during training:

[0078]

[0079] In the formula, the value of i is global, local or cat; w i is the corresponding weight parameter; L i (o i ,y) is the corresponding label smoothed cross-entropy loss function; y is the real label; o i is the predicted probability, and o i =Softmax(W i f i +b i ), Softmax() is the softmax function, W i is the weight matrix, and b i is the bias term.

[0080] For the initial model of glaucoma fundus image classification of binocular images, the cross-entropy loss function is used as the loss function.

[0081] The present invention also provides an imaging method including the glaucoma fundus image classification method, further comprising the following steps:

[0082] S6. Marking and re-imaging the obtained glaucoma fundus image classification result on the actual glaucoma fundus image to obtain a glaucoma fundus image with the glaucoma fundus image classification result.

[0083] The glaucoma fundus image classification method and imaging method provided by the present invention construct a glaucoma fundus image classification network with a feature fusion process through a ConvNeXt model. Therefore, the present invention can not only realize the classification of glaucoma fundus images, but also is applicable to the classification of monocular images and binocular images. It is not only highly reliable and accurate, but also has a wide range of applications and is more objective and scientific. BRIEF DESCRIPTION OF THE DRAWINGS

[0084] Figure 1 Schematic diagram of the process flow of the classification method of the present invention.

[0085] Figure 2 Schematic diagram of the imaging method of the present invention. DETAILED DESCRIPTION

[0086] like Figure 1 The figure shows a flow chart of the classification method of the present invention: The glaucoma fundus image classification method provided by the present invention comprises the following steps:

[0087] S1. Obtain existing glaucoma fundus image data information and classify and annotate it;

[0088] S2. Preprocess the data information obtained in step S1 to obtain a training data set; specifically, the steps include:

[0089] For the fundus image data obtained in step S1, detect whether there is an optic disc area:

[0090] If the optic disc area does not exist, the test fails and the corresponding fundus image is directly deleted;

[0091] If the optic disc area exists, it is detected; the square boundary area of ​​the optic disc area is calculated (the Hough circle transform is used to crop the field of view, and the shorter edges are filled with zeros to adjust the image size to ensure that the width and height of the cropped area are balanced), the corresponding fundus image is cropped to a set size (preferably 512*512), and the corresponding optic disc area is extracted;

[0092] All fundus images that have passed detection and cropping are judged and classified:

[0093] After data enhancement, all monocular images are used to form a monocular image data training set;

[0094] Pair all binocular images and perform data enhancement to form a binocular image data training set;

[0095] In specific implementation, data augmentation includes random rotation, horizontal flipping, and vertical flipping;

[0096] In the specific implementation, a pre-trained YOLOv5 model is used to detect whether an optic disc area exists; for a fundus image with an optic disc area, the pre-trained YOLOv5 model is used to extract the optic disc area to obtain an optic disc area image;

[0097] S3. Based on the ConvNeXt model, construct an initial model for glaucoma fundus image classification; the steps include:

[0098] Based on the ConvNeXt model and pooling layer, an initial model for monocular glaucoma fundus image classification is constructed; specifically, the following steps are involved:

[0099] The input images of the initial model for classifying monocular glaucoma fundus images are the global fundus image and the corresponding local optic disc image of the monocular image; the global fundus image is the monocular image, and the local optic disc image is the optic disc area image corresponding to the monocular image;

[0100] The global fundus image is input into the ConvNeXt model for processing, and the global image feature f is obtained through the activation function global ; Global image feature f global Expressed as

[0101] f global =φ(ConvNeXt(x global θ global ))

[0102] Where x global is the global fundus image; θ global are the parameters of the ConvNeXt model corresponding to the global fundus image; ConvNeXt() represents the processing process of the ConvNeXt model corresponding to the global fundus image; φ() represents the processing process of the activation function corresponding to the global fundus image;

[0103] The local optic disc image is input into the ConvNeXt model for processing, and the local image feature f is obtained through the activation function local ; Local image feature f local Expressed as

[0104] f local =φ(ConvNeXt'(x local θ local ))

[0105] Where x local is the local optic disc image; θ local are the parameters of the ConvNeXt model corresponding to the local optic disc image; ConvNeXt'() is the processing process of the ConvNeXt model corresponding to the local optic disc image; φ'() is the processing process of the activation function corresponding to the local optic disc image;

[0106] The obtained global image feature f global Through adaptive average pooling, it is converted into a global one-dimensional vector f g ' lobal f g ' lobal =AdaptiveAvgPool(f global ); AdaptiveAvgPool() represents the adaptive average pooling process;

[0107] The obtained local image feature f local Through adaptive average pooling, it is converted into a local one-dimensional vector f l ' ocal f l ' ocal =AdaptiveAvgPool(f local );

[0108] The global one-dimensional vector f g ' lobal and the local one-dimensional vector f l ' ocal Connect and get the classification feature f cat f cat =Concat(f g ' lobal ,f l ' ocal ); Concat() represents the connection process;

[0109] The classification feature f cat After passing through the adaptive average pooling and fully connected layers in sequence, the classification result of the input monocular image is obtained;

[0110] Based on the ConvNeXt model, pooling layer, linear layer and dropout layer, an initial model for glaucoma fundus image classification of binocular images is constructed; the specific steps include the following:

[0111] The processing of the initial model for classifying glaucoma fundus images of binocular images includes the following steps:

[0112] The input images of the initial model for glaucoma fundus image classification of binocular images are the left eye global fundus image and the left eye local optic disc image of the left eye image, and the right eye global fundus image and the right eye local optic disc image of the right eye image;

[0113] The left eye feature extraction network is constructed based on the ConvNeXt model, and the left eye global fundus image and the left eye local optic disc image are input into the left eye feature extraction network for processing to obtain the left eye preliminary feature f l ' eft ;

[0114] In specific implementation, the construction of the left eye feature extraction network based on the ConvNeXt model specifically includes the following steps:

[0115] The left eye global fundus image is input into the ConvNeXt model for processing, and the left eye global image features are obtained through the activation function Left eye global image features Expressed as

[0116]

[0117] In the formula is the global fundus image of the left eye; are the parameters of the ConvNeXt model corresponding to the global fundus image of the left eye; Represents the processing process of the ConvNeXt model corresponding to the global fundus image of the left eye; The processing process of the activation function corresponding to the global fundus image of the left eye;

[0118] The left eye local optic disc image is input into the ConvNeXt model for processing, and the left eye local image features are obtained through the activation function Left eye local image features Expressed as

[0119]

[0120] In the formula It is the local optic disc image of the left eye; are the parameters of the ConvNeXt model corresponding to the local optic disc image of the left eye; The processing process of the ConvNeXt model corresponding to the local optic disc image of the left eye; The processing process of the activation function corresponding to the local optic disc image of the left eye;

[0121] The obtained left eye global image features Through adaptive average pooling, it is converted into a global one-dimensional vector for the left eye for

[0122] The obtained left eye local image feature Convert to a left eye local one-dimensional vector through adaptive average pooling processing For

[0123] Connect the left eye global one-dimensional vector And the left eye local one-dimensional vector To obtain the left eye preliminary feature f l ' eft For

[0124] Based on the ConvNeXt model, a right eye feature extraction network is constructed, and the right eye global fundus image and the right eye local optic disc image are input into the right eye feature extraction network for processing to obtain the right eye preliminary feature f r ' ight ;

[0125] In specific implementation, the right eye feature extraction network constructed based on the ConvNeXt model specifically includes the following steps:

[0126] The right eye global fundus image is input into the ConvNeXt model for processing, and the right eye global image feature is obtained through an activation function The right eye global image feature Is expressed as

[0127]

[0128] In the formula, The right eye global fundus image; The parameters of the ConvNeXt model corresponding to the right eye global fundus image; Indicates the processing process of the ConvNeXt model corresponding to the right eye global fundus image; The processing process of the activation function corresponding to the right eye global fundus image;

[0129] The right eye local optic disc image is input into the ConvNeXt model for processing, and the right eye local image feature is obtained through an activation function The right eye local image feature Is expressed as

[0130]

[0131] In the formula, The right eye local optic disc image; The parameters of the ConvNeXt model corresponding to the right eye local optic disc image; The processing process of the ConvNeXt model corresponding to the right eye local optic disc image; The processing procedure of the activation function corresponding to the right eye local optic disc image is as follows:

[0132] The obtained right eye global image feature is converted into a right eye global one-dimensional vector through adaptive average pooling processing The processing procedure of the activation function corresponding to the right eye local optic disc image is as follows:

[0133] The obtained right eye local image feature is converted into a right eye local one-dimensional vector through adaptive average pooling processing The processing procedure of the activation function corresponding to the right eye local optic disc image is as follows:

[0134] The right eye global one-dimensional vector and the right eye local one-dimensional vector are connected to obtain a right eye preliminary feature f r ' ight The processing procedure of the activation function corresponding to the right eye local optic disc image is as follows:

[0135] The obtained left eye preliminary feature f l ' eft is obtained through adaptive average pooling processing to obtain a left eye feature f left .

[0136] The obtained right eye preliminary feature f r ' ight is obtained through adaptive average pooling processing to obtain a right eye feature f right .

[0137] The left eye feature f left and the right eye feature f right are connected in the order of the left eye feature and the right eye feature to obtain a left eye classification feature f left-cat is f left-cat = Concat(f left ,f right ).

[0138] The right eye feature f right and the left eye feature f left are connected in the order of the right eye feature and the left eye feature to obtain a right eye classification feature f right-cat is f right-cat = Concat(f right ,f left ).

[0139] A left eye classification network is constructed based on a pooling layer, a linear layer and a dropout layer, and the left eye classification feature f left-cat is input into the left eye classification network for processing to obtain a final left eye image classification result.

[0140] In a specific implementation, the left-eye classification network is constructed based on the pooling layer, the linear layer, and the dropout layer, and specifically includes the following steps:

[0141] The left eye classification feature f left-cat After feature extraction through the linear layer, processing through the maximum pooling layer, activation through the Relu function, and finally processing through the dropout layer, the first feature f of the left eye classification is obtained. left-cat-1 ;

[0142] Classify the left eye with the first feature f left-cat-1 The feature extraction is performed again through the linear layer, processed through the maximum pooling layer, activated by the Relu function, and finally processed through the dropout layer to obtain the second feature f of the left eye classification. left-cat-2 ;

[0143] Classify the left eye with the second feature f left-cat-2 The final left eye image classification result is obtained by processing through the linear layer;

[0144] The right eye classification network is constructed based on the pooling layer, linear layer and dropout layer, and the right eye classification feature f right-cat Input into the right eye classification network for processing to obtain the final right eye image classification result;

[0145] In a specific implementation, the right eye classification network is constructed based on the pooling layer, the linear layer and the dropout layer, which specifically includes the following steps:

[0146] The right eye classification feature f right-cat After feature extraction through the linear layer, processing through the maximum pooling layer, activation through the Relu function, and finally processing through the dropout layer, the first feature f of the right eye classification is obtained. right-cat-1 ;

[0147] Classify the right eye with the first feature f right-cat-1 The feature is extracted again through the linear layer, processed through the maximum pooling layer, activated by the Relu function, and finally processed through the dropout layer to obtain the second feature f of the right eye classification. right-cat-2 ;

[0148] Classify the right eye with the second feature f right-cat-2 Processed through the linear layer to obtain the final right eye image classification result;

[0149] S4. Using the training data set obtained in step S2, the initial glaucoma fundus image classification model constructed in step S3 is trained to obtain a glaucoma fundus image classification model;

[0150] During specific training, for the initial model of monocular glaucoma fundus image classification, the following formula is used as the loss function L during training:

[0151]

[0152] Where i is global, local or cat; w i is the corresponding weight parameter; L i (o i ,y) is the corresponding label smoothed cross entropy loss function; y is the true label; o i is the predicted probability, and o i =Softmax(W i f i +b i ), Softmax() is the softmax function, W i is the weight matrix, b i is the bias term;

[0153] The initial model for glaucoma fundus image classification for binocular images uses the cross entropy loss function as the loss function;

[0154] During training, we started with an initial learning rate of 0.0001 and trained for more than 50 epochs. We used the ReduceLROnPlateau scheduler to dynamically adjust the learning rate and improve the training process. We used the Adam optimizer with the multi-loss cross entropy loss function and set the batch size to 8.

[0155] S5. Use the glaucoma fundus image classification model obtained in step S4 to classify the actual glaucoma fundus image.

[0156] The effect of the method of the present invention is described below in conjunction with a comparative example:

[0157] The method of the present invention was compared with the existing schemes to classify the same fundus image data on the PAPILA dataset, the OIA-ODIR dataset, and the ORIGA dataset. The comparison indicators included AUC (area under the ROC curve), BAcc (balanced accuracy), Sen (sensitivity), and Spe (specificity). The threshold that produced the highest BAcc score was selected as the final threshold for reporting model performance.

[0158] Among them, the calculation formula of BAcc index is: The calculation formula of Sen indicator is: The calculation formula of Spe index is: Where TP is the number of correctly classified positive (abnormal) images, TN is the number of correctly classified negative (normal) images, FP is the number of incorrectly classified positive (abnormal) images, and FN is the number of incorrectly classified negative (normal) images.

[0159] Specific comparative data are shown in Table 1, Table 2 and Table 3:

[0160] Table 1 Schematic diagram of classification comparison data of PAPILA dataset

[0161]

[0162] Table 2 Classification comparison data of OIA-ODIR dataset

[0163]

[0164] Table 3 Classification comparison data of ORIGA dataset

[0165]

[0166] As can be seen from Tables 1 to 3, the proposed solution demonstrated significant advantages on all three datasets, particularly in terms of AUC, BAcc, and sensitivity. On the first two datasets (including paired binocular images), the proposed solution significantly enhanced detection performance. On the third dataset, which did not include paired images, the proposed solution combined with the triple loss strategy maintained high recognition accuracy. These results demonstrate that the proposed solution effectively improves the accuracy and reliability of glaucoma classification in both paired and monocular scenarios.

[0167] In addition, this example also uses a 512×512 global image as input for comparison with other models. The results of these experiments are shown in the "Global Image" column in Tables 1, 2, and 3. The inventive solution continues to demonstrate significant performance advantages on all three datasets, particularly in terms of AUC, BAcc, and sensitivity. Whether using paired binocular or monocular datasets, the inventive solution consistently improves the accuracy and robustness of glaucoma fundus image classification.

[0168] like Figure 2 The figure shows a flow chart of the imaging method of the present invention: the imaging method disclosed in the present invention, including the glaucoma fundus image classification method, comprises the following steps:

[0169] S1. Obtain existing glaucoma fundus image data information and classify and annotate it;

[0170] S2. Preprocess the data information obtained in step S1 to obtain a training data set;

[0171] S3. Build an initial model for glaucoma fundus image classification based on the ConvNeXt model;

[0172] S4. Using the training data set obtained in step S2, the initial glaucoma fundus image classification model constructed in step S3 is trained to obtain a glaucoma fundus image classification model;

[0173] S5. Using the glaucoma fundus image classification model obtained in step S4, the actual glaucoma fundus image classification is performed;

[0174] S6. Marking and re-imaging the obtained glaucoma fundus image classification result on the actual glaucoma fundus image to obtain a glaucoma fundus image with the glaucoma fundus image classification result.

[0175] The imaging method provided by the present invention can be directly applied to an existing fundus image imaging device (such as a fundus camera) or directly applied to a terminal (such as a computer); in specific application, an existing scheme is used to obtain an actual fundus image, and then the obtained data is input into a corresponding machine device (fundus camera) or terminal. At this time, the machine device or terminal can obtain the actual fundus image classification result according to the imaging method disclosed by the present invention, and display the fundus image classification result on the original image through different types of representation (such as color), and then perform secondary imaging and output; at this time, the output image is an image with the fundus image classification result, which can reflect the actual fundus image classification result, thereby greatly facilitating clinical medical staff and laboratory experimenters to carry out subsequent work.

Claims

1. A glaucoma fundus image classification method comprising the following steps: S1. Obtain existing glaucoma fundus image data information and classify and annotate it; S2. Preprocess the data information obtained in step S1 to obtain a training data set; S3. Based on the ConvNeXt model, construct an initial model for glaucoma fundus image classification; the steps include: Based on the ConvNeXt model and pooling layer, an initial model for monocular glaucoma fundus image classification was constructed; Based on the ConvNeXt model, pooling layer, linear layer and dropout layer, an initial model for glaucoma fundus image classification of binocular images is constructed; the specific steps include the following: The processing of the initial model for classifying glaucoma fundus images of binocular images includes the following steps: The input images of the initial model for glaucoma fundus image classification of binocular images are the left eye global fundus image and the left eye local optic disc image of the left eye image, and the right eye global fundus image and the right eye local optic disc image of the right eye image; The left eye feature extraction network is constructed based on the ConvNeXt model, and the left eye global fundus image and the left eye local optic disc image are input into the left eye feature extraction network for processing to obtain the left eye preliminary feature f′ left ; Based on the ConvNeXt model, a right eye feature extraction network is constructed, and the right eye global fundus image and the right eye local optic disc image are input into the right eye feature extraction network for processing to obtain the right eye preliminary feature f′ right ; The obtained left eye preliminary feature f′ left Through adaptive average pooling processing, the left eye feature f is obtained left ; The obtained preliminary feature f′ of the right eye right Through adaptive average pooling processing, the right eye feature f is obtained right ; In the order of left eye features and right eye features, the left eye features f left and right eye feature f right Connect and get the left eye classification feature f left-cat f left-cat =Concat(f left ,f right ); In the order of right eye features and left eye features, the right eye features f right and left eye feature f left Connect and get the right eye classification feature f right-cat f right-cat =Concat(f right ,f left ); The left eye classification network is constructed based on the pooling layer, linear layer and dropout layer, and the left eye classification feature f left-cat Input into the left eye classification network for processing to obtain the final left eye image classification result; The right eye classification network is constructed based on the pooling layer, linear layer and dropout layer, and the right eye classification feature f right-cat Input into the right eye classification network for processing to obtain the final right eye image classification result; S4. Using the training data set obtained in step S2, the initial glaucoma fundus image classification model constructed in step S3 is trained to obtain a glaucoma fundus image classification model; S5. Use the glaucoma fundus image classification model obtained in step S4 to classify the actual glaucoma fundus image.

2. The glaucoma fundus image classification method according to claim 1, characterized in that Step S2 pre-processes the data information obtained in step S1 to obtain a training data set, which specifically includes the following steps: For the fundus image data obtained in step S1, detect whether there is an optic disc area: If the optic disc area does not exist, the test fails and the corresponding fundus image is directly deleted; If the optic disc area exists, it is detected; the square boundary area of ​​the optic disc area is calculated, the corresponding fundus image is cropped to a set size, and the corresponding optic disc area is extracted; All fundus images that have passed detection and cropping are judged and classified: All monocular images are used to form a monocular image data training set; All binocular images are paired to form a binocular image data training set.

3. The glaucoma fundus image classification method according to claim 2, characterized in that The pre-trained YOLOv5 model is used to detect whether the optic disc area exists; for the fundus image with the optic disc area, the pre-trained YOLOv5 model is used to extract the optic disc area to obtain the optic disc area image.

4. The glaucoma fundus image classification method according to claim 3, characterized in that The aforementioned method of constructing an initial model for classifying glaucoma fundus images of monocular images based on the ConvNeXt model and the pooling layer specifically includes the following steps: The processing of the initial model for classifying monocular glaucoma fundus images includes the following steps: The input images of the initial model for classifying monocular glaucoma fundus images are the global fundus image and the corresponding local optic disc image of the monocular image; the global fundus image is the monocular image, and the local optic disc image is the optic disc area image corresponding to the monocular image; The global fundus image is input into the ConvNeXt model for processing, and the global image feature f is obtained through the activation function global ; Global image feature f global Expressed as f global =φ(ConvNeXt(x global ;θ global )) Where x global is the global fundus image; θ global are the parameters of the ConvNeXt model corresponding to the global fundus image; ConvNeXt() represents the processing process of the ConvNeXt model corresponding to the global fundus image; φ() represents the processing process of the activation function corresponding to the global fundus image; The local optic disc image is input into the ConvNeXt model for processing, and the local image feature f is obtained through the activation function local ; Local image feature f local Expressed as f local =φ'(ConvNeXt'(x local ;θ local )) Where x local is the local optic disc image; θ local are the parameters of the ConvNeXt model corresponding to the local optic disc image; ConvNeXt'() is the processing process of the ConvNeXt model corresponding to the local optic disc image; φ'() is the processing process of the activation function corresponding to the local optic disc image; The obtained global image feature f global Through adaptive average pooling, it is converted into a global one-dimensional vector f′ global f′ global =AdaptiveAvgPool(f global ); AdaptiveAvgPool() represents the adaptive average pooling process; The obtained local image feature f local Through adaptive average pooling, it is converted into a local one-dimensional vector f′ local f′ local =AdaptiveAvgPool(f local ); The global one-dimensional vector f′ global and the local one-dimensional vector f′ local Connect and get the classification feature f cat f cat =Concat(f′ global ,f′ local ); Concat() represents the connection process; The classification feature f cat After passing through the adaptive average pooling and fully connected layers in sequence, the classification result of the input monocular image is obtained.

5. The glaucoma fundus image classification method according to claim 4, characterized in that The left eye feature extraction network is constructed based on the ConvNeXt model, which specifically includes the following steps: The left eye global fundus image is input into the ConvNeXt model for processing, and the left eye global image features are obtained through the activation function Left eye global image features Expressed as In the formula is the global fundus image of the left eye; are the parameters of the ConvNeXt model corresponding to the global fundus image of the left eye; Represents the processing process of the ConvNeXt model corresponding to the global fundus image of the left eye; The processing process of the activation function corresponding to the global fundus image of the left eye; The left eye local optic disc image is input into the ConvNeXt model for processing, and the left eye local image features are obtained through the activation function Left eye local image features Expressed as In the formula It is the local optic disc image of the left eye; are the parameters of the ConvNeXt model corresponding to the local optic disc image of the left eye; The processing process of the ConvNeXt model corresponding to the local optic disc image of the left eye; The processing process of the activation function corresponding to the local optic disc image of the left eye; The obtained left eye global image features Converted into a global one-dimensional vector for the left eye through adaptive average pooling for The obtained left eye local image features Through adaptive average pooling, it is converted into a local one-dimensional vector of the left eye for The left eye global one-dimensional vector and the left eye local one-dimensional vector Connect and get the preliminary feature f′ of the left eye left for The right eye feature extraction network is constructed based on the ConvNeXt model, which specifically includes the following steps: The right eye global fundus image is input into the ConvNeXt model for processing, and the right eye global image features are obtained through the activation function Right eye global image features Expressed as In the formula is the global fundus image of the right eye; are the parameters of the ConvNeXt model corresponding to the global fundus image of the right eye; Represents the processing process of the ConvNeXt model corresponding to the global fundus image of the right eye; The processing process of the activation function corresponding to the global fundus image of the right eye; The right eye local optic disc image is input into the ConvNeXt model for processing, and the right eye local image features are obtained through the activation function Right eye local image features Expressed as In the formula This is the partial optic disc image of the right eye; are the parameters of the ConvNeXt model corresponding to the local optic disc image of the right eye; The processing process of the ConvNeXt model corresponding to the local optic disc image of the right eye; The processing process of the activation function corresponding to the local optic disc image of the right eye; The obtained right eye global image features Converted into a global one-dimensional vector for the right eye through adaptive average pooling for The obtained right eye local image features Through adaptive average pooling, it is converted into a local one-dimensional vector of the right eye for The right eye global one-dimensional vector and the right eye local one-dimensional vector Connect and get the preliminary feature f′ of the right eye right for 6. The glaucoma fundus image classification method according to claim 5, characterized in that The left-eye classification network is constructed based on the pooling layer, the linear layer, and the dropout layer, and specifically includes the following steps: The left eye classification feature f left-cat After feature extraction through the linear layer, processing through the maximum pooling layer, activation through the Relu function, and finally processing through the dropout layer, the first feature f of the left eye classification is obtained. left-cat-1 ; Classify the left eye with the first feature f left-cat-1 The feature extraction is performed again through the linear layer, processed through the maximum pooling layer, activated by the Relu function, and finally processed through the dropout layer to obtain the second feature f of the left eye classification. left-cat-2 ; Classify the left eye with the second feature f left-cat-2 The final left eye image classification result is obtained by processing through the linear layer; The right eye classification network is constructed based on the pooling layer, the linear layer and the dropout layer, and specifically includes the following steps: The right eye classification feature f right-cat After feature extraction through the linear layer, processing through the maximum pooling layer, activation through the Relu function, and finally processing through the dropout layer, the first feature f of the right eye classification is obtained. right-cat-1 ; Classify the right eye with the first feature f right-cat-1 The feature is extracted again through the linear layer, processed through the maximum pooling layer, activated by the Relu function, and finally processed through the dropout layer to obtain the second feature f of the right eye classification. right-cat-2 ; Classify the right eye with the second feature f right-cat-2 After processing through the linear layer, the final right eye image classification result is obtained.

7. The glaucoma fundus image classification method according to claim 6, characterized in that The training described in step S4 specifically includes the following steps: For the initial model of monocular glaucoma fundus image classification, the following formula is used as the loss function L during training: Where i is global, local or cat; w i is the corresponding weight parameter; L i (o i ,y) is the corresponding label smoothed cross entropy loss function; y is the true label; o i is the predicted probability, and o i =Softmax(W i f i +b i ), Softmax() is the softmax function, W i is the weight matrix, b i is the bias term; The initial model for glaucoma fundus image classification for binocular images uses the cross entropy loss function as the loss function.

8. An imaging method comprising the glaucoma fundus image classification method according to any one of claims 1 to 7, characterized in that The following steps are also included: S6. Marking and re-imaging the obtained glaucoma fundus image classification result on the actual glaucoma fundus image to obtain a glaucoma fundus image with the glaucoma fundus image classification result.

Images