Glaucoma fundus image classification method and imaging method

Abstract

The invention discloses a glaucoma fundus image classification method, which comprises the steps of acquiring existing glaucoma fundus image data information, and performing classification labeling and preprocessing to obtain a training data set; constructing a glaucoma fundus image classification initial model based on a ConvNeXt model, and training to obtain a glaucoma fundus image classification model; and adopting the obtained glaucoma fundus image classification model to classify actual glaucoma fundus images. The invention further discloses an imaging method comprising the glaucoma fundus image classification method. According to the method, the glaucoma fundus image classification network with the feature fusion process is constructed through the ConvNeXt model, so that classification of the glaucoma fundus images can be achieved, the method is suitable for classification of monocular images and binocular images, reliability is high, accuracy is good, the application range is wide, and the method is more objective and scientific.

Description

Technical Field

[0001] The invention belongs to the field of image processing, and in particular relates to a glaucoma fundus image classification method and an imaging method. Background Art

[0002] With the development of economy and technology and the improvement of people's living standards, people are paying more and more attention to health. Glaucoma is a common eye disease; therefore, medical research on glaucoma has always been one of the hot topics.

[0003] The classification of glaucoma fundus images is of great significance both in the clinical stage and in the laboratory research stage. The traditional glaucoma fundus image classification scheme adopts a manual classification scheme, that is, medical professionals use their own medical literacy, professional knowledge and experience to manually classify glaucoma fundus images; however, this manual classification scheme is not only time-consuming and labor-intensive, but also has poor efficiency and reliability.

[0004] At present, with the rapid development of neural network technology, some researchers have applied neural network technology to the fundus classification process of glaucoma. Such solutions are generally divided into machine learning solutions and deep learning solutions. However, traditional machine learning solutions rely on the extraction of artificial features, which are easily interfered by human factors. Although the deep learning solution has achieved certain application effects, the generalization ability of this type of model is poor, and there are still problems of poor reliability and poor accuracy in specific applications. Summary of the invention

[0005] One of the purposes of the present invention is to provide a glaucoma fundus image classification method which is highly reliable, accurate, objective and scientific.

[0006] A second object of the present invention is to provide an imaging method including the glaucoma fundus image classification method.

[0007] The glaucoma fundus image classification method provided by the present invention comprises the following steps:

[0008] S1. Obtain existing glaucoma fundus image data information and classify and annotate it;

[0009] S2. Preprocess the data information obtained in step S1 to obtain a training data set;

[0010] S3. Based on the ConvNeXt model, an initial model for glaucoma fundus image classification was constructed;

[0011] S4. Using the training data set obtained in step S2, the initial model for glaucoma fundus image classification constructed in step S3 is trained to obtain a glaucoma fundus image classification model;

[0012] S5. Use the glaucoma fundus image classification model obtained in step S4 to classify the actual glaucoma fundus image.

[0013] Step S2 preprocesses the data information obtained in step S1 to obtain a training data set, which specifically includes the following steps:

[0014] For the fundus image data obtained in step S1, detect whether there is an optic disc area:

[0015] If the optic disc area does not exist, the detection fails and the corresponding fundus image is directly deleted;

[0016] If the optic disc area exists, it is detected; the square boundary area of ​​the optic disc area is calculated, the corresponding fundus image is cropped to a set size, and the corresponding optic disc area is extracted;

[0017] All fundus images that have passed detection and cropping are judged and classified:

[0018] All monocular images are used to form a monocular image data training set;

[0019] All binocular images are paired to form a binocular image data training set.

[0020] The pre-trained YOLOv5 model is used to detect whether an optic disc area exists; for fundus images with an optic disc area, the pre-trained YOLOv5 model is used to extract the optic disc area to obtain an optic disc area image.

[0021] The step S3 described in which the initial model for glaucoma fundus image classification is constructed based on the ConvNeXt model includes the following steps:

[0022] Based on the ConvNeXt model and pooling layer, an initial model for glaucoma fundus image classification of monocular images was constructed;

[0023] Based on the ConvNeXt model, pooling layer, linear layer and dropout layer, an initial model for glaucoma fundus image classification of binocular images was constructed.

[0024] The aforementioned method of constructing an initial model for classifying glaucoma fundus images of monocular images based on the ConvNeXt model and the pooling layer specifically includes the following steps:

[0025] The processing process of the constructed initial model for classification of glaucoma fundus images of monocular images includes the following steps:

[0026] The input image of the initial model for classifying the glaucoma fundus image of the monocular image is the global fundus image of the monocular image and the corresponding local optic disc image; the global fundus image is the monocular image, and the local optic disc image is the optic disc area image corresponding to the monocular image;

[0027] The global fundus image is input into the ConvNeXt model for processing, and the global image feature f is obtained through the activation function global ; Global image feature f global Expressed as

[0028] f global =φ(ConvNeXt(x global θ global ))

[0029] Where x global is the global fundus image; θ global are the parameters of the ConvNeXt model corresponding to the global fundus image; ConvNeXt() represents the processing process of the ConvNeXt model corresponding to the global fundus image; φ() is the processing process of the activation function corresponding to the global fundus image;

[0030] The local optic disc image is input into the ConvNeXt model for processing, and the local image feature f is obtained through the activation function local ; Local image feature f local Expressed as

[0031] f local =φ(ConvNeXt'(x local θ local ))

[0032] Where x local is the local optic disc image; θ local are the parameters of the ConvNeXt model corresponding to the local optic disc image; ConvNeXt'() is the processing process of the ConvNeXt model corresponding to the local optic disc image; φ'() is the processing process of the activation function corresponding to the local optic disc image;

[0033] The obtained global image feature f global Through adaptive average pooling, it is converted into a global one-dimensional vector f g ' lobal f g ' lobal =AdaptiveAvgPool(f global ); AdaptiveAvgPool() represents the adaptive average pooling process;

[0034] The local image feature f localThrough adaptive average pooling, it is converted into a local one-dimensional vector f l ' ocal f l ' ocal =AdaptiveAvgPool(f local );

[0035] The global one-dimensional vector f g ' lobal and the local one-dimensional vector f l ' ocal Connect and get the classification feature f cat f cat =Concat(f g ' lobal ,f l ' ocal ); Concat() represents the connection process;

[0036] The classification feature f cat After passing through the adaptive average pooling and fully connected layers in sequence, the classification result of the input monocular image is obtained.

[0037] The aforementioned method of constructing an initial model for classifying glaucoma fundus images of binocular images based on the ConvNeXt model, the pooling layer, the linear layer and the dropout layer specifically includes the following steps:

[0038] The processing process of constructing the initial model for classifying glaucoma fundus images of binocular images includes the following steps:

[0039] The input images of the initial model for classifying glaucoma fundus images of binocular images are the left eye global fundus image and the left eye local optic disc image of the left eye image, and the right eye global fundus image and the right eye local optic disc image of the right eye image;

[0040] Based on the ConvNeXt model, a left eye feature extraction network is constructed, and the left eye global fundus image and the left eye local optic disc image are input into the left eye feature extraction network for processing to obtain the left eye preliminary feature f l ' eft ;

[0041] Based on the ConvNeXt model, a right eye feature extraction network is constructed, and the right eye global fundus image and the right eye local optic disc image are input into the right eye feature extraction network for processing to obtain the right eye preliminary feature f r ' ight ;

[0042] The obtained left eye preliminary feature f l ' eft Through adaptive average pooling processing, the left eye feature f is obtained left ;

[0043] The obtained preliminary feature f of the right eye r ' ight Through adaptive average pooling processing, the right eye feature f is obtained right ;

[0044] In the order of left eye features and right eye features, the left eye features f left and right eye feature f right Connect and get the left eye classification feature f left-cat f left-cat =Concat(f left ,f right );

[0045] In the order of right eye features and left eye features, the right eye features f right and left eye feature f left Connect and get the right eye classification feature f right-cat f right-cat =Concat(f right ,f left );

[0046] The left eye classification network is constructed based on the pooling layer, linear layer and dropout layer, and the left eye classification feature f left-cat Input to the left eye classification network for processing to obtain the final left eye image classification result;

[0047] The right eye classification network is constructed based on the pooling layer, linear layer and dropout layer, and the right eye classification feature f right-cat Input into the right eye classification network for processing to obtain the final right eye image classification result.

[0048] The construction of the left eye feature extraction network based on the ConvNeXt model specifically includes the following steps:

[0049] The left eye global fundus image is input into the ConvNeXt model for processing, and the left eye global image features are obtained through the activation function. Left eye global image features Expressed as

[0050]

[0051] In the formula is the global fundus image of the left eye; are the parameters of the ConvNeXt model corresponding to the global fundus image of the left eye; Represents the processing process of the ConvNeXt model corresponding to the global fundus image of the left eye; The processing process of the activation function corresponding to the global fundus image of the left eye;

[0052] The left eye local optic disc image is input into the ConvNeXt model for processing, and the left eye local image features are obtained through the activation function Left eye local image features Expressed as

[0053]

[0054] In the formula It is the local optic disc image of the left eye; are the parameters of the ConvNeXt model corresponding to the local optic disc image of the left eye; The processing process of the ConvNeXt model corresponding to the local optic disc image of the left eye; The processing process of the activation function corresponding to the local optic disc image of the left eye;

[0055] The obtained left eye global image features Through adaptive average pooling, it is converted into a global one-dimensional vector for the left eye for

[0056] The obtained left eye local image features Through adaptive average pooling, it is converted into a local one-dimensional vector of the left eye for

[0057] The left eye global one-dimensional vector and the left eye local one-dimensional vector Connect and get the preliminary feature f of the left eye l ' eft for

[0058] The right eye feature extraction network is constructed based on the ConvNeXt model, and specifically includes the following steps:

[0059] The right eye global fundus image is input into the ConvNeXt model for processing, and the right eye global image features are obtained through the activation function Right eye global image features Expressed as

[0060]

[0061] In the formula is the global fundus image of the right eye; are the parameters of the ConvNeXt model corresponding to the global fundus image of the right eye; It represents the processing process of the ConvNeXt model corresponding to the global fundus image of the right eye; The processing process of the activation function corresponding to the global fundus image of the right eye;

[0062] The right eye local optic disc image is input into the ConvNeXt model for processing, and the right eye local image features are obtained through the activation function Right eye local image features Expressed as

[0063]

[0064] In the formula It is the partial optic disc image of the right eye; are the parameters of the ConvNeXt model corresponding to the local optic disc image of the right eye; The processing process of the ConvNeXt model corresponding to the local optic disc image of the right eye; The processing process of the activation function corresponding to the local optic disc image of the right eye;

[0065] The obtained right eye global image features Through adaptive average pooling, it is converted into a global one-dimensional vector for the right eye for

[0066] The obtained right eye local image features Through adaptive average pooling, it is converted into a local one-dimensional vector of the right eye for

[0067] The right eye global one-dimensional vector and the right eye local one-dimensional vector Connect and get the initial feature f of the right eye r ' ight for

[0068] The construction of the left eye classification network based on the pooling layer, the linear layer and the dropout layer specifically includes the following steps:

[0069] The left eye classification feature f left-cat After feature extraction through the linear layer, processing through the maximum pooling layer, activation through the Relu function, and finally processing through the dropout layer, the first feature f of the left eye classification is obtained. left-cat-1 ;

[0070] Classify the left eye with the first feature f left-cat-1 The feature is extracted again through the linear layer, processed through the maximum pooling layer, activated by the Relu function, and finally processed by the dropout layer to obtain the second feature f of the left eye classification. left-cat-2 ;

[0071] Classify the left eye with the second feature f left-cat-2 After processing through the linear layer, the final left eye image classification result is obtained.

[0072] The right eye classification network is constructed based on the pooling layer, the linear layer and the dropout layer, and specifically includes the following steps:

[0073] The right eye classification feature f right-cat After feature extraction through the linear layer, processing through the maximum pooling layer, activation through the Relu function, and finally processing through the dropout layer, the first feature f of the right eye classification is obtained. right-cat-1 ;

[0074] Classify the right eye with the first feature f right-cat-1 The feature is extracted again through the linear layer, processed through the maximum pooling layer, activated by the Relu function, and finally processed by the dropout layer to obtain the second feature f of the right eye classification. right-cat-2 ;

[0075] Classify the right eye with the second feature f right-cat-2 After processing through the linear layer, the final right eye image classification result is obtained.

[0076] The training described in step S4 specifically includes the following steps:

[0077] For the initial model of glaucoma fundus image classification of monocular images, the following formula is used as the loss function L during training:

[0078]

[0079] Where i is global, local or cat; w i is the corresponding weight parameter; L i (o i ,y) is the corresponding label smoothed cross entropy loss function; y is the true label; o i is the predicted probability, and o i =Softmax(W i f i +b i ), Softmax() is the softmax function, W i is the weight matrix, b i Bias term;

[0080] For the initial model of glaucoma fundus image classification of binocular images, the cross entropy loss function is used as the loss function.

[0081] The present invention also provides an imaging method including the glaucoma fundus image classification method, further comprising the following steps:

[0082] S6. Marking and re-imaging the obtained glaucoma fundus image classification result on the actual glaucoma fundus image to obtain a glaucoma fundus image with the glaucoma fundus image classification result.

[0083] The glaucoma fundus image classification method and imaging method provided by the present invention construct a glaucoma fundus image classification network with a feature fusion process through a ConvNeXt model. Therefore, the present invention can not only realize the classification of glaucoma fundus images, but also is suitable for the classification of monocular images and binocular images. It not only has high reliability and good accuracy, but also has a wide range of applications and is more objective and scientific. BRIEF DESCRIPTION OF THE DRAWINGS

[0084] Figure 1 The figure is a flow chart of the classification method of the present invention.

[0085] Figure 2 The figure is a schematic diagram of the method flow of the imaging method of the present invention. DETAILED DESCRIPTION

[0086] like Figure 1 The schematic diagram of the method flow of the classification method of the present invention is shown as follows: The glaucoma fundus image classification method provided by the present invention comprises the following steps:

[0087] S1. Obtain existing glaucoma fundus image data information and classify and annotate it;

[0088] S2. Preprocess the data information obtained in step S1 to obtain a training data set; specifically, the steps include:

[0089] For the fundus image data obtained in step S1, detect whether there is an optic disc area:

[0090] If the optic disc area does not exist, the detection fails and the corresponding fundus image is directly deleted;

[0091] If the optic disc area exists, it is detected; the square boundary area of ​​the optic disc area is calculated (the Hough circle transform is used to crop the field of view, and zeros are filled on the shorter edges to adjust the image size to ensure that the width and height of the cropped area are balanced), the corresponding fundus image is cropped to a set size (preferably 512*512), and the corresponding optic disc area is extracted;

[0092] All fundus images that have passed detection and cropping are judged and classified:

[0093] After data enhancement, all monocular images are used to form a monocular image data training set;

[0094] Pair all binocular images and perform data enhancement to form a binocular image data training set;

[0095] In specific implementation, data augmentation includes random rotation, horizontal flipping, and vertical flipping;

[0096] In the specific implementation, a pre-trained YOLOv5 model is used to detect whether an optic disc area exists; for a fundus image with an optic disc area, the pre-trained YOLOv5 model is used to extract the optic disc area to obtain an optic disc area image;

[0097] S3. Based on the ConvNeXt model, an initial model for glaucoma fundus image classification is constructed; the steps include:

[0098] Based on the ConvNeXt model and the pooling layer, an initial model for glaucoma fundus image classification of monocular images is constructed; specifically, the following steps are included:

[0099] The input image of the initial model for classifying the glaucoma fundus image of the monocular image is the global fundus image of the monocular image and the corresponding local optic disc image; the global fundus image is the monocular image, and the local optic disc image is the optic disc area image corresponding to the monocular image;

[0100] The global fundus image is input into the ConvNeXt model for processing, and the global image feature f is obtained through the activation function global ; Global image feature f global Expressed as

[0101] f global =φ(ConvNeXt(x global θ global ))

[0102] Where x global is the global fundus image; θ global are the parameters of the ConvNeXt model corresponding to the global fundus image; ConvNeXt() represents the processing process of the ConvNeXt model corresponding to the global fundus image; φ() is the processing process of the activation function corresponding to the global fundus image;

[0103] The local optic disc image is input into the ConvNeXt model for processing, and the local image feature f is obtained through the activation function local ; Local image feature f local Expressed as

[0104] f local =φ(ConvNeXt'(x local θ local ))

[0105] Where x local is the local optic disc image; θ local are the parameters of the ConvNeXt model corresponding to the local optic disc image; ConvNeXt'() is the processing process of the ConvNeXt model corresponding to the local optic disc image; φ'() is the processing process of the activation function corresponding to the local optic disc image;

[0106] The obtained global image feature f global Through adaptive average pooling, it is converted into a global one-dimensional vector f g ' lobal f g ' lobal =AdaptiveAvgPool(f global ); AdaptiveAvgPool() represents the adaptive average pooling process;

[0107] The local image feature f local Through adaptive average pooling, it is converted into a local one-dimensional vector f l ' ocal f l ' ocal =AdaptiveAvgPool(f local );

[0108] The global one-dimensional vector f g ' lobal and the local one-dimensional vector f l ' ocal Connect and get the classification feature f cat f cat =Concat(f g ' lobal ,f l ' ocal ); Concat() represents the connection process;

[0109] The classification feature f cat After passing through the adaptive average pooling and fully connected layers in sequence, the classification result of the input monocular image is obtained;

[0110] Based on the ConvNeXt model, pooling layer, linear layer and dropout layer, an initial model for glaucoma fundus image classification of binocular images is constructed; specifically, the following steps are included:

[0111] The processing process of constructing the initial model for classifying glaucoma fundus images of binocular images includes the following steps:

[0112] The input images of the initial model for classifying glaucoma fundus images of binocular images are the left eye global fundus image and the left eye local optic disc image of the left eye image, and the right eye global fundus image and the right eye local optic disc image of the right eye image;

[0113] Based on the ConvNeXt model, a left eye feature extraction network is constructed, and the left eye global fundus image and the left eye local optic disc image are input into the left eye feature extraction network for processing to obtain the left eye preliminary feature f l ' eft ;

[0114] In specific implementation, the construction of the left eye feature extraction network based on the ConvNeXt model specifically includes the following steps:

[0115] The left eye global fundus image is input into the ConvNeXt model for processing, and the left eye global image features are obtained through the activation function. Left eye global image features Expressed as

[0116]

[0117] In the formula is the global fundus image of the left eye; are the parameters of the ConvNeXt model corresponding to the global fundus image of the left eye; Represents the processing process of the ConvNeXt model corresponding to the global fundus image of the left eye; The processing process of the activation function corresponding to the global fundus image of the left eye;

[0118] The left eye local optic disc image is input into the ConvNeXt model for processing, and the left eye local image features are obtained through the activation function Left eye local image features Expressed as

[0119]

[0120] In the formula It is the local optic disc image of the left eye; are the parameters of the ConvNeXt model corresponding to the local optic disc image of the left eye; The processing process of the ConvNeXt model corresponding to the local optic disc image of the left eye; The processing process of the activation function corresponding to the local optic disc image of the left eye;

[0121] The obtained left eye global image features Through adaptive average pooling, it is converted into a global one-dimensional vector for the left eye for

[0122] The obtained left eye local image features Through adaptive average pooling, it is converted into a local one-dimensional vector of the left eye for

[0123] The left eye global one-dimensional vector and the left eye local one-dimensional vector Connect and get the preliminary feature f of the left eye l ' eft for

[0124] Based on the ConvNeXt model, a right eye feature extraction network is constructed, and the right eye global fundus image and the right eye local optic disc image are input into the right eye feature extraction network for processing to obtain the right eye preliminary feature f r ' ight ;

[0125] In specific implementation, the right eye feature extraction network is constructed based on the ConvNeXt model, which specifically includes the following steps:

[0126] The right eye global fundus image is input into the ConvNeXt model for processing, and the right eye global image features are obtained through the activation function Right eye global image features Expressed as

[0127]

[0128] In the formula is the global fundus image of the right eye; are the parameters of the ConvNeXt model corresponding to the global fundus image of the right eye; It represents the processing process of the ConvNeXt model corresponding to the global fundus image of the right eye; The processing process of the activation function corresponding to the global fundus image of the right eye;

[0129] The right eye local optic disc image is input into the ConvNeXt model for processing, and the right eye local image features are obtained through the activation function Right eye local image features Expressed as

[0130]

[0131] In the formula It is the partial optic disc image of the right eye; are the parameters of the ConvNeXt model corresponding to the local optic disc image of the right eye; The processing process of the ConvNeXt model corresponding to the local optic disc image of the right eye; The processing process of the activation function corresponding to the local optic disc image of the right eye;

[0132] The obtained right eye global image features Through adaptive average pooling, it is converted into a global one-dimensional vector for the right eye for

[0133] The obtained right eye local image features Through adaptive average pooling, it is converted into a local one-dimensional vector of the right eye for

[0134] The right eye global one-dimensional vector and the right eye local one-dimensional vector Connect and get the initial feature f of the right eye r ' ight for

[0135] The obtained left eye preliminary feature f l ' eft Through adaptive average pooling processing, the left eye feature f is obtained left ;

[0136] The obtained preliminary feature f of the right eye r ' ight Through adaptive average pooling processing, the right eye feature f is obtained right ;

[0137] In the order of left eye features and right eye features, the left eye features f left and right eye feature f right Connect and get the left eye classification feature f left-cat f left-cat =Concat(f left ,f right );

[0138] In the order of right eye features and left eye features, the right eye features f right and left eye feature f left Connect and get the right eye classification feature f right-cat f right-cat =Concat(f right ,f left );

[0139] The left eye classification network is constructed based on the pooling layer, linear layer and dropout layer, and the left eye classification feature f left-cat Input to the left eye classification network for processing to obtain the final left eye image classification result;

[0140] In specific implementation, the left eye classification network is constructed based on the pooling layer, the linear layer and the dropout layer, and specifically includes the following steps:

[0141] The left eye classification feature f left-cat After feature extraction through the linear layer, processing through the maximum pooling layer, activation through the Relu function, and finally processing through the dropout layer, the first feature f of the left eye classification is obtained. left-cat-1 ;

[0142] Classify the left eye with the first feature f left-cat-1 The feature is extracted again through the linear layer, processed through the maximum pooling layer, activated by the Relu function, and finally processed by the dropout layer to obtain the second feature f of the left eye classification. left-cat-2 ;

[0143] Classify the left eye with the second feature f left-cat-2 After processing through the linear layer, the final classification result of the left eye image is obtained;

[0144] The right eye classification network is constructed based on the pooling layer, linear layer and dropout layer, and the right eye classification feature f right-cat Input to the right eye classification network for processing to obtain the final right eye image classification result;

[0145] In a specific implementation, the right eye classification network is constructed based on the pooling layer, the linear layer and the dropout layer, and specifically includes the following steps:

[0146] The right eye classification feature f right-cat After feature extraction through the linear layer, processing through the maximum pooling layer, activation through the Relu function, and finally processing through the dropout layer, the first feature f of the right eye classification is obtained. right-cat-1 ;

[0147] Classify the right eye with the first feature f right-cat-1 The feature is extracted again through the linear layer, processed through the maximum pooling layer, activated by the Relu function, and finally processed by the dropout layer to obtain the second feature f of the right eye classification. right-cat-2 ;

[0148] Classify the right eye with the second feature f right-cat-2 After processing through the linear layer, the final right eye image classification result is obtained;

[0149] S4. Using the training data set obtained in step S2, the initial model for glaucoma fundus image classification constructed in step S3 is trained to obtain a glaucoma fundus image classification model;

[0150] During specific training, for the initial model of glaucoma fundus image classification of monocular images, the following formula is used as the loss function L during training:

[0151]

[0152] Where i is global, local or cat; w i is the corresponding weight parameter; L i (o i ,y) is the corresponding label smoothed cross entropy loss function; y is the true label; o i is the predicted probability, and o i =Softmax(W i f i +b i ), Softmax() is the softmax function, W i is the weight matrix, b i is the bias term;

[0153] The initial model for glaucoma fundus image classification for binocular images uses the cross entropy loss function as the loss function;

[0154] During training, the training started with an initial learning rate of 0.0001 and the training time exceeded 50 epochs. The ReduceLROnPlateau scheduler was used to dynamically adjust the learning rate to improve the training process. The Adam optimizer was used with the multi-loss cross entropy loss function and the batch size was set to 8.

[0155] S5. Use the glaucoma fundus image classification model obtained in step S4 to classify the actual glaucoma fundus image.

[0156] The effect of the method of the present invention is described below in conjunction with a comparative example:

[0157] The method of the present invention was compared with the existing schemes to classify the same fundus image data on the PAPILA dataset, the OIA-ODIR dataset, and the ORIGA dataset; the comparison indicators included AUC (area under the ROC curve), BAcc (balanced accuracy), Sen (sensitivity), and Spe (specificity); the threshold that produced the highest BAcc score was selected as the final threshold for reporting model performance

[0158] Among them, the calculation formula of BAcc index is: The calculation formula of Sen indicator is: The calculation formula of Spe indicator is: Among them, TP is the number of correctly classified positive (abnormal) pictures, TN is the number of correctly classified negative (normal) pictures, FP is the number of incorrectly classified positive (abnormal) pictures, and FN is the number of incorrectly classified negative (normal) pictures;

[0159] Specific comparative data are shown in Table 1, Table 2 and Table 3:

[0160] Table 1 Classification comparison data of PAPILA dataset

[0161]

[0162] Table 2 Classification comparison data of OIA-ODIR dataset

[0163]

[0164] Table 3 Classification comparison data of ORIGA dataset

[0165]

[0166] It can be seen from Tables 1 to 3 that the scheme of the present invention shows significant advantages on all three datasets, especially in terms of AUC, BAcc and sensitivity. On the first two datasets (including paired binocular images), the scheme of the present invention significantly enhances the detection performance. On the third dataset that does not contain paired images, the scheme of the present invention combined with the triple loss strategy maintains a high recognition accuracy. These results show that the scheme of the present invention effectively improves the accuracy and reliability of glaucoma classification in paired and monocular scenarios.

[0167] In addition, this embodiment also uses a 512×512 global image as input to compare other models. The results of these experiments are shown in the "Global Image" column of Table 1, Table 2, and Table 3. The scheme of the present invention continues to show significant performance advantages on all three datasets, especially in terms of AUC, BAcc, and sensitivity. Whether using paired binocular datasets or monocular datasets, the scheme of the present invention can continuously improve the accuracy and robustness of the classification of glaucoma fundus images.

[0168] like Figure 2 The method flow diagram of the imaging method of the present invention is shown as follows: the imaging method disclosed in the present invention, which includes the glaucoma fundus image classification method, comprises the following steps:

[0169] S1. Obtain existing glaucoma fundus image data information and classify and annotate it;

[0170] S2. Preprocess the data information obtained in step S1 to obtain a training data set;

[0171] S3. Based on the ConvNeXt model, an initial model for glaucoma fundus image classification was constructed;

[0172] S4. Using the training data set obtained in step S2, the initial model for glaucoma fundus image classification constructed in step S3 is trained to obtain a glaucoma fundus image classification model;

[0173] S5. Using the glaucoma fundus image classification model obtained in step S4, classify the actual glaucoma fundus image;

[0174] S6. Marking and re-imaging the obtained glaucoma fundus image classification result on the actual glaucoma fundus image to obtain a glaucoma fundus image with the glaucoma fundus image classification result.

[0175] The imaging method provided by the present invention can be directly applied to an existing fundus image imaging device (such as a fundus camera), or directly applied to a terminal (such as a computer); in specific application, the existing scheme is used to obtain the actual fundus image, and then the obtained data is input into the corresponding machine equipment (fundus camera) or terminal. At this time, the machine equipment or terminal can obtain the actual fundus image classification result according to the imaging method disclosed in the present invention, and display the fundus image classification result on the original image through different types of representation (such as color), and then perform secondary imaging and output; at this time, the output image is an image with the fundus image classification result, which can reflect the actual fundus image classification result, thereby greatly facilitating clinical medical personnel and laboratory experimenters to carry out subsequent work.

Claims

1. A method for classifying glaucoma fundus images, comprising the following steps: S1. Obtain existing glaucoma fundus image data information and classify and annotate it; S2. Preprocess the data information obtained in step S1 to obtain a training data set; S3. Based on the ConvNeXt model, an initial model for glaucoma fundus image classification was constructed; S4. Using the training data set obtained in step S2, the initial model for glaucoma fundus image classification constructed in step S3 is trained to obtain a glaucoma fundus image classification model; S5. Use the glaucoma fundus image classification model obtained in step S4 to classify the actual glaucoma fundus image.

2. The glaucoma fundus image classification method according to claim 1, characterized in that Step S2 preprocesses the data information obtained in step S1 to obtain a training data set, which specifically includes the following steps: For the fundus image data obtained in step S1, detect whether there is an optic disc area: If the optic disc area does not exist, the detection fails and the corresponding fundus image is directly deleted; If the optic disc area exists, it is detected; the square boundary area of ​​the optic disc area is calculated, the corresponding fundus image is cropped to a set size, and the corresponding optic disc area is extracted; All fundus images that have passed detection and cropping are judged and classified: All monocular images are used to form a monocular image data training set; All binocular images are paired to form a binocular image data training set.

3. The glaucoma fundus image classification method according to claim 2, characterized in that The pre-trained YOLOv5 model is used to detect whether an optic disc area exists; for fundus images with an optic disc area, the pre-trained YOLOv5 model is used to extract the optic disc area to obtain an optic disc area image.

4. The glaucoma fundus image classification method according to claim 3, characterized in that The step S3 described in which the initial model for glaucoma fundus image classification is constructed based on the ConvNeXt model includes the following steps: Based on the ConvNeXt model and pooling layer, an initial model for glaucoma fundus image classification of monocular images was constructed; Based on the ConvNeXt model, pooling layer, linear layer and dropout layer, an initial model for glaucoma fundus image classification of binocular images was constructed.

5. The glaucoma fundus image classification method according to claim 4, characterized in that The aforementioned method of constructing an initial model for classifying glaucoma fundus images of monocular images based on the ConvNeXt model and the pooling layer specifically includes the following steps: The processing process of the constructed initial model for classification of glaucoma fundus images of monocular images includes the following steps: The input image of the initial model for classifying the glaucoma fundus image of the monocular image is the global fundus image of the monocular image and the corresponding local optic disc image; the global fundus image is the monocular image, and the local optic disc image is the optic disc area image corresponding to the monocular image; The global fundus image is input into the ConvNeXt model for processing, and the global image feature f is obtained through the activation function global ; Global image feature f global Expressed as f global =φ(ConvNeXt(x global ;θ global )) Where x global is the global fundus image; θ global are the parameters of the ConvNeXt model corresponding to the global fundus image; ConvNeXt() represents the processing process of the ConvNeXt model corresponding to the global fundus image; φ() is the processing process of the activation function corresponding to the global fundus image; The local optic disc image is input into the ConvNeXt model for processing, and the local image feature f is obtained through the activation function local ; Local image feature f local Expressed as f local =φ(ConvNeXt'(x local ;θ local )) Where x local is the local optic disc image; θ local are the parameters of the ConvNeXt model corresponding to the local optic disc image; ConvNeXt'() is the processing process of the ConvNeXt model corresponding to the local optic disc image; φ'() is the processing process of the activation function corresponding to the local optic disc image; The obtained global image feature f global Through adaptive average pooling, it is converted into a global one-dimensional vector f g ' lobal f g ' lobal =AdaptiveAvgPool(f global ); AdaptiveAvgPool() represents the adaptive average pooling process; The local image feature f local Through adaptive average pooling, it is converted into a local one-dimensional vector f l ' ocal f l ' ocal =AdaptiveAvgPool(f local ); The global one-dimensional vector f g ' lobal and the local one-dimensional vector f l ' ocal Connect and get the classification feature f cat f cat =Concat(f g ' lobal ,f l ' ocal ); Concat() represents the connection process; The classification feature f cat After passing through the adaptive average pooling and fully connected layers in sequence, the classification result of the input monocular image is obtained.

6. The glaucoma fundus image classification method according to claim 5, characterized in that The aforementioned method of constructing an initial model for classifying glaucoma fundus images of binocular images based on the ConvNeXt model, the pooling layer, the linear layer and the dropout layer specifically includes the following steps: The processing process of constructing the initial model for classifying glaucoma fundus images of binocular images includes the following steps: The input images of the initial model for classifying glaucoma fundus images of binocular images are the left eye global fundus image and the left eye local optic disc image of the left eye image, and the right eye global fundus image and the right eye local optic disc image of the right eye image; Based on the ConvNeXt model, a left eye feature extraction network is constructed, and the left eye global fundus image and the left eye local optic disc image are input into the left eye feature extraction network for processing to obtain the left eye preliminary feature f l ' eft ; Based on the ConvNeXt model, a right eye feature extraction network is constructed, and the right eye global fundus image and the right eye local optic disc image are input into the right eye feature extraction network for processing to obtain the right eye preliminary feature f r ' ight ; The obtained left eye preliminary feature f l ' eft Through adaptive average pooling processing, the left eye feature f is obtained left ; The obtained preliminary feature f of the right eye r ' ight Through adaptive average pooling processing, the right eye feature f is obtained right ; In the order of left eye features and right eye features, the left eye features f left and right eye feature f right Connect and get the left eye classification feature f left-cat f left-cat =Concat(f left ,f right ); In the order of right eye features and left eye features, the right eye features f right and left eye feature f left Connect and get the right eye classification feature f right-cat f right-cat =Concat(f right ,f left ); The left eye classification network is constructed based on the pooling layer, linear layer and dropout layer, and the left eye classification feature f left-cat Input to the left eye classification network for processing to obtain the final left eye image classification result; The right eye classification network is constructed based on the pooling layer, linear layer and dropout layer, and the right eye classification feature f right-cat Input into the right eye classification network for processing to obtain the final right eye image classification result.

7. The glaucoma fundus image classification method according to claim 6, characterized in that The construction of the left eye feature extraction network based on the ConvNeXt model specifically includes the following steps: The left eye global fundus image is input into the ConvNeXt model for processing, and the left eye global image features are obtained through the activation function. Left eye global image features Expressed as In the formula is the global fundus image of the left eye; are the parameters of the ConvNeXt model corresponding to the global fundus image of the left eye; Represents the processing process of the ConvNeXt model corresponding to the global fundus image of the left eye; The processing process of the activation function corresponding to the global fundus image of the left eye; The left eye local optic disc image is input into the ConvNeXt model for processing, and the left eye local image features are obtained through the activation function Left eye local image features Expressed as In the formula It is the local optic disc image of the left eye; are the parameters of the ConvNeXt model corresponding to the local optic disc image of the left eye; The processing process of the ConvNeXt model corresponding to the local optic disc image of the left eye; The processing process of the activation function corresponding to the local optic disc image of the left eye; The obtained left eye global image features Through adaptive average pooling, it is converted into a global one-dimensional vector for the left eye for The obtained left eye local image features Through adaptive average pooling, it is converted into a local one-dimensional vector of the left eye for The left eye global one-dimensional vector and the left eye local one-dimensional vector Connect and get the preliminary feature f of the left eye l ' eft for The right eye feature extraction network is constructed based on the ConvNeXt model, and specifically includes the following steps: The right eye global fundus image is input into the ConvNeXt model for processing, and the right eye global image features are obtained through the activation function Right eye global image features Expressed as In the formula is the global fundus image of the right eye; are the parameters of the ConvNeXt model corresponding to the global fundus image of the right eye; It represents the processing process of the ConvNeXt model corresponding to the global fundus image of the right eye; The processing process of the activation function corresponding to the global fundus image of the right eye; The right eye local optic disc image is input into the ConvNeXt model for processing, and the right eye local image features are obtained through the activation function Right eye local image features Expressed as In the formula It is the partial optic disc image of the right eye; are the parameters of the ConvNeXt model corresponding to the local optic disc image of the right eye; The processing process of the ConvNeXt model corresponding to the local optic disc image of the right eye; The processing process of the activation function corresponding to the local optic disc image of the right eye; The obtained right eye global image features Through adaptive average pooling, it is converted into a global one-dimensional vector for the right eye for The obtained right eye local image features Through adaptive average pooling, it is converted into a local one-dimensional vector of the right eye for The right eye global one-dimensional vector and the right eye local one-dimensional vector Connect and get the initial feature f of the right eye r ' ight for 8. The glaucoma fundus image classification method according to claim 7, characterized in that The construction of the left eye classification network based on the pooling layer, the linear layer and the dropout layer specifically includes the following steps: The left eye classification feature f left-cat After feature extraction through the linear layer, processing through the maximum pooling layer, activation through the Relu function, and finally processing through the dropout layer, the first feature f of the left eye classification is obtained. left-cat-1 ; Classify the left eye with the first feature f left-cat-1 The feature is extracted again through the linear layer, processed through the maximum pooling layer, activated by the Relu function, and finally processed by the dropout layer to obtain the second feature f of the left eye classification. left-cat-2 ; Classify the left eye with the second feature f left-cat-2 After processing through the linear layer, the final classification result of the left eye image is obtained; The right eye classification network is constructed based on the pooling layer, the linear layer and the dropout layer, and specifically includes the following steps: The right eye classification feature f right-cat After feature extraction through the linear layer, processing through the maximum pooling layer, activation through the Relu function, and finally processing through the dropout layer, the first feature f of the right eye classification is obtained. right-cat-1 ; Classify the right eye with the first feature f right-cat-1 The feature is extracted again through the linear layer, processed through the maximum pooling layer, activated by the Relu function, and finally processed by the dropout layer to obtain the second feature f of the right eye classification. right-cat-2 ; Classify the right eye with the second feature f right-cat-2 After processing through the linear layer, the final right eye image classification result is obtained.

9. The glaucoma fundus image classification method according to claim 8, characterized in that The training described in step S4 specifically includes the following steps: For the initial model of glaucoma fundus image classification of monocular images, the following formula is used as the loss function L during training: Where i is global, local or cat; w i is the corresponding weight parameter; L i (o i ,y) is the corresponding label smoothed cross entropy loss function; y is the true label; o i is the predicted probability, and o i =Softmax(W i f i +b i ), Softmax() is the softmax function, W i is the weight matrix, b i is the bias term; For the initial model of glaucoma fundus image classification of binocular images, the cross entropy loss function is used as the loss function.

10. An imaging method comprising the glaucoma fundus image classification method according to any one of claims 1 to 9, characterized in that The following steps are also included: S6. Marking and re-imaging the obtained glaucoma fundus image classification result on the actual glaucoma fundus image to obtain a glaucoma fundus image with the glaucoma fundus image classification result.

Images