Eye fundus image quality grading method and device, equipment and storage medium

Abstract

The invention provides an eye fundus image quality grading method, device and equipment and a storage medium, and relates to the technical field of computer vision, and the method comprises the steps: obtaining an eye fundus image to be subjected to image quality grading, inputting the eye fundus image into a pre-trained eye fundus image quality grading model, and carrying out the image quality grading processing, the pre-trained eye fundus image quality grading model is obtained by training based on an OfficientNetV2 network improved by introducing a CBAM attention mechanism module, and a grading result of the eye fundus image to be subjected to image quality grading is obtained according to the pre-trained eye fundus image quality grading model. Therefore, accurate evaluation of the quality of the eye fundus image is realized through the deep learning model, artificial grading errors can be reduced, and the grading accuracy is improved.

Description

Technical Field

[0001] The present application relates to the field of computer vision technology, and in particular to a fundus image quality grading method, device, equipment and storage medium. Background Art

[0002] Fundus images are an important basis for ophthalmologists to diagnose a variety of eye diseases. However, due to the influence of various factors such as shooting conditions, equipment performance, and patient cooperation, the quality of fundus images often varies. Grading the quality of fundus images helps to screen out high-quality images for clinical diagnosis and improve the accuracy and efficiency of diagnosis. At present, the quality of fundus images is highly dependent on the subjective judgment of the photographer, which is inefficient and highly subjective, and the accuracy needs to be improved.

[0003] The main problems with fundus image quality grading at present include: Lack of unified evaluation standards: Different doctors may have different evaluation standards for image quality, resulting in inconsistent grading results. Inability to provide real-time feedback: In the clinical diagnosis process, the quality of fundus images cannot be evaluated in real time, which affects the continuity and efficiency of the diagnostic process. High labor costs: Traditional methods rely on manual image quality grading, which is inefficient and easily affected by factors such as fatigue and emotions. Inability to process large amounts of data: In clinical practice, a large number of fundus images need to be processed, and manual grading is difficult to meet the needs. Lack of objectivity: Subjective judgment is easily affected by personal experience and cognitive bias, resulting in grading results that are not objective enough. Poor system adaptability: Existing grading systems are often designed for specific types or brands of fundus cameras and lack adaptability to different devices and shooting conditions. In short, the existing fundus image quality grading methods are based on manual grading, which makes it difficult to quickly and accurately complete the grading of fundus images. Summary of the invention

[0004] In view of the above problems, the present application provides a fundus image quality grading method, including the following contents:

[0005] In a first aspect, the present application provides a method for grading fundus image quality, the method comprising:

[0006] Acquiring a fundus image to be graded for image quality;

[0007] Inputting the fundus image into a pre-trained fundus image quality grading model for image quality grading processing, wherein the pre-trained fundus image quality grading model is obtained by training an EfficientNetV2 network improved by introducing a CBAM attention mechanism module;

[0008] Obtaining a grading result of the fundus image to be graded according to the pre-trained fundus image quality grading model, wherein the grading result is any one of good, usable and rejected;

[0009] When the classification result is available, it is transferred to manual processing;

[0010] When the classification result is rejection, an instruction to retake an image is output.

[0011] Optionally, the training process of the pre-trained fundus image quality grading model includes:

[0012] Acquire a fundus image dataset and a classification label corresponding to the fundus image dataset, wherein the classification label is classified into good, usable, and rejected according to a preset fundus image quality classification rule;

[0013] Performing data preprocessing on the fundus image data set, and dividing the preprocessed data into a training set and a test set according to a preset ratio;

[0014] Building a fundus image quality grading network based on the EfficientNetV2 network, and introducing the CBAM attention mechanism module into the EfficientNetV2 network to obtain an improved EfficientNetV2 network;

[0015] The training set is input into the improved EfficientNetV2 network for training to obtain a trained fundus image quality grading model.

[0016] Optionally, it is characterized in that the CBAM attention mechanism module includes:

[0017] A channel attention module and a spatial attention module, wherein the channel attention module is used to output a 1D channel attention feature image, and the spatial attention module is used to output a 2D spatial attention feature image.

[0018] Optionally, the channel attention module outputs a 1D channel attention feature image including:

[0019] Input feature F, perform maximum pooling and average pooling processing on the feature F, and generate two independent feature images;

[0020] Processing the two feature images through a multi-layer perceptron MLP respectively to obtain an output result of the MLP;

[0021] After adding the output results of the MLP, a nonlinear transformation is performed using a Sigmoid function to generate a channel attention feature Mc(F);

[0022] The attention feature Mc(F) is element-wise multiplied with the original input feature F to obtain a preliminary 1D channel attention feature image F′.

[0023] Optionally, the spatial attention module outputs a 2D spatial attention feature image including:

[0024] Taking the 1D channel attention feature image F′ output by the channel attention module as input, performing maximum pooling and average pooling operations on F′ to obtain two new feature maps;

[0025] The two new feature maps are concatenated based on the channel dimension, and a convolution operation is performed through a 7×7 convolution kernel to reduce the feature dimension to a single channel;

[0026] The features reduced to a single channel are processed by the Sigmoid function to generate the spatial attention feature Ms;

[0027] Perform element-wise multiplication of the spatial attention feature Ms and F′ to obtain a 2D spatial attention feature image F".

[0028] Optionally, the pre-trained fundus image quality grading model is a regression model, and obtaining the grading result of the fundus image to be graded according to the pre-trained fundus image quality grading model includes:

[0029] A prediction score is obtained according to the pre-trained fundus image quality grading model, and a result corresponding to the prediction score is obtained according to a preset mapping rule as the grading result.

[0030] Optionally, the pre-trained fundus image quality grading model is a classification model, and obtaining the grading result of the fundus image to be graded according to the pre-trained fundus image quality grading model includes:

[0031] The category of the fundus image to be graded in terms of image quality is directly obtained according to the pre-trained fundus image quality grading model as the grading result.

[0032] In a second aspect, the present application provides a fundus image quality grading device, the device comprising:

[0033] An acquisition unit, used for acquiring a fundus image to be graded for image quality;

[0034] An input unit is used to input the fundus image into a pre-trained fundus image quality grading model for image quality grading processing, wherein the pre-trained fundus image quality grading model is obtained by training an EfficientNetV2 network improved by introducing a CBAM attention mechanism module;

[0035] A processing unit, configured to obtain a grading result of the fundus image to be graded according to the pre-trained fundus image quality grading model, wherein the grading result is any one of good, usable and rejected;

[0036] When the classification result is available, it is transferred to manual processing;

[0037] When the classification result is rejection, an instruction to retake an image is output.

[0038] Optionally, the device further comprises a model training unit, which is used to train a fundus image quality grading model, including:

[0039] Acquire a fundus image dataset and a classification label corresponding to the fundus image dataset, wherein the classification label is classified into good, usable, and rejected according to a preset fundus image quality classification rule;

[0040] Performing data preprocessing on the fundus image data set, and dividing the preprocessed data into a training set and a test set according to a preset ratio;

[0041] Building a fundus image quality grading network based on the EfficientNetV2 network, and introducing the CBAM attention mechanism module into the EfficientNetV2 network to obtain an improved EfficientNetV2 network;

[0042] The training set is input into the improved EfficientNetV2 network for training to obtain a trained fundus image quality grading model.

[0043] Optionally, the CBAM attention mechanism module includes:

[0044] A channel attention module and a spatial attention module, wherein the channel attention module is used to output a 1D channel attention feature image, and the spatial attention module is used to output a 2D spatial attention feature image.

[0045] Optionally, the channel attention module outputs a 1D channel attention feature image including:

[0046] Input feature F, perform maximum pooling and average pooling processing on the feature F, and generate two independent feature images;

[0047] Processing the two feature images through a multi-layer perceptron MLP respectively to obtain an output result of the MLP;

[0048] After adding the output results of the MLP, a nonlinear transformation is performed using a Sigmoid function to generate a channel attention feature Mc(F);

[0049] The attention feature Mc(F) is element-wise multiplied with the original input feature F to obtain a preliminary 1D channel attention feature image F′.

[0050] Optionally, the spatial attention module outputs a 2D spatial attention feature image including:

[0051] Taking the 1D channel attention feature image F′ output by the channel attention module as input, performing maximum pooling and average pooling operations on F′ to obtain two new feature maps;

[0052] The two new feature maps are concatenated based on the channel dimension, and a convolution operation is performed through a 7×7 convolution kernel to reduce the feature dimension to a single channel;

[0053] The features reduced to a single channel are processed by the Sigmoid function to generate the spatial attention feature Ms;

[0054] Perform element-wise multiplication of the spatial attention feature Ms and F′ to obtain a 2D spatial attention feature image F".

[0055] Optionally, the pre-trained fundus image quality grading model is a regression model, and the processing unit obtains the grading result of the fundus image to be graded according to the pre-trained fundus image quality grading model, including:

[0056] A prediction score is obtained according to the pre-trained fundus image quality grading model, and a result corresponding to the prediction score is obtained according to a preset mapping rule as the grading result.

[0057] Optionally, the pre-trained fundus image quality grading model is a classification model, and the grading result of the fundus image to be graded for image quality obtained by the processing unit according to the pre-trained fundus image quality grading model includes:

[0058] The category of the fundus image to be graded in terms of image quality is directly obtained according to the pre-trained fundus image quality grading model as the grading result.

[0059] In a third aspect, the present application provides a device comprising a memory and a processor, wherein the memory is used to store instructions or codes, and the processor is used to execute the instructions or codes so that the device performs the fundus image quality grading method introduced in any implementation of the first aspect.

[0060] In a fourth aspect, the present application provides a computer-readable storage medium, in which a code is stored. When the code is executed, the device executing the code implements the fundus image quality grading method introduced in any implementation of the first aspect.

[0061] The present application provides a fundus image quality grading method. When executing the method, a fundus image to be graded for image quality is obtained, and the fundus image is input into a pre-trained fundus image quality grading model for image quality grading processing. The pre-trained fundus image quality grading model is obtained based on the improved EfficientNetV2 network training after introducing the CBAM attention mechanism module. The grading result of the fundus image to be graded for image quality is obtained according to the pre-trained fundus image quality grading model. The grading result is any one of good, available and rejected. When the grading result is available, it is transferred to manual processing. When the grading result is rejected, an instruction to retake the image is output. In this way, the accurate assessment of the fundus image quality is achieved through a deep learning model, which can reduce human grading errors and improve the accuracy of grading. In this application, a channel attention mechanism and a spatial attention mechanism are added on the basis of EfficientNetV2 feature extraction, and the feature information of the two dimensions of channel and space are considered at the same time, so that the model can pay more attention to the important features of the input data in detail, effectively improving the performance of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] In order to more clearly illustrate the technical solutions in this embodiment or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0063] Figure 1 A flow chart of a method for grading fundus image quality provided in an embodiment of the present application;

[0064] Figure 2 A schematic diagram of the structure of an improved EfficientNetV2 network provided in an embodiment of the present application;

[0065] Figure 3A flowchart of a training process of a fundus image quality grading model provided in an embodiment of the present application;

[0066] Figure 4 A schematic diagram of the structure of a fundus image quality grading device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0067] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0068] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0069] Figure 1 A flow chart of a method for grading fundus image quality provided in an embodiment of the present application. Figure 1 As shown, the fundus image quality grading method provided in the embodiment of the present application may include:

[0070] S101, obtaining a fundus image to be graded for image quality.

[0071] Fundus images refer to a standard objective diagnostic image obtained by a fundus camera in ophthalmology. The fundus is a general term for the tissue structure at the back of the eye, mainly including the retina, choroid, optic nerve and macula.

[0072] S102: Input the fundus image into a pre-trained fundus image quality grading model to perform image quality grading processing.

[0073] The pre-trained fundus image quality grading model is obtained by training the improved EfficientNetV2 network by introducing the CBAM attention mechanism module. The structure of the improved EfficientNetV2 network is as follows: Figure 2 As shown, Figure 2A structural schematic diagram of an improved EfficientNetV2 network provided in an embodiment of the present application, wherein the input features are processed according to the ConvolutionalBlockAttention Module attention mechanism module to obtain improved features. The improved EfficientNetV2 network in the embodiment of the present application is introduced in detail below.

[0074] The EfficientNetV2 network is a deep learning model. In the embodiment of the present application, the fundus image quality grading model is constructed based on an improved version of the EfficientNetV2 network. The present application improves the EfficientNetV2 network by introducing the CBAM attention mechanism module. Based on CBAM, not only the importance of different feature channels is considered to be different, but also the importance of different positions of the same feature channel is considered to be different. This improves the fundus image quality grading model obtained by introducing the CBAM attention mechanism module based on the EfficientNetV2 network, which can more accurately capture the key information in the fundus image and significantly improve the grading accuracy.

[0075] ConvolutionalBlockAttention Module attention mechanism module (CBAM for short) is a lightweight attention module used to enhance the feature expression ability of convolutional neural network (CNN). It combines the two mechanisms of channel attention module (ChannelAttentionModule) and spatial attention module (SpatialAttentionModule), which can adaptively adjust the importance of different channels and positions in the feature map, so that the model can focus more on key information. In an embodiment of the present application, the channel attention module is used to output a 1D channel attention feature image, and the spatial attention module is used to output a 2D spatial attention feature image. Specifically, an intermediate feature image F is taken as input, where F∈R C×H×W , the 1D channel attention image Mc is inferred in turn through the CBAM attention mechanism module, where Mc∈R C×1×1 ; Perform element-wise multiplication of the channel attention image Mc and F to generate an improved feature F′. The spatial attention module generates a 2D spatial attention image Ms based on the improved feature F′, where Ms∈R 1×H×W . The spatial attention image Ms is element-wise multiplied with F′ to generate the final improved feature F". In the above description, C is (Channels, number of channels), H is (Height, height), W is (Width, width), and R is a set of real numbers.

[0076] The following is a detailed description of the process of obtaining the improved feature image using the channel attention module and the spatial attention module:

[0077] Channel Attention Module:

[0078] The input feature map F is processed by the maximum pooling (MaxPooling) and average pooling (AveragePooling) based on width and height respectively, and two C×1×1 feature images, AvgPool(F) and MaxPool(F) are obtained. These two feature images are sent to the multi-layer perceptron (MLP) neural network respectively, and the features MLP (AvgPool(F)) and MLP (MaxPool(F)) output by the MLP are subjected to element-based addition operations, and then the sigmoid activation function is passed to generate the final channel attention feature Mc(F). Mc(F) and the input feature map F are element-wise multiplied to generate the improved feature F′, that is, in It is an element-wise multiplication operation.

[0079] The calculation formula of the channel attention module is:

[0080] Mc(F)=σ(MLP(AvgPool(F))+MLP(MaxPool(F)))

[0081] =σ(W1(W0(F c avg ))+W1(W0(F c max )))

[0082] Where r is the reduction rate, W0 and W1 represent different weight matrices, and σ represents the Sigmoid function.

[0083] The spatial attention module aims to focus on the spatial location information of the feature map. The specific steps are as follows:

[0084] The feature map F′ output by the channel attention module is used as the input feature map. F′ is subjected to channel-based maximum pooling and average pooling to obtain two 1×H×W feature maps, namely: F s avg ∈R 1×H×W and F s max ∈R 1×H×W. The two feature maps are concatenated based on the channel to obtain a 2×H×W feature map. Then, after a 7×7 convolution operation, the number of channels of the feature map is reduced to 1, that is, 1×H×W. Then, the sigmoid is used to generate the spatial attention feature, that is, Ms(F′). Finally, the feature is multiplied with the input feature of the module. The final improved feature F" is obtained.

[0085] The calculation formula of the spatial attention module is:

[0086] Ms(F′)=σ(f 7×7 ([AvgPool(F′);MaxPool(F′)]))

[0087] =σ(f 7×7 ([F s avg ; F s max ]))

[0088] Among them, σ is the sigmoid activation function, f 7×7 is a 7×7 convolution operation, F s avg represents the average pooling feature, F s max Represents the maximum pooling feature.

[0089] By combining channel attention and spatial attention mechanisms, CBAM can adaptively adjust the importance of different channels and positions in the feature map, thereby improving the performance of the model.

[0090] The training process of the fundus image quality grading model is described in detail below and will not be expanded here.

[0091] S103. Obtaining a grading result of the fundus image to be graded according to the pre-trained fundus image quality grading model.

[0092] The trained fundus image quality grading model can automatically analyze and process the input fundus image, and the processing results include any one of good, usable and rejected. Good means that the image quality is good and can be used directly, usable means that the image quality generally requires further manual judgment to determine whether it can be used, and rejected means that the image quality is very poor and cannot be used and needs to be re-photographed.

[0093] The above embodiment introduces a method for grading fundus image quality. In the above method, the fundus image is input into a pre-trained fundus image quality grading model for image quality grading processing, and finally the pre-trained fundus image quality grading model can output the classification result of the image. The training process of the pre-trained fundus image quality grading model provided in the present application is introduced in conjunction with a specific embodiment. Figure 3 As shown, Figure 3 A flowchart of a training process of a fundus image quality grading model provided in an embodiment of the present application, the process comprising:

[0094] S301, acquiring a fundus image dataset and classification labels corresponding to the fundus image dataset, wherein the classification labels are classified into good, usable, and rejected according to a preset fundus image quality classification rule.

[0095] When constructing a fundus image dataset, you can collect open-source fundus image datasets from the Internet. These datasets are usually publicly provided by research institutions, universities, or medical organizations for scientific research or algorithm development. Or you can obtain desensitized fundus images from hospitals. Desensitization usually means removing personal identification information from the image to protect patient privacy while retaining the medical features in the image for research use.

[0096] The classification labels corresponding to the fundus image data set are set according to the preset fundus image quality classification rules. These classification labels are generally used to evaluate the quality of fundus images for subsequent analysis, processing or algorithm development. In an embodiment of the present application, the classification labels are set to three levels: "good", "usable" and "rejected". These levels are generally divided based on the characteristics of image clarity, contrast, noise level, integrity, etc. "Good" means that the image quality is high, the clarity is high, the contrast is strong, and the noise level is low, which is suitable for subsequent image analysis or algorithm development. "Usable" means that the image quality is acceptable, although there are some defects (such as slight blur, noise, etc.), but it does not affect the basic image analysis or algorithm development. "Reject" means that the image quality is poor, there are serious defects (such as blur, too much noise, missing key information, etc.), which is not suitable for subsequent image analysis or algorithm development.

[0097] Specifically, in the embodiments of the present application, an image quality grading standard (classification rule) is formulated according to clinical practice, which can be divided into the following three levels:

[0098] "Good" grade, recorded as 0:

[0099] 1. The retinal image does not contain low-quality factors (overexposure, poor exposure, blur, artifacts) and the characteristics of retinal lesions are clearly visible;

[0100] 2. The anatomical structure is complete and the edges are clear.

[0101] "Available" level, denoted as 1:

[0102] The retinal images have some minor low-quality indicators, such as poor exposure, overexposure, artifacts, and blur, but the main structures such as the optic disc, macular area, and lesions are clear enough.

[0103] 1. There is a slight poor exposure problem, which makes less than 1 / 3 of the blood vessels in the retinal image unclear;

[0104] 2. There is a slight overexposure problem, which makes less than 1 / 3 of the blood vessels in the retinal image unclear;

[0105] 3. There is a slight blur problem, which makes less than 1 / 3 of the blood vessels in the retinal image unclear;

[0106] 4. There is a slight artifact problem, which makes less than 1 / 3 of the blood vessels in the retinal image unclear;

[0107] 5. The edges of the optic disc and macula are not clear, but the structures of the optic disc and macula are intact;

[0108] 6. The optic disc is rotated and only one side of the blood vessels is intact.

[0109] The 'rejection' level is 2:

[0110] The retinal images had serious quality issues and the structures of the optic disc and macular areas were incomplete.

[0111] 1. There is a serious exposure problem, which makes more than 1 / 3 of the blood vessels in the retinal image unclear;

[0112] 2. There is a serious overexposure problem, which makes more than 1 / 3 of the blood vessels in the retinal image unclear;

[0113] 3. There is a serious blurring problem, which makes more than 1 / 3 of the blood vessels in the retinal image unclear;

[0114] 4. There are serious artifact problems, which make more than 1 / 3 of the blood vessels in the retinal image unclear;

[0115] 5. The optic disc and macula were not photographed or were incompletely photographed.

[0116] S302: preprocess the fundus image data set, and divide the preprocessed data into a training set and a test set according to a preset ratio.

[0117] Data preprocessing is a key step in the development of machine learning or deep learning models. It aims to improve the data quality and make it more suitable for subsequent model training. For fundus image datasets, the data preprocessing process includes the following steps:

[0118] Image enhancement: In order to increase the diversity and richness of the training data set, thereby improving the generalization and robustness of the deep learning model and improving the generalization ability of the model, the image can be enhanced to increase the diversity of the data set and help the model learn more features. In the embodiment of the present application, 16 data enhancement operations are included, including AutoContrast: Automatically adjust the contrast of the image to maximize the difference between the brightest and darkest parts of the image. Equalize: Use histogram equalization technology to adjust the grayscale distribution of the image so that the grayscale value is more evenly distributed in the image. Invert: Invert the color of the image, that is, white becomes black, and black becomes white (for color images, the color of each channel is inverted). Rotate: Randomly rotate the image by a certain angle (for example, any angle between 0° and 360°). Posterize: Convert the image to a version with fewer color levels, simulating old photos or printing effects. Solarize: Invert the pixel values ​​of the image according to a threshold (pixel values ​​below the threshold become the maximum value minus the original value, and those above the threshold remain unchanged). Color: Randomly change the color balance of the image, such as increasing or decreasing the value of the red, green, or blue channel. Contrast: Adjust the contrast of the image, increase or decrease the difference between the light and dark parts of the image. Brightness: Adjust the brightness of the image, increase or decrease the overall brightness of the image. Sharpness: Adjust the sharpness of the image to make the image edges clearer or more blurred. ShearX: Shear the image along the X-axis to produce a parallelogram effect. ShearY: Shear the image along the Y-axis. TranslateX: Translate the image along the X-axis. TranslateY: Translate the image along the Y-axis. Cutout: Randomly cut out one or more rectangular areas in the image and set the pixel values ​​of these areas to 0 or a fixed value to simulate occlusion. SolarizeAdd: Similar to Solarize, but with additional adjustment parameters to more flexibly control the inversion of pixel values. When performing data augmentation, randomly select n (n∈[1,3]) of the above 16 data augmentation operations to augment the data each time. This random selection method increases the diversity and unpredictability of data augmentation, which helps the model learn more feature representations. In specific implementation, different data augmentation operations can be randomly applied to each batch of data during the data loading phase. This way, each batch of data will be different, increasing the generalization ability of the model. At the same time, since data augmentation is applied in real time during training, there is no need to increase the storage space overhead.

[0119] Noise Removal: Remove noise from the image to improve the clarity of the image. This can be achieved through filtering methods (such as mean filtering, Gaussian filtering, etc.).

[0120] Normalization / Standardization: Normalize or standardize the pixel values ​​of an image to a specific range (such as 0-1 or -1 to 1) to ensure that the model can learn stably during training.

[0121] Data cleaning: Check and remove outliers or corrupted images in the dataset to ensure the completeness and accuracy of the dataset.

[0122] After data preprocessing is completed, the dataset needs to be divided into training set and test set. This is a standard practice in machine learning or deep learning, which aims to evaluate the performance of the model on unseen data. According to the specific needs of the project and the size of the dataset, a suitable ratio is preset to divide the training set and test set. Common ratios include 80% training set and 20% test set, or 70% training set, 15% validation set and 15% test set, etc. In order to ensure that the training set and test set are representative, a random division method is usually adopted. This means that the probability of each image being assigned to the training set or the test set is equal.

[0123] S303. Construct a fundus image quality grading network based on the EfficientNetV2 network, and introduce a CBAM attention mechanism module into the EfficientNetV2 network to obtain an improved EfficientNetV2 network.

[0124] The implementation process of the improved EfficientNetV2 network obtained by introducing the CBAM attention mechanism module into the EfficientNetV2 network refers to the implementation steps of step S102 in the previous embodiment, which will not be repeated here.

[0125] S304, inputting the training set into the improved EfficientNetV2 network for training to obtain a trained fundus image quality grading model.

[0126] When training, use pre-trained weights for transfer learning and model fine-tuning. Transfer learning is a machine learning technique that uses a pre-trained model on a task (usually called the source task) to accelerate the model training process on a new task (target task) and improve model performance. In deep learning, this usually means using models trained on large datasets (such as ImageNet) as a starting point. These models have learned many useful feature representations.

[0127] Pre-trained weights refer to the parameters of a neural network model that has been pre-trained on a large dataset. These models typically cover a wide range of tasks such as image recognition and object detection, and have learned rich feature representations. These features are useful for many related tasks and can therefore serve as a starting point for transfer learning.

[0128] In this application, by utilizing transfer learning, a model trained on a large image dataset can be used as a starting point and fine-tuned for the specific task of fundus image quality grading. Doing so can speed up the training process. Since the pre-trained model has already learned many useful feature representations, we can start training from a model that already has good initialization weights, thereby significantly reducing training time. Improve model performance. By fine-tuning the pre-trained model to adapt to the specific task of fundus image quality grading, we can obtain better model performance than training from scratch. This is because the pre-trained model has learned many general image-related features that are also beneficial for fundus image quality grading tasks.

[0129] Set the model parameter adjustment strategy to optimize the training process. Initially set a small learning rate (such as 0.001) and use the Adam optimizer for training. Apply the Warmup strategy in the first 5 epochs and gradually increase the learning rate. Starting from the 6th epoch, monitor the test accuracy of the model. If the test accuracy does not improve significantly within 10 consecutive epochs, reduce the learning rate by 0.1. If needed, you can repeatedly apply the learning rate decay strategy until the model reaches satisfactory performance or the training stops.

[0130] In an implementation method of an embodiment of the present application, the pre-trained fundus image quality grading model is a regression model, and obtaining the grading result of the fundus image to be graded according to the pre-trained fundus image quality grading model includes:

[0131] A prediction score is obtained according to the pre-trained fundus image quality grading model, and a result corresponding to the prediction score is obtained according to a preset mapping rule as the grading result.

[0132] When training a regression model, the loss function is the mean squared error (MSE), and the output of the last layer of the model is set to 1.

[0133] The model prediction score is score, and the category mapping is:

[0134]

[0135] Wherein, true is the true category of the fundus image, predict is the mapping category of the predicted score, N is the number of fundus images, and i = (1, 2, 3...N).

[0136] Through the regression model, the trained fundus image quality grading model processes the input fundus image to be graded to obtain a prediction score, for example, 0.4, and the classification result corresponds to good.

[0137] In an implementation method of an embodiment of the present application, the pre-trained fundus image quality grading model is a classification model, and obtaining the grading result of the fundus image to be graded according to the pre-trained fundus image quality grading model includes:

[0138] The category of the fundus image to be graded in terms of image quality is directly obtained according to the pre-trained fundus image quality grading model as the grading result.

[0139] When the pre-trained fundus image quality grading model is a classification model, when training the classification model, the last layer output of the model is set to 3 categories, the activation function is sofamax, and the loss function is CategoricalCrossentropy(), so that the output result is directly the result corresponding to the image, such as good.

[0140] The above embodiment introduces a flowchart of the training process of the fundus image quality grading model. After the training is completed, in order to ensure the quality of the model, it is necessary to optimize the model, use the trained model to infer new images, generate category documents for new images, correct image categories, add new data, and continuously iterate and optimize the model until the model performance meets the requirements. In addition, the model can also be evaluated by indicators such as precision, recall, and accuracy. Select the optimal model for model transplantation and deployment.

[0141] The above are some specific implementations of a fundus image quality grading method provided in the embodiment of the present application. Based on this, the present application also provides a corresponding device. The device provided in the embodiment of the present application will be introduced from the perspective of functional modularization.

[0142] Figure 4 A schematic diagram of the structure of a fundus image quality grading device provided in an embodiment of the present application. Figure 4 As shown, the fundus image quality grading device 400 provided in the embodiment of the present application includes:

[0143] An acquisition unit 410 is used to acquire a fundus image to be graded for image quality;

[0144] An input unit 420 is used to input the fundus image into a pre-trained fundus image quality grading model for image quality grading processing, wherein the pre-trained fundus image quality grading model is obtained by training an EfficientNetV2 network improved by introducing a CBAM attention mechanism module;

[0145] A processing unit 430 is used to obtain a grading result of the fundus image to be graded according to the pre-trained fundus image quality grading model, wherein the grading result is any one of good, usable and rejected;

[0146] When the classification result is available, it is transferred to manual processing;

[0147] When the classification result is rejection, an instruction to retake an image is output.

[0148] In an implementation of the embodiment of the present application, the device further includes a model training unit, which is used to train a fundus image quality grading model, including:

[0149] Acquire a fundus image dataset and a classification label corresponding to the fundus image dataset, wherein the classification label is classified into good, usable, and rejected according to a preset fundus image quality classification rule;

[0150] Performing data preprocessing on the fundus image data set, and dividing the preprocessed data into a training set and a test set according to a preset ratio;

[0151] Building a fundus image quality grading network based on the EfficientNetV2 network, and introducing the CBAM attention mechanism module into the EfficientNetV2 network to obtain an improved EfficientNetV2 network;

[0152] The training set is input into the improved EfficientNetV2 network for training to obtain a trained fundus image quality grading model.

[0153] In one implementation of the embodiment of the present application, the CBAM attention mechanism module includes:

[0154] A channel attention module and a spatial attention module, wherein the channel attention module is used to output a 1D channel attention feature image, and the spatial attention module is used to output a 2D spatial attention feature image.

[0155] In one implementation of the embodiment of the present application, the channel attention module outputs a 1D channel attention feature image including:

[0156] Input feature F, perform maximum pooling and average pooling processing on the feature F, and generate two independent feature images;

[0157] Processing the two feature images through a multi-layer perceptron MLP respectively to obtain an output result of the MLP;

[0158] After adding the output results of the MLP, a nonlinear transformation is performed using a Sigmoid function to generate a channel attention feature Mc(F);

[0159] The attention feature Mc(F) is element-wise multiplied with the original input feature F to obtain a preliminary 1D channel attention feature image F′.

[0160] In one implementation of the embodiment of the present application, the spatial attention module outputs a 2D spatial attention feature image including:

[0161] Taking the 1D channel attention feature image F′ output by the channel attention module as input, performing maximum pooling and average pooling operations on F′ to obtain two new feature maps;

[0162] The two new feature maps are concatenated based on the channel dimension, and a convolution operation is performed through a 7×7 convolution kernel to reduce the feature dimension to a single channel;

[0163] The features reduced to a single channel are processed by the Sigmoid function to generate the spatial attention feature Ms;

[0164] The spatial attention feature Ms is element-wise multiplied by the F′ to obtain the 2D spatial attention feature image F".

[0165] In an implementation of the embodiment of the present application, the pre-trained fundus image quality grading model is a regression model, and the processing unit obtains the grading result of the fundus image to be graded according to the pre-trained fundus image quality grading model, including:

[0166] A prediction score is obtained according to the pre-trained fundus image quality grading model, and a result corresponding to the prediction score is obtained according to a preset mapping rule as the grading result.

[0167] In an implementation of the embodiment of the present application, the pre-trained fundus image quality grading model is a classification model, and the processing unit obtains the grading result of the fundus image to be graded according to the pre-trained fundus image quality grading model, including:

[0168] The category of the fundus image to be graded in terms of image quality is directly obtained according to the pre-trained fundus image quality grading model as the grading result.

[0169] The embodiments of the present application also provide corresponding devices and computer storage media for implementing the solutions provided by the embodiments of the present application.

[0170] The device includes a memory and a processor, the memory is used to store instructions or codes, and the processor is used to execute the instructions or codes so that the device executes the method described in any embodiment of the present application.

[0171] The computer storage medium stores codes, and when the codes are executed, a device executing the codes implements the method described in any embodiment of the present application.

[0172] Through the description of the above implementation methods, it can be known that those skilled in the art can clearly understand that all or part of the steps in the above-mentioned embodiment method can be implemented by means of software plus a general hardware platform. Based on such an understanding, the technical solution of the present application can be embodied in the form of a software product, which can be stored in a storage medium, such as a read-only memory (ROM) / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network communication device such as a router) to execute the methods described in each embodiment of the present application or some parts of the embodiments.

[0173] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0174] It should also be noted that the various embodiments in this specification are described in a progressive manner, and the same and similar parts between the various embodiments can refer to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device and apparatus embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can refer to the partial description of the method embodiments. The device and apparatus embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components indicated as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0175] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for grading fundus image quality, characterized in that: The method comprises: Acquiring a fundus image to be graded for image quality; Inputting the fundus image into a pre-trained fundus image quality grading model for image quality grading processing, wherein the pre-trained fundus image quality grading model is obtained by training an EfficientNetV2 network improved by introducing a CBAM attention mechanism module; Obtaining a grading result of the fundus image to be graded according to the pre-trained fundus image quality grading model, wherein the grading result is any one of good, usable and rejected; When the classification result is available, it is transferred to manual processing; When the classification result is rejection, an instruction to retake an image is output.

2. The method according to claim 1, characterized in that: The training process of the pre-trained fundus image quality grading model includes: Acquire a fundus image dataset and a classification label corresponding to the fundus image dataset, wherein the classification label is classified into good, usable, and rejected according to a preset fundus image quality classification rule; Performing data preprocessing on the fundus image data set, and dividing the preprocessed data into a training set and a test set according to a preset ratio; Building a fundus image quality grading network based on the EfficientNetV2 network, and introducing the CBAM attention mechanism module into the EfficientNetV2 network to obtain an improved EfficientNetV2 network; The training set is input into the improved EfficientNetV2 network for training to obtain a trained fundus image quality grading model.

3. The method according to claim 2, characterized in that The CBAM attention mechanism module includes: A channel attention module and a spatial attention module, wherein the channel attention module is used to output a 1D channel attention feature image, and the spatial attention module is used to output a 2D spatial attention feature image.

4. The method according to claim 3, characterized in that: The channel attention module outputs a 1D channel attention feature image including: Input feature F, perform maximum pooling and average pooling processing on the feature F, and generate two independent feature images; Processing the two feature images through a multi-layer perceptron MLP respectively to obtain an output result of the MLP; After adding the output results of the MLP, a nonlinear transformation is performed using a Sigmoid function to generate a channel attention feature Mc(F); The attention feature Mc(F) is element-wise multiplied with the original input feature F to obtain a preliminary 1D channel attention feature image F′.

5. The method according to claim 4, characterized in that: The spatial attention module outputs a 2D spatial attention feature image including: Taking the 1D channel attention feature image F′ output by the channel attention module as input, performing maximum pooling and average pooling operations on F′ to obtain two new feature maps; The two new feature maps are concatenated based on the channel dimension, and a convolution operation is performed through a 7×7 convolution kernel to reduce the feature dimension to a single channel; The features reduced to a single channel are processed by the Sigmoid function to generate the spatial attention feature Ms; Perform element-wise multiplication of the spatial attention feature Ms and F′ to obtain a 2D spatial attention feature image F".

6. The method according to claim 1, characterized in that The pre-trained fundus image quality grading model is a regression model, and obtaining the grading result of the fundus image to be graded according to the pre-trained fundus image quality grading model includes: A prediction score is obtained according to the pre-trained fundus image quality grading model, and a result corresponding to the prediction score is obtained according to a preset mapping rule as the grading result.

7. The method according to claim 1, characterized in that The pre-trained fundus image quality grading model is a classification model, and the grading result of the fundus image to be graded according to the pre-trained fundus image quality grading model includes: The category of the fundus image to be graded in terms of image quality is directly obtained according to the pre-trained fundus image quality grading model as the grading result.

8. A fundus image quality grading device, characterized in that: The device comprises: An acquisition unit, used for acquiring a fundus image to be graded for image quality; An input unit is used to input the fundus image into a pre-trained fundus image quality grading model for image quality grading processing, wherein the pre-trained fundus image quality grading model is obtained by training an EfficientNetV2 network improved by introducing a CBAM attention mechanism module; A processing unit, configured to obtain a grading result of the fundus image to be graded according to the pre-trained fundus image quality grading model, wherein the grading result is any one of good, usable and rejected; When the classification result is available, it is transferred to manual processing; When the classification result is rejection, an instruction to retake an image is output.

9. A computing device, characterized in that The computing device includes: a memory and a processor; The memory is used to store computer programs; The processor is configured to implement the method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images