Quality control module for eye fundus image and film reading system comprising quality control module

Abstract

The invention discloses a quality control module for eye fundus images and a film reading system comprising the quality control module. The quality control module comprises a negative quality control module and a positive quality control module; the negative quality control module receives the negative result image, obtains a negative quality control result of the negative result image by using a first quality control model which configures a first confidence threshold based on a preset negative prediction rate, and obtains a final classification result of the negative result image based on the negative result image or takes the negative result image as a first to-be-arbitrated image; and the positive quality control module receives the positive result image, acquires a positive quality control result of the positive result image by using a second quality control model which configures a second confidence threshold based on a preset positive prediction rate, and acquires a final classification result of the positive result image based on the positive result image or takes the positive result image as a second to-be-arbitrated image. Therefore, a negative result image with low risk and a positive result image with high risk can be distinguished.

Description

[0001] This application is a divisional application of a patent application with an application date of 2021-03-24, an application number of 2021103163509, and an invention name of a reading system and a reading method for fundus image classification. Technical Field

[0002] The present disclosure generally relates to a quality control module for fundus images and an image reading system including the quality control module. Background Art

[0003] Medical images often contain many details of body structures or tissues. In modern hospitals, most treatment information comes from medical images such as fundus images. In clinical practice, understanding these details in medical images can help doctors identify related diseases. Medical images have developed into the main method for clinical disease identification. However, traditional medical image-based disease information identification mainly relies on professional physicians to make judgments based on experience. In this case, developing an automatic reading technology that can assist doctors in identifying related diseases has become a hot topic in the field of medical imaging. With the development of artificial intelligence technology, reading technologies based on computer vision and artificial intelligence such as machine learning have been developed and applied in medical image recognition.

[0004] For example, patent document 1 (CN105513077A) discloses a system for screening diabetic retinopathy, which includes: a fundus image acquisition device, an image processing and screening device, and a report output device. The fundus image acquisition device is used to collect or receive fundus images of the person being examined; the image processing and screening device is used to process the fundus image and detect whether there are lesions therein, and then transmit the detection results to the report output device; the report output device outputs a corresponding detection report based on the detection results.

[0005] However, in actual clinical applications, due to the diversity of fundus images, the screening system described in Patent Document 1 may output erroneous or inaccurate detection reports when processing certain fundus images, resulting in a decrease in the classification accuracy of the screening system. Summary of the invention

[0006] The present disclosure is proposed in view of the above-mentioned situation, and its purpose is to provide a fundus image classification reading system and reading method that can improve the classification accuracy.

[0007] To this end, the first aspect of the present disclosure provides a fundus image classification reading system, including: an acquisition module, which is used to acquire a fundus image; a preprocessing module, which is used to preprocess the fundus image to obtain a preprocessed fundus image; a first classification module, which receives the preprocessed fundus image, classifies the preprocessed fundus image using a first classification model based on deep learning to obtain a first classification result, and obtains a classification result type including whether re-classification is required based on the first classification result; a grouping module, which divides the preprocessed fundus image into a negative result image, a positive result image and an image to be reclassified based on the first classification result and the classification result type; a first quality control module, which includes a negative quality control module and a positive quality control module, the negative quality control module receives the negative result image, and obtains a negative quality control result of the negative result image using a first quality control model configured with a first confidence threshold based on a preset negative prediction rate, if the negative quality control result is consistent with the first classification result, the negative quality control result is used as the final classification result, otherwise the negative quality control result is used as the final classification result. The negative result image is used as the first image to be arbitrated. The positive quality control module receives the positive result image, and obtains the positive quality control result of the positive result image by using the second quality control model configured with the second confidence threshold based on the preset positive prediction rate. If the positive quality control result is consistent with the first classification result, the positive quality control result is used as the final classification result, otherwise the positive result image is used as the second image to be arbitrated; a second classification module receives the image to be reclassified, and classifies the image to be reclassified by using the second classification model based on deep learning and trained for the image to be reclassified to obtain a second classification result. If the second classification result is consistent with the first classification result, the second classification result is used as the final classification result, otherwise the image to be reclassified is used as the third image to be arbitrated; and an arbitration module receives the first image to be arbitrated, the second image to be arbitrated, or the third image to be arbitrated as the images to be arbitrated, and arbitrates the images to be arbitrated to obtain an arbitration classification result as the final classification result. In this case, the fundus images are divided into negative result images, positive result images and images to be reclassified. For negative result images and positive result images, negative result images with lower risk are obtained based on the negative prediction rate and positive result images with higher risk are obtained based on the positive prediction rate, and consistency judgment is performed. For the images to be reclassified, the second classification model is used for further classification, and finally the images to be arbitrated are arbitrated. In this way, the classification accuracy of the film reading system can be improved.

[0008] In addition, in the film reading system involved in the first aspect of the present disclosure, optionally, the first classification model includes multiple sub-classification models for receiving the pre-processed fundus image and obtaining sub-classification results for each type of diabetic retinopathy, and the first classification module obtains the first classification result based on the multiple sub-classification results. Thus, the first classification result can be obtained based on multiple sub-classification models.

[0009] In addition, in the film reading system involved in the first aspect of the present disclosure, optionally, the preset negative predictive rate is 95% to 99%, and the preset positive predictive rate is 95% to 99%. Thus, the preset negative predictive rate and the preset positive predictive rate can be obtained.

[0010] In addition, in the film reading system involved in the first aspect of the present disclosure, optionally, the first classification module outputs the first classification result according to the retinopathy grading system used in the UK National Retinopathy Screening Program. In this case, based on the mature retinopathy grading system, the classification accuracy of the film reading system can be further improved.

[0011] In addition, in the film reading system involved in the first aspect of the present disclosure, optionally, the first classification result includes no retinopathy, background period, pre-proliferation period and proliferative period; the negative result image includes the pre-processed fundus image whose first classification result is no retinopathy and the classification result type is that it does not need to be reclassified; the positive result image includes the pre-processed fundus image whose first classification result is pre-proliferation period or proliferative period and the classification result type is that it does not need to be reclassified; the image to be reclassified includes the pre-processed fundus image whose classification result type is that it needs to be reclassified. In this case, dividing the pre-processed fundus image into a negative result image, a positive result image and an image to be reclassified can facilitate subsequent targeted processing of each image. Thereby, the classification accuracy of the film reading system can be further improved.

[0012] In addition, in the film reading system involved in the first aspect of the present disclosure, optionally, the film reading system also includes a self-checking module, which is used to perform a random check on the fundus images of the negative quality control results to determine whether the first confidence threshold meets the requirements and to perform a random check on the fundus images of the positive quality control results to determine whether the second confidence threshold meets the requirements. In this case, it can be further confirmed whether the first confidence threshold and the second confidence threshold meet the requirements. Thereby, the classification accuracy of the film reading system can be improved.

[0013] In addition, in the film reading system involved in the first aspect of the present disclosure, optionally, the first confidence threshold is configured using gold standard data based on the preset negative prediction rate; and the second confidence threshold is configured using gold standard data based on the preset positive prediction rate. Thus, the confidence threshold can be determined.

[0014] In addition, in the film reading system involved in the first aspect of the present disclosure, optionally, the film reading system further includes an output module, and the output module is used to output a result report. Thus, the result report can be output.

[0015] A second aspect of the present disclosure provides a method for classifying fundus images, comprising: acquiring a fundus image; preprocessing the fundus image to acquire a preprocessed fundus image; classifying the preprocessed fundus image using a first classification model based on deep learning to acquire a first classification result and acquiring a classification result type including whether reclassification is required based on the first classification result; dividing the preprocessed fundus image into a negative result image, a positive result image and an image to be reclassified based on the first classification result and the classification result type; acquiring a negative quality control result of the negative result image using a first quality control model that configures a first confidence threshold based on a preset negative prediction rate; if the negative quality control result is consistent with the first classification result, taking the negative quality control result as the final classification result; otherwise taking the negative result image as the first image to be classified, using a first quality control model that configures a first confidence threshold based on a preset negative prediction rate The second quality control model configured with the second confidence threshold by the positive prediction rate obtains the positive quality control result of the positive result image. If the positive quality control result is consistent with the first classification result, the positive quality control result is used as the final classification result, otherwise the positive result image is used as the second image to be arbitrated; the image to be reclassified is classified by the second classification model based on deep learning and trained for the image to be reclassified to obtain the second classification result. If the second classification result is consistent with the first classification result, the second classification result is used as the final classification result, otherwise the image to be reclassified is used as the third image to be arbitrated; and the first image to be arbitrated, the second image to be arbitrated or the third image to be arbitrated is used as the image to be arbitrated, and the image to be arbitrated is arbitrated to obtain the arbitration classification result and used as the final classification result. In this case, the fundus image is divided into a negative result image, a positive result image and an image to be reclassified. For the negative result image and the positive result image, a negative result image with a lower risk is obtained based on the negative prediction rate and a positive result image with a higher risk is obtained based on the positive prediction rate and a consistency judgment is performed, and for the image to be reclassified, the second classification model is used to further classify, and finally the image to be arbitrated is arbitrated. This can improve classification accuracy.

[0016] In addition, in the film reading method involved in the second aspect of the present disclosure, optionally, the fundus images of the negative quality control results are sampled to determine whether the first confidence threshold meets the requirements, and the fundus images of the positive quality control results are sampled to determine whether the second confidence threshold meets the requirements. In this case, it can be further confirmed whether the first confidence threshold and the second confidence threshold meet the requirements. In this way, the classification accuracy can be improved.

[0017] According to the present disclosure, a system and method for classifying fundus images with improved classification accuracy can be provided. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The present disclosure will now be explained in further detail by way of example only with reference to the accompanying drawings, in which:

[0019] Figure 1 is a diagram showing an application scenario of the fundus image classification and interpretation method involved in the example of the present disclosure.

[0020] Figure 2 is a block diagram showing a system for classifying fundus images according to an example of the present disclosure.

[0021] FIG. 3( a ) is a schematic diagram showing a fundus image according to an example of the present disclosure.

[0022] FIG. 3( b ) is a schematic diagram showing a fundus image according to an example of the present disclosure.

[0023] Figure 4 is a schematic diagram showing the convolution kernel used in the convolutional neural network of the first classification module involved in the example of the present disclosure.

[0024] Figure 5 is a block diagram showing a system for classifying fundus images according to an example of the present disclosure.

[0025] Figure 6 is a flowchart showing a method for interpreting fundus images for classification according to an example of the present disclosure. DETAILED DESCRIPTION

[0026] Hereinafter, with reference to the accompanying drawings, the preferred embodiments of the present disclosure are described in detail. In the following description, the same symbols are given to the same parts, and repeated descriptions are omitted. In addition, the accompanying drawings are only schematic diagrams, and the ratio of the sizes of the parts to each other or the shapes of the parts, etc. may be different from the actual ones. It should be noted that the terms "including" and "having" in the present disclosure and any variations thereof, such as the processes, methods, systems, products or devices of a series of steps or units included or possessed are not necessarily limited to those steps or units clearly listed, but may include or have other steps or units that are not clearly listed or inherent to these processes, methods, products or devices. All methods described in the present disclosure may be performed in any suitable order, unless otherwise indicated herein or clearly contradictory to the context.

[0027] The present disclosure relates to a fundus image classification reading system 200 and a reading method capable of improving classification accuracy. The fundus image classification reading system 200 may sometimes be referred to as the reading system 200, and the fundus image classification reading method may sometimes be referred to as the reading method.

[0028] Figure 1 is a diagram showing an application scenario of the fundus image classification and interpretation method involved in the example of the present disclosure.

[0029] In some examples, the image reading method (described later) can be applied to Figure 1 In the application scenario 100 shown. In the application scenario 100, the operator 110 can control the acquisition device 130 connected to the terminal 120 to acquire the fundus image of the fundus of the human eye 140. After the acquisition device 130 completes the fundus image acquisition, the terminal 120 can submit the fundus image to the server 150 through the computer network. The server 150 can implement the film reading method by executing the computer program instructions stored on the server 150. The film reading method receives the fundus image and generates a result report of the fundus image. The server 150 can return the generated result report of the fundus image to the terminal 120. In some examples, the terminal 120 can display the result report. In other examples, the result report can be stored in the memory of the terminal 120 or the server 150 as an intermediate result. In other examples, the fundus image received by the film reading method can be a fundus image stored in the terminal 120 or the server 150.

[0030] In some examples, the operator 110 may be a professional such as an ophthalmologist. In other examples, the operator 110 may be an ordinary person who has been trained in film reading. Film reading training may include but is not limited to the operation of the acquisition device 130 and the operation of the terminal 120 involving the film reading method. In some examples, the terminal 120 may include but is not limited to a laptop, a tablet computer, or a desktop computer. In some examples, the acquisition device 130 may be a camera. The camera may be, for example, a handheld fundus camera or a desktop fundus camera. In some examples, the acquisition device 130 may be connected to the terminal 120 via a serial port. In some examples, the acquisition device 130 may be integrated in the terminal 120.

[0031] In some examples, the server 150 may include one or more processors and one or more memories. The processor may include a central processing unit, a graphics processing unit, and any other electronic component capable of processing data, and may execute computer program instructions. The memory may be used to store computer program instructions. In some examples, the server 150 may implement the film reading method by executing the computer program instructions on the memory. In some examples, the server 150 may also be a cloud server.

[0032] Hereinafter, the image reading system 200 involved in the present disclosure will be described in detail with reference to the accompanying drawings. The image reading system 200 involved in the present disclosure is used to implement the above-mentioned image reading method. Figure 2 is a block diagram showing an image interpretation system 200 for classifying fundus images according to an example of the present disclosure.

[0033] In some examples, such as Figure 2As shown, the film reading system 200 may include an acquisition module 210, a preprocessing module 220, a first classification module 230, a grouping module 240, a first quality control module 250, a second classification module 260, and an arbitration module 270. In some examples, the acquisition module 210 may be used to acquire a fundus image, the preprocessing module 220 may be used to preprocess the fundus image to acquire a preprocessed fundus image, the first classification module 230 may be used to classify the preprocessed fundus image and acquire a first classification result and a classification result type, the grouping module 240 may classify the preprocessed fundus image into a negative result image, a positive result image, and an image to be reclassified, the first quality control module 250 may acquire a final classification result, a first image to be arbitrated, and a second image to be arbitrated based on the negative result image and the positive result image, the second classification module 260 may acquire a final classification result and a third image to be arbitrated based on the image to be reclassified, and the arbitration module 270 may be used to arbitrate the image to be arbitrated to acquire an arbitration classification result as the final classification result. In this case, the fundus images are divided into negative result images, positive result images and images to be reclassified. For the negative result images and positive result images, the negative result images with lower risk are obtained based on the negative prediction rate and the positive result images with higher risk are obtained based on the positive prediction rate and consistency judgment is performed. For the images to be reclassified, the second classification model is used for further classification, and finally the images to be arbitrated are arbitrated. In this way, the classification accuracy of the film reading system 200 can be improved.

[0034] Fig. 3(a) is a schematic diagram showing a fundus image involved in an example of the present disclosure. Fig. 3(b) is a schematic diagram showing a fundus image involved in an example of the present disclosure.

[0035] In some examples, the acquisition module 210 can be used to acquire fundus images. In some examples, the fundus image can be a color fundus image. The color fundus image can clearly present rich fundus information such as the optic disc, optic cup, macula, blood vessels, etc. In addition, the fundus image can be an image in RGB mode, CMYK mode, Lab mode or grayscale mode, etc. In some examples, the fundus image can be acquired by the acquisition device 130. In other examples, the fundus image can be a fundus image stored in the terminal 120 or the server 150. As examples of fundus images, for example, Figure 3 (a) and Figure 3 (b) are fundus images of different human eyes 140, respectively.

[0036] In some examples, the preprocessing module 220 may be used to preprocess the fundus image to obtain a preprocessed fundus image. Specifically, the preprocessing module 220 may obtain the fundus image output by the acquisition module 210, and preprocess the fundus image to obtain a preprocessed fundus image.

[0037] In some examples, the preprocessing module 220 can crop the fundus image. Generally speaking, since the fundus image acquired by the acquisition module 210 may have different image formats or sizes, it is necessary to crop the fundus image to convert the fundus image into an image in a fixed standard format. The fixed standard format may refer to images with the same format and size. For example, in some examples, the size of the fundus image after preprocessing can be unified into a fundus image of 256×256, 374×374, 512×512, 768×768 or 1024×1024 pixels.

[0038] In some examples, the preprocessing module 220 may perform normalization processing on the fundus image. In some examples, the normalization processing may include operations such as coordinate centering and scaling normalization on the fundus image. Thus, the differences between different fundus images can be overcome and the performance of the first classification model can be improved. In addition, in some examples, the preprocessing module 220 may include noise reduction, grayscale processing, etc. on the fundus image. Thus, the features of the fundus image can be highlighted. In some examples, the fundus image may not need to be preprocessed, and the fundus image may be directly classified later.

[0039] In some examples, the first classification module 230 can be used to classify the preprocessed fundus image and obtain the first classification result and the classification result type. In some examples, the first classification module 230 can also obtain the confidence of the first classification result. In some examples, the first classification module 230 can output the first classification result according to the retinopathy grading system used by the British National Retinopathy Screening Program. In some examples, the first classification result may include at least no retinopathy (R0), background period (R1), pre-proliferation period (R2) and proliferative period (R3). In this case, based on the mature retinopathy grading system, the classification accuracy of the film reading system 200 can be further improved. In some examples, the first classification result may also include no diabetic macular edema (M0) and macular edema (M1).

[0040] However, the examples of the present disclosure are not limited thereto, and in other examples, the first classification result may include at least a negative result and a positive result. In some examples, the first classification module 230 may filter out pre-processed fundus images that cannot be classified (e.g., pre-processed fundus images that cannot be classified due to poor image quality).

[0041] In some examples, the classification result type may include whether reclassification is required (e.g., reclassification is required and reclassification is not required). In some examples, the classification result type can be obtained based on the first classification result. In some examples, the classification result type of the preprocessed fundus image whose first classification result is the background period can be set to require reclassification, and other preprocessed fundus images can be set to not require reclassification. In some examples, whether reclassification is required to obtain the classification result type can be determined based on the confidence of the first classification result. For example, the preprocessed fundus image with a first classification result lower than a preset confidence (e.g., 40% or 50%) is set to require reclassification, and other preprocessed fundus images are set to not require reclassification.

[0042] In some examples, the first classification module 230 can classify the fundus image using a machine learning algorithm to obtain a first classification result. In some examples, the machine learning algorithm can be at least one of a traditional machine learning algorithm and a deep learning algorithm. In this case, a suitable machine learning algorithm can be selected according to actual needs. In some examples, a first classification model can be established based on a machine learning algorithm.

[0043] Figure 4 is a schematic diagram showing the convolution kernel used in the convolutional neural network of the first classification module 230 involved in the example of the present disclosure.

[0044] In some examples, the first classification model established based on the deep learning algorithm can be a convolutional neural network (CNN). In some examples, the convolutional neural network (CNN) can use a 3×3 convolution kernel (see Figure 4 ) automatically identifies features in fundus images. However, the examples disclosed herein are not limited thereto. In other examples, the convolution kernel of the convolution neural network (CNN) may be a 5×5 convolution kernel, a 2×2 convolution kernel, or a 7×7 convolution kernel. In this case, since the convolution neural network (CNN) has the characteristic of high efficiency in image feature recognition, the performance of the film reading system 200 can be effectively improved.

[0045] However, the examples of the present disclosure are not limited thereto. In other examples, the machine learning algorithm of the first classification module 230 may be a traditional machine learning algorithm. In some examples, the traditional machine learning algorithm may include, but is not limited to, a linear regression algorithm, a logistic regression algorithm, a decision tree algorithm, a support vector machine algorithm, or a Bayesian algorithm. In this case, the fundus features in the fundus image may be first extracted using an image processing algorithm, and then the fundus features may be input into a first classification model established based on a traditional machine learning algorithm to classify the fundus image.

[0046] In some examples, the first classification model may include multiple sub-classification models. Each sub-classification model may be for each type of diabetic retinopathy. Each sub-classification model may receive a pre-processed fundus image and obtain a sub-classification result. In some examples, the first classification module 230 may obtain a first classification result based on multiple sub-classification results. Thus, the first classification result can be obtained based on multiple sub-classification models.

[0047] Specifically, different sub-classification models can be established for no retinopathy, background stage, pre-proliferative stage, proliferative stage, no diabetic macular edema, and macular edema, and training can be performed to obtain a sub-classification result of whether it is a certain diabetic retinopathy (for example, background stage or not background stage) and a confidence level, and then the first classification result can be obtained according to each sub-classification result and confidence level. For example, the sub-classification result with the highest confidence level can be obtained as the first classification result.

[0048] In some examples, the grouping module 240 can divide the pre-processed fundus image into a negative result image, a positive result image, and an image to be reclassified. In some examples, the pre-processed fundus image can be divided into a negative result image, a positive result image, and an image to be reclassified based on the first classification result and the classification result type. In this case, the pre-processed fundus image is divided into a negative result image, a positive result image, and an image to be reclassified, which can facilitate the subsequent targeted processing of each image. As a result, the classification accuracy of the film reading system 200 can be further improved.

[0049] Specifically, a negative result image may include a preprocessed fundus image whose first classification result is no retinal lesions and whose classification result type is that no reclassification is required. A positive result image may include a preprocessed fundus image whose first classification result is preproliferative period or proliferative period and whose classification result type is that no reclassification is required. The image to be reclassified may include a preprocessed fundus image whose classification result type is that reclassification is required. In some examples, the preprocessed fundus images that need to be reclassified may include preprocessed fundus images whose first classification result is background period and preprocessed fundus images that cannot be classified.

[0050] In some examples, the first quality control module 250 can obtain the final classification result, the first image to be arbitrated, and the second image to be arbitrated based on the negative result image and the positive result image. Figure 2 As shown, in some examples, the first quality control module 250 may include a negative quality control module 251 and a positive quality control module 252 .

[0051] In some examples, the negative quality control module 251 can receive a negative result image, and obtain a negative quality control result using a first quality control model configured with a first confidence threshold based on a preset negative prediction rate. Generally speaking, the higher the negative prediction rate, the more sensitive the first quality control model is to negative results (i.e., no retinopathy), and uncertain negative result images will be classified as positive results (i.e., the presence of a certain type of diabetic retinopathy). Specifically, before the film reading system 200 is officially released to the formal environment, a preset negative prediction rate can be set according to requirements (e.g., customer requirements or default requirements). Based on the preset negative prediction rate, the first confidence threshold is continuously adjusted and tested to obtain a first confidence threshold corresponding to the preset negative prediction rate. In some examples, the preset negative prediction rate can be 95% to 99%. For example, the preset negative prediction rate can be 95%, 96%, 97%, 98%, or 99%, etc.

[0052] In some examples, the first confidence threshold can be configured using gold standard data. Thus, the first confidence threshold can be determined. In some examples, the first confidence threshold can be reversely solved based on the gold standard data according to the preset negative prediction rate. In some examples, in the reverse solution, the confidence threshold within the preset range (for example, the preset range can be 90% to 100%) can be traversed according to the preset step size and the performance index can be solved to obtain the correspondence between multiple confidence thresholds and multiple groups of performance indicators, and the first confidence threshold is determined based on the correspondence between multiple confidence thresholds and multiple groups of performance indicators and the preset negative prediction rate. In this case, the first confidence threshold can be conveniently and quickly determined using the correspondence between multiple confidence thresholds and multiple groups of performance indicators. In some examples, the performance indicators may include sensitivity, specificity, positive prediction rate, and negative prediction rate.

[0053] Specifically, since the performance indicators such as sensitivity, specificity, positive prediction rate (positive prediction rate can be the number of true positives / (the number of true positives + the number of false positives)) and negative prediction rate (negative prediction rate can be the number of true negatives / (the number of true negatives + the number of false negatives)) corresponding to each confidence threshold are determined on the gold standard data. In some examples, the confidence thresholds within a preset range can be traversed according to a preset step size and the relevant performance indicators can be solved. For example, the preset step size can be 0.01, 0.001 or 0.0001, etc. In this case, a table recording a set of performance indicators corresponding to each confidence threshold can be created based on the correspondence between multiple confidence thresholds and multiple groups of performance indicators, and the confidence threshold corresponding to the performance indicator of the preset negative prediction rate included in the table is the first confidence threshold.

[0054] However, the examples of the present disclosure are not limited thereto. In other examples, the first confidence threshold corresponding to the preset negative prediction rate can be obtained by continuously adjusting the first confidence threshold and testing based on the preset negative prediction rate. For example, the first initial confidence threshold can be set based on the gold standard data and the negative prediction rate based on the first initial confidence threshold can be obtained. If the absolute difference between the negative prediction rate and the preset negative prediction rate is greater than a preset value (e.g., 1%, 2%, or 3%), the first initial confidence threshold is adjusted and the negative prediction rate is continuously compared with the preset negative prediction rate. Otherwise, the first initial confidence threshold is used as the first confidence threshold.

[0055] In some examples, the first quality control model may be the same as the first classification model. In other examples, the first quality control model may be a model retrained for negative result images. In some examples, the negative quality control result may include part of the first classification result. For example, the negative quality control result may include no retinopathy. In some examples, the confidence may be the probability that the negative result image belongs to a certain negative quality control result. In some examples, the first confidence threshold may include a positive release threshold, a negative release threshold, and a result threshold. In some examples, if the first quality control model includes multiple first sub-quality control models for each type of diabetic retinopathy, the first confidence threshold may be multiple groups. For example, if there are n first sub-quality control models, n groups of first confidence thresholds are required. In some examples, the first quality control model may output a negative quality control result according to the result threshold. In some examples, negative quality control results with a first quality control model confidence between the negative release threshold and the positive release threshold may be arbitrated.

[0056] In some examples, if the negative quality control result of the negative result image is consistent with the first classification result, the negative quality control result is used as the final classification result of the negative result image, otherwise the negative result image is used as the first image to be arbitrated. In this case, by setting a preset negative prediction rate, negative result images with lower risks can be distinguished and compared with the first classification result.

[0057] In some examples, the first quality control module 250 may include a positive quality control module 252. In some examples, the positive quality control module 252 may receive a positive result image, and obtain a positive quality control result using a second quality control model configured based on a preset positive prediction rate for a second confidence threshold. Generally speaking, the higher the positive prediction rate, the more sensitive the second quality control model is to a positive result (i.e., the presence of a certain type of diabetic retinopathy), and an uncertain positive result image will be classified as a negative result (i.e., no retinopathy). Specifically, before the film reading system 200 is officially released to the formal environment. The preset positive prediction rate can be set according to requirements (e.g., customer requirements or default requirements). Based on the preset positive prediction rate, the second confidence threshold is continuously adjusted and tested to obtain a second confidence threshold corresponding to the preset positive prediction rate. In some examples, the preset positive prediction rate can be 95% to 99%. For example, the preset positive prediction rate can be 95%, 96%, 97%, 98% or 99%, etc.

[0058] In some examples, the second confidence threshold can be configured using gold standard data. Thus, the second confidence threshold can be determined. In some examples, the second confidence threshold can be reversely solved based on the gold standard data according to the preset positive prediction rate. In some examples, the second confidence threshold can be determined based on the corresponding relationship between the above-mentioned multiple confidence thresholds and multiple groups of performance indicators and the preset positive prediction rate. In this case, the second confidence threshold can be easily and quickly determined using the corresponding relationship between the multiple confidence thresholds and multiple groups of performance indicators. For specific content, see the relevant description of the reverse solution in the first confidence threshold.

[0059] However, the examples of the present disclosure are not limited thereto. In other examples, the second confidence threshold corresponding to the preset positive prediction rate can be obtained by continuously adjusting the second confidence threshold and performing tests based on the preset positive prediction rate.

[0060] In some examples, the second quality control model can be the same as the first classification model. In other examples, the second quality control model can be a model retrained for the positive result image. In some examples, the positive quality control result can include part of the first classification result. For example, the positive quality control result can include the early proliferation period and the proliferation period. In some examples, the confidence can be the probability that the positive result image belongs to a certain positive quality control result. In some examples, the second confidence threshold can include a positive release threshold, a negative release threshold and a result threshold. The specific content can refer to the description related to the first confidence threshold.

[0061] In some examples, if the positive quality control result of the positive result image is consistent with the first classification result, the positive quality control result is used as the final classification result of the positive result image, otherwise the positive result image is used as the second image to be arbitrated. In this case, by setting a preset positive prediction rate, it is possible to distinguish positive result images with higher risks and compare them with the first classification result.

[0062] As described above, the film reading system 200 may include a second classification module 260 (see Figure 2 ). In some examples, the second classification module 260 may obtain a final classification result and a third image to be arbitrated based on the image to be reclassified.

[0063] In some examples, the second classification module 260 can receive the image to be reclassified, and classify the image to be reclassified using the second classification model to obtain a second classification result. The second classification model can be based on deep learning and trained for the image to be reclassified. In some examples, the second classification result may include part of the first classification result. For example, the second classification result may include no retinopathy, background period, pre-proliferation period and proliferative period. In some examples, when training for the image to be reclassified, features related to the image to be reclassified such as the image to be reclassified can be extracted and the second classification model can be trained in combination with the image to be reclassified. In some examples, the relevant features may include microaneurysms, hemorrhage, exudation, cotton wool spots, neovascularization or macular lesions. In some examples, the relevant features may include health conditions, age and medical history records. In some examples, the second classification model can also be trained in combination with the color features, texture features and shape features of the image to be reclassified.

[0064] In some examples, if the second classification result of the image to be reclassified is consistent with the first classification result, the second classification result is used as the final classification result of the image to be reclassified; otherwise, the image to be reclassified is used as the third image to be arbitrated.

[0065] In some examples, the arbitration module 270 may be used to arbitrate the image to be arbitrated to obtain an arbitration classification result. In some examples, the arbitration classification result may be used as the final classification result. In some examples, the image to be arbitrated may be the first image to be arbitrated, the second image to be arbitrated, or the third image to be arbitrated. In some examples, the arbitration classification result may be consistent with the first classification result. In some examples, the arbitration doctor may judge the image to be arbitrated to obtain the arbitration classification result.

[0066] Figure 5 is a block diagram showing an image interpretation system 200 for classifying fundus images according to an example of the present disclosure.

[0067] In some examples, such as Figure 5As shown, the film reading system 200 also includes a self-check module 280. In some examples, the self-check module 280 can be used to perform random checks on fundus images of negative quality control results to determine whether the first confidence threshold meets the requirements. In some examples, the self-check module 280 can be used to perform random checks on fundus images of positive quality control results to determine whether the second confidence threshold meets the requirements. In some examples, sampling methods can be used for random checks. For example, random sampling can be used for random checks. In some examples, the degree of random checks for the newly released film reading system 200 can be tightened (for example, the sampling ratio can be increased). In this case, it can be further confirmed whether the first confidence threshold and the second confidence threshold meet the requirements. As a result, the classification accuracy of the film reading system 200 can be improved.

[0068] In some examples, such as Figure 5 As shown, the film reading system 200 may further include an output module 290. In some examples, the output module may be used to output a result report. In some examples, the output module 290 may judge at least one of the first classification result, the negative quality control result, the positive quality control result, the second classification result, the arbitration classification result, and the final classification result to output a result report of the fundus image. In some examples, the result report may include the confidence level of each result.

[0069] The following, combined Figure 6 The fundus image classification reading method of the present disclosure is described in detail. The fundus image classification reading method involved in the present disclosure may sometimes be referred to as the reading method. The reading method is applied to the above-mentioned reading system 200. Figure 6 is a flowchart showing a method for interpreting fundus images for classification according to an example of the present disclosure.

[0070] In some examples, such as Figure 6As shown, the image reading method may include acquiring a fundus image (step S110), preprocessing the fundus image to acquire a preprocessed fundus image (step S120), classifying the preprocessed fundus image and acquiring a first classification result and a classification result type (step S130), dividing the preprocessed fundus image into a negative result image, a positive result image, and an image to be reclassified (step S140), acquiring a final classification result, a first image to be arbitrated, and a second image to be arbitrated based on the negative result image and the positive result image (step S150), acquiring a final classification result and a third image to be arbitrated based on the image to be reclassified (step S160), and arbitrating the image to be arbitrated to acquire an arbitration classification result as the final classification result (step S170). In this case, the fundus image is divided into a negative result image, a positive result image, and an image to be reclassified, and for the negative result image and the positive result image, a negative result image with a lower risk is acquired based on the negative prediction rate and a positive result image with a higher risk is acquired based on the positive prediction rate and a consistency judgment is performed, and for the image to be reclassified, the second classification model is used for further classification, and finally the image to be arbitrated is arbitrated. This can improve classification accuracy.

[0071] In some examples, in step S110, a fundus image may be acquired. The fundus image may be a color fundus image. The color fundus image can clearly present rich fundus information such as the optic disc, optic cup, macula, and blood vessels. For a specific description, see the relevant description of the acquisition module 210 in the film reading system 200.

[0072] In some examples, in step S120, the fundus image may be preprocessed to obtain a preprocessed fundus image. In some examples, the fundus image may be cropped, normalized, denoised, grayed, etc. For a specific description, see the relevant description of the preprocessing module 220 in the film reading system 200.

[0073] In some examples, in step S130, the pre-processed fundus image can be classified using a first classification model based on deep learning to obtain a first classification result. In some examples, the classification result type can be obtained based on the first classification result. In some examples, the classification result type includes whether the type needs to be classified again. In some examples, the first classification result can be output according to the retinopathy grading system used by the British National Retinopathy Screening Program. In some examples, the first classification result can include at least no retinopathy (R0), background period (R1), pre-proliferation period (R2) and proliferative period (R3). In this case, based on the mature retinopathy grading system, the classification accuracy can be further improved. In some examples, a pre-processed fundus image that cannot be classified (for example, a pre-processed fundus image that cannot be classified due to poor image quality) can be obtained. In some examples, the first classification model can include multiple sub-classification models. Each sub-classification model can be for each type of diabetic retinopathy. Each sub-classification model can receive the pre-processed fundus image and obtain a sub-classification result. In some examples, the first classification module 230 can obtain the first classification result based on multiple sub-classification results. Thus, the first classification result can be obtained based on multiple sub-classification models. For a detailed description, please refer to the relevant description of the first classification module 230 in the film reading system 200.

[0074] In some examples, in step S140, the pre-processed fundus image can be divided into a negative result image, a positive result image, and an image to be reclassified based on the first classification result and the classification result type. In this case, the pre-processed fundus image is divided into a negative result image, a positive result image, and an image to be reclassified, which can facilitate the subsequent targeted processing of each image. Thus, the classification accuracy can be further improved. Specifically, the negative result image can include a pre-processed fundus image whose first classification result is no retinal lesions and the classification result type is not required to be reclassified. The positive result image can include a pre-processed fundus image whose first classification result is a proliferative pre-stage or a proliferative stage and the classification result type is not required to be reclassified. The image to be reclassified can include a pre-processed fundus image whose classification result type is required to be reclassified. In some examples, the pre-processed fundus image that needs to be reclassified can include a pre-processed fundus image whose first classification result is a background stage and a pre-processed fundus image that cannot be classified. For a specific description, please refer to the relevant description of the grouping module 240 in the film reading system 200.

[0075] In some examples, in step S150, the negative quality control result of the negative result image can be obtained using the first quality control model. In some examples, the first quality control model can be configured based on a preset negative prediction rate to the first confidence threshold. In some examples, if the negative quality control result is consistent with the first classification result, the negative quality control result is used as the final classification result, otherwise the negative result image is used as the first image to be arbitrated. In some examples, the positive quality control result of the positive result image can be obtained using the second quality control model. The first quality control model can be configured based on a preset positive prediction rate to the second confidence threshold. In some examples, if the positive quality control result is consistent with the first classification result, the positive quality control result is used as the final classification result, otherwise the positive result image is used as the second image to be arbitrated. In some examples, the first confidence threshold can be configured using gold standard data (i.e., the first confidence threshold is continuously adjusted using gold standard data and the first confidence threshold is finally determined). Thus, the first confidence threshold can be determined. In some examples, the second confidence threshold can be configured using gold standard data (i.e., the second confidence threshold is continuously adjusted using gold standard data and finally determined). Thus, the second confidence threshold can be determined. For a specific description, please refer to the relevant description of the first quality control module 250 in the film reading system 200.

[0076] In some examples, in step S160, the image to be reclassified may be classified using a second classification model based on deep learning to obtain a second classification result. In some examples, the second classification model may be trained for the image to be reclassified. In some examples, if the second classification result is consistent with the first classification result, the second classification result is used as the final classification result, otherwise the image to be reclassified is used as the third image to be arbitrated. For a specific description, please refer to the relevant description of the second classification module 260 in the film reading system 200.

[0077] In some examples, in step S170, the image to be arbitrated may be arbitrated to obtain an arbitration classification result as the final classification result. In some examples, the image to be arbitrated may be the first image to be arbitrated, the second image to be arbitrated, or the third image to be arbitrated. For a specific description, see the relevant description of the arbitration module 270 in the film reading system 200.

[0078] In some examples, the film reading method also includes a self-checking step (not shown). In some examples, in the self-checking step, the fundus images of the negative quality control results can be sampled to determine whether the first confidence threshold meets the requirements. In some examples, the fundus images of the positive quality control results are sampled to determine whether the second confidence threshold meets the requirements. In this case, it can be further confirmed whether the first confidence threshold and the second confidence threshold meet the requirements. Thus, the classification accuracy of the film reading system 200 can be improved. For a specific description, please refer to the relevant description of the self-check module 280 in the film reading system 200.

[0079] In some examples, the film reading method further includes an output step. In some examples, the output step can be used to output a result report. For a specific description, see the relevant description of the output module 290 in the film reading system 200.

[0080] Although the present invention is specifically described above in conjunction with the accompanying drawings and embodiments, it is to be understood that the above description does not limit the present invention in any form. Those skilled in the art can deform and change the present invention as needed without departing from the essential spirit and scope of the present invention, and these deformations and changes all fall within the scope of the present invention.

Claims

1. A quality control module for fundus images, characterized in that: It includes a negative quality control module and a positive quality control module; the negative quality control module receives a negative result image, obtains a negative quality control result of the negative result image by using a first quality control model configured with a first confidence threshold based on a preset negative prediction rate, obtains a final classification result of the negative result image based on the negative result image, or uses the negative result image as a first image to be arbitrated; the positive quality control module receives a positive result image, obtains a positive quality control result of the positive result image by using a second quality control model configured with a second confidence threshold based on a preset positive prediction rate, obtains a final classification result of the positive result image based on the positive result image, or uses the positive result image as a second image to be arbitrated.

2. The quality control module according to claim 1, characterized in that: The negative quality control module is configured as follows: if the negative quality control result is consistent with the first classification result of the negative result image, the negative quality control result is used as the final classification result; otherwise, the negative result image is used as the first image to be arbitrated; the positive quality control module is configured as follows: if the positive quality control result is consistent with the first classification result of the positive result image, the positive quality control result is used as the final classification result; otherwise, the positive result image is used as the second image to be arbitrated.

3. The quality control module according to claim 1, characterized in that: The first confidence threshold is configured using gold standard data based on the preset negative prediction rate; and the second confidence threshold is configured using gold standard data based on the preset positive prediction rate.

4. The quality control module according to claim 1, characterized in that: The preset negative predictive rate is 95% to 99%, and the preset positive predictive rate is 95% to 99%.

5. The quality control module according to claim 1, characterized in that: The first quality control model is a model retrained for the negative result image; and the second quality control model is a model retrained for the positive result image.

6. A fundus image classification reading system, characterized in that: It comprises an acquisition module, a preprocessing module, a first classification module, a grouping module, a quality control module as described in any one of claims 1 to 5, a second classification module and an arbitration module; the acquisition module is used to acquire a fundus image; the preprocessing module is used to preprocess the fundus image to acquire a preprocessed fundus image; The first classification module is used to classify the pre-processed fundus image and obtain a first classification result and a classification result type, wherein the classification result type includes whether a re-classification is required; the grouping module is used to classify the pre-processed fundus image into a negative result image, a positive result image, and an image to be re-classified based on the first classification result and the classification result type; The quality control module is used to receive the negative result image and the positive result image to output the final classification result of the negative result image or use the negative result image as the first image to be arbitrated, and to output the final classification result of the positive result image or use the positive result image as the second image to be arbitrated; The second classification module is used to obtain a final classification result of the image to be reclassified based on the image to be reclassified or use the image to be reclassified as a third image to be arbitrated; The arbitration module is used to receive the first image to be arbitrated, the second image to be arbitrated, or the third image to be arbitrated as the images to be arbitrated, and to arbitrate the images to be arbitrated to obtain an arbitration classification result as the final classification result.

7. The film reading system according to claim 6, characterized in that: The first classification module is used to classify the preprocessed fundus image based on a first classification model of deep learning. The first classification model includes multiple sub-classification models for each type of diabetic retinopathy for receiving the preprocessed fundus image and obtaining sub-classification results. The first classification module obtains the first classification result based on the multiple sub-classification results.

8. The film reading system according to claim 7, characterized in that: The first quality control model of the quality control module is the same as the first classification model; the second quality control model of the quality control module is the same as the first classification model.

9. The film reading system according to claim 6, characterized in that: The second classification module uses a second classification model based on deep learning and trained for the image to be reclassified to classify the image to be reclassified to obtain a second classification result. If the second classification result is consistent with the first classification result, the second classification result is used as the final classification result; otherwise, the image to be reclassified is used as the third image to be arbitrated.

10. The film reading system according to claim 9, characterized in that: The second classification model is trained by combining features related to the image to be reclassified with the image to be reclassified.

Images