System and method for evaluating diabetic retinopathy of fundus image

Abstract

The disclosure describes an evaluation system and an evaluation method for evaluating diabetic retinopathy of fundus images, which comprises an acquisition module for acquiring and preprocessing fundus images; a first judgment module for judging whether the fundus images have diabetic retinopathy and outputting a first judgment result; a quality control grading module for dividing the fundus images into primary quality control images or high-level quality control images based on the confidence of the sub-model result corresponding to the first judgment result of the fundus images; a first sampling module for extracting the primary quality control images as sample fundus images; a second judgment module for judging the primary quality control images and the sample fundus images to obtain a second judgment result; a sampling monitoring module for determining whether the first judgment result of the primary quality control images is qualified; and a third judgment module for obtaining a final judgment result based on each judgment result. Thus, the evaluation accuracy of diabetic retinopathy of fundus images can be improved.

Description

Technical Field

[0001] This disclosure generally relates to an assessment system and method for evaluating diabetic retinopathy in fundus images. Background Technology

[0002] Medical images often contain numerous details of body structures or tissues. In modern hospitals, a large portion of treatment information comes from medical images, such as fundus images. Clinically, understanding these details in medical images helps doctors identify relevant diseases. Medical imaging has become a primary method for clinical disease identification. However, traditional disease identification based on medical images mainly relies on the experience of professional physicians. Therefore, developing a system that assists doctors in identifying relevant diseases has become a hot topic in the field of medical imaging. With the development of artificial intelligence technology, disease identification systems based on computer vision and artificial intelligence, such as machine learning, have been developed and applied in medical image recognition.

[0003] For example, Patent Document 1 (CN105513077A) discloses a system for screening diabetic retinopathy. The system includes: a fundus image acquisition device, an image processing and screening device, and a report output device. The fundus image acquisition device is used to collect or receive fundus images of the examinee; the image processing and screening device is used to process the fundus images and detect whether there are lesions in them, and then transmit the detection results to the report output device; the report output device outputs a corresponding detection report based on the detection results.

[0004] However, in actual clinical applications, due to the diversity of fundus images, the screening system described in Patent Document 1 may output incorrect or inaccurate test reports when evaluating certain fundus images, resulting in a decrease in the evaluation accuracy of the screening system. Summary of the Invention

[0005] This disclosure is made in view of the above-mentioned situation, and its purpose is to provide an assessment system and method for assessing diabetic retinopathy in fundus images that can improve the accuracy of assessment of diabetic retinopathy in fundus images.

[0006] To this end, the first aspect of this disclosure provides an assessment system for evaluating diabetic retinopathy in fundus images, comprising an acquisition module, a first judgment module, a quality control grading module, a first sampling module, a second judgment module, a sampling monitoring module, and a third judgment module; the acquisition module is used to acquire fundus images and preprocess the fundus images; the first judgment module judges whether diabetic retinopathy exists in the fundus images based on a machine learning judgment model and outputs a first judgment result, the first judgment result including sub-model results for each stage of diabetic retinopathy and the confidence level corresponding to the sub-model results; the quality control grading module classifies the fundus images into primary quality control images or advanced quality control images based on the first judgment result of the fundus images, the primary quality control images being fundus images in which no sub-model results in the first judgment result have a confidence level less than a preset confidence level. The advanced quality control image is a fundus image in the first judgment result where the confidence level of the sub-model result corresponding to the sub-model result is less than the preset confidence level. The first sampling module is used to sample the primary quality control image and use the sampled primary quality control image as the sampled fundus image. The second judgment module is used to use the advanced quality control image and / or the sampled fundus image as the quality control image and judge the quality control image to obtain a second judgment result. The sampling monitoring module is used to obtain the second judgment result of the primary quality control image and determine whether the first judgment result of the primary quality control image is qualified. If it is not qualified, the sampling is stopped, the primary quality control image is used as the quality control image and judged to obtain the second judgment result. The third judgment module is used to confirm and judge the first judgment result of the first judgment module and the second judgment result of the second judgment module to obtain the final judgment result. In this case, the first judgment result is obtained by using a machine learning-based judgment model, the fundus image is divided based on the confidence level in the first judgment result, and different judgment modules are used to make targeted judgments on the divided fundus images and to sample the judgment modules. This can improve the accuracy of assessing diabetic retinopathy in fundus images.

[0007] Furthermore, in the evaluation system according to the first aspect of this disclosure, optionally, the first judgment module, the second judgment module, and the third judgment module further judge the image quality of the fundus image to obtain an image quality level. In this case, obtaining the image quality level of the fundus image allows for subsequent evaluation of the fundus image in conjunction with the image quality level. This further improves the accuracy of evaluating diabetic retinopathy using fundus images.

[0008] Furthermore, in the evaluation system disclosed in the first aspect, optionally, the sampling monitoring module acquires the second judgment result of the primary quality control image and calculates the sampling error rate. If the sampling error rate is greater than a preset error rate, the first judgment result of the primary quality control image is determined to be unqualified. Thus, it is possible to determine whether the first judgment result is qualified based on the sampling error rate.

[0009] Additionally, in the assessment system disclosed in the first aspect, the staging of diabetic retinopathy may optionally include no retinopathy, background stage, preproliferative stage, and proliferative stage.

[0010] Additionally, in the evaluation system disclosed in the first aspect, optionally, the judgment model includes multiple sub-judgment models for receiving the fundus images and obtaining sub-judgment results for each stage of diabetic retinopathy. Each sub-judgment result includes a negative result and a confidence level of the negative result, or a positive result and a confidence level of the positive result. The confidence level of the negative result and the confidence level of the positive result are compared among the sub-judgment results to obtain a result with high confidence and this result is used as the sub-model result of the sub-judgment model.

[0011] Furthermore, in the evaluation system according to the first aspect of this disclosure, optionally, the first sampling module samples the primary quality control images based on a sampling plan, wherein the sampling plan includes uniformly sampling the primary quality control images, or classifying the primary quality control images according to one or more different dimensions and sampling separately for each category. Thus, sampling can be performed based on multiple sampling plans.

[0012] Additionally, in the evaluation system according to the first aspect of this disclosure, the evaluation system optionally includes a recall module, which is used to perform recall processing on fundus images corresponding to judgment results that have been determined to be unqualified by the sampling monitoring module. This can further improve the accuracy of fundus image evaluation for diabetic retinopathy.

[0013] Additionally, in the evaluation system disclosed in the first aspect, the preset confidence level is optionally 95% to 98%.

[0014] Additionally, in the assessment system disclosed in the first aspect, the assessment system may optionally include an output module for outputting a result report. The result report includes user information, medical history, the fundus image, the image quality level of the fundus image, a comprehensive assessment result, comprehensive recommendations, reviewer information, and an interpretation based on the final judgment result. Thus, a result report can be output.

[0015] The second aspect of this disclosure provides an assessment method for evaluating diabetic retinopathy in fundus images, comprising: acquiring fundus images and preprocessing the fundus images; judging whether diabetic retinopathy exists in the fundus images based on a machine learning-based judgment model and outputting a first judgment result, the first judgment result including sub-model results for each stage of diabetic retinopathy and the confidence level corresponding to the sub-model results; classifying the fundus images into primary quality control images or advanced quality control images based on the first judgment result, wherein the primary quality control images are fundus images in the first judgment result for which no sub-model result has a confidence level lower than a preset confidence level, and the advanced quality control images are... The first judgment result includes fundus images of sub-model results with confidence levels lower than the preset confidence level. The primary quality control images are sampled, and these sampled primary quality control images are used as sampled fundus images. The advanced quality control images and / or the sampled fundus images are used as quality control images, and the quality control images are judged to obtain a second judgment result. The second judgment result of the primary quality control image is obtained, and it is determined whether the first judgment result of the primary quality control image is qualified. If it is unqualified, the sampling is stopped, and the primary quality control image is used as the quality control image and judged to obtain the second judgment result. The first judgment result and the second judgment result are then confirmed and judged to obtain the final judgment result. In this case, a machine learning-based judgment model is used to obtain the first judgment result, and the fundus images are segmented based on the confidence level in the first judgment result. Targeted judgments are then performed on the segmented fundus images, and sampling is conducted. This improves the accuracy of fundus image assessment for diabetic retinopathy.

[0016] According to this disclosure, an assessment system and method for assessing diabetic retinopathy based on fundus images can be provided to improve the accuracy of assessment of diabetic retinopathy based on fundus images. Attached Figure Description

[0017] This disclosure will now be explained in further detail by way of example only with reference to the accompanying drawings, in which:

[0018] Figure 1 This is an application scenario diagram illustrating the evaluation method for assessing diabetic retinopathy in fundus images, as described in this disclosure.

[0019] Figure 2 This is a block diagram illustrating an evaluation system for assessing diabetic retinopathy in fundus images, as described in this disclosure example.

[0020] Figure 3 This is a schematic diagram illustrating a fundus image relevant to an example of this disclosure.

[0021] Figure 4 This is a schematic diagram illustrating how the quality control grading module involved in this disclosure divides fundus images.

[0022] Figure 5 This is a block diagram illustrating an evaluation system for assessing diabetic retinopathy in fundus images, as described in this disclosure example.

[0023] Figure 6 This is a flowchart illustrating an evaluation method for assessing diabetic retinopathy in fundus images, as illustrated in this disclosure. Detailed Implementation

[0024] The preferred embodiments of this disclosure are described in detail below with reference to the accompanying drawings. In the following description, the same reference numerals are used for the same components, and repeated descriptions are omitted. Furthermore, the drawings are merely schematic diagrams, and the proportions of the components or the shapes of the components may differ from actual figures. It should be noted that the terms "comprising" and "having," and any variations thereof, in this disclosure, do not necessarily limit the process, method, system, product, or apparatus to the explicitly listed steps or units, but may include or have other steps or units not explicitly listed or inherent to these processes, methods, products, or apparatuses. All methods described in this disclosure may be performed in any suitable order unless otherwise indicated herein or clearly contradicted by the context.

[0025] Figure 1 This is an application scenario diagram illustrating the evaluation method for assessing diabetic retinopathy in fundus images, as described in this disclosure.

[0026] In some examples, the evaluation method (described later) can be applied to, for example... Figure 1 In application scenario 100, operator 110 can control acquisition device 130 connected to terminal 120 to acquire fundus images of the fundus of human eye 140. After acquisition device 130 completes fundus image acquisition, terminal 120 can submit the fundus images to server 150 via computer network. Server 150 can execute computer program instructions stored on server 150 to implement an evaluation method. This evaluation method can receive fundus images, evaluate the presence of diabetic retinopathy in the fundus images, and generate a fundus image result report. Server 150 can return the generated fundus image result report to terminal 120. In some examples, terminal 120 can display the result report. In other examples, the result report can be stored as an intermediate result in the memory of terminal 120 or server 150.

[0027] In some examples, the results report can be displayed on the user's device for viewing, such as a patient. In other examples, the fundus images received by the evaluation method can be fundus images stored in terminal 120 or server 150.

[0028] In some examples, operator 110 may be a professional, such as an ophthalmologist. In other examples, operator 110 may be a trained general staff member. The training may include, but is not limited to, the operation of the acquisition device 130 and the operation of the terminal 120 related to the assessment methods. In some examples, terminal 120 may include, but is not limited to, a laptop, tablet, or desktop computer. In some examples, acquisition device 130 may be a camera. The camera may be, for example, a handheld fundus camera or a desktop fundus camera. In some examples, acquisition device 130 may be connected to terminal 120 via a serial port. In some examples, acquisition device 130 may be integrated into terminal 120.

[0029] In some examples, server 150 may include one or more processors and one or more memories. The processor may include a central processing unit, a graphics processing unit, and any other electronic components capable of processing data and executing computer program instructions. The memory may be used to store computer program instructions. In some examples, server 150 may implement an evaluation method by executing computer program instructions stored in memory. In some examples, server 150 may also be a cloud server.

[0030] The evaluation system 200 disclosed herein is described in detail below with reference to the accompanying drawings. The evaluation system 200 disclosed herein is used to implement the evaluation methods described above. Figure 2 This is a block diagram illustrating an evaluation system 200 for assessing diabetic retinopathy in fundus images, as described in this disclosure example.

[0031] like Figure 2As shown, the evaluation system 200 may include an acquisition module 210, a first judgment module 220, a quality control grading module 230, a first sampling module 240, a second judgment module 250, a sampling monitoring module 260, and a third judgment module 270. The acquisition module 210 can acquire fundus images. The first judgment module 220 can judge whether diabetic retinopathy exists in the fundus images based on a deep learning judgment model and output a first judgment result. The quality control grading module 230 can classify the fundus images into primary quality control images or advanced quality control images based on the first judgment result. The first sampling module 240 can sample the primary quality control images and use the extracted primary quality control images as sampled fundus images. The second judgment module 250 can use advanced quality control images and / or sampled fundus images as quality control images and judge the quality control images to obtain a second judgment result. The sampling monitoring module 260 can monitor whether the first judgment result of the primary quality control images is qualified. The third judgment module 270 can be used to confirm and judge the judgment results of each judgment module to obtain a final judgment result. In this case, a first judgment result is obtained using a deep learning-based judgment model, the fundus image is segmented based on the confidence level in the first judgment result, and different judgment modules are used to perform targeted judgments on the segmented fundus images and to sample the judgment modules. This improves the accuracy of assessing diabetic retinopathy in fundus images.

[0032] Figure 3 This is a schematic diagram illustrating a fundus image relevant to an example of this disclosure.

[0033] In some examples, the acquisition module 210 can be used to acquire fundus images. In some examples, the fundus images may include fundus images from the user's left and right eyes. In some examples, the fundus images may be color fundus images. Color fundus images can clearly present rich fundus information such as the optic disc, optic cup, macula, and blood vessels. Additionally, the fundus images may be images in RGB mode, CMYK mode, Lab mode, or grayscale mode. In some examples, the fundus images may be acquired by the acquisition device 130. In other examples, the fundus images may be fundus images stored in the terminal 120 or server 150. Examples of fundus images include... Figure 3 Image of the fundus at 140° of the human eye.

[0034] In some examples, after acquiring the fundus image, the acquisition module 210 can preprocess the fundus image. In some instances, preprocessing may include cropping the fundus image. Generally, because the fundus images acquired by the acquisition module 210 may have different image formats or sizes, it is necessary to crop the fundus images to convert them into a fixed standard format. A fixed standard format can refer to images with the same format and consistent size. For example, in some examples, the preprocessed fundus images may be uniformly sized to 256×256, 374×374, 512×512, 768×768, or 1024×1024 pixels.

[0035] In some examples, preprocessing may include normalizing the fundus image. Normalization may include operations such as coordinate centering and scaling normalization of the fundus image. This overcomes the differences between various fundus images and improves the performance of the evaluation system 200. In other examples, preprocessing may include denoising and grayscale conversion of the fundus image. This highlights the features of the fundus image. In some examples, the fundus image may not require preprocessing, and subsequent judgments may be made directly from the fundus image.

[0036] In some examples, the first judgment module 220 can judge whether there is diabetic retinopathy in the fundus image from the acquisition module 210 and output the first judgment result.

[0037] In some examples, the first judgment module 220 can output a first judgment result based on the retinal disease grading system used by the UK National Retinopathy Screening Program. Specifically, the fundus image can be processed to determine the stage of diabetic retinopathy and its corresponding confidence level. The stages of diabetic retinopathy can include no retinopathy (R0), background stage (R1), preproliferative stage (R2), and proliferative stage (R3). A stage of no retinopathy (R0) can be set as a negative result, and stages of background stage (R1), preproliferative stage (R2), and proliferative stage (R3) can be set as positive results. In this case, based on the well-established retinal disease grading system, the accuracy of assessing diabetic retinopathy in fundus images can be further improved. In some examples, the stages of diabetic retinopathy can include no diabetic macular edema (M0) and macular edema (M1). A stage of diabetic retinopathy without retinopathy (R0) and without diabetic macular edema (M0) can be set as a negative result, and others as positive results.

[0038] In some examples, the first judgment module 220 can use a machine learning-based judgment model to determine whether diabetic retinopathy exists in a fundus image. In some examples, the machine learning-based judgment model can be at least one of a traditional machine learning-based judgment model and a deep learning-based judgment model. Therefore, it is possible to establish judgment models based on multiple machine learning methods.

[0039] In some examples, the judgment model based on deep learning can be a convolutional neural network (CNN). In this case, due to the high efficiency of CNN in image feature recognition, it can effectively improve the performance of the evaluation system 200.

[0040] However, the examples disclosed herein are not limited to this. In other examples, the machine learning-based judgment model of the first judgment module 220 can be a traditional machine learning-based judgment model. In some examples, traditional machine learning algorithms may include, but are not limited to, linear regression, logistic regression, decision tree, support vector machine, or Bayesian algorithms. In this case, image processing algorithms can be used to extract fundus features from the fundus image, and then the fundus features can be input into the traditional machine learning-based judgment model to make judgments on the fundus image.

[0041] Figure 4 This is a schematic diagram illustrating how the quality control grading module 230, as described in this disclosure, divides fundus images.

[0042] In some examples, the diagnostic model may include multiple sub-diagnostic models for each stage of diabetic retinopathy. For example... Figure 4 As shown, the number of sub-judgment models can be n. Specifically, the sub-judgment models can include sub-judgment model 1, sub-judgment model 2, ..., sub-judgment model n. In some examples, each sub-judgment model can be used to receive fundus images and obtain sub-judgment results. In some examples, each sub-judgment result can include a negative result and its confidence level, or a positive result and its confidence level. The confidence level of a negative result can be the probability that it does not belong to a single stage of diabetic retinopathy, and the confidence level of a positive result can be the probability that it belongs to a single stage of diabetic retinopathy. Specifically, fundus images can be input into each sub-judgment model for judgment. Each sub-judgment model can output the probability that the fundus image belongs to the diabetic retinopathy stage corresponding to that sub-judgment model (i.e., a positive result) or the probability that it does not belong to the diabetic retinopathy stage corresponding to that sub-judgment model (i.e., a negative result).

[0043] In some examples, the confidence levels of negative and positive results in the sub-judgment results of a fundus image can be compared to obtain the result with the higher confidence level, and this result can be used as the sub-model result of the sub-judgment model. For example, if the confidence level of a negative result in the sub-judgment results of a fundus image is 10% and the confidence level of a positive result is 90%, then the sub-model result of the sub-judgment model corresponding to that fundus image can be a positive result. In some examples, if the sub-model result is negative, the fundus image may not belong to the stage of diabetic retinopathy corresponding to the sub-judgment model; if the sub-model result is positive, the fundus image may belong to the stage of diabetic retinopathy corresponding to the sub-judgment model. In this case, multiple sub-judgment models can obtain multiple sub-model results and the corresponding confidence levels for each sub-model result. Figure 4 As shown, the number of sub-model results can be n. Specifically, the multiple sub-model results and the confidence levels corresponding to each sub-model result can include sub-model result 1 and the confidence level corresponding to sub-model result 1, sub-model result 2 and the confidence level corresponding to sub-model result 2, ..., sub-model result n and the confidence level corresponding to sub-model result n.

[0044] In some examples, the first judgment may include sub-model results (i.e., sub-model results for each stage of diabetic retinopathy) and the confidence level corresponding to the sub-model results (see [reference]). Figure 4 In some examples, the number of sub-model results in the first judgment result can be multiple. In some examples, the stage of diabetic retinopathy to which the fundus image belongs can be determined based on multiple sub-model results and their corresponding confidence levels. For example, the sub-model result with the highest confidence level among multiple sub-judgment models that has a positive result can be obtained and used as the stage of diabetic retinopathy to which the fundus image belongs. In some examples, the first judgment result may include the stage of diabetic retinopathy to which the fundus image belongs.

[0045] In some examples, the first judgment module 220 can also be used to judge the image quality of the fundus image to obtain the image quality level of the fundus image. For example, a machine learning-based image quality grading model can be trained using fundus images of different image quality levels to enable the model to judge the image quality of the fundus image and obtain its image quality level. In this case, obtaining the image quality level of the fundus image allows for subsequent evaluation of the fundus image based on that level. This further improves the accuracy of the assessment of diabetic retinopathy in fundus images. In some examples, the image quality level of the fundus image includes five levels: very good, good, fair, poor, and very poor. This allows for the determination of the image quality level. In some examples, the first judgment result can include the image quality level of the fundus image. In some examples, the first judgment result can also include the presence of lesion features.

[0046] In some examples, the quality control grading module 230 can classify the fundus image into a primary quality control image or a high-level quality control image based on the first judgment result of the fundus image. In some examples, the primary quality control image can be a fundus image in which the first judgment result does not contain a sub-model result with a confidence level lower than a preset confidence level. In some examples, the high-level quality control image can be a fundus image in which the first judgment result contains a sub-model result with a confidence level lower than a preset confidence level. Specifically, such as... Figure 4 As shown, assuming a fundus image outputs n sub-model results from n sub-judgment models, along with the corresponding confidence levels for each sub-model result, the quality control grading module 230 is configured to classify the fundus image as a high-level fundus image if any sub-model result has a confidence level lower than a preset confidence level; otherwise, it is classified as a low-level fundus image. In some examples, the preset confidence level can be set according to the actual situation. In some examples, the preset confidence level can be between 95% and 98%. For example, the preset confidence level can be 95%, 96%, 97%, or 98%, etc.

[0047] However, the examples disclosed herein are not limited to this. In other examples, the first judgment result may also include the sub-model result of multiple sub-judgment models where the sub-model result is positive and the confidence level of the positive result is the highest. In this case, the primary quality control image may be a fundus image with a confidence level greater than a preset confidence level corresponding to a positive result, and the advanced quality control image may be a fundus image with a confidence level less than or equal to the preset confidence level corresponding to a positive result. In other examples, the judgment model may not include multiple sub-judgment models, that is, the diabetic retinopathy stage to which the fundus image belongs and the corresponding confidence level of the diabetic retinopathy stage can be directly obtained through the judgment model. In this case, the first judgment result may include the diabetic retinopathy stage to which the fundus image belongs and the corresponding confidence level of the diabetic retinopathy stage, the primary quality control image may be a fundus image with a confidence level greater than a preset confidence level corresponding to the diabetic retinopathy stage, and the advanced quality control image may be a fundus image with a confidence level less than or equal to the preset confidence level corresponding to the diabetic retinopathy stage.

[0048] In some examples, the first sampling module 240 can sample primary quality control images and use the extracted primary quality control images as sampled fundus images. Specifically, the first sampling module 240 can sample primary quality control images according to a preset time (e.g., every hour) or a preset number of images (e.g., 100 images). In some examples, the extracted primary quality control images can be marked to distinguish them from the sampled fundus images. In some examples, primary quality control images that are not sampled can be input into the third judgment module 270 (described later).

[0049] In some examples, primary fundus images can be sampled based on a sampling plan. In some examples, the sampling plan may include uniform sampling of primary fundus images. Specifically, a sampling method can be performed on all primary fundus images. In some examples, the sampling method may include, but is not limited to, random sampling, stratified sampling, population sampling, or systematic sampling. In some examples, the sampling method may be based on national standards for sampling inspection. In some examples, national standards for sampling inspection may include, but are not limited to, GB / T2828.1-2012, GB / T2828.5-2011, or GB / T8052-2002.

[0050] Additionally, in some examples, the sampling scheme may include classifying primary fundus images according to one or more different dimensions and sampling separately for each category (e.g., several dimensions can be considered as one category). In this case, primary fundus images can be classified based on different dimensions and sampled separately for each category. This improves the reliability of the sampling. In some examples, the dimensions may include at least one of the following: the source of the fundus image, the model of the camera used to capture the fundus image, information about the person who captured the fundus image, the image quality of the fundus image, and information about the person who judged the fundus image. In some examples, the machine learning-based judgment model used to determine whether a fundus image contains diabetic retinopathy can be considered as the judge; if there are multiple judgment models, then there can be multiple judges.

[0051] In some examples, the sampling plan can be adjusted based on dimensions or the duration of the evaluation system's 200 release. In some examples, the sampling plan can have different sampling degrees. In some examples, the sampling degree can be general, specific, stringent, or relaxed. In some examples, different sampling degrees can correspond to different sampling methods or different sampling parameters. In some examples, sampling parameters can include, but are not limited to, sampling proportions or AQL (Acceptable Quality Limit) values.

[0052] In some examples, an initial sampling proportion can be obtained and adjusted based on the degree of sampling to determine the final sampling proportion. For example, if the degree of sampling is relaxed, the initial sampling proportion can be reduced to determine the final sampling proportion.

[0053] Generally, AQL refers to the worst-case process average quality level that can be tolerated when a continuous series of batches are submitted for acceptance sampling. If the quality level is expressed as a percentage of nonconformities, the AQL value should not exceed 10%; if the quality level is expressed as the number of nonconformities per 100 units of fundus images to be sampled, the AQL value should not exceed 10. Different levels of sampling can correspond to different AQL values.

[0054] In some examples, the sampling scheme can be adjusted based on the release time of the evaluation system 200. For instance, the sampling level for a newly released evaluation system 200 can be tightened, and after a period of operation, once the accuracy of the evaluation system 200 is determined to meet the requirements, the sampling level can be relaxed. In some examples, the sampling scheme can be adjusted based on the dimension. For instance, the sampling level for primary fundus images judged by less experienced assessors can be tightened.

[0055] In some examples, the second judgment module 250 can be used to judge the quality control image to obtain a second judgment result. The second judgment result may include the stage of diabetic retinopathy to which it belongs. In some examples, the quality control image may be a high-level quality control image from the first judgment module 220 and / or a sampled fundus image from the first sampling module 240. In some examples, the second judgment module 250 may also judge the image quality of the fundus image to obtain the image quality level of the fundus image. In some examples, the second judgment result may also include the image quality level of the fundus image. In some examples, the second judgment result may also include the presence of lesion features.

[0056] Specifically, the second judgment module 250 can acquire and display basic information of the quality control image. This basic information may include, but is not limited to, eye identification (e.g., left or right eye), creation time, affiliated medical institution, camera model, and the current judgment module (e.g., the second judgment module 250). After the judge assesses the quality control image based on the basic information and generates judgment information, the second judgment module 250 can receive the judge's judgment information and generate a second judgment result for the quality control image based on that information. In some examples, the basic information may also include medical history, visual acuity information, gender, and age.

[0057] In some examples, the information used for judgment may include, but is not limited to, the image quality level of the fundus image, the stage of diabetic retinopathy, or the characteristics of the existing lesions.

[0058] As described above, the second judgment module 250 can be used to judge the quality control image to obtain a second judgment result. The quality control image can be a high-level quality control image from the first judgment module 220 and / or a sampled fundus image from the first sampling module 240. Therefore, the sampled fundus image has a second judgment result (that is, the extracted primary quality control image has a second judgment result).

[0059] In some examples, the sampling monitoring module 260 can acquire the second judgment result of the primary quality control image and determine whether the first judgment result of the primary quality control image is qualified. In some examples, the sampling monitoring module 260 can acquire the second judgment result of primary quality control images within a preset time period (e.g., the current day, morning, or afternoon) and determine whether the first judgment result of the corresponding primary quality control images is qualified. In other examples, the sampling monitoring module 260 can acquire the second judgment result of a preset number of primary quality control images and determine whether the first judgment result of the corresponding primary quality control images is qualified. In still other examples, the sampling monitoring module 260 can acquire the second judgment result of primary quality control images within a preset time period and determine whether the first judgment result of the primary quality control images in at least one dimension is qualified according to at least one dimension (e.g., the person making the judgment).

[0060] In some examples, if the quality is not up to standard, the first sampling module 240 is stopped and the corresponding primary quality control image (e.g., primary quality control image for a preset time period, primary quality control image for at least one dimension, primary quality control image for a preset time period and at least one dimension, or all primary quality control images) is input into the second judgment module 250 for judgment. That is, the sampling of the corresponding primary quality control image is stopped, and the corresponding primary quality control image is used as the quality control image and judged to obtain the second judgment result.

[0061] As described above, in some other examples, the sampling monitoring module 260 can obtain a second judgment result for a preset number of primary quality control images. Specifically, the sampling monitoring module 260 can monitor the primary quality control images in the evaluation system 200, and when the number of primary quality control images in the evaluation system 200 reaches a preset number, such as 500, it determines the first judgment result of the preset number of primary quality control images based on the second judgment result of the preset number of primary quality control images.

[0062] In some examples, the sampling monitoring module 260 can acquire the second judgment result of the primary quality control image and calculate the sampling error rate. If the sampling error rate is greater than a preset error rate, the first judgment result of the corresponding primary quality control image (i.e., the primary quality control image used to calculate the sampling error rate) is deemed unqualified. Thus, it is possible to determine whether the first judgment result is qualified based on the sampling error rate. In some examples, the preset error rate can be set according to actual conditions. In some examples, the sampling monitoring module 260 can acquire the second judgment result of the primary quality control image within a preset time period and calculate the sampling error rate. In some examples, the sampling monitoring module 260 can acquire the second judgment result of the primary quality control image within a preset time period and calculate the sampling error rate according to at least one dimension.

[0063] In some examples, if the first judgment result of the primary quality control image is inconsistent with the second judgment result, then the first judgment result of the primary quality control image is an incorrect judgment result. In some examples, the consistency between the first and second judgment results can be determined based on the image quality level of the fundus image or the stage of diabetic retinopathy to which it belongs.

[0064] As described above, the sampling monitoring module 260 can acquire the second judgment result of the primary quality control images and calculate the sampling error rate. In some examples, the sampling error rate can be the number of sampling errors divided by the total number of samples. The sampling monitoring module 260 can acquire the number of primary quality control images used to calculate the sampling error rate, the total number of samples, and the number of sampling errors. The total number of samples can be the number of primary fundus images that have been sampled by the first sampling module 240 in the primary quality control images used to calculate the sampling error rate. The number of sampling errors can be the number of primary quality control images corresponding to erroneous judgment results in the primary quality control images used to calculate the sampling error rate.

[0065] In some examples, the sampling plan can be adjusted based on the sampling error rate. For instance, if the sampling error rate for a certain dimension is low, the sampling level of the sampling plan can be relaxed.

[0066] In some examples, the third judgment module 270 can be used to confirm and judge the first judgment result of the first judgment module 220 and the second judgment result of the second judgment module 250 to obtain a final judgment result. In some examples, the third judgment module 270 can also confirm and judge the first judgment result of the primary quality control image that has not been extracted to obtain a final judgment result.

[0067] In some examples, the third judgment module 270 can check the judgment results of each module and confirm whether to retain the judgment result of the fundus image in the last judgment module (e.g., the first judgment module 220 or the second judgment module 250) as the final judgment result. If not, a re-judgment is performed to obtain the final judgment result. This allows for unified confirmation of the judgment results of each judgment module. In some examples, the final judgment result may include the stage of diabetic retinopathy. In some examples, the third judgment module 270 can also judge the image quality of the fundus image to obtain the image quality level of the fundus image. In some examples, the final judgment result may also include the image quality level of the fundus image. In some examples, the final judgment result may also include the existing lesion characteristics.

[0068] Figure 5 This is a block diagram illustrating an evaluation system 200 for assessing diabetic retinopathy in fundus images, as described in this disclosure example.

[0069] In some examples, the evaluation system 200 may also include an output module 280 (see [link to example]). Figure 5 The output module 280 can be used to output a result report. In some examples, the result report may include the final judgment result. In some examples, the result report may include user information, medical history, fundus images, image quality levels of the fundus images, comprehensive assessment results, comprehensive recommendations, reviewer information, and interpretation based on the final judgment result. Thus, a result report can be output.

[0070] In some examples, user information may include, but is not limited to, name, gender, and age. Additionally, in some examples, medical history may include, but is not limited to, a history of diabetes, hypertension, and eye disease. Furthermore, in some examples, the fundus images in the results report may include fundus images of the left and right eyes. Additionally, in some examples, the comprehensive assessment results may include assessments of the left and right eyes. As an example of a comprehensive assessment result, it could be that no obvious diabetic retinopathy was observed in the left fundus, while proliferative diabetic retinopathy was observed in the right fundus. Furthermore, in some examples, the reviewer information in the results report may be the information of the person who confirmed and judged each result using the third judgment module 270. In some examples, the interpretation based on the final judgment result may include at least one of medical explanations, common causes, and guidance recommendations.

[0071] In some examples, the evaluation system 200 also includes a recall module 290 (see...). Figure 5 The recall module 290 can be used to perform recall processing on fundus images (e.g., fundus images of at least one dimension or fundus images within a preset time period) corresponding to judgment results (e.g., the first judgment result mentioned above) that were determined to be unqualified by the sampling monitoring module 260. In some examples, the recalled fundus images can be fully inspected, the sampling ratio of at least one dimension corresponding to the recalled fundus images can be increased, and at least one activity in the hardware and software of the inspection and evaluation system 200 can be performed. This can further improve the accuracy of fundus image assessment of diabetic retinopathy.

[0072] The following, combined with Figure 6 This disclosure describes in detail the assessment method for diabetic retinopathy in fundus images. The assessment method for diabetic retinopathy in fundus images disclosed herein may sometimes be simply referred to as the assessment method. The assessment method is applied in the aforementioned assessment system 200. Figure 6 This is a flowchart illustrating an evaluation method for assessing diabetic retinopathy in fundus images, as illustrated in this disclosure.

[0073] In some examples, such as Figure 6As shown, the evaluation method may include acquiring fundus images (step S110), using a deep learning-based judgment model to determine whether diabetic retinopathy exists in the fundus images and outputting a first judgment result (step S120), dividing the fundus images into primary quality control images or advanced quality control images based on the first judgment result (step S130), sampling the primary quality control images and using the extracted primary quality control images as sampled fundus images (step S140), using the advanced quality control images and / or sampled fundus images as quality control images and judging the quality control images to obtain a second judgment result (step S150), monitoring whether the first judgment result of the primary quality control images is qualified (step S160), and confirming and judging each judgment result to obtain a final judgment result (step S170). In this case, the first judgment result is obtained using a deep learning-based judgment model, the fundus images are divided based on the confidence level in the first judgment result, and targeted judgments and sampling are performed on the divided fundus images. Therefore, the accuracy of the evaluation of diabetic retinopathy in fundus images can be improved.

[0074] In some examples, a fundus image may be acquired in step S110. See the relevant description of the acquisition module 210 for details.

[0075] In some examples, in step S120, a deep learning-based judgment model can be used to determine whether diabetic retinopathy exists in the fundus image and output a first judgment result. In some examples, the judgment model may include multiple sub-judgment models for receiving fundus images and obtaining sub-judgment results for each stage of diabetic retinopathy. In some examples, each sub-judgment result may include a negative result and its confidence level, or a positive result and its confidence level. In some examples, the confidence levels of negative and positive results in the sub-judgment results can be compared to obtain a result with higher confidence, which is then used as the sub-model result of the sub-judgment model. In some examples, if the sub-model result is negative, the fundus image may indicate the absence of the diabetic retinopathy stage corresponding to the sub-judgment model. In some examples, if the sub-model result is positive, the fundus image may indicate the presence of the diabetic retinopathy stage corresponding to the sub-judgment model. In some examples, the first judgment result may include the sub-model result and its corresponding confidence level. In some examples, in step S120, the image quality of the fundus image can also be judged to obtain the image quality level of the fundus image. In this context, the image quality level of the fundus image is obtained, and the fundus image can then be evaluated in conjunction with this image quality level. This further improves the accuracy of fundus image assessment for diabetic retinopathy. In some examples, the image quality level of the fundus image can be categorized into five levels: very good, good, fair, poor, and very poor. This allows for the determination of the image quality level. See the relevant description of the first judgment module 220 for a detailed description. Additionally, in some examples, the staging of diabetic retinopathy may include no retinopathy, background stage, preproliferative stage, and proliferative stage.

[0076] In some examples, in step S130, the fundus image can be classified into a primary quality control image or a high-level quality control image based on the first judgment result. In some examples, the fundus image can be classified into a primary quality control image or a high-level quality control image based on the first judgment result. In some examples, the primary quality control image can be a fundus image in which the first judgment result does not contain a sub-model result with a confidence level lower than a preset confidence level. In some examples, the high-level quality control image can be a fundus image in which the first judgment result contains a sub-model result with a confidence level lower than a preset confidence level. For a detailed description, please refer to the relevant description of the quality control grading module 230.

[0077] In some examples, in step S140, primary quality control images can be sampled, and the sampled primary quality control images can be used as sampled fundus images. In some examples, the primary quality control images that are not sampled can be processed by step S170 (described later). In some examples, during sampling, the sampling scheme can be adjusted according to dimensions or the release duration of the evaluation system 200. This improves the reliability of sampling. In some examples, dimensions may include at least one of the following: the source of the fundus image, the model of the fundus image capturing device, the information of the person capturing the fundus image, the image quality of the fundus image, and the information of the person judging the fundus image. In this case, fundus images can be classified based on different dimensions, and sampling can be performed separately for each category. This improves the reliability of sampling. For a detailed description, please refer to the relevant description of the first sampling module 240.

[0078] In some examples, in step S150, a high-level quality control image and / or a sampled fundus image can be used as a quality control image, and the quality control image is judged to obtain a second judgment result. See the relevant description of the second judgment module 250 for details.

[0079] In some examples, in step S160, the first judgment result of the primary quality control image can be monitored to determine whether it is qualified. In some examples, the second judgment result of the primary quality control image within a preset time period can be obtained, and the first judgment result of the primary quality control image in at least one dimension can be determined according to at least one dimension. If it is not qualified, the sampling in step S140 is stopped, and the primary quality control image is transferred to step S160 for judgment. That is, the sampling of the primary quality control image is stopped, and the primary quality control image is used as the quality control image for judgment to obtain the second judgment result. In some examples, the determination of whether the first judgment result of the primary quality control image in at least one dimension is qualified can be based on the sampling error rate. Specifically, the sampling monitoring module 260 can obtain the second judgment result of the primary quality control image within a preset time period and calculate the sampling error rate according to at least one dimension. If the sampling error rate is greater than the preset error rate, the first judgment result of the primary quality control image in at least one dimension is determined to be unqualified. Thus, the first judgment result can be determined based on the sampling error rate. For a detailed description, please refer to the relevant description of the sampling monitoring module 260.

[0080] In some examples, step S170 may involve confirming and judging each judgment result to obtain a final judgment result. In some examples, the first judgment result and the second judgment result may be confirmed and judged to obtain a final judgment result. In some examples, step S170 may also involve obtaining the image quality level of the fundus image. For a detailed description, please refer to the relevant description of the third judgment module 270.

[0081] In some examples, the assessment method may also output a results report. In some examples, the results report may include the final judgment. In some examples, the results report may include user information, medical history, fundus images, image quality levels of the fundus images, comprehensive assessment results, comprehensive recommendations, reviewer information, and interpretation based on the final judgment. This enables the output of a results report. See the relevant description of output module 280 for a detailed description.

[0082] In some examples, the evaluation method can also perform recall processing on fundus images corresponding to at least one dimension of the unqualified judgment results (such as the first judgment result mentioned above). In some examples, at least one activity in the hardware and software of the evaluation system 200 can be performed on the recalled fundus images, the sampling ratio of at least one dimension corresponding to the recalled fundus images can be increased, and the evaluation system 200 can be improved. This can further improve the accuracy of fundus image evaluation for diabetic retinopathy.

[0083] While the present disclosure has been specifically described above in conjunction with the accompanying drawings and examples, it is to be understood that the foregoing description does not limit the present disclosure in any way. Those skilled in the art can make modifications and variations to the present disclosure as needed without departing from its essential spirit and scope, and all such modifications and variations shall fall within the scope of the present disclosure.

Claims

1. An evaluation system for evaluating diabetic retinopathy of a fundus image, characterized by, The system comprises an acquisition module, a first judgment module, a quality control grading module, a first sampling module, a second judgment module, a sampling monitoring module, and a third judgment module. The acquisition module is configured to acquire fundus images and pre-process the fundus images. The first judgment module is configured to judge whether the fundus images have diabetic retinopathy based on a machine learning judgment model and output a first judgment result, which comprises a sub-model result for each stage of diabetic retinopathy and a confidence level corresponding to the sub-model result. The judgment model comprises a plurality of sub-judgment models for receiving the fundus images and obtaining sub-judgment results for each stage of diabetic retinopathy. Each sub-judgment result comprises a negative result and a confidence level of the negative result, or a positive result and a confidence level of the positive result. The confidence levels of the negative result and the positive result in the sub-judgment result are compared to obtain a result with a high confidence level as the sub-model result of the sub-judgment model. The quality control grading module is configured to divide the fundus images into primary quality control images or advanced quality control images based on the first judgment result of the fundus images. The primary quality control images are fundus images without sub-model results corresponding to a confidence level less than a pre-set confidence level in the first judgment result. The advanced quality control images are fundus images with sub-model results corresponding to a confidence level less than the pre-set confidence level in the first judgment result. The first sampling module is configured to sample the primary quality control images and extract the primary quality control images as sampling fundus images. The second judgment module is configured to take the advanced quality control images and / or the sampling fundus images as quality control images and judge the quality control images to obtain a second judgment result. The sampling monitoring module is configured to obtain the second judgment result of the primary quality control images and determine whether the first judgment result of the primary quality control images is qualified. If not, the sampling is stopped, the primary quality control images are taken as the quality control images and judged to obtain the second judgment result. The third judgment module is configured to confirm and judge the first judgment result of the first judgment module and the second judgment result of the second judgment module to obtain a final judgment result.

2. The evaluation system according to claim 1, wherein: The first judgment module, the second judgment module, and the third judgment module are further configured to judge the image quality of the fundus images to obtain an image quality level.

3. The evaluation system according to claim 1, wherein: The sampling monitoring module obtains the second judgment result of the primary quality control images and calculates a sampling error rate. If the sampling error rate is greater than a pre-set error rate, it is determined that the first judgment result of the primary quality control images is unqualified.

4. The evaluation system according to claim 1, wherein: The stages of diabetic retinopathy include no retinopathy, background stage, pre-proliferative stage, and proliferative stage.

5. The evaluation system according to claim 1, wherein: The first sampling module samples the primary quality control image based on a sampling scheme, the sampling scheme including uniformly sampling the primary quality control image, or classifying the primary quality control image according to one or more different dimensions and separately sampling each category.

6. The evaluation system of claim 1, wherein: The evaluation system further comprises a recall module configured to perform recall processing on the fundus image corresponding to the judgment result determined as unqualified by the sampling monitoring module.

7. The evaluation system of claim 1, wherein: The pre-set confidence is 95% to 98%.

8. The evaluation system of any one of claims 1 to 7, wherein: The evaluation system further comprises an output module configured to output a result report, the result report including user information, medical history, the fundus image, image quality level of the fundus image, comprehensive evaluation result, comprehensive suggestion, reviewer information, and interpretation based on the final judgment result.

9. An evaluation method of diabetic retinopathy of a fundus image, characterized by, including: acquiring a fundus image and pre-processing the fundus image; judging whether the fundus image has diabetic retinopathy based on a machine learning judgment model and outputting a first judgment result, the first judgment result including sub-model results for each stage of diabetic retinopathy and confidence corresponding to the sub-model results, wherein the judgment model includes multiple sub-judgment models for receiving the fundus image and obtaining sub-judgment results for each stage of diabetic retinopathy, each sub-judgment result including a negative result and confidence of the negative result, or a positive result and confidence of the positive result, and by comparing the confidence of the negative result and the confidence of the positive result in the sub-judgment result to obtain a result with high confidence as the sub-model result of the sub-judgment model; dividing the fundus image into a primary quality control image or a high-level quality control image based on the first judgment result of the fundus image, the primary quality control image being the fundus image without sub-model results corresponding to confidence less than a pre-set confidence in the first judgment result, and the high-level quality control image being the fundus image with sub-model results corresponding to confidence less than the pre-set confidence in the first judgment result; sampling the primary quality control image and taking the sampled primary quality control image as a sampling fundus image; taking the high-level quality control image and / or the sampling fundus image as a quality control image and judging the quality control image to obtain a second judgment result; obtaining the second judgment result of the primary quality control image and determining whether the first judgment result of the primary quality control image is qualified, if not, stopping the sampling, taking the primary quality control image as the quality control image and judging to obtain the second judgment result; and confirming and judging the first judgment result and the second judgment result to obtain a final judgment result.

Images