Method and system for reading medical images and server

Abstract

This disclosure describes a method, system, and server for viewing medical images. The method includes: receiving medical images; outputting screening results based on the medical images, the screening results including at least a lesion judgment result based on the medical images using a machine learning algorithm to determine the presence of lesions, and a quality control judgment result based on information including the lesion judgment result to determine whether the medical images require quality control; dividing the medical images into qualified images and first images to be quality controlled based on the screening results, and using the first images to be quality controlled and a portion of the qualified images as images to be quality controlled; and outputting quality control results based on the images to be quality controlled. This improves the accuracy of medical image screening.

Description

[0001] This application is a divisional application of the patent application filed on 2021-12-08, with application number 2021114930504 and invention titled "Image Reading System and Method for Fundus Images". Technical Field

[0002] This disclosure relates to a method, system, and server for reading medical images. Background Technology

[0003] Medical images often contain numerous details of body structures or tissues. In modern hospitals, a large portion of treatment information comes from medical images, such as fundus images. Clinically, understanding these details in medical images helps doctors identify related diseases. Medical imaging has become a primary method for clinical disease identification. However, traditional medical image-based disease identification relies mainly on the experience of professional physicians. Therefore, developing an automated image interpretation technology that assists doctors in identifying related diseases has become a hot topic in the field of medical imaging. With the development of artificial intelligence technology, image interpretation techniques based on computer vision and artificial intelligence, such as machine learning, have been developed and applied in medical image recognition.

[0004] For example, Patent Document 1 (CN105513077A) discloses a system for screening diabetic retinopathy. The system includes: a fundus image acquisition device, an image processing and screening device, and a report output device. The fundus image acquisition device is used to collect or receive fundus images of the examinee; the image processing and screening device is used to process the fundus images and detect whether there are lesions in them, and then transmit the detection results to the report output device; the report output device outputs a corresponding detection report based on the detection results.

[0005] However, in actual clinical applications, due to the diversity of fundus images, the screening system described in Patent Document 1 may output incorrect or inaccurate test reports when processing certain fundus images, resulting in a decrease in the screening accuracy of the screening system. Summary of the Invention

[0006] This disclosure is made in view of the above-mentioned circumstances, and its purpose is to provide a system and method for reading fundus images that can improve the accuracy of screening.

[0007] To this end, a first aspect of this disclosure provides a system for viewing fundus images, comprising: an input module for receiving fundus images; a screening module for outputting screening results based on the fundus images, the screening results including at least a lesion judgment result based on the fundus images using a machine learning algorithm to determine the presence of lesions, a confidence level of the lesion judgment result, and a quality control judgment result based on information including the lesion judgment result to determine whether the fundus images require quality control, wherein the machine learning algorithm is one of a traditional machine learning algorithm and a deep learning algorithm, and the lesion judgment result includes two results: a negative result and a positive result; a first classification module for classifying the fundus images into screening qualified images and first images to be quality controlled based on the quality control judgment result, and designating at least one of the first images to be quality controlled and the screening qualified images as images to be quality controlled; and a quality control module for receiving the images to be quality controlled and outputting quality control results based on the images to be quality controlled. In this disclosure, the screening module utilizes machine learning algorithms to output screening results, including quality control judgment results and lesion judgment results, based on the fundus images received by the input module. The first classification module, based on the quality control judgment results, divides the fundus images into screening-qualified images and first images to be quality controlled, and uses the first images to be quality controlled and a portion of the screening-qualified images as images to be quality controlled. The quality control module outputs quality control results based on the images to be quality controlled. In this configuration, multi-level image interpretation of fundus images can be performed based on the screening results and quality control results. Therefore, the screening accuracy of the image interpretation system can be improved.

[0008] Furthermore, in the image reading system according to the first aspect of this disclosure, optionally, the image reading system further includes a verification module, which divides the image to be quality controlled into a quality-controlled qualified image and a first image to be arbitrated based on the lesion judgment result and the quality control result, and uses at least one of the first image to be arbitrated and the quality-controlled qualified image as the image to be arbitrated, wherein the quality-controlled qualified image is the image to be arbitrated whose lesion judgment result is the same as the quality control result, and the first image to be arbitrated is the image to be arbitrated whose lesion judgment result is different from the quality control result. In this case, by further processing the image to be arbitrated, the screening accuracy of the image reading system can be effectively improved.

[0009] Additionally, in the image reading system according to the first aspect of this disclosure, the system may optionally include an arbitration module for receiving the image to be arbitrated and outputting an arbitration result based on the image. In this case, an arbitration result can be obtained, thereby further improving the screening accuracy of the image reading system.

[0010] Furthermore, in the image reading system according to the first aspect of this disclosure, optionally, the image reading system further includes a second classification module, which classifies the screened qualified images into negative result images and positive result images based on the lesion judgment result. In this case, obtaining positive result images from the screened qualified images can improve the screening accuracy of the image reading system.

[0011] Additionally, in the image reading system according to the first aspect of this disclosure, the system may optionally include a review module, which outputs a review result based on the positive result image. In this case, by further processing the positive result image, the screening accuracy of the image reading system can be effectively improved.

[0012] Furthermore, in the image reading system according to the first aspect of this disclosure, optionally, the screening module outputs the lesion judgment result according to the retinal lesion grading system used by the UK National Retinopathy Screening Program. Specifically, the lesion judgment result for fundus images with a retinal lesion grade of "no retinal lesion" in the UK National Retinopathy Screening Program grading system is set as a negative result, while the lesion judgment result for fundus images with retinal lesions in the background, preproliferative, and proliferative phases is set as a positive result. In this case, based on the already maturely applied retinal lesion grading system, the screening accuracy of the image reading system can be further improved.

[0013] Furthermore, in the image reading system according to the first aspect of this disclosure, optionally, the screening module obtains the quality control judgment results of the fundus images based on preset rules of different dimensions. Specifically, the dimensions of the misjudged fundus images are determined by analyzing misjudged fundus images within a preset time period of the image reading system, and then the preset rules are updated based on the dimensions of the misjudgments. In this case, fundus images requiring quality control can be promptly screened based on the updated preset rules.

[0014] Furthermore, in the image reading system according to the first aspect of this disclosure, optionally, the misjudged fundus images are classified to determine the category of the misjudgment, and the category of the misjudgment is used as the dimension of the misjudgment. Thus, the dimension of the misjudgment can be determined.

[0015] Furthermore, in the image reading system disclosed in the first aspect, optionally, the screening module also outputs the lesion judgment result in conjunction with the patient's health status, age, and medical history records. In this case, considering the patient's health status, age, and medical history records as factors in the lesion judgment result can further improve the screening accuracy of the image reading system.

[0016] The second aspect of this disclosure provides a method for reading fundus images, comprising: an input step of receiving fundus images; a screening step of outputting a screening result based on the fundus images, the screening result including at least a lesion judgment result based on the fundus images using a machine learning algorithm to determine whether a lesion exists, a confidence level of the lesion judgment result, and a quality control judgment result based on information including the lesion judgment result to determine whether the fundus images require quality control, wherein the machine learning algorithm is one of a traditional machine learning algorithm and a deep learning algorithm, and the lesion judgment result includes two results: a negative result and a positive result; a first classification step of dividing the fundus images into screening qualified images and first images to be quality controlled based on the quality control judgment result, and designating at least one image from the first images to be quality controlled and the screening qualified images as images to be quality controlled; and a quality control step of outputting a quality control result based on the images to be quality controlled. In this disclosure, a machine learning algorithm is used to output screening results, including quality control judgment results and lesion judgment results, based on fundus images. Based on the quality control judgment results, the fundus images are divided into qualified screening images and first images to be quality controlled, and the first images to be quality controlled and a portion of the qualified screening images are used as images to be quality controlled. Quality control results are output based on the images to be quality controlled. In this way, multi-level image interpretation of fundus images can be performed based on screening results and quality control results. Therefore, the screening accuracy of the image interpretation system can be improved.

[0017] According to this disclosure, a system and method for viewing fundus images that can improve the accuracy of screening are provided. Attached Figure Description

[0018] Embodiments of this disclosure will now be explained in further detail by way of example only with reference to the accompanying drawings, in which: Figure 1 This is an illustration of an application scenario of the image reading method for fundus images as described in this disclosure.

[0019] Figure 2 This is a block diagram illustrating a first type of image reading system for fundus images as described in this disclosure.

[0020] Figure 3(a) is a schematic diagram illustrating a fundus image involved in an example of this disclosure.

[0021] Figure 3(b) is a schematic diagram illustrating a fundus image involved in an example of this disclosure.

[0022] Figure 4 This is a schematic diagram illustrating the convolutional kernel used in the convolutional neural network of the screening module involved in the example of this disclosure.

[0023] Figure 5 This is a block diagram illustrating a second type of image reading system for fundus images as described in this disclosure.

[0024] Figure 6 This is a block diagram illustrating a third type of image reading system for fundus images as described in this disclosure.

[0025] Figure 7 This is a block diagram illustrating a fourth image reading system for fundus images as described in the examples of this disclosure.

[0026] Figure 8 This is a flowchart illustrating the first method for viewing fundus images as described in this disclosure.

[0027] Figure 9 This is a flowchart illustrating a second method for viewing fundus images as described in this disclosure.

[0028] Figure 10 This is a flowchart illustrating a third method for viewing fundus images as described in this disclosure. Detailed Implementation

[0029] Hereinafter, preferred embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In the following description, the same reference numerals are used for the same components, and repeated descriptions are omitted. Furthermore, the drawings are merely schematic diagrams, and the proportions of the components or the shapes of the components may differ from actual figures.

[0030] It should be noted that the terms "comprising" and "having" and any variations thereof in this disclosure, such as a process, method, system, product, or device that includes or has a series of steps or units, are not necessarily limited to those steps or units that are explicitly listed, but may include or have other steps or units that are not explicitly listed or that are inherent to such processes, methods, products, or devices.

[0031] This disclosure relates to a system and method for reading fundus images that can improve the accuracy of screening. The system for reading fundus images is sometimes simply referred to as the reading system, and the method for reading fundus images is sometimes simply referred to as the reading method.

[0032] Figure 1 This is an illustration of an application scenario of the image reading method for fundus images as described in this disclosure.

[0033] In some examples, the image viewing method (described later) can be applied to, for example... Figure 1In application scenario 1, as shown, operator 11 can control acquisition device 13 connected to terminal 12 to acquire medical images (e.g., fundus images). After acquisition device 13 completes the acquisition, terminal 12 can submit the medical images to server 14 via a computer network. Server 14 can execute computer program instructions stored on server 14 to implement an image reading method. This method receives the medical images and generates a result report, which is then returned to terminal 12. In some examples, terminal 12 can display the result report. In other examples, the result report can be stored as an intermediate result in the memory of terminal 12 or server 14. In still other examples, the medical images received by the image reading method can be medical images stored in terminal 12 or server 14.

[0034] In some examples, operator 11 may be a professional, such as an ophthalmologist. In other examples, operator 11 may be a general staff member who has received training in image interpretation. Image interpretation training may include, but is not limited to, the operation of the acquisition device 13 and the operation of the terminal 12 involved in the image interpretation method.

[0035] In some examples, terminal 12 may be, but is not limited to, a laptop, tablet, or desktop computer.

[0036] In some examples, the acquisition device 13 may be a camera. The camera may be, for example, a handheld fundus camera or a desktop fundus camera. In some examples, the acquisition device 13 may be connected to the terminal 12 via a serial port. In some examples, the acquisition device 13 may be integrated into the terminal 12.

[0037] In some examples, server 14 may include one or more processors and one or more memories. The processor may include a central processing unit, a graphics processing unit, and any other electronic components capable of processing data and executing computer program instructions. The memory may be used to store computer program instructions. In some examples, server 14 can implement a video viewing method by executing computer program instructions stored in memory. In some examples, server 14 may also be a cloud server.

[0038] The film viewing system 2 disclosed herein will now be described in detail with reference to the accompanying drawings. The film viewing system 2 disclosed herein is used to implement the film viewing method described above. Figure 2 This is a block diagram illustrating the first type of image reading system 2 for fundus images as described in this disclosure.

[0039] In some examples, the image reading system 2 can be applied to fields of screening based on medical images. Examples include screening for diabetic retinopathy, pulmonary nodules, tuberculosis, lung tumors, lumbar osteophyte formation, or various types of fractures. This enables rapid and accurate screening of medical images, obtaining screening results such as negative / positive.

[0040] In some examples, such as Figure 2 As shown, the image reading system 2 may include an input module 10, a screening module 20, a first classification module 30, and a quality control module 40. In the image reading system 2, the input module 10 can receive fundus images; the screening module 20 can output screening results based on the fundus images; the first classification module 30 can classify the fundus images into qualified images and images requiring quality control based on the screening results; and the quality control module 40 can output quality control results based on the images requiring quality control. In this configuration, multi-level image reading of fundus images can be performed based on the screening and quality control results. Therefore, the screening accuracy of the image reading system 2 can be improved.

[0041] In some examples, the image viewing system 2 may be a computer device including an input module 10, a screening module 20, a first classification module 30, and a quality control module 40.

[0042] Figure 3(a) is a schematic diagram showing a fundus image related to an example of this disclosure. Figure 3(b) is a schematic diagram showing a fundus image related to an example of this disclosure.

[0043] In some examples, input module 10 can be used to receive fundus images. In some examples, fundus images can be acquired by acquisition device 13. In other examples, fundus images can be fundus images stored in terminal 12 or server 14. As examples of fundus images, Figures 3(a) and 3(b) show two fundus images of a human eye. However, the examples of this disclosure are not limited to these; in other examples, input module 10 can be used to receive other medical images, such as lung images, brain images, or chest X-ray images. In this case, the image reading system 2 is able to screen different medical images.

[0044] Figure 4 This is a schematic diagram illustrating the convolutional kernel used in the convolutional neural network of the screening module involved in the example of this disclosure.

[0045] In some examples, the screening module 20 can output screening results based on fundus images. In some examples, the screening results may include at least lesion assessment results and quality control assessment results.

[0046] In some examples, the screening module 20 can use machine learning algorithms to output lesion judgment results based on fundus images.

[0047] In some examples, the lesion assessment result can be used to determine whether a lesion is present in the fundus image. In some examples, the lesion assessment result can include both negative and positive results.

[0048] In some examples, the machine learning algorithm can be at least one of traditional machine learning algorithms and deep learning algorithms. In this case, the appropriate machine learning algorithm can be selected according to the actual needs. In some examples, a screening and classification model can be built based on a machine learning algorithm.

[0049] In some examples, the screening and classification model built using deep learning algorithms can be a convolutional neural network (CNN). In some examples, the CNN can use 3×3 convolutional kernels (see...). Figure 4 The system automatically identifies features in fundus images. However, the examples disclosed herein are not limited to this. In other examples, the convolutional neural network (CNN) kernel can be a 5×5 kernel, a 2×2 kernel, or a 7×7 kernel, etc. In this case, due to the high efficiency of the convolutional neural network (CNN) in recognizing image features, the performance of the image reading system 2 can be effectively improved.

[0050] In some examples, such as in diabetic retinopathy (DR) screening, a convolutional neural network (CNN) can learn the features of fundus images with negative results and fundus images with positive results, thereby enabling the CNN to output a lesion judgment result (e.g., negative / positive result) from the fundus images.

[0051] However, the examples disclosed herein are not limited to this. In other examples, the machine learning algorithm of the screening module 20 can be a conventional machine learning algorithm. In some examples, the conventional machine learning algorithm may include, but is not limited to, linear regression, logistic regression, decision tree, support vector machine, or Bayesian algorithms. In this case, image processing algorithms can be used to extract fundus features (such as color features, shape features, or texture features) from the fundus image, and then the fundus features can be input into a screening classification model built based on a conventional machine learning algorithm to achieve screening of the fundus image.

[0052] As described above, the image reading system 2 can be applied to screening for diabetic retinopathy. In some examples, the fundus images in the screening module 20 can be screened according to the retinopathy grading system used by the UK National Retinopathy Screening Programme.

[0053] For example, in some cases, the lesion assessment results of fundus images with a retinal lesion grade of no retinal lesion (R0) in the retinal lesion grading system used by the UK National Retinopathy Screening Program can be set as negative, as can the lesion assessment results of fundus images with retinal lesion grades of background (R1), preproliferative (R2), and proliferative (R3). In this case, based on the already well-established retinal lesion grading system, the screening accuracy of the image reading system 2 can be further improved.

[0054] In some examples, fundus images with a macular degeneration grade of M0 can be classified as negative results, while fundus images with a macular degeneration grade of M1 can be classified as positive results.

[0055] However, the examples disclosed herein are not limited to these. In other examples, fundus images with a retinal lesion grade of R0 and a macular lesion grade of M0 can be determined as negative results, while fundus images in other cases can be determined as positive results.

[0056] As mentioned above, screening results can include at least quality control assessment results and lesion assessment results. In some examples, quality control assessment results can be output based on information related to fundus images. Fundus image-related information can include lesion assessment results. For example, fundus image-related information can include lesion assessment results, the confidence level of the lesion assessment results, the location of the fundus image source, the model of the imaging equipment used to capture the fundus image, the quality of the fundus image, or patient information (e.g., health status or medical history). In some examples, quality control assessment results can be used to determine whether fundus images require quality control. In some examples, quality control assessment results can include both "quality control required" and "quality control not required" results.

[0057] In some examples, the screening module 20 can obtain the quality control judgment results of the fundus images based on preset rules. That is, the screening module 20 can classify the fundus images into two types based on preset rules: those that require quality control and those that do not.

[0058] In some examples, the preset rules can have different dimensions. In some examples, the dimensions can include, but are not limited to, at least one of the following: lesion diagnosis result, confidence level of lesion diagnosis result, source location of fundus image, model of fundus image acquisition device, quality of fundus image, or patient information (e.g., health condition or medical history).

[0059] In some examples, the dimensions of misclassified fundus images can be determined by analyzing misclassified fundus images within a preset time period (e.g., by day, month, or quarter) in the image reading system 2. The preset rules are then continuously updated based on these dimensions. In this case, the updated preset rules can promptly filter out fundus images requiring quality control.

[0060] For example, preliminary preset rules can be set based on dimensions such as lesion assessment results, confidence levels of lesion assessment results, and patient information, and these preset rules can be continuously updated. Specifically, the preliminary preset rules can be to classify fundus images with positive lesion assessment results, existing medical history, and a confidence level of lesion assessment results below a preset value (e.g., 90%) as requiring quality control, otherwise classifying them as not requiring quality control. If, when analyzing misjudgments in the image reading system 2, it is detected that fundus images not included in the preset rules require quality control, or fundus images not requiring quality control are classified as requiring quality control, then the dimension of the misjudgment is determined and the preset rules are updated.

[0061] In some examples, misclassified fundus images can be categorized to determine the misclassification category, which is then used as the dimension of the misclassification. In other examples, categorizing the misclassified fundus images can determine the number or proportion of misclassified fundus images in each category, and then the misclassification category can be determined based on this number or proportion, serving as the dimension of the misclassification. For example, if the number of misclassified fundus images in a certain category exceeds a preset number or the proportion of misclassified fundus images exceeds a preset value, then that category is considered the misclassified category. This allows for the determination of the dimension of the misclassification.

[0062] In some examples, there can be multiple preset rules. In some examples, updating preset rules can include adding, deleting, and modifying preset rules.

[0063] In some examples, the screening module 20 can also output the confidence level of the lesion diagnosis result. Therefore, the reliability of the lesion diagnosis result can be determined based on the confidence level.

[0064] In addition, in some examples, the screening module 20 can also combine health status, age, and medical history records to output lesion judgment results. For example, in some examples, fundus features in fundus images can be extracted first, and the fundus images can be screened based on fundus features, health status, age, and medical history records. In this case, considering the patient's health status, age, and medical history records as factors in the lesion judgment results can further improve the screening accuracy of the image reading system 2.

[0065] In some examples, the screening module 20 can combine other features of the fundus image, such as microaneurysms, hemorrhages, exudates, cotton wool spots, neovascularization, or macular degeneration, to output the lesion judgment result of the fundus image.

[0066] As described above, the image reading system 2 may include a first classification module 30. In some examples, the first classification module 30 may classify fundus images into qualified images and images to be controlled based on screening results.

[0067] In some examples, the first classification module 30 can classify fundus images into screening qualified images and first quality control images based on the quality control judgment results. For example, fundus images whose quality control judgment results indicate that no quality control is required can be used as screening qualified images, and fundus images whose quality control judgment results indicate that quality control is required can be used as first quality control images.

[0068] In some examples, the first classification module 30 can also classify fundus images into qualified screening images and first quality control images based on the confidence level of the lesion judgment results output by the screening module 20. For example, fundus images with a confidence level higher than 80% can be considered qualified screening images, and fundus images with a confidence level lower than 80% can be considered first quality control images; or fundus images with a confidence level higher than 90% can be considered qualified screening images, and fundus images with a confidence level lower than 90% can be considered first quality control images.

[0069] In some examples, fundus images that the screening module 20 cannot determine can be used as the first quality control images.

[0070] In some examples, the first classification module 30 can make a comprehensive judgment based on the confidence levels of the quality control judgment results and the lesion judgment results to divide the fundus images into screening qualified images and first quality control images. For example, fundus images with a quality control judgment result of "no quality control required" and a confidence level higher than 80% can be classified as screening qualified images; fundus images with a quality control judgment result of "quality control required" and a confidence level lower than 80% can be classified as first quality control images; fundus images with a quality control judgment result of "quality control required" and a confidence level higher than 80% can be classified as first quality control images; and fundus images with a quality control judgment result of "no quality control required" and a confidence level lower than 80% can be classified as first quality control images.

[0071] In some examples, the first classification module 30 can use at least one image from the first image to be quality controlled and the images that have passed screening as the image to be quality controlled. In this case, by further processing the partially screened images, the error rate of the partially screened images can be analyzed, and then it can be determined whether all the screened images need to be re-screened based on the error rate. This can effectively improve the screening accuracy of the image reading system 2. Specifically, at least one image from the partially screened images can be considered as the target screened image.

[0072] In some examples, the target screening qualified images may be 10% of the total qualified images. However, the examples disclosed herein are not limited to this. In other examples, the number of target screening qualified images can be adjusted according to actual needs. For example, the number of target screening qualified images can be a preset percentage of the qualified images. For example, the preset percentage can be 5%, 15%, 20%, or 30%, etc. In some examples, the target screening qualified images can have a predetermined number, such as 1000, 2000, or 3000 images.

[0073] As described above, the image viewing system 2 may include a quality control module 40. In some examples, the quality control module 40 may be used to receive images to be quality controlled and output quality control results based on the images. In some examples, the quality control results may include both negative and positive results. In some examples, the quality control module 40 may also output the confidence level of the quality control results.

[0074] In some examples, the quality control module 40 may include a quality control classification model, which may have the same network architecture as the screening classification model in the screening module 20; for example, the quality control classification model may be a convolutional neural network (CNN). In other examples, the quality control classification model in the quality control module 40 may be a more complex classification model than that in the screening module 20. This allows for the acquisition of quality control results with higher confidence levels.

[0075] In some examples, the quality control classification model in the quality control module 40 can be trained to have the ability to output quality control results based on the image to be quality controlled.

[0076] Specifically, in some examples, such as in diabetic retinopathy (DR) screening, a physician with quality control experience can assess the target image to be monitored to obtain a target quality control result, such as a negative / positive result, as the ground truth. The quality control classification model in quality control module 40 is trained and optimized using the target image and the ground truth, so that the model has the same ability as the physician to output quality control results, such as negative / positive results, based on the image to be monitored. In some examples, the target image to be monitored can be obtained based on the target fundus image and using the first classification module 30. In some examples, the target fundus image can be from a cooperating hospital and, for example, 50,000 to 200,000 fundus images with patient information removed. In this case, the quality control result output by the quality control classification model in quality control module 40 is closer to the quality control result of a physician with quality control experience. This improves the screening accuracy of the image reading system 2.

[0077] In some examples, the quality control results can be obtained by a physician with quality control experience judging the images to be controlled. Specifically, in some examples, the images to be controlled can be output in the image reading system 2, and after the physician with quality control experience completes the quality control result analysis of the images, the quality control results are saved into the image reading system 2 so that a result report can be output based on the quality control results later.

[0078] Figure 5 This is a block diagram illustrating a second type of image reading system based on fundus images, as illustrated in this disclosure.

[0079] like Figure 5As shown, in some examples, the image reading system 2 may include a verification module 50. In some examples, the verification module 50 may divide the images to be quality controlled into quality control qualified images and first images to be arbitrated based on the lesion judgment result and quality control result. In some examples, the verification module 50 may use at least one of the first images to be arbitrated and the quality control qualified images as the images to be arbitrated. In this case, by further processing the images to be arbitrated, the screening accuracy of the image reading system 2 can be effectively improved.

[0080] In some examples, a quality control qualified image can be the image to be quality controlled if the lesion assessment result is the same as the quality control result. For example, if the lesion assessment result is negative and the quality control result is also negative, then the image to be quality controlled can be used as the quality control qualified image. In some examples, the first image to be arbitrated can be the image to be quality controlled if the lesion assessment result is different from the quality control result. For example, if the lesion assessment result is negative and the quality control result is positive, then the image to be quality controlled can be used as the first image to be arbitrated.

[0081] In some examples, the verification module 50 can use at least one image from the first image to be arbitrated and the quality control qualified images as the image to be arbitrated. In this case, by further processing the partially qualified images, the error rate of the partially qualified images can be analyzed, and then it can be determined whether all the quality control qualified images need to be re-controlled based on the error rate. This can effectively improve the screening accuracy of the image reading system 2. Specifically, at least one image from the quality control qualified images can be considered as the target quality control qualified image.

[0082] In some examples, the target quality control qualified images can be 10% of the quality control qualified images. However, the examples disclosed herein are not limited to this. In other examples, the number of target quality control qualified images can be adjusted according to actual needs. For example, the number of target quality control qualified images can be a preset percentage of the quality control qualified images. For example, the preset percentage can be 5%, 15%, 20%, or 30%, etc. In some examples, the target quality control qualified images have a predetermined number, such as 1000, 2000, or 3000 images.

[0083] In some examples, the viewing system 2 may include an arbitration module 60 (see [link to relevant documentation]). Figure 5 The arbitration module 60 can receive the image to be arbitrated and output the arbitration result based on the image. In this case, the arbitration result can be obtained, thereby further improving the screening accuracy of the image reading system 2. In some examples, the arbitration result can include both negative and positive results. In some examples, the arbitration module 60 can also output the confidence level of the arbitration result.

[0084] In some examples, the arbitration module 60 may include an arbitration classification model, which may have the same network architecture as the screening classification model in the screening module 20 and the arbitration classification model in the quality control module 40. For example, the arbitration classification model may be a convolutional neural network (CNN).

[0085] In some examples, the arbitration classification model in arbitration module 60 can be trained to output arbitration results based on the image to be arbitrated.

[0086] Specifically, in some examples, such as in diabetic retinopathy (DR) screening, a physician with arbitration experience can judge the target image to be arbitrated to obtain a target arbitration result for the target image, and use the target arbitration result, such as a negative / positive result, as the ground truth. The arbitration classification model in arbitration module 60 is trained and optimized using the target image to be arbitrated and this ground truth, so that the arbitration classification model in arbitration module 60 has the same ability as the physician to output arbitration results, such as negative / positive results, based on the image to be arbitrated. In some examples, the target image to be arbitrated can be obtained based on the target fundus image and using verification module 50. In this case, the arbitration result output by the arbitration classification model in arbitration module 60 is closer to the arbitration result of a physician with arbitration experience, thereby improving the screening accuracy of image reading system 2.

[0087] In some examples, a physician with arbitration experience can assess the images to be arbitrated to obtain an arbitration result. Specifically, in some examples, the images to be arbitrated can be output in the image reading system 2. After the physician with arbitration experience completes the arbitration result analysis of the images, the arbitration result is saved in the image reading system 2 for subsequent output of a result report based on the arbitration result.

[0088] Figure 6 This is a block diagram illustrating a third type of image reading system based on fundus images, as illustrated in this disclosure.

[0089] like Figure 6 As shown, in some examples, the image reading system 2 may include a second classification module 70. In some examples, the second classification module 70 may classify qualified images into negative result images and positive result images based on the lesion judgment result (e.g., positive result or negative result). In this case, obtaining positive result images for qualified images can improve the screening accuracy of the image reading system.

[0090] In some examples, the results report can be output using output module 90 (described later) for screening qualified images that are negative results.

[0091] However, the examples disclosed herein are not limited to this. In other examples, the second classification module 70 may classify quality control qualified images into negative result images and positive result images based on quality control results (e.g., positive or negative results). In other examples, the second classification module 70 may classify images to be arbitrated into negative result images and positive result images based on arbitration results (e.g., positive or negative results).

[0092] In some examples, the image viewing system 2 may include a review module 80 (see [link to relevant documentation]). Figure 6 The re-examination module 80 can output re-examination results based on positive result images. Clinically, positive result images often receive more attention than negative result images. In this case, further processing of positive result images can effectively improve the screening accuracy of the image reading system 2.

[0093] In some examples, the review module 80 may include a review classification model. In some examples, the review classification model in the review module 80 can be trained to learn the ability to output review results based on positive result images.

[0094] Specifically, in some examples, such as in diabetic retinopathy (DR) screening, a doctor with experience in follow-up examinations can judge the target positive result image to obtain the target follow-up result of the target positive result image, and take the target follow-up result, such as negative / positive, as the ground truth. The follow-up classification model in the follow-up module 80 is trained and optimized using the target positive result image and this ground truth, so that the follow-up classification model in the follow-up module 80 has the same ability as the doctor to output follow-up results, such as negative / positive, based on the positive result image. In some examples, the target positive result image can be obtained based on the target fundus image using the second classification module 70. In this case, the follow-up result output by the follow-up classification model in the follow-up module 80 is closer to the follow-up result of a doctor with follow-up experience, thereby improving the screening accuracy of the image reading system 2.

[0095] In some examples, a doctor with experience in follow-up examinations can interpret the positive result images to obtain the follow-up results. Specifically, in some examples, the positive result images can be output in the image reading system 2. After the doctor with experience in follow-up examinations completes the follow-up result analysis of the positive result images, the follow-up results are saved into the image reading system 2 so that a result report can be output based on the follow-up results later.

[0096] Figure 7 This is a block diagram illustrating a fourth image reading system for fundus images as described in the examples of this disclosure.

[0097] like Figure 7As shown, in some examples, the image reading system 2 may also include an output module 90. The output module 90 can output a result report of the fundus images. In some examples, the output module 90 can judge at least one of the lesion judgment result, quality control result, arbitration result, and re-examination result to output a result report of the fundus images.

[0098] In some examples, output module 90 can evaluate at least two of the following: lesion assessment results, quality control results, arbitration results, and follow-up results, to output a result report of the fundus image. In this case, multiple results from the image reading system 2 can be combined to output a result report of the fundus image. In some examples, the result report may also include information about the patient's medical history, health status, and age.

[0099] In some examples, the output module 90 can assign corresponding priorities to different results. Specifically, in the output module 90, at least two results among the lesion judgment result, quality control result, arbitration result, and re-examination result can be assigned corresponding priorities. In some examples, the result with the highest priority can be output to the result report. In some examples, priorities can be set from high to low according to the order of re-examination result, arbitration result, quality control result, and lesion judgment result. However, the examples in this disclosure are not limited to these, and the priority setting method can be adjusted according to actual needs.

[0100] In some examples, the results report may include a table with at least six columns. The first column may be the ID of the fundus image, the second column may be the lesion assessment result of the fundus image, the third column may be the quality control result of the fundus image, the fourth column may be the arbitration result of the fundus image, the fifth column may be the re-examination result of the fundus image, and the sixth column may be the final result of the fundus image. In some examples, the table may also have a seventh column of data, which may contain other information about the fundus image. In this case, the assessment process and results of the fundus image can be shown in detail. Thus, a comprehensive assessment of the fundus image can be made.

[0101] The following, combined with Figure 8 This disclosure describes in detail the method for reading fundus images. The method for reading fundus images disclosed herein may sometimes be simply referred to as the reading method. The reading method is applied in the aforementioned reading system 2. Figure 8 This is a flowchart illustrating the first method for viewing fundus images as described in this disclosure.

[0102] In some examples, such as Figure 8As shown, the image reading method may include receiving fundus images (input step S10), outputting screening results based on the fundus images (screening step S20), dividing the fundus images into qualified images and images to be quality controlled (first classification step S30), and outputting quality control results based on the images to be quality controlled (quality control step S40). In this case, multi-level image reading of fundus images can be performed based on screening results and quality control results. Therefore, the screening accuracy of image reading can be improved.

[0103] In some examples, fundus images can be received in input step S10. For a detailed description, please refer to input module 10 in the image reading system 2; it will not be repeated here.

[0104] In some examples, screening results can be output based on fundus images in screening step S20. In some examples, the screening results may include at least lesion assessment results and quality control assessment results. In some examples, machine learning algorithms can be used to output lesion assessment results based on fundus images. In some examples, lesion assessment results can be used to determine whether lesions exist in the fundus images. In some examples, lesion assessment results may include both negative and positive results. In some examples, the machine learning algorithm can be at least one of traditional machine learning algorithms and deep learning algorithms. In this case, an appropriate machine learning algorithm can be selected according to actual needs. In some examples, the screening classification model built based on deep learning algorithms can be a convolutional neural network (CNN). In this case, since convolutional neural networks (CNNs) are highly efficient in recognizing image features, they can effectively improve the performance of image reading. In some examples, fundus images can be screened according to the retinal lesion grading system used by the UK National Retinopathy Screening Project in screening step S20. In this case, based on the already maturely applied retinal lesion grading system, the screening accuracy of image reading can be further improved. For a detailed description, please refer to screening module 20 in image reading system 2, which will not be repeated here.

[0105] In some examples, in screening step S20, a quality control judgment result can be output based on information related to the fundus image. This information may include lesion judgment results. In some examples, the quality control judgment result can be used to determine whether the fundus image requires quality control. In some examples, the quality control judgment result may include both "requires quality control" and "does not require quality control." Additionally, in some examples, the confidence level of the lesion judgment result can also be output in screening step S20. This allows for the assessment of the reliability of the lesion judgment result based on the confidence level. Furthermore, in some examples, the lesion judgment result can be output in combination with the patient's health status, age, and medical history in screening step S20. In this case, considering the patient's health status, age, and medical history as factors in the lesion judgment result can further improve the screening accuracy of the image reading. For a detailed description, please refer to screening module 20 in the image reading system 2; it will not be repeated here.

[0106] In some examples, in the first classification step S30, fundus images can be divided into screening qualified images and images to be quality controlled. In some examples, fundus images can be divided into screening qualified images and first images to be quality controlled based on the quality control judgment results. In some examples, at least one image from the first image to be quality controlled and the screening qualified images can be used as the image to be quality controlled. In this case, by further processing the partially screening qualified images, the error rate of the partially screening qualified images can be analyzed, and then it can be determined whether all screening qualified images need to be re-screened based on the error rate. Thus, the screening accuracy of image reading can be effectively improved. For a detailed description, please refer to the relevant description of the first classification module 30, which will not be repeated here.

[0107] In some examples, quality control step S40 can output quality control results based on the image to be quality controlled. In other examples, quality control step S40 can establish a quality control classification model based on machine learning algorithms, and train the model to output quality control results based on the image to be quality controlled. For a detailed description, please refer to the relevant description of quality control module 40, which will not be repeated here.

[0108] In some examples, during quality control step S40, a physician with quality control experience can evaluate the image to be controlled to obtain the quality control results. Specifically, in some examples, the image to be controlled can be output, and a physician with quality control experience can analyze the quality control results of the image and save the results for subsequent output of a report based on the quality control results.

[0109] Figure 9 This is a flowchart illustrating a second method for viewing fundus images as described in this disclosure.

[0110] like Figure 9As shown, the image reading method may include a verification step S50. In some examples, in verification step S50, the image to be quality controlled can be divided into a quality control qualified image and an image to be arbitrated. In some examples, the image to be quality controlled can be divided into a quality control qualified image and a first image to be arbitrated based on the lesion judgment result and the quality control result. In some examples, at least one of the first image to be arbitrated and the quality control qualified image can be used as the image to be arbitrated. In this case, by further processing the image to be arbitrated, the screening accuracy of the image reading system 2 can be effectively improved. In some examples, the quality control qualified image is the image to be quality controlled whose lesion judgment result is the same as the quality control result, and the first image to be arbitrated is the image to be quality controlled whose lesion judgment result is different from the quality control result. For a detailed description, please refer to the relevant description of the verification module 50, which will not be repeated here.

[0111] In some examples, the image viewing method may also include arbitration step S60 (see Figure 9 In some examples, arbitration step S60 can output an arbitration result based on the image to be arbitrated. In some examples, arbitration step S60 can establish an arbitration classification model based on machine learning algorithms, and train the model to enable it to output an arbitration result based on the image to be arbitrated. For a detailed description, please refer to the relevant description of arbitration module 60, which will not be repeated here.

[0112] In some examples, during arbitration step S60, a physician with arbitration experience can assess the image to be arbitrated to obtain an arbitration result. Specifically, in some examples, the image to be arbitrated can be output, and after the physician with arbitration experience completes the arbitration result analysis of the image, the arbitration result is saved for subsequent output of a result report based on the arbitration result.

[0113] Figure 10 This is a flowchart illustrating a third method for viewing fundus images as described in this disclosure.

[0114] like Figure 10 As shown, the image reading method may include a second classification step S70. In some examples, in the second classification step S70, qualified images can be classified into negative result images and positive result images. In some examples, in the second classification step S70, qualified images can be classified into negative result images and positive result images based on the lesion judgment result. In this case, obtaining a positive result image from the qualified images can improve the screening accuracy of the image reading system. For a detailed description, please refer to the relevant description of the second classification module 70, which will not be repeated here.

[0115] In some examples, the image review method may include a review step S80 (see Figure 10In some examples, in the re-examination step S80, the re-examination result can be output based on the positive result image. In this case, by further processing the positive result image, the screening accuracy of image reading can be effectively improved. In some examples, in the re-examination step S80, a re-examination classification model can be built based on a machine learning algorithm, and through training, the re-examination classification model can learn to output the re-examination result based on the positive result image. For a detailed description, please refer to the relevant description of the re-examination module 80, which will not be repeated here.

[0116] In some examples, during the follow-up step S80, a doctor with follow-up experience can interpret the positive result image to obtain the follow-up result. Specifically, in some examples, the positive result image can be output, and after the doctor with follow-up experience completes the follow-up result analysis of the positive result image, the follow-up result is saved for subsequent output of a result report based on the follow-up result.

[0117] In some examples, the image reading method may include an output step (not shown) that outputs a result report of the fundus image. In this case, a result report of the fundus image can be output. For a detailed description, please refer to the relevant description of the output module 90, which will not be repeated here.

[0118] While the present disclosure has been specifically described above in conjunction with the accompanying drawings and embodiments, it is to be understood that the above description does not limit the present disclosure in any way. Those skilled in the art can make modifications and variations to the present disclosure as needed without departing from its essential spirit and scope, and all such modifications and variations fall within the scope of the present disclosure.

Claims

1. A method for interpreting medical images, characterized in that, include: Receive medical images; Based on the medical image output screening results, the screening results include at least a lesion judgment result based on the medical image output using a machine learning algorithm to determine whether a lesion exists, and a quality control judgment result based on information including the lesion judgment result to determine whether the medical image needs quality control. Based on the screening results, the medical images are divided into qualified images and first images to be quality controlled, and the first images to be quality controlled, along with a portion of the qualified images, are used as images to be quality controlled; and The quality control results are output based on the image to be controlled.

2. The image reading method according to claim 1, characterized in that, Also includes: Based on the lesion judgment results and the quality control results, the images to be quality controlled are divided into quality control qualified images and first arbitration images; At least one of the first image to be arbitrated and the quality control qualified image is used as the image to be arbitrated, wherein the quality control qualified image is the image to be arbitrated whose lesion judgment result is the same as the quality control result, and the first image to be arbitrated is the image to be arbitrated whose lesion judgment result is different from the quality control result; The arbitration result is output based on the image to be arbitrated.

3. The image reading method according to claim 1, characterized in that, Also includes: Based on the lesion judgment results, the qualified screening images are divided into negative result images and positive result images, and the re-examination results are output based on the positive result images.

4. The image reading method according to claim 1, characterized in that, A quality control classification model is established based on a machine learning algorithm, and through training, the quality control classification model is made capable of outputting the quality control results based on the image to be quality controlled.

5. The image reading method according to claim 1, characterized in that, The quality control judgment results of the medical images are obtained based on preset rules of different dimensions. Specifically, the dimensions of misjudgment are determined by analyzing the misjudged medical images within a preset time period, and then the preset rules are updated based on the dimensions of misjudgment.

6. The image reading method according to claim 5, characterized in that, The dimensions include at least one of the following: the lesion diagnosis result of the medical image, the confidence level of the lesion diagnosis result of the medical image, the source location of the medical image, the model of the medical image capturing equipment, the quality of the medical image, or patient information.

7. The image reading method according to claim 1, characterized in that, Based on the screening results, the medical images are divided into the qualified screening images and the first images to be quality control, including at least one of the following methods: Based on the quality control judgment results, the medical images are divided into the screened qualified images and the first images to be quality controlled; Based on the confidence level of the lesion judgment results, the medical images are divided into the qualified screening images and the first quality control images; Based on the confidence levels of the quality control judgment results and the lesion judgment results, a comprehensive judgment is made to divide the medical images into the screening qualified images and the first quality control images; The medical image that cannot be determined is selected as the first image to be quality controlled.

8. The image viewing method according to any one of claims 1 to 7, characterized in that, The medical image is a fundus image.

9. A server, characterized in that, The server includes one or more processors and one or more memories, and implements the image viewing method according to any one of claims 1 to 8 by executing computer program instructions on the memory.

10. A medical image reading system, characterized in that, The image viewing method is used to implement any one of claims 1 to 8.

Images