Medical image processing device, medical image processing system, medical image processing method and program

Abstract

To achieve both accuracy and efficiency of image reading.SOLUTION: A medical image processor according to the present invention includes: acquisition means that acquires information on a medical image and an image reading doctor; estimation means that estimates a degree of difficulty of diagnostic imaging for the medical image; and determination means that determines an image reading doctor in charge of image reading, on the basis of the estimated degree of difficulty of diagnostic imaging and the information on the image reading doctor.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a medical image processing apparatus, a medical image processing system, a medical image processing method, and a program. [Background technology]

[0002] In recent years, the number of medical image data to be interpreted is increasing due to the improvement of imaging device performance and the increase in the number of imaging sessions, while the number of highly skilled doctors to interpret images is insufficient. As a result, not all medical images can be interpreted by highly skilled doctors, which poses the problem that doctors may not be skilled enough to perform appropriate image diagnosis.

[0003] For example, Non-Patent Document 1 explains the accuracy of image interpretation during eye examinations, and points out that the accuracy of image interpretation differs between ophthalmologists and general practitioners.

[0004] In addition, to address the issue of a shortage of doctors who can interpret images, there has been active development of technology that uses computers to support the workflow of doctors who interpret images.

[0005] Patent Document 1 describes a technology that determines the number of doctors who will read images based on information about abnormal shadow candidate regions detected from medical images using artificial intelligence. The efficiency of image diagnosis is improved by determining the number of doctors who will read images according to the type, number, and probability of abnormal shadow candidate regions, or the department responsible for reading images.

[0006] Patent Document 2 describes a technology for remote diagnosis of an ophthalmologic device, in which an ophthalmologic imaging device transmits information indicating the disease name or candidate disease name, and the degree of difficulty of interpretation to an external device, and requests interpretation. By selecting a doctor who can interpret images of that level of difficulty, appropriate interpretation can be performed remotely. [Prior art documents] [Patent documents]

[0007] [Patent Document 1] JP 2020-60857 A [Patent Document 2] JP 2019-213734 A [Non-patent literature]

[0008] [Non-Patent Document 1] Japanese Ophthalmology, Vol. 88, No. 1, Appendix: A collection of research results from the "Adult Eye Examination" research group Summary of the Invention [Problem to be solved by the invention]

[0009] However, the methods described in Patent Documents 1 and 2 may not improve the accuracy or efficiency of diagnosis. For example, there may be cases where the skill and expertise of the image interpreting physicians varies. When there is a variation in the skill and expertise of the image interpreting physicians, the effect of interpretation by multiple people is reduced. For example, when an image interpreting physician with low skill and an image interpreting physician with high skill are in charge, there is little point in the image interpreting physician with low skill, and from the viewpoint of efficiency, the image interpreting by the image interpreting physician with low skill is wasted.

[0010] In addition, the method described in Patent Document 2 does not describe a method for assigning a level of difficulty to image interpretation. When an examiner who requests image interpretation assigns a level of difficulty, the level of difficulty assigned will vary depending on the level of proficiency of the examiner. In particular, when an examiner with a level of proficiency who requires remote imaging support assigns a level of difficulty, it is expected that there will be variation. On the other hand, it is inefficient for an experienced person such as an image interpretation doctor to accurately assign a level of difficulty. If there is variation in the level of difficulty assigned, it becomes difficult to assign it to an appropriate image interpretation doctor, and the accuracy and efficiency of image interpretation decreases. In the case of a large-scale and large number of people in charge of image acquisition and image interpretation, such as a health check, it becomes even more difficult to standardize the proficiency of examiners and image interpretation doctors to increase accuracy and efficiency.

[0011] As described above, it has been desired to achieve both accuracy and efficiency in image interpretation.

[0012] In view of the above problems, an object of the present invention is to provide a system that achieves both diagnostic accuracy and efficiency in diagnosing medical images.

[0013] In addition to the above-mentioned object, another object of the present invention is to achieve an effect that is derived from each configuration shown in the description of the embodiment of the invention described below, and that cannot be obtained by conventional techniques. [Means for solving the problem]

[0014] The medical image processing apparatus according to the present invention comprises: Information on medical images and radiologists and An acquisition means for acquiring the An image finding for the medical image is estimated using a learning model that has learned a pair of an input image and an image finding determined for the input image, and the estimated image finding is used to An estimation means for estimating a degree of difficulty of image diagnosis for the medical image; The estimated difficulty of the image diagnosis and Information about the interpreting physician and A determination means for determining an allocation to a radiologist in charge of reading the image based on the above information; a transmitting means for transmitting the medical image and the estimated image findings to an information terminal related to the image interpretation doctor based on the determined allocation; Equipped with do. Effect of the Invention

[0015] According to the present invention, it is possible to provide a system that achieves both diagnostic accuracy and efficiency in diagnosing medical images. [Brief description of the drawings]

[0016] [Figure 1] FIG. 1 is a schematic diagram showing an example of the configuration of a medical image processing system according to the present invention. [Diagram 2] FIG. 1 is a schematic diagram showing an example of the configuration of a medical image processing apparatus according to the present invention. [Diagram 3] FIG. 1 is a flow chart showing an example of the outline of the process of the present invention. [Figure 4] FIG. 1 is a diagram showing an example of a medical image according to the present invention. [Diagram 5] FIG. 1 is a diagram showing an example of a combination of medical image interpretation results according to the present invention. [Figure 6]FIG. 1 is a diagram illustrating an example of a display screen of the present invention. [Figure 7] FIG. 1 is a diagram illustrating an example of a display screen of the present invention. [Figure 8] FIG. 1 is a diagram showing an example of a medical image according to the present invention. [Figure 9] FIG. 1 is a diagram showing an example of a medical image according to the present invention. [Figure 10] FIG. 1 is a diagram showing an example of a medical image according to the present invention. [Figure 11] FIG. 1 is a schematic diagram showing an example of the configuration of a medical image processing system according to the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0017] A preferred embodiment of the medical image processing system according to the present invention will be described in detail below with reference to the attached drawings. However, the components described in this embodiment are merely examples, and the technical scope of the medical image processing system according to the present invention is determined by the claims, and is not limited to the individual embodiments described below. Furthermore, the present invention is not limited to the following embodiments, and various modifications (including organic combinations of each embodiment) are possible based on the spirit of the present invention, and are not excluded from the scope of the present invention. In other words, configurations that combine each of the embodiments described below and their modifications are all included in the embodiments disclosed in this specification.

[0018] In the following embodiment, a case where a fundus camera is used as a representative example of an imaging device for capturing medical images used to estimate the difficulty of image diagnosis will be described, but the present invention is not limited to this, and other imaging devices can also be suitably applied. For example, other ophthalmic examination devices such as optical coherence tomography devices capable of capturing images of the fundus or anterior eye may be used depending on the disease for which the difficulty is to be estimated. In addition, imaging devices used for screening and diagnosing diseases, such as general X-ray imaging, X-ray computed tomography, nuclear magnetic resonance imaging, ultrasound examination, positron emission tomography, and single photon emission tomography, may also be used.

[0019] [Embodiment 1] The medical image processing system according to this embodiment is characterized by estimating the difficulty of image diagnosis for a medical image, and determining the image interpreter who will interpret the image based on the estimated diagnostic difficulty and the proficiency of the image interpreter.

[0020] More specifically, for example, the medical image processing system assigns medical images that are estimated to have a low level of difficulty in image diagnosis to radiologists with low proficiency, and assigns medical images that are estimated to have a high level of difficulty in image diagnosis to radiologists with high proficiency.

[0021] Hereinafter, one embodiment of the present invention will be described in detail with reference to the drawings.

[0022] FIG. 1 is a diagram showing the overall configuration of a medical image processing system 100 including a medical image processing apparatus according to this embodiment.

[0023] The medical image processing system 100 includes a fundus camera 101 , a data server 102 , a medical image processing device 103 , and an image interpretation terminal 104 .

[0024] The fundus camera 101 captures a fundus image, which is a medical image used for image diagnosis.

[0025] An example of a fundus image taken by the fundus camera 101 is shown in FIG. 4. The fundus camera 101 shines strong light on the eye and takes a color image of the fundus (retina). As shown in FIG. 4, the optic disc 401 where the optic nerve gathers, the macula 402, and the surrounding retina are photographed, and the size and course of blood vessels in the fundus, as well as the presence or absence of bleeding or retinal detachment, can be determined. From the fundus image, doctors can check for the presence or absence of eye diseases such as glaucoma, diabetic retinopathy, age-related macular degeneration, pathological myopia, and cataracts. It has also been reported that the image can reveal signs of cerebrovascular disease and Alzheimer's syndrome in addition to eye diseases.

[0026] The data server 102 is communicably connected to the fundus camera 101 and the medical image processing device 103 via a network, and stores and manages data such as images captured by the fundus camera 101 and related medical information.

[0027] As shown in FIG. 2, the medical image processing device 103 has, as its functional configuration, a communication IF (Interface) 111, a ROM (Read Only Memory) 112, a RAM (Random Access Memory) 113, a storage unit 114, an operation unit 115, a display unit 116, and a control means 117.

[0028] The communication IF 111 is realized by a LAN card or the like, and controls communication between an external device (e.g., the data server 102) and the medical image processing device 103. The ROM 112 is realized by a non-volatile memory or the like, and stores various programs and the like. The RAM 113 is realized by a volatile memory or the like, and temporarily stores various information. The storage unit 114 is an example of a computer-readable storage medium, and is realized by a large-capacity information storage device represented by a hard disk drive (HDD) or a solid-state drive (SSD), and stores various information. The operation unit 115 is realized by a keyboard, a mouse, or the like, and inputs instructions from a user to the device. The display unit 116 is realized by a display or the like, and displays various information to the user. The control means 117 is realized by a CPU (Central Processing Unit), a GPU (Graphical Processing Unit), or the like, and controls each process in the medical image processing device 104.

[0029] The control means 117 includes, as its functional configuration, an acquisition means 118, an estimation means 119, a display control means 120, a determination means 121, and a transmission means 122.

[0030] The acquiring means 118 reads and acquires data such as a fundus image of the subject captured by the fundus camera 101 from the data server 102 .

[0031] The estimation means 119 estimates the degree of difficulty of image diagnosis from the fundus image acquired by the acquisition means 118.

[0032] The display control means 120 displays the fundus image acquired by the acquisition means 118 and the result estimated by the estimation means 119 on the display unit.

[0033] The determining means 121 determines an image interpreting doctor who will perform the image diagnosis based on the difficulty of the image diagnosis estimated by the estimating means 119 and the proficiency of the image interpreting doctor.

[0034] The transmitting means 122 transmits the medical image to the interpretation terminal 104 based on the result determined by the determining means 121 .

[0035] The image interpretation terminal 104 includes a receiving means 124 for receiving a medical image from the medical image processing device 103, a display control means 125 for displaying the received image on the display unit 123, a receiving means 126 for receiving an input of an image interpretation result, and a transmitting means 127 for transmitting the image interpretation result. The image interpretation terminal 104 may take the form of, for example, a PC or a tablet terminal equipped with a monitor and an input means. The image interpretation terminal 104 may be a dedicated terminal or a general-purpose terminal program. The image interpretation terminal 104 may be located within the facility where the image interpretation is performed, or may be located outside the facility. In addition, one image interpretation terminal may be shared by multiple image interpretation doctors, or, for example, when there are multiple image interpretation doctors, an image interpretation terminal 104a may be provided for image interpretation doctor A, and an image interpretation terminal 104b may be provided for image interpretation doctor B, as shown in FIG. 1.

[0036] Next, the processing procedure of the medical image processing apparatus 103 according to this embodiment will be described with reference to the flowchart of FIG.

[0037] (S1) (Acquire medical images) In S1, the acquisition means 118 reads and acquires data such as a fundus image of the subject captured by the fundus camera 101 and information on the image interpreting doctor used to determine the image interpreting doctor who will perform the diagnosis from the data server 102. The data server 102 stores a plurality of image interpreting doctors who are in charge of image interpretation, divided into two or more classes corresponding to their proficiency. The information on the image interpreting doctor is, for example, information indicating the proficiency of the image interpreting doctor who performs the image diagnosis, and includes items such as years of experience, specialty (whether the doctor is a specialist or a general practitioner), the number of images interpreted, and the accuracy rate of image interpretation in a test for evaluating proficiency. The information on the image interpreting doctor is sufficient if it is possible to acquire at least which class the doctor belongs to among the classes classified based on the proficiency. The information on the image interpreting doctor is not limited to the above, and for example, the specialty may be divided into three classes, such as a glaucoma specialist, a general ophthalmologist, and a general practitioner, and may be information indicating the specialty field of the image interpreting doctor.

[0038] The acquisition means 118 may acquire all information related to the image-reading physician stored in the data server 102, or may acquire only some of the items. When acquiring some of the items, the acquisition means 118 may automatically acquire predetermined information, or may acquire information on the items selected by the user via the operation unit 115.

[0039] Moreover, the data does not necessarily have to be acquired from the data server 102, and for example, data directly transmitted from the fundus camera 101 may be acquired. Also, the server from which the data is acquired may differ depending on the information to be acquired. For example, the medical image and the information on the image interpreter may be acquired from different data servers.

[0040] (S2) (Estimate the difficulty of the diagnosis) In S2, the estimation means 119 estimates the diagnostic difficulty of the medical image acquired in S1 by inputting the medical image into a learning model.

[0041] In this embodiment, a score related to the difficulty of image diagnosis, which will be described in detail below, is estimated from medical images using a learning model.

[0042] In this embodiment, the learning model is constructed by performing supervised learning using a neural network in which a pair of a medical image, which is input data, and a label, which is a score related to the difficulty of diagnosis, which is the output result, is used as training data. The learning model may be generated by a learning means (not shown) included in the control means 117, or may be generated by a processing device different from the medical image processing device 103. The learning model may be any model that can output a score related to the difficulty (or a calculation of the probability for each score) from an input image. For example, a convolutional network (CNN), a support vector machine (SVM), or a learning algorithm that combines multiple filters and networks in a multi-layered manner may be used.

[0043] Next, we will explain the difficulty of diagnosing medical images. The difficulty of diagnosing medical images refers to an index of how accurately the presence or absence of a disease can be determined from a medical image. An example of a combination of one disease and one test will be explained using Figure 5. The true results of the presence or absence of a disease and the positive or negative results of the test to determine that disease are divided into a 2 x 2 table as shown in Figure 5. Based on the combination of the presence or absence of a disease and the results, they can be classified as a: true positive, b: false positive, c: false negative, and d: true negative. Of these, true positives and true negatives correspond to cases where a diagnosis is made accurately. Conversely, false negatives and false positives correspond to oversights and false positives, respectively.

[0044] Even if the medical images are normal, there are variations in the images to be interpreted, such as individual differences, camera models, and shooting conditions. Furthermore, if there are morphological changes due to other diseases, the difficulty of interpretation changes. The difficulty of interpreting the images also changes depending on the type, progression, degree, and location of the disease to be interpreted.

[0045] As a specific example, a fundus photograph will be used. Since the interpretation of a fundus photograph is basically qualitative, even if the fundus change is normal, it may be difficult to detect the disease. For example, in severe myopia, the axial length may elongate and the fundus may be deformed. In addition, the size and shape of the optic disc vary depending on the case of inclined optic disc, congenital large optic disc (enlarged cup), optic nerve hypoplasia, and optic disc hypoplasia. In addition, in myopic eyes and elderly people, the retina becomes thin, resulting in a leopard-shaped fundus where the choroidal blood vessels can be seen through. Furthermore, in the case of diseased eyes, the condition of the fundus changes depending on the disease, so the difficulty of interpretation changes.

[0046] Therefore, it is considered that a doctor with high proficiency can accurately judge an image with high difficulty. Note that the degree of difficulty of image interpretation in this embodiment may be calculated for all image interpretation doctors, or may be calculated for a partial group of all image interpretation doctors. For example, the degree of difficulty of image interpretation may be calculated for a group of doctors with low proficiency who are in charge of image interpretation.

[0047] Next, the score related to the difficulty of image diagnosis will be described. The score may be any one that indicates the difficulty of interpretation, and may be a continuous numerical value or a discrete classification. In the case of a continuous numerical value, for example, a numerical value from 0 to 100 may be used. In the case of a discrete classification, for example, two or more stages such as low difficulty and high difficulty may be used. In addition, even if the numerical value is continuous, it can be handled as discrete by setting a threshold value and classifying it. Conversely, even in the case of a discrete classification, it is possible to handle it as continuous by including a probability.

[0048] A method for determining the score used as the training data will be described. The score may be an index related to the difficulty of interpretation. For example, the accuracy rate when multiple radiologists diagnose an image with a definite diagnosis may be used. In addition, the probability that the results of multiple radiologists actually interpreting each image will be true positive, true negative, false positive, or false negative may be used as the score.

[0049] Then, a specific image reading doctor or multiple image reading doctors judge the score. Note that when a specific image reading doctor or multiple image reading doctors judge the difficulty score for each medical image, the judgment may be subjective or a judgment criterion may be set.

[0050] The score may also be determined based on image findings such as the presence or absence of a specific disease, the classification of the progression of the disease, the classification of the severity of the disease or the probability of belonging to that classification, the site or range of the lesion, or the presence or absence of a change in shape. Alternatively, the score may be determined based on the image quality of the medical image. For example, contrast, degree of focus, signal intensity, amount of random noise, SN ratio, fixed noise, reflection such as flare, presence or absence of artifacts, or resolution may be used.

[0051] For example, images without shape changes or disease may be classified into a score corresponding to a low degree of difficulty, and the rest may be classified into a score corresponding to a high degree of difficulty. Alternatively, medical images with a specific disease that is difficult to judge may be classified into a high degree of difficulty, and conversely, diseases that are easy to judge may be classified into a score corresponding to a low degree of difficulty. Alternatively, the early stage of the progression of the disease may be classified into a high degree of difficulty, and the later stage may be classified into a low degree of difficulty. Alternatively, the degree of the disease may be classified into a mild (or narrow range) degree of high difficulty, and the degree of the disease may be classified into a severe (or wide range) degree of low difficulty. Furthermore, the region where the lesion is located in the center (in the case of the fundus, the vicinity of the macula or optic disc) may be classified into a low degree of difficulty, and the peripheral region may be classified into a high degree of difficulty. In terms of image quality, images with low contrast or images with artifacts may be classified into a high degree of difficulty, and images with high contrast may be classified into a low degree of difficulty.

[0052] In the above, an example was shown in which the estimation means 119 estimates the degree of difficulty using a learning model that has been trained using, as labels, image findings such as the presence or absence of a specific disease, or a score related to the degree of difficulty determined based on the image quality of a medical image. However, for example, image findings may be estimated using a learning model that has been trained using medical images and image findings for the medical images as labels, and the degree of difficulty may be determined based on the estimated image findings, and a doctor who will read the image may be assigned to the latter stage. Alternatively, the degree of difficulty may be estimated using any learning model.

[0053] In other words, the estimation means 119 corresponds to an example of an estimation means that estimates the difficulty of image diagnosis for the medical image using at least one of a first learning model that has learned a pair of an input image and a score related to the difficulty of image diagnosis for the input image, and a second learning model that has learned a pair of an input image and an image finding determined for the input image.

[0054] Alternatively, in order to output the information as diagnostic support information for image interpretation physicians, estimation of image findings may be performed separately from estimation of difficulty level.

[0055] In this case, the image findings estimated by the estimation means 119 may be the presence or absence of a specific disease, a disease progression classification, or a probability of belonging to a severity classification. The specific disease may be one or more of glaucoma, diabetic retinopathy, age-related macular degeneration, pathological myopia, cataract, cerebrovascular disease, and Alzheimer's syndrome. Furthermore, a visualization of the site, range, and size of the disease may be combined.

[0056] In this case, for example, when estimating the presence or absence of a specific disease, images with a high probability of being estimated to have a specific disease (probability of existence estimated to be 80% to 100%) and images with a low probability of being estimated to have a specific disease (probability of existence 0% to 20%) are determined to have a low degree of difficulty in interpretation. Images with neither a high nor a low probability of being estimated to have a specific disease (probability of existence 30% to 70%) are determined to have a high degree of difficulty in interpretation. Note that the above thresholds and determination methods are merely examples and are not intended to be limiting.

[0057] In addition, pre-processing and post-processing of medical images may be performed using a learning model. Also, subject information such as the subject's gender and age may be added to the input.

[0058] For the judgment of items other than the difficulty of image diagnosis, one common learning model may be used to obtain multiple outputs from input images. Alternatively, multiple learning models may be prepared for each output, such as a model for estimating the difficulty and a model for estimating image findings. In either case, pairs of input data and corresponding labels are used as training data for learning.

[0059] By estimating the difficulty level using a learning model, it is possible to efficiently estimate the difficulty level of image diagnosis without human assistance. It is also possible to obtain information on the presence or absence of associated diseases.

[0060] (S3) (Determine the allocation of radiologists) In S3, the determination means 121 determines the allocation of image interpretation doctors who will interpret the medical images based on the scores related to the difficulty of interpreting the medical images estimated by the estimation means 119.

[0061] More specifically, the determination means 121 determines the allocation of medical images to be interpreted based on the information on the image interpretation physician acquired in S1 and the score on the difficulty of image interpretation estimated by the estimation means 119.

[0062] For example, a correspondence is established in advance between the score regarding the difficulty of image interpretation and the group of doctors to whom the image interpretation is assigned, and medical images with a low score regarding the difficulty of image diagnosis are assigned to a group with low proficiency, and medical images with a high score regarding the difficulty of image diagnosis are preferentially assigned to a group with high proficiency.

[0063] In this way, by performing image interpretation by matching the proficiency of image interpretation with the difficulty of the image, image interpretation can be performed efficiently while maintaining accuracy.

[0064] In addition, a group of highly skilled doctors may be assigned medical images with low difficulty. When doctors are divided into three or more groups, the scores may be assigned to the groups of doctors in a similar manner.

[0065] (S4) (Send the image to the radiologist) In S4, the transmission means 122 transmits the medical images to the image interpretation terminal 104 via a communication line or the like. Specifically, the images are transmitted to the image interpretation terminals of the image interpretation doctors based on the allocation determined in S3. For example, a medical image estimated to be difficult to interpret is transmitted to the image interpretation terminal 104a of a highly skilled image interpretation doctor, and a medical image estimated to be difficult to interpret is transmitted to the image interpretation terminal 104b of a less skilled image interpretation doctor.

[0066] In addition, the transmission means 122 may transmit various medical information such as the imaging device and the imaging site for diagnostic purposes in addition to the raw medical images. For example, it may transmit images that have been processed to remove noise and artifacts from the medical images, or images whose appearance has been corrected in terms of contrast, brightness, angle, magnification, etc. Furthermore, it may transmit image analysis results such as numerical data obtained by analyzing the medical images, maps, charts, or differences from previously obtained images.

[0067] Furthermore, the transmission means 122 may transmit subject information in addition to medical images. For example, information that is useful for reference in interpretation, such as age, sex, items filled in a medical questionnaire, medical history, past examination history, and medical images of the same subject measured in the past, may be included. In addition to subject information, images similar to a suspected disease may also be transmitted.

[0068] (S5) (Accepts reading input) In S5, the receiving means 124 included in the image interpretation terminal 104 receives the medical image transmitted from the medical image processing device 103. Then, the display control means 125 displays the received medical image on the display unit, and the receiving means 126 receives an input from the image interpretation doctor regarding the medical image displayed on the display unit.

[0069] Here, an example of the display screen of the image interpretation terminal 104 is shown in FIG. 6. The display screen 600 has an image interpretation doctor information screen 601 that displays information about the image interpretation doctor, and an image display screen 602 that displays medical images for image interpretation. The image display screen 602 has at least an image interpretation image display field 603 that displays the medical image to be interpreted, an examinee information display field 604 that displays examinee information, and an image interpretation result input field 605 for inputting the image interpretation result. When there are multiple medical images, a switch button 606 for switching the images may be provided. For example, in the case of a fundus image, a switch for switching the left and right eyes may be provided. In addition, an information display field 607 for displaying information to assist image interpretation and a selection button 608 for switching the contents to be displayed may be provided. The image interpretation result input field 605 has, for example, a field 609 for entering whether or not a detailed examination is necessary, a field 610 for entering comments, and a field 611 for entering the name of the image interpretation doctor. The information may be entered using an input means such as a keyboard or a mouse (not shown). Alternatively, if the display unit 123 is a GUI (Graphical User Interface) screen, the user may perform operational input on the GUI screen.

[0070] The screen also includes a save button 612 for saving the image interpretation results, a send button 613 for confirming and sending the image interpretation results, and a stop button 614 for stopping the image interpretation.

[0071] Furthermore, in S2, when the medical image processing device 103 estimates a diagnosis result for the medical image, such as the presence or absence of a disease, an estimation result display field 620 may be provided to display the estimation result based on the learning model. Examples of the estimation result to be displayed include the presence or absence of a disease and its probability. By displaying the estimation result based on the learning model, the radiologist can refer to it when making an image diagnosis.

[0072] Furthermore, based on the estimation result of the learning model, diagnosis candidates to be input may be displayed in advance in the image interpretation result input field 605. By displaying diagnosis candidates in advance, the efficiency of image interpretation can be improved.

[0073] Furthermore, there may be a difficulty level display field 621 that displays the difficulty level of image diagnosis. By displaying the difficulty level, the radiologist can refer to whether the medical image requires special attention. Conversely, a display selection button 622 may allow the radiologist to select whether to display or hide the estimation results based on the learning model. When the estimation results based on the learning model are hidden, the radiologist can perform the interpretation without being influenced by the estimation results.

[0074] The display screen 600 may change the display contents according to the group of radiologists in charge of interpretation or according to information registered in advance for each radiologist. The display contents settings may be stored in a storage unit (not shown) provided in the interpretation terminal 104. By changing the display contents, it is possible to provide information suited to each radiologist, and interpretation can be performed more efficiently. For example, the number of medical images or additional information displayed for a group of radiologists with high proficiency may be increased compared to a group with low proficiency. A group of radiologists with high proficiency is assigned difficult medical images, and therefore requires relatively more detailed information.

[0075] Examples of additional information include past captured images and, if there are capture results using other modalities, other capture results may be displayed.

[0076] For example, when there is an image taken by an OCT (Optical Coherence Tomography), a SLO (Scanning Laser Ophthalmoscope), or a slit lamp other than the fundus camera, the image may be displayed on the display screen. As an example, a tomographic image of the retina by OCT is shown in FIG. 7. The OCT image may be displayed in the read image display field 703 or in the information display field 707. In the example of FIG. 7, a fundus image is displayed in the read image display field 703, and a tomographic image is displayed in the information display field 707. Furthermore, when an image of a different modality such as an OCT image is displayed, the operation UI may be changed. In the example of FIG. 7, the cross section of the tomographic image can be changed by operating the operation unit 730. Although an OCT image can obtain a tomographic image and three-dimensional information, it requires more skill to interpret than a fundus camera. Therefore, depending on the proficiency of the doctor who interprets the image and the difficulty of the image interpretation, displaying an OCT image may be useful for image interpretation, or may not be useful for image interpretation and may reduce efficiency. Also, if an OCTA (Optical Coherence Tomography Angiography) image is available, the OCTA image may be displayed. An example of an OCTA blood vessel image is shown in FIG. 8(a). An OCTA image can obtain the shape of retinal blood vessels and obtain more detailed information, but requires more skill to interpret than an OCT. For a group with a low level of proficiency, for example, a chart diagram 801 may be displayed as an analysis result of OCTA, as shown in FIG. 8(b). Alternatively, an image in which the analysis result is superimposed on a fundus image of a fundus camera may be displayed. FIG. 9 shows an example in which contour lines 902 representing shape data are superimposed on a fundus camera. Similarly, a thickness map of the optic nerve fiber layer may be superimposed. Alternatively, a fundus image may be displayed three-dimensionally using OCT, as shown in FIG. 10. These may be displayed as images that can be interpreted by the doctor in charge. The image display screen 602 may be set to display from the beginning of interpretation, or may be configured to be switchable at any time using a selection button 608 or the like. By switching the display for each doctor, the accuracy and efficiency of interpretation can be improved.

[0077] As additional information, images that have been subjected to image analysis may be displayed. Conversely, for the group of radiologists with low proficiency, diagnostic support may be provided by displaying images in which areas estimated to be diseased are colored and highlighted, or by displaying the results of numerical judgments based on image analysis instead of images. For example, the vertical Cup / Disc ratio (C / D ratio) is an index for diagnosing glaucoma from fundus images. The C / D ratio in normal eyes is 0.3 or less, and when it exceeds 0.7, glaucoma is suspected. The C / D ratio may be calculated by image analysis, and this numerical value may be displayed for reference to the group of radiologists with low proficiency.

[0078] (S6) (Notify the interpretation result) In S6, when the image interpretation doctor confirms the image interpretation result, the transmission means 127 transmits the image interpretation result to at least one of the data server 102 and the medical image processing device 103. Then, the transmitted image interpretation result is registered in the subject information stored in at least one of the data server 102 and the medical image processing device 103.

[0079] Then, the medical image processing device 103 notifies the subject of the diagnosis result by a notification means (not shown). The notification method may be to notify the subject directly or indirectly via a medical examination institution, a health insurance association, etc.

[0080] The processing of the medical image processing system 100 is carried out as described above.

[0081] According to the above, by estimating the degree of difficulty of image diagnosis for a medical image and determining an image interpretation doctor who will interpret the image based on the estimated degree of difficulty of diagnosis, it is possible to achieve both accuracy and efficiency of diagnosis.

[0082] (Variation 1) In this modification, in S5, the interpreted image display section 603 on the display screen 600 for displaying the medical image to be interpreted displays the image that has been subjected to image processing such as image processing and correction.

[0083] For example, noise may be removed, and things that hinder image diagnosis such as flare, eyelashes, artifacts, etc. Image processing may be performed in advance before the image is transmitted, or may be performed by the image interpretation terminal 104.

[0084] For image processing, for example, a learning model that outputs an image that has been subjected to image processing, different from the learning model that estimates the difficulty of diagnosis used in S2, can be used.

[0085] For example, even with the same fundus camera, there are subtle differences in color, contrast, and color balance between different manufacturers and different models.Furthermore, the angle of view and the appearance of the periphery are also different.Even with monochrome images, the gradation display method may be black and white or pseudo color.

[0086] Therefore, when a medical image taken with a different model is input, a learning model may be generated that processes the image so that it resembles an image from a model that the radiologist in charge is familiar with, and the image processing may be performed using the learning model. Note that the learning model may learn a data set of images (input data) taken with an imaging device that actually takes medical images of the subject and images (output data) taken with an imaging device that the radiologist is familiar with.

[0087] By switching the display for each interpreting physician, the accuracy and efficiency of interpretation can be improved.

[0088] (Variation 2) In this modification, in S2, image findings are estimated using a learning model, and the estimated image findings are associated with the medical image to be interpreted and transmitted to the image interpreting physician.

[0089] Then, in S5, if the interpretation result by the radiologist and the evaluation result of the learning model differ, a warning display section 623 that calls attention is displayed on the display screen 600 as shown in Fig. 6. If the judgments of the learning model and the radiologist differ, input errors can be prevented by displaying a warning in the warning display section 623. If the radiologist determines that there is no problem with the interpretation result, even if a warning is displayed in the warning display section 623, the radiologist can confirm the interpretation result after checking.

[0090] Alternatively, in S5, if the interpretation result by the image interpreter and the evaluation result of the learning model differ, the image diagnosis may be reassigned to another image interpreter. First, the evaluation result of the learning model is compared with the judgment of the image interpreter, and if the interpretation result matches, the interpretation result is confirmed.

[0091] On the other hand, if the image reading physician inputs a different judgment, the image reading result is provisionally registered and a request is made for reassignment to an image reading group with a higher level of proficiency. The result is then returned to the server. If there is a request for reassignment, the reassignment is made. This configuration can further improve the accuracy of image reading.

[0092] (Variation 3) In this modified example, a method for evaluating the proficiency of an image reading doctor will be described. The image reading doctor to be evaluated for proficiency reads the training data of the learning model or an equivalent medical image. Since the correct answer of the training data is known, the accuracy rate of the image reading doctor can be calculated by comparing it with the image reading result of the image reading doctor. The proficiency of the image reading doctor can be evaluated from the accuracy rate. The evaluation of the proficiency may be performed separately from normal image reading. Alternatively, the evaluation may be performed by mixing medical images for evaluation with medical images for normal image reading.

[0093] Furthermore, the proficiency level can be evaluated, for example, when a new radiologist is registered, or the proficiency level can be reevaluated. The proficiency level of radiologists can be calculated based on the accuracy rate of image diagnosis, and the radiologists can be divided into two or more groups according to their proficiency level and stored in the radiologist information storage means. In this way, the proficiency level of radiologists can be evaluated and classified with more objective data.

[0094] When performing image interpretation for proficiency evaluation, the doctor may be presented with correct or incorrect image diagnosis. By presenting correct or incorrect images, the doctor can be trained.

[0095] (Variation 4) The remuneration for interpretation may be different for groups of doctors divided according to their proficiency. The remuneration may be different per image, or per hour. Alternatively, the remuneration may be different according to the difficulty of each medical image. In this way, a remuneration system according to the level of proficiency and difficulty can be established.

[0096] (Variation 5) The learning model may have another learning model that learns from a data group in which medical images are input data and the type and urgency of a disease are output data as teacher data. By configuring in this way, the priority of image diagnosis can be determined. In the case of fundus images, images with high priority include, for example, images in which fundus bleeding is observed and images in which shape abnormalities such as retinal detachment are in the process of developing.

[0097] Furthermore, the determining means 121 may be configured to preferentially assign medical images with a high priority to image interpretation doctors over other medical images based on the degree of difficulty and urgency of image diagnosis estimated by the estimating means 119. With such a configuration, medical images of subjects who may have a disease with a high degree of urgency can be preferentially interpreted.

[0098] (Variation 6) A method of remote diagnosis using the medical image processing apparatus 103 of the present invention will be described with reference to FIG.

[0099] An image diagnosis facility 1103 having a medical image processing device 103 is connected to an image acquisition facility 1101 having a fundus camera 101 via a communication means 1100 such as an internet line. The image acquisition facility 1101 may be a general hospital or clinic, a specialized clinic such as an ophthalmology clinic, or a medical examination facility. If the image acquisition means does not require a specialized examiner or can be worn by a subject, the image acquisition facility may be a facility other than a hospital or a general home.

[0100] A subject has medical images acquired at an image acquisition facility 1101, either directly or through a health checkup provided by a health insurance association or a public institution, or by using medical services provided by a company. The medical image processing device 103 is also connected to an image interpretation terminal 1104 at the hand of an image interpretation doctor via a communication means 1100. With this configuration, the medical images acquired at the image acquisition facility 1101 are interpreted by an image interpretation doctor via the image diagnosis facility 1101. The image interpretation results are finally transmitted from the image diagnosis facility 1101 to the subject. The image interpretation results may be transmitted by sending a written document or by sending it to an information terminal 1105 possessed by the subject. The subject determines the presence or absence of a disease and the need for a detailed examination based on the image interpretation results.

[0101] Other embodiments The present invention can also be realized by a process in which a program for realizing one or more functions of the above-mentioned embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. Also, the present invention can be realized by a circuit for realizing one or more functions.

[0102] The processor or circuitry may include a central processing unit (CPU), a microprocessing unit (MPU), a graphics processing unit (GPU), an application specific integrated circuit (ASIC), a field programmable gateway (FPGA), a digital signal processor (DSP), a data flow processor (DFP), or a neural processing unit (NPU).

[0103] The medical image processing system in each of the above-mentioned embodiments may be realized as a single device, or may be realized as a form in which multiple devices are combined so as to be able to communicate with each other and execute the above-mentioned processing, and either form is included in the embodiments of the present invention. The above-mentioned processing may be executed by a common server device or a group of servers. The medical image processing device and the multiple devices constituting the medical image processing system only need to be able to communicate at a predetermined communication rate, and do not need to be located in the same facility or the same country.

[0104] The embodiments of the present invention include a form in which a software program that realizes the functions of the above-described embodiments is supplied to a system or device, and the computer of the system or device reads and executes the code of the supplied program.

[0105] Therefore, the program code itself installed in a computer to realize the processing according to the embodiment is also one embodiment of the present invention. Also, the OS or the like running on the computer performs part or all of the actual processing based on the instructions contained in the program read by the computer, and the functions of the above-mentioned embodiment can also be realized by that processing.

[0106] Furthermore, the present invention is not limited to the above-mentioned embodiment, and various modifications (including organic combinations of the respective examples) are possible based on the spirit of the present invention, and are not excluded from the scope of the disclosure of this specification. In other words, all configurations that combine the above-mentioned respective examples and their modifications are included in the embodiments disclosed in this specification. [Explanation of symbols]

[0107] 100 Medical image processing system 101 Fundus Camera 102 Data Server 103 Medical image processing device 104 Image Reading Terminal 117 Control Means 118 Acquisition means 119 Estimation means 120 Display control means 121 Decision-making procedures 122 Transmission means

Claims

1. An acquisition means for acquiring information on a medical image and an image interpreting physician; an estimation means for estimating image findings for the medical image using a learning model that has learned pairs of an input image and image findings determined for the input image, and estimating a degree of difficulty of image diagnosis for the medical image using the estimated image findings; A determination means for determining an allocation to a radiologist in charge of interpretation based on the estimated difficulty of image diagnosis and information on the radiologist; a transmitting means for transmitting the medical image and the estimated image findings to an information terminal related to the image interpretation doctor based on the determined allocation; A medical image processing device comprising:

2. The medical image processing apparatus according to claim 1 , wherein the transmission means further transmits a similar image related to the estimated image finding to an information terminal related to the image interpretation doctor.

3. 3. The medical image processing device according to claim 1, wherein the score related to the difficulty of the image diagnosis is determined based on at least one of the following information: the accuracy rate when a radiologist interprets an input image, and at least one of the following image findings: the presence or absence of a specific disease, a classification of the progression of the disease, a classification of the severity of the disease, the location of the lesion, the extent of the lesion, and the presence or absence of a change in shape.

4. The medical image processing apparatus according to claim 1 , wherein the medical image is an image captured by any one of a fundus camera, an OCT, an SLO, and a slit lamp.

5. The medical image processing apparatus according to claim 1 , wherein the information about the image-reading doctor is information indicating to which class the image-reading doctor belongs, out of a plurality of classes classified based on proficiency.

6. The medical image processing apparatus according to claim 5 , wherein the information on the image interpretation doctor further includes at least one of years of experience, specialty, number of image interpretations, and accuracy rate of image interpretation in a test for proficiency evaluation.

7. 7. The medical image processing device according to claim 5 or 6, wherein the determination means allocates medical images estimated by the estimation means to have a high degree of difficulty in image diagnosis to radiologists belonging to a class with a high level of proficiency among the plurality of classes, and allocates medical images estimated to have a low degree of difficulty in image diagnosis to radiologists belonging to a class with a low level of proficiency among the plurality of classes.

8. The medical image processing apparatus according to claim 1 , wherein the transmission means further transmits information to be filled in on a medical questionnaire.

9. The medical image processing apparatus according to claim 1 , wherein the transmission means transmits to the information terminal, among the medical images, a medical image associated with information indicating a high urgency of interpretation with priority.

10. A medical image processing system including the medical image processing apparatus according to any one of claims 1 to 9 and an information terminal, The information terminal includes: a display control means for displaying the medical image transmitted from the medical image processing device on a display unit; a reception unit for receiving an input of an interpretation result for the medical image; A medical image processing system comprising:

11. The medical image processing system according to claim 10 , wherein the display control means displays, on the display unit, the degree of difficulty of image diagnosis of the medical image estimated by the estimation means together with the medical image.

12. 12. The medical image processing system according to claim 10, wherein the display control means changes the display contents based on the proficiency level of the radiologist who displays the medical image.

13. The medical image processing system according to any one of claims 10 to 12, wherein the display control means displays a warning on the display unit when the interpretation result accepted by the acceptance means differs from information regarding the estimated image findings for the medical image.

14. The medical image processing system according to any one of claims 10 to 12, wherein the information terminal transmits a notification to the medical image processing device to reallocate the medical image to a different information terminal when the interpretation result accepted by the acceptance means differs from the information regarding the estimated image findings for the medical image.

15. A method for operating a medical image processing apparatus, comprising: An acquisition step of acquiring a medical image and information regarding a doctor who interprets the image; an estimation step in which the medical image processing device estimates image findings for the medical image using a learning model that has learned pairs of an input image and image findings determined for the input image, and the medical image processing device estimates a degree of difficulty of image diagnosis for the medical image using the estimated image findings; A determination step in which the medical image processing device determines an allocation to a radiologist in charge of interpretation based on the estimated difficulty of image diagnosis and information on the radiologist; a transmission step in which the medical image processing device transmits the medical image and the estimated image findings to an information terminal related to the image interpretation doctor based on the determined allocation; A method of operating a medical imaging device comprising:

16. A program for causing a computer to execute the method for operating a medical image processing apparatus according to claim 15.

Images