Medical image display device, method, and program

Abstract

To display an analysis result on a medical image without executing work for receiving an instruction on display each time the analysis result is displayed or work for displaying the medical image for a specified time or more in a medical image display device, method, and program.SOLUTION: A processor displays a medical image, acquires an analysis result for the medical image, and displays an annotation indicating the analysis result in addition to the medical image on which an instruction for display is given when the instruction to display the medical image is given in the case where the analysis result has already been referred to on the basis of reference information indicating whether or not the analysis result has already been referred to by a user.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present disclosure relates to a medical image display device, a method, and a program. [Background technology]

[0002] In recent years, advances in medical equipment such as CT (Computed Tomography) and MRI (Magnetic Resonance Imaging) have made it possible to perform diagnostic imaging using higher quality, higher resolution medical images. In particular, diagnostic imaging using CT and MRI images allows for accurate identification of lesion areas, leading to the provision of appropriate treatment based on the results of such identification.

[0003] Medical images are also analyzed using computer-aided diagnosis (CAD) software, which uses machine learning models based on deep learning and other techniques, to detect abnormalities, such as lesions, contained in the medical images. The analysis results generated by the CAD analysis process are associated with examination information, such as the patient's name, gender, age, and the modality used to acquire the medical image, and are stored in a database for use in diagnosis. Doctors interpret the medical images by referring to the delivered medical images and analysis results on their own image-reading devices. Based on the interpretation results, the doctor annotates the abnormal shadows contained in the medical images. For example, the doctor may annotate the area surrounding the abnormal shadow, add an arrow indicating the abnormal shadow, or add text indicating the type and size of the disease. When the analysis results are referenced, annotations representing the analysis results are displayed on the medical image.

[0004] On the other hand, doctors' interpretation of medical images should not be guided by the results of CAD analysis. For this reason, CAD analysis results are often used as a secondary interpretation (second reading) in clinical settings. For example, when interpreting an image, a doctor first performs a primary interpretation of the medical image without referring to the results of CAD analysis, and then adds annotations to the medical image representing the interpretation results. After that, the doctor displays the medical image annotated based on the results of CAD analysis, and performs a secondary interpretation of the medical image while referring to the annotations representing the analysis results. By performing primary and secondary interpretations in this manner, it is possible to prevent disease areas from being overlooked.

[0005] (The terms primary reading and secondary reading are not used in the prior art, so they have been deleted.) Patent Document 1 proposes a method of displaying the CAD analysis results superimposed on or in parallel with the doctor's reading results after the doctor has completed the primary reading. Specifically, when the doctor signs off to indicate that the reading is complete, or when the medical image has been displayed for more than a specified time, it is determined that the reading is complete and the CAD analysis results are displayed. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Japanese Patent Application Laid-Open No. 2006-115921 Summary of the Invention [Problem to be solved by the invention]

[0007] The interpretation results of medical images on which CAD analysis results are displayed may be affected by the CAD analysis results. For this reason, for medical images on which CAD analysis results have already been displayed, it is extremely cumbersome for doctors to have to accept display instructions every time the CAD results are displayed, or to display the medical image for longer than the specified time.

[0008] The present disclosure has been made in consideration of the above circumstances, and aims to make it possible to display analysis results for medical images without having to accept display instructions each time analysis results are displayed, or displaying medical images for longer than the specified time. [Means for solving the problem]

[0009] A medical image display device according to the present disclosure includes at least one processor, The processor displays the medical image; Obtaining analysis results for medical images, Based on reference information indicating whether the analysis results have been referenced by the user, if the analysis results have been referenced, when a command to display a medical image is given, an annotation indicating the analysis results is displayed in addition to the requested medical image.

[0010] In the medical image display device according to the present disclosure, the reference information may be managed on a per-user basis.

[0011] Furthermore, in the medical image display device according to the present disclosure, the reference information may be managed in units of groups each including a plurality of users.

[0012] In this case, when a user in the group instructs the display of a medical image for which the analysis results have been referenced for all users in the group, the processor may display an annotation representing the analysis results in addition to the requested medical image, and when a user in the group instructs the display of a medical image for which the analysis results have not been referenced for at least one user in the group, the processor may display only the requested medical image.

[0013] In addition, in the medical image display device according to the present disclosure, the processor may visually display whether or not the analysis result has been referred to for the medical image to be displayed.

[0014] In addition, in the medical image display device according to the present disclosure, the processor manages a setting value indicating whether or not the analysis result that has already been referred to is to be displayed when the user next displays the medical image, even if the user does not issue an instruction to display the analysis result; If the setting value is set to display the analysis results, when the user next displays a medical image, an annotation representing the analysis results is displayed in addition to the medical image; If the setting value is set so that the analysis results are not displayed, the next time the user displays a medical image, only the medical image may be displayed.

[0015] In this case, the setting values ​​may be managed on a per-user basis.

[0016] A medical image display method according to the present disclosure includes displaying a medical image, Obtaining analysis results for medical images, Based on reference information indicating whether the analysis results have been referenced by the user, if the analysis results have been referenced, when an instruction to display a medical image is given, an annotation indicating the analysis results is displayed in addition to the requested medical image.

[0017] The medical image display method according to the present disclosure may be provided as a program for causing a computer to execute the method. [Effects of the Invention]

[0018] According to the present disclosure, analysis results for medical images can be displayed without receiving a display instruction each time an analysis result is displayed or without displaying a medical image for a specified time or longer. [Brief explanation of the drawings]

[0019] [Figure 1] FIG. 1 is a diagram showing a schematic configuration of a medical information system to which a medical image display device according to an embodiment of the present disclosure is applied. [Figure 2] FIG. 1 is a diagram showing a schematic configuration of a medical image display device according to an embodiment of the present invention. [Figure 3]Functional configuration diagram of the medical image display device according to this embodiment [Figure 4] A flowchart showing the processing performed in this embodiment [Figure 5] A flowchart showing the processing performed in this embodiment [Figure 6] Diagram showing the display screen for primary reading [Figure 7] A diagram showing a display screen on which the primary interpretation results are displayed. [Figure 8] A diagram showing a display screen with a notification that editing is prohibited [Figure 9] A diagram showing the display screen showing the analysis results [Figure 10] FIG. 1 illustrates management information including reference information associated with a medical image. [Figure 11] A diagram showing a display screen on which secondary interpretation results are displayed. [Figure 12] FIG. 10 shows another example of reference information associated with a medical image. [Figure 13] A flowchart showing a process performed using reference information in this embodiment. [Figure 14] A diagram showing the display screen showing the analysis results [Figure 15] FIG. 10 is a diagram showing another example of management information including reference information. [Figure 16] A flowchart showing a process performed using reference information managed in groups. [Figure 17] FIG. 10 is a diagram showing a display screen that visually indicates whether or not the analysis results have been referenced. [Figure 18] FIG. 10 is a diagram showing a display screen on which analysis results are displayed in another embodiment. [Figure 19] FIG. 10 shows management information including reference information associated with a medical image in another embodiment. [Figure 20] A flowchart illustrating the processing performed using reference information in another embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0020] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. First, the configuration of a medical information system 1 to which a medical image display device according to this embodiment is applied will be described. FIG. 1 is a diagram showing a schematic configuration of the medical information system 1. The medical information system 1 shown in FIG. 1 is a system for capturing an examination target region of a subject, storing the medical images acquired by capturing the images, having a radiologist interpret the medical images and create an interpretation report, and allowing the requesting doctor from the medical department to view the interpretation report and observe the details of the medical image to be interpreted, based on an examination order from a doctor from a medical department using a known ordering system.

[0021] As shown in Figure 1, the medical information system 1 is composed of multiple imaging devices 2, multiple interpretation WSs (Workstations) 3 which are interpretation terminals, a medical treatment WS 4, an image server 5, an image database (hereinafter referred to as image DB (DataBase)) 6, a report server 7, a report database (hereinafter referred to as report DB) 8, and an analysis server 9, all connected in a state where they can communicate with each other via a wired or wireless network 10.

[0022] Each device is a computer installed with an application program that causes the device to function as a component of the medical information system 1. The application program is stored in an externally accessible state in a storage device of a server computer connected to the network 10 or in network storage, and is downloaded and installed into the computer upon request. Alternatively, the application program is recorded on a recording medium such as a DVD (Digital Versatile Disc) or CD-ROM (Compact Disc Read Only Memory) and distributed, and then installed into the computer from the recording medium.

[0023] The imaging device 2 is a device (modality) that generates a medical image representing the diagnostic target part by imaging the diagnostic target part of the subject. Specifically, it is a plain X-ray imaging device, a CT device, an MRI device, a PET (Positron Emission Tomography) device, etc. The medical images generated by the device 2 are transmitted to the image server 5 and stored in the image database 6. Note that a simple X-ray imaging device is a device that obtains a two-dimensional radiographic image, which is a transmitted image of a subject, by irradiating the subject with radiation once.

[0024] The image interpretation WS3 is a computer used by, for example, a radiologist to interpret medical images and create image interpretation reports, and includes the medical image display device 20 according to this embodiment. The image interpretation WS3 performs tasks such as sending requests to the image server 5 to view medical images, performing various image processing on medical images received from the image server 5, displaying the medical images, and accepting input of findings related to the medical images. The image interpretation WS3 also performs tasks such as creating image interpretation reports, sending requests to the report server 7 to register and view image interpretation reports, and displaying image interpretation reports received from the report server 7. These processes are performed by the image interpretation WS3 executing software programs for each process.

[0025] The medical treatment WS4 is a computer used by doctors in the medical department to observe images in detail, view radiology reports, and create electronic medical records, and is composed of a processing device, a display device such as a monitor, and input devices such as a keyboard and a mouse. The medical treatment WS4 sends image viewing requests to the image server 5, displays images received from the image server 5, requests radiology reports to the report server 7, and displays radiology reports received from the report server 7. These processes are performed by the medical treatment WS4 executing software programs for each process.

[0026] The image server 5 is a general-purpose computer installed with a software program that provides the functions of a database management system (DBMS). The image server 5 also has storage in which the image DB 6 is configured. This storage may be a hard disk device connected to the image server 5 via a data bus, or a disk device connected to a NAS (Network Attached Storage) or SAN (Storage Area Network) connected to the network 10. When the image server 5 receives a request to register a medical image from the imaging device 2, it formats the medical image in a database format and registers it in the image DB 6.

[0027] The image DB 6 stores image data and additional information of medical images acquired by the imaging device 2. The additional information includes, for example, an image ID (identification) for identifying each medical image, a patient ID for identifying the subject, an examination ID for identifying the examination, a unique ID (UID: unique identification) assigned to each medical image, and a time stamp (time stamp) for generating the medical image. This information includes the date and time of the examination, the type of imaging device used in the examination to obtain the medical images, patient information such as the patient's name, age, and gender, the area examined (imaged area), imaging information (imaging protocol, imaging sequence, imaging method, imaging conditions, use of contrast agent, etc.), and the series number or collection number if multiple medical images are obtained in one examination.

[0028] In addition, when the image server 5 receives a viewing request from the image interpretation WS3 and medical treatment WS4 via the network 10, it searches for medical images registered in the image DB6 and transmits the searched medical images to the image interpretation WS3 and medical treatment WS4 that made the request.

[0029] A software program that provides a general-purpose computer with the functions of a database management system is installed in the report server 7. When the report server 7 receives a request to register an interpretation report from the interpretation WS 3, it converts the interpretation report into a database format and registers it in the report DB 8.

[0030] The report DB 8 registers image interpretation reports created by image interpretation physicians using the image interpretation WS 3. The image interpretation report may include information such as the medical image to be interpreted, an image ID for identifying the medical image, an image interpretation physician ID for identifying the image interpretation physician who performed the image interpretation, the disease name, location information of the disease, and information for accessing the medical image.

[0031] In addition, when the report server 7 receives a request to view an interpretation report from the interpretation WS3 and medical treatment WS4 via the network 10, it searches for the interpretation report registered in the report DB8 and sends the retrieved interpretation report to the interpretation WS3 and medical treatment WS4 that made the request.

[0032] The analysis server 9 is a computer installed with a software program that provides the function of analyzing medical images using CAD or the like to detect abnormal shadows such as lesions. The analysis server 9 analyzes medical images in response to instructions from the medical treatment WS 4 or the image interpretation WS 3. Upon receiving an instruction to analyze a medical image, the analysis server 9 acquires the medical image to be processed from the image server 5. The analysis server 9 then analyzes the acquired medical image and transmits the analysis results to the medical treatment WS 4 or image interpretation WS 3 that issued the instruction. The analysis results are also transmitted to the image server 5 and registered in the image DB 6. The analysis results include area information indicating the position of the abnormal shadow in the medical image identified by the analysis, contour information indicating the position of the contour of the abnormal shadow, coordinate information indicating the center position of the abnormal shadow, size information indicating the size of the abnormal shadow, and text information indicating the characteristics of the abnormal shadow.

[0033] CAD analysis can be performed using a learning model that has undergone machine learning to detect abnormal shadows in medical images. The learning model is a convolutional neural network (CNN) that has undergone deep learning using training data to determine whether each pixel (voxel) in a medical image represents an abnormal shadow.

[0034] The learning model is constructed by training a CNN using a large number of training data, including training images containing abnormal shadows and ground truth data representing the regions of abnormal shadows and the characteristics of the abnormal shadows in the training images, as well as training images containing no abnormal shadows. The learning model derives a confidence (likelihood) indicating that each pixel in a medical image is an abnormal shadow, detects regions consisting of pixels where the confidence is equal to or greater than a predetermined threshold as regions of abnormal shadows, and derives the characteristics of the detected abnormal shadows. Here, the confidence is a value between 0 and 1.

[0035] As the learning model, in addition to the convolutional neural network, any learning model such as a support vector machine (SVM) can be used.

[0036] In this embodiment, the medical image is a radiographic image of the lung as the diagnostic target, acquired by a plain X-ray imaging device, and abnormal shadows contained in the lung are detected by CAD in the analysis server 9, and the properties of the abnormal shadows are derived. The image is not limited to a radiographic image, and may be a CT image or an MRI image consisting of multiple cross-sectional images.

[0037] The network 10 is a wired or wireless local area network that connects various devices within the hospital. If the interpretation WS3 is installed in other hospitals or clinics, the network 10 may be configured by connecting the local area networks of each hospital via the Internet or a dedicated line.

[0038] Next, the medical image display device according to this embodiment will be described. Fig. 2 describes the hardware configuration of the medical image display device according to this embodiment. As shown in Fig. 2, the medical image display device 20 includes a CPU (Central Processing Unit) 11, non-volatile storage 13, and memory 16 as a temporary storage area. The medical image display device 20 also includes a display 14 such as a liquid crystal display, an input device 15 consisting of a keyboard, a pointing device such as a mouse, and the like, and a network I / F (Interface) 17 connected to a network 10. The CPU 11, storage 13, display 14, input device 15, and the like are all connected to the network 10. The processor 15, the memory 16, and the network I / F 17 are connected to a bus 18. The CPU 11 is an example of a processor in the present disclosure.

[0039] The storage 13 is realized by a hard disk drive (HDD), a solid state drive (SSD), a flash memory, etc. The storage 13 as a storage medium stores a medical image display program 12. The CPU 11 reads the medical image display program 12 from the storage 13, expands it in the memory 16, and executes the expanded medical image display program 12.

[0040] In this embodiment, when creating an interpretation report in the interpretation WS3, the radiologist instructs the analysis server 9 to analyze the medical image. The radiologist then displays the medical image on the display 14 and interprets the medical image with his or her own eyes. The radiologist then obtains the analysis results of the medical image from the analysis server 9, displays annotations representing the analysis results on the medical image, and performs a second interpretation by referring to the displayed annotations representing the analysis results. The first interpretation is referred to as the primary interpretation, and the second interpretation with reference to the displayed analysis results is referred to as the secondary interpretation.

[0041] Next, the functional configuration of the medical image display device according to this embodiment will be described. Fig. 3 is a diagram showing the functional configuration of the medical image display device according to this embodiment. As shown in Fig. 3, the medical image display device 20 includes an information acquisition unit 21, a display control unit 22, an input acceptance unit 23, an edit control unit 24, and a management unit 25. When the CPU 11 executes the medical image display program 12, the CPU 11 functions as the information acquisition unit 21, the display control unit 22, the input acceptance unit 23, the edit control unit 24, and the management unit 25.

[0042] The information acquisition unit 21 acquires the medical image G0 to be processed for creating an interpretation report from the image server 5 in response to an instruction from the radiologist, who is the operator, via the input device 15. Also, in response to an instruction from the input device 15, the information acquisition unit 21 instructs the analysis server 9 to analyze the medical image G0 to be processed, and acquires the analysis results from the analysis server 9.

[0043] The display control unit 22 controls various displays, including the display of the medical image G0 and the display of annotations representing the interpretation results and analysis results (described later). In addition, as described later, the display control unit 22 controls the display of the analysis results according to reference information.

[0044] As will be described later, after the annotation representing the analysis result is displayed, the editing control unit 24 prohibits editing of the primary interpretation result input by the radiologist in the primary interpretation before the display of the annotation representing the analysis result. Furthermore, when the annotation representing the analysis result is displayed, the editing control unit 24 notifies that editing of the primary interpretation result is prohibited. Furthermore, the editing control unit 24 associates the annotation representing the primary interpretation result and the annotation representing the secondary interpretation result input by the radiologist in the secondary interpretation after the display of the annotation representing the analysis result with the medical image G0 and saves them.

[0045] The management unit 25 associates the medical image with reference information indicating whether the analysis result for the medical image has been referred to by the user, and also associates the reference information on a user-by-user basis.

[0046] The processing performed in this embodiment will be described in detail below. Figures 4 and 5 are flowcharts showing the processing performed in this embodiment. It is assumed that the acquisition of the medical image G0 to be diagnosed and the instruction to the analysis server 9 to analyze it have been completed. In addition, in the flowcharts shown in Figures 4 and 5, processing using reference information, which will be described later, is not performed. The processing starts when an instruction to display the medical image G0 is given to create an interpretation report, and the display control unit 22 displays the medical image G0 on the display 14 (step ST1). As a result, the interpreting physician performs a primary interpretation of the displayed medical image G0.

[0047] FIG. 6 is a diagram showing a display screen for primary interpretation. As shown in FIG. 6, an image display area 41 and a text display area 42 are displayed on a display screen 40. A confirm button 43 for confirming the input content is also displayed. A medical image G0 is displayed in the image display area 41. As shown in FIG. 6, the medical image G0 is a radiological image of the chest of a human body. The medical image G0 includes two abnormal shadows A1 and A2. Furthermore, the text display area 42 displays the findings entered by the radiologist.

[0048] Next, the input receiving unit 23 determines whether or not the primary interpretation result of the medical image G0 has been input by the radiologist (step ST2). If the result of step ST2 is affirmative, the display control unit 22 displays an annotation representing the primary interpretation result on the medical image G0 (step ST3). If the result of step ST2 is negative, the process proceeds to step ST14.

[0049] FIG. 7 is a diagram showing a display screen on which annotations representing primary image interpretation results are displayed. As shown in FIG. 7, a circular mark 30 is attached to an abnormal shadow A1 identified by a radiologist in a medical image G0 on a display screen 40A as an annotation representing the primary image interpretation results. Although only one mark 30 is displayed in FIG. 7, if multiple abnormal shadows are found as a result of the interpretation, a mark is attached to each of them. The mark is not limited to a circle, but may be a polygonal shape, an arrow, or any other shape. Furthermore, the annotation representing the primary image interpretation results is not limited to a mark, but may be text describing the doctor's interpretation results. In the case of text, a display area for the text is displayed as an annotation adjacent to the abnormal shadow. For the sake of explanation, it is assumed here that an abnormal shadow A2 was not identified in the primary image interpretation.

[0050] Next, the input receiving unit 23 determines whether an instruction to display the analysis results has been input (step ST4), and if step ST4 is positive, the information acquiring unit 21 acquires the analysis results of the medical image G0 (step ST5). Note that the analysis results may be acquired while the processes of steps ST1 to ST4 are being performed, or may be acquired before the process of step ST1. On the other hand, if step ST4 is negative, the process proceeds to step ST14.

[0051] Next, the editing control unit 24 notifies the radiologist that editing of the primary interpretation results entered by the radiologist is prohibited (step ST6). FIG. 8 is a diagram showing a display screen on which the editing prohibition notice is displayed. As shown in FIG. 8, an editing prohibition notice 44 is displayed on the display screen 40B. The editing prohibition notice 44 displays text 45 saying, "After viewing the analysis results, it will be clearly stated that the interpretation was performed with reference to the analysis results, and editing of the interpretation results before viewing the analysis results will become impossible. Are you sure?" along with a YES button 46 and a NO button 47. If the NO button 47 is selected (step ST7; NO), the analysis results will not be displayed and will not be referenced by the radiologist (user), so the management unit 25 associates reference information indicating that the analysis results have not been referenced with the medical image G0 (associating reference information for analysis results not referenced; step ST8), and proceeds to the processing of step ST14, which will be described later. The reference information is included in the management information, which will be described later.

[0052] When the YES button 46 is selected (step ST7; YES), the editing control unit 24 prohibits editing of the primary interpretation results (step ST9). Specifically, an editing prohibition flag is set for the information representing the primary interpretation results so that the mark 30, which is the primary interpretation result, cannot be deleted from the medical image G0, the mark 30 cannot be changed, and the position of the mark 30 cannot be changed. Then, the display control unit 22 displays the acquired analysis results on the medical image G0 (step ST10). Furthermore, the management unit 25 associates reference information indicating that the analysis results have been referred to by the radiologist with the medical image G0 (association of reference information with analysis results referred to; step ST11). The processing of step ST11 may be performed before the processing of step ST10, or may be performed in parallel with the processing of step ST10.

[0053] 9 is a diagram showing a display screen on which annotations representing analysis results are displayed. As shown in Fig. 9, in addition to a circular mark 30 which is an annotation representing the primary interpretation result, a triangular mark 31 which is an annotation representing the analysis result is displayed on the medical image G0 displayed on the display screen 40C.

[0054] Here, the annotation representing the primary interpretation result is a circular mark 30, and the annotation representing the analysis result is a triangular mark 31, so the annotation representing the primary interpretation result and the annotation representing the analysis result are displayed separately. In addition to or instead of having different shapes, the marks 30 and 31 may also have different colors or different line types. Thereafter, the radiologist performs further interpretation of the medical image G0, i.e., secondary interpretation, by referring to the analysis results.

[0055] Note that text indicating the characteristics included in the analysis results may be displayed as annotations representing the analysis results. In this case, a text box is associated with each abnormal shadow detected by the analysis results, and the characteristics are described in the text box.

[0056] 10 is a diagram showing management information including reference information associated with a medical image. As shown in FIG. 10, the management information 50 includes an image interpretation physician ID 51 that identifies the image interpretation physician currently interpreting the medical image G0, a file name 52 of the medical image G0, reference information 53 indicating whether the analysis results have been referenced, and a reference date 54 if the analysis results have been referenced. If the analysis results have not been referenced, the information for the reference date 54 is left blank. The management information 50 is stored in the storage 13.

[0057] Next, the input receiving unit 23 determines whether or not the secondary interpretation result of the medical image G0 has been input by the radiologist with reference to the analysis result (step ST12). If the result of step ST12 is affirmative, the display control unit 22 displays an annotation representing the secondary interpretation result on the medical image G0 (step ST13). If the result of step ST12 is negative, the process proceeds to step ST14.

[0058] 11 is a diagram showing a display screen on which annotations representing secondary interpretation results are displayed. As shown in Fig. 11, a medical image G0 on a display screen 40D displays a circular mark 30, which is an annotation representing the primary interpretation result, a triangular mark 31, which is an annotation representing the analysis result, and a rectangular mark 32, which is an annotation representing the secondary interpretation result. Note that the rectangular mark 32 is assigned to an abnormal shadow A2 identified by the analysis result, and therefore the triangular mark 31 and the rectangular mark 32 are displayed overlapping each other.

[0059] Here, the annotation representing the primary interpretation result is a circular mark 30, and the annotation representing the secondary interpretation result is a rectangular mark 32, so that the annotation representing the primary interpretation result and the annotation representing the secondary interpretation result are displayed separately. In addition to or instead of having different shapes, the marks 30 and 32 may also have different colors or different line types.

[0060] Next, the editing control unit 24 accepts an input of a finding by the radiologist in the text display area 42 (step ST14). Then, the editing control unit 24 determines whether the confirm button 43 is selected (step ST15), and if step ST15 is negative, the process returns to step ST14. If step ST15 is positive, the editing control unit 24 associates the annotations representing the primary interpretation results and the annotations representing the secondary interpretation results in the medical image G0 with the medical image G0 and saves them in the storage 13 (step ST16). Note that if the processing in step ST2 is negative, the annotations representing the interpretation results are not associated with the medical image G0. If the processing in steps ST4, ST7, and ST12 is negative, only the annotations representing the primary interpretation results are associated with the medical image G0 and saved.

[0061] Furthermore, the editing control unit 24 generates an interpretation report containing the findings and sends it to the report server 7, and also sends the medical image G0 stored in the storage 13 and management information including the reference information to the image server 5 (step ST17), and then terminates the processing.

[0062] Here, "saving the annotations representing the primary interpretation results and the annotations representing the secondary interpretation results in association with the medical image G0" means saving the annotations representing the primary interpretation results and the annotations representing the secondary interpretation results as an inseparable part of the medical image G0. Specifically, the annotations can be superimposed on the medical image G0 or written into the tag information of the medical image G0. As a result, when the medical image G0 is later displayed, the annotations saved in association with the medical image G0 can be displayed together, thereby allowing the annotations representing the primary interpretation results and the annotations representing the secondary interpretation results to be displayed in the same manner as the display screen 40C shown in FIG. 9 or the display screen 40D shown in FIG. 11.

[0063] Meanwhile, the image server 5 stores management information 50 including reference information associated with the medical image G0. Here, when multiple radiology doctors interpret the same medical image G0, the image server 5 manages the reference information for each medical image on a physician-by-physician basis. For example, in the management information 50A shown in FIG. 12, when two radiology doctors interpret the medical image G0 with the file name IMG0001.jpg, the reference information is managed for each radiology doctor. In FIG. 12, the radiology doctor with the radiology doctor ID ABC1234 has already referred to the reference information 53, while the radiology doctor with the radiology doctor ID DEF6789 has not yet referred to the reference information 53.

[0064] Next, the processing using the reference information in this embodiment will be described. Fig. 13 is a flowchart showing the processing performed using the reference information in this embodiment. It is assumed that the acquisition of the medical image G0 to be diagnosed and management information including the reference information from the image server 5, and the acquisition of the analysis results from the analysis server 9 have been completed. The processing is started when an instruction to display the medical image G0 is given to create an interpretation report, and the management unit 25 determines whether the analysis results for the medical image G0 have been referenced for the radiologist who is about to perform the interpretation, based on the reference information associated with the medical image G0 (step ST21).

[0065] If step ST21 is positive, the display control unit 22 displays the analysis result in addition to the medical image G0 without accepting an instruction to display the analysis result (step ST22), and ends the process. On the other hand, if step ST22 is negative, the display control unit 22 displays only the medical image G0 (step ST23), and ends the process.

[0066] FIG. 14 is a diagram showing a display screen on which the analysis results are displayed. As shown in FIG. 14, a display screen 60 displays an image display area 61, a text display area 62, and a confirm button 63. A medical image G0 is displayed in the image display area 61. A mark 64, which is an annotation indicating the analysis results, is displayed on the medical image G0. In this case, the radiologist has already interpreted the medical image G0 once. Therefore, if the primary interpretation results are associated with the medical image G0, an annotation indicating the primary interpretation results may be displayed in addition to the annotation indicating the analysis results. In this case, the radiologist performs the interpretation while referring to the analysis results, if necessary, and creates an interpretation report, if necessary.

[0067] On the other hand, if step ST22 is negative and only the medical image G0 is displayed, the display control unit 22 displays the display screen 40 shown in Fig. 6. The radiologist can create an interpretation report by performing the same process as in the flowcharts shown in Figs. 4 and 5.

[0068] As described above, in this embodiment, based on the reference information indicating whether the analysis results have been referenced by the radiologist (user), when an instruction to display the medical image G0 is given, an annotation indicating the analysis results for the specified medical image G0 is displayed in addition to the specified medical image G0. Therefore, if the analysis results have been referenced, the next time the medical image G0 is displayed, an annotation indicating the analysis results is displayed in addition to the medical image G0 without receiving an instruction from the radiologist to display the analysis results. Therefore, according to this embodiment, the analysis results for a medical image can be displayed without receiving an instruction to display each time the analysis results are displayed or without displaying the medical image for more than a specified time.

[0069] In addition, in this embodiment, annotations representing the primary interpretation results before the analysis results are displayed and annotations representing the secondary interpretation results after the annotations representing the analysis results are displayed are displayed separately, so that it is possible to check how the CAD analysis results of the medical image have affected the doctor's interpretation results in the displayed medical image G0.

[0070] Furthermore, by prohibiting editing of the primary interpretation results after the annotations representing the analysis results are displayed, it is possible to prevent the primary interpretation results from being altered by the radiologist before the annotations representing the analysis results are displayed, which makes it possible to more accurately confirm how the CAD analysis results of medical images have affected the doctor's interpretation results.

[0071] Furthermore, by notifying the radiologist that editing of the primary interpretation results is prohibited before the annotations representing the analysis results are displayed, the radiologist can know in advance that editing of the primary interpretation results is prohibited. This allows the radiologist to be more careful when displaying the analysis results, and as a result, it is possible to encourage the radiologist to be more careful when interpreting the images before the analysis results are displayed.

[0072] Furthermore, by saving annotations representing the primary interpretation results and the secondary interpretation results in association with the medical image G0, when the medical image G0 is displayed later, it is possible to check how the CAD analysis results of the medical image affected the doctor's interpretation results.

[0073] In the above embodiment, reference information is managed on a per-user basis in the management information, but this is not limited to this. For example, medical images of a patient managed by multiple doctors are also interpreted by multiple doctors. Therefore, reference information may be managed on a group basis including multiple doctors. FIG. 15 is a diagram showing management information including reference information managed on a group basis. In management information 50B shown in FIG. 15, a group 55 consisting of four doctors is set for medical image G0 with the file name IMG0002.jpg. Furthermore, in management information 50B, reference information 53 of the analysis results and a reference date 54 if the medical image has been referenced are set for each doctor included in group 55.

[0074] Next, a process using reference information managed on a group-by-group basis will be described. Fig. 16 is a flowchart showing a process performed using reference information managed on a group-by-group basis. It is assumed that acquisition of management information including the medical image G0 to be diagnosed and reference information from the image server 5 and acquisition of analysis results from the analysis server 9 have been completed. The process starts when an instruction to display the medical image G0 is given to create an interpretation report, and the management unit 25 determines whether the analysis results have been referenced for all radiologists included in the group to which the radiologist attempting to perform the interpretation belongs, based on the reference information associated with the medical image G0 (step ST31).

[0075] If step ST31 is positive, the display control unit 22 displays the analysis results in addition to the medical image G0 without accepting an instruction to display the analysis results (step ST32), and ends the process. On the other hand, if step ST31 is negative because the analysis results have not been referred to for at least one radiological interpreter included in the group, the display control unit 22 displays only the medical image G0 (step ST33), even if the radiological interpreter who is about to perform the interpretation has referred to the analysis results for the medical image G0, and ends the process.

[0076] When managing reference information by group, there may be cases where a radiologist performs an interpretation while also referring to the analysis results, and then requests that the medical image G0 be interpreted by multiple doctors. In such cases, a group including multiple doctors is created, and the medical image G0 is registered in that group. During registration, the reference information associated with the medical image G0 for the radiologists who have referred to the analysis results and are included in the group may be changed to "not referred to."

[0077] In the above embodiment, when displaying the medical image G0, whether or not the analysis result has been referenced may be visually displayed. Fig. 17 is a diagram showing a display screen that visually displays whether or not the analysis result has been referenced. As shown in Fig. 17, a display screen 70 includes an examination list area 71, a thumbnail area 72, and an image display area 73.

[0078] The examination list area 71 displays the examination date, patient name, and reference information for each examination. The radiologist can select the patient to be examined by clicking on the desired examination. In FIG. 17, the row of the examination for which the patient name "Fuji Taro" is selected in the examination list area 71 is surrounded by a thick line. In addition, for the examinations displayed in the examination list area 71, it is visually indicated whether the analysis results have been referenced or not. In FIG. 17, it can be seen that the analysis results for Fuji Taro have not been referenced, but the analysis results for Fuji Hanako and Yamada Taro have been referenced.

[0079] The thumbnail area 72 displays a thumbnail image of a representative image of the medical image G0 of the selected patient. It is assumed here that multiple types of medical images, such as CT images, MRI images, and plain radiographic images, are acquired during a single examination. Therefore, thumbnail images for each of the multiple types of medical images are displayed in the thumbnail area 72. For comparative interpretation, thumbnail images of past images of the selected patient are also displayed in the thumbnail area 72. In FIG. 17 , thumbnail images C1 to C3 for three medical images G1 to G3 and thumbnail images C4 and C5 for two past images G4 and G5 are displayed. Here, if the medical image is a three-dimensional image, a reduced image of a tomographic image of a predetermined cross-sectional plane included in the three-dimensional image can be used as the representative image. If the medical image is a plain radiographic image, the reduced image can be used as the representative image. By selecting a thumbnail image displayed in the thumbnail area 72, the radiologist can display the medical image corresponding to the selected thumbnail image in the image display area 73.

[0080] It is also assumed that the analysis results for medical image G2 have been acquired by the analysis server 9. Here, the currently selected patient to be examined is Fujitaro, and the analysis results have not yet been referenced. Therefore, the thumbnail image C2 for medical image G2 displayed in the thumbnail area 72 is blacked out or grayed out to indicate that the analysis results have not yet been referenced. In FIG. 17, the thumbnail image C2 is shaded in black or grayed out to indicate that it is blacked out or grayed out.

[0081] The image display area 73 displays the image selected in the thumbnail area 72. As shown in Fig. 17, the image display area 73 includes four display areas 73A to 73D. Therefore, a maximum of four medical images can be displayed in the image display area 73.

[0082] As shown in Figure 17, when displaying a medical image, by visually displaying whether or not the analysis results have been referenced, the radiologist can check whether or not the analysis results have been referenced for the medical image of the patient he or she is about to interpret before displaying it. Here, the analysis results affect the interpretation results of the radiologist. Therefore, by visually displaying whether or not the analysis results have been referenced, it is possible to prevent the erroneous display of the analysis results of a medical image G0 for which the analysis results have not been referenced.

[0083] In the above embodiment, if the analysis results have already been referred to, a setting value indicating whether or not to display the analysis results when the radiologist next displays the medical image G0 may be managed even if the radiologist does not issue an instruction to display the analysis results. This will be described below as another embodiment.

[0084] Fig. 18 is a diagram showing a display screen on which annotations representing analysis results are displayed in another embodiment. Display screen 40E shown in Fig. 18 displays a setting area 80 in addition to display screen 40C shown in Fig. 9. In setting area 80, text 81 saying "Do you want to display analysis results on the next display?", a YES button 82, and a NO button 83 are displayed.

[0085] When the radiologist selects the YES button 82 in the setting area 80, the management unit 25 sets the setting value for whether or not to display the analysis results for the currently displayed medical image G0 the next time the radiologist displays the medical image G0 to "display" even if there is no instruction to display the analysis results. When the radiologist selects the NO button 83, the management unit 25 sets the setting value for whether or not to display the analysis results for the currently displayed medical image G0 the next time the radiologist displays the medical image G0 to "do not display." The setting value is managed together with the reference information.

[0086] Fig. 19 is a diagram showing management information including reference information associated with a medical image in another embodiment. In addition to the management information 50 shown in Fig. 10, the management information 50C shown in Fig. 19 includes a setting value 56 indicating whether or not the analysis results will be displayed the next time the radiologist displays the medical image G0, even if no instruction to display the analysis results is given. The setting value for the management information 50C shown in Fig. 19 is "Do not display." Therefore, even if the analysis results have already been referenced for that medical image G0, the analysis results will not be displayed the next time the medical image G0 is displayed.

[0087] Next, processing using reference information in another embodiment will be described. Fig. 20 is a flowchart showing processing performed using reference information in another embodiment. It is assumed that acquisition of management information including the medical image G0 to be diagnosed and reference information from the image server 5 and acquisition of analysis results from the analysis server 9 have been completed. Processing begins when an instruction to display the medical image G0 is given to create an interpretation report, and the management unit 25 determines whether the analysis results for the medical image G0 have been referenced for the radiologist who is about to perform the interpretation, based on the reference information associated with the medical image G0 (step ST41).

[0088] If step ST41 is positive, the display control unit 22 determines whether the setting value for whether or not to display the analysis results when the medical image G0 is next displayed for the radiologist who is about to perform the radiological interpretation is "display" even if there is no instruction to display the analysis results (step ST42). If step ST42 is positive, the analysis results are displayed in addition to the medical image G0 without receiving an instruction to display the analysis results (step ST43), and the process ends. On the other hand, if steps ST41 and ST42 are negative, the display control unit 22 displays only the medical image G0 (step ST44), and the process ends.

[0089] In the above embodiment, the reference information is managed in association with the medical image, but the present invention is not limited to this. The reference information may be managed in association with the analysis result.

[0090] In the above embodiment, the medical image G0 is analyzed in the analysis server 9, but this is not limiting. The medical image G0 may be analyzed in the image interpretation WS3.

[0091] In addition, in the above-described embodiments, the technology of the present disclosure is applied to the case where an interpretation report is created using medical images in which the diagnostic target is the lungs or the liver, but the diagnostic target is not limited to the lungs or the liver. In addition to the lungs, any part of the human body, such as the heart, brain, kidneys, and limbs, can be the diagnostic target.

[0092] Furthermore, in each of the above embodiments, the following various processors can be used as the hardware structure of a processing unit that executes various processes, such as the information acquisition unit 21, the display control unit 22, the input acceptance unit 23, the editing control unit 24, and the management unit 25. As described above, the various processors include a CPU, which is a general-purpose processor that executes software (programs) and functions as various processing units, as well as dedicated electrical circuits that are processors having a circuit configuration specifically designed to execute specific processes, such as a programmable logic device (PLD), which is a processor whose circuit configuration can be changed after manufacture, such as an FPGA (Field Programmable Gate Array), and an ASIC (Application Specific Integrated Circuit).

[0093] A single processing unit may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs or a combination of a CPU and an FPGA). Also, multiple processing units may be configured with a single processor.

[0094] Examples of configuring multiple processing units with a single processor include, first, a form in which one processor is configured with a combination of one or more CPUs and software, and this processor functions as multiple processing units, as typified by computers such as client and server. Second, a form in which a processor is used to realize the functions of an entire system including multiple processing units with a single IC (Integrated Circuit) chip, as typified by systems on chips (SoCs). In this way, various processing units are configured using one or more of the above-mentioned various processors as a hardware structure.

[0095] Furthermore, more specifically, the hardware structure of these various processors can be an electric circuit that combines circuit elements such as semiconductor elements. [Explanation of symbols]

[0096] 1 Medical Information System 2. Imaging equipment 3 Image Reading Workshop 4. Medical Workshop 5 Image Server 6. Image Database 7 Report Server 8 Report DB 9 Analysis Server 10 Network 11 CPU 12 Medical image display program 13. Storage 14 Display 15 Input Devices 16 memory 17 Network I / F 18 Bus 20 Medical image display device 21 Information Acquisition Department 22 Display control unit 23 Input reception section 24 Editing control section 25 Management Department 30, 31, 32, 33, 64 marks 40, 40A~40E, 60, 70 display screen 41,61,73 Image display area 42,62 Text display area 43,63 Confirm button 50,50A,50B,50C Management information 51 Image Interpreter ID 52 File Name 53 References 54 Reference date 55 Group 56 Setting value 71 Examination List Area 72 Thumbnail Area 73A~73D Display area 80 Setting Area 81 Text 82 YES button 83 NO button C1~C5 thumbnail images G0 Medical Imaging

Claims

1. at least one processor; The processor: Displaying medical images, obtaining an analysis result for the medical image; A medical image display device that, based on reference information indicating whether the analysis result has been referenced by the user, displays an annotation indicating the analysis result in addition to the specified medical image when an instruction to display the medical image is given, if the analysis result has been referenced.

2. The medical image display device according to claim 1 , wherein the reference information is managed on a per-user basis.

3. The medical image display device according to claim 2 , wherein the reference information is managed in units of a group including a plurality of users.

4. The medical image display device of claim 3, wherein when a user included in the group instructs the display of a medical image for which the analysis results have been referenced for all users included in the group, the processor displays an annotation representing the analysis results in addition to the instructed medical image, and when a user included in the group instructs the display of a medical image for which the analysis results have not been referenced for at least one user included in the group, the processor displays only the instructed medical image.

5. The medical image display device according to claim 1 , wherein the processor visually indicates whether or not the analysis result has been referenced for the medical image to be displayed.

6. The processor manages a setting value indicating whether or not the analysis result already referred to is to be displayed when the user next displays the medical image, even if the user does not issue an instruction to display the analysis result; If the setting value indicates that the analysis result is to be displayed, when the user next displays the medical image, an annotation representing the analysis result is displayed in addition to the medical image; A medical image display device according to any one of claims 1 to 5, wherein if the setting value is set to not display the analysis results, the next time the user displays the medical image, only the medical image is displayed.

7. The medical image display device according to claim 6 , wherein the setting value is managed on a per-user basis.

8. the processor accepts input of a primary interpretation result obtained by a user interpreting the medical image; displaying annotations representing the primary interpretation results; obtaining an analysis result for the medical image; displaying annotations representing the analysis results; receiving input of a secondary interpretation result obtained by the user interpreting the medical image after the analysis result is displayed; The medical image display device according to claim 1 , wherein annotations representing the primary interpretation results and annotations representing the secondary interpretation results are displayed in a distinguishable manner.

9. Displaying medical images, obtaining an analysis result for the medical image; A medical image display method in which, based on reference information indicating whether the analysis result has been referenced by a user, when an instruction to display the medical image is given, an annotation indicating the analysis result is displayed in addition to the specified medical image if the analysis result has been referenced.

10. a procedure for displaying medical images; obtaining an analysis result for the medical image; A medical image display program that causes a computer to execute a procedure that, based on reference information indicating whether the analysis result has been referenced by a user, when an instruction to display the medical image is given, displays an annotation representing the analysis result in addition to the specified medical image, if the analysis result has been referenced.

Images