Image display apparatus, method, and program

Abstract

To provide an image display device capable of displaying only a necessary annotation without displaying an unnecessary annotation even when reading a medical image to which a plurality of annotations are imparted.SOLUTION: The processor performs control to specify a region of interest based on an observation condition for specifying the region of interest of a user in a medical image and display only related information concerning the specified region of interest.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present disclosure relates to an image display device, a method, and a program. [Background technology]

[0002] In recent years, advances in medical equipment such as CT (Computed Tomography) and MRI (Magnetic Resonance Imaging) have made it possible to perform diagnostic imaging using higher quality, higher resolution medical images. In particular, diagnostic imaging using CT and MRI images allows for accurate identification of lesion areas, leading to the provision of appropriate treatment based on the results of such identification.

[0003] Medical images are also analyzed using computer-aided diagnosis (CAD) software, which uses a learning model based on machine learning techniques such as deep learning, to detect disease regions, such as lesions, contained in the medical image as regions of interest. The analysis results generated by the CAD analysis process are associated with examination information, such as the patient's name, gender, age, and the modality used to acquire the medical image, and stored in a database for use in diagnosis. Doctors interpret the medical images using their own image-reading terminals, referring to the delivered medical images and analysis results. Based on the analysis results, annotations are displayed on the image-reading terminals, superimposed on the regions of interest containing diseases. Annotations are a collective term for figures, characters, and symbols added to medical images. For example, annotations include rectangles enclosing regions of interest, arrows indicating the regions of interest, and text indicating the type and size of the disease. The radiologist then creates an interpretation report by referring to the annotations added to the regions of interest.

[0004] When a medical image contains multiple lesions, annotations are assigned to each of the lesions. Therefore, when a medical image with annotations is displayed, the multiple annotations are superimposed on the medical image. When multiple annotations are superimposed on the medical image, it becomes difficult to clearly identify the location of the lesion indicated by the displayed annotations. For this reason, a method has been proposed in which multiple annotations are grouped and associated with medical images on a group-by-group basis, thereby clarifying the association between the multiple annotations and the medical image (see, for example, Japanese Patent Application Laid-Open No. 2013-132514). The method described in Japanese Patent Application Laid-Open No. 2013-132514 also proposes a method of further associating a character string of findings included in a radiology report with the medical image and the annotations. Summary of the Invention [Problem to be solved by the invention]

[0005] On the other hand, when interpreting a medical image with multiple annotations, annotations other than those assigned to the region of interest in the medical image can be a nuisance to the radiologist. Using the method described in JP 2013-132514 A, it is possible to display only annotations included in a specific group, but these may include annotations that are unnecessary for interpretation. In such cases, by clicking on the unnecessary annotations or on the region to which the unnecessary annotations are assigned, the unnecessary annotations can be hidden and only the necessary annotations can be displayed. However, because radiologists interpret a large number of medical images every day, such operations can be very cumbersome for them.

[0006] The present disclosure has been made in consideration of the above circumstances, and aims to enable radiologists to refer to only annotations for regions of interest without having to perform cumbersome operations. [Means for solving the problem]

[0007] The image display device according to the present disclosure includes at least one processor, which identifies an area of ​​interest that a user focuses on within a medical image based on observation conditions for identifying the area of ​​interest, and controls the display of only relevant information related to the identified area of ​​interest.

[0008] In addition, in the image display device according to the present disclosure, the processor may identify an organ to be observed from among a plurality of regions of interest contained in the medical image based on the observation conditions, and identify, from among a plurality of regions of interest contained in the identified organ, a region of interest that corresponds to the observation conditions as the region of interest.

[0009] In the image display device according to the present disclosure, the viewing condition may be a window level or a window width.

[0010] In the image display device according to the present disclosure, the processor may perform control to hide or weakly display information related to regions of interest other than the region of interest.

[0011] In the image display device according to the present disclosure, the related information may include an annotation related to the region of interest.

[0012] In the image display device according to the present disclosure, the annotation may include at least one of a recognition result of the region of interest and text information related to the region of interest.

[0013] The image display method according to the present disclosure involves a computer executing a process to identify an area of ​​interest that a user focuses on within a medical image based on observation conditions for identifying the area of ​​interest, and to control the display of only relevant information related to the identified area of ​​interest.

[0014] The image display program according to the present disclosure causes a computer to execute a process of identifying an area of ​​interest that a user focuses on in a medical image based on observation conditions for identifying the area of ​​interest, and controlling the display of only relevant information related to the identified area of ​​interest. [Effects of the Invention]

[0015] According to the present disclosure, radiologists can refer to only annotations for regions of interest without having to perform cumbersome operations. [Brief explanation of the drawings]

[0016] [Figure 1] FIG. 1 is a diagram showing a schematic configuration of a medical information system to which an image display device according to an embodiment of the present disclosure is applied; [Figure 2] 1 is a diagram showing a schematic configuration of an image display device according to an embodiment of the present invention; [Figure 3] Functional configuration diagram of an image display device according to this embodiment [Figure 4] A diagram showing the relationship between the target image and annotations. [Figure 5] Annotations added to the tomographic images shown in Figure 4. [Figure 6] Diagram showing the display screen [Figure 7] Figure showing the property input window [Figure 8] Figure showing the observations input window [Figure 9] A diagram showing the display screen with annotations displayed. [Figure 10] A diagram showing the display screen with annotations displayed. [Figure 11] A diagram showing the display screen with annotations displayed. [Figure 12] Diagram for explaining the identification of an area of ​​interest [Figure 13] A diagram showing an enlarged target image. [Figure 14] Figure showing a cross-sectional image with annotations [Figure 15] Figure showing a cross-sectional image with annotations [Figure 16] Figure showing a cross-sectional image with annotations [Figure 17] Figure showing a cross-sectional image with annotations [Figure 18] A flowchart showing the processing performed in this embodiment [Figure 19] A diagram showing a tomographic image in which some annotations are weakly displayed. [Figure 20] A diagram showing a cross-sectional image in which some annotations are hidden. [Figure 21] A diagram showing a cross-sectional image in which some annotations are hidden. [Figure 22] A diagram showing a cross-sectional image in which some annotations are hidden. [Figure 23] FIG. 10 shows a state in which a mask representing the region extraction result is superimposed on an image as related information. [Figure 24] Diagram showing adjacent regions of interest [Figure 25] A diagram showing a cross-sectional image in which some annotations are hidden. DETAILED DESCRIPTION OF THE INVENTION

[0017] Hereinafter, an embodiment of the present disclosure will be described with reference to the drawings. First, the configuration of a medical information system 1 to which an image display device according to this embodiment is applied will be described. FIG. 1 is a diagram showing a schematic configuration of the medical information system 1. The medical information system 1 shown in FIG. 1 is a system for capturing an examination target region of a subject, storing the medical images acquired by capturing the images, having a radiologist interpret the medical images and create an interpretation report, and allowing the requesting doctor from the department to view the interpretation report and observe the details of the medical image to be interpreted, based on an examination order from a doctor from a department using a known ordering system.

[0018] As shown in Figure 1, the medical information system 1 is composed of multiple imaging devices 2, multiple interpretation WSs (Workstations) 3 which are interpretation terminals, a medical treatment WS 4, an image server 5, an image database (hereinafter referred to as image DB (DataBase)) 6, a report server 7, and a report database (hereinafter referred to as report DB) 8, all of which are connected in a state where they can communicate with each other via a wired or wireless network 10.

[0019] Each device is a computer installed with an application program that causes the device to function as a component of the medical information system 1. The application program is stored in an externally accessible state in a storage device of a server computer connected to the network 10 or in network storage, and is downloaded and installed into the computer upon request. Alternatively, the application program is recorded on a recording medium such as a DVD (Digital Versatile Disc) or CD-ROM (Compact Disc Read Only Memory) and distributed, and then installed into the computer from the recording medium.

[0020] The imaging device 2 is a device (modality) that captures an image of a diagnostic target part of a subject, thereby generating a medical image representing the diagnostic target part. Specifically, it is a plain X-ray imaging device, a CT device, an MRI device, a PET (Positron Emission Tomography) device, etc. The medical image generated by the imaging device 2 is transmitted to the image server 5 and stored in the image DB 6.

[0021] The image interpretation WS3 is a computer used by, for example, a radiologist to interpret medical images and create image interpretation reports, and includes the image display device according to this embodiment. The image interpretation WS3 performs tasks such as sending requests to the image server 5 to view medical images, performing various image processing on medical images received from the image server 5, displaying the medical images, and accepting input of findings related to the medical images. The image interpretation WS3 also performs tasks such as creating image interpretation reports, sending requests to the report server 7 to register and view image interpretation reports, and displaying image interpretation reports received from the report server 7. These processes are performed by the image interpretation WS3 executing software programs for each process.

[0022] The medical treatment WS4 is a computer used by doctors in the medical department to observe images in detail, view radiology reports, and create electronic medical records, and is composed of a processing device, a display device such as a monitor, and input devices such as a keyboard and a mouse. The medical treatment WS4 sends image viewing requests to the image server 5, displays images received from the image server 5, requests radiology reports to the report server 7, and displays radiology reports received from the report server 7. These processes are performed by the medical treatment WS4 executing software programs for each process.

[0023] The image server 5 is a general-purpose computer installed with a software program that provides the functions of a database management system (DBMS). The image server 5 also has storage in which the image DB 6 is configured. This storage may be a hard disk device connected to the image server 5 via a data bus, or a disk device connected to a NAS (Network Attached Storage) or SAN (Storage Area Network) connected to the network 10. When the image server 5 receives a request to register a medical image from the imaging device 2, it formats the medical image in a database format and registers it in the image DB 6.

[0024] The image DB 6 stores image data and additional information of medical images acquired by the imaging device 2. The additional information includes, for example, an image ID (identification) for identifying each medical image, a patient ID for identifying the subject, an examination ID for identifying the examination, a unique ID (UID: unique identification) assigned to each medical image, the examination date and time when the medical image was generated, patient information such as the type of imaging device used in the examination to acquire the medical image, the patient's name, age, and sex, the examination site (imaging site), imaging information (imaging protocol, imaging sequence, imaging method, imaging conditions, use of contrast agent, etc.), and a series number or collection number when multiple medical images are acquired in one examination.

[0025] In addition, when the image server 5 receives a viewing request from the image interpretation WS3 and medical treatment WS4 via the network 10, it searches for medical images registered in the image DB6 and transmits the searched medical images to the image interpretation WS3 and medical treatment WS4 that made the request.

[0026] A software program that provides a general-purpose computer with the functions of a database management system is installed in the report server 7. When the report server 7 receives a request to register an interpretation report from the interpretation WS 3, it converts the interpretation report into a database format and registers it in the report DB 8.

[0027] The report DB 8 registers image interpretation reports created by image interpretation physicians using the image interpretation WS 3. The image interpretation report may include, for example, the medical image to be interpreted, an image ID for identifying the medical image, an image interpretation physician ID for identifying the image interpretation physician who performed the image interpretation, the disease name, location information of the disease, and information for accessing the medical image.

[0028] In addition, when the report server 7 receives a request to view an interpretation report from the interpretation WS3 and medical treatment WS4 via the network 10, it searches for the interpretation report registered in the report DB8 and sends the retrieved interpretation report to the interpretation WS3 and medical treatment WS4 that made the request.

[0029] In this embodiment, the diagnostic target is the chest and abdomen of a human body, and the medical image is a three-dimensional CT image consisting of multiple cross-sectional images including the chest and abdomen. Then, in the image interpretation WS3, a radiologist interprets the CT image to create an interpretation report including findings about diseases of the lungs, liver, and other organs included in the chest and abdomen. The medical image is not limited to a CT image, and any medical image, such as an MRI image or a plain X-ray image obtained by a plain X-ray imaging device, can be used.

[0030] In this embodiment, when creating an interpretation report, the medical image is analyzed to detect a region of interest contained in the medical image. The doctor interprets the image while referring to the detection result. The detection of the region of interest will be described later.

[0031] The network 10 is a wired or wireless local area network that connects various devices within the hospital. If the interpretation WS3 is installed in other hospitals or clinics, the network 10 may be configured by connecting the local area networks of each hospital via the Internet or a dedicated line.

[0032] Next, an image display device according to this embodiment will be described. Fig. 2 illustrates the hardware configuration of the image display device according to this embodiment. As shown in Fig. 2, the image display device 20 includes a CPU (Central Processing Unit) 11, non-volatile storage 13, and memory 16 as a temporary storage area. The image display device 20 also includes a display 14 such as a liquid crystal display, an input device 15 including a keyboard, a pointing device such as a mouse, and a network I / F (Interface) 17 connected to a network 10. The CPU 11, storage 13, display 14, input device 15, memory 16, and network I / F 17 are connected to a bus 18. The CPU 11 is an example of a processor in the present disclosure.

[0033] The storage 13 is realized by a hard disk drive (HDD), a solid state drive (SSD), a flash memory, etc. The storage 13 as a storage medium stores an image display program 12. The CPU 11 reads the image display program 12 from the storage 13, loads it into the memory 16, and executes the loaded image display program 12.

[0034] Next, the functional configuration of the image display device according to this embodiment will be described. Fig. 3 is a diagram showing the functional configuration of the image display device according to this embodiment. As shown in Fig. 3, the image display device 20 includes an image acquisition unit 21, an analysis unit 22, a display control unit 23, and an identification unit 24. When the CPU 11 executes the image display program 12, the CPU 11 functions as the image acquisition unit 21, the analysis unit 22, the display control unit 23, and the identification unit 24.

[0035] The image acquisition unit 21 acquires medical images to be interpreted for creating an interpretation report from the image server 5 in response to instructions from the input device 15 by the radiologist, who is the operator. The medical image to be interpreted will be referred to as the target image G0 in the following description. Note that multiple images may be acquired in one examination. Since all images acquired in one examination are interpreted, there may be multiple target images G0. Furthermore, multiple images may have been acquired in the past for the patient under examination for whom the target image G0 is to be interpreted and stored in the image server 5. In this embodiment, when interpreting the target image G0 of the patient under examination, all images acquired by photographing the patient under examination, including the target image G0, are acquired from the image server 5. In the following description, of all images of the patient acquired from the image server 5, images other than the target image G0 will be referred to as past images GP0 because they were acquired earlier than the target image G0.

[0036] In this embodiment, the target image G0 is a three-dimensional CT image consisting of multiple tomographic images acquired by imaging the chest and abdomen of a patient to be examined. When multiple target images G0 are acquired in a single examination, each target image G0 may include, in addition to a CT image, an X-ray image acquired by simple X-ray photography, etc. Furthermore, the past image GP0 may also include, in addition to a CT image, an X-ray image acquired by simple X-ray photography, etc.

[0037] The analysis unit 22 detects regions of abnormal shadows included in the target image G0 as regions of interest and derives annotations for the detected regions of interest. The analysis unit 22 uses a known computer-aided image diagnosis (i.e., CAD) algorithm to detect regions of shadows of multiple types of diseases from the target image G0 as regions of interest and derives the properties of the regions of interest, thereby deriving the annotations.

[0038] To detect regions of interest and derive annotations, the analysis unit 22 has a learning model 22A that has been machine-learned to detect abnormal shadows of multiple types of diseases as regions of interest from the target image G0 and further derive the properties of the abnormal shadows.

[0039] The learning model 22A consists of a convolutional neural network (CNN) that has undergone deep learning using training data to determine whether each pixel (voxel) in the target image G0 represents a shadow of various diseases or an abnormal shadow.

[0040] The learning model 22A is constructed by training a CNN using, for example, training data consisting of training images containing abnormal shadows and ground truth data representing the region of the abnormal shadow in the training images and the characteristics of the abnormal shadow, as well as training data consisting of training images not containing abnormal shadows. The learning model 22A derives a degree of certainty (likelihood) that indicates that each pixel in a medical image is an abnormal shadow, and detects, as a region of interest, a region consisting of pixels whose certainty is equal to or greater than a predetermined threshold. Here, the degree of certainty is a value between 0 and 1. The learning model 22A also derives characteristics of the detected region of interest. The characteristics include the position, size, type of disease, etc. of the region of interest.

[0041] The learning model 22A may detect a region of interest from the three-dimensional target image G0, or may detect a region of interest from each of the multiple tomographic images that make up the target image G0. In addition to a convolutional neural network, any learning model such as a support vector machine (SVM) can be used as the learning model 22A.

[0042] The analysis unit 22 derives annotations such as a figure surrounding the region of interest detected by the learning model, text representing the characteristics, and lines connecting the figure and the text. The annotations are derived as images in a channel separate from the target image G0 and are assigned to the target image G0. Therefore, if the image in the annotation channel is displayed superimposed on the target image G0, the annotations will be displayed superimposed on the target image G0.

[0043] FIG. 4 is a diagram schematically illustrating the relationship between a target image G0 and annotations. As shown in FIG. 4, the target image G0 includes a plurality of tomographic images Dj. Furthermore, it is assumed that an abnormal shadow is detected as a region of interest in each of tomographic image groups GA1, GA2, and GA3 among the plurality of tomographic images Dj, and annotations are added to the detected regions of interest. In this embodiment, the tomographic images Dj represent axial cross sections of a human body, but are not limited to this.

[0044] FIG. 5 is a diagram showing annotations added to the tomographic image groups GA1 to GA3 shown in FIG. 4. As shown in FIG. 5, a region of interest R1 is detected in a tomographic image DA1 included in the tomographic image group GA1, and annotation 31 is added to the region of interest R1. Annotation 31 is text information related to the region of interest R1, and its content is "10 mm of SOL is observed in S1." SOL is an abbreviation for Space Occupying Lesion. Note that annotation 31 is added to all tomographic images included in the tomographic image group GA1 in which the region of interest R1 is detected.

[0045] Furthermore, in a tomographic image DA2 included in the tomographic image group GA2, a region of interest R2 is detected in the left lung, and a region of interest R3 is detected in the right lung. Furthermore, in the tomographic image DA2, an annotation 32 for the region of interest R2 in the left lung and an annotation 33 for the region of interest R3 in the right lung are respectively added. The annotation 32 consists of a graphic 32A surrounding the region of interest in the left lung included in the tomographic image DA2, text 32B describing the characteristics of the region of interest surrounded by the graphic 32A, and a leader line 32C connecting the graphic 32A and the text 32B. The annotation 33 consists of a graphic 33A surrounding the region of interest in the right lung included in the tomographic image DA2, text 33B describing the characteristics of the region of interest surrounded by the graphic 33A, and a leader line 33C connecting the graphic 33A and the text 33B. The text 32B and 33B relating to the region of interest are, for example, the position, size, and characteristics of the region of interest. The annotations 32 and 33 are added to all the tomographic images included in the tomographic image group GA2 in which the regions of interest R2 and R3 are detected, respectively.

[0046] Furthermore, in a tomographic image DA3 included in the tomographic image group GA3, a region of interest R4 included in the liver has been detected, and an annotation 34 has been added to the liver region of interest R4. The annotation 34 consists of a graphic 34A surrounding the liver region of interest included in the tomographic image DA3, text 34B indicating the characteristics of the region of interest surrounded by graphic 34A, and a leader line 34C connecting graphic 34A and text 34B. The annotation 34 is added to all tomographic images included in the tomographic image group GA3 in which the region of interest R4 has been detected.

[0047] The display control unit 23 displays images of the patient being examined, including the target image G0 and the past image GP0, on the display 14. Fig. 6 is a diagram showing the image display screen. As shown in Fig. 6, the display screen 40 includes an examination list area 41 that displays a list of examinations, a thumbnail image area 42 that displays thumbnail images of the target image G0 included in the selected examination and past images of the patient being examined, and an image display area 43 that displays the images.

[0048] The examination list area 41 displays the patient name, patient ID, and examination date and time for each examination. The operator can select the patient to be examined by clicking on the desired examination list. In Figure 6, the row of the patient to be examined selected in the examination list area 41 is shaded.

[0049] The thumbnail image area 42 displays thumbnail images of representative images of the target image G0 and past image GP0 of the selected patient under examination. Here, assuming that two target images G1 and G2 and four past images GP1 to GP4 have been acquired for the patient under examination, six thumbnail images C1 to C6 are displayed in the thumbnail image area 42, as shown in FIG. 6. If the image is a three-dimensional image, a reduced image of a tomographic image of a predetermined tomographic plane included in the three-dimensional image can be used as the representative image. If the image is a plain X-ray image, a reduced image of the tomographic image can be used as the representative image. In this embodiment, thumbnail images C1 and C2 are thumbnail images of the target images G1 and G2, and thumbnail images C3 to C6 are thumbnail images of the past images GP1 to GP4. It is assumed that the target image G1 corresponds to the target image G0 shown in FIG. 4, and that abnormal shadows have been detected as regions of interest in a group of tomographic images GA1, GA2, and GA3 among the multiple tomographic images Dj included in the target image G1, and that annotations have been added to the detected regions of interest.

[0050] The image display area 43 displays an image selected in the thumbnail image area 42. As shown in FIG. 6, the image display area 43 includes four display areas 43A to 43D. In FIG. 6, four representative images selected from six thumbnail images C1 to C6 are displayed in the display areas 43A to 43D of the image display area 43, respectively. In the thumbnail image area 42, thumbnail images other than the image displayed in the image display area 43 are shaded to indicate that they are not displayed in the image display area 43. In this embodiment, since thumbnail images C1 and C2 correspond to target images G1 and G2, respectively, the target images G1 and G2 are displayed in the display areas 43A and 43B, respectively. In addition, it is assumed that past images GP1 and GP2 are displayed in the display areas 43C and 43D.

[0051] Although annotations are added to the target image G1 as shown in Figure 4, the annotations are not superimposed on the representative image of the target image G1 displayed on the initial display screen 40 immediately after it is displayed on the display 14.

[0052] In this embodiment, the radiologist interprets the target image G1 displayed on the display screen 40 to confirm the region of interest detected by the analysis unit 22 and identify any abnormal shadows that the analysis unit 22 was unable to detect. At this time, the radiologist can perform a paging operation to switch the slice plane of the target image G1 displayed in the display area 43A by moving the mouse cursor 45 to the display area 43A and rotating the mouse wheel. The radiologist can also perform a predetermined resizing operation using the input device 15 to change the size of the displayed target image G1. An example of a resizing operation is to move the mouse cursor 45 to a reference position in the target image G1 where the resizing is desired, press the ctrl key, and rotate the mouse wheel. In this case, the size of the displayed target image G1 is changed around the cursor position. The paging operation and the resizing operation are examples of other operations related to the display of medical images.

[0053] If necessary, the radiologist can also add tag information indicating the characteristics of a newly found abnormal shadow to the target image or input a comment about the target image G1. The operations of adding tag information and inputting a comment are examples of other operations related to the interpretation of medical images.

[0054] The operation of adding tag information representing characteristics to a target image is, for example, to identify a newly found abnormal shadow with the mouse cursor 45 and right-click the identified abnormal shadow to pop up a characteristic input window 50 shown in FIG. 7 . The characteristic input window 50 displays, for example, the following items as characteristics of an abnormal lung shadow: boundary, shape, sawtooth, spicule, lobulation, straight line, absorption value, bronchial radiogram, cavity, calcification, fat, pleural indentation, and pleural contact. Negative (-) and positive (+) can be selected for the boundary, sawtooth, spicule, lobulation, straight line, bronchial radiogram, cavity, calcification, fat, pleural indentation, and pleural contact items. Irregular and regular shapes can be selected for the shape item. Solid and ground-glass shapes can be selected for the absorption value. The operation of inputting a finding is performed by right-clicking the mouse on the displayed target image, thereby popping up a finding input window 51 shown in FIG.

[0055] The identification unit 24 identifies a region of interest that the radiologist focuses on among multiple regions of interest included in the target image. In this embodiment, the identification unit 24 identifies at least one region of interest based on identification information including information other than the information dedicated to identifying the region of interest. In this embodiment, the detection result of another operation related to the display or interpretation of the medical image is used as the other information. The identification unit 24 identifies the region of interest based on the identification information including the detection result of the other operation as the other information. Note that, an example of the dedicated information for identifying the region of interest is the detection result of an operation of clicking the region of interest using a mouse.

[0056] For example, if a radiologist finds an abnormal shadow in a tomographic image while interpreting the target image G1, he or she performs a paging operation to repeatedly interpret the tomographic images before and after the tomographic image containing the abnormal shadow. The paging operation for repeated interpretation refers to repeatedly switching between tomographic planes by rotating the mouse wheel back and forth along the body axis. When the identification unit 24 detects such a paging operation, if the tomographic images repeatedly displayed during the paging operation contain annotations, the identification unit 24 identifies the region of interest to which the annotations are added as the region of interest that the radiologist focuses on.

[0057] For example, when a paging operation is performed near the tomographic image group GA1 of the target image G1 shown in Fig. 4, the identification unit 24 identifies the region of interest R1 included in the tomographic image DA1 shown in Fig. 5 as the region of interest. Furthermore, when a paging operation is performed near the tomographic image group GA2 in the target image G1 shown in Fig. 4, the identification unit 24 identifies the region of interest R2 and the region of interest R3 included in the tomographic image DA2 shown in Fig. 5 as the region of interest. Furthermore, when a paging operation is performed near the tomographic image group GA3 in the target image G1 shown in Fig. 4, the identification unit 24 identifies the region of interest R4 included in the tomographic image DA3 shown in Fig. 5 as the region of interest.

[0058] The display control unit 23 displays, on the display screen 40, annotations assigned to the region of interest identified by the identification unit 24. That is, when a paging operation is performed near the tomographic image group GA1 in the target image G1 displayed in the display area 43A, the display control unit 23 displays an annotation 31 superimposed on the target image G1 (tomographic image DA1) as shown in Fig. 9. When a paging operation is performed near the tomographic image group GA2 in the target image G1, the display control unit 23 displays an annotation 32 and 33 superimposed on the target image G1 (tomographic image DA2) as shown in Fig. 10. When a paging operation is performed near the tomographic image group GA3 in the target image G1, the display control unit 23 displays an annotation 34 superimposed on the target image G1 (tomographic image DA3) as shown in Fig. 11.

[0059] When the paging operation is stopped, the display control unit 23 may continue to display the annotation, or may hide the annotation.Furthermore, the display control unit 23 may display the annotation for a predetermined time and then hide the annotation.

[0060] On the other hand, when a radiologist finds an abnormal shadow in the target image G1, the radiologist may enlarge the abnormal shadow. In this case, the radiologist performs an operation to enlarge the displayed target image G1. For example, in the target image G1 (tomographic image DA2) on which annotations 32 and 33 are superimposed as shown in FIG. 10, if the radiologist wants to enlarge the target image G1 based on the region of interest R3 to which annotation 33 is added, the radiologist moves the mouse cursor 45 to the region of interest R3 and rotates the mouse wheel while pressing a specific key such as the ctrl key, as shown in FIG. 12. Based on the detection result of such a resizing operation, the identification unit 24 identifies the region of interest R3 where the mouse cursor 45 is located as the region of interest.

[0061] The display control unit 23 displays on the display screen 40 the annotations assigned to the region of interest identified by the identification unit 24. Here, as shown in FIG. 12, assume that the region of interest R3 is identified as the region of interest when the currently displayed tomographic image DA2 includes two regions of interest R2 and R3, each with annotations 32 and 33 superimposed thereon. In this case, the display control unit 23 displays only the annotation 33 for the region of interest R3 and hides the annotation 32 for the region of interest R2. As a result, as shown in FIG. 13, the display area 43A displays the target image G1 (tomographic image DA2) enlarged around the region of interest R3, the annotation 32 is hidden, and only the annotation 33 assigned to the region of interest R3 is displayed.

[0062] When the resizing operation is stopped, the display control unit 23 may continue to display the annotation, or may hide the annotation, or may display the annotation for a predetermined time and then hide the annotation.

[0063] Furthermore, when an image interpreter finds an abnormal shadow in the target image G1, the image interpreter may pop up a finding input window 51 shown in FIG. 8 and input a finding about the abnormal shadow. The identification unit 24 may identify a region of interest based on a detection result of the image interpreter's operation of inputting a finding. Specifically, the identification unit 24 may analyze the input finding and identify a region of interest corresponding to a term included in the finding as a region of interest. Here, as shown in FIG. 12, suppose that the currently displayed tomographic image DA2 includes two regions of interest R2 and R3, each with annotations 32 and 33 superimposed thereon, and the input finding is "A nodule is observed in the left lung." The identification unit 24 analyzes the finding and identifies the region of interest "left lung." Based on the identified term "left lung," the identification unit 24 identifies the region of interest R2 of the left lung included in the tomographic image DA2 as a region of interest. In addition, the identification unit 24 may identify the term "nodule" contained in the finding text, refer to the descriptions in the text 32B and 33B of the annotations 32 and 33 assigned to the regions of interest R2 and R3, and identify the regions of interest to which annotations including descriptions about nodules are assigned as the regions of interest.

[0064] In this case, the display control unit 23 displays only the annotation 32 for the region of interest R2 and hides the annotation 33 for the region of interest R3. As a result, as shown in Fig. 14, the finding input window 51 is displayed in the display area 43A, and based on the finding "A nodule is seen in the left lung" input in the finding input window 51, the annotation 33 assigned to the region of interest R3 is hidden and only the annotation 32 assigned to the region of interest R2 is displayed.

[0065] Assume that the radiologist further inputs the finding "A cavity is also seen" into the finding input window 51. In this case, the identification unit 24 analyzes the newly input finding and identifies the term "cavity." In this case, as shown in FIG. 15 , if the text 33B of the annotation 33 assigned to the region of interest R3 includes a description about a cavity, the annotation 33 assigned to the region of interest R3 may be further displayed.

[0066] In this case, when the input cursor 38 is positioned at the position of "A cavity is also seen" in the finding sentence input window 51, the display control unit 23 may display the annotation 33 assigned to the region of interest R3 corresponding to the finding sentence "A cavity is also seen," and may hide the annotation 32 assigned to the region of interest R2, as shown in Fig. 16. Note that when the input cursor 38 is positioned at the finding sentence "A nodule is seen in the left lung," the display control unit 23 may display the annotation 32 assigned to the region of interest R2 corresponding to the finding sentence "A nodule is seen in the left lung," and may hide the annotation 33 assigned to the region of interest R3.

[0067] When the operation of inputting the observation sentence is stopped, the display control unit 23 may continue to display the annotation, or may hide the annotation, or may hide the annotation after displaying it for a predetermined time.

[0068] Furthermore, when an image interpreting physician finds an abnormal shadow in the target image G1, the physician may perform an operation to input characteristics. In this case, the physician identifies the found abnormal shadow with the mouse cursor 45 and right-clicks on the identified abnormal shadow, thereby popping up a characteristic input window 50 as shown in FIG. 7. Based on the detection result of this operation, the identification unit 24 identifies the region of interest where the mouse cursor 45 is located as the region of interest. For example, when an operation to input characteristics for the region of interest R2 is performed in the target image G1 (tomographic image DA2) on which annotations 32 and 33 are superimposed as shown in FIG. 10, the physician moves the mouse cursor 45 to the region of interest R2 and right-clicks. Upon detecting this operation, the identification unit 24 identifies the region of interest R2 where the mouse cursor 45 is located as the region of interest.

[0069] In this case, the display control unit 23 hides the annotation 33 assigned to the region of interest R3 in the target image G0 (tomographic image DA2) and displays only the annotation 32 assigned to the region of interest R2, as shown in Fig. 17. The radiologist can view the annotation 32 displayed for the region of interest R2 and add missing characteristics or correct incorrect characteristics.

[0070] When the operation of inputting the properties is stopped, the display control unit 23 may continue to display the annotation, or may hide the annotation, or may hide the annotation after displaying it for a predetermined time.

[0071] Next, the processing performed in this embodiment will be described. FIG. 18 is a flowchart showing the processing performed in this embodiment. It is assumed that a plurality of images including the target image G0 and the past image GP0 of the patient under examination are acquired from the image server 5 and stored in the storage 13. It is also assumed that the display control unit 23 is displaying the display screen 40 on the display 14, an examination list is selected on the display screen 40, and images related to the selected examination are displayed in the thumbnail image area 42. When an image to be displayed in the image display area 43 is selected in the thumbnail image area 42 (step ST1; Yes), the display control unit 23 displays the selected image in the image display area 43 (step ST2). At this time, even if a region of interest has been detected in the image, the display control unit 23 first hides the annotation (step ST3).

[0072] In this state, the radiologist can perform operations such as paging, inputting findings, and attaching characteristic tags to newly found abnormal shadows.

[0073] Next, the identification unit 24 determines whether or not another operation related to the display of the target image or another operation related to the interpretation of the image has been detected (detection of another operation; step ST4). If step ST4 is positive, the identification unit 24 identifies a region of interest among the multiple regions of interest based on identification information including other information that is the detection result of the other operation (step ST5).

[0074] The display control unit 23 switches to displaying annotations for the identified region of interest (step ST6). This allows the radiologist to interpret the region of interest by referring to the displayed annotations. The radiologist can also perform operations such as inputting a finding statement and attaching a characteristic tag to a newly found abnormal shadow. The inputted finding statement is saved in the storage 13 as an interpretation report. The characteristic tag is also attached to the target image G0 as tag information.

[0075] Next, the display control unit 23 determines whether other operations have been completed (step ST7), and if step ST7 is negative, the process returns to step ST6. If step ST7 is positive, the process switches the annotation to hidden (step ST8). If step ST4 is negative or following step ST8, the process determines whether an instruction to end the interpretation has been given (step ST9), and if step ST9 is negative, the process returns to step ST4. If step ST9 is positive, the process ends. The interpretation report generated by interpretation of the target image G0 and the target image G0 to which the attribute tag has been added are transferred to the report server 7. The report server 7 stores the transferred interpretation report together with the target image G0.

[0076] If step ST7 is positive, the annotation may be switched to hidden mode after a predetermined time has elapsed. Alternatively, the annotation may be continued to be displayed. If the annotation is continued to be displayed, another operation is detected, a region of interest is identified according to the detected operation, and an annotation related to the identified region of interest is displayed.

[0077] In this manner, in this embodiment, at least one region of interest is identified based on identification information including information other than the information dedicated to identifying the region of interest that the user focuses on, and annotations for the identified region of interest are switched to display, so that the radiologist can refer to only the annotations for the region of interest without performing any cumbersome operations.

[0078] In particular, by hiding annotations related to regions of interest other than the region of interest, it is possible to prevent annotations about regions of interest that are not of interest from interfering with interpretation of the region of interest that is of interest.

[0079] In addition, by identifying an area of ​​interest based on the detection results of other operations related to the display or interpretation of a medical image, the radiologist can identify the area of ​​interest and display annotations related to the area of ​​interest without having to perform a dedicated operation to identify the area of ​​interest.

[0080] In the above embodiment, all annotations are hidden when the target image G0 is displayed, but this is not limited to this. All annotations may be displayed when the target image G0 is displayed. In this case, when an attention area is identified based on identification information such as the detection result of another operation, the display control unit 23 may hide annotations related to areas of interest other than the attention area, thereby displaying only annotations related to the attention area.

[0081] In the above embodiment, annotations related to the region of interest are displayed, and annotations related to regions of interest other than the region of interest are hidden. However, this is not limited to this. Annotations related to regions of interest other than the region of interest may be weakly displayed. Here, weak display refers to weakening the display level of annotations compared to displayed annotations by increasing the transparency of the annotation, displaying it with a dashed line, or displaying it with a lower density or a lighter color than displayed annotations. For example, if the region of interest R2 in the tomographic image DA2 shown in FIG. 10 is identified as the region of interest, annotation 32 related to region of interest R2 may be displayed, and annotation 33 related to region of interest R3 may be displayed with a dashed line, as shown in FIG. 19.

[0082] Furthermore, in the above embodiment, the detection result of another operation is used as the other information included in the identification information, but this is not limited to this. The observation conditions of the target image G0 may be used as the other information to identify the region of interest. Here, if the target image G0 is a CT image, a window level (WL) and a window width (WW) are used as the observation conditions when displaying the CT image. WL is the CT value that is the center of the region to be observed in the gradation that the display 14 can display when displaying the CT image on the display 14. WW is the width between the lower limit and upper limit of the CT value of the region to be observed.

[0083] Known observation conditions include mediastinal conditions that make it easy to observe bones and abdominal organs, and lung field conditions that make it easy to observe lung fields, depending on the region. Furthermore, not only CT images but also other medical images, such as MRI images or radiographic images acquired by plain radiography, are often displayed under observation conditions that make it easy to observe a specific region. Therefore, the identification unit 24 may identify an organ to be observed according to the observation conditions, and identify a region of interest included in the identified organ as the region of interest.

[0084] For example, suppose that a region of interest R5 is detected in the left lung and a region of interest R6 is detected in the liver in a tomographic image DA4 including a portion of the liver and left lung as shown in FIG. 20. Furthermore, suppose that an annotation 35 is added to the region of interest R5 in the left lung and an annotation 36 is added to the region of interest R6 in the liver. In this case, when the observation condition is the mediastinal condition, the identification unit 24 identifies the region of interest R6 in the liver as the region of interest. Therefore, the display control unit 23 displays the annotation 36 added to the region of interest R6 and hides the annotation 35 added to the region of interest R5, as shown in FIG. 21.

[0085] In the above embodiment, annotations related to the region of interest are displayed, but annotations superimposed on the region of interest may be hidden. For example, as shown in Fig. 22, when a region of interest R4 included in a tomographic image DA3 is identified as the region of interest, a figure 34A and leader line 34C surrounding the region of interest R4 of the annotation 34 may be hidden, and only text 34B outside the human body region may be displayed.

[0086] Furthermore, in the above embodiment, annotations are used as related information, but the present invention is not limited to this. Examples of related information related to a region of interest include, in addition to annotations, region extraction results of structures included in the target image G0. For example, as shown in FIG. 23 , a mask 60 for a liver region included in the target image G0 may be superimposed on a tomographic image DA5 of the target image as the region extraction result. In this case, if the liver is used as the region of interest and the mask 60 for the liver is used as related information, the mask 60 may be displayed on the liver based on the detection result of another operation related to the display of the image or another operation related to the interpretation of the image.

[0087] Furthermore, in the above embodiment, the display and non-display of annotations is described only for the target image G0 (G1) displayed in the display area 43A, but if annotations are also added to images displayed in the other display areas 43B to 43D, the display and non-display of annotations may be switched in the same way as in the display area 43A.

[0088] In the above embodiment, the region of interest is identified based on information other than the information dedicated to identifying the region of interest of the user. However, the present invention is not limited to this. The detection result of a dedicated operation for identifying the region of interest, such as a mouse click by the radiologist, may also be used as the identification information. For example, as shown in FIG. 24 , assume that a region of interest R7 is included in the pleura of the lung field included in the tomographic image DA6, and a region of interest R8 is included in the mediastinum adjacent to the region of interest R7. In this case, when the radiologist selects the region of interest R7 in the lung field by clicking the mouse, the radiologist may erroneously select the region of interest R8 adjacent to the region of interest R7. In such a case, the identification unit 24 may display an annotation using the detection result of the radiologist's mouse click to select the region of interest and the observation conditions of the target image G0 including the tomographic image DA6 as identification information.

[0089] For example, if the observation condition of the target image G0 is a lung field condition, the region of interest that the radiologist focuses on is included in the lung field. Therefore, when the observation condition is a lung field condition, the identification unit 24 determines that the selected region of interest is a region of interest R7 included in the lung field based on the observation condition and the detection result of the operation to select the region of interest, and identifies the region of interest R7 as the region of interest. In this case, as shown in FIG. 25, the annotation 37 assigned to the region of interest R7 is displayed. Note that when the observation condition is a mediastinum condition, the identification unit 24 determines that the selected region of interest is a region of interest R8 included in the mediastinum based on the observation condition and the detection result of the operation to select the region of interest, and identifies the region of interest R8 as the region of interest. In this case, only the annotation assigned to the region of interest R8 may be displayed.

[0090] In addition, in the above embodiment, multiple images are simultaneously displayed on the display screen 40, but this is not limiting. The technology of the present disclosure can also be applied to a case where only one image is displayed on the display screen 40.

[0091] Furthermore, in the above embodiment, the image display device included in the image interpretation WS3 according to this embodiment is equipped with the analysis unit 22, and the target image is analyzed in the image interpretation WS3, but this is not limited to this. The target image may be analyzed by an external analysis device separate from the image interpretation WS3, and the analysis results may be acquired in the image interpretation WS3. Also, annotations may already be superimposed on images stored in the image server 5. Even in such cases, as in the above embodiment, the image on which the annotations are superimposed can be displayed in the image interpretation WS3, and the annotations can be hidden based on the detection results of other operations related to the display of the image or other operations related to the image interpretation.

[0092] Furthermore, in each of the above embodiments, the following various processors can be used as the hardware structure of a processing unit that executes various processes, such as the image acquisition unit 21, the analysis unit 22, the display control unit 23, and the identification unit 24. As described above, the various processors include a CPU, which is a general-purpose processor that executes software (programs) and functions as various processing units, as well as dedicated electrical circuits that are processors having a circuit configuration specifically designed to execute specific processes, such as a programmable logic device (PLD), which is a processor whose circuit configuration can be changed after manufacture, such as an FPGA (Field Programmable Gate Array), and an ASIC (Application Specific Integrated Circuit).

[0093] A single processing unit may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs or a combination of a CPU and an FPGA). Also, multiple processing units may be configured with a single processor.

[0094] Examples of configuring multiple processing units with a single processor include, first, a form in which one processor is configured with a combination of one or more CPUs and software, and this processor functions as multiple processing units, as typified by computers such as client and server. Second, a form in which a processor is used to realize the functions of an entire system including multiple processing units with a single IC (Integrated Circuit) chip, as typified by systems on chips (SoCs). In this way, various processing units are configured using one or more of the above-mentioned various processors as a hardware structure.

[0095] Furthermore, more specifically, the hardware structure of these various processors can be an electric circuit that combines circuit elements such as semiconductor elements. [Explanation of symbols]

[0096] 1 Medical Information System 2. Imaging equipment 3 Image Reading Workshop 4. Medical Workshop 5 Image Server 6. Image Database 7 Report Server 8 Report DB 10 Network 11 CPU 12 Image display program 13. Storage 14 Display 15 Input Devices 16 memory 17 Network I / F 18 Bus 20 Image display device 21 Image acquisition unit 22 Analysis Department 23 Display control unit 24 Specific section 31~37 Annotation 32A, 33A, 34A Shapes 32B, 33B, 34B Text 32C,33C,34C leader line 40 display screen 41 Examination List Area 42 Thumbnail Image Area 43 Image display area 43A~43D Display Area 45 Mouse Cursor 50 Property input window 51 Observation entry window 60 Mask C1~C6 thumbnail images Dj, DA1~DA6 Tomographic images G0, G1, G2 target images GA1~GA3 tomographic images GP1~GP4 past images R1~R8 Region of interest

Claims

1. at least one processor; The processor: Identifying a region of interest that a user focuses on in a medical image based on observation conditions for identifying the region of interest; performing control to display only information related to the identified area of ​​interest; Image display device.

2. the processor identifies an organ to be observed from among a plurality of regions of interest included in the medical image based on the observation conditions; identifying, as the region of interest, a region of interest corresponding to the observation conditions from among a plurality of regions of interest included in the identified organ; The image display device according to claim 1 .

3. The viewing condition is a window level or a window width.

3. The image display device according to claim 1 or 2.

4. The processor controls to hide or weakly display related information regarding regions of interest other than the region of interest. The image display device according to any one of claims 1 to 3.

5. the related information includes annotations related to the region of interest; The image display device according to claim 2 .

6. the annotation includes at least one of a recognition result of the region of interest and text information related to the region of interest; The image display device according to claim 5 .

7. Identifying a region of interest that a user focuses on in a medical image based on observation conditions for identifying the region of interest; A process of controlling to display only the related information related to the identified attention area, A computer-implemented method for displaying images.

8. Identifying a region of interest that a user focuses on in a medical image based on observation conditions for identifying the region of interest; A process of controlling to display only the related information related to the identified attention area, An image display program to be executed by a computer.

Images