Medical image display device, medical image display control method, and program

Abstract

To reduce the time and effort required to switch between multiple images and display them.SOLUTION: A medical image display device includes: an acquisition unit; an image processing unit; a generation unit; a display control unit; a reception unit; and a display switching unit. The acquisition unit acquires medical images. The image processing unit sequentially performs a plurality of types of image processing on the medical images. The generation unit generates cross-sectional images on the basis of the image processing. The display control unit causes the cross-sectional images to be displayed in a display region of a display unit. The reception unit receives user operations on the cross-sectional images. The display switching unit switches between the cross-sectional images to be displayed in the display region on the basis of the processing state of the image processing, the display state of the display region, and the user's operation history.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The embodiments disclosed in this specification and the drawings relate to a medical image display device, a medical image display control method, and a program. [Background technology]

[0002] For example, when treating the left atrial appendage or making a treatment plan for the left atrial appendage, a user such as a doctor uses a medical image to confirm information such as the morphology of the left atrial appendage and the positional relationship between the left atrial appendage and surrounding cardiac structures. Conventionally, there is a technology that automatically extracts cardiac structures including the left atrial appendage from a medical image, and identifies and displays cross-sectional positions suitable for confirming the above information in the medical image based on the extracted cardiac structure. This technology reduces the effort required for a doctor to identify the cross-section.

[0003] During treatment or treatment planning, it is necessary to observe a plurality of different types of cross sections or groups of cross sections, and it is necessary to switch between and display images of the plurality of cross sections or groups of cross sections. However, switching between these images is, for example, performed manually by a user, which is time-consuming. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2019-202142 A Summary of the Invention [Problem to be solved by the invention]

[0005] The problem to be solved by the embodiments disclosed in this specification and the drawings is to reduce the effort required to switch and display multiple images. However, the problem to be solved by the embodiments disclosed in this specification and the drawings is not limited to the above problem. Problems corresponding to the effects of each configuration shown in the embodiments described below can also be positioned as other problems. [Means for solving the problem]

[0006] The medical image display device of the embodiment has an acquisition unit, an image processing unit, a generation unit, a display control unit, a reception unit, and a display switching unit. The acquisition unit acquires medical images. The image processing unit sequentially performs multiple types of image processing on the medical images. The generation unit generates cross-sectional images based on the image processing. The display control unit displays the cross-sectional images in a display area of ​​the display unit. The reception unit receives user operations on the cross-sectional images. The display switching unit switches the cross-sectional images to be displayed in the display area based on a processing state of the image processing, a display state of the display area, and an operation history of the user. [Brief description of the drawings]

[0007] [Figure 1] 1 is a diagram showing an example of the arrangement of a medical image display device 100 according to an embodiment. [Diagram 2] FIG. 4 is a diagram showing an example of a first image GA1. [Diagram 3] FIG. 13 is a diagram showing an example of a second image GA2. [Figure 4] FIG. 13 is a diagram showing an example of a third image GA3. [Diagram 5] FIG. 13 is a diagram showing an example of a fourth image GA4. [Figure 6] FIG. 13 is a diagram showing an example of a fifth image GA5. [Figure 7] FIG. 13 is a diagram showing an example of a sixth image GA6. [Figure 8] FIG. 13 is a diagram showing an example of a seventh image GA7. [Figure 9] FIG. 13 shows an example of an eighth image GA8. [Figure 10] FIG. 13 shows an example of a 9th image GA9. [Figure 11] A figure showing an example of a tenth image GA10. [Figure 12] 4 is a flowchart showing an example of processing in the medical image display device 100 of the embodiment. [Figure 13] FIG. 13 is a diagram showing an example of the contents of a display condition switching determination table 152. [Figure 14] FIG. 13 is a diagram showing an example of a GUI used to determine a priority order. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] Hereinafter, a medical image display device, a medical image display control method, and a program according to an embodiment will be described with reference to the drawings.

[0009] Fig. 1 is a diagram showing an example of the configuration of a medical image display device 100 according to an embodiment. For example, as shown in Fig. 1, the medical image display device 100 according to this embodiment is communicably connected to a medical image diagnostic device 10 and a medical image storage device 20 via a network. The network shown in Fig. 1 may also be connected to various other devices and systems. The medical image display device 100 has a viewing function for displaying medical images and an editing function for editing the displayed medical images.

[0010] The medical image diagnostic apparatus 10 captures an image of a subject to generate a medical image. The medical image diagnostic apparatus 10 then transmits the generated medical image to various devices on a network. For example, the medical image diagnostic apparatus 10 is an X-ray diagnostic apparatus, an X-ray CT (Computed Tomography) apparatus, an MRI (Magnetic Resonance Imaging) apparatus, an ultrasound diagnostic apparatus, a SPECT (Single Photon Emission Computed Tomography) apparatus, a PET (Positron Emission computed Tomography) apparatus, or the like.

[0011] The medical image storage device 20 stores various medical images related to subjects. Specifically, the medical image storage device 20 receives medical images from the medical image diagnostic device 10 via a network, and stores the medical images in a memory circuit within the device. For example, the medical image storage device 20 is realized by a computer device such as a server or a workstation. Also, for example, the medical image storage device 20 is realized by a Picture Archiving and Communication System (hereinafter, PACS) or the like, and stores medical images in a format conforming to DICOM (Digital Imaging and Communications in Medicine).

[0012] The medical image display device 100 performs various information processing related to a subject. Specifically, the medical image display device 100 receives medical images from a medical image diagnostic device 10 or a medical image storage device 20 via a network, and performs various information processing using the received medical images. For example, the medical image display device 100 is realized by a computer device such as a server or a workstation.

[0013] The medical image display device 100 includes, for example, a communication interface 110, an input interface 120, a display 130, a processing circuitry 140, and a memory 150. The communication interface 110 controls the transmission and communication of various data transmitted and received between the medical image display device 100 and other devices connected via a network. The communication interface 110 is connected to the processing circuitry 140, and transmits data received from other devices to the processing circuitry 140, or transmits data transmitted from the processing circuitry 140 to other devices. The communication interface 110 is realized by, for example, a network card, a network adapter, a NIC (Network Interface Controller), or the like.

[0014] The input interface 120 accepts various input operations by a user, converts the accepted input operations into electrical signals, and outputs the electrical signals to the processing circuitry 140. For example, when an input operation is performed by a user, the input interface 120 generates input operation information according to the input operation and outputs the input operation information to the processing circuitry 140.

[0015] The input interface 120 includes, for example, a mouse, a keyboard, a trackball, a switch, a button, a joystick, a touch panel, etc. The input interface 120 may be, for example, a user interface that accepts voice input from a microphone or the like. When the input interface 120 is a touch panel, the input interface 120 may also have the display function of the display 130. The input interface 120 is an example of an input device.

[0016] In this specification, the input interface is not limited to an interface having physical operation parts such as a mouse, a keyboard, etc. For example, an example of the input interface also includes an electrical signal processing circuit that receives an electrical signal corresponding to an input operation from an external input device provided separately from the device and outputs the electrical signal to a control circuit.

[0017] The display 130 displays various types of information. For example, the display 130 displays an image generated by the processing circuit 140, a GUI (Graphical User Interface) for receiving various input operations from an operator, and the like. For example, the display 130 is an LCD (Liquid Crystal Display), a CRT (Cathode Ray Tube) display, an organic EL (Electro Luminescence) display, or the like. The medical image may be a two-dimensional image or a three-dimensional image.

[0018] The processing circuitry 140 includes, for example, an acquisition function 141, an image processing function 142, a generation function 143, a display control function 144, a reception function 145, and a switching function 146. The processing circuitry 140 realizes these functions by, for example, a hardware processor (computer) executing a program stored in a memory (storage circuitry) 150.

[0019] The hardware processor refers to circuitry such as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), an Application Specific Integrated Circuit (ASIC), a programmable logic device (e.g., a Simple Programmable Logic Device (SPLD) or a Complex Programmable Logic Device (CPLD)), and a Field Programmable Gate Array (FPGA).

[0020] Instead of storing the program in the memory 150, the program may be directly embedded in the circuit of the hardware processor. In this case, the hardware processor realizes the function by reading and executing the program embedded in the circuit. The above program may be stored in the memory 150 in advance, or may be stored in a non-transitory storage medium such as a DVD or CD-ROM, and installed in the memory 150 from the non-transitory storage medium by mounting the non-transitory storage medium in a disk drive (not shown) of the medical image display device 100.

[0021] The hardware processor is not limited to being configured as a single circuit, but may be configured as a single hardware processor by combining multiple independent circuits to realize each function. Also, multiple components may be integrated into a single hardware processor to realize each function.

[0022] The acquisition function 141 in the processing circuit 140 acquires a medical image. For example, the acquisition function 141 acquires a medical image of a subject from the medical image diagnostic device 10 or the medical image storage device 20 via the communication interface 110. The acquisition function 141 can also acquire image data of a two-dimensional image, or multiple volume data obtained by capturing multiple images in three dimensions in the time direction.

[0023] The acquisition function 141 acquires, for example, a CT image, an ultrasound image, an MRI image, an X-ray image, an Angio image, a PET image, a SPECT image, etc. as the above-mentioned volume data. By executing the above-mentioned acquisition function 141, the processing circuitry 140 receives medical images of a subject from the medical image diagnostic apparatus 10 or the medical image storage apparatus 20, and stores the received medical images in the memory 150. The acquisition function 141 is an example of an acquisition unit.

[0024] The image processing function 142 sequentially executes a plurality of types of image processing on the medical image acquired by the acquisition function 141. The image processing function 142 processes the medical image based on, for example, a program stored in the memory 150 or an instruction corresponding to a user's input operation to the input interface 120. As image processing, the image processing function 142 executes, for example, setting display conditions when displaying the medical image, identifying and extracting an area showing a biological organ of interest in the image acquired by the acquisition function 141 (hereinafter, an area of ​​interest), and other processing. The image processing function 142 is an example of an image processing unit.

[0025] The generating function 143 generates a cross-sectional image based on the image processing by the image processing function 142. The generating function 143 generates a plurality of cross-sectional images related to the processing result based on the processing result of the image processing. The generating function 143 generates, for example, a cross-section at a predetermined position of the subject, for example, a cross-sectional image of an Ostium plane or a Fossa plane. In this specification, the Ostium plane is a cross-section specified based on Ostium, which is the entrance of the left atrial appendage region (hereinafter, left atrial appendage region). Also, in this specification, the Fossa plane is a cross-section specified by three points, namely, the center position of Ostium, which is the entrance of the left atrial appendage region, the center position of the fossa ovalis region, and the center position of the entrance of the inferior vena cava to the right atrium (RA). The generating function 143 is an example of a generating unit.

[0026] The display control function 144 displays the cross-sectional images generated by the generation function 143 in the display area of ​​the display 130. The display control function 144 displays the cross-sectional images on the display based on any one of a plurality of display conditions that are set through a predetermined procedure. For example, the display control function 144 divides the display area to display a plurality of cross-sectional images. The display procedure and display mode of the cross-sectional images by the display control function 144 will be described further below. The display control function 144 is an example of a display control unit.

[0027] The reception function 145 receives a user's operation on the cross-sectional image. For example, the reception function 145 receives a user's operation based on input operation information transmitted by the input interface 120 in response to the user's input operation on the input interface 120. The reception function 145 is an example of a reception unit.

[0028] The switching function 146 switches the display mode of the display 130 based on the processing state of the image processing, the display state of the display area, and the user's operation history. The switching function 146 switches the display area displayed on the display 130 based on, for example, the processing state of the image processing in the image processing function 142, the display state of the display area by the display control function 144, and the user's operation history of the cross-sectional image. The processing state of the image processing is, for example, the state of a segmentation process that identifies a region of interest. The display state of the display area is based on, for example, display conditions controlled by the display control function 144. The switching function 146 is an example of a switching unit.

[0029] The memory 150 is realized by, for example, a random access memory (RAM), a semiconductor memory element such as a flash memory, a hard disk, an optical disk, etc. The memory 150 does not have to be built into the medical image display device 100 as long as the medical image display device 100 can access the memory 150 over a network. The memory 150 stores a display condition switching determination table 152. The display condition switching determination table 152 is a table used to determine whether or not to switch the display state of the display area.

[0030] Next, the display contents displayed on the display 130 by the medical image display device 100 will be described. The medical image display device 100 causes the display 130 to display images including cross-sectional images by a display control function 144 in the processing circuitry 140. When displaying images, the display control function 144, for example, divides the display area of ​​the display 130 into multiple areas and displays cross-sectional images and other information in each of the divided display areas. Here, some examples of images displayed on the display 130 by the display control function 144 will be described.

[0031] 2 is a diagram showing an example of a first image GA1. The first image GA1 is displayed in a divided display area, which is a thumbnail display area 201, a main image display area 202, a menu bar display area 203, a sub-image display area 204, and an analysis result display area 205. Of these, medical images such as cross-sectional images are displayed mainly in the main image display area 202 and the sub-image display area 204, which has a smaller display area than the main image display area 202, thereby exerting the viewing function of the medical image display device 100. The main image display area is an example of a first display area. The sub-image display area is an example of a second display area.

[0032] The main image display area 202 is divided and displayed into a first main image divided area 202a, a second main image divided area 202b, a third main image divided area 202c, and a fourth main image divided area 202d. The sub-image display area 204 is divided and displayed into four divided areas (divided display areas), a first sub-image divided area 204a, a second sub-image divided area 204b, a third sub-image divided area 204c, and a fourth sub-image divided area 204d.

[0033] The thumbnail display area 201 is a display area for displaying thumbnail images. The thumbnail display area 201 displays a list of images acquired from an image database or an examination device in the hospital based on a user's instruction in response to a user's operation accepted by the reception function 145. The images displayed in the list are images that satisfy the conditions specified by the user using an interface for specifying image conditions (not shown).

[0034] The image conditions include information about the patient who is the subject of the image, such as the name, ID, date of birth, and weight, as well as the type of modality of the image, the name of the imaging device, the imaging date, the imaging conditions, the reconstruction conditions, etc. The user can specify the image conditions by operating the input interface 120.

[0035] These image conditions can be acquired from the DICOM header of the image, PACS, electronic medical record, Radiology Information System (hereinafter, RIS), Hospital Information System (hereinafter, HIS), etc. In the following, an example in which one CT image of a specific patient is specified will be described, but images of multiple patients or images of different types of modalities (for example, CT image and ultrasound image, etc.) may be specified.

[0036] A plurality of icons representing images that satisfy the specified conditions are displayed in the thumbnail display area 201. As the icon to be displayed, a two-dimensional image of a representative cross section of the specified image may be reduced to fit the size of the thumbnail display area 201 and the reduced image may be used as it is, or a character string or symbol simply indicating the image may be used, or various figures, images, or schematic images prepared in advance and stored in the memory 150 may be used.

[0037] In the thumbnail display area 201, basic information about the image, such as the date of imaging, the number of slices, reconstruction function, etc., may be displayed side by side or overlaid on the icon. The basic information to be displayed may be determined in advance or may be specified by the user. This information may also be obtained from, for example, the DICOM header of the image, PACS, electronic medical record, RIS, HIS, etc.

[0038] The user can select an icon displayed in thumbnail display area 201 and drag and drop it into main image display area 202 or sub-image display area 204 to display an image corresponding to that icon in main image display area 202 or sub-image display area 204. If an image is already being displayed in main image display area 202 when drag and drop is performed, a warning (not shown) may be displayed to the user, and then the already displayed image may be removed from the display area and the image corresponding to the dragged and dropped icon may be displayed.

[0039] When displaying an image such as a cross-sectional image in a display area, the display control function 144 may display the image based on a predetermined display condition, or may display the image based on a display condition specified based on a process described later. The display condition is the allocation of images to be displayed in a plurality of display areas (for example, what image is to be displayed in which area of ​​the main image first division area 202a to main image fourth division area 202d), the cross-sectional position when displaying a cross-sectional image, the magnification ratio, the window level, the window width, etc. The display conditions will be described later in detail.

[0040] The main image display area 202 and the sub-image display area 204 are image display areas that display various images such as cross-sectional images. The main image display area 202 and the sub-image display area 204 are each divided into an arbitrary number or size of areas (for example, a main image first division area 202a to a main image fourth division area 202d and a sub-image first division area 204a to a sub-image fourth division area 204d). In each of the divided areas, a cross-sectional image at a predetermined position in the CT image acquired by the acquisition function 141 or a processed image calculated by applying image processing to the CT image is displayed.

[0041] In each divided area in the main image display area 202 and the sub-image display area 204, an image is displayed under a different associated display condition. The type of image and the display condition are determined in advance. In the example shown in Fig. 2, an image is displayed in the main image display area 202 under a first display condition, which will be described later, and an image is displayed in the sub-image display area 204 under a second display condition.

[0042] The display control function 144 can display the images displayed in the main image display area 202 and the sub-image display area 204, or their divided areas, on the display 130 by changing the observation cross section, scrolling through slices (browsing), changing the magnification ratio, center position (parallel movement), window level, window width, etc., based on user instructions in response to user operations accepted by the acceptance function 145.

[0043] The display control function 144 can also change the size of the divided areas in the main image display area 202 and the sub-image display area 204. The user can change the size of the divided areas, for example, by performing an operation (drag & drop, etc.) on the frame of each divided area (the dividing line dividing the display area) or a specific operation on each display area (double-clicking the mouse button, clicking the mouse button while holding down the Ctrl key on the keyboard, etc.).

[0044] Furthermore, at a specific position on each display region or divided region (for example, the position of the edge of a region such as the edge region 202e), information designated in advance or by the user is displayed superimposed. The information displayed in the edge region 202e includes information about the patient who is the subject of the image, such as the name, ID, date of birth, and weight, and information about the image, such as the type of modality of the image, the name of the imaging device, the imaging date, the imaging conditions, and the reconstruction conditions. The user can specify the information to be displayed in the edge region 202e by operating the input interface 120. The information displayed in the edge region 202e can be obtained from the DICOM header of the image, PACS, electronic medical record, IS, HIS, etc.

[0045] Icons and buttons corresponding to various functions are displayed in the menu bar display area 203. A switching instruction button 203a, a split instruction button 203b, a first operation system icon 203c to a fifth operation system icon 203g (operation system icons), a first drawing and measurement system icon 203h to a seventh drawing and measurement system icon 203n (drawing and measurement system icons), a first check box 203o to a third check box 203q, an execution button group 203r, a reset button 203s, and a setting screen display button 203t are displayed in the menu bar display area 203.

[0046] The medical image display device 100 mainly performs an editing function based on instructions given by the reception function 145 receiving user operations on drawing and measurement system icons. The medical image display device 100 may have only a viewing function without an editing function. The user can activate a function indicated by an icon or button by selecting and operating an icon or button displayed in the menu bar display area 203 using the input interface 120, for example, an instruction device such as a mouse. Hereinafter, selecting and operating an icon or button may be abbreviated to simply "selecting."

[0047] The switching instruction button 203a is a button for instructing switching between display and non-display of the thumbnail display area 201. When the thumbnail display area 201 is displayed, the user performs an operation to select the switching instruction button 203a, and the reception function 145 accepts the user's operation, thereby hiding the thumbnail display area 201. In this case, the display areas of the main image display area 202, the sub-image display area 204, and the analysis result display area 205 are enlarged by the amount by which the thumbnail display area 201 is hidden. When the thumbnail display area 201 is not displayed, the user performs an operation to select the switching instruction button 203a, and the reception function 145 accepts the user's operation, thereby displaying the thumbnail display area 201.

[0048] The division instruction button 203b is a button for instructing a change in the number of divisions in the main image display area 202 or the sub-image display area 204. When the user performs an operation to select the division instruction button 203b and the reception function 145 receives the operation by the user, the number of divisions is changed in one or both of the main image display area 202 and the sub-image display area 204. The number of divisions may be changed in any manner. For example, the area may be divided into a number of divisions specified by the user, or may be divided into a predetermined number of divisions. Whether the divided area in which the number of divisions is to be changed is the main image display area 202 or the sub-image display area 204 may be set in advance, or may be specified by the user.

[0049] Instead of or in addition to changing the number of divisions, the matrix of the divided regions may be changed by selecting the division instruction button 203b. In the example shown in FIG. 2, the main image display region 202 and the sub-image display region 204 are both divided into two rows and two columns, but the number of rows and columns may be changeable. Alternatively, the size of the divided regions may be changeable. For the number of divisions, matrix, and size of the divided regions, several patterns may be registered as presets, and the corresponding display form may be set by specifying the registered set. Also, the user may be allowed to register a new preset.

[0050] The first operation icon 203c to the fifth operation icon 203g are a group of icons for functions to which the operation system of a mouse included in the input interface 120 is assigned. The display control function 144 controls so that, for example, when any one of the first operation icon 203c to the fifth operation icon 203g is selected, the operation system of the left click and drag of the mouse is assigned to the operation system corresponding to the selected icon.

[0051] Specifically, the first operation icon 203c is a browse operation system for continuously displaying images in the slice direction, the fourth operation icon 203f is an operation system for changing the magnification ratio of an image, the third operation icon 203e is an operation system for moving an image in parallel, the second operation icon 203d is an operation system for changing the gradation of an image (WL or WW in a CT image), the fifth operation icon 203g is an operation system for rotating an image, etc. are assigned to the operation system of left clicking and dragging of a mouse. Of course, any of the first operation icon 203c to the fifth operation icon 203g may be assigned to the operation system of right clicking and dragging, the operation system of mouse wheel clicking and dragging, or the operation system of right and left simultaneous clicking and dragging.

[0052] Also, for the operation icons, the speed or amount of slice forwarding in the browse function relative to the amount of mouse movement, the amount of change in magnification, the amount of translation, the amount of change in gradation, and the amount of rotation may be set.Also, the allocation of the operation icons may be changed according to the mouse operation when selecting the operation icons.For example, when the icon is selected by left clicking, the operation system corresponding to the icon is assigned to the left click operation system, when the icon is selected by right clicking, the operation system corresponding to the icon is assigned to the right click operation system, when the icon is selected by simultaneous right and left clicking, the operation system corresponding to the icon is assigned to the right and left simultaneous click operation system, and when the icon is selected by mouse wheel clicking, the operation system corresponding to the icon is assigned to the mouse wheel click operation system.

[0053] The first drawing and measurement icon 203h to the seventh drawing and measurement icon 203n are all icons for drawing and measuring various figures. When the user performs an operation to select one of the first drawing and measurement icon 203h to the seventh drawing and measurement icon 203n and the reception function 145 receives the operation by the user, the drawing and measuring function of various figures is executed by selection according to the operation received by the reception function 145.

[0054] Specifically, the first drawing and measurement system icon 203h is an icon for designating drawing a straight line on an image and measuring and displaying the length of the straight line. The user can adjust the display form, such as the positions of the start and end points of the line, the color and thickness, and the font of the measurement value, by operating the input interface 120.

[0055] The second drawing and measurement icon 203i is an icon that specifies drawing two straight lines on an image and measuring and displaying the acute angle between the two straight lines. The user can adjust the display form, such as the positions of the start and end points of the two straight lines, their colors and thickness, and the font of the measurement values, by operating the input interface 120.

[0056] The third drawing and measurement icon 203j is an icon that specifies drawing an ellipse on an image and calculating and displaying the perimeter, inner area, and statistics of the pixel values ​​inside the ellipse (average, maximum, minimum, etc.). The user can adjust the display form of the ellipse, such as the center position, major axis, minor axis, color, thickness, and font of the measurement values, by operating the input interface 120.

[0057] The fourth drawing and measurement icon 203k is an icon for designating drawing an arrow icon on the image. The user can adjust the display form of the arrow icon, such as the start and end positions, color, thickness, and tip shape, by operating the input interface 120.

[0058] The fifth drawing and measurement icon 203l is an icon for designating the display of an arbitrary character string on an image. The character string to be displayed is designated by the user operating the input interface 120. The display form, such as the position where the character string is displayed, the font of the character string, and the color of the background, can be adjusted by the user operating the input interface 120.

[0059] The sixth drawing and measurement icon 203m is an icon for designating drawing a closed curve of any shape on an image and calculating and displaying the perimeter of the closed curve, the area inside the closed curve, and statistics of the pixel values ​​inside the closed curve (average, maximum, minimum, etc.). The user can adjust the display form such as the center position, color, thickness, and font of the measurement value of the closed curve by operating the input interface 120.

[0060] The closed curve that is drawn and displayed by selecting the sixth drawing / measurement system icon 203m can be set to a predetermined shape (circle, ellipse, rectangle, square, triangle, etc.), and the length of each side of the shape, the angle between two sides, the diameter, major axis, minor axis, etc. can be adjusted, or a freeform shape can be drawn.

[0061] The seventh drawing and measurement icon 203n is an icon that specifies drawing an open curve of any shape on an image and calculating and displaying the perimeter of the open curve. The display form, such as the center position, color, thickness, and font of the measurement value of the open curve, can be adjusted by the user operating the input interface 120. The display control function 144 may control the selection of the seventh drawing and measurement icon 203n to adjust the display form, as well as to set a three-dimensional figure (sphere, oval sphere, rectangular parallelepiped, triangular pyramid, etc.) and calculate and display the surface area, volume, etc. of the figure.

[0062] The first check box 203o is a check box that instructs switching of the reference display condition (whether to display or not display) that indicates the cross-sectional position of an image displayed in another divided area on the image. The display condition of the reference is switched by selecting the first check box 203o. The reference is displayed by checking the first check box 203o, and is not displayed by unchecking the first check box 203o. In addition, the user can change the display form of the image displayed in the corresponding divided area by performing an operation on the reference line displayed as a reference.

[0063] 3 is a diagram showing an example of the second image GA2. In the second image GA2, a first reference line 301, a second reference line 302, a first mark 303, and a second mark 304 are displayed as references. The first reference line 301 is a line that faces in the vertical direction. The second reference line 302 is a line that is perpendicular to the first reference line 301. The display form (color, thickness, type (solid line or dotted line, dotted line fineness, etc.)) of the reference lines 301 and 302 may be set in advance, or may be specified by the user by operating the input interface 120.

[0064] The first mark 303 is displayed, for example, at the upper end of the first reference line 301 and the right end of the second reference line 302, superimposed on the first reference line 301 and the second reference line 302. The first mark 303 is displayed, for example, as a hollow circle. The second mark 304 is displayed, for example, sandwiching the lower end of the first reference line 301 and the left end of the second reference line 302. The second mark 304 is displayed, for example, as two triangles, with the tips of both triangles sandwiching the lower end of the first reference line 301 and the left end of the second reference line 302, respectively.

[0065] The user can move the intersection 305 by, for example, operating the mouse to select the intersection 305 of the reference lines or its surroundings and drag and drop it. When the intersection 305 is moved, the cross-sectional positions indicated by the first reference line 301 and the second reference line 302 also change, and the cross-sectional images displayed in each divided area change accordingly.

[0066] For example, the user can translate the selected reference lines 301, 302 by selecting and dragging and dropping the reference lines 301, 302. By moving the reference lines 301, 302, the user can change the cross-sectional positions indicated by the reference lines 301, 302, and can change the cross-sectional images displayed in each display area accordingly.

[0067] A user can rotate the selected reference lines 301, 302 by selecting the reference lines 301, 302 and dragging and dropping them in a specific direction. By rotating the reference lines 301, 302, a user can change the cross-sectional positions indicated by the reference lines 301, 302, and can change the cross-sectional images displayed in each display area accordingly.

[0068] When rotating multiple reference lines, the display control function 144 may control the operation of one reference line to operate the other reference lines in conjunction with the operation of the other reference lines, so that the angle formed by the multiple reference lines is always kept constant. In this case, the angle kept constant may be, for example, an angle obtained by equally dividing 180 degrees by the number of reference lines.

[0069] For example, when there are two reference lines, the display control function 144 may rotate all the reference lines while maintaining the state in which the reference lines are perpendicular to each other (forming an angle of 90 degrees) (when there are three reference lines, the angle between adjacent reference lines is 60 degrees). The display control function 144 may control the multiple reference lines to move independently of each other.

[0070] The display control function 144 may switch between parallel translation and rotation of the reference lines 301, 302 depending on a position specified by the pointer on the reference lines 301, 302. In this case, for example, when the inside of the first mark 303 on the reference lines 301, 302 is specified with the pointer and the pointer (mouse) is moved, the display control function 144 rotates the reference lines 301, 302. On the other hand, when the position outside the first mark 303 on the reference lines 301, 302 is specified with the pointer and the pointer (mouse) is moved, the display control function 144 translates the reference lines 301, 302.

[0071] The display control function 144 may switch between parallel translation and rotation of the reference lines 301, 302 in response to a keyboard operation. In this case, when the reference lines 301, 302 are designated and moved while the Ctrl key on the keyboard is pressed, for example, the display control function 144 translates the reference lines 301, 302. On the other hand, when the reference lines 301, 302 are designated and moved without pressing a key on the keyboard, for example, the display control function 144 rotates the reference lines 301, 302.

[0072] The display control function 144 may create a projection image such as a maximum intensity projection image or a minimum intensity projection image in a range based on the size of a certain width by specifying the width using the reference lines 301, 302, and display the projection image in a display area corresponding to the reference lines 301, 302. The width may be specified by moving the second mark 304, for example.

[0073] For example, when specifying the width of the first reference line 301, the right and left triangles of the second mark 304 that sandwich the lower end of the first reference line 301 may be moved, and the range from the position indicated by the tip of the right triangle to the position indicated by the tip of the left triangle may be specified as the width of the first reference line 301. Similarly, when specifying the width of the second reference line 302, the upper and lower triangles of the second mark 304 that sandwich the left end of the second reference line 302 may be moved, and the range from the position indicated by the tip of the upper triangle to the position indicated by the tip of the lower triangle may be specified as the width of the second reference line 302. In this way, a projection image such as a maximum intensity projection image or a minimum intensity image in the specified range is created.

[0074] The second check box 203p is a check box for instructing whether or not to display a two-dimensional cross section superimposed (overlapped) on an image showing a three-dimensional region. The three-dimensional region is included in a rendering image such as a VR (Volume Rendering) image or an SR (Surface Rendering) image.

[0075] Selecting the second check box 203p switches between superimposing and displaying a two-dimensional cross section on a three-dimensional region. Checking the second check box 203p displays a two-dimensional cross section superimposed on a three-dimensional region, and unchecking the second check box 203p hides the two-dimensional cross section.

[0076] 4 is a diagram showing an example of a third image GA3. In the third image GA3, a VR image of the left atrial appendage is displayed in a main image first divided region 202a, and a cross-sectional image (two-dimensional image) of the left atrial appendage is displayed in a main image second divided region 202b.

[0077] When the second check box 203p is checked, the cross-sectional image of the left atrial appendage displayed in the main image second divided area 202b is displayed, for example, in correspondence with a three-dimensional position in the VR image of the left atrial appendage displayed in the main image first divided area 202a. Fig. 5 is a diagram showing an example of the fourth image GA4. The fourth image GA4 is an image in which the cross-sectional image of the left atrial appendage is superimposed on a three-dimensional position in the VR image of the left atrial appendage displayed in the main image first divided area 202a in Fig. 4.

[0078] As seen in the fourth image GA4, when a cross-sectional image of the left atrial appendage is superimposed on a three-dimensional position in the VR image of the left atrial appendage, a VR image located in front of the cross-section in the observation direction is displayed, and a VR image located behind the cross-section is not displayed. In the third image GA3 shown in Fig. 4, a cross-sectional image is displayed in each of the main image second division region 202b, the main image third division region 202c, and the main image fourth division region 202d. The user specifies the cross-sectional image to be superimposed on the main image first division region 202a by specifying any one of the main image second division region 202b, the main image third division region 202c, and the main image fourth division region 202d.

[0079] The divided area may be specified, for example, using a context menu that is displayed by right-clicking on each divided area, or a divided area (for example, main image second divided area 202b) in which the cross-sectional image to be superimposed is displayed may be determined in advance, and the cross-sectional image displayed in that divided area may be superimposed.

[0080] The cross-sectional position and size of the cross-sectional image superimposed on the VR image are matched to the position and magnification of the VR image, for example, but the cross-sectional position and size of the cross-sectional image may be determined by other conditions. For example, the display conditions of the VR image may be matched to the display conditions of the cross-sectional image, or a predetermined condition may be used as a fixed condition. Of course, when the second check box 203p is checked and the display conditions are changed (slice forward, etc.) by a user operation in the divided area in which the cross-sectional image superimposed on the VR image is displayed, the display conditions of the cross-sectional image superimposed on the VR image may also be changed in conjunction with the display conditions.

[0081] The third check box 203q is a check box for instructing whether or not to automatically remeasure various measurement values ​​displayed in the measurement value display area based on a user operation and update the measurement values. Checking the third check box 203q controls to perform remeasurement, and unchecking it controls not to perform remeasurement.

[0082] When the third checkbox 203q is checked, control may be performed so that the display conditions cannot be changed to a range where re-measurement is not possible. For example, when various measurement values ​​are recalculated based on the position of the Ostium surface, control may be performed so that the Ostium surface (the cross-sectional position of the main image third divided area 202c in Figure 2) cannot be set anywhere other than in the vicinity of the left atrial appendage area (for example, a position a specified distance away from the center of gravity of the left atrial appendage or a position outside the left atrium (LA)).

[0083] The execution button group 203r includes a plurality of execution buttons for instructing the execution of extraction of a region of interest, which will be described later. Regions that can be extracted by the execution buttons include, for example, "left atrial appendage (LAA)", "Fossa", "Chamber", and "IVC / SVC". A user selects one of the execution buttons included in the execution button group 203r to extract the corresponding region of interest. When extracting a region of interest, the execution button corresponding to the region of interest that has already been extracted may be controlled to be in an unselectable (disabled) state.

[0084] The reset button 203s is a button that, when selected, designates that the display state of the main image display area 202 and the sub-image display area 204 is to be returned to a specific display state. Examples of specific states include the state when the application is launched, the state when an image is first displayed in the display area, and the state before a specific number of operations. The display conditions and display form in a specific state are recorded and restored to return the display state to a specific display state. The display conditions and form for a certain period of time may be recorded so that the display state after the previous or next operation can be continuously restored (forward and back buttons).

[0085] The setting screen display button 203t is a button for displaying a setting screen for setting display conditions of an image showing a three-dimensional area such as a rendering image such as a VR image or an SR image, or an area to be superimposed on a two-dimensional image. Fig. 6 is a diagram showing an example of the fifth image GA5. Fig. 7 is a diagram showing an example of the sixth image GA6.

[0086] When the setting screen display button 203t shown in Fig. 2 is selected, a fifth image GA5 (hereinafter also referred to as a setting screen) shown in Fig. 6 is displayed on the display 130. The user can perform various settings by operating the setting screen (fifth image GA5) displayed on the display 130.

[0087] The fifth image GA5 includes setting items related to the first item display KN1 indicated by the characters "Priority", the second item display KN2 indicated by the characters "Color", the third item display KN3 indicated by the characters "Transparency", the fourth item display KN4 indicated by the characters "VR", the fifth item display KN5 indicated by the characters "MPR", and the fifth item display KN6 indicated by the characters "Name". The display priority of the specified area (specified from the "Area Name" combo box on the right) is set in the first item display KN1. For example, the higher the area specified on the setting screen, the higher the display priority, and when multiple areas correspond to the same coordinates on the image, the area with the higher priority is displayed.

[0088] In the second item display KN2, a color is set to be assigned when the area of ​​the corresponding column (specified from the "Area Name" combo box on the right) is superimposed on a VR image or a cross-sectional image. For example, a color sample is displayed in the second item display KN2. For example, when the user selects an area showing the color of the sample, the display control function 144 displays a color map and an input box for RGB values, as in the sixth image GA6 shown in FIG. 7. The user can assign any color to the target area by selecting a color from the color map or inputting RGB values.

[0089] Additionally, the third item display KN3 sets the transparency of the superimposed display on the VR image or cross-sectional image of the corresponding area (specified from the "Area Name" combo box on the right). For example, the third item display KN3 can be specified in 1% increments from 0 to 99% using the slider bar, and at 0%, it is superimposed without any transparency (i.e. the background image is not visible).

[0090] Although not shown in Fig. 6 and Fig. 7, it is also possible to set display conditions such as saturation and brightness, or to set texture instead of color. Also, by selecting the link command button BT1 indicated by the characters "link" shown in the lower left of the figure, all transparency can be set together. Here, when linking, it is possible to control so that all transparency is set to the same value, or to control so that the overall transparency is increased or decreased while maintaining the relationship of the transparency value corresponding to each area at the time when the link command button BT1 is operated and the check box on the left is checked.

[0091] The fourth item display KN4 is a check box for specifying an area to be displayed on the VR image. The fifth item display KN5 is a check box for specifying an area to be displayed on the MPR image. Although not shown in Fig. 6 and Fig. 7, a button may be set to simultaneously check or uncheck all the check boxes of the fourth item display KN4 or the fifth item display KN5.

[0092] In the sixth item display KN6, the name of the area to be displayed is set according to the set priority and display conditions. For example, the user specifies an area using a combo box arranged in the column of the sixth item display KN6. Note that it is possible to control so that the same area cannot be set in multiple combo boxes (for example, if an area that has already been set in another combo box is specified in a certain combo box, control may be performed so that it cannot be specified, or the setting of the existing combo box may be canceled), or it may be possible to set the area but control so that the setting with the highest priority is used first.

[0093] The hide command button BT2 labeled "Close" is a button for hiding this setting screen. The reset command button BT3 labeled "Reset" is a button for returning the settings to their initial state. The timing for reflecting the display conditions set on this setting screen in each area may be immediately after each condition is set, or may be all at once after the hide command button BT2 is operated.

[0094] The analysis result display area 205 is a display area that displays the analysis results obtained by analyzing the medical image. The display control function 144 displays values ​​(measurement values) that indicate the characteristics (measurement items) of each area of ​​interest that are calculated based on each area of ​​interest identified and extracted by the image processing function 142, in the analysis result display area 205. The display control function 144 displays the calculated analysis results in the analysis result display area 205 as a list that indicates the relationship between the names of various measurement items and the measurement values.

[0095] The measurement items include, for example, morphological characteristics of the left atrial appendage (e.g., volume, surface area, sphericity, etc.), characteristics related to ostium (e.g., ostium area, circumference, major axis, minor axis, etc.), and characteristics related to the relationship between the left atrial appendage (LAA) and other anatomical structures (e.g., distance and accuracy between the ostium center and the Fossa center, etc.). The various measurement values ​​are calculated, for example, by a predetermined method based on known technology.

[0096] When the third checkbox 203q is checked and remeasurement is automatically performed, the display conditions related to calculation of the measurement value may be changed in the main image display area 202 and the sub-image display area 204. In this case, the display control function 144 displays the recalculated measurement value based on the change and updates the measurement value.

[0097] For example, a cross-sectional image corresponding to an Ostium surface is displayed in the main image third divided region 202c shown in Fig. 2, and various measurement values ​​related to Ostium are calculated based on the position of the Ostium surface. In this case, when the user changes the position of the cross-sectional image in the main image third divided region 202c using the browse function, rotation function, or the like of the device, the display control function 144 updates the display contents of the list with various measurement values ​​related to Ostium that are remeasured based on the changed cross-sectional position.

[0098] In the analysis result display area 205, the measurement values ​​are displayed in a list, but as long as the user can understand the relationship between various measurement items and the measurement values, the measurement values ​​may be displayed in any form, such as a graph instead of a list. In the analysis result display area 205, an output button 205a and a check box 205b are displayed.

[0099] When the output button 205a is selected, a table showing the relationship between the measurement values ​​of various measurement items and the phase or slice is output as a file in a format such as CSV to a storage area on a computer designated by the user. The check boxes 205b correspond to the respective measurement items. When the check boxes 205b are checked, the measurement points may be displayed in the main image display area 202 and the sub-image display area 204.

[0100] A controller 206 is displayed in the analysis result display area 205. The controller 206 is a controller for cine display. The controller 206 is set with a play button, a stop button, a speed increase button, a speed decrease button, a button to return to the start image, a button to advance to the final image, a button to advance to the image of the next cardiac phase, a button to advance to the image of the previous cardiac phase, and the like, and controls so that the functions assigned to the various buttons are executed when the user designates them by an operation such as clicking the mouse.

[0101] The display order during cine display may be determined based on the selection order of thumbnails, or based on the order of imaging dates and times obtained from a DICOM header or the like, or the order of cardiac phases set based on RR intervals. In addition, if the user gives some kind of instruction to the image during cine display (slice advance, translation, change of magnification ratio, change of gradation, measurement using various measurement functions, etc.), the controller may be hidden.

[0102] Next, a description will be given of the display conditions when images are displayed in the main image display area 202 and the sub-image display area 204. In the embodiment, four display conditions, a first display condition to a fourth display condition, are set as the display conditions. These display conditions will be described below in order.

[0103] First, the first display condition will be described. The first display condition is a display condition suitable for observing the left atrial appendage. Fig. 8 is a diagram showing an example of a seventh image GA7. The seventh image GA7 shows an example of an image of the main image display area 202 displayed under the first display condition. The main image display area 202 displayed under the first display condition, shown in the seventh image GA7, is divided into four divided areas of two rows and two columns.

[0104] The four divided areas include a first divided area under the first display condition 411, a second divided area under the first display condition 412, a third divided area under the first display condition 413, and a fourth divided area under the first display condition 414. The first divided area under the first display condition 411, the second divided area under the first display condition 412, the third divided area under the first display condition 413, and the fourth divided area under the first display condition 414 are displayed, for example, in the upper left, lower left, upper right, and lower right areas of the main image display area 202, respectively.

[0105] A VR image of the left atrial appendage region is displayed in the first display condition first divided region 411. The acquisition function 141 acquires the left atrial appendage region when the image processing function 142 extracts a region of interest. The generation function 143 generates a VR image of the left atrial appendage region based on the left atrial appendage region acquired by the acquisition function 141. The display control function 144 displays the VR image of the left atrial appendage region generated by the generation function 143 in the first display condition first divided region 411 of the display 130. The display control function 144 sets, for example, the center of gravity position of the left atrial appendage region to be located at the center of the first display condition first divided region 411, and displays it in red, for example, with a magnification rate of 100%.

[0106] A cross-sectional image corresponding to the Ostium plane is displayed in the first display condition second divided region 412. The display control function 144 sets the center position of Ostium to be located at the center of the first display condition second divided region 412, sets the magnification rate to 150%, the window level to 300, and the window width to 650, and displays the position corresponding to the left atrial appendage region in red.

[0107] Two cross-sectional images perpendicular to the Ostium plane and perpendicular to each other are displayed in the third divided region 413 under first display condition and the fourth divided region 414 under first display condition. As with the display for the second divided region 412 under first display condition, the display control function 144 sets the center position of Ostium to be located at the center of the third divided region 413 under first display condition and the fourth divided region 414 under first display condition, sets the magnification to 150%, the window level to 300, and the window width to 650, and displays the position corresponding to the left atrial appendage region in red.

[0108] That is, the display control function 144 displays three orthogonal cross sections based on the Ostium plane in the second divided region 412 under the first display condition, the third divided region 413 under the first display condition, and the fourth divided region 414 under the first display condition. Note that the Ostium plane is specified by a known method based on the boundary information of the left atrial appendage region acquired by the acquisition function 141 when the region of interest is extracted by the image processing function 142, for example.

[0109] Next, the second display condition will be described. The second display condition is a display condition suitable for observing the Fossa plane. Fig. 9 is a diagram showing an example of an eighth image GA8. The eighth image GA8 shows an example of an image of the main image display area 202 displayed under the second display condition. The main image display area 202 displayed under the second display condition shown in the eighth image GA8 is divided into four divided areas of two rows and two columns.

[0110] The four divided areas include a first divided area under the second display condition 421, a second divided area under the second display condition 422, a third divided area under the second display condition 423, and a fourth divided area under the second display condition 424. The first divided area under the second display condition 421, the second divided area under the second display condition 422, the third divided area under the second display condition 423, and the fourth divided area under the second display condition 424 are displayed, for example, in the upper left, lower left, upper right, and lower right areas of the main image display area 202, respectively.

[0111] In the second display condition first divided region 421, VR images of both the left atrial appendage region and the fossa ovalis region (hereinafter, fossa ovalis region) are displayed. The acquisition function 141 acquires the left atrial appendage region and the fossa ovalis region when the image processing function 142 extracts a region of interest. The generation function 143 generates a VR image of the left atrial appendage region and the fossa ovalis region acquired by the acquisition function 141. The display control function 144 displays the VR image of the left atrial appendage region and the fossa ovalis region generated by the generation function 143 in the second display condition first divided region 421 of the display 130. For example, the display control function 144 sets the center positions of the left atrial appendage region and the fossa ovalis region to be located at the center of the second display condition first divided region 421, and displays the left atrial appendage region in red and the fossa ovalis region in yellow with a magnification of 100%, for example.

[0112] A cross-sectional image corresponding to the Fossa plane is displayed in the second display condition third divided region 423. The display control function 144 sets the center of gravity of the fossa ovalis region to be located at the center of the second display condition third divided region 423, sets the magnification rate to 156%, the window level to 300, and the window width to 650, and displays the position corresponding to the left atrial appendage region in red and the position corresponding to the fossa ovalis region in yellow.

[0113] Two cross-sectional images perpendicular to the Fossa plane and perpendicular to each other are displayed in the second divided region 422 under the second display condition and the fourth divided region 424 under the second display condition. The display control function 144 sets the center of gravity of the fossa ovalis region to be located at the center of the second divided region 422 under the second display condition and the fourth divided region 424 under the second display condition, as in the display for the third divided region 423 under the second display condition, sets the magnification to 156%, the window level to 300, and the window width to 650, and displays the position corresponding to the left atrial appendage region in red and the position corresponding to the fossa ovalis region in yellow.

[0114] That is, the display control function 144 displays three orthogonal cross sections based on the Fossa plane in the second divided region 422 under the second display condition, the third divided region 423 under the second display condition, and the fourth divided region 424 under the second display condition. Note that the Fssa plane is specified by a known method based on the respective regional information of the left atrial appendage region, the fossa ovalis region, and the inferior vena cava region acquired by the acquisition function 141 when the region of interest is extracted by the image processing function 142.

[0115] Next, the third display condition will be described. The third display condition is a display condition suitable for 4Camber observation. Fig. 10 is a diagram showing an example of a ninth image GA9. The ninth image GA9 shows an example of an image of the main image display area 202 displayed under the third display condition. The main image display area 202 displayed under the third display condition, shown in the ninth image GA9, is divided into four divided areas of two rows and two columns.

[0116] The four divided areas include a first divided area under the third display condition 431, a second divided area under the third display condition 432, a third divided area under the third display condition 433, and a fourth divided area under the third display condition 434. The first divided area under the third display condition 431, the second divided area under the third display condition 432, the third divided area under the third display condition 433, and the fourth divided area under the third display condition 434 are displayed, for example, in the upper left, lower left, upper right, and lower right areas of the main image display area 202, respectively.

[0117] In the third display condition first division region 431, VR images corresponding to all regions of the left atrium, right atrium, left ventricle (LV), and right ventricle (RV) which are 4Camber are displayed. The acquisition function 141 acquires the regions of the left atrium, right atrium, left ventricle, and right ventricle when the image processing function 142 extracts the region of interest. The generation function 143 generates a VR image of the region acquired by the acquisition function 141. The display control function 144 displays the VR image generated by the generation function 143 in the third display condition first division region 431 on the display 130. The display control function 144 sets the positions of the centers of the left atrium, right atrium, left ventricle, and right ventricle to be located at the center of the third display condition first division region 431, and displays the left atrium region in blue, the right atrium region in red, the left ventricle region in yellow, and the right ventricle region in green, for example, with a magnification rate of 100%. The color of each region may be other colors.

[0118] A cross-sectional image corresponding to a short-axis view of the heart is displayed in the second division area 432 under the third display condition, a cross-sectional image corresponding to a two-chamber view is displayed in the third division area 433 under the third display condition, and a cross-sectional image corresponding to a four-chamber view is displayed in the fourth division area 434 under the third display condition. For example, for the short-axis view, the display control function 144 sets the center of gravity of the left ventricle to be located at the center of the division area of ​​the second division area 432 under the third display condition. For example, for the two-chamber view, the display control function 144 sets the center of gravity of the left ventricle and the center of gravity of the left atrium to be located at the center of the third division area 433 under the third display condition. For example, for the four-chamber view, the display control function 144 sets the center of gravity of the entire area of ​​the left atrium, right atrium, left ventricle, and right ventricle to be located at the center of the fourth division area 434 under the third display condition.

[0119] The display control function 144 displays these images, for example, with a magnification of 135%, a window level of 300, and a window width of 650, and displays the left atrium region in blue, the right atrium region in red, the left ventricle region in yellow, and the right ventricle region in green. The colors of each region may be other colors. Note that the cross sections corresponding to the short-axis image, the two-chamber image, and the four-chamber image are identified by a known method from the region information of the left atrium, right atrium, left ventricle, and right ventricle acquired by the acquisition function 141 when the region of interest is extracted by the image processing function 142.

[0120] Next, the fourth display condition will be described. Fig. 11 is a diagram showing an example of a tenth image GA10. The tenth image GA10 shows an example of an image of the main image display area 202 displayed under the fourth display condition. The main image display area 202 displayed under the fourth display condition shown in the tenth image GA10 is divided into a total of three divided areas, two on the right side and one on the left side.

[0121] The three divided regions include a first divided region 441 under the fourth display condition, a second divided region 442 under the fourth display condition, and a third divided region 443 under the fourth display condition. The first divided region 441 under the fourth display condition and the second divided region 442 under the fourth display condition are of the same size, and the third divided region 443 under the fourth display condition is twice as large vertically as the first divided region 441 under the fourth display condition and the second divided region 442 under the fourth display condition. The first divided region 441 under the fourth display condition, the second divided region 442 under the fourth display condition, and the third divided region 443 under the fourth display condition are displayed in the entire upper left, lower left, and right portions of the main image display region 202, respectively.

[0122] A cross-sectional image corresponding to an axial image is displayed in the first division area 441 under the fourth display condition. A cross-sectional image corresponding to a coronal image is displayed in the second division area 442 under the fourth display condition. A cross-sectional image corresponding to a sagittal image is displayed in the third division area 443 under the fourth display condition.

[0123] The display control function 144 sets the center position of each of the transverse, coronal, and sagittal images to be located at the center of each divided area, and displays them in the main image display area 202 with a magnification of 123%, a window level of 300, and a window width of 2758.

[0124] The above display conditions are merely examples, and the display conditions may be determined in any way as long as they can be set based on the CT image to be processed and information on each anatomical structure identified when extracting the region of interest. However, when the display region is divided into multiple regions, it is desirable to set the display conditions so that each divided region displays a related image.

[0125] Next, the processing of the medical image display device of the embodiment will be described. Fig. 12 is a flowchart showing an example of processing in the medical image display device 100 of the embodiment. Fig. 12 shows processing in the medical image display device 100 of the embodiment when a user performs interpretation while looking at the display 130.

[0126] First, the medical image display device 100 acquires a medical image of a subject, for example, a CT image, from an in-hospital image database or an examination device by the acquisition function 141 (step S101). The image database is included in, for example, the medical image display device 100 or the medical image storage device 20. The image database may be included in another device, a cloud, or the like.

[0127] The type of medical image may be any type of image that stores morphological information of the three-dimensional anatomical structure of the target biological tissue. The medical image may be a CT image, or may be a medical image obtained by other imaging devices such as an ultrasound image, an MRI image, an X-ray image, a PET image, or a SPECT image, or may be a four-dimensional image obtained by capturing a plurality of such images in the time direction.

[0128] Next, the image processing function 142 identifies a region (hereinafter, referred to as a region of interest) indicating a biological organ of interest in the image acquired by the acquisition function 141 (step S103). The region of interest may be a plurality of regions. The region of interest for which coordinate information is acquired may be determined in advance, or a region selected by the user by operating the execution button group 203r may be set as the region of interest.

[0129] In the embodiment, the coordinate information of each pixel in the CT image representing the left atrial appendage, the fossa ovalis, the left atrium, the left ventricle, the right atrium, the right ventricle, and the inferior vena cava on the CT image is extracted as the region of interest. As a specific process for identifying the region of interest, the user may manually specify the position of the region of interest using the input interface 120, or the region of interest may be identified based on the anatomical structure depicted in the CT image by a known region extraction technique.

[0130] Known region extraction techniques include, for example, Otsu's binarization method based on CT values, region growing method, snake method, graph cut method, and mean shift method. The processing here may be any method that can identify the region of interest from the medical image, and is not limited to the method of identifying the region. For example, the region of interest may be identified using a shape model of the region of interest that is constructed based on learning data prepared in advance using machine learning techniques (including deep learning).

[0131] The timing to start the process of extracting the region of interest in step S103 may be explicitly instructed by the user, or may be the time when the image acquisition in step S101 is completed. If the timing to start the extraction of the region of interest is the time when the image acquisition is completed, the extraction of the region of interest may be started automatically.

[0132] When multiple attention regions are specified, the attention regions whose extraction starts automatically and the attention regions whose extraction starts by the user's instruction may be set separately. This setting may be, for example, predetermined, or may be set by the user using the input interface 120.

[0133] The timing for ending the process of extracting the regions of interest in step S103 (the timing for moving to step S107) may be when all regions of interest have been specified, or when any or any number of regions of interest have been specified. Alternatively, the process may move to step S107 even before any region of interest has been specified.

[0134] If the process proceeds to step S107 before one or more areas of interest have been identified, the processing device that performs the processing from step S107 onwards (e.g., some threads in a CPU or GPU) and the processing device that performs the processing of the remaining step S103 (e.g., other some threads in a CPU or GPU) may be separated and the processing may be carried out simultaneously.

[0135] When proceeding to step S107 when one or more attention areas have not been specified, the progress status of step S103, for example, the status of which attention areas have been specified and which attention areas have not been specified, may be notified to the user by displaying it on the display 130 by the display control function 144, or it may not be displayed. Alternatively, it may be possible to start a process of specifying a different new attention area based on a user's instruction at any timing after step S107. In other words, instead of executing step S103 after step S101 as in the flow shown in FIG. 12, it may be possible to start the process of step S103 at any timing after step S107 if there is a user's instruction.

[0136] After identifying the region of interest, the image processing function 142 processes the image acquired by the acquisition function 141 (step S105). The generation function 143 generates a cross-sectional image based on the image processing by the image processing function 142 (step S105). At this time, the display control function 144 causes the display 130 to display the cross-sectional image generated by the generation function 143 (step S105). The image processing function 142 executes image processing sequentially, and the generation function 143 generates a cross-sectional image every time the image processing function 142 performs image processing.

[0137] Next, the switching function 146 acquires the progress status of the process (segmentation process, hereinafter, the identification process) for identifying the region of interest in step S105 (step S107). The progress status of the identification process is a situation in which, when identification of the positions of multiple regions of interest is set in step S103, identification of the positions of some regions of interest has been completed and others has not been completed. In other words, the image processing function 142 acquires information on whether the process for identifying each region of interest has been completed. When the identification process of each region of interest is composed of multiple processing steps, the switching function 146 may acquire, as the progress status of the identification process, not only the completion of the identification process but also information on the processing steps that have been completed at the time of execution of step S103.

[0138] Next, the switching function 146 acquires the display states of the main image display area 202 and the sub-image display area 204 (step S109). That is, the switching function 146 acquires the display conditions of the images displayed in the main image display area 202 and the sub-image display area 204 when the process of step S109 is executed. The switching function 146 may acquire the display conditions themselves in each display area as the display conditions, or may acquire only identifiers indicating the types of the display conditions, such as the first to fourth display conditions, as described above.

[0139] Next, the switching function 146 acquires the user's operation history for the main image display area 202 and the sub-image display area 204 (step S111). The user's operation history for the main image display area 202 and the sub-image display area 204 refers to the history of operations performed on the images displayed on the display 130 by operating the input interface 120 for the images displayed in the main image display area 202 and the sub-image display area 204 at the time of executing step S103, using the functions of the split instruction button 203b, the first operation icon 203c to the fifth operation icon 203g (operation system icons), the first drawing and measurement icon 203h to the seventh drawing and measurement icon 203n (drawing and measurement system icons), and the first check box 203o.

[0140] The image processing function 142 performs control so that the operation history executed during the processing of step S103 is recorded in advance in the memory 150 or the like at the time of a predetermined operation. The image processing function 142 may store all operation histories by the user in the memory 150, or may store only a specific operation history that is determined in advance. The image processing function 142 may store in the memory 150 only the presence or absence of an operation history, that is, whether or not an operation has been performed even once.

[0141] Next, the switching function 146 determines whether or not to switch the display conditions of the main image display area 202 and the sub-image display area 204 (step S113). Whether or not to switch the display conditions of the display areas is determined by referring to the progress status acquired in step S103, the display state acquired in step S109, and the operation history acquired in step S111 in the display condition switching determination table 152.

[0142] Here, the display condition switching determination table 152 will be described. Fig. 13 is a diagram showing an example of the contents of the display condition switching determination table 152. The display condition switching determination table 152 shows the relationship between the conditions for switching the main image display area 202 and the sub-image display area 204 and the display conditions of the switched main image display area 202 and the sub-image display area 204. For example, when there is no operation history for the display area, the switching function 146 maintains the display conditions of the display unit, and changes the display conditions on the condition that there is the operation history for the display area.

[0143] The "circled numbers" in Fig. 13 indicate the display conditions, with "circled number 1" indicating the first display condition, "circled number 2" indicating the second display condition, "circled number 3" indicating the third display condition, and "circled number 4" indicating the fourth display condition. The display condition switching determination table 152 shown in Fig. 13 is an example. The display condition switching determination table may be in other forms. The display condition switching determination table may be, for example, a table for all combinations, a table prepared only for combinations that need to be changed, or a table that has no settings for combinations that do not need to be changed.

[0144] For example, when the process of identifying the left atrial appendage is completed but the process of identifying the 4Camber and Fossa planes is not completed, no cross-sectional images are displayed in either the main image display area 202 or the sub-image display area 204, and there is no operation history by the user. In this case, the switching function 146 determines that the display conditions have been changed and that an image of the first display condition is to be displayed in the main image display area 202 and an image of the fourth condition is to be displayed in the sub-image display area 204.

[0145] Also, for example, assume that the process of identifying the left atrial appendage and 4Camber has been completed but the process of identifying the Fossa plane has not been completed, an image is displayed in the main image display area 202 under the first display condition, and in the sub-image display area 204 under the fourth display condition, there is an operation history for the main image display area 202, and there is no operation history for the sub-image display area 204. In this case, the switching function 146 determines that the display conditions are not changed and that the image under the first display condition is to be left displayed in the main image display area 202 and the image under the fourth display condition is to be left displayed in the sub-image display area 204.

[0146] If it is determined in step S113 that the display is not to be switched, the switching function 146 skips the process of step S115 and advances the process to step S117. If the switching function 146 determines that the display is to be switched, the display control function 144 causes the display 130 to display the first image GA1 shown in FIG. 2 under the switched display conditions (step S115).

[0147] The display control function 144 may display a warning to the user regarding the change before switching the display of the main image display area 202 and the sub-image display area 204. The display control function 144 may also allow the user to set whether or not to switch using a pop-out screen or the like so that the user can determine whether or not to switch before switching the display of the main image display area 202 and the sub-image display area 204. In this case, the display control function 144 may display on the display 130 the text "Do you want to change?" along with buttons for selecting "OK or NG," or the like.

[0148] Next, the switching function 146 judges whether the user's interpretation is completed or not (step S117). The judgment of whether the user's interpretation is completed or not may be performed in any manner. For example, this judgment may be made by the user designating an interpretation end button (not shown) provided on the input interface 120.

[0149] If the switching function 146 determines not to end the interpretation, the process returns to step S 107. On the other hand, if the switching function 146 determines not to end the interpretation, the medical image display device 100 ends the process shown in FIG.

[0150] Next, an example of a flow for switching the display conditions of the main image display area 202 and the sub-image display area 204 will be described with reference to the display condition switching determination table 152. The display condition switching determination table 152 is a table created such that the priority of specific processes to be displayed in the main image display area 202 is set in the order of high to low for the left atrial appendage, 4Camber, and Fossa surface.

[0151] For example, when the left atrial appendage identification process is completed with no cross-sectional images displayed in either the main image display area 202 or the sub-image display area 204, the main image display area 202 is displayed under the first display condition. Here, the 4Camber and Fossa planes have not yet been identified, and the sub-image display area 204 is displayed under the fourth condition. The user performs image interpretation while viewing the main image display area 202 and the sub-image display area 204.

[0152] Furthermore, when the left atrial appendage and 4Camber are specified but the Fossa plane is not specified, if there is no operation history in the sub-image display area 204, the sub-image display area 204 is switched to and displayed under the third display condition, regardless of the presence or absence of an operation history in the main image display area 202. On the other hand, if there is an operation history in the sub-image display area 204, the display conditions of neither the main image display area 202 nor the sub-image display area 204 are switched.

[0153] Furthermore, when the left atrial appendage, 4Camber, and Fossa plane have all been identified, if there is no operation history for the main image display area 202 displayed under the first display condition, the main image display area 202 is switched to the second display condition. On the other hand, if there is an operation history for the main image display area 202 displayed under the first display condition, the display condition of the sub-image display area 204 may be switched based on the display condition and operation history in the sub-image display area 204.

[0154] In addition, when the main image display area 202 is displayed in the second display condition and there is a user operation history of the main image display area 202, the sub-image display area 204 may be displayed in the first display condition. When the Fossa plane specification process is completed, if the main image display area 202 is displayed in a display condition other than the first display condition, the main image display area 202 may be displayed in the third display condition, and even if the main image display area 202 is displayed in the first display condition and there is no user operation history, the main image display area 202 may be displayed in the third display condition. In this case, the sub-image display area 204 may be displayed in the first display condition.

[0155] When the main image display area 202 is displayed under the first display condition and there is a user operation history of the main image display area 202, the sub-image display area 204 may be displayed under the second display condition. When the 4Chamber specification process is completed, the display condition of the main image display area 202 is not changed, and when the sub-image display area 204 is displayed under the first display condition, the display of the sub-image display area 204 is not switched, and when the sub-image display area 204 is displayed under the second display condition, the sub-image display area 204 is displayed under the third display condition only if there is no operation history, and when the sub-image display area 204 is displayed under the fourth display condition, the sub-image display area 204 is displayed under the fourth display condition even if there is a user operation history. These are examples, but by setting only the priority order of the attention area, it is also possible to set the relationship between various states and histories and the display conditions of each display area after the change.

[0156] Instead of or in addition to the display condition switching determination table 152, a priority order may be set for each of the left atrial appendage, 4Camber, and Fossa surface, and the switching function 146 may determine whether or not to change the display conditions based on the set priority order. For example, when the left atrial appendage has the highest priority order, the 4Camber has the next highest priority order, and the Fossa surface has the lowest priority order, a determination as to whether or not to change the display conditions is made in the same manner as the determination shown in the display condition switching determination table 152.

[0157] The relationship between the conditions for changing the main image display area 202 and the sub-image display area 204 and the display conditions of the changed main image display area 202 and the sub-image display area 204 is determined, for example, as follows.

[0158] When the process of identifying the left atrial appendage is completed, the left atrial appendage is displayed in the main image display area 202 under the first display condition regardless of the display state of the main image display area 202. The first display condition is a display condition suitable for observing the left atrial appendage. At this time, if the main image display area 202 is displayed under the second display condition suitable for observing the Fossa plane, which has the second highest priority, the sub-image display area 204 is displayed under the second display condition.

[0159] The priority order for each of the left atrial appendage, 4Camber, and Fossa surface may be set by the user operating a GUI displayed on the display 130. Fig. 14 is a diagram showing an example of a GUI used for determining the priority order. For example, as shown in the upper part of Fig. 14, a first combo box CB1 to a third combo box CB3 may be provided, and the user may operate the input interface 120 to display "LAA", "Fossa", or "4Chamber" in any of the first combo box CB1 to the third combo box CB3, thereby allowing the user to set the priority order for each of the left atrial appendage, 4Camber, and Fossa surface.

[0160] Also, as shown in the middle of Fig. 14, the degree of the first priority and the second priority, and the second priority and the third priority may be adjusted by specifying and moving the second combo box 2 up and down. The degree of the first priority and the second priority, and the second priority and the third priority may be represented by a symbol X indicating the degree of the degree, as shown in the lower part of Fig. 14.

[0161] The medical image display device of the embodiment includes a switching function 146 that switches the display mode of the display 130 based on the processing state of image processing, the display state of the display area, and the operation history of the user. This reduces the effort required for switching and displaying multiple images when the user performs processing such as image interpretation.

[0162] Although the medical image display device 100 of the embodiment has been described as displaying a 2D or 3D medical image, the medical image display device 100 may be capable of displaying a 4D image such as a 4DCT image by cine display, etc. When displaying a 3D image by cine display, the medical image display device 100 may switch the display screen based not only on the progress of the process of identifying the region of interest for the image of the cardiac phase at the time of switching, but also on the process of identifying the region of interest at the time of switching.

[0163] When the segmentation process is not completed at the switching destination, the switching to the phase where the segmentation process is not completed may not be performed. When the segmentation process is completed at the switching destination, the medical image display device 100 may display the cross-sectional image of the switching source.

[0164] According to at least one of the embodiments described above, by having an acquisition unit that acquires medical images, an image processing unit that sequentially performs multiple types of image processing on the medical images, a generation unit that generates cross-sectional images based on the image processing, a display control unit that displays the cross-sectional images in a display area of ​​a display unit, a reception unit that receives user operations on the cross-sectional images, and a display switching unit that switches the display mode of the display unit based on the processing state of the image processing, the display state of the display area, and the user's operation history, it is possible to reduce the effort of switching between and displaying multiple images.

[0165] Although some embodiments have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included in the scope of the invention and its equivalents as described in the claims, as well as in the scope and spirit of the invention. [Explanation of symbols]

[0166] 10 Medical imaging diagnostic equipment 20 Medical image storage device 100 Medical image display device 110 Communication Interface 120 Input Interface 130 Display 140 Processing circuit 141 Acquisition Function 142 Image Processing Functions 143 Generation function 144 Display control function 145 Reception Function 146 Switching Function 150 Memory 152 Display condition switching judgment table

Claims

1. An acquisition unit for acquiring medical images; an image processing unit that sequentially performs a plurality of types of image processing on the medical image; A generation unit that generates a cross-sectional image based on the image processing; a display control unit that displays the cross-sectional image in a display area of ​​a display unit; a reception unit that receives a user's operation on the cross-sectional image; a display switching unit that switches a display mode of the display unit based on a processing state of the image processing, a display state of the display area, and an operation history of the user. Medical image display device.

2. The display control unit causes the display unit to display the plurality of display areas; The display switching unit switches the cross-sectional images to be displayed in the display areas based on a display state of each of the plurality of display areas. The medical image display device according to claim 1 .

3. The generating unit generates a plurality of cross-sectional images related to a processing result of the image processing based on the processing result of the image processing, The display control unit causes a plurality of cross-sectional images to be displayed in the display area.

3. The medical image display device according to claim 1.

4. The display control unit causes the display unit to display the cross-sectional image based on any one of a plurality of display conditions. The medical image display device according to claim 1 .

5. The display conditions include at least one of a first display condition suitable for observing the left atrial appendage, a second display condition suitable for observing the Fossa plane at a predetermined position of the subject, and a third display condition suitable for observing 4 cambers including the left atrium, the right atrium, the left ventricle, and the right ventricle. The medical image display device according to claim 4 .

6. the first display condition is to display at least one of a VR image of a left atrial appendage region, a cross-sectional image corresponding to an Ostium plane at a predetermined position of a subject, and two cross-sectional images perpendicular to the Ostium plane and perpendicular to each other; The medical image display device according to claim 5 .

7. The second display condition is to display at least one of a VR image of both the left atrial appendage region and the fossa ovalis region, a cross-sectional image corresponding to the Fossa plane, or two cross-sectional images perpendicular to the Fossa plane and perpendicular to each other. The medical image display device according to claim 5 .

8. the third display condition is to display at least one of a 4-Camber VR image, a cross-sectional image corresponding to a short-axis view of the heart, a cross-sectional image corresponding to a two-chamber view, and a cross-sectional image corresponding to a four-chamber view; The medical image display device according to claim 5 .

9. the display conditions include at least one of a cross-sectional position, a magnification ratio, a window level, and a window width when displaying a cross-sectional image; The medical image display device according to claim 4 .

10. the display control unit causes the cross-sectional image to be displayed in a first display area on the display unit and a second display area having a display area smaller than that of the first display area. The medical image display device according to claim 1 .

11. the display control unit causes at least one of an Ostium plane or a Fossa plane cross-sectional image at a predetermined position of the subject to be displayed on the display unit as the cross-sectional image; The medical image display device according to claim 1 .

12. the display switching unit maintains a display condition of the display unit when there is no operation history for the display area. The medical image display device according to claim 4 .

13. the display switching unit changes the display condition on condition that the operation history exists for the display area. The medical image display device according to claim 4 .

14. The display switching unit switches the display conditions based on a predetermined priority order. The medical image display device according to claim 4 .

15. The priority can be specified by a user. The medical image display device according to claim 14.

16. The display control unit further causes the display unit to display a reference indicating a cross-sectional position of the image. The medical image display device according to claim 1 .

17. The computer Acquire medical images; Sequentially performing a plurality of types of image processing on the medical image; generating a cross-sectional image based on the image processing; Displaying the cross-sectional image in a display area of ​​a display unit; accepting a user's operation on the cross-sectional image; switching the cross-sectional image to be displayed in the display area based on a processing state of the image processing, a display state of the display area, and an operation history of the user; A medical image display control method.

18. On the computer, Acquire medical images; Sequentially performing a plurality of types of image processing on the medical image; generating a cross-sectional image based on the image processing; Displaying the cross-sectional image in a display area of ​​a display unit; accepting a user's operation on the cross-sectional image; switching the cross-sectional image to be displayed in the display area based on a processing state of the image processing, a display state of the display area, and an operation history of the user; program.

Images