Medical image processing apparatus, medical image processing method, and program

Abstract

To assist in grasping information on medical image data.SOLUTION: A medical image processing device in an embodiment includes an acquisition unit, a reception unit and a display control unit. The acquisition unit acquires medical image data for an analyte. The reception unit receives designation of measurement items related to the medical image data. The display control unit controls a display mode of the medical image data based on the designated measurement items.SELECTED DRAWING: Figure 4

Description

Technical Field

[0001] The embodiments disclosed in this specification and the drawings relate to a medical image processing apparatus, a medical image processing method, and a program.

Background Art

[0002] Medical image data can provide useful information to users in various aspects such as diagnosis, treatment planning, and determination of treatment effects. For example, based on medical image data, a user can refer to any cross-section in a patient's body or refer to measurement values obtained by analyzing the medical image data.

[0003] Here, the information obtained from medical image data is enormous, and it is not easy for users to grasp all of it. For example, when the medical image data is three-dimensional data, it is possible to display various cross-sections with changed positions and angles. Also, there are a wide variety of measurement values obtained by analyzing medical image data. Further, in order to execute the display and analysis of medical image data as desired, the user needs to input various instructions.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Patent Document 2

Patent Document 3

Patent Document 4

Summary of the Invention

Problems to be Solved by the Invention

[0005] One of the problems to be solved by the embodiments disclosed in this specification and the drawings is to assist in grasping information regarding medical image data. However, the problems to be solved by the embodiments disclosed in this specification and the drawings are not limited to the above problems. It is also possible to position the problems corresponding to the respective effects of each configuration shown in the embodiments described later as other problems.

Means for Solving the Problems

[0006] The medical image processing apparatus according to the embodiment includes an acquisition unit, a reception unit, and a display control unit. The acquisition unit acquires medical image data regarding a subject. The reception unit receives a designation of a measurement item regarding the medical image data. The display control unit controls a display mode of the medical image data based on the designated measurement item.

Brief Description of the Drawings

[0007]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6A

Figure 6B

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13A

Figure 13B

Figure 13C

Figure 14A

Figure 14B

Figure 14C

Figure 15

Figure 16A

Figure 16B

Figure 16C

Figure 17

Figure 18A

Figure 18B

Figure 18C

Figure 19

Figure 20

Figure 21

Figure 22

Figure 23

Figure 24

Figure 25

Figure 26

Figure 27A

Figure 27B

Figure 27C

Figure 28A

Figure 28B

DETAILED DESCRIPTION OF THE INVENTION

[0008] Hereinafter, embodiments of a medical image processing apparatus, a medical image processing method, and a program will be described in detail with reference to the accompanying drawings.

[0009] (First Embodiment) In this embodiment, a medical image processing system 1 including a medical image processing apparatus 20 will be described as an example. For example, as shown in FIG. 1, the medical image processing system 1 includes a medical image diagnostic apparatus 10, a medical image processing apparatus 20, and an image storage apparatus 30. FIG. 1 is a block diagram showing an example of the configuration of the medical image processing system 1 according to the first embodiment. The medical image diagnostic apparatus 10, the medical image processing apparatus 20, and the image storage apparatus 30 are connected to each other via a network NW.

[0010] As long as they can be connected via the network NW, the locations where the respective apparatuses included in the medical image processing system 1 are installed are arbitrary. For example, the image storage apparatus 30 may be installed in a hospital different from the hospital where the medical image diagnostic apparatus 10 and the medical image processing apparatus 20 are installed, or in another facility. That is, the network NW may be configured by a closed local network within the facility, or may be a network via the Internet.

[0011] The medical image diagnostic apparatus 10 is an apparatus that captures a subject and collects medical image data. Note that various types of data handled in this specification are typically digital data. The medical image diagnostic apparatus 10 is, for example, a medical modality such as an X-ray diagnostic apparatus, an X-ray CT (Computed Tomography) apparatus, an MRI (Magnetic Resonance Imaging) apparatus, an ultrasonic diagnostic apparatus, a SPECT (Single Photon Emission Computed Tomography) apparatus, or a PET (Positron Emission computed Tomography) apparatus. Although a single medical image diagnostic apparatus 10 is shown in FIG. 1, the medical image processing system 1 may include a plurality of medical image diagnostic apparatuses 10. Further, the medical image processing system 1 may include a plurality of types of medical image diagnostic apparatuses 10. For example, the medical image processing system 1 may include an X-ray CT apparatus and an MRI apparatus as the medical image diagnostic apparatuses 10.

[0012] The image storage device 30 is an image database that stores the medical image data collected by the medical image diagnostic apparatus 10. For example, the image storage device 30 includes an arbitrary storage device inside or outside the device, and manages the medical image data acquired from the medical image diagnostic apparatus 10 via the network NW in the form of a database. For example, the image storage device 30 is a server of a PACS (Picture Archiving and Communication System). Further, the image storage device 30 may be realized by a server group (cloud) connected to the medical image processing system 1 via the network NW.

[0013] The medical image processing apparatus 20 is an apparatus that acquires the medical image data collected by the medical image diagnostic apparatus 10 and performs various processes described later. For example, as shown in FIG. 1, the medical image processing apparatus 20 includes a memory 21, a display 22, an input interface 23, and a processing circuit 24.

[0014] The memory 21 is implemented by, for example, a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory, a hard disk, an optical disk, or the like. For example, the memory 21 stores medical image data. Further, the memory 21 stores a program for the circuits included in the medical image processing apparatus 20 to realize their functions.

[0015] The display 22 is, for example, a liquid crystal display or a CRT (Cathode Ray Tube) display. The display 22 may be a desktop type, or may be configured as a tablet terminal or the like that can communicate wirelessly with the main body of the medical image processing apparatus 20. The control of the display on the display 22 will be described later.

[0016] The input interface 23 receives various input operations from the user, converts the received input operations into electrical signals, and outputs them to the processing circuit 24. For example, the input interface 23 is realized by a mouse, a keyboard, a trackball, a switch, a button, a joystick, a touch pad that performs an input operation by touching the operation surface, a touch screen in which the display screen and the touch pad are integrated, a non-contact input circuit using an optical sensor, a voice input circuit, or the like. Note that the input interface 23 may be configured as a tablet terminal or the like that can communicate wirelessly with the main body of the medical image processing apparatus 20. Further, the input interface 23 may be a circuit that receives an input operation from the user by motion capture. For example, the input interface 23 can receive the body movement, the line of sight, etc. of the user as input operations by processing a signal acquired via a tracker or an image collected about the user. Further, the input interface 23 is not limited to only those provided with physical operation components such as a mouse and a keyboard. For example, an electrical signal processing circuit that receives an electrical signal corresponding to an input operation from an external input device provided separately from the medical image processing apparatus 20 and outputs this electrical signal to the processing circuit 24 is also included in the examples of the input interface 23.

[0017] The processing circuit 24 controls the operation of the entire medical image processing apparatus 20 by executing a control function 24a, an acquisition function 24b, a reception function 24c, an extraction function 24d, a calculation function 24e, and a display control function 24f. The acquisition function 24b is an example of an acquisition unit. The reception function 24c is an example of a reception unit. The extraction function 24d is an example of an extraction unit. The calculation function 24e is an example of a calculation unit. The display control function 24f is an example of a display control unit.

[0018] For example, the processing circuit 24 reads out and executes a program corresponding to the control function 24a from the memory 21, and based on various input operations received from the user via the input interface 23, controls various functions such as the acquisition function 24b, the reception function 24c, the extraction function 24d, the calculation function 24e, and the display control function 24f.

[0019] Also, the processing circuit 24 reads out and executes a program corresponding to the acquisition function 24b from the memory 21 to acquire medical image data regarding the subject. For example, the acquisition function 24b receives the medical image data imaged by the medical image diagnostic apparatus 10 via the network NW and stores it in the memory 21. Here, the acquisition function 24b may directly acquire the medical image data from the medical image diagnostic apparatus 10, or may acquire the medical image data via another apparatus such as the image storage device 30.

[0020] Also, the processing circuit 24 receives the specification of measurement items regarding medical image data by reading out and executing a program corresponding to the reception function 24c from the memory 21. Also, the processing circuit 24 extracts a predetermined region included in the medical image data by reading out and executing a program corresponding to the extraction function 24d from the memory 21. Also, the processing circuit 24 calculates a measurement value for the specified measurement item by reading out and executing a program corresponding to the calculation function 24e from the memory 21. Also, the processing circuit 24 controls the display on the display 22 by reading out and executing a program corresponding to the display control function 24f from the memory 21. Details of the processing by the reception function 24c, the extraction function 24d, the calculation function 24e, and the display control function 24f will be described later.

[0021] In the medical image processing apparatus 20 shown in FIG. 1, each processing function is stored in the memory 21 in the form of a program executable by a computer. The processing circuit 24 is a processor that realizes the functions corresponding to the respective programs by reading out and executing the programs from the memory 21. In other words, the processing circuit 24 in the state of having read out the program has the functions corresponding to the read-out program.

[0022] In FIG. 1, the control function 24a, the acquisition function 24b, the reception function 24c, the extraction function 24d, the calculation function 24e, and the display control function 24f are described as being realized by a single processing circuit 24. However, it is also possible to configure the processing circuit 24 by combining a plurality of independent processors, and each processor realizes the functions by executing programs. Also, each processing function of the processing circuit 24 may be realized by being appropriately distributed or integrated into a single or a plurality of processing circuits.

[0023] Further, the processing circuit 24 may also implement functions by using the processor of an external device connected via the network NW. For example, the processing circuit 24 reads out and executes a program corresponding to each function from the memory 21, and also uses a group of servers (cloud) connected to the medical image processing apparatus 20 via the network NW as computing resources, thereby implementing each function shown in FIG. 1.

[0024] The configuration example of the medical image processing system 1 including the medical image processing apparatus 20 has been described above. Under such a configuration, the processing circuit 24 in the medical image processing apparatus 20 assists in grasping information regarding medical images.

[0025] Hereinafter, the processing performed by the processing circuit 24 will be described with reference to the flowchart of FIG. 2. FIG. 2 is a flowchart showing an example of the processing by the processing circuit 24 of the medical image processing apparatus 20 according to the first embodiment.

[0026] First, the acquisition function 24b acquires medical image data regarding a subject (step S1). The acquisition function 24b may directly acquire medical image data from the medical image diagnostic apparatus 10, or may acquire medical image data via another device such as the image storage apparatus 30.

[0027] The type of the medical image data acquired by the acquisition function 24b is not particularly limited. For example, the acquisition function 24b acquires any type of medical image data in which the morphological information of the three-dimensional anatomical structure of the target living tissue is stored. For example, the acquisition function 24b can acquire, as medical image data regarding the subject, an X-ray CT image, an ultrasonic image, an MRI image, a three-dimensional X-ray image, a PET image, a SPECT image, or the like. Further, the acquisition function 24b may acquire a plurality of three-dimensional images (four-dimensional images) in a time series obtained by imaging a three-dimensional image such as an X-ray CT image a plurality of times in the time direction as medical image data regarding the subject. Note that a plurality of three-dimensional images in a time series collected by one imaging is also referred to as one series of images.

[0028] For example, the acquisition function 24b acquires medical image data triggered by an instruction received from a user via the input interface 23. Alternatively, the acquisition function 24b may automatically acquire medical image data newly collected by the medical image diagnostic apparatus 10. Further, the acquisition function 24b may acquire the newly collected medical image data when the newly collected medical image data satisfies a predetermined condition. For example, the acquisition function 24b sets a condition related to an imaging target as a predetermined condition, and acquires the medical image data when medical image data including an imaging target satisfying the predetermined condition in the imaging range is imaged, or when medical image data including an imaging target satisfying the predetermined condition in the imaging range is stored in the image storage device 30. Also, for example, the acquisition function 24b sets a condition related to an imaging protocol as a predetermined condition, and acquires the medical image data when the medical image data is imaged with an imaging protocol satisfying the predetermined condition. Also, for example, the acquisition function 24b sets a condition related to a reconstruction method as a predetermined condition, and acquires the medical image data when the medical image data is reconstructed by a reconstruction method satisfying the predetermined condition. More specifically, the acquisition function 24b sets the implementation of the magnified reconstruction as a predetermined condition, and acquires the medical image data when the magnified reconstruction is implemented, or when the medical image data with the magnified reconstruction implemented is stored in the image storage device 30. Also, the acquisition function 24b may combine a plurality of conditions, such as a condition related to an imaging protocol and a condition related to a reconstruction method, as a predetermined condition, and acquire the medical image data when the medical image data satisfying the predetermined condition is imaged or reconstructed or stored.

[0029] Regarding the medical image data acquired in step S1, it may be displayed before proceeding to step S2, or may proceed to step S2 without being displayed. When displaying, the display control function 24f causes, for example, a predetermined cross section in the medical image data to be displayed on the display 22.

[0030] An example of displaying medical image data will be described with reference to FIG. 3. FIG. 3 is a diagram showing an example of display according to the first embodiment. In FIG. 3, as an example, a display screen including a display area D11, a display area D12, and a display area D13 is illustrated.

[0031] The display area D11 is a menu bar, and icons, buttons, characters, etc. corresponding to various functions are displayed. The user can activate the corresponding function by inputting an operation on the icon or the like via the input interface 23. For example, the user can operate the mouse to move the mouse cursor on the display screen of FIG. 3 or click on an icon corresponding to a desired function. Also, when the display 22 is a touch panel, the user can tap an icon corresponding to a desired function or perform character input by flicking.

[0032] For example, in the display area D115 in the display area D11, a group of icons for functions that assign the operation system of the mouse is displayed. For example, the control function 24a controls so as to assign the left click and drag operation systems of the mouse to the operation system corresponding to the selected icon by selecting each icon. Specifically, the control function 24a can assign an operation system for continuously displaying an image in the slice direction, an operation system for changing the magnification rate of the image, an operation system for translating the image, an operation system for changing the tone of the image, an operation system for rotating the image, etc. to the left click and drag operation systems of the mouse. Note that the tone of an image is, for example, WL (Window level) or WW (Window Width) in the case of an X-ray CT image.

[0033] The above assignment of the operating system is merely an example, and various modifications are possible. For example, the same assignment may be made for the right-click and drag operating system, the mouse wheel click and drag operating system, the simultaneous right and left click and drag operating system, etc. Also, it may be possible to set the speed or amount of slice feed during the browsing function with respect to the amount of mouse movement, the amount of change in magnification, the amount of translation, the amount of tone change, the amount of rotation, etc. Further, the assignment may be changed according to the mouse operation when selecting the icon. For example, when the icon is selected by a left click, the operating system corresponding to the icon is assigned to the left click operating system; when the icon is selected by a right click, the operating system corresponding to the icon is assigned to the right click operating system; when the icon is selected by a simultaneous right and left click, the operating system corresponding to the icon is assigned to the simultaneous right and left click operating system; when the icon is selected by a mouse wheel click, the operating system corresponding to the icon may be assigned to the mouse wheel click operating system for control.

[0034] Also, for example, icons for various graphic drawing and measurement functions may be displayed in the display area D115, and control may be performed so that the various graphic drawing and measurement functions can be executed by selecting the icon. For example, control may be performed to assign a function that can draw a straight line, a broken line, or a curve on an image, and the distance of the straight line, broken line, or curve, the angle formed by the broken line, the curvature of the curve, etc. may be calculated and displayed. Also, an arbitrary closed curve may be drawn, and the perimeter of the closed curve, the area inside the closed curve, and statistical quantities (average value, maximum value, minimum value, etc.) of the pixel values inside may be calculated and displayed for control. The closed curve may be set to a predetermined shape (circle, ellipse, rectangle, square, triangle, etc.), and the length of each side of the shape, the angle formed by two sides, the diameter, the major axis, the minor axis, etc. may be adjustable, or a free-form shape may be drawn. It may be possible to set three-dimensional figures (sphere, ellipsoid, rectangular parallelepiped, triangular pyramid, etc.), and control may be performed to calculate and display their surface area, volume, etc.

[0035] In addition, various icons can be displayed in the display area D115. For example, in the display area D115, icons corresponding to functions such as drawing a text box or an arrow on the image may be displayed. Also, in the text box, the display form (font, character size, color, etc.) of the text to be input may be settable. Further, in the display area D115, icons corresponding to the function of drawing schematic diagrams of various treatment devices may be displayed. For example, a schematic diagram may be defined in advance from known shapes, sizes, etc. of various treatment devices such as various artificial valves, and it may be possible to draw it on each image in a size corresponding to the magnification of the image.

[0036] In the display area D116, a group of icons for setting various conditions for the display area D13 where an image is displayed is shown. For example, the display control function 24f changes the display form of the display area D13 according to the selection of each icon.

[0037] FIG. 3 shows a case where six display areas (display areas D131 to D136) of two rows and three columns are set in the display area D13. Here, the display control function 24f may change the number of rows or columns of the display areas included in the display area D13 according to the operation on the icons in the display area D116. Also, the display control function 24f may be able to change the size and shape (aspect ratio, etc.) of each display area included in the display area D13 according to the operation on the icons. Further, display areas with preset arrangements and sizes may be set. That is, the initial arrangements and sizes of the display areas included in the display area D13 may be set, and changes from the initial settings may be possible according to the operation on the icons in the display area D116.

[0038] Further, the display control function 24f may control the display and non-display of various information to be superimposed and displayed on each display area included in the display area D13. Examples of such information include the supplementary information of each image (such as a part of the information described in the DICOM (Digital Imaging and Communications in Medicine) header). For example, the display control function 24f displays the supplementary information of each image at a position on a predetermined display area. Further, the display control function 24f may switch the on / off of the display of the supplementary information, or change the display position, size, etc. according to an operation on the icon in the display area D116. Also, it may be possible to specify the type of information to be displayed. For example, the display control function 24f may control to display only the supplementary information of the specified item among a plurality of items such as patient ID, imaging conditions, and reconstruction conditions.

[0039] The button D111 is a button for instructing the execution of the processing after step S2 described later. That is, the processing circuit 24 starts the processing after step S2 triggered by receiving an operation of selecting the button D111. Note that the processing circuit 24 may omit the display of the button D111 and automatically start the processing after step S2 after step S1.

[0040] Hereinafter, as an example of medical image data, X-ray CT images will be described. For example, as shown in FIG. 3, the display control function 24f causes an X-ray CT image to be displayed in the display area D13. For example, the display control function 24f performs known three-dimensional image processing on the X-ray CT image dragged and dropped from the display area D12 to generate display images such as an arbitrary cross-section (tomographic image) and a rendering image in an arbitrary viewpoint direction, and causes them to be displayed in each display area included in the display area D13. Examples of three-dimensional image processing include volume rendering (VR), surface rendering (SR), image value projection processing, multi-planar reconstruction (MPR) processing, curved MPR (CPR) processing, maximum intensity projection (MIP), minimum intensity projection (MinIP), and the like.

[0041] More specifically, FIG. 3 shows a case where a horizontal cross-section (axial plane) based on the imaging direction of the X-ray CT image is displayed in the display area D131. Further, FIG. 3 shows a case where a coronal cross-section (coronal plane) based on the imaging direction of the X-ray CT image is displayed in the display area D132. Further, FIG. 3 shows a case where a sagittal cross-section (sagittal plane) based on the imaging direction of the X-ray CT image is displayed in the display area D133. Further, FIG. 3 shows a case where no image to be displayed in the display areas D134 to D136 is selected and they are blank. Regarding the cross-section direction and type of the image to be displayed in each of the display areas D131 to D136, they may be preset for each display area, or the user may select them as appropriate. For example, the display areas D131 to D133 may be areas for displaying cross-sections in a predetermined direction, and the display areas D134 to D136 may be areas for displaying cross-sections and rendering images in the direction selected by the user. Further, information other than images, such as measurement values and meshes described later, may be displayed in the display areas D134 to D136.

[0042] Buttons D112 to D114 are buttons for activating the function of displaying X-ray CT images in a predetermined display mode. For example, when button D112 (Mitral View) is selected, the display control function 24f displays the X-ray CT image in a display mode focusing on the mitral valve. Also, when button D113 (Aortic View) is selected, the display control function 24f displays the X-ray CT image in a display mode focusing on the aortic valve. Further, when button D114 (Heart View) is selected, the display control function 24f displays the X-ray CT image in a display mode focusing on the heart.

[0043] For example, when button D112 is selected, the display control function 24f causes a cross-section related to the mitral valve to be displayed in a predetermined area of the display area D13. That is, when button D112 is selected, the display control function 24f displays a cross-section specified as a cross-section suitable for observing the mitral valve.

[0044] For example, as shown in FIG. 4, the display control function 24f displays the mitral valve horizontal cross-section I21 in the display area D131. Similarly, the display control function 24f displays the mitral valve coronal cross-section I22 in the display area D132 and the mitral valve sagittal cross-section I23 in the display area D133. Note that FIG. 4 is a diagram showing a display example according to the first embodiment. Cross-sections such as the mitral valve horizontal cross-section I21, the mitral valve coronal cross-section I22, and the mitral valve sagittal cross-section I23 can be specified in step S3. The details of step S3 will be described later.

[0045] In FIG. 4, the image to be displayed in the display area D134 can be arbitrarily selected by the user, and as an example, a horizontal tomogram I24 at a predetermined position is illustrated. Also, in FIG. 4, the mitral valve mesh M1 is displayed in the display area D135. The mitral valve mesh M1 is displayed so that the anterior cusp and the posterior cusp can be distinguished. Further, in FIG. 4, a table V1 showing various measurement values related to the mitral valve is displayed in the display area D136.

[0046] Here, regarding the mitral valve mesh M1 in the display area D135, as shown in FIG. 5, it can also be displayed superimposed on the image I25 of an arbitrary cross-section. For example, in step S3 described later, the extraction function 24d extracts the area related to the mitral valve from the X-ray CT image. Here, the display control function 24f can generate the mitral valve mesh M1 based on the shape of the extracted area. Also, the display control function 24f generates the image I25 of an arbitrary cross-section in the X-ray CT image based on the input operation from the user. Both the mitral valve mesh M1 and the image I25 are generated based on the X-ray CT image, and the positional relationship between the mitral valve mesh M1 and the image I25 in the three-dimensional space is known. From the above, the display control function 24f can three-dimensionally align and superimpose the mitral valve mesh M1 and the image I25 of an arbitrary cross-section. For example, as shown in FIG. 5, the display control function 24f can control to display the portion of the mitral valve mesh M1 that is located in front of the image I25 with respect to the observation direction and not display the portion that is located behind the image I25. Note that the mitral valve mesh M1 is an example of a mesh showing the shape of the part included in the medical image data. Also, FIG. 5 is a diagram showing a display example of the mitral valve mesh M1 according to the first embodiment.

[0047] Regarding the superimposed image of the mitral valve mesh M1 and the image I25 of an arbitrary cross-section, as shown in FIGS. 6A and 6B, it may be observable from an arbitrary direction. For example, the display control function 24f can display the superimposed image of the mitral valve mesh M1 and the image I25 of an arbitrary cross-section so that it can be rotated in response to an input operation from the user such as a mouse operation. FIGS. 6A and 6B are diagrams showing a display example of the mitral valve mesh M1 according to the first embodiment.

[0048] The image I25 of an arbitrary cross-section shown in FIGS. 5 to 6B can also be sliced. That is, the display control function 24f can also display a superimposed image of a cross-section that is parallel to the image I25 and has a different position in the slice direction and the mitral valve mesh M1 instead of the superimposed image in FIGS. 5 to 6B.

[0049] In addition, instead of the superimposed images in FIGS. 5 to 6B, the display control function 24f can also display a superimposed image of an image of a cross-section that is not parallel to the image I25 and the mitral valve mesh M1. For example, the display control function 24f can change any one cross-section to be superimposed on the mitral valve mesh M1 in the rotational direction with respect to the axis of the mitral valve. That is, the display control function 24f can change the cross-sectional direction of the cross-section to be superimposed on the mitral valve mesh M1 according to an input operation from the user such as a mouse operation.

[0050] For example, a plurality of cross-sectional directions are preset as the cross-sectional direction of the cross-section to be superimposed on the mitral valve mesh M1. For example, as shown in FIG. 7, a plurality of cross-sectional directions such as cross-section P1, cross-section P2, cross-section P3, and cross-section P4 are set with respect to the mitral valve mesh M1. The display control function 24f can superimpose any one of cross-sections P1 to P4 on the mitral valve mesh M1 and display it in a display area D135 or the like, for example, according to a mouse wheel operation or the like. FIG. 7 is a diagram for explaining the cross-section to be superimposed on the mitral valve mesh M1 according to the first embodiment.

[0051] Also, for example, when the button D113 is selected, the display control function 24f displays a cross-section related to the aortic valve in a predetermined area of the display area D13. That is, when the button D113 is selected, the display control function 24f displays a cross-section specified as a cross-section suitable for observing the aortic valve. Such a cross-section can be specified in step S3 described later. That is, in FIG. 2, step S3 is described as a step of "specifying a cross-section related to the mitral valve", but the extraction function 24d may specify a cross-section related to the aortic valve instead of or in addition to the cross-section related to the mitral valve.

[0052] For example, when button D113 is selected, the display control function 24f can display the horizontal cross-section of the aortic valve, the coronal cross-section of the aortic valve, and the sagittal cross-section of the aortic valve instead of the mitral valve horizontal cross-section I21, the mitral valve coronal cross-section I22, and the mitral valve sagittal cross-section I23 shown in FIG. 4. Also, similar to the case shown in FIG. 4, the display control function 24f may display an arbitrary image in the display area D134, display the aortic valve mesh M2 showing the shape of the aortic valve in the display area D135, or display a table showing various measurement values related to the aortic valve in the display area D136. Further, similar to the cases shown in FIGS. 5 to 6B, the aortic valve mesh M2 may be displayed superimposed on an arbitrary cross-section based on the X-ray CT image.

[0053] Also, for example, when button D114 is selected, the display control function 24f displays a cross-section of the heart in a predetermined area of the display area D13. That is, when button D114 is selected, the display control function 24f displays a cross-section specified as a cross-section suitable for observing the heart. Such a cross-section can be specified in step S2 of FIG. 2.

[0054] For example, when the button D114 is selected, as shown in FIG. 8, the display control function 24f causes the 4-chamber image I31 to be displayed in the display area D131, the 3-chamber image to be displayed in the display area D132, the 2-chamber image to be displayed in the display area D133, the vertical long axis (VLA) image of the heart to be displayed in the display area D134, the horizontal long axis (HLA) image to be displayed in the display area D135, and the short axis (SA) image to be displayed in the display area D136. Note that FIG. 8 is a diagram showing a display example according to the first embodiment. FIG. 8 is merely an example, and the selection and arrangement of the images to be displayed can be arbitrarily changed. For example, similar to the case shown in FIG. 4, the display control function 24f may display an arbitrary image in the display area D134, display the heart mesh M3 showing the shape of the heart in the display area D135, or display a table showing various measurement values related to the heart in the display area D136. Also, similar to the cases shown in FIGS. 5 to 6B, the heart mesh M3 may be displayed superimposed on an arbitrary cross-section based on the X-ray CT image.

[0055] In addition, various icons for activating the various functions provided in a general medical image display device can be set in the display area D11. For example, the display control function 24f may accept the selection of an icon to be displayed in the display area D117 from the user. For example, the display control function 24f may display predetermined icons in the display areas D115 and D116, and display an icon freely set by the user in the display area D117.

[0056] The display area D12 is a display area for thumbnail images, and the X-ray CT images acquired by the acquisition function 24b are displayed in a list. For example, the acquisition function 24b acquires X-ray CT images from an image database such as the image storage device 30 or an inspection device such as the medical image diagnostic device 10 based on a user's instruction. Then, the display control function 24f displays the X-ray CT images as thumbnails in the display area D12. For example, the display control function 24f generates a reduced image of a predetermined size based on the X-ray CT image, and displays the reduced image in a list in the display area D12.

[0057] Each X-ray CT image thumbnail-displayed in the display area D12 is an image with different imaging phases, times, imaging conditions, reconstruction conditions, etc. Here, the X-ray CT image may be a four-dimensional image obtained by imaging a plurality of three-dimensional images in the time direction. In this case, the display control function 24f may display any one of the three-dimensional images included in the four-dimensional image as a representative image, or may thumbnail-display each of the plurality of three-dimensional images included in the four-dimensional image. Also, although not shown in FIGS. 3, 4, and 8, information regarding the X-ray CT image may be displayed together with the thumbnail image. For example, when the acquisition function 24b acquires a four-dimensional image as an X-ray CT image, the display control function 24f may display, together with the representative image, basic information (imaging date, number of slices, reconstruction function, etc.) predetermined in the X-ray CT image side by side. For example, the user can select the image to be displayed in the display area D13 by selecting the thumbnail image displayed in the display area D12 and dragging and dropping it into the display area D13.

[0058] Here, in each display area included in the display area D13, a plurality of display images based on the X-ray CT image can be sequentially switched and displayed. That is, the display control function 24f can display the X-ray CT image as a moving image.

[0059] For example, the display control function 24f can generate a plurality of cross-sections that are parallel to each other and have different positions in the slice direction as display images based on the X-ray CT image. The display control function 24f can sequentially switch and display these plurality of cross-sections with different positions in the slice direction in any of the display areas D131 to D136. That is, the display control function 24f can display a moving image while slicing a plurality of cross-sections.

[0060] Also, when the X-ray CT image is a four-dimensional image, the display control function 24f can generate a plurality of cross-sections that have the same position and angle and are different in the time direction as display images based on the X-ray CT image. For example, the display control function 24f causes a representative image, which is any one of the three-dimensional images included in the four-dimensional image, to be displayed in the display area D12. Then, when the representative image is selected by the user, the display control function 24f generates a plurality of cross-sections from each of the plurality of three-dimensional images included in the four-dimensional image, and causes a moving image to be displayed in any one of the display areas D131 to D136. Also, for example, the display control function 24f causes each of the plurality of three-dimensional images included in the four-dimensional image to be displayed as a thumbnail in the display area D12. Then, when a plurality of thumbnail images are selected, for example, by being selected all at once and dragged & dropped, the display control function 24f generates a plurality of cross-sections from each of the plurality of three-dimensional images corresponding to the selected plurality of thumbnail images, and causes a moving image to be displayed in any one of the display areas D131 to D136.

[0061] Note that the display order in the moving image display may be determined based on a user operation, or may be automatically determined by the display control function 24f. For example, when the user selects a plurality of thumbnail images in the display area D12, the display control function 24f can determine the display order based on the selection order of the thumbnails. Also, for example, the display control function 24f can also automatically determine the display order based on, for example, the imaging date and time order obtained from a DICOM header or the like, or the order of cardiac phases set based on the R-R interval.

[0062] Also, when sequentially switching and displaying a plurality of display images based on X-ray CT images, the display control function 24f may display a controller for display control. For example, as shown in FIG. 9, the display control function 24f causes the horizontal cross-section I41 to be displayed in the display area D131, the coronal cross-section I42 to be displayed in the display area D132, the sagittal cross-section I43 to be displayed in the display area D133, and the arbitrary cross-section I44 selected by the user to be displayed in the display area D134. FIG. 9 is a diagram showing an example of a controller for display control according to the first embodiment. In FIG. 9, a plurality of cross-sections different in the slice direction or the time direction are selected as the horizontal cross-section I41, and the case where the horizontal cross-section I41 is displayed as a moving image will be described.

[0063] For example, as shown in FIG. 9, the display control function 24f causes a controller D14 for controlling the display of the moving image of the horizontal cross-section I41 in the display area D131 to be displayed. The controller D14 includes, for example, icons such as a play button, a stop button, a speed-up button, a speed-down button, a button to return to the start image, and a button to advance to the final image. The user designates an icon included in the controller D14 by an operation such as a mouse click, and the display control function 24f executes the function assigned to the designated icon.

[0064] Here, as shown in FIG. 9, each icon included in the controller D14 may have a transparent background color. Thereby, even when the controller D14 is displayed, it is possible to suppress a decrease in the visibility of the horizontal cross-section I41. Note that the display control function 24f may make the background color of each icon included in the controller D14 semi-transparent. In this case, the display control function 24f may accept a designation of the transparency of the background color from the user.

[0065] The controller D14 may be appropriately hidden. For example, the display control function 24f switches the display / hiding of the controller D14 based on an instruction from the user. Also, for example, the display control function 24f may hide the controller D14 when the user gives an instruction with respect to the horizontal cross-section I41. To give an example, when an instruction such as slice feed, translation, magnification change, harmonic change, measurement by various measurement functions, etc. is input from the user with respect to the horizontal cross-section I41, the display control function 24f can hide the controller D14.

[0066] Also, FIG. 9 shows the case where the controller D14 is displayed on the horizontal cross-section I41. When an icon included in such a controller D14 is specified, the display control function 24f may execute the function assigned to the specified icon only for the horizontal cross-section I41, or may further execute it for other cross-sections. For example, when an icon included in the controller D14 is specified, the display control function 24f may control only the moving image display of the horizontal cross-section I41, or may simultaneously control the moving image display of all or part of other cross-sections such as the coronal cross-section I42, the sagittal cross-section I43, and the arbitrary cross-section I44.

[0067] Next, the process of step S2 will be described. The extraction function 24d specifies a cross-section related to the heart from the X-ray CT image acquired by the acquisition function 24b. For example, the extraction function 24d extracts a region related to the heart from the X-ray CT image, and based on the extracted region, specifies a cross-section related to the heart. The region related to the heart is an example of the first region. Also, the cross-section related to the heart is an example of the first cross-section.

[0068] Specifically, the extraction function 24d first extracts regions corresponding to the heart and each cardiac chamber (right atrium, left atrium, right ventricle, left ventricle) from the X-ray CT image. That is, the extraction function 24d acquires the coordinate information of each pixel indicating the heart and each cardiac chamber in the X-ray CT image. The extraction function 24d may extract the region related to the heart based on a user operation or automatically. For example, the extraction function 24d extracts the region related to the heart by receiving the designation of the positions of the regions of the heart and each cardiac chamber from the user via the input interface 23. Also, for example, the extraction function 24d may extract the regions of the heart and each cardiac chamber based on the anatomical structure depicted in the X-ray CT image by a known region extraction technique. Examples of the known region extraction techniques include discriminant analysis methods (also called Otsu's binarization method) based on pixel values such as CT values, region growing methods, snake methods, graph cut methods, mean shift methods, and the like. In addition, the extraction function 24d can extract the region related to the heart by any method. For example, the extraction function 24d can also extract the region related to the heart by a machine learning technique such as deep learning. For example, the extraction function 24d may extract the region related to the heart using a heart shape model constructed based on the learning data prepared in advance.

[0069] Next, the extraction function 24d specifies a cross-section related to the heart based on the extracted region. For example, the extraction function 24d specifies a predetermined specific cross-section as the cross-section related to the heart using the morphological information of the regions of the heart and each cardiac chamber. For example, the extraction function 24d specifies the range in which each cardiac chamber is depicted from the position of the center of gravity of each cardiac chamber, and specifies cross-sections such as four-chamber view, three-chamber view, and two-chamber view.

[0070] For example, the extraction function 24d extracts an axis related to the heart based on the extracted region. Specifically, the extraction function 24d extracts an axis (hereinafter referred to as the four-chamber image axis) that passes through the range in which each cardiac chamber is depicted and is along the line-of-sight direction in which a four-chamber image is obtained. Further, the extraction function 24d extracts an axis (hereinafter referred to as the three-chamber image axis) that passes through the range in which each cardiac chamber is depicted and is along the line-of-sight direction in which a three-chamber image is obtained. Further, the extraction function 24d extracts an axis (hereinafter referred to as the two-chamber image axis) that passes through the range in which each cardiac chamber is depicted and is along the line-of-sight direction in which a two-chamber image is obtained. Further, the extraction function 24d extracts axes such as VLA, HLA, and SA. Then, based on the extracted axes, the extraction function 24d identifies four-chamber images, three-chamber images, two-chamber images, VLA images, HLA images, SA images, etc. Hereinafter, the axes related to the heart (four-chamber image axis, three-chamber image axis, two-chamber image axis, VLA, HLA, SA, etc.) are also described as the first axis or the cardiac axis.

[0071] The process of step S2 may be any method as long as it can identify a cross-section related to the heart from the X-ray CT image, and is not limited to the method via region extraction. For example, machine learning techniques such as deep learning may be used to identify the cross-section. For example, the cross-section can also be identified using models for each cross-section related to the heart constructed based on pre-prepared learning data. Further, the process of step S2 may be explicitly started by a user's instruction, or may be automatically operated after the process of step S1.

[0072] Next, the process of step S3 will be described. The extraction function 24d identifies a cross-section related to the mitral valve from the X-ray CT image acquired by the acquisition function 24b. For example, the extraction function 24d extracts a region related to the mitral valve from the X-ray CT image, and based on the extracted region, identifies a cross-section related to the mitral valve. The region related to the mitral valve is an example of the second region. Note that the second region is a region different from the first region, but may be an overlapping region. For example, the extraction function 24d may extract a part of the region related to the heart as the region related to the mitral valve. Further, the cross-section related to the mitral valve is an example of the second cross-section.

[0073] Specifically, the extraction function 24d first extracts the region related to the mitral valve from the X-ray CT image. That is, the extraction function 24d acquires the coordinate information of each pixel indicating the mitral valve in the X-ray CT image. The extraction function 24d may extract the region related to the mitral valve based on a user operation or automatically. For example, the extraction function 24d receives the specification of the position of the mitral valve from the user via the input interface 23, and extracts the region related to the mitral valve. Also, for example, the extraction function 24d may extract the region related to the mitral valve based on the anatomical structure depicted in the X-ray CT image by a known region extraction technique. Examples of the known region extraction techniques include discriminant analysis methods based on pixel values such as CT values, region growing methods, snake methods, graph cut methods, mean shift methods, and the like. In addition, the extraction function 24d can extract the region related to the mitral valve by any method. For example, the extraction function 24d can also extract the region related to the mitral valve by a machine learning technique such as deep learning. For example, the extraction function 24d may extract the region related to the mitral valve using a shape model of the mitral valve constructed based on the learning data prepared in advance.

[0074] In addition, during the region extraction process, a plurality of methods may be presented for the user to select. For example, the display control function 24f displays the UI (User Interface) of FIG. 10 on the display 22, and the extraction function 24d executes the extraction process of the region related to the mitral valve by the method selected by the user referring to the UI of FIG. 10. FIG. 10 is a diagram showing an example of the UI according to the first embodiment.

[0075] Specifically, "fully automatic" in the UI of FIG. 10 is a method of identifying the region related to the mitral valve from the input image based on a predetermined equation. Also, "semi-automatic" is a method of prompting user input such as predetermined feature points, and identifying the region related to the mitral valve based on the user input and the image. Further, "fully manual" is a method in which the user designates all regions corresponding to the region related to the mitral valve. Here, the display control function 24f may further display, for each method, the assumed processing time and the operations that the user will perform when that method is selected. For example, as shown in FIG. 10, the display control function 24f displays text such as "It takes about 2 minutes for the process" for "fully automatic", "Please specify the position of the commissure on the left and right. The subsequent process takes about 1 minute" for "semi-automatic", and "Please specify all pixels corresponding to the mitral valve" for "fully manual".

[0076] Also, the display control function 24f can change the UI according to which method is selected. For example, when "fully automatic" is selected, the display control function 24f displays the assumed processing time, the progress information of the process, etc. For example, as progress information, the display control function 24f can display the ratio of the processed process to the entire process and the assumed remaining processing time. Also, when "semi-automatic" or "fully manual" is selected, the display control function 24f displays a UI that prompts user input, and the extraction function 24d records the coordinate positions and the like specified by the user. For example, the display control function 24f displays a cross-section in which the mitral valve is drawn on the display 22, and the extraction function 24d records the position clicked by the user as the coordinate position of a predetermined feature point. Another example is that the display control function 24f displays a color palette or the like to allow the user to specify a region on the image together with the cross-section in which the mitral valve is drawn, and the extraction function 24d records the coordinate positions of the region specified by the user.

[0077] When extracting the region related to the mitral valve, the extraction function 24d may extract a plurality of anatomical structures constituting the mitral valve as different regions. For example, the extraction function 24d may extract the anterior leaflet and the posterior leaflet as separate regions. Further, the extraction function 24d may identify characteristic points and lines constituting the mitral valve. For example, the extraction function 24d may identify the anterior commissure and the posterior commissure based on the positional relationship between the anterior leaflet and the posterior leaflet, or may identify the positions of the left and right fibrous triangles based on the positional relationship with the aortic valve.

[0078] Fig. 11 shows the morphology of the mitral valve and the positional relationship of characteristic points. Fig. 11 is a diagram showing the anatomical structure of the mitral valve and the aortic valve according to the first embodiment. As shown in different patterns in Fig. 11, the extraction function 24d can extract the anterior leaflet region A11 and the posterior leaflet region A12 as separate regions. Specifically, in Fig. 11, the anterior leaflet region A11 is shown in a sparse dot pattern, and the posterior leaflet region A12 is shown in a dense dot pattern. Further, the extraction function 24d can extract the annulus shape A13 and the tip shape A14 as closed curves, respectively. Further, the extraction function 24d can extract the anterior commissure A15, the left fibrous triangle (trigone) A16, the annular position closest to the left fibrous triangle A16 (hereinafter, the left fibrous triangle point) A17, the posterior commissure A18, the right fibrous triangle A19, the annular position closest to the right fibrous triangle A19 (hereinafter, the right fibrous triangle point) A20, the central position on the annulus on the posterior leaflet side between the left fibrous triangle point A17 and the right fibrous triangle point A20 (hereinafter, the central point) A21, etc.

[0079] Here, the extraction function 24d can generate the mitral valve mesh M1 by extracting the area related to the mitral valve as shown in the figure in the lower part of FIG. 11, then extracting any number of points from the area and connecting adjacent points. Alternatively, the extraction function 24d may directly extract the mitral valve mesh M1 from the X-ray CT image. Specifically, the extraction function 24d extracts any number of points instead of all the pixels indicating the mitral valve in the X-ray CT image. Then, the extraction function 24d extracts the mitral valve mesh M1 in a lattice form by connecting the adjacent arbitrary number of points. That is, the extraction function 24d may extract the mitral valve mesh M1 as the area related to the mitral valve.

[0080] FIG. 12 is a diagram showing an example of the mitral valve mesh M1 according to the first embodiment. More specifically, FIG. 12 is a mesh shape created by extracting the coordinates of a large number of points on the mitral valve, and is a diagram showing the shape of the mitral valve in a lattice form by connecting the left, right, front, and rear points of the large number of points with straight lines. In FIG. 12, the mesh corresponding to the anterior cusp region A11 is shown by a solid line, and the mesh corresponding to the posterior cusp region A12 is shown by a broken line. In addition, in the mitral valve mesh M1, characteristic positions in the mitral valve, such as the anterior commissure A15, the left fibrous trigone point A17, the posterior commissure A18, the right fibrous trigone point A20, and the central point A21, can be defined respectively.

[0081] Then, the extraction function 24d specifies a cross-section related to the mitral valve based on the specified area related to the mitral valve and characteristic points, lines, etc. For example, the extraction function 24d can specify, as the mitral valve horizontal cross-section I21 shown in FIG. 4, a plane parallel to the cross-section passing through the left fibrous trigone point A17, the right fibrous trigone point A20, and the central point A21 and having the minimum sum of the distances from the anterior commissure A15 and the posterior commissure A18. In addition, the extraction function 24d can specify, as the mitral valve coronal cross-section I22 shown in FIG. 4, a plane perpendicular to the mitral valve horizontal cross-section I21 and passing through the anterior commissure A15 and the posterior commissure A18. In addition, the extraction function 24d can specify, as the mitral valve sagittal cross-section I23 shown in FIG. 4, a plane perpendicular to both the mitral valve horizontal cross-section I21 and the mitral valve coronal cross-section I22 and passing through the midpoint of the line segment connecting the anterior commissure A15 and the posterior commissure A18.

[0082] For example, the extraction function 24d extracts an axis (hereinafter referred to as the mitral valve horizontal section axis) that is orthogonal to a cross-section passing through the left fibrous trigone A17, the right fibrous trigone A20, and the central point A21. Then, the extraction function 24d specifies a plane that is orthogonal to the mitral valve horizontal section axis and for which the sum of the distance from the anterior commissure A15 and the distance from the posterior commissure A18 is minimized as the mitral valve horizontal section I21. Further, for example, the extraction function 24d extracts an axis (hereinafter referred to as the mitral valve coronal section axis) that is perpendicular to the mitral valve horizontal section I21 and perpendicular to a plane passing through the anterior commissure A15 and the posterior commissure A18. Then, the extraction function 24d specifies a plane that is orthogonal to the mitral valve coronal section axis and passes through the anterior commissure A15 and the posterior commissure A18 as the mitral valve coronal section I22. Also, the extraction function 24d extracts an axis (hereinafter referred to as the mitral valve sagittal section axis) that is perpendicular to a plane perpendicular to both the mitral valve horizontal section I21 and the mitral valve coronal section I22. Then, the extraction function 24d specifies a plane that is orthogonal to the mitral valve sagittal section axis and passes through the midpoint of the line segment connecting the anterior commissure A15 and the posterior commissure A18 as the mitral valve sagittal section I23. Note that the mitral valve horizontal section axis, the mitral valve coronal section axis, and the mitral valve sagittal section axis are examples of the second axis. The extraction function 24d can specify second sections such as the mitral valve horizontal section I21, the mitral valve coronal section I22, and the mitral valve sagittal section I23 based on the extracted second axis.

[0083] For the process of step S3, any method may be used as long as a cross-section related to the mitral valve can be specified from the X-ray CT image. For example, the extraction function 24d may specify each cross-section based on the centroid of the region related to the mitral valve or the center of the valve annulus. Also, the extraction function 24d may use the cross-section related to the heart specified in step S2 to specify the cross-section related to the mitral valve. Further, the extraction function 24d may set three axes with respect to the extending directions of the respective planes, with the intersection point of the three extracted cross-sections as the origin. That is, the extraction function 24d may set a mitral valve horizontal axis, a mitral valve coronal axis, and a mitral valve sagittal axis with respect to the extending directions of the respective planes, with the intersection point of the mitral valve horizontal cross-section I21, the mitral valve coronal cross-section I22, and the mitral valve sagittal cross-section I23 as the origin. Also, the process of step S3 is not limited to the method via region extraction. For example, machine learning techniques such as deep learning may be used to specify the cross-section. For example, the cross-section can also be specified using models for each cross-section related to the mitral valve constructed based on pre-prepared learning data. Also, the process of step S3 may be explicitly started according to the user's instruction, or may be made to operate automatically after the process of step S2.

[0084] The region indicating the mitral valve extracted by the extraction function 24d can be displayed together with other images in the display region D13. For example, as shown in FIGS. 5 to 6B, the display control function 24f can superimpose and display the image I25 of an arbitrary cross-section and the mitral valve mesh M1.

[0085] Alternatively, the display control function 24f may display the intersection portion of the arbitrary cross-section and the mitral valve mesh M1 on the arbitrary cross-section. For example, the mitral valve mesh M1 is three-dimensional data composed of lines (straight lines or curves). When the mitral valve mesh M1 and the arbitrary cross-section are aligned three-dimensionally, the display control function 24f can specify the points where the arbitrary cross-section intersects the lines constituting the mitral valve mesh M1. The display control function 24f can display such points on the arbitrary cross-section as the intersection portion of the arbitrary cross-section and the mitral valve mesh M1.

[0086] For example, as shown in FIG. 13A, the display control function 24f displays the mitral valve horizontal section I51 as an arbitrary cross section and also displays the intersection of the mitral valve horizontal section I51 and the mitral valve mesh M1. FIG. 13A is a diagram showing an example of displaying the intersection according to the first embodiment. Specifically, in FIG. 13A, the point where the line constituting the anterior cusp side of the mitral valve mesh M1 intersects the mitral valve horizontal section I51 is indicated by a black diamond point. Also, in FIG. 13A, the point where the line constituting the posterior cusp side of the mitral valve mesh M1 intersects the mitral valve horizontal section I51 is indicated by a cross-shaped point.

[0087] In addition, in FIG. 13A, the anterior commissure A15 and the posterior commissure A18 are further illustrated. The display control function 24f can display the anterior commissure A15 and the posterior commissure A18 regardless of whether the anterior commissure A15 and the posterior commissure A18 are located on the mitral valve horizontal section I51. That is, the display control function 24f can indicate the longitudinal and transverse positions of the anterior commissure A15 and the posterior commissure A18 on the mitral valve horizontal section I51 even when the positions of the anterior commissure A15 and the posterior commissure A18 and the mitral valve horizontal section I51 are shifted in the slice direction.

[0088] Similarly, as shown in FIG. 13B, the display control function 24f displays the mitral valve coronary section I52 as an arbitrary cross section and also displays the intersection of the mitral valve coronary section I52 and the mitral valve mesh M1. Also, as shown in FIG. 13C, the display control function 24f displays the mitral valve sagittal section I53 as an arbitrary cross section and also displays the intersection of the mitral valve sagittal section I53 and the mitral valve mesh M1. FIGS. 13B and 13C are diagrams showing examples of displaying the intersection according to the first embodiment.

[0089] As another example, the display control function 24f may display, as the intersection, a line where a mask based on the lines constituting the mitral valve mesh M1 intersects an arbitrary cross section. Specifically, the display control function 24f generates a mask by approximating the straight lines or curves constituting the mitral valve mesh M1 with a plurality of polygons or by approximating them with a curved surface. Such a mask becomes three-dimensional data showing the shape of the mitral valve in a plane.

[0090] For example, as shown in FIG. 14A, the display control function 24f causes the mitral valve horizontal section I54 to be displayed as an arbitrary section, and also causes a line where the mitral valve horizontal section I54 intersects with a mask based on the lines forming the mitral valve mesh M1 to be displayed. Specifically, in FIG. 14A, the line where the mask based on the line forming the anterior cusp side of the mitral valve mesh M1 intersects with the mitral valve horizontal section I54 is shown as a solid line. Also, in FIG. 14A, the line where the mask based on the line forming the posterior cusp side of the mitral valve mesh M1 intersects with the mitral valve horizontal section I51 is shown as a dashed line. Similarly, as shown in FIG. 14B, the display control function 24f causes the mitral valve coronary section I55 to be displayed as an arbitrary section, and also causes the intersection between the mitral valve coronary section I55 and the mitral valve mesh M1 to be displayed. Further, as shown in FIG. 14C, the display control function 24f causes the mitral valve sagittal section I56 to be displayed as an arbitrary section, and also causes the intersection between the mitral valve sagittal section I53 and the mitral valve mesh M1 to be displayed. FIGS. 14A, 14B, and 14C are diagrams showing display examples of the intersections according to the first embodiment.

[0091] When displaying the intersection between an arbitrary section and the mitral valve mesh M1, the reception function 24c may accept corrections from the user regarding the intersection. For example, the user performs an operation to correct the shape of the mitral valve mesh M1 so as to match the shape of the mitral valve depicted in the arbitrary section, and the reception function 24c accepts the operation.

[0092] For example, as shown in FIG. 15, the display control function 24f causes the mitral valve sagittal section I57 to be displayed as an arbitrary section, and also causes, as the intersection, the points where the lines forming the mitral valve mesh M1 intersect with the mitral valve sagittal section I57 to be displayed. FIG. 15 is a diagram showing a display example when correcting the intersection according to the first embodiment. In FIG. 15, the points where the line forming the anterior cusp side of the mitral valve mesh M1 intersects with the mitral valve sagittal section I57 are shown as black diamond-shaped points, and the points where the line forming the posterior cusp side of the mitral valve mesh M1 intersects with the mitral valve sagittal section I57 are shown as cross-shaped points.

[0093] For example, the user operates the mouse cursor C1 to specify the point to be modified. In FIG. 15, the case where one of the points indicated by the black rhombus is specified will be described. The display control function 24f may, for example, as shown in the upper right figure of FIG. 15, change the color of the specified point from black to white and perform display so that the user can recognize the specified point. Next, the user operates the mouse cursor C1 to change the position of the specified point and determine the position after the change. For example, as shown in the lower left and lower right figures of FIG. 15, the user moves the specified point by a drag-and-drop operation. Thereby, the reception function 24c can receive a modification from the user regarding the intersection and further modify the shape of the mitral valve mesh M1.

[0094] When receiving a modification of the intersection, the reception function 24c may provide a modifiable range. For example, as shown in FIG. 15, when a point where the line constituting the mitral valve mesh M1 intersects the mitral valve sagittal section I57 is displayed and the reception function 24c receives an operation from the user to move the point, the reception function 24c restricts the range in which the point can be moved. For example, the reception function 24c may control so that only a predetermined amount (for example, 10 pixels, etc.) can be moved. Also, for example, the reception function 24c may determine a movable range (for example, within 10 mm from the region related to the mitral valve) based on the region related to the mitral valve extracted by the extraction function 24d and control so that movement outside that range is impossible. Regarding the modifiable range, the same range may be determined for each point, or it may be changed for each point. For example, the reception function 24c may decide not to provide a modifiable range for points at characteristic locations (such as the intersection part), or conversely, control so that they cannot be moved.

[0095] Further, the display control function 24f may display the set modifiable range. For example, the display control function 24f displays the points where the lines forming the mitral valve mesh M1 intersect the mitral valve sagittal section I57, accepts from the user the designation of the points to be moved, and changes the color of the designated points from black to white. Further, as shown in FIG. 16A, the display control function 24f displays a circle indicating the range within which the designated point (white diamond point) can be moved. As another example, as shown in FIG. 16B, the display control function 24f displays a curve indicating the range within which the designated point (white diamond point) can be moved. FIGS. 16A and 16B are diagrams showing display examples when correcting the intersection part according to the first embodiment.

[0096] As another example, as shown in FIG. 16C, the display control function 24f displays the points where the lines forming the mitral valve mesh M1 intersect the mitral valve sagittal section I57, and displays the range within which each point can be moved before accepting from the user the designation of the points to be moved. Specifically, in FIG. 16C, the points where the lines forming the anterior cusp side of the mitral valve mesh M1 intersect the mitral valve sagittal section I57 are indicated by three black diamond points, and the range within which these three points can be moved is displayed as an elliptical region. Also, in FIG. 16C, the points where the lines forming the posterior cusp side of the mitral valve mesh M1 intersect the mitral valve sagittal section I57 are indicated by three cross-shaped points, and the range within which these three points can be moved is displayed as an elliptical region. FIG. 16C is a diagram showing a display example when correcting the intersection part according to the first embodiment.

[0097] As shown in FIGS. 16A, 16B, and 16C, the timing of displaying the modifiable range may be the timing when the point to be corrected is designated, or may be before the point to be corrected is designated. Alternatively, the display control function 24f may display the modifiable range when a predetermined button (not shown) is selected by the user or the like.

[0098] Also, in FIGS. 15 to 16C, the case where the point where the line constituting the mitral valve mesh M1 intersects the mitral valve sagittal section I57 is displayed and the correction of the point is accepted has been described. However, the same applies to the case where the line where the mask based on the line constituting the mitral valve mesh M1 intersects the mitral valve sagittal section I57 is displayed and the correction of the line is accepted. For example, the display control function 24f receives an operation from the user to specify a point on the line where the mask intersects the mitral valve sagittal section I57, displays the range within which the specified point can be moved, receives an operation to move the point within the range, and modifies the shape of the mask so as to pass through the moved point. That is, the display control function 24f may correct the pixel group (mask) corresponding to the region related to the mitral valve by specifying the pixels to be added or deleted in the region related to the mitral valve.

[0099] Although the case of extracting the region related to the mitral valve has been described, it is similarly possible to extract the regions related to other valves included in the heart, such as the aortic valve. That is, the extraction function 24d can identify the cross-section related to any valve included in the heart in step S3. Also, the order of performing step S2 and step S3 is arbitrary and they may be performed in parallel.

[0100] Next, the process of step S4 will be described. In step S4, the calculation function 24e sets a set of measurement items. The set of measurement items is a group of measurement items that the user wants to observe or refer to, and is set according to, for example, the use of the measurement value and the target site.

[0101] For example, a series of processes for the diagnosis and treatment plan regarding mitral valve diseases can be organized in a flowchart as shown in FIG. 17. That is, a user such as a doctor first determines whether or not to perform surgical treatment (step T1). If it is determined that surgical treatment should be performed, then it is determined whether or not to perform catheter treatment (step T2). If it is determined that catheter treatment should be performed, then it is determined which device should be used (step T3). Further, according to the device selected in step T3, the user makes a determination in step T4 or step T6. For example, the user determines whether or not to perform valve replacement (step T4). If it is determined that valve replacement should be performed, then it is determined where the replacement should be made (step T5). Also, the user determines whether or not to perform Edge to Edge repair (step T6). If it is determined that Edge to Edge repair should be performed, then it is determined where the clip should be placed (step T7). The display control function 24f displays the flowchart shown in FIG. 17 as a UI, and the calculation function 24e sets a measurement item set according to a user operation on the UI of FIG. 17. FIG. 17 is a diagram showing an example of a UI related to the setting of the measurement item set according to the first embodiment.

[0102] That is, the display control function 24f presents the purposes of using the software in the form of the clinical decision-making level by presenting steps T1 to T7 in FIG. 17. The user operates, for example, the mouse cursor C1 to select any one of steps T1 to T7. Then, the calculation function 24e sets a set of measurement items corresponding to the selected purpose (clinical decision-making level). That is, the calculation function 24e accepts the selection of the clinical decision-making level from the user and sets a set of measurement items according to the selected level.

[0103] For example, the calculation function 24e pre-sets a set of measurement items for each of steps T1 to T7. For example, for step T2 in FIG. 17, as shown in FIG. 18A, a set of measurement items including left ventricular volume, left atrial volume, mitral valve volume (Tenting Area), mitral valve height (Tenting Height), distance between commissures, anterior leaflet area, posterior leaflet area, valve orifice area, and calcification amount is pre-set. Also, for example, for step T6 or step T7, as shown in FIG. 18B, a set of measurement items including left ventricular volume, left atrial volume, Coaptation height, valve orifice area, distance between commissures, anterior leaflet length, posterior leaflet length, A1 Prolapse Area, A2 Prolapse Area, A3 Prolapse Area, P1 Prolapse Area, P2 Prolapse Area, and P3 Prolapse Area is pre-set. Further, for example, for step T3, as shown in FIG. 18C, a set of measurement items including measurement items related to Edge to Edge (valve orifice area, anterior leaflet length, posterior leaflet length, Coaptation height, valve annulus perimeter), measurement items related to valve replacement (D-shaped mitral valve perimeter, neoLVOT clearance, angle between A valve and M valve, anterior leaflet length, valve annulus perimeter), and measurement items related to valve annulus correction (distance between valve annulus and coronary artery, posterior leaflet side valve annulus perimeter, calcification amount of valve annulus), etc. is pre-set. FIGS. 18A to 18C are diagrams showing an example of a set of measurement items according to the first embodiment.

[0104] The calculation function 24e may set the pre-set items as the measurement item set as they are, or may set them after receiving addition or deletion of calculation items by the user. Further, even when the measurement item set is not set in advance, for example, when the user designates measurement items one by one to set the measurement item set, it may be the case. As other calculation items in the example shown in FIGS. 18A to 18C, the volume of the entire heart, body surface area, morphological features such as the volume and roundness of each cardiac chamber, features observable from the entire heart such as the angle formed by the line segment connecting the centroid of each cardiac chamber and the centroid position of each valve, the circumference and area of the annulus of the mitral valve, the area and calcification amount of the anterior and posterior leaflets, the distance between commissures, features observable only from the features around the mitral valve such as the circumference, area, and roundness of the closed curve of the valve orifice, features observable from the mitral valve and a part of the region such as the shortest distance between the mitral annulus and the coronary artery and the distance between the papillary muscle and the mitral valve, etc. can be exemplified.

[0105] Next, the process of step S5 will be described. In step S5, the calculation function 24e calculates the measurement values corresponding to each measurement item in the measurement item set set in step S4 based on the X-ray CT image acquired in step S1.

[0106] The method for calculating the measurement values for each measurement item is not particularly limited. For example, the calculation function 24e can calculate the measurement values for each measurement item based on the regions and axes related to the heart and mitral valve acquired in steps S2 and S3. For example, the calculation function 24e can calculate the distance between the commissures based on the coordinate positions of the anterior commissure A15 and the posterior commissure A18 extracted in step S3. Further, the calculation function 24e can calculate the calcification amount of the anterior or posterior leaflet, for example, by the calculation formula of the Agatston score, based on the anterior leaflet region A11 or the posterior leaflet region A12 extracted in step S3 and the pixel values of the pixels included in the region.

[0107] The measured values calculated by the calculation function 24e can be displayed, for example, in the form of a table. For example, as shown in the display area D136 of FIG. 4, the display control function 24f can display a table V1 showing the calculated measured values together with various cross-sections related to the mitral valve, the mitral valve mesh M1, and the like.

[0108] Here, when a plurality of X-ray CT images are acquired in step S1, the calculation function 24e can calculate measured values for each of the plurality of X-ray CT images. The order of the X-ray CT images for which the measured values are calculated is not particularly limited, but the calculation function 24e can calculate measured values for each of the plurality of X-ray CT images, for example, in the order of the imaging dates obtained from the DICOM header or the like.

[0109] Alternatively, the calculation function 24e may calculate measured values for the X-ray CT image displayed in the display area D13. For example, the display control function 24f displays the X-ray CT image in the display area D13 based on an input operation from the user, and the calculation function 24e calculates measured values corresponding to each measurement item in the measurement item set set in step S4 for the X-ray CT image displayed in the display area D13, and the display control function 24f further displays a table showing the calculated measured values in the display area D13. Here, when the X-ray CT image to be displayed in the display area D13 is changed, the calculation function 24e calculates measured values for the changed X-ray CT image, and the display control function 24f updates the measured values in the table to be displayed in the display area D13.

[0110] Incidentally, the X-ray CT image may be displayed as a moving image in the display area D13. That is, the display control function 24f can display a plurality of display images based on the X-ray CT image while sequentially switching them. For example, the display control function 24f can display a moving image while slicing a plurality of cross-sections. Also, for example, the display control function 24f can display a plurality of cross-sections having the same position and angle and different in the time direction as a moving image. In this case, the calculation function 24e sequentially calculates the measured values for the newly displayed display image, and the display control function 24f can sequentially display the newly calculated measured values. However, in this case, the display of the measured values may change rapidly at short time intervals, making it difficult for the user to visually recognize the measured values.

[0111] Therefore, when performing moving image display in the display area D13, the display control function 24f may change the display mode so that the user can easily visually recognize the measured value. For example, as shown in FIG. 19, the display control function 24f may display a graph showing the change in the measured value. FIG. 19 is a diagram showing an example of the display mode of the measured value according to the first embodiment.

[0112] Specifically, the display control function 24f sequentially switches and displays a plurality of cross-sections having the same position and angle and different cardiac phases at the time of imaging in the display area D13. Also, the calculation function 24e sequentially calculates the measured values for the newly displayed cross-sections. Note that FIG. 19 shows the case where the measured values are calculated using "valve ring circumference" and "distance between commissures" as measurement items. Then, as shown in FIG. 19, the display control function 24f can associate the measured values calculated for "valve ring circumference (mm)" and "distance between commissures (mm)" with the cardiac phase respectively and display them as a graph.

[0113] In FIG. 19, a graph associating measurement values with the cardiac phase is shown. However, the display control function 24f may display a graph associating measurement values with another time axis such as the imaging date and time. Further, when displaying a moving image while slicing a plurality of cross-sections, the display control function 24f may display a graph associating measurement values with positions in the slice direction.

[0114] Also, as shown in FIG. 19, there may be cases where the units are common depending on the measurement items. In this case, the display control function 24f can display the measurement values calculated for a plurality of measurement items on one graph. Further, even when the units of the measurement items are not common, or even when the units of the measurement items are common, the display control function 24f may show the changes in each measurement value by a plurality of graphs.

[0115] Also, when a large number of measurement items are included in the measurement item set, a lot of information is displayed on one graph, or a large number of graphs are displayed, which may make it difficult for the user to visually recognize the measurement values. Therefore, the display control function 24f may accept from the user the designation of the measurement items to be displayed on the graph, and display only the measurement values calculated for the designated measurement items in a graph.

[0116] Also, although the graph display has been described as a display mode when performing moving image display in the display area D13, the embodiment is not limited to this. For example, the display control function 24f causes a plurality of display images based on the X-ray CT image to be sequentially switched and displayed in the display area D13, and the calculation function 24e sequentially calculates measurement values for the newly displayed display images. Here, the display control function 24f displays, for example, a part of the newly calculated measurement values. That is, the display control function 24f updates the measurement values to be displayed at a frequency lower than the frame rate in the moving image display. Thereby, the display control function 24f can gently change the display of the measurement values and improve the visibility of the measurement values.

[0117] Further, the display control function 24f may change the display form when the measured value in each measurement item exceeds a preset threshold value. For example, the display control function 24f displays the calculated measured value in a table and emphasizes the display by changing the character color of the measured value that exceeds the threshold value. Further, for example, the display control function 24f displays the calculated measured value in a graph and emphasizes the display by displaying a predetermined figure at the position on the graph corresponding to the measured value that exceeds the threshold value.

[0118] Next, the process of step S6 will be described. In step S6, the reception function 24c receives the designation of the measurement item. For example, the user designates the measurement location in each measurement item or the measurement item for which the user wants to confirm on the image or mesh, and the reception function 24c receives the designation operation.

[0119] For example, as shown in FIG. 20, the display control function 24f displays the measured values calculated for each of the plurality of measurement items as a table V2. Further, the display control function 24f displays a checkbox column for each measurement item in the table V2. Then, the reception function 24c receives the designation of the measurement item from the user who refers to the display in FIG. 20. Note that FIG. 20 shows the case where "distance between cross-linking parts" is designated as the measurement item. Further, FIG. 20 is a diagram for explaining the designation of the measurement item according to the first embodiment.

[0120] The reception function 24c may receive an operation on the checkbox column, an operation on the measurement item, or an operation on the measured value. For example, when the user clicks on the character of "distance between cross-linking parts" shown in FIG. 20, the reception function 24c may determine that "distance between cross-linking parts" is designated as the measurement item and display a check mark in the corresponding checkbox column.

[0121] Next, the process of step S7 will be described. In step S7, the display control function 24f controls the display mode of the X-ray CT image based on the measurement item designated in step S6.

[0122] For example, the display control function 24f causes a cross-section where the measurement location of the measurement value measured in step S5 in the measurement item specified in step S6 is visible to be displayed in the display area D13. Also, the display control function 24f changes other display images according to the classification of the measurement item. For example, when a cross-section based on an X-ray CT image is already displayed in the display area D131 or the like, the display control function 24f changes the position and angle of the cross-section to be displayed according to the classification of the measurement item.

[0123] FIG. 21 shows an example of display control in step S7. FIG. 21 is a diagram showing a display example according to the first embodiment. As shown in Table V2 of the display area D136, the display control function 24f causes the measurement values calculated for each of the plurality of measurement items to be displayed. Also, FIG. 21 shows the case where the reception function 24c has received the designation of the measurement item "distance between cross-connecting portions".

[0124] Here, as shown in the display area D134, the display control function 24f causes a cross-section I64 where the measurement location of the measurement value in the specified measurement item is visible to be displayed based on the specified measurement item. Specifically, the "distance between cross-connecting portions" specified in FIG. 21 is the distance between the front cross-connecting portion A15 and the rear cross-connecting portion A18 extracted from the X-ray CT image in step S3. Therefore, the display control function 24f generates a cross-section I64 including the front cross-connecting portion A15 and the rear cross-connecting portion A18 from the X-ray CT image and causes it to be displayed in the display area D134. Note that the display control function 24f may further display a line segment connecting the front cross-connecting portion A15 and the rear cross-connecting portion A18 as shown in FIG. 21.

[0125] Also, the "distance between commissures" specified in FIG. 21 is a measurement item related to the mitral valve. Therefore, the display control function 24f controls the display to be performed in a manner based on the region related to the mitral valve. That is, the display control function 24f controls the display mode of the X-ray CT image according to which part the specified measurement item relates to. For example, when the display areas D131 to D133 are blank or other images are being displayed, the display control function 24f causes cross-sections related to the mitral valve to be displayed in the display areas D131 to D133. For example, as shown in FIG. 21, the display control function 24f causes the mitral valve horizontal cross-section I61 to be displayed in the display area D131, the mitral valve coronal cross-section I62 to be displayed in the display area D132, and the mitral valve sagittal cross-section I63 to be displayed in the display area D133. These cross-sections related to the mitral valve are specified in advance in step S3.

[0126] Also, since the "distance between commissures" specified in FIG. 21 is a measurement item related to the mitral valve, the display control function 24f may display the mitral valve mesh M1 as shown in the display area D135 of FIG. 21. Further, the display control function 24f may indicate the measurement location of the measured value in the specified measurement item on the mitral valve mesh M1. For example, the display control function 24f may display the anterior commissure A15, the posterior commissure A18, or a line segment connecting these on the mitral valve mesh M1, similar to the case of the cross-section I64 in the display area D135.

[0127] The display example shown in FIG. 21 is merely an example, and various modifications are possible. For example, in FIG. 21, six display areas D131 to D136 of the same size are shown, and various displays are described as being performed in these display areas. However, the sizes and arrangements of these display areas may be changed as appropriate. For example, it may be possible for the user to arbitrarily change the sizes and arrangements of the respective display areas. Further, for example, as shown in FIG. 22, the display control function 24f may display a cross-section I64 in which the measurement location of the measured value in the specified measurement item is visible in a larger size as compared with the mitral valve horizontal cross-section I61, the mitral valve coronal cross-section I62, the mitral valve sagittal cross-section I63, the mitral valve mesh M1, the table V2, and the like. FIG. 22 is a diagram showing a display example according to the first embodiment.

[0128] Also, when the specified measurement item shows two-dimensional features, it is preferably illustrated two-dimensionally. For example, when the specified measurement item is a distance or an area on a plane, etc., the display control function 24f enlarges and displays the cross-section I64 as shown in FIG. 22, and displays the measurement location on the cross-section I64. On the other hand, when the specified measurement item shows three-dimensional features, it is preferably illustrated three-dimensionally. For example, when the specified measurement item is a volume or a surface area of a curved surface, etc., the display control function 24f enlarges and displays the mitral valve mesh M1 as shown in FIG. 23, and displays the measurement location on the mitral valve mesh M1. FIG. 23 is a diagram showing a display example according to the first embodiment. Note that FIG. 23 shows a case where the mitral valve mesh M1 is enlarged and displayed, but the display control function 24f may enlarge and display a VR image, an SR image, or the like instead of the mitral valve mesh M1.

[0129] In FIGS. 21 to 23, the case of performing display in a manner based on the area related to the mitral valve has been described. In a state where such display is being performed, the reception function 24c can further receive the specification of the measurement item.

[0130] As shown in Table V2 of FIG. 24, a case where the measurement item of "left ventricular volume" is further specified will be described. FIG. 24 is a diagram showing a display example according to the first embodiment. Here, "left ventricular volume" is a measurement item related to the heart. In this case, the display control function 24f changes the display mode to a mode based on the region related to the heart. That is, the display control function 24f controls the display mode of the X-ray CT image to be either a mode based on the region related to the heart or a mode based on the region related to the mitral valve according to the specified measurement item.

[0131] Note that "left ventricular volume" is a measurement item related to the region related to the heart and is an example of the first item. Also, "distance between commissures" is a measurement item related to the region related to the mitral valve and is an example of the second item. The display control function 24f controls the display mode of the X-ray CT image to be either the first mode based on the region related to the heart or the second mode based on the region related to the mitral valve according to which of the first item and the second item is specified.

[0132] When the measurement item of "left ventricular volume" is specified, the display control function 24f displays the VR image I74 of the heart as shown in the display area D134 of FIG. 24. Here, the display control function 24f can highlight the measurement location of the measured value in the specified measurement item in the VR image I74. That is, the display control function 24f can perform highlighting by coloring or the like at the position corresponding to the left ventricle in the VR image I74.

[0133] Also, for example, the display control function 24f can display cross-sections related to the heart in the display areas D131 to D133. For example, as shown in FIG. 24, the display control function 24f causes the four-chamber image I71 to be displayed in the display area D131, the two-chamber image I72 to be displayed in the display area D132, and the SA image I73 to be displayed in the display area D133. These cross-sections related to the heart are specified in advance in step S2. Also, in these cross-sections related to the heart, similar to the case of the VR image I74, the measurement locations of the measured values in the specified measurement items can be highlighted. Also, in FIG. 24, the mitral valve mesh M1 is shown in the display area D135, but instead of the mitral valve mesh M1, a heart mesh M3 showing the shape of the heart may be displayed.

[0134] As described above, according to the first embodiment, the acquisition function 24b acquires medical image data related to a subject, such as an X-ray CT image. Also, the reception function 24c receives a designation of a measurement item related to the medical image data. Also, the display control function 24f controls the display mode of the medical image data based on the designated measurement item. Therefore, the medical image processing apparatus 20 according to the first embodiment can assist in grasping information related to the medical image data. That is, the medical image processing apparatus 20 can perform display based on the medical image data in an appropriate manner based on the designated measurement item, and assist the user in grasping image information.

[0135] For example, the display control function 24f causes a first cross-section specified based on a region related to the heart to be displayed as a first mode, or a second cross-section specified based on a region related to the mitral valve to be displayed as a second mode, based on the designated measurement item. Therefore, the medical image processing apparatus 20 according to the first embodiment can display a cross-section related to the designated measurement item, and assist in grasping information such as the image information of the cross-section and the calculated value calculated for the measurement item.

[0136] In the flowchart of FIG. 2, before step S4, it has been described that a cross section related to the heart and a cross section related to the mitral valve are specified (steps S2 and S3). However, the embodiment is not limited thereto. For example, steps S2 and S3 may be integrated into step S7. That is, the cross section related to the heart may be specified when it is determined in step S7 that the display of the cross section related to the heart is to be performed. Similarly, the cross section related to the mitral valve may be specified when it is determined in step S7 that the display of the cross section related to the mitral valve is to be performed.

[0137] Also, in the flowchart of FIG. 2, it has been described that after calculating the measurement value, the designation of the measurement item is received (steps S5 and S6). However, the embodiment is not limited thereto. For example, the order of steps S5 and S6 may be interchanged, or they may be performed in parallel. For example, the calculation function 24e may calculate the measurement value for the measurement item specified in step S6.

[0138] (Second Embodiment) In the first embodiment, the case of assisting in grasping information on medical image data by changing the image to be displayed has been described. In the second embodiment, the case of assisting in grasping information on medical image data without changing the image to be displayed will be described. The medical image processing system 1 according to the second embodiment has the same configuration as the medical image processing system 1 shown in FIG. 1, and a part of the processing by the reception function 24c and the display control function 24f is different. Hereinafter, for the points described in the above embodiment, the same reference numerals as those in FIG. 1 will be given, and the description will be omitted.

[0139] Specifically, in the second embodiment, the display control function 24f displays information for receiving a specification of display conditions for medical image data. Further, the reception function 24c receives a specification of display conditions from a user who refers to the information. Here, the display control function 24f changes a display mode of information for receiving a specification of display conditions based on at least one of an operation of a user who specifies a measurement item related to the medical image data and an operation of the user on the medical image data. Further, the display control function 24f displays the medical image data based on the specified display conditions.

[0140] For example, as shown in the left diagram of FIG. 25, the display control function 24f displays a cross-section I81 based on medical image data. Here, the user may wish to change the position or angle of the cross-section to be displayed. In such a case, the display control function 24f lists candidates for cross-sections such as "mitral valve horizontal cross-section", "mitral valve coronal cross-section", "mitral valve sagittal cross-section", "4 chamber", "3 chamber", "2 chamber", "VLA", "HLA", "SA", "Axial", "Coronal", "Saggittal" as information for receiving a specification of display conditions for the medical image data. For example, when the user operates the mouse cursor C1 and performs a right-click operation on the cross-section I81, the display control function 24f lists the candidates for cross-sections shown in the left diagram of FIG. 25 as a context menu. Note that FIG. 25 is a diagram showing an example of display of display conditions according to the second embodiment.

[0141] Here, the user can move the position of the mouse cursor C1 with respect to the cross-section I81. For example, the user can move the mouse cursor C1 to the position of interest among the anatomical structures depicted in the cross-section I81. The display control function 24f changes the display mode based on such user operations. For example, as shown in the left figure of FIG. 25, when the mouse cursor C1 is positioned at the position corresponding to the mitral valve, the display control function 24f controls the display order in the context menu to be in the order of "Mitral valve horizontal section", "Mitral valve coronal section", "Mitral valve sagittal section", "4 chamber", "3 chamber", "2 chamber", "VLA", "HLA", "SA", "Axial", "Coronal", "Saggittal". That is, when the mouse cursor C1 is positioned at the position corresponding to the mitral valve, the display control function 24f preferentially displays the display conditions suitable for observing the mitral valve. Here, as shown in the right figure of FIG. 25, when the mouse cursor C1 is moved to the position corresponding to the cardiac chamber (left ventricle), the display control function 24f changes the display order in the context menu to be in the order of "4 chamber", "3 chamber", "2 chamber", "VLA", "HLA", "SA", "Mitral valve horizontal section", "Mitral valve coronal section", "Mitral valve sagittal section", "Axial", "Coronal", "Saggittal". That is, when the mouse cursor C1 is moved to the position corresponding to the cardiac chamber, the display control function 24f preferentially displays the display conditions suitable for observing the cardiac chamber. Thereby, the user can easily recognize that the desired display conditions are being displayed and can also easily select them.

[0142] As another example, the display control function 24f can change the display mode of information for receiving a specification of display conditions based on an operation of a user who specifies a measurement item related to medical image data. For example, the display control function 24f displays a list of cross-section candidates as a context menu in the same manner as shown in FIG. 25. Also, as described in step S6 of FIG. 2, the reception function 24c receives a specification of a measurement item. Then, the display control function 24f changes the display mode of the context menu according to the specified measurement item.

[0143] For example, when a measurement item related to the mitral valve such as "distance between commissures" is specified, the display control function 24f controls the display order in the context menu to be in the order of "mitral valve horizontal section", "mitral valve coronal section", "mitral valve sagittal section", "4 chamber", "3 chamber", "2 chamber", "VLA", "HLA", "SA", "Axial", "Coronal", "Saggittal". That is, when a measurement item related to the mitral valve is specified, the display control function 24f preferentially displays display conditions suitable for observing the mitral valve. Also, for example, when a measurement item related to a cardiac chamber such as "left ventricular volume" is specified, the display control function 24f changes the display order in the context menu to be in the order of "4 chamber", "3 chamber", "2 chamber", "VLA", "HLA", "SA", "mitral valve horizontal section", "mitral valve coronal section", "mitral valve sagittal section", "Axial", "Coronal", "Saggittal". That is, when a measurement item related to a cardiac chamber is specified, the display control function 24f preferentially displays display conditions suitable for observing the cardiac chamber. Thereby, the user can easily recognize that the desired display conditions are being displayed and can also easily select them.

[0144] (Third Embodiment) In the first embodiment, for example, as shown in FIGS. 21 to 23, an example of calculating a measurement value for a specified measurement item and displaying the measurement location of the measurement value was described. In the third embodiment, a case where an operation for correcting the displayed measurement location is received from the user will be further described. The medical image processing system 1 according to the third embodiment has the same configuration as the medical image processing system 1 shown in FIG. 1, and a part of the processing by the reception function 24c and the display control function 24f is different. Hereinafter, for the points described in the above-described embodiments, the same reference numerals as those in FIG. 1 are given, and the description thereof is omitted.

[0145] Specifically, in the third embodiment, the display control function 24f displays the measurement location of the calculated measurement value, as in the case shown in FIGS. 21 to 23. Further, the reception function 24c receives an operation for correcting the measurement location from the user. Further, the calculation function 24e recalculates the measurement value for the corrected measurement location.

[0146] For example, when "distance between articulation parts" is specified as the measurement item, the display control function 24f displays a cross section I82 including the anterior articulation part A15 and the posterior articulation part A18, as shown in the left diagram of FIG. 26. Further, the display control function 24f displays the anterior articulation part A15, the posterior articulation part A18, and a line segment connecting them on the cross section I82. FIG. 26 is a diagram for explaining the correction of the measurement location according to the third embodiment.

[0147] Here, the reception function 24c receives an operation for correcting the measurement location from the user. For example, the user operates the mouse cursor C1 to move the anterior articulation part A15, as shown in the middle and right diagrams of FIG. 26. For example, the user moves the anterior articulation part A15 by drag & drop. Thereby, the reception function 24c can acquire the position coordinates of the anterior articulation part A15 after the movement.

[0148] Further, the calculation function 24e recalculates the measurement values for the corrected measurement locations. For example, the calculation function 24e recalculates the measurement values for various measurement items such as the "distance between the cross-linking parts" based on the position coordinates of the anterior cross-linking part A15 after movement. Further, the display control function 24f causes the recalculated measurement values to be displayed. Here, the display control function 24f may highlight the measurement items whose measurement values have changed due to the correction of the measurement location. For example, in the case shown in FIG. 26, since the position of the anterior cross-linking part A15 has been changed, the measurement value of the "distance between the cross-linking parts" has increased and the measurement value of the "posterior cusp length" has decreased. For example, the display control function 24f can attach a marker to these measurement items whose measurement values have changed and highlight them. Further, the display control function 24f may highlight the measurement items whose measurement values have changed and indicate how the measurement values have changed by color-coding the measurement items with increased measurement values and the measurement items with decreased measurement values.

[0149] (Fourth Embodiment) In the first embodiment, the case of controlling the display mode of medical image data based on the specified measurement items has been described. In the fourth embodiment, the case of accepting a designation for the device used in the treatment and controlling the display mode of medical image data based on the designated device will be described. The medical image processing system 1 according to the fourth embodiment has the same configuration as the medical image processing system 1 shown in FIG. 1, and a part of the processing by the reception function 24c and the display control function 24f is different. Hereinafter, for the points described in the above-described embodiments, the same reference numerals as those in FIG. 1 are given and the description is omitted.

[0150] For example, in the fourth embodiment, the acquisition function 24b acquires medical image data in the same manner as in step S1 of FIG. 2. Further, the extraction function 24d identifies the cross-section related to the heart and the cross-section related to the mitral valve in the same manner as in steps S2 and S3 of FIG. 2. Further, the display control function 24f displays a list of device candidates, and the reception function 24c accepts a designation for the device from the user who has referred to the device candidates. Then, the display control function 24f controls the display mode of the medical image data based on the designated device.

[0151] For example, in the treatment planning stage or the like, the display control function 24f can draw a schematic diagram of the device on the medical image data and provide it to the user. Here, the reception function 24c can receive a specification regarding the type of device to be drawn on the medical image data. Examples of such devices include a clip for performing Edge to Edge repair, an artificial valve for valve replacement, a device for reducing the valve annulus diameter by placing and tightening anchors at the proximal and distal portions of the coronary sinus, and the like.

[0152] For example, when a clip is specified, the display control function 24f displays a cross-section suitable for observing the mitral valve. For example, the display control function 24f displays cross-sections related to the mitral valve such as a mitral valve horizontal cross-section, a mitral valve coronary cross-section, and a mitral valve sagittal cross-section.

[0153] Also, for example, when an artificial valve is specified, the display control function 24f displays a cross-section suitable for observing the heart. For example, the display control function 24f displays cross-sections related to the heart such as the 4-chamber axis, the 3-chamber axis, the 2-chamber axis, VLA, HLA, SA, and the like.

[0154] Also, for example, when a device for reducing the valve annulus diameter by placing and tightening anchors at the proximal and distal portions of the coronary sinus is specified, the display control function 24f displays a cross-section suitable for observing the mitral valve including the coronary vein. For example, the display control function 24f displays a cross-section related to the mitral valve and capable of depicting the coronary vein with a reduced magnification ratio. Alternatively, the extraction function 24d further extracts the coronary vein when identifying a cross-section related to the heart in step S2 of FIG. 2, for example, identifies a cross-section capable of depicting both the mitral valve and the coronary vein, and the display control function 24f displays a cross-section capable of depicting both the mitral valve and the coronary vein when a device for reducing the valve annulus diameter by placing and tightening anchors at the proximal and distal portions of the coronary sinus is specified.

[0155] As described above, according to the fourth embodiment, the reception function 24c receives a designation for a device. Further, the display control function 24f controls the display mode of the medical image data based on the designated device. That is, the medical image processing apparatus 20 can display the medical image data in an appropriate manner as reference information when selecting a device, and assist the operations performed by the user.

[0156] (Fifth Embodiment) In the fifth embodiment, a case will be described in which the grasping of information regarding medical image data is assisted by controlling the display mode when displaying a plurality of devices. The medical image processing system 1 according to the fifth embodiment has the same configuration as the medical image processing system 1 shown in FIG. 1, and a part of the processing by the reception function 24c and the display control function 24f is different. Hereinafter, for the points described in the above-described embodiments, the same reference numerals as those in FIG. 1 are given, and the description thereof is omitted.

[0157] Specifically, in the fifth embodiment, the display control function 24f displays medical image data and a plurality of devices used for treatment. Further, the reception function 24c receives a designation of any one of the plurality of devices from the user who refers to the display. Here, the display control function 24f changes the display mode of the plurality of devices based on the display mode of the medical image data or the user's operation on the medical image data.

[0158] For example, as shown in FIG. 27A, the display control function 24f displays a cross-section I83 based on the medical image data. Further, the display control function 24f displays "Device E1", "Device E2", "Device E3", "Device E4", "Device E5", and "Device E6" as a plurality of devices used for treatment. FIG. 27A is an example of the display of medical image data and a plurality of devices according to the fifth embodiment.

[0159] For example, the display control function 24f can draw a schematic diagram of the device on the medical image data and provide it to the user during the treatment planning stage or the like. Here, the reception function 24c can receive a specification regarding the type of device to be drawn on the medical image data. For example, in the case shown in FIG. 27A, the user can specify the type of device to be drawn on the cross-section I83 from among the devices E1 to E6.

[0160] For example, the display control function 24f first displays the cross-section I83. Here, for example, when the user performs a right-click operation on the cross-section I83, the display control function 24f displays the devices E1 to E6 as a context menu as shown in FIG. 27A. Here, FIG. 27A shows the case where the cross-section I83 is displayed, but the display mode of the medical image data is various and may also be changed during the display. For example, instead of the cross-section I83, the cross-section I84 in FIG. 27B or the cross-section I85 in FIG. 27C may be displayed. FIGS. 27B and 27C are display examples of the medical image data and the plurality of devices according to the fifth embodiment.

[0161] The display control function 24f changes the display modes of a plurality of devices based on the display modes of medical image data such as cross-section I83, cross-section I84, and cross-section I85. For example, the display control function 24f selects the device to be displayed or changes the display order based on the display mode of the medical image data. For example, when cross-section I83 is being displayed, the display control function 24f causes "Device E1", "Device E2", "Device E3", "Device E4", "Device E5", and "Device E6" to be displayed in the order shown in FIG. 27A. Also, when cross-section I84 is being displayed, the display control function 24f causes "Device E3", "Device E1", "Device E4", "Device E5", and "Device E6" to be displayed in the order shown in FIG. 27B. Further, when cross-section I85 is being displayed, the display control function 24f causes "Device E4", "Device E3", "Device E1", "Device E5", and "Device E6" to be displayed in the order shown in FIG. 27C. For example, when the medical image data is being displayed in a mode in which the coronary sinus is depicted, the display control function 24f causes a device that reduces the annulus diameter by placing and tightening anchors at the proximal and distal portions of the coronary sinus to be displayed at the top.

[0162] Also, the display control function 24f can change the display modes of a plurality of devices based on a user operation on the medical image data. For example, the display control function 24f first causes cross-section I86 to be displayed. Next, for example, when the user performs a right-click operation on cross-section I86, the display control function 24f causes devices E1 to E6 to be displayed as a context menu.

[0163] Here, the user can right-click at any position in cross-section I86 to display a context menu. Then, the display control function 24f can change the display modes of a plurality of devices according to the position where the right-click is performed. For example, when a right-click is performed at the position indicated by the mouse cursor C11 in FIG. 28A, the display control function 24f causes "Device E1", "Device E2", "Device E3", "Device E4", "Device E5", and "Device E6" to be displayed in the order shown in FIG. 28A. Also, for example, when a right-click is performed at the position indicated by the mouse cursor C12 in FIG. 28B, the display control function 24f causes "Device E4", "Device E3", "Device E1", "Device E5", and "Device E6" to be displayed in the order shown in FIG. 28B. FIGS. 28A and 28B are display examples of medical image data and a plurality of devices according to the fifth embodiment.

[0164] For example, the display control function 24f identifies a region indicating the mitral valve or a region indicating the coronary sinus in cross-section I86. Note that the display control function 24f may use the region specified in step S3 of FIG. 2. Then, the display control function 24f changes the order of the context menu according to the distance from the region indicating the mitral valve. For example, when the distance from the region indicating the mitral valve to the position where the right-click for displaying the context menu is performed is short, the display control function 24f displays the device used for the mitral valve at the top. Also, when the distance from the region indicating the coronary sinus to the position where the right-click for displaying the context menu is performed is short, the display control function 24f displays at the top a device that reduces the valve annulus diameter by placing and tightening anchors at the proximal and distal portions of the coronary sinus.

[0165] Note that in FIGS. 27A to 27C, the case where the display modes of a plurality of devices are changed based on the display mode of medical image data has been described. Also, in FIGS. 28A to 28B, the case where the display modes of a plurality of devices are changed based on a user operation on medical image data has been described. Here, the display control function 24f may change the display modes of a plurality of devices based on both the display mode of medical image data and a user operation on the medical image data. That is, the display control function 24f changes the display modes of a plurality of devices based on at least one of the display mode of medical image data and a user operation on the medical image data.

[0166] The term "processor" used in the above description means a circuit such as a CPU, a GPU (Graphics Processing Unit), an application specific integrated circuit (ASIC), or a programmable logic device (for example, a simple programmable logic device (SPLD), a complex programmable logic device (CPLD), and a field programmable gate array (FPGA)). When the processor is, for example, a CPU, the processor realizes its function by reading and executing a program stored in a storage circuit. On the other hand, when the processor is, for example, an ASIC, instead of storing a program in a storage circuit, the function is directly incorporated as a logic circuit in the circuit of the processor. Note that each processor in the embodiment is not limited to being configured as a single circuit for each processor, and a plurality of independent circuits may be combined to be configured as one processor to realize its function. Further, a plurality of components in each figure may be integrated into one processor to realize its function.

[0167] Also, in FIG. 1, a single memory 21 has been described as storing programs corresponding to the respective processing functions of the processing circuit 24. However, the embodiment is not limited to this. For example, a plurality of memories 21 may be arranged in a distributed manner, and the processing circuit 24 may be configured to read the corresponding programs from the individual memories 21. Further, instead of storing the program in the memory 21, the program may be directly incorporated into the circuit of the processor. In this case, the processor realizes its functions by reading and executing the program incorporated into the circuit.

[0168] Each component of each device according to the above-described embodiment is a functional concept and does not necessarily have to be physically configured as shown in the drawings. That is, the specific form of the distribution and integration of each device is not limited to that shown in the drawings, and all or part of it can be functionally or physically distributed and integrated in any unit according to various loads, usage situations, etc. Further, each processing function performed by each device can be realized in whole or in any part by a CPU and a program analyzed and executed by the CPU, or can be realized as hardware by wired logic.

[0169] Also, the medical image processing method described in the above-described embodiment can be realized by executing a program prepared in advance on a computer such as a personal computer or a workstation. This program can be distributed via a network such as the Internet. Further, this program can be recorded on a non-transitory recording medium readable by a computer such as a hard disk, a flexible disk (FD), a CD-ROM, an MO, a DVD, etc., and can also be executed by being read from the recording medium by the computer.

[0170] According to at least one of the embodiments described above, it is possible to assist in grasping information regarding medical image data.

[0171] Although several embodiments have been described, these embodiments are presented by way of example and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, replacements, changes, and combinations of embodiments can be made without departing from the gist of the invention. These embodiments and their modifications are included in the scope and gist of the invention, as well as in the invention described in the claims and its equivalent scope.

Explanation of Reference Numerals

[0172] 1 Medical image processing system 10 Medical image diagnostic device 20 Medical image processing device 21 Memory 22 Display 23 Input interface 24 Processing circuit 24a Control function 24b Acquisition function 24c Reception function 24d Extraction function 24e Calculation function 24f Display control function 30 Image storage device

Claims

1. An acquisition unit that acquires medical image data regarding a subject, A calculation unit that calculates measurement values for measurement items regarding the medical image data, A reception unit that receives a designation for the measurement items for which the measurement values have been calculated, An extraction unit that extracts a region related to the heart included in the medical image data and a region related to a valve included in the medical image data, extracts a first axis related to the heart based on the region related to the heart, and extracts a second axis related to the valve based on the region related to the valve, A display control unit that displays a first cross-section in the medical image data specified based on the first axis or a second cross-section in the medical image data specified based on the second axis based on the designated measurement item A medical image processing apparatus comprising the above.

2. The medical image processing apparatus according to Claim 1, wherein the types and display order of a plurality of cross-sections when displaying the first cross-section are different from the types and display order of a plurality of cross-sections when displaying the second cross-section.

3. The reception unit receives a designation of either a first item regarding the region related to the heart among the measurement items or a second item regarding the region related to the valve among the measurement items, The display control unit displays the first cross-section or the second cross-section according to which of the first item and the second item is designated. The medical image processing apparatus according to Claim 1.

4. The display control unit displays a cross-section in which the measurement location of the measurement value in the measurement item is visible based on the designated measurement item. The medical image processing apparatus according to any one of Claims 1 to 3.

5. The display control unit controls the display mode of the medical image data according to which part the designated measurement item relates to. The medical image processing apparatus according to any one of Claims 1 to 4.

6. The display control unit controls the display mode of the medical image data according to whether the designated measurement item indicates a two-dimensional feature or a three-dimensional feature. The medical image processing apparatus according to any one of Claims 1 to 4.

7. The display control unit further displays the measurement value. The medical image processing apparatus according to Claim 1.

8. The medical image processing apparatus according to claim 7, wherein the calculation unit sets a measurement item set and calculates a measurement value for each of the measurement items included in the measurement item set.

9. The medical image processing apparatus according to claim 8, wherein the calculation unit receives a selection of a clinical decision-making level from a user and sets a set of the measurement items according to the selected level.

10. The medical image processing apparatus according to any one of claims 7 to 9, wherein the calculation unit calculates the measurement value for the displayed medical image data.

11. The display control unit causes a plurality of display images based on the medical image data to be sequentially switched and displayed, the calculation unit sequentially calculates the measurement value for the newly displayed display image, and the display control unit displays a graph showing changes in the plurality of measurement values calculated for the plurality of display images. The medical image processing apparatus according to claim 10.

12. The display control unit further causes the measurement location of the measurement value to be displayed, the reception unit receives an operation for correcting the measurement location from a user, and the calculation unit recalculates the measurement value for the corrected measurement location. The medical image processing apparatus according to any one of claims 7 to 11.

13. The medical image processing apparatus according to any one of claims 1 to 12, wherein the display control unit further causes a mesh indicating the shape of a part included in the medical image data to be displayed.

14. The medical image processing apparatus according to claim 13, wherein the display control unit causes a superimposed image of an arbitrary cross section based on the medical image data and the mesh to be rotatably displayed in response to an input operation from a user.

15. The medical image processing apparatus according to claim 14, wherein the display control unit changes a cross section direction of the arbitrary cross section to be superimposed on the mesh in response to an input operation from a user.

16. The display control unit causes a plurality of display images based on the medical image data to be sequentially switched and displayed and causes a display controller to be displayed, and the display controller includes an icon having a transparent or translucent background color. The medical image processing apparatus according to any one of claims 1 to 15.

17. The display control unit displays an intersection between an arbitrary cross-section based on the medical image data and a mesh indicating the shape of a part included in the medical image data on the arbitrary cross-section. The medical image processing apparatus according to any one of claims 1 to 12.

18. The intersection is a point where a line constituting the mesh intersects with the arbitrary cross-section. The medical image processing apparatus according to claim 17.

19. The intersection is a line where a mask based on a line constituting the mesh intersects with the arbitrary cross-section. The medical image processing apparatus according to claim 17.

20. The reception unit further receives a correction of the intersection. The medical image processing apparatus according to any one of claims 17 to 19.

21. When the reception unit receives a correction of the intersection, the display control unit displays a range in which correction is possible. The medical image processing apparatus according to claim 20.

22. Obtain medical image data regarding a subject, Calculate measurement values for measurement items regarding the medical image data, Receive a designation for the measurement item for which the measurement value has been calculated, Extract a region related to the heart included in the medical image data and a region related to a valve included in the medical image data, extract a first axis related to the heart based on the region related to the heart, and extract a second axis related to the valve based on the region related to the valve, Based on the designated measurement item, display a first cross-section in the medical image data specified based on the first axis or a second cross-section in the medical image data specified based on the second axis A medical image processing method including this.

23. Obtain medical image data regarding a subject, Calculate measurement values for measurement items regarding the medical image data, Receive a designation for the measurement item for which the measurement value has been calculated, Extract a region related to the heart included in the medical image data and a region related to a valve included in the medical image data, extract a first axis related to the heart based on the region related to the heart, and extract a second axis related to the valve based on the region related to the valve, Based on the designated measurement item, display a first cross-section in the medical image data specified based on the first axis or a second cross-section in the medical image data specified based on the second axis A program that causes a computer to execute each process.

24. The types and display order of a plurality of cross-sections when displaying the first cross-section are different from the types and display order of a plurality of cross-sections when displaying the second cross-section. The medical image processing method according to Claim 22.

25. The types and display order of a plurality of cross-sections when displaying the first cross-section are different from the types and display order of a plurality of cross-sections when displaying the second cross-section. The program according to Claim 23.

Images