Medical image processing system, medical image processing method and medical image processing program

Abstract

To display an image through which a correspondence between a three-dimensional image and a second-dimensional image is easily grasped and which can effectively be used in diagnosis and medical treatment.SOLUTION: A medical image processing system 1 includes a display device 13, an input device 14, a data storage unit 12 for storing a data set which can create a three-dimensional image, and an image display control unit 11 for controlling the display of an image to the display device 13. The image display control unit 11 creates at least a three-dimensional image from the data set, simultaneously displays the created three-dimensional image and a second image having a different form from that of the three-dimensional image and associated with the three-dimensional image in the display device, creates an auxiliary image from information included in at least one of the three-dimensional image and the second image, and displays the auxiliary image in the other of the three-dimensional image and the second image.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a medical image processing system, a medical image processing method, and a medical image processing program for processing the display of medical image data obtained from a subject.

Background Art

[0002] Conventionally, diagnosis and treatment using medical images obtained by imaging devices such as CT devices have been performed. The medical images obtained by the imaging device are used, for example, for confirming the presence or absence of lesions, preoperative simulation, and intraoperative navigation. The medical images used are two-dimensional images and / or three-dimensional images. The image data acquired by the above-described imaging device is a plurality of slice image data, that is, two-dimensional image data. A three-dimensional image can be obtained by creating volume data from a plurality of two-dimensional image data. For example, Patent Documents 1 and 2 disclose a medical image display device capable of displaying a two-dimensional image and a three-dimensional image on the same display device.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the inventions described in Patent Documents 1 and 2, a three-dimensional image and a two-dimensional image corresponding thereto are displayed on the same display device. However, there is also a demand for more detailed knowledge of the correspondence between the two-dimensional image and the three-dimensional image. For example, at a site where arterial and venous blood vessels are running complexly, the positional relationship between multiple blood vessels is grasped by visually comparing the two-dimensional image and the three-dimensional image. However, simply arranging and displaying the two-dimensional image and the three-dimensional image side by side may make it difficult to grasp the correspondence between the three-dimensional image and the two-dimensional image, and thus it was not convenient for use in diagnosis and treatment.

[0005] An object of the present invention is to provide a medical image processing apparatus or the like that can easily grasp the correspondence between various images displayed on the same display device and display an image that can be effectively used in diagnosis and treatment.

Means for Solving the Problems

[0006] The medical image processing system of the present invention includes a display device that displays an image, an input device that receives an input by an operator, a data storage unit that stores at least one data set capable of creating a three-dimensional image representing a three-dimensional arrangement of a plurality of anatomical structures, an image display control unit that controls the display of the image on the display device according to the input received by the input device, and has The image display control unit performs a process of creating at least the three-dimensional image from the data set, a process of simultaneously displaying the created three-dimensional image and a second image that has a different format from the three-dimensional image but is associated with the three-dimensional image on the display device, According to the input received by the input device or independently of the input, an auxiliary image is created from information included in at least one of the three-dimensional image and the second image, and the auxiliary image is displayed on the other of the three-dimensional image and the second image. is configured to perform.

[0007] The medical image processing method of the present invention is a medical image processing method using a display device for displaying an image, an input device for receiving an input by an operator, and a data storage unit for storing at least one data set capable of creating a three-dimensional image representing a three-dimensional arrangement of a plurality of anatomical structures, creating the three-dimensional image from the data set, simultaneously displaying the created three-dimensional image and a second image that has a different format from the three-dimensional image but is associated with the three-dimensional image on the display device, creating an auxiliary image from information included in at least one of the three-dimensional image and the second image in response to an input received by the input device or independently of the input, and displaying the auxiliary image on the other of the three-dimensional image and the second image, including.

[0008] The medical image processing program of the present invention is a medical image processing program that causes a computer to execute a process of displaying at least the three-dimensional image created from at least one data set capable of creating a three-dimensional image representing a three-dimensional arrangement of a plurality of anatomical structures stored in a data storage unit on a display device, causing the computer to, create the three-dimensional image from the data set, simultaneously display the created three-dimensional image and a second image that has a different format from the three-dimensional image but is associated with the three-dimensional image on the display device, create an auxiliary image from information included in at least one of the three-dimensional image and the second image, and display the auxiliary image on the other of the three-dimensional image and the second image, execute.

[0009] (Definition of terms used in the present invention) "Anatomical structure" refers to an object (such as an organ, bone, blood vessel, etc.) that can be recognized within the subject, including lesions such as fat and tumors. Also, even if it is an anatomical structure as a whole, an anatomical structure that can be divided into multiple units or has multiple separate roles may be treated as multiple anatomical structures. For example, the lungs can be treated as separate anatomical structures for the upper lobe, middle lobe, and lower lobe respectively, and blood vessels can be treated as separate anatomical structures for the arterial blood vessels and venous blood vessels respectively.

[0010] "Translucency" in the display of an image is one of the display forms of anatomical structures in a three-dimensional image, which means that the transmittance is set so that other hidden anatomical structures can be visually recognized, and it also includes the display in a state where almost no anatomical structure to be made translucent can be visually recognized. As a specific transmittance, it can be between 30% and 90%.

[0011] "Two-dimensional image" means an image representing a cross-section of an anatomical structure, and "three-dimensional image" means an image representing three-dimensional information two-dimensionally. "Two-dimensional image" is sometimes referred to as "cross-sectional image".

Advantages of the Invention

[0012] According to the present invention, when a plurality of types of mutually related images are displayed on a display device, the correspondence relationship of the displayed images can be easily grasped visually, and images that can be effectively used in diagnosis and treatment can be displayed.

Brief Description of the Drawings

[0013]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 5A

Figure 5B

Figure 5C

Figure 5D

Figure 6

Figure 6A

Figure 7

Figure 8A

Figure 8B

Figure 9

Figure 9A

Figure 10A

Figure 10B

Figure 10C

Figure 11A

Figure 11B

Figure 11C

Figure 12A

Figure 13A

Figure 13B

Figure 13C

Figure 13D

Figure 13E

Figure 13F

Figure 13G

Figure 14A

Figure 15

Embodiments for Carrying Out the Invention

[0014] Referring to FIG. 1, a block diagram of a medical image processing system 1 according to an embodiment of the present invention is shown, which includes at least an image display control unit 11, a data storage unit 12, a display device 13, and an input device 14. In addition to these, the medical image processing system 1 may further include a data input / output interface 15, a medical information management device 21, a chemical solution injection device 22, and an imaging device 23. The medical image processing system 1 of this embodiment is configured to be able to simultaneously display a three-dimensional image and a second image, which is another type of image associated with the three-dimensional image, on the display device 13. In the following, the case where the second image is a two-dimensional image will be described as an example.

[0015] The display device 13 may be any display that can display the image created by the image display control unit 11, such as a liquid crystal display and an organic EL display. The input device 14 may be any device that can receive an input operation by an operator, such as a keyboard or a mouse, and input data or the like to the image display control unit 11. Also, a touch panel combining a display and a touch screen can be used as the display device 13 and the input device 14.

[0016] Furthermore, a non-contact input unit capable of performing non-contact input can be combined as the input device 14. The non-contact input unit can be classified into those using gesture recognition technology and those using voice recognition technology.

[0017] As an example of a non-contact input unit using gesture recognition technology, the "Leap Sensor" (manufactured by Leap Motion) can be cited. The "Leap Sensor" is an input device that can recognize the movements of an operator's fingers, etc. without contact, and has an infrared irradiation unit and an infrared camera, etc. The infrared camera captures the reflected light when the infrared light irradiated from the infrared irradiation unit hits the operator's hand, and by performing image analysis, it can detect in real time the position, movement, and shape of the operator's hand and fingers in a three-dimensional space.

[0018] As another example of a non-contact input unit using gesture recognition technology, "RealSense" (manufactured by Intel) can be cited. "RealSense" is a modularized device composed of a 3D camera consisting of an RGB camera and an infrared camera, an infrared sensor, etc. In addition to color information, it can acquire depth information and can recognize the movements of an operator's fingers, etc. in three dimensions.

[0019] In this embodiment, both the Leap Sensor and RealSense can be used. In either case, in addition to the same operations as the operator's operations on the touch panel (for example, tap, double tap, swipe, flick, pinch in, pinch out, etc.), operations in the depth direction can also be used as the operator's operations for input. In actual input operations, a transparent flat plate made of acrylic resin or the like is placed at an appropriate position, and by performing the input operation with this flat plate as a reference plane, misrecognition due to displacement in the depth direction can be prevented.

[0020] As an example of a non-contact input unit using voice recognition technology, a voice recognition unit can be cited. The voice recognition unit can have a microphone that acquires the voice generated by the operator and a voice recognition device that recognizes the voice acquired by the microphone and converts it into an operation signal. The installation location of the voice recognition device can be arbitrary, but it is preferable to install the microphone near the operator.

[0021] The data storage unit 12 stores at least one data set capable of creating a three-dimensional image representing the three-dimensional arrangement of a plurality of anatomical structures. The data storage unit 12 can include at least one of an HDD (Hard Disc Drive), an SSD (Solid State Drive), and various memories. In addition to these two-dimensional image data and three-dimensional image data, the data storage unit 12 may store at least one program, table, database, etc. necessary for the processing performed by the image display control unit 11.

[0022] The data set stored in the data storage unit 12 can be acquired from the medical information management device 21 through the data input / output interface 15. The data input / output interface 15 may be wirelessly connected or wiredly connected to the medical information management device 21. Examples of the medical information management device 21 include a PACS (Picture Archiving and Communication Systems), a RIS (Radiology Information System), and a HIS (Hospital Information System).

[0023] The medical information management device 21 manages the medical image data of a subject in which a drug solution (for example, a contrast agent) is injected by the drug solution injection device 22 and imaged by the imaging device 23. As the drug solution injection device 22, any injection device that injects a drug solution such as a contrast agent filled in a syringe or a drug solution bag into a subject according to preset injection conditions can be used. The imaging device 23 may be any device capable of imaging a medical image composed of image data, such as a CT (Computed Tomography) device, an MRI (Maganetic Resonance Imaging) device, an angiography device, a PET (Positoron Emission Tomography) device, and an ultrasonic diagnostic device. The data set stored in the data storage unit 12 can also be acquired from the imaging device 23.

[0024] Here, the data set stored in the data storage unit 12 will be described. As an example of the data set, a volume data set for a plurality of anatomical structures can be mentioned. The volume data set is a data set obtained by arranging a plurality (for example, 300) of slice image data continuously taken at regular intervals (for example, 1 mm intervals) in a specific direction (for example, the body axis direction, the left - right direction, the front - back direction, a direction inclined with respect to at least one of these) with respect to the subject by the imaging device 23 in the body axis direction. Also, the data set stored in the data storage unit 12 may be raw data directly or indirectly acquired from the imaging device 23. In this case, it is preferable that the image display processing unit 11 is configured to reconstruct the raw data stored in the data storage unit 12 to obtain an arbitrary image.

[0025] The volume data set consists of a plurality of voxels, and a plurality of anatomical structures can be extracted by a predetermined process based on the voxel values of each voxel. Then, for each voxel, a transmittance and a hue are set for each extracted anatomical structure. Therefore, each voxel includes coordinate information, transmittance information, and hue information. In this way, by performing a rendering process on the volume data set in which the anatomical structure is extracted and the transmittance and hue are set for each anatomical structure, a three - dimensional image can be created. Examples of the rendering process include volume rendering (VR), maximum intensity projection (MIP), and the like.

[0026] Each process for creating a three - dimensional image from the volume data set, such as the process for extracting the anatomical structure, setting the transmittance and hue for each voxel, and the rendering process, may be a known process, so detailed descriptions thereof are omitted here. Also, the extraction of the anatomical structure and the setting of the transmittance and hue in the volume data set may be performed by the image display control unit 11, or a volume data set in which the anatomical structure has been extracted and the transmittance and hue have been set in advance may be stored in the data storage unit 12.

[0027] A volume data set with transmittance and hue set for each anatomical structure is divided into a plurality of layers for each anatomical structure, and each voxel may contain layer information. In this case, the volume data set may be stored in the data storage unit 12 as one data file having a plurality of layer information for each anatomical structure, or may be stored in the data storage unit 12 as a plurality of data files divided for each layer information. Whether the volume data set is stored in which form may be arbitrarily selected by the operator at the time of storing the volume data set.

[0028] By cutting out and reconstructing the volume data set on an arbitrary plane, a two-dimensional image can be created. Such a process is called multi-planar reformation (MPR). Basically, as two-dimensional images used in the medical field, there are an axial cross-section perpendicular to the body axis direction of the subject, a sagittal cross-section perpendicular to the left-right direction of the subject, and a coronal cross-section perpendicular to the front-back direction of the subject.

[0029] These two-dimensional images can be created using two-dimensional image data at a desired cross-sectional position created from the volume data set. The creation of the two-dimensional image data can be performed by the image display control unit 11 using the volume data set stored in the data storage unit 12. Alternatively, a plurality of two-dimensional image data (two-dimensional image data set) created in advance using the volume data set may be stored in the data storage unit 12 together with the original volume data set. In any case, at the stage where the two-dimensional image data is created from the volume data, the coordinate information of each voxel of the volume data is inherited by the two-dimensional image data. Therefore, the three-dimensional image and the two-dimensional image based on the common volume data have common coordinate data. Also, when displaying the two-dimensional image, interpolation processing of the two-dimensional image data may be performed.

[0030] For example, when a two-dimensional image data set in a plurality of cross-sections such as the above-mentioned axial cross-section, sagittal cross-section, and coronal cross-section is stored in the data storage unit 12, the two-dimensional image data set may be stored in the data storage unit 12 as one data file having a plurality of two-dimensional image data for all cross-sections, or may be stored in the data storage unit 12 as a plurality of data files each having a plurality of two-dimensional image data for each cross-section.

[0031] Referring again to FIG. 1, when the connection between the data input / output interface 15 and the medical information management device 21 is a wired connection and the data input / output interface 15 is a cable detachable connector, and when the connection between the data input / output interface 15 and the medical information management device 21 is a wireless connection, the medical image processing terminal 10 can be configured as the medical image processing system 1. The medical image processing terminal 10 is preferably configured such that the image display control unit 11, the data storage unit 12, the display device 13, the input device 14, and the data input / output interface 15 are housed in a single housing. The medical image processing terminal 10 is a portable terminal such as a tablet terminal or a notebook personal computer having a touch panel display. However, the medical image processing terminal 10 may be configured as a workstation in which the display device 13 and the input device 14 are configured as separate units from the image display control unit 11.

[0032] The image display control unit 11 can be configured by a computer unit including a CPU (Central Processing Unit), a ROM (Read Only Memory), and a RAM (Random Access Memory), and executes various processes for controlling the display of images on the display device 13 according to the input received by the input device 14. The processes performed by the image display control unit 11 may be realized by a computer program or may be realized by hardware using a logic circuit.

[0033] When the processing performed by the image display control unit 11 is realized by a computer program, the computer program can be stored in the data storage unit 12 as described above. The computer program stored in the data storage unit 12 is executed by being loaded into the RAM of the image display control unit 11, and various processes are executed by cooperating with hardware such as a CPU.

[0034] The computer program may be stored in a computer-readable recording medium. The recording medium storing the computer program may be a non-transitory recording medium. The non-transitory recording medium is not particularly limited, and may be, for example, a recording medium such as a memory card, a CD-ROM, etc. The computer program stored in the recording medium can be installed in the computer unit via an appropriate reader. Examples of the appropriate reader include a card reader when the recording medium is a memory card, a CD drive when the recording medium is a CD-ROM, and the like.

[0035] Alternatively, the computer program may be downloaded to the computer unit from an external server via a communication network.

[0036] The image display control unit 11 conceptually includes an input determination unit 11a, a display processing unit 11b, and an auxiliary image processing unit 11c in order to execute various processes for controlling the display of an image.

[0037] The input determination unit 11a determines what processing to perform on the image displayed on the display device 13 based on the type of the input device 14 that received the input, the change in the input signal, etc.

[0038] The display processing unit 11b mainly performs processing on the display area of the display device 13 and processing on image display within the display area. Taking the case where a volume data set is stored in the data storage unit 12 as an example, these processes will be further described with reference to FIGS. 2 and 3.

[0039] In the processing of the display area, as shown in FIG. 2, the display processing unit 11b divides the image display area on the screen of the display device 13 into a first display area and a second display area, specifically, a three-dimensional image display area 100a and a two-dimensional image display area 100b, and causes them to be displayed. Further, the display processing unit 11b can also cause the display device 13 to display buttons (icons) used for input operations by the input device 14 and / or various marks and the like corresponding to the input from the input device 14.

[0040] In the processing regarding the image display within the display area, as shown in FIG. 3, a three-dimensional image 300 created from the volume data set stored in the data storage unit 12 is displayed in the three-dimensional image display area 100a, and at the same time, a two-dimensional image 200 corresponding to the position specified on the three-dimensional image 300 is created from the volume data set stored in the data storage unit 12 and displayed in the two-dimensional image display area 100b.

[0041] The auxiliary image processing unit 11c performs processing for displaying the two-dimensional image 200 displayed in the two-dimensional image display area 100b and for displaying auxiliary images on the three-dimensional image 300 displayed in the three-dimensional image display area 100a. In particular, it is configured to be able to create an auxiliary image from information included in at least one of the two-dimensional image 200 and the three-dimensional image 300, and perform processing for displaying the auxiliary image on the other of the two-dimensional image 200 and the three-dimensional image 300. This processing may be processing according to the input received by the input device 14, or may be processing according to a preset procedure independent of the input from the input device 14.

[0042] Next, regarding the image processing by the medical image processing system 1 described above, the case where the medical image processing system 1 is applied to the tablet-type medical image processing terminal 10 will be described as an example. The tablet-type medical image processing terminal 10 described below houses an image display control unit 11, a data storage unit 12, a display device 13, an input device 14, and a data input / output interface 15 in one housing, and is a portable medical image processing terminal 10 equipped with a touch panel as the display device 13 and the input device 14.

[0043] When the power of the medical image processing terminal 10 is turned on and the medical image processing program is started, a series of processes described below are performed by the image display control unit 11.

[0044] First, as shown in FIG. 2, the display processing unit 11b divides the image display area of the display device 13 of the medical image processing terminal 10 into a three-dimensional image display area 100a and a two-dimensional image display area 100b for display. Further, the display processing unit 11b causes the display device 13 to display various icons used for input operations by the operator. The various icons to be displayed on the display device 13 include at least one data readout icon 111 used to read image data from the data storage unit 12, and an input selection icon 114 used for selection of the input device 14, etc. The input selection icon 114 is valid when other input devices 14 such as a Leap sensor or a voice recognition unit are connected in addition to the touch panel as the input device 14.

[0045] When the operator taps the data read icon 111, this operation is judged by the input judgment unit 11a. The display processing unit 11b reads the volume data set from the data storage unit 12, creates a two-dimensional image 200 and a three-dimensional image 300 as shown in Fig. 3, performs appropriate processing such as cross-sectional reconstruction processing or rendering processing, and displays them in the two-dimensional image display area 100b and the three-dimensional image display area 100a. The two-dimensional image 200 displayed in the two-dimensional image display area 100b is the two-dimensional image 200 corresponding to a specified position in the three-dimensional image 300 displayed in the three-dimensional image display area 100a.

[0046] The orientation of the three-dimensional image 300 displayed at this time (whether it is an image seen from the front-back direction of the subject, an image seen from the left-right direction of the subject, or an image seen from the body axis direction of the subject) may be predetermined by the display processing unit 11b or may be input from the input device 14. The two-dimensional image 200 corresponds to a specified position in the three-dimensional image 300, but which position in the three-dimensional image 300 it corresponds to and which cross section (an axial cross section perpendicular to the body axis direction of the subject, a sagittal cross section perpendicular to the left-right direction of the subject, or a coronal cross section perpendicular to the front-back direction of the subject) may be predetermined by the display processing unit 11b or may be input from the input device 14. As an example, the three-dimensional image 300 can be determined so that an image seen from the front of the subject is displayed, and the two-dimensional image 200 can be determined so that an axial cross section passing through the center of the three-dimensional image 300 is displayed. The orientation of the three-dimensional image 300 and the cross section of the two-dimensional image 200 can be arbitrarily switched by tapping the display switching icons 112 and 113 displayed on the display device 13.

[0047] Furthermore, the display processing unit 11b processes the display of the three-dimensional image 300 displayed in the three-dimensional image display area 100a and the two-dimensional image 200 displayed in the two-dimensional image area 100b, either individually or in conjunction with each other, in response to an input from the input device 14, or according to a predetermined setting. Some examples of the display processing will be described below.

[0048] (Rotation, enlargement / reduction, translation, etc. of three-dimensional images) The three-dimensional image 300 displayed in the three-dimensional image display area 100a can be changed in display, such as rotation in an arbitrary direction, enlargement / reduction, and translation, by a predetermined input operation using the input device 14. When the input device 14 is a touch panel, operations specific to the touch panel can be used for these display changes. This operation is determined by the input determination unit 11a, and based on this, the display processing unit 11b changes the display of the three-dimensional image 300 according to the determination result of the input determination unit 11a. Examples of display changes include rotating the three-dimensional image 300 according to the swipe direction by a swipe operation, and reducing / enlarging the three-dimensional image 300 by a pinch-in / pinch-out operation. Also, the three-dimensional image 300 can be translated by sliding while touching two points.

[0049] (Change in cross-section position, enlargement / reduction, etc. of two-dimensional images) The two-dimensional image 200 displayed in the two-dimensional image display area 100b can also be changed in display, such as changing the cross-section position, enlargement / reduction, etc., in the same manner as the three-dimensional image 300, by a predetermined input operation using the input device 14. Examples of display changes include changing the cross-section position represented by the displayed two-dimensional image 200 by performing a swipe operation in the vertical direction in a specific area (such as the right end) within the two-dimensional image display area 100b. Also, similar to the three-dimensional image 300, it is possible to reduce / enlarge the display of the two-dimensional image 200 by a pinch-in / pinch-out operation.

[0050] The auxiliary image processing unit 11c creates an auxiliary image from information included in at least one of the two-dimensional image 200 and the three-dimensional image 300, and performs processing to display the created auxiliary image on the other of the two-dimensional image 200 and the three-dimensional image 300. Some examples of the processing by the auxiliary image processing unit 11c will be described below.

[0051] (Color display of a specific anatomical structure in a two-dimensional image) Generally, the three-dimensional image 300 is composed of a plurality of layers set for each anatomical structure. In addition to the coordinate information, it has a plurality of layer information indicating these layers and hue information that varies for each layer (for each anatomical structure). These pieces of information are the information inherited from each voxel of the volume dataset when creating the three-dimensional image 300 from the volume dataset. The display processing unit 11b displays the three-dimensional image 300 in the three-dimensional image display area 100a according to these pieces of information. Therefore, the three-dimensional image 300 is displayed in different hues for each layer.

[0052] On the other hand, the two-dimensional image 200 is a monochrome image represented by, for example, shades according to the magnitude of CT values. In such a two-dimensional image 200, when various anatomical structures are complexly arranged, it may be difficult to read the two-dimensional image 200.

[0053] Therefore, as shown in FIGS. 3 and 4, the auxiliary image processing unit 11c creates two-dimensional images 201 and 202 for at least one layer 301 and 302 as auxiliary images. The two-dimensional images 201 and 202 created here are two-dimensional images corresponding to the positions specified on the three-dimensional image 300, that is, the two-dimensional images 201 and 202 for each layer 301 and 302 in the same cross-section as the two-dimensional image 200 displayed in the two-dimensional image display area 100b. Also, these two-dimensional images 201 and 202 are displayed in the same hue as the hue set for the corresponding layers 301 and 302. The auxiliary image processing unit 11c superimposes the created two-dimensional images 201 and 202 on the two-dimensional image 200 displayed in the two-dimensional image display area 100b.

[0054] By the above processing, since a specific anatomical structure of the two-dimensional image 200 is displayed in the same hue as the three-dimensional image 300, it becomes possible to visually easily distinguish the anatomical structures in the two-dimensional image 200. The layers to be displayed as auxiliary images on the two-dimensional image 200 may be predetermined or may be input by the input device 14.

[0055] The display of the auxiliary image on the two-dimensional image 200 is particularly effective when applied to the vasculature. Blood vessels are observed as being approximately circular in cross-section perpendicular to their running direction. Also, lymph nodes are observed to have an approximately circular cross-section. When an approximately circular anatomical structure is displayed in the two-dimensional image 200, in order to determine whether it is a blood vessel or a lymph node, the two-dimensional image 200 must be scrolled (changing the cross-sectional position). This is because lymph nodes are spherical while blood vessels are tubular, and the difference in their structures can be grasped by scrolling.

[0056] Therefore, in the three-dimensional image 300, when the layer 301 corresponding to the arterial blood vessels is displayed in red and the layer 302 corresponding to the venous blood vessels is displayed in blue, by overlapping and displaying the red two-dimensional image 201 corresponding to the layer 301 and the blue two-dimensional image 202 corresponding to the layer 302 on the two-dimensional image 200, the distinction between the arterial and venous blood vessels can be easily made on the two-dimensional image 200. And if an approximately circular anatomical structure that is not colored either red or blue is displayed in the two-dimensional image 200 shown in the two-dimensional image display area 100b, it can be inferred that the anatomical structure is a lymph node without scrolling the two-dimensional image 200.

[0057] (Displaying the cross-sectional position of the two-dimensional image on the three-dimensional image) The two-dimensional image 200 and the three-dimensional image 300 each have coordinate information corresponding to each other in the subject's body axis direction, left-right direction, and front-back direction. Therefore, using the coordinate information of the two-dimensional image 200 and the three-dimensional image 300, the cross-sectional position corresponding to the two-dimensional image 200 shown in the two-dimensional image display area 100b can be represented on the three-dimensional image 300 shown in the three-dimensional image display area 100a.

[0058] An example thereof is shown in FIG. 5. For example, when a swipe operation is performed in a specific area (such as the right end) within the two-dimensional image display area 100b as described in the change of the cross-sectional position of the two-dimensional image 200 described above, the input determination unit 11a determines that coordinates only in the body axis direction are input according to the position where the swipe operation is performed. When such an arbitrary coordinate is input in the two-dimensional image display area 100b from the input device 14 (a touch panel in the example described here), the auxiliary image processing unit 11c displays, as an auxiliary image, a planar image 310 corresponding to the cross-section represented by the two-dimensional image 200 displayed in the two-dimensional image display area 100b at a position passing through the input coordinate in the three-dimensional image display area 100a. The planar image 310 is preferably displayed semi-transparently so that the anatomical structure is not hidden by the planar image 310.

[0059] In this way, by displaying the planar image 310 corresponding to the cross-section represented by the two-dimensional image 200 in the three-dimensional image display area 100a, it is possible to intuitively grasp at which position of the three-dimensional image 300 displayed in the three-dimensional image display area 100a the two-dimensional image 200 displayed in the two-dimensional image display area 100b represents a cross-section.

[0060] In the three-dimensional image display area 100a, in addition to the above-described planar image 310, a highlight contour image 311 may be further displayed as an auxiliary image at a position corresponding to the contour portion of the anatomical structure at the cross-section passing through the planar image 310. More specifically, when an arbitrary coordinate is input in the two-dimensional image display area 100b, the auxiliary image processing unit 11c displays the contour portion of the anatomical structure at the cross-section represented by the two-dimensional image 200 displayed in the two-dimensional image display area 100b in the three-dimensional image 300 displayed in the three-dimensional image display area 100a in a hue different from other portions of this anatomical structure. By highlighting the contour portion of the anatomical structure in this way, the above-described effect can be exerted more favorably. "Highlight display" means displaying in a visually distinguishable hue different from the color of the target anatomical structure.

[0061] In the above example, the case where the coordinates in the body axis direction are input by an input operation in a specific area on the two-dimensional image display area 100b has been described. However, when a swipe operation is performed on the two-dimensional image 200, the input determination unit 11a determines that coordinate information in the left-right direction (x-axis direction), front-back direction (y-axis direction), and body axis direction (z-axis direction) has been input according to the position where the swipe operation is being performed. When arbitrary coordinates are input on the two-dimensional image 200 in this way, the auxiliary image processing unit 11c displays the cursor 320 as an auxiliary image at the position corresponding to the input coordinates of the three-dimensional image 300 displayed in the three-dimensional image display area 100a as shown in FIG. 5A. Thereby, it is possible to intuitively grasp which position on the three-dimensional image 300 corresponds to the position specified on the two-dimensional image 200.

[0062] In this case, the auxiliary image processing unit 11c preferably displays a line segment 210 representing the position of the input coordinates in the left-right direction (x-axis direction) and a line segment 211 representing the position of the input coordinates in the front-back direction (y-axis direction) on the two-dimensional image 200. The intersection of these line segments 210 and 211 indicates the input coordinates. Thereby, the correspondence between the two-dimensional image 200 and the three-dimensional image 300 can be visually grasped.

[0063] The above-described display of the planar image 310, the display of the highlighted contour image 311, and the display of the cursor 320 using the coordinate information may be performed individually or in combination of two or more.

[0064] (Parallel Processing of Multiple Inputs) When the input device 14 can receive a plurality of inputs simultaneously, such as when the input device 14 is a touch panel, the image display control unit 11 can execute the processing according to the input related to the two-dimensional image and the input related to the three-dimensional image in parallel. Hereinafter, an example of parallel processing of a plurality of inputs when the input device 14 is a touch panel will be described.

[0065] The medical image processing terminal 10 is configured to be able to receive an input from the touch panel on the three-dimensional image display area 100a while an input from the touch panel on the two-dimensional image display area 100b is being received. When the image display control unit 11 receives inputs from both the two-dimensional image display area 100b and the three-dimensional image display area 100a, it can simultaneously execute predetermined processes according to the inputs received respectively. The state where an input on the two-dimensional image display area 100b is being received means a state where operations on the touch panel such as touch and slide are being performed by the operator on the two-dimensional image display area 100b.

[0066] For example, as shown in FIG. 5B, when the operator touches an arbitrary position on the two-dimensional image display area 100b to specify coordinates on the two-dimensional image display area 100b, as described above, the auxiliary image processing unit 11c displays at least one of the auxiliary images such as the planar image 310, the highlight contour image 311, and the cursor 320 on the three-dimensional image area 100a. Further, the auxiliary image processing unit 11c displays line segments 210, 211 representing the positions of the specified coordinates on the two-dimensional image display area 100b.

[0067] In this state, when an operation for rotating, enlarging, reducing, or translating the three-dimensional image is received as an input by the operator on the three-dimensional image display area 100a, the image display control unit 11 executes a process including rotation, enlargement, reduction, or translation of the three-dimensional image displayed on the three-dimensional image display area 100a according to the operation on the three-dimensional image display area 100a by the operator while displaying at least one of the auxiliary images on the three-dimensional image display area 100a. At this time, the auxiliary image processing unit 11c moves the positions of the auxiliary images (such as the planar image 310, the highlight contour image 311, and the cursor 320) displayed on the three-dimensional image display area 100a corresponding to the rotation, enlargement, reduction, or translation of the three-dimensional image, as shown in FIGS. 5C to 5D, for example.

[0068] In the examples shown in FIGS. 5B to 5D, the case where the input on the two-dimensional image display area 100b is not changed is shown. However, it is also possible to accept the input on the three-dimensional image display area 100a while accepting the change of the input on the two-dimensional image display area 100b. Also in this case, the image display control unit 11 can execute a predetermined process according to the change of the input on the two-dimensional image display area 100b and a predetermined process according to the input on the three-dimensional image display area 100a simultaneously. Here, the change of the input on the two-dimensional image display area 100b includes the change of the coordinate position and the change of the cross-sectional position described above.

[0069] (Specify the two-dimensional image to be displayed on the three-dimensional image) In the examples described so far, the change of the two-dimensional image 200 displayed in the two-dimensional image display area 100b is due to the input to the two-dimensional image 200, such as an input operation within the two-dimensional image display area 100b. However, since the three-dimensional image 300 and the two-dimensional image 200 have corresponding position information with each other, it is also possible to display an arbitrary two-dimensional image 200 as an auxiliary image by inputting coordinates in the three-dimensional image 300. An example thereof will be described below with reference to FIGS. 6 and 6A.

[0070] First, the operator inputs, by the input device 14, an arbitrary coordinate on the anatomical structure of the three-dimensional image 300 displayed in the three-dimensional image display area 100a. When the input device 14 is a touch panel, the input of the coordinates can be performed by tapping an arbitrary position on the three-dimensional image 300, and the input determination unit 11a determines that the coordinates at the tapped position are input. When the coordinates are input, the auxiliary image processing unit 11c reads out the two-dimensional image data including the input coordinates from the data storage unit 12, and as shown in FIG. 6, displays a thumbnail 400 of the two-dimensional image created from the read two-dimensional image data on the display device 13 as an auxiliary image.

[0071] The thumbnail 400 does not have to represent the entire anatomical structure in the cross-section including the input coordinates, and may be a two-dimensional image within a predetermined range centered on the input coordinates. The size of the range to be displayed as the thumbnail 400 can be set in advance. Also, the thumbnail 400 is preferably displayed in an area separate from the three-dimensional image display area 100a and the two-dimensional image display area 100b so as to be distinguishable from the two-dimensional image 200 and the three-dimensional image 300 that are already being displayed. In the example shown in FIG. 6, the thumbnail 400 is displayed between the three-dimensional image display area 100a and the two-dimensional image display area 100b, overlapping the three-dimensional image display area 100a and the two-dimensional image display area 100b. Note that, in order for the operator to recognize the position of the input coordinates on the three-dimensional image 300, the auxiliary image processing unit 11c preferably displays a pointer 330 at the position corresponding to the input coordinates on the three-dimensional image 300.

[0072] When approving the displayed thumbnail 400, the operator makes an input to that effect from the input device 14. When the input device 14 is a touch panel, tapping the thumbnail 400 causes the input determination unit 11a to determine that an input indicating approval of the thumbnail 400 has been made, and thus an input for approving the thumbnail 400 can be made. If the thumbnail 400 is not approved, the coordinates are input again.

[0073] When the thumbnail is approved, the auxiliary image processing unit 11c deletes the display of the thumbnail and displays the two-dimensional image 200 that was being displayed as the thumbnail in the two-dimensional image display area 100b, as shown in FIG. 6A. As described above, the two-dimensional image 200 at the coordinates specified on the three-dimensional image 300 can be displayed. In this way, by enabling the two-dimensional image 200 to be specified on the three-dimensional image 300 for display in the two-dimensional image display area 100b, the two-dimensional image 200 at a desired position can be more easily displayed. Such processing is particularly effective when it is desired to carefully check a point of interest in the three-dimensional image 300 with the two-dimensional image 200.

[0074] When the two-dimensional image 200 to be displayed in the two-dimensional image display area 100b can be specified on the three-dimensional image 300, depending on the size of the anatomical structure displayed as the three-dimensional image 300, it may be difficult to specify the intended coordinates. Therefore, for example, as shown in FIG. 7, when an arbitrary coordinate on the anatomical structure of the three-dimensional image 300 is specified (input), it is preferable that the auxiliary image processing unit 11c displays the anatomical structure including the specified coordinate in a semi-transparent manner. Thereby, it is possible to easily confirm whether the specified coordinate is the intended coordinate.

[0075] (Semi-transparent display of anatomical structures, etc. in three-dimensional images) As shown in FIGS. 5, 5A, and 7, the auxiliary image processing unit 11c can display a part of the image displayed in the three-dimensional image display area 100a in a semi-transparent manner. In FIGS. 5 and 5A, the planar image 310 indicating the position of the cross-section represented by the two-dimensional image 200, and the part below the planar image 310 of the anatomical structure displayed as the three-dimensional image 300 are displayed in a semi-transparent manner. Also, in FIG. 7, the specified anatomical structure of the anatomical structure displayed as the three-dimensional image 300 is displayed in a semi-transparent manner. Thus, by displaying a specific display in the three-dimensional image 300 in a semi-transparent manner, the state of the anatomical structure, etc. can be easily grasped.

[0076] (Changing the size of the image display area) The image display control unit 11 may perform a size change process of arbitrarily changing the sizes of the three-dimensional image display area 100a and the two-dimensional image display area 100b in response to an input from the input device 14. The case where the input device 14 is a touch panel will be described as an example for the size change.

[0077] As shown in FIG. 8A, the display processing unit 11b displays the size change icon 115 at the boundary between the three-dimensional image display area 100a and the two-dimensional image display area 100b. When the operator swipes the size change icon 115 in the left-right direction as an input from the input device 14, the input determination unit 11a determines that this operation is an input operation for changing the size of the image display area. The display processing unit 11b changes the display position of the size change icon 115 according to the input operation, specifically according to the swipe operation direction, and changes the sizes of the three-dimensional image display area 100a and the two-dimensional image display area 100b so that the position of the size change icon 115 becomes the boundary between the three-dimensional image display area 100a and the two-dimensional image display area 100b as shown in FIGS. 8A and 8B. At the same time, the auxiliary image processing unit 11c changes the size of the three-dimensional image 300 according to the changed size of the three-dimensional image display area 100a, and changes the size of the two-dimensional image 200 according to the size of the two-dimensional image display area 100b. Such a size change is effective when it is desired to check the three-dimensional image 300 or the two-dimensional image 200 in detail.

[0078] (Various functions related to display) In addition to the functions described above, the medical image processing system 1 can have various functions related to display as functions executed by the image processing control unit 11. These functions can also be separately possessed by the two-dimensional image and the three-dimensional image. When having a plurality of functions, the image display control unit 11 can be configured to display a function call icon group including a plurality of icons corresponding to the respective functions for calling the various functions on the display device 13. In this case, in order to secure a sufficient effective display area for the three-dimensional image display area 100a and the two-dimensional image display area 100b, it is preferable that the image display control unit 11 normally makes the function call icon group non-displayed and displays it as necessary.

[0079] Therefore, as shown in FIG. 9 for example, the image display control unit 11 can display menu button icons 500a and 500b in the three-dimensional image display area 100a and the two-dimensional image display area 100b, respectively. When the operator taps these menu button icons 500a and 500b, the image display control unit 11 displays a function call icon group in the three-dimensional image display area 100a and the two-dimensional image display area 100b. When the user taps a desired icon from the icon group, the function corresponding to the tapped icon is selected.

[0080] The types and numbers of display-related functions that the medical image processing system 1 can have are not particularly limited and can be arbitrary. Some examples of display-related functions will be described below.

[0081] (1) Revert function As described above, the three-dimensional image can arbitrarily change the display direction and size, and the two-dimensional image can arbitrarily change the display cross-sectional position and size. The revert function is a function for initializing the display of the three-dimensional image and / or the two-dimensional image. Initializing the display means that the display processing unit 11b reads a data set from the data storage unit 12, executes appropriate processing, and displays the initial image to be first displayed in the three-dimensional image display area 100a and the two-dimensional image display area 100b.

[0082] As shown in FIG. 9A, revert icons 520 and 560 are displayed in the three-dimensional image display area 100a and the two-dimensional image display area 100b, respectively. When the revert icon 520 on the three-dimensional image display area 100a is tapped, the image display control unit 11 executes the revert function on the three-dimensional image. When the revert icon 560 on the two-dimensional image display area 100b is tapped, the image display control unit 11 executes the revert function on the two-dimensional image.

[0083] The revert icon 520 or 560 may be displayed only in either the three-dimensional image display area 100a or the two-dimensional image display area 100b. In that case, when the displayed revert icon 520 or 560 is tapped, the revert function may be executed for both the three-dimensional image and the two-dimensional image. Also, when initializing the image, it may be changed all at once from the currently displayed image to the initial image, or it may be changed gradually. By gradually changing from the currently displayed image to the initial image, the user can easily grasp the correspondence between the currently displayed image and the initial image.

[0084] (2) Clipping function The clipping function is a function of cutting out and displaying an arbitrary part of the three-dimensional image displayed in the three-dimensional image display area 100a. In the form shown in FIG. 9A, when the clipping icon 530 displayed as “BBox” is tapped, the image display control unit 11 executes a clipping process. Hereinafter, an example of the clipping process of the three-dimensional image by the image display control unit 11 will be described.

[0085] When the clipping process is executed, as shown in FIG. 10A, a three-dimensional image 600 in which an anatomical structure is arranged inside a hexahedron is displayed in the three-dimensional image display area 100a. The hexahedron is displayed semi-transparently so that the user can visually recognize the anatomical structure located inside it. Also, in the illustrated form, the hexahedron is a rectangular parallelepiped composed of rectangular faces, but the shape of each face may be arbitrary according to the three-dimensional shape of the anatomical structure. Note that, in the description here, for the purpose of making the visual understanding easier by displaying the three-dimensional image largely, the description will be made in a state where the size of the three-dimensional image display area 100a is enlarged.

[0086] When the user taps one surface 610 to be clipped out of the three-dimensional image 600, the tapped surface 610 is made active. It is preferable to visually distinguish the active surface from other surfaces so that the user can visually recognize which surface of the three-dimensional image 600 is active. For visual distinction, for example, as shown in FIG. 10B, the made-active surface 610 can be displayed in a color different from other surfaces. Alternatively, the color of the frame (side) of the made-active surface 610 can be displayed in a color different from the frames (sides) of other surfaces, or a combination of these can be used.

[0087] When the user slides in a direction corresponding to the direction perpendicular to the made-active surface 610 of the displayed three-dimensional image 600 while keeping the made-active surface 610 touched, as shown in FIG. 10C, the three-dimensional image 600 is clipped according to the sliding direction. At this time, if there is an anatomical structure on the clipped surface, the anatomical structure is displayed in cross-section on the clipped surface. When another surface is tapped in this state, the tapped other surface is made active, and in the same manner as above, the three-dimensional image 600 can be further clipped with another surface.

[0088] Here, the processing of the three-dimensional image displayed on the three-dimensional image display area 100a has been described, but in response to the clipping of the three-dimensional image, the display of the two-dimensional image on the two-dimensional image display area 100b may be changed. For example, when the portion cut out by clipping the three-dimensional image is displayed as a two-dimensional image on the two-dimensional image display area 100b, the portion corresponding to the cut-out portion can be made non-displayed or displayed in another color, etc., so that it can be made a display that can be visually distinguished from the non-cut-out portion.

[0089] (3) Perspective display function Normally, parallel projection (also referred to as "orthographic display") is used for the display of three-dimensional images. This is because the dimensional ratios of each part are accurately drawn. Perspective display is one of the expression methods that gives a sense of depth by representing a three-dimensional image by perspective projection.

[0090] In the form shown in FIG. 9A, as one of the function selection icons, it has a perspective angle icon 540 that represents a three-dimensional image by perspective projection. When the user taps this perspective angle icon 540, the image display control unit 11 displays an angle-of-view adjustment icon 700 in the three-dimensional image display area 100a as shown in FIG. 11A. When the user slides while touching the angle-of-view adjustment icon 700, as shown in FIGS. 11B and 11C, according to the sliding amount, the angle of view of the three-dimensional image displayed in the three-dimensional display area 100a becomes wider, the part at a closer position is displayed larger, and the part at a farther position is displayed smaller. As a result, the sense of depth of the three-dimensional image is emphasized. At this time, by changing the size (for example, length) of the angle-of-view adjustment icon 700 according to the sliding amount of the angle-of-view adjustment icon 700, the user can easily grasp the degree of perspective display.

[0091] By representing the three-dimensional image by perspective projection, an image similar to the image seen through an endoscope can be displayed. Therefore, the perspective display function can be effectively used in endoscopic surgery and the like, such as proceeding with the treatment while comparing the image obtained from the imaging device with the actual organ image obtained by the endoscope.

[0092] (4) Split window function The medical image processing system can have a split window function. In that case, as shown in FIG. 9A, the icon group for function selection includes a split window icon 550. When the user taps the split window icon 550, the image display control unit 11 executes the split window function.

[0093] The split window function is a function that divides the three-dimensional image display area 100a into a plurality of areas 101a and 102a as shown in Fig. 12A, and displays three-dimensional images in the respective areas 101a and 102a. The three-dimensional images displayed in the respective areas 101a and 102a can be independently changed in terms of orientation, size, anatomical structure to be displayed, etc. by the processing of the image display control unit 11 based on a predetermined operation by the user on the respective areas 101a and 102a. The split window function can be effectively used when checking anatomical structures from different orientations simultaneously.

[0094] In the form shown in Fig. 12A, the three-dimensional image display area 100a is divided into left and right, but it may also be divided into upper and lower. Also, each time the split window icon 550 is tapped, the split into left and right, the split into upper and lower, and no split may be switched. Furthermore, the number of divisions of the three-dimensional image display area 100a may be three or more.

[0095] (5) Scene saving function The medical image processing system can have a scene saving function. In that case, as shown in Fig. 9A, the group of icons for function selection includes scene saving icons 570a and 570b. When the user taps the scene saving icon 570a or 570b, the image processing control unit 11 executes the scene saving function.

[0096] The scene saving function is a function for saving the currently displayed image in the medical image processing system. Specifically, for the currently displayed image, it is a function for newly storing in the data storage unit 12 a data set including various data necessary for display, such as orientation, size, position, and display target. When the scene saving icon 570a on the three-dimensional image display area 100a is tapped, the data set of the three-dimensional image displayed on the three-dimensional image display area 100a at that time is stored in the data storage unit 12 (see FIG. 1). When the scene saving icon 570b on the two-dimensional image display area 100b is tapped, the data set of the two-dimensional image displayed on the two-dimensional image display area 100b at that time is stored in the data storage unit 12.

[0097] The saved image can be displayed as a thumbnail 580 at an appropriate position. For example, the three-dimensional image can be displayed in the three-dimensional image display area 100a, and the two-dimensional image can be displayed in the two-dimensional image display area 100b. When the user taps a desired thumbnail 580, the image processing control unit 11 reads out the corresponding data set from the data storage unit 12 and displays it as a three-dimensional image or a two-dimensional image in the three-dimensional image display area 100a or the two-dimensional image display area 100b. Thereby, when it is necessary to resume work after interrupting the work by this medical image processing system, it is possible to quickly access the image before interruption.

[0098] Here, the case where both the scene saving icon 570a for the three-dimensional image display area 100a and the scene saving icon 570b for the two-dimensional image display area 100b are provided has been described, but only one of them may be provided.

[0099] (6) Non-display of object and re-display of non-displayed object The medical image processing system can also have a function for non-displaying an object. In this case, it can further have a function for re-displaying an object that has been set as an emergency. The object may be an anatomical structure.

[0100] For example, in the state shown in FIG. 13A, when the user selects an arbitrary object (the parenchymal organ of the liver) 800 among the three-dimensional images including a plurality of objects displayed in the three-dimensional image display area 100a by tapping, as shown in FIG. 13B, the selected object 800 is displayed semi-transparently. Also, as described above, a thumbnail 400 of the two-dimensional image corresponding to the selected object 800 may be displayed. When the selected object 800 is tapped again, the display of the object 800 is changed from semi-transparent to the original display, in other words, to a non-transparent display with a transmittance of zero. Thus, in this embodiment, each time the object 800 is selected, the display of the object 800 is switched between semi-transparent display and non-transparent display.

[0101] The semi-transparent display of the object 800 can include a plurality of semi-transparent displays with different transmittances. In this case, it is preferable that the switching of the semi-transparent display of the object 800 is switched so that the transmittance gradually increases each time the user taps the object 800.

[0102] When the object 800 is selected, as shown in FIG. 13B, an icon group 850 for selecting additional processing for the selected object 800 can be further displayed. As the additional processing, for example, it can include hiding the selected object. In this case, the icon group 850 can include a hide icon 851.

[0103] When the user taps the hide icon 851 while the object 800 is selected, as shown in FIG. 13C, the selected object 800 is hidden, and only other objects 801, 802, 803 (for example, the hepatic artery, hepatic vein, and portal vein) are displayed.

[0104] In the state shown in FIG. 13C, when the user selects another object (hepatic artery) 801 by tapping it, as shown in FIG. 13D, the object 801 is displayed semi-transparently, and further, as described above, the icon group 850 is displayed. Here, when the user further taps the hidden icon 851, as shown in FIG. 13D, the selected object 801 is hidden, and only the other objects 802 and 803 are displayed.

[0105] Here, the case where the next object 801 is selected after hiding the first object 800 has been described. However, it is also possible to select the next object 801 in the state where the first object 800 is selected and displayed semi-transparently (FIG. 13B). In this case, the first object 800 remains semi-transparently displayed and the next object 801 is also semi-transparently displayed. Also, the initial setting when the objects 800, 801, 802, and 803 are selected does not necessarily have to be semi-transparent display and may be hidden. For example, when the object 800 is selected in the state shown in FIG. 13A, as shown in FIG. 13C, the selected object 800 may be hidden. In this case, the hidden icon 851 for additional processing is not necessary.

[0106] There may be a case where it is desired to display the hidden object again. However, in this case, since the object to be displayed does not exist on the screen, the object cannot be selected.

[0107] Therefore, the medical image processing system can have a function of redisplaying hidden objects. The redisplay function can be called, for example, from the menu button icon 500a. As shown in FIG. 9A, the function selection icon displayed by tapping the menu button icon 500a can include a redisplay icon 510. However, when there are no hidden objects, the redisplay icon 510 is displayed in a different color and / or size from the other icons (520-550) of the function call icons, in other words, in an inactive state. When there are hidden objects, as shown in FIG. 13F, the redisplay icon 510 is displayed in the same color and size as the other icons of the function call icon group, in other words, in an active state.

[0108] When the user taps the active redisplay icon 510, as shown in FIG. 13G, a hidden object bar 910 is displayed. Hidden objects are displayed on the hidden object bar 910. The display format of the hidden objects may be arbitrary, such as a list format, but it is preferable to display a thumbnail of the image representing the hidden object so that the user can intuitively understand it. In one form, as shown in FIG. 13G, thumbnails 901, 902 are displayed. When the user taps the thumbnail of the object to be redisplayed among these thumbnails, the object corresponding to the tapped thumbnail is redisplayed on the three-dimensional image display area 100a. The hidden object bar 910 may be always displayed regardless of the presence or absence of hidden objects, or may be automatically displayed without user input when there are hidden objects. In this case, the redisplay icon 510 is not necessary.

[0109] (7) Vascular territory calculation function If the three-dimensional image includes an object of a parenchymal organ and an object of a blood vessel, the vascular supply area of the parenchymal organ (when the blood vessel is divided into a plurality of blood vessel parts at a branch, the supply area for each divided blood vessel part) can be calculated. The medical image processing system can have this vascular supply area calculation function. For the calculation of the vascular supply area, a known method such as the Voronoi method can be used.

[0110] For example, as shown in FIG. 13B, an icon group 850 for selecting additional processing includes a split icon 852, and when the user taps the split icon 852, the image processing control unit 11 can be made to execute the vascular supply area calculation function. In the state shown in FIG. 13B, as objects of blood vessels, it has an object (artery) 801, an object (vein) 802, and an object (portal vein) 803, and an object 800 (parenchymal organ of the liver) 800 is selected. When the user taps the split icon 852 in this state, the image processing control unit 11 calculates the vascular supply area and changes the display of the object (parenchymal organ of the liver) 800 so that the object (parenchymal organ of the liver) 800 can be visually distinguished for each supply area of the blood vessel objects 801, 802, 803. When the calculation of the vascular supply area is completed, as shown in FIG. 14A, the calculation result 950 is displayed.

[0111] Also, in order to recombine and display the divided parenchymal organs, for example, as shown in FIG. 13B, an icon group 850 for selecting additional processing includes a combine icon 853, and when the user taps the combine icon 853, the image processing control unit 11 can be made to execute a process of recombining the divided parenchymal organs.

[0112] Above, mainly about the processes performed by the auxiliary image processing unit 11c, several processes performed by the image display control unit 11 have been described. However, each of the above-described processes can be performed alone, or any two or more of them can be combined and performed.

[0113] Furthermore, the present invention is not limited to the above-described forms and can be appropriately modified within the scope of the technical idea of the present invention. Modification examples will be described below.

[0114] (Data set stored in the data storage unit) The data set stored in the data storage unit 12 may be a two-dimensional image data set composed of a plurality of slice image data captured by the imaging device 23. In this case, the display processing unit 11b can create the two-dimensional image 200 using the stored two-dimensional image data set and also create the three-dimensional image 300 by performing the above-described appropriate rendering processing.

[0115] (Second image) The second image, which has a different format from the three-dimensional image, is not limited to a two-dimensional image. The second image that can be used in the present invention may be a video image during an actual surgery. The three-dimensional image includes the organ that is the target of the surgery. By displaying the three-dimensional image including the organ simultaneously with the video image during the surgery in this way, the treatment can be advanced while comparing the three-dimensional image with the actual image. The video image is captured by the image display control unit 11 via the data input / output interface 15 and can be displayed on the display device 13 together with the three-dimensional image 300. It is preferable that the display sizes of the three-dimensional image 300 and the video image are enlarged or reduced as necessary so that they match on the display device 13. The change in the display size can be caused by an input by an operator via the input device 14, or if the video image includes coordinate information, the display size can be automatically changed by processing by the image display control unit 11 so as to match the coordinate information of the three-dimensional image.

[0116] Other examples of the second image include two-dimensional or three-dimensional images of artificial grafts such as artificial heart valves and stents, and various instruments used during surgery. These artificial grafts and instruments are usually manufactured based on CAD data, so CAD data exists. Since the CAD data includes information about dimensions, the display processing unit 11b can associate this information with coordinate information such as a volume data set, and then display a three-dimensional image created based on the volume data set and an image created based on the CAD data on the display device 1 at equal sizes, and can display a change in the display direction and size of one image in conjunction with the other image. This allows a preoperative simulation of whether the artificial graft or instrument is suitable for the patient. The CAD data may be stored in the data storage unit 12, or may be obtained from the outside through the data input / output interface 15. In addition, the three-dimensional image based on the volume data set and the image based on the CAD data may be displayed side by side, or may be displayed overlapping each other.

[0117] (Arrangement of two display areas on a display device) In the above embodiment, the screen of the display device 13 is divided into left and right halves, with the left side being the first display area and the right side being the second display area. However, the arrangement of the first display area and the second display area may be reversed. Also, the left and right may be arbitrarily switched by an input by the operator via the input device 14.

[0118] Furthermore, when the medical image processing system 1 is a portable medical image processing terminal 10, since the orientation of the medical image processing terminal 10 can be arbitrarily changed, for example, when the medical image processing terminal 10 is in a landscape orientation, the first display area and the first display area are arranged side by side, and when the medical image processing terminal 10 is in a portrait orientation, the first display area and the second display area are arranged vertically. The arrangement of each display area may be changed according to the orientation of the medical image processing terminal 10. The change in the orientation of the arrangement of each display area may be due to an input through the input device 14 by the operator, or the orientation of the medical image processing terminal 10 may be detected by an appropriate sensor such as an acceleration sensor, and automatically switched based on the detection result.

[0119] (Transparency of anatomical structures in the display of three-dimensional images) In the three-dimensional image 300, it may be displayed semi-transparently according to the number of anatomical structures to be displayed and / or the importance of the display. In this case, the transparency may be set for each file of the dataset, or may be arbitrarily set by the operator through an input from the input device 14. It is also possible to set the transparency in multiple steps for each anatomical structure.

[0120] Furthermore, when the arterial and venous vessels are displayed at least partially overlapping in the depth direction of the screen, when a complexly shaped anatomical structure is displayed at least partially overlapping with another anatomical structure in the depth direction of the screen, or when many anatomical structures are displayed at least partially overlapping in the depth direction of the screen, etc., if all of the multiple anatomical structures displayed overlapping are displayed semi-transparently, it may be difficult to grasp the front-back relationship of the anatomical structures. In such a case, for the anatomical structure that is displayed at least partially overlapping behind the semi-transparently displayed anatomical structure, the part that overlaps with the anterior anatomical structure may not be displayed. By doing so, the front-back relationship between the semi-transparently displayed anatomical structures can be easily grasped. This process can be performed by the display processing unit 11b.

[0121] (Temporal phase of the dataset) When acquiring a dataset such as a volume dataset by the imaging device 23, a contrast agent is often injected into the subject and the contrast effect of the contrast agent is utilized. In this case, due to the difference in the time from the injection of the contrast agent until it reaches each blood vessel, the time when the contrast effect appears may vary depending on the type of anatomical structure, etc. For example, usually, the contrast effect appears later in veins than in arteries. Therefore, as methods for acquiring volume datasets for a plurality of anatomical structures, there are mainly the following two methods.

[0122] The first method is a method of injecting a contrast agent and performing an imaging operation by the imaging device 23 after time t1, and then determining the type of anatomical structure based on the standard contrast effect value at time t1, for example, the CT value, for the data obtained thereby. In this case, it is divided for each anatomical structure, and a plurality of volume data acquired at the same time are created for each.

[0123] The second method is a method of injecting a contrast agent and performing an imaging operation by the imaging device 23 at time t1, acquiring volume data for, for example, arterial blood vessels, and then performing an imaging operation by the imaging device 23 again at a subsequent time t2 to acquire volume data for, for example, venous blood vessels. In this case, it is divided for each anatomical structure, and a plurality of volume data acquired at different times are created for each.

[0124] In the present invention, volume data obtained by any method can be used, and the volume data divided for each anatomical structure are combined to form a volume dataset.

[0125] However, since the volume data set obtained by the first method has different contrast effect values for each anatomical structure, clear images may not be obtained for anatomical structures with low contrast effect values. In addition, since the volume data set obtained by the second method is data acquired at different times for each anatomical structure, due to changes in the subject's respiration and the position of the anatomical structure caused by differences in time, there may be a shift in the positional relationship for each anatomical structure.

[0126] To eliminate these problems, it is preferable to use a data set obtained by injecting a contrast agent by an injection method that can obtain an appropriate contrast effect for a plurality of anatomical structures in a single imaging operation. Therefore, it is preferable to divide the injection phase of the contrast agent into a plurality of phases corresponding to the plurality of target anatomical structures, and to inject the contrast agent by an injection method in which the interval of the phases is set so that the contrast agent injected in each phase gives an appropriate contrast effect within the range of the imaging operation time for each anatomical structure. Thereby, a data set can be obtained in which each anatomical structure is clearly depicted and an image without positional displacement between the anatomical structures can be created.

[0127] In the above injection method, the injection amount of the contrast agent can be determined by a known method based on the subject's body weight and the like. Then, the determined injection amount is distributed to each phase. The injection rate of the contrast agent in each phase may be constant, or may be increased or decreased according to the passage of time. Also, the injection rates may be different in each phase. The interval time can be set between 10 seconds and 120 seconds. In addition, physiological saline may be injected during the interval between each phase and / or after the last phase.

[0128] (Output of the image to other devices) The display processing unit 11b causes at least one of the created three-dimensional image 300 and two-dimensional image 200 (second image) to be displayed on the display device 13, and can also output the image as data to other devices via the data input / output interface 15. For outputting an image to other devices, the other devices are connected to the image display control unit 11 via the data input / output interface 15, either wired or wirelessly.

[0129] As an example of another device that outputs the image created by the display processing unit 11b, a projection device can be mentioned. As shown in FIG. 15, the projection device 30 can include a light source 31, an image forming processing unit 32 that forms a projection image based on the image input as data by the light from the light source, and a projection optical system 33 that projects the light emitted from the image forming processing unit 32. The image forming processing unit 32 can have, for example, an image forming element such as a liquid crystal panel and a controller that controls the image forming element. By arranging the projection device 30 at a position where an image is projected onto the body of a patient who undergoes a procedure such as surgery, for example, the projection device 30 can be combined with, for example, the medical image processing terminal 10 to constitute a treatment support system.

[0130] In such a treatment support system, prior to the treatment of the patient, a data set acquired in advance for that patient is stored in the data storage unit 12 of the medical image processing terminal 10. The image display control unit 11 (specifically, the display processing unit 11b) of the medical image processing terminal 10 creates an image including at least the three-dimensional image 300 from the data set stored in the data storage unit 12 according to the above-described processing procedure. The created image is displayed on the display device 13 and is output to the projection device 30 manually by the operator or automatically.

[0131] When the generated image is output to the projection device 30, the image output to the projection device 30 is projected from the projection device 30 onto the patient's body. For example, when the three-dimensional image 300 is output to the projection device 30, the same three-dimensional image 300 as that displayed on the display device 13 is projected onto the patient's body by the projection device 30. The operator can adjust the display size and position of the three-dimensional image 300 displayed on the display device 13 on the display device 13 by appropriately operating the input device 14 so that the size and position of the three-dimensional image 300 projected onto the patient's body match the corresponding part on the patient's body. By doing so, the treatment of the patient can be efficiently performed while confirming it with the anatomical structure projected onto the patient's body as an image.

[0132] As described above, in the three-dimensional image 300, the hue is set for each anatomical structure. However, depending on the relationship with the color of the actual anatomical structure, the image displayed on the body may be difficult to see. Therefore, it is preferable that the image display control unit 11 is configured to be able to arbitrarily set the hue of the anatomical structure of the three-dimensional image 300 created by the display processing unit 11b. In addition, when the size of the projected image is determined, the image display control unit 11 may have a function of locking at least one or any combination of two or more of the size change, rotation, and movement of the display image (a function that makes the change impossible) by the operation of the operator or the like.

[0133] The projection device 30 may have a correction function such that, depending on the three-dimensional shape of the patient's body onto which the image is to be projected, a projection image similar to that obtained when projected onto a plane can be obtained. For such a correction function, for example, a computer program for performing so-called "projection mapping" can be used. In this case, for example, a grid (square grid) is projected onto the patient, analyzed to obtain data, and while using the data, image processing is performed to correct the assumed image distortion so that an image can be obtained without being affected by the three-dimensional shape of the patient's body.

[0134] (Actual size display function) When the data set that is the basis of the three-dimensional image 300 is a volume data set, the actual dimensions of each voxel can be obtained from the coordinate information of the volume data set. On the other hand, the display device 13 can display the image in actual size by appropriately changing the size of the displayed image according to the pixel pitch of the display device 13. Therefore, by obtaining the actual dimension per voxel of the volume data set and associating the obtained actual dimension per voxel with the pixel pitch of the display device 13, the image can be displayed in actual size on the display device 13. The above actual size display process can be performed by the display processing unit 11b. Further, after executing the actual size display process, the image display control unit 11 may have a function of locking at least one or any combination of two or more of the size change, rotation, and movement of the display image (a function that makes the change impossible) by an operation of the operator or the like.

[0135] The actual size display of the image on the display device 13 can be performed by an operation of the operator. For the operation of the operator, the display processing unit 11b causes the display device 13 to display an actual size display icon for executing the actual size display, and by the operator tapping this icon, the above actual size display process can be executed.

[0136] As the pixel pitch used for the actual size display process, a value defined as the specification of the display device 13 can be used. Alternatively, the pixel pitch can also be calculated from the screen size and the number of pixels in the vertical and horizontal directions of the display device 13.

[0137] As described above, by displaying the image in actual size on the display device 13, for example, in the preparation stage of the procedure for transplanting an artificial heart valve to a patient, with the cross-sectional image near the patient's heart displayed in actual size on the display device 13, the artificial heart valve to be transplanted (or its physical model) can be placed on the image displayed on the display device 13, and it can be used to confirm whether the artificial heart valve is of a size suitable for the patient.

[0138] (Supplementary Note) Although the present invention has been described above, this specification discloses the invention described below. However, the disclosed matters in this specification are not limited to the following invention.

[0139] [1] A display device for displaying an image, An input device for receiving an input by an operator, A data storage unit for storing at least one data set capable of creating a three-dimensional image representing a three-dimensional arrangement of a plurality of anatomical structures, An image display control unit for controlling the display of the image on the display device according to the input received by the input device, and the image display control unit, performs a process of creating at least the three-dimensional image from the data set, a process of simultaneously displaying the created three-dimensional image and a second image that has a different format from the three-dimensional image but is associated with the three-dimensional image on the display device, a process of creating an auxiliary image from information included in at least one of the three-dimensional image and the second image in response to the input received by the input device or independently of the input, and displaying the auxiliary image on the other of the three-dimensional image and the second image, A medical image processing system configured to perform the above.

[0140] [2] The medical image processing system according to [1], wherein the second image is a two-dimensional image representing a cross-section of the plurality of anatomical structures.

[0141] [3] The three-dimensional image is composed of a plurality of layers set for each of the anatomical structures, the information includes a plurality of layer information that is information indicating the plurality of layers, in the process of displaying the auxiliary image, the image display control unit, processing for displaying the three-dimensional image in different hues for each layer; processing for creating, as the auxiliary image, a cross-sectional image of the three-dimensional image at a cross-section corresponding to the cross-section represented by the two-dimensional image in at least one layer among the plurality of layers, in the same hue as the color displayed in the three-dimensional image; processing for superimposing the cross-sectional image on the two-dimensional image; The medical image processing system according to [2], which is configured to perform the above.

[0142] [4] The plurality of anatomical structures include arterial vessels and venous vessels. The medical image processing system according to [3], wherein the image display control unit is configured to perform processing for creating the cross-sectional image for the layer set as the arterial vessel and the layer set as the venous vessel in the processing for creating the cross-sectional image.

[0143] [5] The medical image processing system according to any one of [2] to [4], wherein the information includes coordinate information common to the three-dimensional image and the two-dimensional image.

[0144] [6] In the processing for displaying the auxiliary image, when an arbitrary coordinate on the two-dimensional image is input from the input device, the image display control unit performs processing including displaying, as the auxiliary image, a planar image corresponding to the cross-section represented by the displayed two-dimensional image at a position passing through the input coordinate on the three-dimensional image. The medical image processing system according to [5].

[0145] [7] In the processing for displaying the auxiliary image, when an arbitrary coordinate on the two-dimensional image is input from the input device, the image display control unit performs processing including displaying, as the auxiliary image, a highlighted contour image in which the contour portion of the anatomical structure at the cross-section represented by the displayed two-dimensional image is represented in a hue different from other portions on the three-dimensional image. The medical image processing system according to [5] or [6].

[0146] [8] In the process of displaying the auxiliary image, when any coordinate on the two-dimensional image is input from the input device, the image display control unit performs a process including a process of displaying a cursor as the auxiliary image at a position corresponding to the input coordinate on the three-dimensional image. The medical image processing system according to any one of [5] to [7].

[0147] [9] In the process of displaying the auxiliary image, when any coordinate on the anatomical structure of the three-dimensional image is input from the input device, the image display control unit performs a process including a process of displaying a thumbnail of the two-dimensional image having the coordinate information matching the input coordinate as the auxiliary image on the display device separately from the two-dimensional image and the three-dimensional image. The medical image processing system according to [5].

[0148]

[10] In the process of displaying the auxiliary image, when an input representing the meaning of approving the thumbnail is input from the input device, the image display control unit further performs a process of enlarging and displaying the thumbnail. The medical image processing system according to [9].

[0149]

[11] When one of the anatomical structures on the three-dimensional image is designated by an input from the input device, the image display control unit is further configured to perform a process of displaying the designated anatomical structure semi-transparently. The medical image processing system according to any one of [1] to

[10] .

[0150]

[12] In response to an input from the input device, the image display control unit is further configured to perform a process of changing the sizes of the three-dimensional image display area where the three-dimensional image is displayed and the second image display area where the second image is displayed. The medical image processing system according to any one of [1] to

[10] .

[0151]

[13] In the size change process, the image display control unit A process of displaying a size-changing icon between the three-dimensional image display area and the second image display area, A process of changing the display position of the size-changing icon according to the input from the input device, The medical image processing system according to

[12] , which performs a process of changing the sizes of the three-dimensional image display area and the second image display area according to the display position of the size-changing icon.

[0152]

[14] A medical image processing method using a display device for displaying an image, an input device for receiving an input by an operator, and a data storage unit for storing at least one data set capable of creating a three-dimensional image representing a three-dimensional arrangement of a plurality of anatomical structures, comprising: Creating the three-dimensional image from the data set, Simultaneously displaying the created three-dimensional image and a second image that has a different format from the three-dimensional image but is associated with the three-dimensional image on the display device, Creating an auxiliary image from information included in at least one of the three-dimensional image and the second image according to the input received by the input device or independently of the input, and displaying the auxiliary image on the other of the three-dimensional image and the second image, A medical image processing method including the above.

[0153]

[15] A medical image processing program that causes a computer to execute a process of displaying at least the three-dimensional image created from at least one data set capable of creating a three-dimensional image representing a three-dimensional arrangement of a plurality of anatomical structures stored in a data storage unit on a display device, comprising: Causing the computer to: Create the three-dimensional image from the data set, Simultaneously display the created three-dimensional image and a second image that has a different format from the three-dimensional image but is associated with the three-dimensional image on the display device, A process of creating an auxiliary image from information included in at least one of the three-dimensional image and the second image, and displaying the auxiliary image on the other of the three-dimensional image and the second image; A medical image processing program for causing the above to be executed.

[0154] A computer-readable recording medium storing the medical image processing program according to

[16]

[15] .

Explanation of Signs

[0155] 1 Medical image processing system 10 Medical image processing terminal 11 Image display control unit 11a Input determination unit 11b Display processing unit 11c Auxiliary image processing unit 12 Data storage unit 13 Display device 14 Input device 15 Data input / output interface 21 Medical information management device 22 Chemical solution injection device 23 Imaging device 100a Three-dimensional image display area 100b Two-dimensional image display area 200 Two-dimensional image 300 Three-dimensional image 400 Thumbnail

Claims

1. A display device for displaying an image; an input device for receiving an input from an operator; a data storage unit for storing at least one data set capable of generating a three-dimensional image representing a spatial configuration of a plurality of anatomical structures; an image display control unit that controls display of the image on the display device in accordance with an input received by the input device; having The image display control unit generating at least said three-dimensional image from said data set; displaying the created three-dimensional image and a second image associated with the three-dimensional image on the display device simultaneously; creating an auxiliary image from information included in at least one of the three-dimensional image and the second image in response to an input received by the input device or independently of the input, and displaying the auxiliary image on the other of the three-dimensional image and the second image; A medical image processing system configured to:

2. The medical image processing system of claim 1 , wherein the second image is a two-dimensional image representing a cross-section of the plurality of anatomical structures.

3. The three-dimensional image is composed of a plurality of layers set for each of the anatomical structures, the information includes a plurality of layer information pieces that are information pieces indicating the plurality of layers, The image display control unit, in the process of displaying the auxiliary image, A process of displaying the three-dimensional image in a different hue for each layer; creating, for at least one layer among the plurality of layers, a cross-sectional image of the three-dimensional image at a cross-section corresponding to a cross-section represented by the two-dimensional image, as the auxiliary image, in the same hue as the color displayed in the three-dimensional image; superimposing the cross-sectional image onto the two-dimensional image; 3. The medical image processing system of claim 2, configured to perform the following:

4. the plurality of anatomical structures includes an arterial system and a venous system; The medical image processing system according to claim 3 , wherein the image display control unit is configured to perform processing for creating the cross-sectional images for a layer set as the arterial blood vessels and a layer set as the venous blood vessels in the processing for creating the cross-sectional images.

5. The medical image processing system according to claim 2 , wherein the information includes coordinate information common to the three-dimensional image and the two-dimensional image.

6. The medical image processing system of claim 5, wherein the image display control unit performs a process that, in the process of displaying the auxiliary image, includes a process of, when any coordinate on the two-dimensional image is input from the input device, displaying a planar image corresponding to the cross section represented by the displayed two-dimensional image as the auxiliary image at a position passing through the input coordinate on the three-dimensional image.

7. The medical image processing system of claim 5 or 6, wherein the image display control unit performs processing including, in the process of displaying the auxiliary image, when any coordinate on the two-dimensional image is input from the input device, displaying a highlighted contour image, in which the contour of the anatomical structure in the cross section represented by the displayed two-dimensional image is represented in a hue different from other parts, as the auxiliary image on the three-dimensional image.

8. The medical image processing system of any one of claims 5 to 7, wherein the image display control unit performs processing including, in the process of displaying the auxiliary image, when any coordinate on the two-dimensional image is input from the input device, displaying a cursor as the auxiliary image at a position on the three-dimensional image corresponding to the input coordinate.

9. The medical image processing system of claim 5, wherein the image display control unit performs a process that, when any coordinate on the anatomical structure of the three-dimensional image is input from the input device, displays a thumbnail of the two-dimensional image having coordinate information that matches the input coordinate on the display device as the auxiliary image, separately from the two-dimensional image and the three-dimensional image, in the process of displaying the auxiliary image.

10. The medical image processing system according to claim 9 , wherein the image display control unit, in the process of displaying the auxiliary image, further performs a process of enlarging and displaying the thumbnail when an input indicating approval of the thumbnail is input from the input device.

11. The medical image processing system according to any one of claims 1 to 10, wherein the image display control unit is configured to further perform processing for displaying the specified anatomical structure semi-transparently when one of the anatomical structures on the three-dimensional image is specified by input from the input device.

12. The medical image processing system of any one of claims 1 to 10, wherein the image display control unit is further configured to perform a resize process to change the size of a three-dimensional image display area in which the three-dimensional image is displayed and a second image display area in which the second image is displayed in response to input from the input device.

13. The image display control unit, in the size change process, A process of displaying a size change icon between the three-dimensional image display area and the second image display area; A process of changing a display position of the size change icon in response to an input from the input device; The medical image processing system according to claim 12 , wherein a process of changing the sizes of the three-dimensional image display area and the second image display area is performed in accordance with the display position of the size change icon.

14. 1. A medical image processing method using a display device for displaying an image, an input device for receiving an input from an operator, and a data storage unit for storing at least one data set capable of generating a three-dimensional image showing a three-dimensional arrangement of a plurality of anatomical structures, comprising: generating said three-dimensional image from said data set; simultaneously displaying the created three-dimensional image and a second image in a different format from the three-dimensional image but associated with the three-dimensional image on the display device; creating an auxiliary image from information included in at least one of the three-dimensional image and the second image in response to an input received by the input device or independently of the input, and displaying the auxiliary image on the other of the three-dimensional image and the second image; A medical image processing method comprising:

15. A medical image processing program that causes a computer to execute a process of displaying at least a three-dimensional image, which is created from at least one data set capable of creating a three-dimensional image representing a three-dimensional arrangement of a plurality of anatomical structures and is stored in a data storage unit, on a display device, the process comprising: The computer includes: generating said three-dimensional image from said data set; displaying the created three-dimensional image and a second image associated with the three-dimensional image on the display device simultaneously; creating an auxiliary image from information included in at least one of the three-dimensional image and the second image, and displaying the auxiliary image on the other of the three-dimensional image and the second image; A medical image processing program that executes the above.

16. A computer-readable recording medium storing the medical image processing program according to claim 15.

Images