Medical image processing apparatus and medical image processing method

Abstract

To support an operation so that an artificial valve can be indwelled at an appropriate angle for each patient.SOLUTION: A medical image processing apparatus includes an acquisition unit, a first region identification unit, a first direction identification unit, a second region identification unit, a second direction identification unit, and a feature amount calculation unit. The processing circuitry is configured to acquire a three dimensional medical image of a patient in whom an artificial valve needs to be placed. The first region specification unit specifies a first region of interest from the three dimensional medical image. The first direction identification unit identifies a first direction of the three dimensional medical image based on the first region of interest. The second region identification unit identifies a second region of interest from the three dimensional medical image. The second direction identification unit identifies a second direction of the three dimensional medical image based on the second region of interest. The feature amount calculation unit calculates a first feature amount related to the indwelling based on the first direction and the second direction.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The embodiments disclosed in this specification and the drawings relate to a medical image processing device and a medical image processing method. [Background technology]

[0002] The aortic valve of the heart has three leaflets that open and close, with the joints (commissures) between each leaflet spaced approximately 120° apart. This aortic valve is located between the left ventricle of the heart and the aorta, and the leaflets open and close in response to the movement of the heart, allowing blood to flow from the heart to the aorta. This aortic valve can develop aortic stenosis, in which the opening of the leaflets is restricted and the aortic valve narrows due to various causes.

[0003] Transcatheter aortic valve implantation (TAVI) is a known treatment for aortic stenosis. In TAVI, a prosthetic valve that replaces the patient's aortic valve is carried along the blood vessels by a catheter and placed so that it roughly overlaps the position of the aortic valve. In the current TAVI surgical procedure, to place the prosthetic valve at the appropriate angle, the prosthetic valve is rotated at a certain angle (currently 90°) relative to the patient before insertion and then maneuvered to the appropriate position by catheter manipulation. Furthermore, in the current surgical procedure, the same procedure may be recommended for patients with different anatomical structures based on statistical analysis of the results of prior clinical evaluations.

[0004] According to the inventor's investigations, while performing this same procedure, the shape of the aortic valve, such as the positional relationship of the commissures, differs from patient to patient, and therefore the appropriate placement position of the artificial valve also differs from patient to patient, leaving room for improvement in terms of the catheter operation procedure, including the angle at which the artificial valve is inserted. That is, according to the inventor's investigations, in order to place an artificial valve at an angle appropriate for each patient, different operation procedures are required for each patient, and therefore it is preferable to provide support for these operation procedures. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Laid-Open No. 2014-083357 Summary of the Invention [Problem to be solved by the invention]

[0006] One of the problems to be solved by the embodiments disclosed in this specification and the drawings is to assist surgery so that an artificial valve can be placed at an appropriate angle for each patient. However, the problems to be solved by the embodiments disclosed in this specification and the drawings are not limited to the above problem. Problems corresponding to the effects of each configuration shown in the embodiments described below can also be positioned as other problems. [Means for solving the problem]

[0007] A medical image processing apparatus according to an embodiment includes an acquisition unit, a first region identification unit, a first direction identification unit, a second region identification unit, a second direction identification unit, and a feature calculation unit. The acquisition unit acquires a three-dimensional medical image including image information related to a patient's aortic valve. The first region identification unit identifies a first region of interest from the three-dimensional medical image. The first direction identification unit identifies a first direction of the three-dimensional medical image based on the first region of interest. The second region identification unit identifies a second region of interest from the three-dimensional medical image. The second direction identification unit identifies a second direction of the three-dimensional medical image based on the second region of interest. The feature calculation unit calculates a first feature related to the shape of the aortic valve based on the first direction and the second direction. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a block diagram showing an example of the configuration of a medical image processing system including a medical image processing apparatus according to an embodiment. [Figure 2] FIG. 2 is a flowchart illustrating the operation in one embodiment. [Figure 3]FIG. 3 is a schematic diagram for explaining the operation in one embodiment. [Figure 4] FIG. 4 is a schematic diagram for explaining the operation of FIG. [Figure 5] FIG. 5 is a schematic diagram for explaining the operation relating to the first region of interest in one embodiment. [Figure 6] FIG. 6 is a schematic diagram for explaining the operation of FIG. [Figure 7] FIG. 7 is a schematic diagram for explaining the operation of FIG. [Figure 8] FIG. 8 is a schematic diagram for explaining the operation of FIG. [Figure 9] FIG. 9 is a schematic diagram for explaining the operation relating to the second region of interest in one embodiment. [Figure 10] FIG. 10 is a schematic diagram for explaining a first modified example of an embodiment. [Figure 11] FIG. 11 is a schematic diagram for explaining a second modified example of the embodiment. [Figure 12] FIG. 12 is a schematic diagram for explaining a second modified example of the embodiment. [Figure 13] FIG. 13 is a schematic diagram for explaining a third modified example of the embodiment. [Figure 14] FIG. 14 is a schematic diagram for explaining a fourth modified example of the embodiment. [Figure 15] FIG. 15 is a schematic diagram showing an example of the menu of FIG. [Figure 16] FIG. 16 is a schematic diagram showing an example of the menu of FIG. [Figure 17] FIG. 17 is a schematic diagram showing an example of the split screen of FIG. [Figure 18] FIG. 18 is a schematic diagram showing an example of the alignment display. [Figure 19] FIG. 19 is a schematic diagram showing an example of a setting screen in the menu of FIG. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, a medical image processing apparatus and a medical image processing method according to an embodiment will be described with reference to the drawings. In the following description, components having substantially the same functions and configurations are designated by the same reference numerals, and redundant descriptions will be provided as appropriate and necessary.

[0010] FIG. 1 is a diagram showing an example of the configuration of a medical image processing system 100 according to an embodiment. The medical image processing system 100 includes a plurality of medical image diagnostic devices 10, an image storage device 20, and a medical image processing device 30. As shown in FIG. 1, the medical image diagnostic devices 10, the image storage device 20, and the medical image processing device 30 are connected to each other via a network, whether wireless or wired, so that they can communicate with each other. The network is, for example, a local area network (LAN). Note that the connection line is not limited to a LAN as long as security is ensured by a virtual private network (VPN) or the like. In this case, the network may be, for example, a public communication line such as the Internet.

[0011] The medical image processing system 100 may be implemented, for example, in the form of a thin client, in which a client device used by an operator performs the minimum necessary processing, and a server device performs most of the processing. In this case, the medical image processing device 30 functions as the server device. When the medical image processing system 100 is implemented in the form of a thin client, the client device (not shown) is connected to the network as, for example, a terminal device having an input interface, a display, a memory, and a processing circuit with a control function, as described below. In this case, the terminal device may function, for example, as a medical image display device.

[0012] The multiple medical image diagnostic devices 10 perform imaging of a patient and collect medical image data. The medical image diagnostic devices 10 collect, for example, medical image data depicting blood vessels in the patient. The medical image diagnostic devices transmit the collected medical image data to an image storage device 20 and a medical image processing device 30. For the sake of concrete explanation, the multiple medical image diagnostic devices 10 are assumed to be X-ray computed tomography (hereinafter referred to as CT) devices. Note that the medical image diagnostic devices 10 are not limited to X-ray CT devices, and may be other imaging devices capable of acquiring medical image data, such as an X-ray diagnostic device or a magnetic resonance imaging device.

[0013] An X-ray CT device collects medical image data about a patient by performing a CT scan on the patient. At this time, scan conditions are attached to the medical image data. The CT scan conditions include, for example, the region to be imaged, the scan method, the number of views, the tube voltage, the tube current, and the patient's position (posture) when the CT scan is performed.

[0014] The image storage device 20 is a device that stores medical image data collected by the medical image diagnostic device 10. The image storage device 20 acquires medical image data from the medical image diagnostic device 10 via a network and stores the acquired medical image data in a memory provided inside or outside the device. For example, the image storage device 20 is realized by a computer device such as a server device.

[0015] The medical image processing device 30 acquires medical image data from the medical image diagnostic device 10 or the image storage device 20 via a communication interface (not shown) over a network, and performs various processes using the acquired medical image data. The medical image processing device 30 is realized by, for example, a computer device such as a workstation. Note that the medical image diagnostic device 10, the image storage device 20, and the medical image processing device 30 may be installed in any location as long as they are connectable over a network. For example, the medical image processing device 30 may be installed in a facility, hospital, or the like different from the medical image diagnostic device 10. Furthermore, the medical image processing device 30 may be mounted on the medical image diagnostic device 10, such as an X-ray diagnostic device.

[0016] As shown in FIG. 1, the medical image processing device 30 includes an input interface 31, a display 32, a memory 33, and a processing circuit .

[0017] The input interface 31 may be realized by a trackball, switch buttons, mouse, keyboard, touchpad (or trackpad) for inputting various instructions, commands, information, selections, and settings from the operator (user) into the medical information processing device main body, a touch panel display (or touch screen) that integrates a display screen and a touchpad, or the like. The input interface 31 is connected to the processing circuitry 34 and converts input operations received from the user into electrical signals and outputs them to the processing circuitry 34. In this case, the input interface 31 may display a GUI (graphical user interface) on the display 32, allowing the user to input various instructions using physical operation components such as a mouse or keyboard. Note that, in this specification, the input interface 31 is not limited to those having physical operation components. For example, an electrical signal processing circuit that receives electrical signals corresponding to input operations from an external input device provided separately from the device and outputs these electrical signals to the processing circuitry 34 is also included as an example of the input interface 31. In the following description, "operation of the input interface 31 by the user" is also referred to as "user operation."

[0018] The display 32 is composed of a display main body that displays any data, an internal circuit that supplies display signals to the display main body, and peripheral circuits such as connectors and cables that connect the display main body to the internal circuitry. The display 32 displays various information under the control of the control function of the processing circuit 34. For example, the display 32 displays various images generated by the processing circuit 34. The display 32 also displays a GUI as the input interface 31. Any of a variety of displays can be used as appropriate for the display 32. For example, a liquid crystal display, an organic electroluminescence (EL) display, or a plasma display can be used for the display 32. The display 32 may be a desktop type, or may be composed of a tablet terminal or the like that is capable of wireless communication with the medical image processing device 30.

[0019] The memory 33 is composed of memories for recording electrical information, such as a read only memory (ROM), a random access memory (RAM), a hard disk drive (HDD), and an image memory, as well as peripheral circuits associated with these memories, such as a memory controller and a memory interface. The memory 33 stores, for example, various programs, such as a medical image processing program for the medical image processing device 30, and various data, such as various tables, data in the middle of processing, and data after processing. The medical image processing program is a program for causing a computer to function as the medical image processing device 30 that executes a medical image processing method. The medical image processing program may be stored in, for example, a non-transitory computer-readable storage medium and distributed, and may be read from the storage medium and installed in the memory 33.

[0020] The processing circuitry 34 reads out a medical image processing program stored in the memory 33 based on instructions input by a user via the input interface 31, and controls the medical image processing device 30 in accordance with the program. For example, the processing circuitry 34 is a processor that implements each function of the medical image processing device 30 in accordance with the medical image processing program read out from the memory 33. The functions include, for example, an acquisition function 34a, a display control function 34b, a region identification function 34c, a direction identification function 34d, and a feature calculation function 34e. Note that each function may be implemented by distributing it among multiple processors as appropriate. Alternatively, each function or part of each function may be executed by another device as appropriate.

[0021] Next, the functions of the processing circuitry 34 will be described in order: the acquisition function 34a, the display control function 34b, the area identification function 34c, the direction identification function 34d, and the feature calculation function 34e. However, the allocation of the functions described below is for convenience and can be changed as appropriate. This is because even if a process that is assigned to one function is assigned to another function, the processing circuitry 34 still executes that process. Note that the ability to change the allocation of the functions also applies to the following embodiments and modified examples.

[0022] The acquisition function 34a acquires three-dimensional medical images of a patient requiring placement of an artificial valve. For example, the acquisition function 34a acquires three-dimensional medical images including image information related to the patient's aortic valve. For example, the acquisition function 34a may acquire the three-dimensional medical images from either the medical image diagnostic device 10 or the image storage device 20. For example, the acquisition function 34a may acquire the three-dimensional medical images from the memory 33 that stores the three-dimensional medical images. The acquisition function 34a is an example of an acquisition unit.

[0023] The display control function 34b displays data during or as a result of processing by the processing circuitry 34 on the display 32. For example, the display control function 34b may display the three-dimensional medical image on the display 32. Alternatively, for example, the display control function 34b may display a first display area that displays the three-dimensional medical image based on a first display orientation and a second display area that displays the three-dimensional medical image based on a second display orientation side by side on the display 32. In this case, the display control function 34b may update the first display orientation or the second display orientation in response to a user operation, and update the display content of the display 32 based on the updated result. The display control function 34b is an example of a display control unit.

[0024] The region identification function 34c identifies a first region of interest from the 3D medical image. The region identification function 34c also identifies a second region of interest from the 3D medical image. The first region of interest is, for example, an image region of a first anatomical structure. The second region of interest is, for example, an image region of a second anatomical structure. The 3D medical image is, for example, a medical image including image information related to a patient's aortic valve. The image information of the aortic valve enables the aortic valve to be displayed as a cross-sectional region of a 3D medical image of the heart region. Here, we will provide additional information about the anatomical structure related to the aortic valve. A typical aortic valve has three leaflets (right coronary cusp, left coronary cusp, and non-coronary cusp) that open and close, with the junctions (commissures) between the leaflets spaced approximately 120° apart. The aortic valve is located between the left ventricle and the aorta of the heart, and controls blood flow from the heart to the aorta by the leaflets opening and closing in response to the movement of the heart. Specifically, blood flows from the left ventricle through the aortic valve in the sinus of Valsalva, then through the ascending aorta, the aortic arch, and the descending aorta. Furthermore, a portion of the blood flow that passes through the aortic valve flows into the left and right coronary arteries via two coronary artery ostia in the sinus of Valsalva. The prosthetic valve placed inside the aortic valve has three valve leaflets (right coronary cusp, left coronary cusp, and non-coronary cusp) that open and close, similar to the anatomical structure of a typical aortic valve, with the junctions (commissures) between the leaflets spaced approximately 120° apart. The region identification function 34c can appropriately identify any structure of interest, such as the ascending aorta, the descending aorta, the commissures, and the coronary artery ostia, as a region of interest. The anatomical structure of the aortic valve is merely a brief example, and any known anatomical structure can be used as appropriate. Similarly, the region of interest is not limited to the above-described example of the structure of interest. The region specifying function 34c is an example of a first region specifying unit and a second region specifying unit.

[0025] The direction identification function 34d identifies a first direction of the three-dimensional medical image based on the first region of interest. Furthermore, the direction identification function 34d identifies a second direction of the three-dimensional medical image based on the second region of interest. The "direction" here refers to a parameter that specifies a two-dimensional image to be displayed on a display from among two-dimensional images that can be displayed from the three-dimensional medical image when the three-dimensional medical image is observed from a predetermined position. Parameters that correspond to the "direction" include, for example, the display direction and a direction corresponding to the observation angle. However, any parameter that can be identified based on the region of interest can be used as the "direction." The direction identification function 34d is an example of a first direction identification unit and a second direction identification unit.

[0026] The feature calculation function 34e calculates a first feature related to the placement of the artificial valve based on the first direction and the second direction. For example, the feature calculation function 34e calculates a first feature related to the shape of the aortic valve based on the first direction and the second direction. When the first direction or the second direction is updated, the feature calculation function 34e may update the display content of the display 32 based on the updated result. The feature calculation function 34e may also calculate, as the first feature, an angle at which the artificial valve is inserted into the patient. The feature calculation function 34e may also calculate, as the angle, an amount of rotation of the artificial valve around the axis of a catheter holding the artificial valve. Here, the angle may be an angle dependent on the commissures of the aortic valve. The amount of rotation may be a rotation angle for aligning the commissures of the artificial valve with the commissures of the aortic valve. The feature calculation function 34e is an example of a feature calculation unit.

[0027] Next, the operation of the medical image processing device configured as described above will be described using the flowchart in Fig. 2 and the schematic diagrams in Figs. 3 to 9. The following description will be given taking as an example a case where the amount of rotation of an artificial valve in TAVI is calculated based on the positional relationship between the ascending aorta and the descending aorta in a CT image and the position of the commissure of the aortic valve in the CT image. Furthermore, the first direction, second direction, and third direction will be described taking as an example the first display direction, second display direction, and third display direction, respectively. However, the operation of the medical image processing device 30 is not limited to this example.

[0028] (Step S1) The processing circuitry 34 of the medical image processing device 30 acquires, via the acquisition function 34a, a 3D medical image containing image information about the patient's aortic valve. For example, the processing circuitry 34 acquires a 3D CT image of the patient from the image storage device 20. The 3D medical image may be of any type as long as it stores morphological information about the 3D anatomical structure of the target biological tissue. For example, the 3D medical image may be a medical image captured by another imaging device, such as an ultrasound image, an MRI image, an X-ray image, a PET image, or a SPECT image, or may be part of a 4D image obtained by capturing multiple 3D medical images over time. Step S1 may be initiated by a user's instruction, or may be automatically initiated when a new medical image is stored in the image storage device 20, such as a PACS. Alternatively, the processing circuitry 34 may determine whether the new medical image satisfies a predetermined condition and, if so, execute the processing. The condition may be any condition that can determine the image status. For example, the processing circuitry 34 may execute the processing when the medical image is captured using an imaging protocol targeting the heart as a condition for the imaging protocol, or when the medical image is enlarged and reconstructed as a condition for the reconstruction method, or when a combination of these conditions is satisfied.

[0029] (Step S2) The processing circuitry 34 uses the region identification function 34c to identify a first region of interest from the 3D medical image. Specifically, the processing circuitry 34 identifies a region of a first biological organ of interest in the acquired CT image as the first region of interest. In one embodiment, the processing circuitry 34 identifies coordinate information for each pixel of the aorta in the CT image, but any region of any biological organ may be identified. The identified aorta as the first region of interest includes both the ascending and descending aorta. Specifically, the first region of interest may be identified by specifying its position through the input interface 31, or by using a known region extraction technique to identify the first region of interest based on an anatomical structure depicted in the CT image. Examples of known region extraction techniques include Otsu's binarization method based on CT values, region growing, snake algorithm, graph cut algorithm, and mean shift algorithm. Any method for identifying the first region of interest from a medical image can be used in step S2. For example, the first region of interest may be identified using a shape model of the region of interest constructed based on training data prepared in advance using machine learning technology (including deep learning). Since applying the above-described processing, such as the graph cut method, to the entire medical image can easily result in excessive computational costs, the first region of interest may be identified by identifying a region (hereinafter referred to as a related region) that is related to the first biological organ of interest and is larger than the first region of interest but smaller than the entire medical image, and applying the above-described processing only to the related region. For example, if the first biological organ of interest is the aorta, the related region may be the cardiac region or the region surrounding the medial canthus. The related region may be set by operating the input interface 31. Furthermore, when the aorta is identified as the first region of interest, the ascending aorta and the descending aorta included in the aorta may be identified as separate regions. In this case, the above-described processing may be applied separately to the related region of the ascending aorta and the related region of the descending aorta. Alternatively, the aorta identified as the first region of interest may be separated into a region of the ascending aorta and a region of the descending aorta based on a feature of the aorta (for example, curvature, etc.). Note that in the process of step S2, the size of the first region of interest may be arbitrary, and for example, the coordinates of only one point (for example, one pixel) may be identified as the first region of interest.

[0030] (Step S3) The processing circuitry 34 uses the direction identification function 34d to identify a first display direction of the 3D medical image based on the first region of interest. Here, the first display direction is a direction in which the patient's 3D anatomical structure is observed from a predetermined direction and is a direction in which the anatomical structure is displayed in 2D on the display 32. In this example, as shown in FIG. 3, a coordinate system is used in which the patient P is placed supine on a tabletop. The coordinate system is a clinical coordinate system with the origin at the center position of the patient P's imaging. In the clinical coordinate system, the axis parallel to the body axis of the patient P is the Z axis, the axis parallel to the front direction of the patient P is the Y axis, and the axis perpendicular to the Z axis and the Y axis is the X axis. As shown in FIG. 3 , the angle of rotation of patient P to the right side around the Y-axis position along the front direction of patient P as the reference position and the Z-axis as the rotation axis is expressed as the first oblique view (Right Anterior Oblique view, hereinafter referred to as RAO), and the angle of rotation to the left side is expressed as the second oblique view (Left Anterior Oblique view, hereinafter referred to as LAO). The angle of rotation of patient P to the head side around the Y-axis position as the reference position and the X-axis as the rotation axis is expressed as the head direction (CRAnial, hereinafter referred to as CRA), and the angle of rotation of patient P to the feet side is expressed as the tail direction (CAUdal, hereinafter referred to as CAU). These are combined to specify the first display direction. That is, LAO (or RAO) of 0° and CRA (or CAU) of 0° represent the angle at which patient P is observed from the front. Furthermore, RAO of 90° and CRA of 0° represent the angle at which patient P is observed from a position tilted 90° to the right side from the front. An LAO of 90° and a CRA of 0° represent angles at which patient P is observed from a position tilted 90° to the left from the front. Figure 4 shows examples of medical images g1 to g3 that display the aorta as a first region of interest from directions corresponding to these observation angles.

[0031] In step S3, the processing circuitry 34 determines the first display orientation based on the positional relationship between the ascending aorta and the descending aorta included in the aorta, which is the first region of interest. For example, the processing circuitry 34 determines the first display orientation based on the overlapping position of the ascending aorta and the descending aorta represented in the 3D medical image relative to the front direction of the patient. Specifically, as shown in FIG. 5, from the medical image g4 showing the aorta sr1, the angle where the CRA is 0° and the RAO at the overlapping position of the descending aorta sr1b and the ascending aorta sr1a is 25° is determined as the first display orientation X1. The first display orientation X1 of the medical image g4 is determined, for example, as shown in FIGS. 6 to 8.

[0032] That is, as shown in FIG. 6, the processing circuit 34 determines, as a feature point fp, the point on the core line L1 within the range of the ascending aorta sr1a in the frontal medical image g1 where the tangent to the core line L1 at each point is closest to being perpendicular to the XY plane. More specifically, the processing circuit 34 determines, as a feature point fp, the point at which the magnitude of the Z component (component in the Z-axis direction) of the unit vector of each tangent is the largest. Next, as shown in FIG. 7, the processing circuit 34 takes a width of several centimeters in both the positive and negative directions of the Z axis around the feature point fp and calculates a region w1a of the ascending aorta sr1a and a region w1b of the descending aorta sr1b within that width. Thereafter, as shown in FIG. 8, the processing circuit 34 rotates the ascending aorta sr1a and the descending aorta sr1b about the Z axis and determines the first display direction X1 from the rotation angle at which the two regions w1a and w1b overlap most. Note that if there are multiple rotation angles at which the two regions w1a and w1b overlap, the first identified angle may be used. Furthermore, the two regions w1a and w1b must overlap with the ascending aorta sr1a positioned closer to the user (front side) than the descending aorta sr1b. The center of rotation may be, for example, the center of gravity of the aortic valve. Alternatively, a point on the descending aorta sr1b with the same Z coordinate as the feature point fp may be identified, and the first display direction X1 may be identified from the angle of the RAO at which the feature point fp and the point on the descending aorta sr1b overlap when the CRA is 0°. Note that the first display direction X1 is not limited to the CRA of 0° and the RAO of 25° shown in FIGS. 5 and 8; any direction can be used as long as it is identified based on the first region of interest. Furthermore, the first display direction X1 may be identified in any manner. For example, a VR (Volume Rendering) image display function of a known medical image display device may be used to change the display direction of a three-dimensional VR image in response to a user's operation, and the angle at which the descending aorta sr1b and the ascending aorta sr1a overlap most closely may be identified and set as the first display direction X1. Note that with the known VR image display function, it is necessary to set a rotation center for changing the display direction, but the rotation center can be set at any position. For example, the rotation center may be set at a point related to the first region of interest (e.g., the center of gravity, etc.).In another example, the amount of overlap between the descending aorta sr1b and the ascending aorta sr1a may be calculated in any of a variety of directions (e.g., 360 directions in 1-degree increments), and the direction with the greatest amount of overlap may be identified as the first display direction X1. In this case, the amount of overlap may be calculated for all regions of the ascending aorta sr1a and the descending aorta sr1b, or, as described above, the amount of overlap may be calculated for only a portion of the ascending aorta sr1a and / or the descending aorta sr1b. For example, the core line L1 of the ascending aorta sr1a or the descending aorta sr1b may be extracted based on a known technique, and feature values ​​at each position on the core line L1 may be calculated to control the region where the overlap position is calculated based on the feature values. For example, the curvature may be calculated at each position on the approximately curved core line L1, and the curvature may be used as the feature value, or the direction calculated by first-order differentiation at each position on the core line L1 may be used as the feature value. Note that the process of step S3 is not limited to a method for specifying the direction and any method can be used as long as it can identify the first display direction X1 based on the first attention area. For example, the first display direction X1 may be identified using a learning model constructed based on learning data prepared in advance using a machine learning technique (including deep learning).

[0033] (Step S4) The processing circuitry 34 uses the region identification function 34c to identify a second region of interest from the 3D medical image. Specifically, the processing circuitry 34 identifies a region indicating a second biological organ of interest in the acquired CT image as the second region of interest. In this example, coordinate information of pixels indicating the commissure between the right and left coronary cusps of the aortic valve included in the CT image is extracted using a known region extraction method, and the region of the pixels indicating the commissure is identified as the second region of interest. In another example, coordinate information of pixels indicating the commissure between the right and non-coronary cusps of the aortic valve included in the CT image and pixels indicating the commissure between the left and non-coronary cusps may be identified as the second region of interest. The known region extraction method is as described in step S2. In step S2, the first and second regions of interest may be identified in parallel, or the second region of interest may be identified by using the first region of interest as a related region of the second region of interest. Alternatively, the second region of interest may be identified using any other method.

[0034] (Step S5) The processing circuitry 34 uses the direction identification function 34d to identify a second display direction of the three-dimensional medical image based on the second region of interest. For example, the processing circuitry 34 identifies the second display direction based on the position of the commissure of the aortic valve represented by the three-dimensional medical image. Specifically, the processing circuitry 34 identifies the second display direction based on the position (coordinate information) of the commissure identified in step S4. That is, as shown in FIG. 9, the processing circuitry 34 identifies, from the medical image g5 showing the ascending aorta sr1a and the descending aorta sr1b, an angle where the CRA is 0° and the RAO where the commissure sr2 is located on the far right side is 35°, as the second display direction Y1. In another example, the processing circuitry 34 identifies, as the second display direction Y1, an angle where the CRA is 0° and the Y coordinate of the pixel indicating the commissure between the right coronary cusp and the non-coronary cusp is the same as that of the pixel indicating the commissure between the left coronary cusp and the non-coronary cusp. The second display direction Y1 is not limited to this, and any direction can be used as long as it is set based on the second attention area. Also, steps S2 and S3 may be executed after steps S4 and S5.

[0035] (Step S6) The processing circuitry 34 calculates a first feature value related to the shape of the aortic valve using the feature value calculation function 34e based on the first display orientation and the second display orientation. For example, the processing circuitry 34 calculates, as the first feature value, the angle at which the artificial valve is inserted into the patient so that the artificial valve is placed at an angle that matches the shape. Specifically, for example, the processing circuitry 34 calculates, as the angle at which the artificial valve is inserted, the amount of rotation of the artificial valve around the axis of the catheter holding the artificial valve. That is, the processing circuitry 34 determines the difference between the angle (2X) twice the angle indicated by the first display orientation X and the angle indicated by the second display orientation Y1 as the correction value for the rotation amount θ. This correction value is a correction value from the rotation angle of 90° currently used to align the commissures of the artificial valve with the commissures of the aortic valve. In other words, the rotation amount θ, which is the operating amount in commissure alignment (commissure positioning), which focuses on the commissure to adjust the placement position of the artificial valve, corresponds to θ = 90 - (2X - Y1). However, the rotation amount θ is not limited to the calculation formula θ = 90 - (2X - Y1). The rotation amount θ can be calculated using any formula based on the first display direction X and the second display direction Y1.

[0036] As described above, according to one embodiment, the processing circuitry 34 acquires a three-dimensional medical image containing image information about the patient's aortic valve as a three-dimensional image of a patient requiring artificial valve placement. The processing circuitry 34 identifies a first region of interest from the three-dimensional medical image. The processing circuitry 34 identifies a first display orientation X of the three-dimensional medical image based on the first region of interest. The processing circuitry 34 identifies a second region of interest from the three-dimensional medical image. The processing circuitry 34 identifies a second display orientation Y1 of the three-dimensional medical image based on the second region of interest. The processing circuitry 34 calculates a first feature amount related to the shape of the aortic valve as a first feature amount related to placement based on the first display orientation X and the second display orientation Y1. In this way, by calculating the first feature amount related to artificial valve placement from a three-dimensional medical image of a patient requiring artificial valve placement, the first feature amount for each patient can be used in surgery, thereby assisting surgery so that the artificial valve can be placed at an appropriate angle for each patient. According to one embodiment, the processing circuitry 34 may determine the first display direction X based on the overlapping position of the ascending aorta and the descending aorta shown in the three-dimensional medical image relative to the patient's front direction. The processing circuitry 34 may determine the second display direction Y1 based on the position of the commissure of the aortic valve shown in the three-dimensional medical image. In addition to the above-described effects, this configuration makes it possible to relatively easily determine the first display direction X and the second display direction Y1 by using positions that are easy to identify from the three-dimensional medical image.

[0037] Furthermore, according to one embodiment, the processing circuitry 34 may calculate, as the first feature, the angle at which the artificial valve is inserted into the patient. In this case, in addition to the effects described above, surgery can be more directly supported.

[0038] Furthermore, according to one embodiment, the processing circuitry 34 may calculate the amount of rotation of the artificial valve around the axis of the catheter holding the artificial valve as the angle at which the artificial valve is inserted. In this case, in addition to the effects described above, surgery can be supported more directly. For example, even if the amount of catheter manipulation and the amount of rotation of the artificial valve differ, the actual amount of rotation of the artificial valve can be used in surgery.

[0039] According to one embodiment, the insertion angle may be an angle dependent on the commissures of the patient's aortic valve. The amount of rotation may be a rotation angle for aligning the commissures of the prosthetic valve with the commissures of the aortic valve. In this case, in addition to the effects described above, surgery can be assisted to ensure that the commissures of the prosthetic valve and the aortic valve are aligned with each other for each patient.

[0040] (First Modification) In the embodiment, two display directions are identified from two regions of interest. However, this is not limiting, and two display directions may be identified from one region of interest. For example, the processing circuitry 34 may use the region identification function 34c to identify the first region of interest and the second region of interest as the same region and identify the first display direction and the second display direction from the same region. For example, the second region of interest may be the same region as the first region of interest. In this case, the first region of interest may include a first partial region and a second partial region that are different from each other. Accordingly, the processing circuitry 34 may use the direction identification function 34d to identify the first display direction X based on a first positional relationship between the first partial region and the second partial region. The first positional relationship may be, for example, a relationship in which the first partial region and the second partial region overlap most closely when the first region of interest is rotated about an arbitrary rotation center and the first region of interest is observed from a predetermined direction. Furthermore, the processing circuitry 34 may use the direction identification function 34d to identify a second display direction Y1 based on a second positional relationship between the first partial region and the second partial region. The second positional relationship may be, for example, a relationship in which, when the first region of interest is rotated about an arbitrary rotation center and the first region of interest is observed from a predetermined direction, parts of the contours of the first partial region and the second partial region are in contact with each other. Each of the first positional relationship and the second positional relationship is preferably a positional relationship when a three-dimensional medical image relative to the patient's front direction (CRA (or CAU) is 0°) is observed from a predetermined RAO (or LAO) direction, but is not limited to the patient's front direction.

[0041] FIG. 10 is a schematic diagram illustrating a first modified example of an embodiment. In FIG. 10, a medical image g11 used to determine the first display orientation X and a medical image g12 used to determine the second display orientation Y1 are displayed on the display 32. That is, the processing circuitry 34 causes the display control function 34b to display a first display region that displays the medical image g11 based on the first display orientation X and a second display region that displays the medical image g12 based on the second display orientation Y1 side by side on the display 32. The medical images g11 and g12 are three-dimensional medical images. The first partial region is the ascending aorta sr1a. The second partial region is the descending aorta sr1b.

[0042] Medical image g11 represents an example of a first positional relationship in which the ascending aorta sr1a and the descending aorta sr1b overlap when the first region of interest is rotated around the center of gravity of the aortic valve and the aorta is observed from the front to the back of the display 32. In this example, medical image g11 represents a first positional relationship in which the feature point fp, at which the tangent to the core line L1 at each point in the ascending aorta sr1a is closest to being perpendicular to the XY plane, overlaps with the core line L1 in the descending aorta sr1b. However, the first positional relationship is not limited to this, and may be, for example, a position at which the amount of overlap between the area indicated by the ascending aorta sr1a and the area indicated by the descending aorta sr1b is maximized in the observation direction. In one embodiment, the processing circuitry 34 uses the direction identification function 34d to identify an angle where CRA is 0° and RAO is 33.6° as the first display direction X based on the first positional relationship between the ascending aorta sr1a and the descending aorta sr1b as shown in the medical image g11.

[0043] Furthermore, medical image g12 represents an example of a second positional relationship in which, when the first region of interest is rotated about the center of gravity of the aortic valve and the aorta is observed from the front to the back of the display 32, portions of the contours of the ascending aorta sr1a and the descending aorta sr1b are in contact with each other. The second positional relationship may also be referred to as a relationship in which the ascending aorta sr1a and the descending aorta sr1b are closest to each other but do not overlap. In any case, the processing circuitry 34, using the direction identification function 34d, identifies the angle of CRA=0° and RAO=53.3° as the second display direction Y1 based on the second positional relationship between the ascending aorta sr1a and the descending aorta sr1b as shown in medical image g12.

[0044] The processing circuit 34 calculates the first feature amount based on a calculation formula different from the above in accordance with such a modification of the first display orientation X and the second display orientation Y1. As the calculation formula for the first feature amount, a formula that matches the modified first display orientation X and second display orientation Y1 is set in advance in the feature amount calculation function 34e.

[0045] According to the first modified example described above, the first region of interest (aorta sr1) includes a first partial region (ascending aorta sr1a) and a second partial region (descending aorta sr1b) that are different from each other. The second region of interest is the same region as the first region of interest. The processing circuit 34 determines a first display direction X based on a first positional relationship between the ascending aorta sr1a and the descending aorta sr1b. The processing circuit 34 determines a second display direction Y1 based on a second positional relationship between the ascending aorta sr1a and the descending aorta sr1b. This not only achieves the effect of the first embodiment, but also reduces the processing required to determine the second region of interest. Note that the first modified example is not limited to the example shown in FIG. 10 , and any configuration that determines two display directions from one region of interest can be used as appropriate.

[0046] Furthermore, according to the first modification, the first positional relationship is a relationship in which the ascending aorta sr1a and the descending aorta sr1b are at a position where they overlap most in a predetermined observation direction. The second positional relationship is a relationship in which the contours of the ascending aorta sr1a and the descending aorta sr1b are at a position where they partially contact each other in a predetermined observation direction. Therefore, in addition to the above-mentioned effects, two display directions can be specified using two easily visible positional relationships.

[0047] (Second Modification) In the embodiment and the first modified example, one feature (first feature) is calculated from the same or different regions of interest. However, this is not limiting. Two or more feature (first feature, second feature, ...) may be calculated from three or more regions of interest. For example, the processing circuitry 34 may use the region identification function 34c to identify a third region of interest different from the first and second regions of interest from the three-dimensional medical image. Furthermore, for example, the processing circuitry 34 may use the direction identification function 34d to identify a third display orientation of the three-dimensional medical image based on the third region of interest. Accordingly, the processing circuitry 34 may use the feature calculation function 34e to further calculate a second feature related to the placement of the artificial valve based on the first, second, and third display orientations. In this case, the processing circuitry 34 may use the feature calculation function 34e to calculate the third feature related to the placement based on a predetermined calculation formula using the first feature and the second feature as variables. For example, the processing circuitry 34 may calculate a third feature amount related to placement of the artificial valve based on a predetermined calculation formula using the first feature amount and the second feature amount as variables. The region specifying function 34c is an example of a third region specifying unit. The direction specifying function 34d is an example of a third direction specifying unit.

[0048] 11 is a schematic diagram illustrating a second modified example of an embodiment. In FIG. 11, a medical image g21 used to determine the first display direction X, a medical image g22 used to determine the second display direction Y1, and a medical image g23 used to determine the third display direction Z are displayed on the display 32. The medical images g21 to g23 are three-dimensional medical images. The first region of interest is the aorta sr1, which includes the ascending aorta sr1a and the descending aorta sr1b. The second region of interest is the commissure sr2 between the right and left coronary cusps of the aortic valve. The third region of interest is the coronary artery ostium sr3 located between the ascending aorta sr1a and the aortic valve.

[0049] In the medical image g21, similarly to the first modified example, an angle of 0° CRA and 33.6° RAO is specified as the first display direction X based on the first positional relationship in which the ascending aorta sr1a and the descending aorta sr1b overlap.

[0050] In the medical image g22, as in step S5 of one embodiment, the angle of CRA is 0° and RAO is 56.2° when the commissure sr2 is located at the rightmost side is specified as the second display direction Y1. Note that the medical image g22 of the second modified example and the medical image g5 of step S5 of one embodiment are medical images of different patients, so even when the commissure sr2 is located at the rightmost side, the RAO values ​​are different from each other.

[0051] In the medical image g23, the angle of the CRA is 0° and the angle of the RAO where the coronary artery ostium sr3 is located on the far right side is 116.7°, which is specified as the third display direction Z.

[0052] As shown in image g24 of FIG. 12 , in addition to the above-described commissure alignment, the processing circuitry 34 performs coronary alignment (coronary alignment), which adjusts the placement position of the artificial valve by focusing on the coronary artery ostium. This further calculates a second feature value related to the shape of the aortic valve based on the first display direction X, the second display direction Y1, and the third display direction Z. For example, the processing circuitry 34 calculates the difference obtained by subtracting the angle indicated by the second display direction Y1 from the angle (2X) twice the angle indicated by the first display direction X, and then adding the angle indicated by the third display direction Z to the difference, subtracting 60°, and sets this difference as the correction value for the rotation amount φ as the second feature value. Note that the value 60° used here comes from the fact that the commissures of the aortic valve are offset by approximately 120°. However, the calculation formula may be modified so that the angle formed by the commissures in each patient is calculated and used. This correction amount is a correction amount from 90°, which is the rotation angle currently used to place an artificial valve so as not to block the coronary artery (coronary artery ostium) branching from the aorta. In other words, the rotation amount φ, which is the manipulation amount in coronary alignment that adjusts the placement position of the artificial valve by focusing on the coronary artery ostium, corresponds to φ = 90 - (2X - Y1 + Z - 60). Note that this rotation amount φ may also be expressed as θ + Z - 60, using the rotation amount θ, which is the manipulation amount in commissar alignment. In this case, the rotation amount φ in coronary alignment is an example of an angle obtained by correcting the rotation amount θ in commissar alignment. Additionally, the rotation amount θ in commissar alignment is an example of a first feature amount, which is the angle when the artificial valve is inserted into the patient. The rotation amount φ in coronary alignment is an example of a second feature amount, which is the angle obtained by correcting the first feature amount so that the artificial valve is placed in a state that does not block the coronary artery ostium. Note that the rotation amount φ is not limited to the above calculation formula. The rotation amount φ can be calculated using any calculation formula based on the first display direction X, the second display direction Y1, and the third display direction Z.

[0053] Alternatively, the processing circuitry 34 may calculate the third feature quantity (Ω) related to the shape of the aortic valve based on a predetermined calculation formula using the first feature quantity (θ) and the second feature quantity (φ) as variables. As the predetermined calculation formula, for example, a formula for adding the first feature quantity (θ) and the second feature quantity (φ) (Ω=θ+φ), a formula for subtracting them (Ω=θ-φ), or a formula for averaging them (Ω=(θ+φ) / 2) can be used as appropriate.

[0054] According to the second modification described above, the processing circuitry 34 identifies a third region of interest (coronary artery ostium sr3) different from the first region of interest (aorta sr1) and the second region of interest (commissure sr2) from the three-dimensional medical image. The processing circuitry 34 identifies a third display direction Z of the three-dimensional medical image based on the third region of interest. The processing circuitry 34 further calculates a second feature amount based on the first display direction X, the second display direction Y1, and the third display direction Z. This provides the effect of the first embodiment, as well as the ability to assist surgery by calculating a second feature amount other than the first feature amount. Furthermore, according to a second modification, the first region of interest is the aorta, including the patient's ascending aorta and descending aorta. The second region of interest is the commissure of the patient's aortic valve. The third region of interest is the coronary artery ostium between the ascending aorta and the aortic valve. The first feature value is the angle at which the artificial valve is inserted into the patient. The second feature value is an angle obtained by correcting the first feature value so that the artificial valve is placed in a state that does not block the coronary artery ostium. In this case, in addition to the effect of the first embodiment, the configuration for calculating the angle obtained by correcting the first feature value so that the artificial valve is placed in a state that does not block the coronary artery ostium can provide thorough support for surgery.

[0055] According to a second modification, the processing circuitry 34 may calculate a third feature value related to the shape of the aortic valve based on a predetermined calculation formula using the first feature value and the second feature value as variables. In this case, in addition to the above-described effects, surgery can be assisted by calculating a third feature value other than the first feature value and the second feature value. Note that the second modification is not limited to the examples shown in FIGS. 11 and 12 , and any configuration that calculates two or more feature values ​​from three or more regions of interest can be used as appropriate.

[0056] (Third Modification) In one embodiment, the determination of the display orientation based on the feature of the core line L1 in step S3, the determination of the display orientation based on the position of the commissure sr2 in step S5, and the calculation of the first feature in step S6 are performed based on predetermined specific methods. However, the scope of the present invention is not limited to this. For example, the processing circuitry 34 may classify the shape of the anatomical structure in the first or second region of interest, select a calculation formula to apply to feature calculation based on the classification result from among a plurality of predetermined calculation formulas capable of calculating the first feature, and calculate the first feature based on the selected calculation formula. That is, the processing circuitry 34 may change the method for determining the display orientation in step S3 or step S5 or the calculation formula to apply in step S6 based on the shape of the first or second region of interest. For example, the processing circuitry 34 may perform a determination process to classify the shape of the region of interest before steps S3 and S5. The determination process calculates shape features of the anatomical structure of the first or second region of interest, compares the shape with predetermined criteria, and determines the feature to be calculated, the position of the region of interest, or the calculation formula to be applied based on the classification result. Specifically, for example, if the target anatomical structure is an aortic valve, the aortic valve may be extracted as the first or second region of interest, and the number of aortic valve leaflets may be calculated as a shape feature, and determination may be made based on the number of leaflets. The normal number of aortic valve leaflets is three, but depending on the disease, the number may be two (bicuspid aortic valve) or four (tetracuspid aortic valve). In this case, it is preferable to change the calculation formula applied in step 6 depending on the number of leaflets. The shape feature may be calculated solely from the shape of the region of interest, such as the size of the valve annulus, or a relative feature, such as the distance from an arbitrary third region of interest different from the region of interest. Alternatively, the shape feature of an arbitrary third region of interest different from the region of interest may be used. For example, when the region of interest is the aortic valve, the calculation formula may be changed depending on the curvature of the ascending aorta as the third region of interest (substantially straight, substantially bent, or substantially arcuate).

[0057] FIG. 13 is a schematic diagram illustrating a third modification of an embodiment. In FIG. 13, a medical image g31 showing the ascending aorta sr1a and its center line L1 of one patient and a medical image g32 showing the ascending aorta sr1a-2 and its center line L2 of another patient are displayed on a display 32. The medical images g31 and g32 also show the descending aorta sr1b and sr1b-2, which are partially hidden by the ascending aorta sr1a and sr1a-2. In the medical images g31 and g32 of different patients, the curvatures may be calculated at each position of the center line L1 of the ascending aorta sr1a and each position of the center line L2 of the ascending aorta sr1a-2, and the calculated curvatures may be used as feature quantities. In this case, the calculation formula to be applied may be determined depending on the calculated curvatures. For example, as shown in schematic diagram g33, a curvature range and a calculation formula for the feature may be set in advance, and the calculation formula may be determined depending on the curvature at points p1 and p2 on the core lines L1 and L2. In this example, when the curvature at point p1 on the core line L1 is large, the calculation formula to be applied is determined to be formula A. When the curvature at point p2 on the core line L2 is small, the calculation formula to be applied is determined to be formula B. Note that when the curvature is 0, the calculation formula to be applied is determined to be formula C. The calculation formula may also be changed depending on the amount of calcification in the aortic valve. That is, the processing circuitry 34 calculates feature values ​​related to the anatomical structure of each patient, determines the feature values ​​based on predetermined criteria, and changes the calculation formula to be applied for each patient depending on the result of the determination.

[0058] According to the third modification as described above, the processing circuitry 34 classifies the shape of the anatomical structure in the first region of interest or the second region of interest, selects a calculation formula corresponding to the classification result from among a plurality of calculation formulas capable of calculating the first feature amount, and calculates the first feature amount based on the selected calculation formula. This makes it possible to calculate the first feature amount based on a calculation formula corresponding to the classification result of the shape of the anatomical structure for each patient, in addition to the effect of the first embodiment.

[0059] (Fourth Modification) Although the first embodiment did not provide a detailed description of the display content, the fourth modification will provide a detailed description of a specific example of the display content of the display 32. Accordingly, the processing circuitry 34 uses the display control function 34b to update the first display orientation X or the second display orientation Y1 in response to a user operation, and updates the display content of the display 32 based on the updated result. Furthermore, the processing circuitry 34 uses the feature calculation function 34e to update the first feature based on the updated result. That is, the processing circuitry 34 updates (recalculates) the first feature related to the shape of the aortic valve based on the updated first display orientation X and second display orientation Y1.

[0060] Fig. 14 is a schematic diagram for explaining a fourth modified example of an embodiment, Fig. 15 and Fig. 16 are schematic diagrams showing an example of a menu, Fig. 17 is a schematic diagram showing an example of a split screen, Fig. 18 is a schematic diagram showing an example of a position adjustment display, Fig. 19 is a schematic diagram showing an example of a setting screen in the menu.

[0061] As shown in FIG. 14, the display screen 200 of the display 32 includes, for example, a thumbnail area 201, a medical image area 202, a menu bar area 203, a divided image area 204, and a feature amount calculation area 205.

[0062] Here, the thumbnail area 201 is a display area for thumbnail images, and displays a list of images acquired from an in-hospital image database or examination device based on a user's instruction. The images displayed in the list are images that satisfy conditions specified by the user using an input interface 31 (not shown) for specifying image conditions. The conditions can specify information about the patient, such as the name, ID, date of birth, and weight of the patient who is the subject of the image, as well as information about the image, such as the type of modality of the image, the name of the imaging device, the imaging date, the imaging conditions, and the reconstruction conditions. These conditions can be acquired from the image's DICOM (digital imaging and communication in medicine) header, a PACS (picture archiving and communication system), an electronic medical record, a RIS (radiology information system), an HIS (hospital information system), etc. In the following, an example is described in which one CT image of a specific patient is specified. However, images of multiple patients or images of different types of modalities (e.g., CT images and ultrasound images) may also be specified. Icons representing images that satisfy the specified conditions are displayed in the thumbnail area 201. The icon may be a reduced image obtained by reducing a two-dimensional image of a representative cross section of the specified image to fit the size of the thumbnail area 201, or may simply be a character string or symbol indicating the specified image. Alternatively, the icon may be any of various shapes, images, or schematic images prepared in advance and stored in a storage device. Basic information about the specified image (such as the imaging date, number of slices, and reconstruction function) may be displayed alongside the icon in the thumbnail area 201. The basic information to be displayed may be predetermined or specified by the user. This basic information can be obtained from the DICOM header of the image, a PACS, an electronic medical record, a RIS, an HIS, or the like. The user can select an icon displayed in the thumbnail area 201 and drag and drop it into the medical image area 202 or the segmented image area 204 to display the image corresponding to the icon in the medical image area 202 or the segmented image area 204.If a medical image is already displayed in the medical image area 202 when a drag-and-drop is performed, a warning (not shown) may be displayed to the user, the medical image being displayed may be removed from the medical image area 202, and the medical image corresponding to the drag-and-dropped icon may be displayed. Note that the drag-and-drop may be called a move operation or a copy operation depending on the content.

[0063] The divided image area 204 is a display area for various images. The divided image area 204 is divided into display areas 204a to 204d of any number or size, and each of the display areas 204a to 204d displays a cross-sectional image at a predetermined position in the acquired medical image or a processed image calculated by applying image processing to the medical image. Note that the display areas 204a to 204d are merely examples, and the number and size of the display areas are arbitrary. The display areas 204a to 204d display images under different related display conditions, and the types and display conditions of the images are predetermined. The display conditions include image allocation, such as which image is displayed in which of the display areas 204a to 204d, the cross-sectional position when displaying the cross-sectional image, the magnification rate, the window level, the window width, etc.

[0064] Furthermore, the images displayed in the medical image region 202, the divided image region 204, or their display regions 202a-202b, 204a-204d can be displayed by changing the observation cross section, slice advance (browsing), magnification ratio, center position (translation), window level, window width, etc. based on user instructions. The sizes of the display regions 202a-202b, 204a-204d can also be changed. For example, the sizes of the display regions can be changed by operations such as dragging and dropping on the frames of the display regions 202a-202b, 204a-204d, or by specific operations on the display regions 202a-202b, 204a-204d. Examples of specific operations that can be used include double-clicking and clicking while holding down the Ctrl key, as appropriate. Additionally, information set in advance or designated by the user is displayed superimposed on a specific position on the display areas 202a-202b and 204a-204d (for example, at the edge 204e of the display area 204b). This information can specify information about the patient, who is the subject of the image, such as the name, ID, date of birth, and weight, as well as information about the image, such as the type of modality of the image, the name of the imaging device, the imaging date, the imaging conditions, and the reconstruction conditions. This information can be acquired from the DICOM header of the image, PACS, electronic medical record, RIS, HIS, etc.

[0065] 14 to 16, the menu bar area 203 displays a menu bar including buttons 203a to 203b, 203q to 203t, icons 203c to 203n, and check boxes 203o to 203p corresponding to various functions. The user selects an icon by operating the input interface 31 such as a mouse, and activates the function of the button, icon, or check box. The processing circuit 34 uses the display control function 34b to update the display content of the display 32 in response to the user's operation of the menu bar area 203.

[0066] The button 203a is a GUI for switching between displaying and hiding the thumbnail area 201. In response to an operation of the button 203a, the processing circuitry 34 hides the thumbnail area 201 and enlarges at least one of the medical image area 202, the divided image area 204, and the feature amount calculation area 205 to match the size of the hidden thumbnail area 201.

[0067] Button 203b is a GUI for changing the number of divisions in the medical image region 202 or the divided image region 204. In FIGS. 14 and 17, the divided image region 204 is set to four divided display regions 204a-204d with two rows and two columns, but the number of rows or columns of the divided regions may be changeable. Also, the sizes of the display regions 202a-202b and 204a-204d may be changeable. Several patterns of the number of divisions and sizes of the display regions 202a-202b and 204a-204d may be registered in advance as presets, and the display regions 202a-202b and 204a-204d may be set to a pattern corresponding to the specified set. Also, the user may be able to register new presets.

[0068] Icons 203c to 203g are a group of icons for functions to which mouse operation systems are assigned. For example, by selecting each icon, the operation systems of left clicking and dragging of the mouse are controlled to be assigned to the operation system corresponding to the selected icon.

[0069] Specifically, the icon 203c allows a browse operation system for displaying images continuously in the slice direction to be assigned to the operation system of left clicking and dragging the mouse.

[0070] Icon 203d allows the operation of changing the image gradation (WL or WW in CT) to be assigned to the operation of left-clicking and dragging the mouse.

[0071] Icon 203e allows the operation system for moving an image in parallel to be assigned to the operation system of left clicking and dragging a mouse.

[0072] Icon 203f allows the operation system for changing the magnification ratio of an image to be assigned to the operation system of left clicking and dragging of a mouse.

[0073] Icon 203g can be assigned an operation system for rotating an image to a left click and drag of a mouse. However, each of icons 203c to 203g may be assigned an operation system for right click and drag, an operation system for mouse wheel click and drag, or an operation system for simultaneous right and left click and drag.

[0074] Furthermore, the icons 203c to 203g may be configured to set the speed or amount of slice advancement in the browse function relative to the amount of mouse movement, the amount of change in magnification, the amount of translation, the amount of change in gradation, and the amount of rotation.

[0075] Furthermore, the allocation may be changed according to the mouse operation when selecting the icons 203c to 203g. For example, when the icon is selected by left-clicking, the operation system corresponding to the icon may be controlled to be assigned to the left-click operation system. When the icon is selected by right-clicking, the operation system corresponding to the icon may be controlled to be assigned to the right-click operation system. When the icon is selected by simultaneous right- and left-clicking, the operation system corresponding to the icon may be controlled to be assigned to the simultaneous right- and left-click operation system. When the icon is selected by mouse-wheel clicking, the operation system corresponding to the icon may be controlled to be assigned to the mouse-wheel-click operation system.

[0076] The icons 203h to 203n are a group of icons for drawing and measuring functions of various figures, and control may be exercised so that the drawing and measuring functions of various figures can be executed by selecting any of the icons 203h to 203n.

[0077] Specifically, icon 203h is a GUI for drawing a straight line on an image and measuring and displaying the length of the line. The display format, such as the position, color, thickness, and font of the measurement value, of the line can be adjusted by user operation.

[0078] Icon 203i is a GUI for drawing two lines on an image and measuring and displaying the acute angle between the two lines. The user can adjust the display format, such as the position, color, thickness, and font of the measurement values, of the start and end points of the two lines.

[0079] Icon 203j is a GUI for drawing an ellipse on an image and calculating and displaying the perimeter, internal area, and statistics of the pixel values ​​inside the ellipse (average, maximum, minimum, etc.). The display format of the ellipse, such as the center position, major axis, minor axis, color, thickness, and font of the measurement values, can be adjusted by user operation.

[0080] Icon 203k is a GUI for drawing an arrow icon on an image. The display form of the arrow icon, such as the start and end positions, color, thickness, and tip shape, can be adjusted by user operation.

[0081] Icon 203l displays any character string on the image. The displayed character string is specified by the user using a keyboard or the like through a GUI. The display format, such as the position where the character string is displayed, the font of the character string, and the background color, can be adjusted by user operation.

[0082] Icon 203m is a GUI for drawing a closed curve of any shape on an image and calculating and displaying the perimeter of the closed curve, the area inside the closed curve, and statistics of the pixel values ​​inside the closed curve (average, maximum, minimum, etc.). The display format of the closed curve, such as the center position, color, thickness, and font of the measurement values, can be adjusted by user operation. The closed curve may be set to a predetermined shape (circle, ellipse, rectangle, square, triangle, etc.), and the length of each side of the shape, the angle between two sides, the diameter, major axis, minor axis, etc. may be adjustable, or a freeform shape may be drawn.

[0083] Icon 203n is a GUI for drawing an open curve of any shape on an image and calculating and displaying the perimeter of the open curve. The display format, such as the center position, color, thickness, and font of the measurement value of the open curve, can be adjusted by user operation. In addition, it is also possible to set a three-dimensional shape (sphere, oval sphere, rectangular parallelepiped, triangular pyramid, etc.), and control the calculation and display of its surface area, volume, etc.

[0084] Meanwhile, the check box 203o is a GUI for switching between displaying and hiding reference lines indicating cross-sectional positions of images displayed in other divided areas on the image, and this can be switched by selecting the check box 203o. Note that in FIG. 17, the user can change the display format of the images displayed in the display areas 204c and 204d corresponding to the reference lines L41 and L42 by performing operations on the reference lines L41 and L42. That is, the processing circuitry 34 uses the display control function 34b to update the display content on the display 32 in response to the user's operation on the reference lines L41 and L42. Similarly, the processing circuitry 34 uses the display control function 34b to update the display content on the display 32 in response to the user's operation on the intersection 301 of the reference lines L41 and L42, the circular mark 302, or the triangular mark 303.

[0085] For example, the processing circuitry 34 allows the user to move the intersection 301 by selecting or dragging and dropping the intersection 301 or its periphery of the reference lines L41 and L42. When the intersection 301 is moved, the cross-sectional positions indicated by the reference lines L41 and L42 also change, and the cross-sectional images displayed in the display areas 204c and 204d also change accordingly. Furthermore, by translating the reference lines L41 and L42 by dragging and dropping, the cross-sectional images in the display areas 204c and 204d can be changed in accordance with the change in the cross-sectional positions indicated by the reference lines L41 and L42. Furthermore, by rotating the reference lines L41 and L42 around the intersection 301 by dragging and dropping, the cross-sectional images in the display areas 204c and 204d can be changed in accordance with the change in the cross-sectional positions indicated by the reference lines L41 and L42. When rotating the reference lines L41 and L42, the other reference lines may be controlled to move in conjunction with an operation on one reference line so that the angle between the reference lines L41 and L42 is always kept constant. The angle may be, for example, 180 degrees divided equally based on the number of reference lines. In this case, the angle is 90 degrees (perpendicular intersection) for two reference lines, and 60 degrees for three reference lines. Alternatively, when rotating the reference lines L41 and L42, each reference line may be moved independently. Furthermore, the translation and rotation operations may be switched between rotation by operating the circular mark 302 on the reference lines L41 and L42 in the display areas 204b to 204d and translation by operating the reference lines L41 or L42 outside the circular mark 302. Alternatively, the translation and rotation operations of the reference lines L41 and L42 may be switched using a keyboard. In the case of a keyboard, for example, it may be possible to switch between rotation by pressing the Ctrl key and translation by releasing the Ctrl key (non-contact).Alternatively, a width may be specified using the reference lines L41 and L42, and maximum intensity projection images or minimum intensity projection images may be created within a range based on the width, and the projection images may be displayed in the display regions 204c and 204d corresponding to the reference lines L41 and L42. The width may be specified by moving one of two triangular marks 303, one of which has one vertex overlapping on the reference lines L41 and L42, away from the reference lines L41 and L42. In this case, a Mip image can be created for the cross-sectional image indicated by the range between the vertices of the two triangular marks 303 on the reference line L41 and L42 side. The display format (color, thickness, line type (solid line, dotted line, dotted line fineness, etc.)) of the reference lines L41 and L42 may be preset or specified by the user.

[0086] Check box 203p is a GUI for switching whether to display an arbitrary two-dimensional cross-sectional image in alignment with an image showing a three-dimensional region, such as a rendering image such as a VR image or an SR (Surface Rendering) image. For example, as shown in FIG. 18, a cross-sectional image in display region 204b is displayed in display region 202a, with its three-dimensional position corresponding to that of the VR image of the aorta displayed in display region 202b. When superimposing, VR images positioned in front of the cross-section relative to the observation direction are displayed, while VR images positioned behind the cross-section are not displayed. The cross-sectional images to be superimposed may be specified by the user as display regions or divided regions. For example, they may be specified using a context menu displayed by right-clicking on each display region. Alternatively, a display region (e.g., display region 204b) that will be the cross-section to be superimposed may be determined in advance, and the cross-section displayed in that display region may be superimposed. Here, when superimposing, the cross-sectional position and size are adjusted to the position and magnification of the VR image, but other display conditions (such as window conditions) may be matched to the display conditions displayed in the divided regions, or these conditions may be fixed as predetermined conditions. In addition, when the check box 203p is set, if a user operation is performed in the display area where the superimposed cross-sectional image is displayed, the display conditions of the cross-sectional image are changed accordingly. For example, if the display conditions are changed (slice advance, etc.) by a user operation, the display conditions of the cross-sectional image superimposed on the VR image are also changed accordingly.

[0087] The buttons 203q to 203s are GUIs for returning the state of the medical image region 202 and the divided image region 204 to a specific state by selection. Examples of specific states include the state when the application is launched (initial state), the state when an image is first displayed in the display region, or the state before a specific number of operations (for example, an image of a back button (undo button), a forward button (redo button), or a reset button). In FIG. 16, the button 203q is a back button, and the button 203r is a forward button. The button 203s is a reset button. In either case, the buttons 203q to 203s are implemented by recording the display conditions and display forms in a specific state and restoring the recorded contents. Alternatively, the display conditions and forms for a certain period may be recorded, and the display state after the previous or next operation may be continuously restored by operating the buttons 203q and 203r.

[0088] The button 203t is a GUI for displaying a setting screen for setting display conditions for an image showing a three-dimensional area, such as a rendering image such as a VR image or an SR image, or an area to be superimposed on a two-dimensional image. FIG. 19 shows an example of the setting screen g40. The processing circuit 34 uses the display control function 34b to update the display content of the display 32 in response to a user's operation on the setting screen g40. Note that the position information of the various areas is specified in step S2. The setting screen g40 will now be described.

[0089] The combo box for specifying "Name" sets the display area for which you want to change the display conditions. "Priority" indicates that the area specified higher on the screen has a higher display priority, and if multiple areas correspond to the same coordinates on the image, the area with the highest priority will be displayed.

[0090] The "Color" column displays a sample of the color to be assigned when the area in the corresponding column (specified from the "Name" combo box) is superimposed on the VR image or cross-sectional image. As shown in Figure 19, the user can change the color to any color by selecting the area that shows the sample color. The color can be changed by specifying the RBG value or by specifying it from the color map.

[0091] In the "Transparency" column, you can set the transparency of the VR image or cross-sectional image of the corresponding column area (specified from the "Name" combo box) to be superimposed. The transparency can be specified in 1% increments from 0 to 99% using the slider bar, and at 0%, the image will be superimposed without any transparency (i.e. the background image will not be visible).

[0092] Although not shown, display conditions such as saturation and brightness may be set, and texture may be set instead of color. Also, all transparency levels may be set together by selecting a "linked" checkbox (not shown). When linking, all transparency levels may be controlled to be set to the same value, or the transparency level may be increased or decreased overall while maintaining the relationship between the transparency levels corresponding to each region when the checkbox is selected.

[0093] The checkboxes in the "VR" column are a GUI for specifying the area to be displayed on the VR image.

[0094] The check boxes in the "MPR" column are a GUI for specifying the area to be displayed on an MPR (Multi Planar Reconstruction) image.

[0095] Although not shown, a button may be set that allows checking or unchecking all checkboxes at the same time. Furthermore, the combo box for the "Name" column can specify the area to be displayed based on the set priority and display conditions. Control may be exercised so that the same area cannot be set for multiple combo boxes. For example, if an area that has already been set for another combo box is specified for a certain combo box, control may be exercised so that it cannot be specified, or the setting for the existing combo box may be canceled. Alternatively, if the same area is set, control may be exercised so that the setting with the highest priority is used first.

[0096] The close button bt1 is a GUI that hides the setting screen g40.

[0097] The reset button bt2 is a GUI for resetting the settings on the setting screen g40 to the initial state. The display conditions set on the setting screen g40 may be reflected in the display area immediately after each setting, or in response to the operation of the close button bt1.

[0098] The display areas 202a and 202b are areas that display, for example, a first attention area and a second attention area. The display areas 202a and 202b are used to specify a first display direction X and a second display direction Y1 of the attention area in steps S3 and S5. The display directions of the VR images displayed in the display areas 202a and 202b are used as the first display direction X and the second display direction Y1 specified in steps S3 and S5. The initial display directions of the display areas 202a and 202b may have RAO and CRA set to 0, or may be display directions calculated by a predetermined method. The first display direction X and the second display direction Y1 corresponding to the display direction of the VR image are displayed in the area 202c.

[0099] The display areas 202a and 202b each include an area 202c and a button 202d, and the display area 202b further includes a button 202e.

[0100] Here, the display of the area 202c is controlled so as to be changed in conjunction with the user changing the display direction of the VR image in the area 202c.

[0101] Button 202d is a GUI for fixing an arbitrary rotation axis when changing the display direction of a VR image, and each of the X-axis, Y-axis, and Z-axis can be fixed arbitrarily. The function of button 202d allows the VR image to be rotated only in a specific direction. For example, button 202d is used when it is desired to fix the CRA and change only the rotation angle of the RAO. In other words, operating button 202d controls the display direction to change only when the mouse is moved in a specified direction, and does not change the display direction when moved in any other direction. The fixed rotation axis may be an axis based on the patient coordinate system or an axis based on the display direction on the screen.

[0102] Button 202e is used when multiple feature quantities are calculated in step S6 based on multiple directions, as in Modification Example 2. In FIG. 14, when button 202e is operated while it is labeled "Commissar," commissar alignment is selected in feature quantity calculation area 205. After button 202e is selected, button 202e may be changed to labeled "Coronary." Alternatively, it may be labeled "Alignment." In either case, when button 202e is selected, the same or different attention areas are displayed in display area 202b, and the display direction is set according to the user's operation. Note that multiple display directions may be set by selecting button 202e, or a display direction may be recorded once, and the display direction of display area 202b may be switched according to the recorded display direction. Note that a button equivalent to button 202e may also be provided in display area 202a, allowing multiple types of first display directions X to be set.

[0103] The feature amount calculated in step S6 is displayed in the feature amount calculation area 205. The feature amount calculation area 205 includes a first feature amount area 205a, a second feature amount area 205b, and a direction display area 205c.

[0104] The first feature amount area 205a displays the calculation formula for the first feature amount calculated in step S6, the first display direction X (angle) identified in step S3, and the second display direction Y1 (angle) identified in step S5. When the user performs an operation on the display area 202a or the display area 202b and the display direction X or the display direction Y1 in the display area 202a or the display area 202b is updated, the display in the first feature amount area 205a is also updated based on the updated display direction. The user can update the value of the display direction X or Y1 displayed in the first feature amount area 205a to an arbitrary value using an input interface, and when updated, the display in the display area 202a or the display area 202b may also be controlled to change to a display form corresponding to the display direction corresponding to the updated value.

[0105] The second feature amount area 205b displays the calculation formula for the second feature amount calculated in the second modification, the first display direction X (angle) identified in step S3, the second display direction Y1 (angle) identified in step S5, and the third display direction Z identified in the second modification. When a user operates the display area 202a or the display area 202b and the display direction X or the display direction Y1 in the display area 202a or the display area 202b is updated, the display in the second feature amount area 205b is also updated based on the updated display direction. The user may update the value of the display direction X or Y1 displayed in the second feature amount area 205b to any value using an input interface, and when updated, the display in the display area 202a or the display area 202b may be controlled to change to a display mode corresponding to the display direction corresponding to the updated value.

[0106] The first feature amount region 205a and the second feature amount region 205b are used when there are two types of feature amounts to be calculated, as in the second modified example. That is, when only one type of feature amount is calculated, the second feature amount region 205b is unnecessary. When three or more types of feature amounts are calculated, the calculation formula and display direction (angle) corresponding to each feature amount may be displayed. For example, the first feature amount region 205a and the second feature amount region 205b may be displayed side by side, or the display may be switched based on the display direction in the display areas 202a and 202b. For example, when a medical image g23 including the third display direction Z is displayed in the display area 202a or 202b as shown in FIG. 11, the first feature amount region 205a not using the third display direction Z may not be displayed, and the second feature amount region 205b using the third display direction Z may be displayed.

[0107] The direction display area 205c is an area for converting the direction into a clock shape and displaying it when the calculated feature value indicates a direction. That is, the direction is converted into a clock shape and displayed by converting one minute of the clock into 6 degrees of direction (angle). That is, a 90-degree angle is 15 minutes of the clock and indicates the 3 o'clock direction. In FIG. 14, the solid line in the direction display area 205c is displayed at the 0.5 o'clock position corresponding to the feature value of the first feature value area 205a. The dashed-dotted line in the direction display area 205c is displayed at the 0.3 o'clock position corresponding to the feature value of the second feature value area 205b.

[0108] According to the fourth modification as described above, the processing circuitry 34 updates the first display orientation X or the second display orientation Y1 in response to a user operation, and updates the display content on the display 32 based on the updated result. The processing circuitry 34 updates the first feature amount based on the updated result. This not only achieves the effect of the first embodiment, but also makes it possible to calculate the first feature amount from the first display orientation X and the second display orientation Y1 in response to a user operation.

[0109] According to at least one of the embodiments and modifications described above, surgery can be assisted so that an artificial valve can be placed at an angle appropriate for each patient.

[0110] The term "processor" used in the above description refers to a circuit such as a CPU, a GPU, an application specific integrated circuit (ASIC), a programmable logic device (e.g., a simple programmable logic device (SPLD), a complex programmable logic device (CPLD), and a field programmable gate array (FPGA)). If the processor is a CPU, for example, the processor realizes its function by reading and executing a program stored in a memory. On the other hand, if the processor is an ASIC, for example, instead of storing a program in a memory, the function is directly incorporated into the circuit of the processor as a logic circuit. Note that each processor in the embodiments is not limited to being configured as a single circuit for each processor, but may be configured as a single processor by combining multiple independent circuits to realize its function. Furthermore, the multiple components in FIG. 1 may be integrated into a single processor to realize its function.

[0111] Although several embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included within the scope and spirit of the invention, as well as within the scope of the invention described in the claims and their equivalents. [Explanation of symbols]

[0112] 10 Medical imaging diagnostic equipment 20 Image Archive 30 Medical image processing device 31 Input Interface 32 Display 33 Memory 34 Processing circuit 34a Acquisition Function 34b Display control function 34c Area identification function 34d direction identification function 34e Feature calculation function 100 Medical Image Processing System

Claims

1. an acquisition unit that acquires three-dimensional medical images of a patient requiring placement of an artificial valve; a first region specifying unit that specifies a first region of interest from the three-dimensional medical image; a first direction specifying unit that specifies a first direction of the three-dimensional medical image based on the first region of interest; a second region specifying unit that specifies a second region of interest from the three-dimensional medical image; a second direction specifying unit that specifies a second direction of the three-dimensional medical image based on the second region of interest; a feature amount calculation unit that calculates a first feature amount related to the placement based on the first direction and the second direction; A medical image processing device comprising:

2. the first region of interest includes a first partial region and a second partial region that are different from each other; the second region of interest is the same region as the first region of interest, the first direction identification unit identifies the first direction based on a first positional relationship between the first partial region and the second partial region; The medical image processing apparatus according to claim 1 , wherein the second direction specifying unit specifies the second direction based on a second positional relationship between the first partial region and the second partial region.

3. the first direction specifying unit specifies the first direction based on a position where the ascending aorta and the descending aorta overlap, the position being represented by the three-dimensional medical image in relation to a front direction of the patient; The medical image processing apparatus according to claim 1 , wherein the second direction specifying unit specifies the second direction based on a position of a commissure of an aortic valve represented by the three-dimensional medical image.

4. a third region specifying unit that specifies a third region of interest different from the first region of interest and the second region of interest from the three-dimensional medical image; a third direction specifying unit that specifies a third direction of the three-dimensional medical image based on the third region of interest; Further provided with The medical image processing apparatus according to claim 1 , wherein the feature amount calculation unit further calculates a second feature amount based on the first direction, the second direction, and the third direction.

5. the first region of interest is the aorta, including the ascending aorta and the descending aorta of the patient; the second region of interest is a commissure of the patient's aortic valve; the third region of interest is a coronary artery ostium located between the ascending aorta and the aortic valve; the first feature value is an angle at which the artificial valve is inserted into the patient; The medical image processing apparatus according to claim 4 , wherein the second feature amount is an angle obtained by correcting the first feature amount so that the stent is placed in a state where it does not block the entrance of the coronary artery.

6. The medical image processing apparatus according to claim 4 , wherein the feature amount calculation unit calculates a third feature amount related to the placement based on a predetermined calculation formula using the first feature amount and the second feature amount as variables.

7. 2. The medical image processing apparatus according to claim 1, wherein the feature calculation unit classifies the shape of the anatomical structure in the first region of interest or the second region of interest, selects a calculation formula corresponding to the classification result from a plurality of calculation formulas capable of calculating the first feature, and calculates the first feature based on the selected calculation formula.

8. The medical image processing apparatus according to claim 1 , wherein the feature amount calculation unit calculates, as the first feature amount, an angle at which the artificial valve is inserted into the patient.

9. The medical image processing apparatus according to claim 8 , wherein the feature amount calculation unit calculates, as the angle, an amount of rotation of the artificial valve around an axis of a catheter that holds the artificial valve.

10. the angle is dependent on the position of the commissures of the patient's aortic valve; The medical image processing apparatus according to claim 9 , wherein the amount of rotation is a rotation angle for aligning the commissures of the artificial valve with the commissures of the aortic valve.

11. 11. The medical image processing device according to claim 1, further comprising a display control unit that causes a display to display a first display area that displays the three-dimensional medical image based on the first direction and a second display area that displays the three-dimensional medical image based on the second direction side by side.

12. the display control unit updates the first direction or the second direction in response to a user operation, and updates display content on the display based on a result of the update; The medical image processing apparatus according to claim 11 , wherein the feature amount calculation unit updates the first feature amount based on the updated result.

13. acquiring, by an acquisition unit, a three-dimensional medical image of a patient in need of placement of an artificial valve; Identifying a first region of interest from the three-dimensional medical image by a first region identification unit; specifying a first direction of the three-dimensional medical image based on the first region of interest by a first direction specifying unit; Identifying a second region of interest from the three-dimensional medical image by a second region identification unit; specifying a second direction of the three-dimensional medical image based on the second region of interest by a second direction specifying unit; calculating a first feature amount related to the placement based on the first direction and the second direction by a feature amount calculation unit; A medical image processing method comprising:

Images