Medical information processing device and medical information processing method

Abstract

When attaching a therapeutic device to a target organ in a patient, accurately calculate indicators of the target organ's condition during the procedure. [Solution] The medical information processing device according to the embodiment comprises an acquisition unit, a target organ region identification unit, a device region identification unit, and an indicator identification unit. The acquisition unit acquires a plurality of images of the target organ taken in chronological order. The target organ region identification unit identifies the target organ region in which the target organ is depicted from each of the plurality of images. The device region identification unit identifies the device region in which a device attached to the target organ is depicted from each of the plurality of images. Based on the identified target organ region and device region, the indicator identification unit identifies an indicator related to the state of the target organ to which the device is attached, from the change in the relative position of the device with respect to the target organ in a specific direction of interest.

Description

Technical Field

[0005] , , ,

[0001] The embodiments disclosed in this specification and the drawings relate to a medical information processing apparatus and a medical information processing method.

Background Art

[0002] Conventionally, a technique for attaching a device for treatment to a patient's organ has been known. For example, as a treatment for mitral regurgitation (MR), a technique for reducing the reverse blood flow volume by attaching a clip-shaped device that grips the tips of the anterior and posterior leaflets of the mitral valve is known.

[0003] In such a technique, the movement of the mitral valve is restricted by the device and a load is applied, and this load varies depending on the attachment position of the device. A doctor determines the attachment position of the device by estimating the position where the load can be suppressed within an appropriate range in the procedure. Conventionally, for example, a doctor has estimated the load on the mitral valve by visually and qualitatively evaluating the magnitude of the pressure or tension applied to the mitral valve by observing a transesophageal echocardiography (TEE) image.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] One of the problems that the embodiments disclosed herein and in the drawings aim to solve is to accurately calculate an index of the condition of a target organ during a procedure when attaching a therapeutic device to that organ of a patient. However, the problems that the embodiments disclosed herein and in the drawings aim to solve are not limited to the above problem. Problems corresponding to the effects of each configuration shown in the embodiments described later can also be positioned as other problems. [Means for solving the problem]

[0006] The medical information processing device according to the embodiment comprises an acquisition unit, a target organ region identification unit, a device region identification unit, and an indicator identification unit. The acquisition unit acquires a plurality of images of the target organ taken in chronological order. The target organ region identification unit identifies the target organ region in which the target organ is depicted from each of the plurality of images. The device region identification unit identifies the device region in which a device attached to the target organ is depicted from each of the plurality of images. Based on the identified target organ region and device region, the indicator identification unit identifies an indicator related to the state of the target organ to which the device is attached, from the change in the relative position of the device with respect to the target organ in a specific direction of interest. [Brief explanation of the drawing]

[0007] [Figure 1] Figure 1 shows an example of the procedure in progress during the attachment of a device to the mitral valve. [Figure 2] Figure 2 shows an example of a device attached to the mitral valve. [Figure 3] Figure 3 shows an example of the configuration of a medical image processing apparatus according to the embodiment. [Figure 4] Figure 4 is a flowchart showing an example of the calculation process for an index related to the state of the mitral valve according to the embodiment. [Figure 5] Figure 5 shows an example of a four-dimensional image according to this embodiment. [Figure 6] Figure 6 shows another example of a four-dimensional image according to the embodiment. [Figure 7]Figure 7 shows an example of a method for identifying the mitral valve region as a mesh according to the embodiment. [Figure 8] Figure 8 shows an example of a mesh definition representing the mitral valve region according to the embodiment. [Figure 9] Figure 9 shows an example of the time-series change in the shape of the mitral valve mesh representing the mitral valve region according to this embodiment. [Figure 10] Figure 10 shows an example of a method for identifying the mitral valve region according to the embodiment. [Figure 11] Figure 11 shows an example of a device region 60 identified as a domain according to the embodiment. [Figure 12] Figure 12 shows an example of a method for identifying a device region as a mesh according to the embodiment. [Figure 13] Figure 13 shows an example of a method for identifying a device region as a region according to the embodiment. [Figure 14] Figure 14 shows an example of a device region 60 identified by multiple three-dimensional coordinates according to the embodiment. [Figure 15] Figure 15 shows an example of the characteristic features of a device according to this embodiment. [Figure 16] Figure 16 shows an example of feature points when the device is closed more tightly than in Figure 15. [Figure 17] Figure 17 shows an example of feature points when the device is open wider than in Figure 15. [Figure 18] Figure 18 shows an example of the time-series change of the position of the three-dimensional coordinates that define the device region according to the embodiment. [Figure 19] Figure 19 shows an example of the coordinates of interest according to the embodiment. [Figure 20] Figure 20 shows an example of the time-series change of the position of the coordinate of interest according to this embodiment. [Figure 21] Figure 21 is a diagram showing an example of a direction of interest according to the embodiment. [Figure 22]FIG. 22 is a diagram showing an example of a method for specifying the attention direction D when the device area according to the embodiment is specified as a domain. [Figure 23] FIG. 23 is a diagram showing an example of the temporal change of the attention direction according to the embodiment. [Figure 24] FIG. 24 is a diagram simulating an example of the coordinates of pixels included in the mitral valve region of the three-dimensional image according to the embodiment. [Figure 25] FIG. 25 is a diagram showing an example of the average coordinate position of each pixel included in the anterior tip region of the mitral valve region of the three-dimensional image according to the embodiment. [Figure 26] FIG. 26 is a diagram showing an example of the position of the average coordinate position with respect to the attention direction at the time according to the embodiment. [Figure 27] FIG. 27 is a diagram showing an example of the display mode of the change amount according to the embodiment. [Figure 28] FIG. 28 is a diagram showing an example of a method for specifying the three-dimensional distance between the mitral valve and the device in the time phase according to Modification 1.

MODE FOR CARRYING OUT THE INVENTION

[0008] Hereinafter, embodiments of a medical information processing apparatus and a medical information processing method will be described in detail with reference to the drawings. Note that the medical information processing apparatus and the medical information processing method according to the present application are not limited to the following embodiments. Although a plurality of features are described in the embodiments, not all of these plurality of features are essential, and the plurality of features may be arbitrarily combined. Further, in the accompanying drawings, the same or similar configurations are denoted by the same reference numerals, and redundant descriptions are omitted.

[0009] The medical information processing apparatus and the medical information processing method in the embodiment are used, for example, for calculating an index that indirectly indicates the load applied to the mitral valve tip at the location where the treatment device is attached. Here, a treatment method for mitral valve repair by attaching a device will be described.

[0010] Various treatment methods have been proposed for treating heart disease. For example, in secondary mitral regurgitation (MR), a physician uses a catheter to transvascularly implant a device into the patient's mitral valve.

[0011] Figure 1 shows an example of the procedure in progress during the attachment of the device 30 to the mitral valve 20. The physician attaches the device 30 to the mitral valve 20 by manipulating the shaft 31. The device 30 is a type of mitral valve repair device that can increase the bonding area by grasping the tips of the anterior leaflet 21 and posterior leaflet 22 of the mitral valve 20.

[0012] This type of technique is called Edge-to-Edge Repair. Examples of devices 30 used in Edge-to-Edge Repair include MitraClip (registered trademark) and PASCAL (Edwards Lifesciences).

[0013] Figure 2 shows an example of the device 30 being attached to the mitral valve 20. After attaching the device 30 to the appropriate position, the physician releases the shaft 31, thereby attaching the device 30 while grasping the tips of the anterior leaflet 21 and posterior leaflet 22 of the mitral valve 20. The attachment position of the device 30 can be changed until the shaft 31 is released, but once the shaft 31 is released, the device 30 is fixed in the position at the time of release. For this reason, physicians generally carefully confirm the attachment position of the device 30 using transesophageal echocardiography (TEE) images or other images of the mitral valve 20 before releasing the shaft 31.

[0014] Here, transcatheter mitral valve repair using device 30 for patients with secondary mitral regurgitation is a widely used and approved treatment method, and its safety and efficacy have been supported by large-scale randomized clinical trials. However, despite its proven safety record, a small number of serious leaflet adverse events (LAEs) associated with the treatment device have been reported. Types of LAEs include leaflet perforation, leaflet rupture, leaflet deformation, partial leaflet grasp, and chordal involvement. One example of a contributing factor to LAEs such as leaflet rupture is the increased tension and other loads on the mitral valve 20 due to the attachment of the device 30, which restrains the periodic movement of the mitral valve 20.

[0015] There are techniques that allow for the prior calculation of the appropriate mounting position of the device 30 or the load on the mitral valve 20, such as blood flow, through computer simulation. However, predicting the load that the mitral valve 20 can withstand with high accuracy in advance can be difficult, as it varies depending on the individual patient's condition. Furthermore, in order to determine whether or not the shaft 31 can be released, the physician is required to check the condition of the mitral valve 20 in real time during the procedure. The medical information processing device and medical information processing method of this embodiment are used to provide the physician with indicators regarding the condition of the mitral valve 20 in such cases.

[0016] Figure 3 shows an example of the configuration of a medical image processing device 100 according to an embodiment. The medical image processing device 100 is, for example, a computer such as a server or a PC (Personal Computer).

[0017] As shown in Figure 3, the medical image processing device 100 includes, for example, a network interface 110, a memory circuit 120, an input interface 130, a display 140, and a processing circuit 150.

[0018] The NW interface 110 is connected to the processing circuit 150 and controls the transmission and communication of various data between the medical image processing device 100 and other devices. Other devices include, but are not limited to, medical image storage devices such as PACS (Picture Archiving and Communication System) for storing medical image data, various modalities (medical imaging devices), and electronic medical record systems. The NW interface 110 is implemented by a network card, network adapter, NIC (Network Interface Controller), etc.

[0019] The memory circuit 120 stores various types of information used by the processing circuit 150 in advance. The memory circuit 120 also stores various programs. The memory circuit 120 is, for example, a non-volatile storage device such as an HDD (Hard Disk Drive), SSD (Solid State Drive), or integrated circuit memory device that stores various types of information. In addition to HDDs and SSDs, the memory circuit 120 may also be a drive device that reads and writes various types of information to portable storage media such as CDs (Compact Discs), DVDs (Digital Versatile Discs), flash memory, or semiconductor memory elements such as RAM (Random Access Memory). The memory circuit 120 is an example of a storage unit.

[0020] The input interface 130 is implemented by a mouse, keyboard, pen tablet (combining a stylus and tablet that accept user input), trackball, switch buttons, touchpad (for input operations by touching the operating surface), touchscreen (integrating a display screen and touchpad), non-contact input circuit using an optical sensor, and audio input circuit, etc. The input interface 130 may include multiple devices that accept user operations. The input interface 130 is connected to the processing circuit 150 and converts the input operations received from the user into electrical signals and outputs them to the processing circuit 150. In this specification, the input interface is not limited to those equipped with physical operating components such as a mouse or keyboard. For example, an electrical signal processing circuit that receives electrical signals corresponding to input operations from an external input device provided separately from the device and outputs these electrical signals to the processing circuit 150 is also included as an example of an input interface.

[0021] The display 140 displays various information under the control of the processing circuit 150. For example, the display 140 outputs various generated images and a GUI (Graphical User Interface) for accepting various operations from the user. Specifically, the display 140 is an LCD display or a CRT (Cathode Ray Tube) display, etc. The input interface 130 and the display 140 may be integrated. For example, the input interface 130 and the display 140 may be implemented by a touch panel. The display 140 is an example of a display unit.

[0022] The processing circuit 150 is a processor that reads programs from the memory circuit 120 and executes them to realize functions corresponding to each program. The processing circuit 150 in this embodiment includes an acquisition function 151, a target organ region identification function 152, a device region identification function 153, a focus coordinate identification function 154, a focus direction identification function 155, an average coordinate identification function 156, a change amount calculation function 157, a display control function 158, and a reception function 159. The acquisition function 151 is an example of an acquisition unit. The target organ region identification function 152 is an example of a target organ region identification unit. The device region identification function 153 is an example of a device region identification unit. The focus coordinate identification function 154 is an example of a focus coordinate identification unit. The focus direction identification function 155 is an example of a focus direction identification unit. The average coordinate identification function 156 is an example of an average coordinate identification unit. The change amount calculation function 157 is an example of a change amount calculation unit and an index identification unit. The display control function 158 is an example of a display control unit. Reception function 159 is an example of a reception area.

[0023] Here, for example, the processing functions of the processing circuit 150, which are components of the processing circuit 150, such as the acquisition function 151, the target organ region identification function 152, the device region identification function 153, the focus coordinate identification function 154, the focus direction identification function 155, the average coordinate identification function 156, the change amount calculation function 157, the display control function 158, and the reception function 159, are stored in the memory circuit 120 in the form of programs that can be executed by a computer. The processing circuit 150 is a processor. For example, the processing circuit 150 realizes the functions corresponding to each program by reading the program from the memory circuit 120 and executing it. In other words, the processing circuit 150 in the state in which each program has been read will have the functions shown in the processing circuit 150 in Figure 3. In Figure 3, the processing functions performed by the acquisition function 151, target organ region identification function 152, device region identification function 153, focus coordinate identification function 154, focus direction identification function 155, average coordinate identification function 156, change amount calculation function 157, display control function 158, and reception function 159 are explained as being realized by a single processor. However, it is also acceptable to configure the processing circuit 150 by combining multiple independent processors, with each processor executing a program to realize the functions. Furthermore, in Figure 3, it is explained that a single memory circuit 120 stores the programs corresponding to each processing function. However, it is also acceptable to configure the processing circuit 150 by distributing multiple memory circuits and reading the corresponding programs from individual memory circuits.

[0024] The above description illustrates an example in which a "processor" reads and executes programs corresponding to each function from a memory circuit, but the embodiments are not limited to this. The term "processor" refers to circuits such as a CPU (Central Processing Unit), GPU (Graphics Processing Unit), Application Specific Integrated Circuit (ASIC), and Programmable Logic Device (e.g., Simple Programmable Logic Device (SPLD), Complex Programmable Logic Device (CPLD), and Field Programmable Gate Array (FPGA)). If the processor is a CPU, for example, it realizes its functions by reading and executing programs stored in a memory circuit. On the other hand, if the processor is an ASIC, instead of storing the program in the memory circuit 120, the function is directly incorporated as a logic circuit within the processor's circuit. In this embodiment, each processor is not limited to being configured as a single circuit; multiple independent circuits may be combined to form a single processor and realize its functions. Furthermore, the multiple components shown in Figure 3 may be integrated into a single processor to realize its functions.

[0025] The acquisition function 151 acquires multiple images of the target organ taken in chronological order at multiple different points in time from a medical image storage device or various modalities, for example, via the NW interface 110. The target organ is the object to which the device 30 is attached, and is the object to measure the state of the organ when the device 30 is attached. In particular, organs that move periodically or regularly are envisioned as target organs. The device 30 also has a function to restrain such periodic or regular movement. In this embodiment, the mitral valve 20 is used as an example of a target organ. The mitral valve 20 moves back and forth regularly and periodically. In this embodiment, as described above, mitral valve repair devices such as MitraClip® and PASCAL (Edwards Lifesciences) are used as examples of devices 30.

[0026] Furthermore, the multiple images are, for example, multiple three-dimensional images of the mitral valve 20 taken at different times. Such images contain spatial three-dimensional information and temporal (one-dimensional) information, and are therefore also called four-dimensional images. More specifically, a four-dimensional image is, for example, an image taken sequentially in a time series that includes morphological information of the three-dimensional anatomical structure of the mitral valve 20. The form of the four-dimensional image is not limited to this, and any image including at least two or more three-dimensional images taken at different times can be used. In this embodiment, the four-dimensional image of the mitral valve 20 is a 4D TEE image. The type of three-dimensional image taken at different times is not limited to TEE images, and may be medical images taken by various modalities such as CT (Computed Tomography) images, MRI (Magnetic Resonance Imaging) images, X-ray images, PET (Positron Emission computed tomography) images, and SPECT (Single Photon Emission Computed Tomography) images.

[0027] The target organ region identification function 152 identifies the region in which the mitral valve 20, which is the target organ, is depicted from the four-dimensional image acquired by the acquisition function 151. Identifying the region in which the mitral valve 20 is depicted means identifying the three-dimensional coordinates of each pixel in the region in which the mitral valve 20 is depicted. More specifically, the target organ region identification function 152 identifies the region in which the mitral valve 20 is depicted from each of multiple three-dimensional images taken at multiple different time points included in the four-dimensional image. Hereinafter, the region in which the mitral valve 20 is depicted will be referred to as the mitral valve region. The mitral valve region is an example of a target organ region in this embodiment.

[0028] The device region identification function 153 identifies the region in which the device 30 attached to the target organ is depicted from the four-dimensional image acquired by the acquisition function 151. Identifying the region in which the device 30 is depicted means identifying the three-dimensional coordinates of each pixel in the region in which the device 30 is depicted. More specifically, the device region identification function 153 identifies the region in which the device 30 is depicted from each of multiple three-dimensional images taken at different time points included in the four-dimensional image. Hereinafter, the region in which the device 30 is depicted will be referred to as the device region.

[0029] The focus coordinate identification function 154 identifies the focus coordinate of device 30 from the device region identified by the device region identification function 153 in each of the multiple three-dimensional images taken at different time points included in the four-dimensional image. The focus coordinate is a coordinate that represents the position of device 30 in the three-dimensional image. For example, the focus coordinate is the three-dimensional coordinate of any single point within the device region. As an example, the focus coordinate may be the centroid of multiple feature points corresponding to the edges of device 30.

[0030] The direction of focus identification function 155 identifies the direction of focus based on the device region identified by the device region identification function 153 from each of multiple three-dimensional images taken at different points in time and included in the four-dimensional image. The direction of focus is, for example, the direction in which the device 30 primarily moves in conjunction with the operation of the mitral valve 20. The device 30 attached to the mitral valve 20 generally reciprocates periodically along the long axis of the device 30 together with the mitral valve 20. Therefore, the direction of focus identification function 155 identifies the long axis of the device 30 in three-dimensional space as the direction of focus.

[0031] The average coordinate identification function 156 identifies the average coordinate position of the target organ at each of several different time points, based on each of the three-dimensional images taken at different time points included in the four-dimensional image. The multiple different time points are the time points at which each of the three-dimensional images included in the four-dimensional image acquired by the acquisition function 151 was taken.

[0032] The change calculation function 157 identifies an indicator of the state of the target organ to which the device 30 is attached, based on the identified target organ region and device region, from the change in the relative position of the device 30 to the target organ in a specific direction of interest between multiple different time points.

[0033] In this embodiment, the change in distance between the coordinate of interest and the mitral valve region at multiple different time points is an example of an indicator of the state of the target organ. The calculation of the change is an example of identifying the indicator. The change in distance between the coordinate of interest and the mitral valve region at multiple different time points varies depending on the magnitude of the pressure or tension on the mitral valve 20. For example, if the mitral valve 20 is under high tension and is tightly stretched, the movement of the mitral valve 20 is strongly restricted, so the change in distance between the coordinate of interest and the mitral valve region becomes small. That is, the greater the tension, the smaller the change in distance between the coordinate of interest and the mitral valve region, and the less the tension, the larger the change in distance between the coordinate of interest and the mitral valve region. Therefore, the change in distance between the coordinate of interest and the mitral valve region at multiple different time points serves as an indirect indicator of the tension on the mitral valve 20, etc. Furthermore, the change in distance between the coordinate of interest and the mitral valve region at multiple different time points also differs depending on the degree of flexibility of the mitral valve 20.

[0034] The display control function 158 displays various images or GUIs on the display 140. For example, the display control function 158 presents indicators related to the state of the target organ identified by the change amount calculation function 157 to the physician during the procedure by displaying them on the display 140. The display control function 158 may also display, for example, the amount of change in the distance between the coordinate of interest and the mitral valve region along with images depicting the mitral valve 20 and the device 30. Alternatively, the display control function 158 may display a graph on the display 140 showing the time-series change in the amount of change in the distance between the coordinate of interest and the mitral valve region.

[0035] The reception function 159 accepts various user operations via the input interface 130. For example, the reception function 159 may accept a user operation specifying the position of the mitral valve 20 in the three-dimensional image. The reception function 159 may also accept a user operation specifying the position of the device 30 in the three-dimensional image. Furthermore, the reception function 159 may accept a user operation specifying the direction of interest in the three-dimensional image.

[0036] Here, the process of calculating an index related to the state of the mitral valve 20, which is performed by the medical image processing device 100 configured as described above, will be explained using a flowchart.

[0037] Figure 4 is a flowchart showing an example of the calculation process for an index related to the state of the mitral valve 20 according to the embodiment. This flowchart is executed, for example, after the device 30 is attached to the mitral valve 20 and before the shaft 31 is released. In the following description, several drawings will be used along with the flowchart, but the shaft 31 will not be shown in each drawing.

[0038] First, the acquisition function 151 acquires a four-dimensional image, such as a 4D TEE image of the mitral valve 20 (S1).

[0039] Here, Figure 5 shows an example of a four-dimensional image according to the embodiment. The four-dimensional image may be three-dimensional images 90a to 90d of the mitral valve 20 taken at consecutive times t0, t1, t2, and t3, respectively, as shown in Figure 5. Figure 6 shows another example of a four-dimensional image according to the embodiment. The acquisition times of the multiple three-dimensional images included in the four-dimensional image do not necessarily have to be consecutive. For example, as shown in Figure 6, three-dimensional images 90a and 90c taken at intervals between times t0 and t2 may constitute the four-dimensional image. In this embodiment, the acquisition function 151 will be described as acquiring three-dimensional images 90a to 90d of the mitral valve 20 taken at consecutive times t0, t1, t2, and t3, respectively, as shown in Figure 5.

[0040] Returning to Figure 4, the target organ region identification function 152 then identifies the region of the target organ (mitral valve 20) (mitral valve region) from each of the multiple three-dimensional images included in the acquired four-dimensional image (S2). For example, the target organ region identification function 152 acquires coordinate information for each pixel in which the mitral valve 20 is depicted from each three-dimensional image 90a to 90d included in the 4D TEE image. The coordinate information for each pixel of the mitral valve 20 is, in other words, the coordinate information for each pixel included in the mitral valve region. The target organ region identification function 152 may acquire the coordinate information for each pixel included in the mitral valve region as a mesh or as a region (segment).

[0041] Figure 7 shows an example of a method for identifying the mitral valve region 200 as a mesh according to the embodiment. As shown in the mesh image 91 in Figure 7, the mitral valve region 200 is represented, for example, by a three-dimensional mesh model. The mitral valve region 200 includes an anterior leaflet region 201 in which the anterior leaflet 21 is depicted and a posterior leaflet region 202 in which the posterior leaflet 22 is depicted.

[0042] Here, the three-dimensional mesh model represents the mitral valve region 200 as a computational grid (mesh) by, for example, setting multiple grid points on the identified mitral valve region. In this case, the number and arrangement of grid points may be predetermined, or the target organ region identification function 152 may determine them based on the size and shape of the mitral valve region 200.

[0043] Figure 8 shows an example of a mesh definition representing the mitral valve region 200 according to the embodiment. The mitral valve mesh representing the mitral valve region 200 in the mesh image 91 of Figure 8 includes an anterior leaflet mesh representing the anterior leaflet region 201 and a posterior leaflet mesh representing the posterior leaflet region 202. The number of grid points for the anterior leaflet mesh is, for example, 171 (19 columns × 9 rows), and the number of grid points for the posterior leaflet mesh is 225 (25 columns × 9 rows). The mitral valve mesh also includes the annular region 70, anterior commissure coordinates 73, posterior commissure coordinates 74, coordinates of two feature points on the annular region (for example, the coordinate of the mitral annulus closest to the right fiber triangle 71, and the coordinate of the mitral annulus closest to the left fiber triangle 72), the valve tip region 75, and the midpoint of the fiber triangles 76 as constituent elements. In the example shown in Figure 8, the annular region 70 corresponds to the grid in row Y=0 of the mitral valve mesh. Furthermore, the valve tip region 75 corresponds to the grid cell in the Y=8 row of the mitral valve mesh. The anterior commissure coordinate 73 indicates the position of the anterior commissure, and in the example shown in Figure 8, it corresponds to the position X=0, Y=8. The posterior commissure coordinate 74 indicates the position of the posterior commissure, and in the example shown in Figure 8, it corresponds to the position X=18, Y=8. With this definition, the shape of the mitral valve region 200 is represented by the mesh. The number and arrangement of grid points of the mesh shown in Figure 8, and the feature points extracted, are examples only, and the definition of the mesh is not limited to this.

[0044] Figure 9 is a diagram showing an example of the time-series change in the shape of the mitral valve mesh representing the mitral valve region 200 according to the embodiment. The mesh images 91a to 91d in Figure 9 each represent the mitral valve regions 200a to 200d identified from the three-dimensional images 90a to 90d shown in Figure 5. As described above, since the mitral valve 20 reciprocates regularly and periodically, the shape of the mitral valve 20 captured in each of the three-dimensional images 90a to 90d is different. The target organ region identification function 152 identifies the shape of the mitral valve regions 200a to 200d at times t0 to t3 by identifying the mitral valve regions 200a to 200d from each of the three-dimensional images 90a to 90d.

[0045] Figure 10 is a diagram showing an example of a method for identifying the mitral valve region 200 as a region according to the embodiment. As shown in the segmentation image 92 in Figure 10, the mitral valve region 200 may be identified, for example, as a continuous three-dimensional region. In this method as well, the anterior leaflet region 201 and the posterior leaflet region 202 included in the mitral valve region 200 can be identified.

[0046] Known techniques can be used to identify the mitral valve region 200 in the three-dimensional image 90. For example, the target organ region identification function 152 may identify the mitral valve region 200 based on the position of the mitral valve 20 manually specified by the user, which is received by the reception function 159 via the input interface 130. Alternatively, the target organ region identification function 152 may identify the mitral valve region 200 based on anatomical structures extracted by known region extraction techniques. Known region extraction methods include, for example, Otsu's binarization method, region expansion method, snake method, graph cut method, and mean shift method based on the brightness values ​​of each three-dimensional image 90a to 90d. Furthermore, the target organ region identification function 152 may identify the mitral valve region 200 using a shape model of the mitral valve region 200 constructed based on pre-prepared training data using machine learning techniques (including deep learning). In addition, the display control function 158 may present the above-mentioned multiple methods, and the user may select one of the presented methods. In this case, the target organ region identification function 152 identifies the mitral valve region 200 using a method selected by the user.

[0047] Returning to Figure 4, the device region identification function 153 then identifies the region of device 30 (device region) from each of the multiple three-dimensional images included in the acquired four-dimensional image (S3). For example, the device region identification function 153 acquires the coordinate information of each pixel in which device 30 is depicted from each three-dimensional image 90a to 90d included in the 4D TEE image. The coordinate information of each pixel of device 30 is, in other words, the coordinate information of each pixel included in the device region.

[0048] The device region identification function 153 may identify the coordinate information of each pixel included in the device region as a domain, or it may identify it using two or more three-dimensional coordinates.

[0049] Figure 11 shows an example of a device region 60 identified as a domain according to the embodiment. The device region identification function 153 may identify a domain from the three-dimensional image 90 by specifying the three-dimensional coordinates of the area corresponding to the contour of the entire device 30. Furthermore, when the device region identification function 153 identifies the device region 60 as a domain, it may identify it as a mesh or as a region (segment).

[0050] Figure 12 shows an example of a method for identifying the device region 60 as a mesh, which is one of the methods for identifying the device region 60 as a domain according to the embodiment. The device region identification function 153 may identify the device region mesh representing the device region 60 using the same method as the identification of the mitral valve mesh representing the mitral valve region 200 described in Figures 7 and 8. The number and arrangement of the mesh grids shown in Figure 12 are just an example and are not limited thereto.

[0051] Figure 13 is a diagram showing an example of a method for identifying the device region 60 as a domain, among the methods for identifying the device region 60 as a domain according to the embodiment. The device region identification function 153 may identify the device region 60 as a continuous three-dimensional region, similar to the identification of the mitral valve region 200 described in Figure 10.

[0052] Figure 14 also shows an example of a method for identifying the device region 60 according to the embodiment as a plurality of three-dimensional coordinates 61 to 63. In the example shown in Figure 14, the device region 60 is identified by the three-dimensional coordinates 61 to 63 of three points in the three-dimensional image 90, but the number of three-dimensional coordinates is not limited to the example shown in Figure 14, as long as it is a number that can represent the spatial shape of the entire device 30.

[0053] As an example, the three three-dimensional coordinates 61-63 shown in Figure 14 correspond to three feature points of device 30.

[0054] Figure 15 shows an example of feature points 301 to 303 of a device 30 according to an embodiment. As shown in Figure 15, feature points 301 to 303 are, for example, three endpoints of the device 30. Generally, the device 30 has a bifurcated portion for gripping the mitral valve 20 and a tip portion connecting the bifurcated portion. The two endpoints of the bifurcated portion for gripping the mitral valve 20 correspond to feature points 302 and 303, respectively. The length of the line segment connecting feature point 302 and feature point 303 is the maximum length of the device 30 in the short-side direction. The endpoint of the tip portion of the device 30 corresponds to feature point 301. The device region identification function 153 can identify the device region 60 corresponding to the entire device 30 in the three-dimensional image 90 by identifying these three endpoints as feature points 301 to 303.

[0055] Note that the arrangement of the endpoints of the device 30 may differ depending on the opening and closing angle of the device 30 in the three-dimensional image 90. For example, Figure 16 shows an example of feature points 301 and 302 when the device 30 is closed more tightly than in Figure 15. When the device 30 firmly grips the mitral valve 20, as shown in Figure 16, the bifurcated parts of the device 30 may almost overlap, causing feature points 302 and 303 to be depicted as a single feature point 302. Also, Figure 17 shows an example of feature points 301 to 303 when the device 30 is open more than in Figure 15. Even when the opening angle of the device 30 is wide in this way, by identifying the three endpoints as feature points 301 to 303, the device region 60 corresponding to the entire device 30 in the three-dimensional image 90 can be identified.

[0056] Known techniques can be used to identify the device region 60 in the three-dimensional image 90. For example, the device region identification function 153 may identify the device region 60 based on the range in which the device 30 is depicted or the positions of feature points 301 to 303 of the device 30, which are manually specified by the user and received by the reception function 159 via the input interface 130. Alternatively, the device region identification function 153 may identify the device region 60 based on anatomical structures extracted by known region extraction techniques. Known region extraction methods include, as mentioned above, Otsu's binarization method, region expansion method, snake method, graph cut method, mean shift method, etc., based on the brightness values ​​of each three-dimensional image 90a to 90d. Furthermore, the device region identification function 153 may identify the device region 60 using a shape model of the device region 60 constructed based on pre-prepared training data using machine learning techniques (including deep learning). Alternatively, the device region identification function 153 may identify the device region 60 based on feature points 301 to 303 of the device 30 extracted by a known feature point extraction technique. Furthermore, the display control function 158 may present the above-mentioned multiple methods, and the user may select one of the presented methods. In this case, the device region identification function 153 identifies the device region 60 using the method selected by the user.

[0057] Figure 18 is a diagram showing an example of the time-series change of the positions of the three-dimensional coordinates 61-63 that define the device region 60 according to the embodiment. In Figure 18, the device region identification function 153 identifies the device region 60 as multiple three-dimensional coordinates 61a-61d, 62a-62d, and 63a-63d from the three-dimensional images 90a-90d, as explained in Figure 14.

[0058] Since device 30 is attached to the mitral valve 20, the three-dimensional coordinates 61-63 move to different positions as the mitral valve 20 moves from time t0 to t3. That is, the device region identification function 153 identifies the change in the position of device 30 from time t0 to t3 by identifying the three-dimensional coordinates 61a-61d, 62a-62d, and 63a-63d from the three-dimensional images 90a-90d, respectively.

[0059] Returning to Figure 4, the next step is for the focus coordinate identification function 154 to identify the focus coordinate of device 30 from among the device regions 60 identified by the device region identification function 153 from each of the multiple three-dimensional images 90a to 90d (S4).

[0060] Figure 19 shows an example of a focus coordinate 650 according to the embodiment. The focus coordinate 650 is any one point among the three-dimensional coordinates included within the device region 60. In the example shown in Figure 19, the focus coordinate 650 corresponds to the centroid of a plurality of three-dimensional coordinates 61 to 63 that correspond to the end of the device 30. For example, the focus coordinate identification function 154 identifies the midpoint 610 of the two three-dimensional coordinates 61 and 63 of the bifurcated part of the device 30. Then, the focus coordinate identification function 154 sets the midpoint between the three-dimensional coordinate 61, which is the tip of the device 30, and the midpoint 610 as the focus coordinate 650. Note that the focus coordinate 650 shown in Figure 19 is just an example and is not limited thereto. For example, as another example, the focus coordinate identification function 154 may set the three-dimensional coordinate 61, which is the tip of the device 30, as the focus coordinate 650.

[0061] Figure 20 shows an example of the time-series change of the positions of the coordinates of interest 650a to 650d according to the embodiment. In Figure 20, the three-dimensional image 90 is omitted from the illustration, but the positions of the coordinates of interest 650a to 650d are identified in the three-dimensional images 90a to 90d, respectively, in the same way as the three-dimensional coordinates 61a to 61d, 62a to 62d, and 63a to 63d in Figure 18.

[0062] Since device 30 is attached to the mitral valve 20, the coordinates 650a to 650d move to different positions as the mitral valve 20 moves during time t0 to t3. In other words, the coordinates 650a to 650d represent multiple pixels contained within the device region 60 and show the change in the position of device 30 during time t0 to t3.

[0063] Returning to Figure 4, the focus direction identification function 155 then identifies the focus direction of the device 30 based on the device region 60 identified by the device region identification function 153 from each of the multiple three-dimensional images 90a to 90d (S5).

[0064] Figure 21 shows an example of a direction of interest D according to the embodiment. The direction of interest D is, for example, the longitudinal axis direction of the device 30. In general, the longitudinal axis direction of the device 30 is the direction along the blood flow direction in the mitral valve 20. The direction of interest identification function 155 may identify the longitudinal axis direction of the device 30 based on the geometric relationship of a plurality of three-dimensional coordinates that define the device region 60.

[0065] For example, in the example shown in Figure 21, the focus direction identification function 155 identifies the direction of focus as the extension direction from the focus coordinate 650 to the midpoint 610 of the line segment connecting the midpoint 610 of the three-dimensional coordinates 61 and 63, which correspond to the two feature points 302 and 303 of the bifurcated portion of the device 30, and the focus coordinate 650.

[0066] Various known techniques can be used to identify the direction of interest D. For example, the direction of interest identification function 155 may identify the direction of interest D based on the user operation received by the reception function 159 via the input interface 130.

[0067] Alternatively, the focus direction identification function 155 may identify the first principal component obtained from principal component analysis of the coordinate information of each pixel included in the device region 60 of each three-dimensional image 90a to 90d as the focus direction D. Figure 22 is a diagram showing an example of a method for identifying the focus direction D when the device region 60 according to the embodiment is identified as a domain. In the example shown in Figure 22, the focus direction identification function 155 identifies the focus direction D by principal component analysis of the coordinate information of each pixel included in the device region 60.

[0068] Alternatively, the focus direction identification function 155 may identify the focus direction D using a model that estimates the direction of the long axis of the device 30, which is constructed based on pre-prepared training data using machine learning techniques (including deep learning). The method for identifying the focus direction is not limited to this; any method that identifies the typical direction of reciprocating motion of an organ moving back and forth in three-dimensional space can be used. Furthermore, the display control function 158 may present multiple methods, and the user may select one of the presented methods. In this case, the focus direction identification function 155 identifies the focus direction D using the method selected by the user.

[0069] Figure 23 shows an example of the time series change of the direction of interest D according to this embodiment.

[0070] Since the mitral valve 20 and the device 30 attached to the mitral valve 20 operate three-dimensionally in three-dimensional space, the direction of interest D will also be different for each of the times t0 to t3. In Figure 23, the direction of interest D is represented as a two-dimensional direction, but in reality, the direction of interest D changes three-dimensionally between each time phase.

[0071] Returning to Figure 4, the mean coordinate identification function 156 then identifies the mean coordinates of the mitral valve region 200 at each time point t0 to t3 when the three-dimensional images 90a to 90d included in the four-dimensional image were taken (S6).

[0072] Figure 24 is a diagram that simulates an example of the coordinates 211-214 and 221-224 of pixels included in the mitral valve region 200 of the three-dimensional image 90 according to the embodiment. The average coordinate identification function 156 may, for example, identify the coordinates 211-214 and 221-224 of each pixel included in the mitral valve region 200, and identify the average of the identified coordinates 211-214 and 221-224 as the average coordinate.

[0073] Furthermore, the average coordinate identification function 156 may identify the average coordinates not for the entire mitral valve 20, but for either the anterior leaflet 21 or the posterior leaflet 22 of the mitral valve 20. Figure 25 shows an example of the average coordinates 210 of each pixel coordinate 211 to 214 included in the anterior leaflet region of the mitral valve region 200 of the three-dimensional image 90 according to the embodiment.

[0074] Returning to Figure 4, the change amount calculation function 157 then identifies the change in distance between the average coordinate 210 and the coordinate of interest 650 in the direction of interest (S7).

[0075] The change amount calculation function 157 identifies, for example, the maximum position and the minimum position in the direction of interest D from the average coordinates 210 at each of the times t0 to t3.

[0076] Figure 26 shows an example of the position of the average coordinates 210a to 210d relative to the direction of interest D at times t0 to t6 according to the embodiment. Up until now, we have explained using times t0 to t3 as an example, but in Figure 26, we will explain using times t4 to t6 in order to illustrate a longer period. Note that times t4 to t6 are times after time t3, and are also the times when different three-dimensional images 90 were taken.

[0077] As described above, the mitral valve 20 reciprocates periodically, so the mean coordinate 210 of the mitral valve region 200 moves back and forth along the direction of interest D over time. In the example shown in Figure 26, the mean coordinate 210 moves from its position at time t0 (mean coordinate 210a) to the opposite direction of the direction of interest D from time t1 to t3 (mean coordinates 210b to 210d). Then, at time t3, the reciprocating motion reverses direction. After that, the mean coordinate 210 moves from its position at time t3 (mean coordinate 210d) to the direction of interest D from time t4 to t6 (mean coordinates 210e to 210g). At time t6, the mean coordinate 210g returns to its position at time t0 (mean coordinate 210a).

[0078] In the example shown in Figure 26, the average coordinates 210a and 210g at times t0 and t6 are the coordinates located furthest towards the extension direction of the direction of interest D. Therefore, the change amount calculation function 157 uses the average coordinates 210a and 210g as the maximum position in the direction of interest D. Also, in the example shown in Figure 26, the average coordinate 210d at time t3 is the coordinate located furthest away from the extension direction of the direction of interest D. The change amount calculation function 157 uses the average coordinate 210d as the minimum position in the direction of interest D.

[0079] In Figure 26, the coordinate of interest 650 is shown as a single position, but in reality, the coordinate of interest 650 also moves along with the movement of the device 30 attached to the mitral valve 20. However, due to the expansion and contraction of the mitral valve 20, a difference arises between the amount of movement of the average coordinate 210 and the amount of movement of the device 30. Therefore, the distance between the coordinate of interest 650 and the average coordinate 210 in the direction of interest D fluctuates over time. Furthermore, the maximum position of the average coordinate of the mitral valve 20 is not limited to a single point; a certain range may be defined as the "maximum position" of the mitral valve 20. For example, the range of the "maximum position" of the mitral valve 20 may be defined as coordinates above a specified threshold in the direction of interest D, and coordinates to be used for calculating the amount of change described later may be selected from the average coordinates of the mitral valve 20 included in that range.

[0080] The change amount calculation function 157 calculates the distance A1 between the coordinate 650 of interest in the direction of interest D and the maximum position (average coordinates 210a, 210g). The change amount calculation function 157 also calculates the distance A2 between the coordinate 650 of interest in the direction of interest D and the minimum position (average coordinate 210d). The change amount calculation function 157 calculates the absolute value of the difference between distance A1 and distance A2 as the change amount C. Distance A1 is an example of a first distance in this embodiment. Distance A2 is an example of a second distance in this embodiment. Since the average coordinate 210 is a coordinate representative of the mitral valve region 200, distances A1 and A2 represent the distance between the coordinate 650 of interest and the mitral valve region 200 at times t0 and t3, respectively.

[0081] Returning to Figure 4, the display control function 158 then displays the change amount C calculated by the change amount calculation function 157 on the display 140 (S8). The change amount C is an example of an indicator related to the state of the mitral valve 20.

[0082] Figure 27 is a diagram showing an example of how the change amount C is displayed according to the embodiment. As shown in Figure 27, the display control function 158 may display the average coordinates 210a and 210d at times t0 and t3, and the target coordinates 650a and 650d at times t0 and t3, together with the change amount C from time t0 to time t3 on the display 140. In this case, the display control function 158 may superimpose the value of the change amount C onto the image 93 in which the mitral valve 20 and device 30 are depicted at times t0 and t3. Note that the display method of the change amount C is not limited to the example shown in Figure 27. The display control function 158 may also display a graph on the display 140 showing the time-series change in the change amount C of the distance between the target coordinate 650 and the average coordinate 210. At this point, the processing of this flowchart ends. Note that until the shaft 31 is released by the physician, the processing of this flowchart is repeatedly executed during the procedure, so that the physician can continuously check indicators regarding the state of the mitral valve 20 during the procedure.

[0083] As described above, the medical image processing device 100 of this embodiment identifies the mitral valve region 200 and the device region 60 from each of a plurality of images of the mitral valve 20 of a subject taken in a time series, and calculates an index regarding the state of the mitral valve 20 to which the device 30 is attached from the change in the relative position of the device 30 to the mitral valve 20 in a specific direction of interest D, based on the identified mitral valve region 200 and device region 60. Therefore, with the medical image processing device 100 of this embodiment, when attaching a therapeutic device 30 to the mitral valve 20 of a subject, an index regarding the state of the mitral valve 20 can be calculated with high accuracy during the procedure. Therefore, with the medical image processing device 100 of this embodiment, it is possible to support the physician's decision-making regarding the release of the shaft 31, the change in the attachment position of the device 30, or the discontinuation of the procedure.

[0084] For example, when estimating the tension of the mitral valve by simulating the three-dimensional shape of the mitral valve or the time-dependent changes in blood flow through the mitral valve, the computational cost is high, making it difficult to use during the procedure. In contrast, the method of this embodiment makes it possible to calculate an index that indirectly indicates the tension of the mitral valve 20 as a numerical value from the geometric positional relationship between the mitral valve 20 and the device 30, thereby reducing the computational cost and making it usable during the procedure.

[0085] Furthermore, the medical image processing device 100 of this embodiment identifies a point of interest coordinate 650, which is the three-dimensional coordinate of a single point included in the device region 60, and a direction of interest D, which is the long axis direction of the device 30 in three-dimensional space, from each of the multiple three-dimensional images 90a to 90d of the mitral valve 20 taken in time series. The device calculates the amount of change C in the distance between the point of interest coordinate 650 and the mitral valve 20 in the direction of interest D at multiple different time points (times t0 to t3) as an indicator of the state of the mitral valve 20. Since the shape of the device 30 does not change over time, by calculating the amount of change C based on the point of interest coordinate 650, which is based on the geometric shape of the device 30, as in the medical image processing device 100 of this embodiment, it is possible to reduce the influence of changes in coordinates in three-dimensional space related to movements other than the mitral valve 20, such as the movement of the ultrasound probe due to the subject's body movement or respiration, and heartbeat. Furthermore, according to the medical image processing device 100 of this embodiment, by using the direction of interest D, which is based on the geometric shape of the device 30 that remains unchanged over time, as the reference direction for calculating the amount of change C, the amount of change C in the coordinates in three-dimensional space in the blood flow direction of greatest interest can be calculated with high accuracy.

[0086] Furthermore, the medical image processing device 100 of this embodiment identifies the maximum and minimum positions relative to the direction of interest D from the mitral valve region 200 identified from each of the multiple three-dimensional images 90a to 90d of the mitral valve 20. The medical image processing device 100 of this embodiment calculates the change amount C as the absolute value of the difference between the distance A1 between the coordinate of interest 650 in the direction of interest D and the maximum position, and the distance A2 between the coordinate of interest 650 in the direction of interest D and the minimum position. Therefore, according to the medical image processing device 100 of this embodiment, it is possible to easily determine the change in the positional relationship between the mitral valve 20 and the device 30 due to the periodic reciprocating motion of the mitral valve 20.

[0087] Furthermore, the medical image processing device 100 of this embodiment identifies the device region 60 using three feature points 301 to 303 corresponding to the three endpoints of the device 30, and identifies the centroid of these three feature points 301 to 303 as the point of interest coordinate 650. Alternatively, the medical image processing device 100 may identify the point of interest direction D as the extension direction of the line segment connecting the midpoint 610 of the three-dimensional coordinates 62 and 63 of feature points 302 and 303, which correspond to the two endpoints in the short-side direction of the device 30, and the point of interest coordinate 650, from the point of interest coordinate 650 toward the midpoint 610. Therefore, according to the medical image processing device 100 of this embodiment, the point of interest direction D can be identified stably and with high accuracy by utilizing the shape of the device 30, which is constant over time.

[0088] Furthermore, the medical image processing device 100 of this embodiment may identify the first principal component obtained from principal component analysis of the coordinate information of each pixel included in the device region 60 identified from each of the multiple three-dimensional images 90a to 90d of the mitral valve 20 as the direction of interest D. According to the medical image processing device 100 of this embodiment, even when this method is adopted, the direction of interest D can be stably and accurately identified by utilizing the shape of the device 30 which does not change over time.

[0089] Furthermore, the medical image processing device 100 of this embodiment displays the calculated change amount C on the display 140 along with images showing the maximum and minimum positions of the coordinates of interest 650. Alternatively, the medical image processing device 100 of this embodiment displays a graph showing the time-series change of the calculated change amount C on the display 140. Therefore, the medical image processing device 100 of this embodiment allows physicians and others to visually grasp indicators related to the state of the mitral valve 20.

[0090] (Variation 1) In the embodiment described above, the medical image processing device 100 calculated the change in distance C between time phases between the average coordinate 210 of the mitral valve region 200 in a plurality of three-dimensional images 90a to 90d and the coordinate 650 of interest of the device 30 in the direction of interest D, as an indicator of the state of the mitral valve 20. However, the method for determining the change in the three-dimensional distance between the mitral valve 20 and the device 30 between time phases is not limited to this.

[0091] Figure 28 shows an example of a method for determining the three-dimensional distance between the mitral valve 20 and the device 30 over time, according to Modification 1. In the example shown in Figure 28, the change amount calculation function 157 calculates the distances Ba to Bc between the valve annular surface 230 and the coordinate of interest 651 at each time t0 to t2 from the three-dimensional images 90a to 90c taken at different times t0 to t2. The change amount calculation function 157 then calculates the change in the distance Ba to Bc between each time t0 to t2 as an indicator of the state of the mitral valve 20.

[0092] Furthermore, in this modified example, the target organ region identification function 152 identifies the annular surface 230 from each of the three-dimensional images 90a to 90c taken at different times t0 to t2. The annular surface 230 is used as a reference for the position of the mitral valve 20 in order to determine the three-dimensional distance between the mitral valve 20 and the device 30. The annular surface 230 is an example of a reference plane in this modified example.

[0093] The target organ region identification function 152 may, for example, identify the least squares plane of coordinates corresponding to the annulus of the mitral valve 20 within the mitral valve region 200 as the annular surface 230. The method for identifying the annular surface 230 is not limited to the above method, as long as it is possible to identify a plane that approximates the annulus of the mitral valve 20.

[0094] Furthermore, the periodic reciprocating motion of the mitral valve 20 occurs along the direction normal to the annular surface 230. Therefore, the coordinate identification function 154 of this modified example utilizes this property of the mitral valve 20 to identify the direction normal to the annular surface 230 as the direction of interest D.

[0095] Furthermore, in this modified example, the focus coordinate identification function 154 may identify the focus coordinate 651 using the same method as in the embodiment described above, or it may identify the focus coordinate 651 indicating the three-dimensional position of the device 30 using other methods.

[0096] Thus, the medical image processing device 100 of this modified example identifies the annular surface 230 of the mitral valve 20 as a reference plane and identifies the amount of change in the distance Ba to Bc between the annular surface 230 and the coordinate 651 of interest in the direction of interest D over multiple different time points as an indicator. Even with this method, it is possible to easily determine the change in the positional relationship between the mitral valve 20 and the device 30 due to the periodic reciprocating motion of the mitral valve 20.

[0097] Furthermore, the modified medical image processing device 100 identifies the normal direction of the valve annular surface 230 as the direction of interest D, and identifies the amount of change in the distance Ba to Bc between the valve annular surface 230 and the coordinate of interest 651 in the direction of interest D between multiple different time points as an indicator. In this way, the direction of the periodic operation of the mitral valve 20 and the device 30 can be identified by methods other than determining the longitudinal direction of the device 30.

[0098] (Modification 2) In the embodiments described above, the target organ was explained using a heart valve, particularly the mitral valve 20, as an example, but the target organ is not limited to this. For example, the target organ may be other areas related to heart valves, or any organ that moves regularly other than the heart valve. For example, the target organ may be the diaphragm, eyelids, joints (adduction, abduction, internal rotation, external rotation of the arms, hips, and legs), etc.

[0099] (Variation 3) In the embodiments described above, specific examples of the device 30 that can be attached to the target organ were given as MitraClip® and PASCAL (Edwards Lifesciences), which can be attached to the mitral valve 20. However, the device 30 is not limited to these, and any therapeutic device that can be attached to a target organ that moves back and forth in three-dimensional space and that restrains the reciprocating motion of the target organ is acceptable.

[0100] (Modification 4) In the above-described embodiment, the calculation process for an index related to the state of the mitral valve 20 was performed during the procedure of attaching the device 30 to the mitral valve 20. However, the timing of the execution of this process is not limited to this. For example, the calculation process for an index related to the state of the mitral valve 20 may be performed after the attachment of the device 30 for the purpose of periodically inspecting the state of the mitral valve 20.

[0101] (Variation 5) In the above-described embodiment, the case where there is one device 30 attached to the mitral valve 20 was explained as an example, but there may be two or more devices 30 attached to the mitral valve 20. The tension applied to the mitral valve 20 may change depending on the number of devices 30 attached to the mitral valve 20.

[0102] Furthermore, the size of the device 30 may vary depending on the product. For example, the larger the width and height of the device 30, the greater the tension on the mitral valve 20. Therefore, even if the value of the change amount C is the same, the state of the mitral valve 20 may differ.

[0103] Therefore, the change amount calculation function 157 may correct the calculated index (e.g., change amount C) according to the size or number of devices 30. As a correction method, the change amount C may be multiplied by a predetermined correction value according to the size or number of devices 30.

[0104] (Experimental variation 6) In the embodiment described above, the change amount calculation function 157 calculated a numerical value of change amount C as an indicator of the state of the target organ, but the method for identifying the indicator is not limited to this. For example, identifying levels that show the magnitude of change amount C in stages may be considered as identifying the indicator. Alternatively, the change amount calculation function 157 may express the magnitude of change amount C according to a predetermined standard, such as large, medium, or small.

[0105] The various types of data discussed in this specification are typically digital data.

[0106] According to at least one embodiment described above, when a therapeutic device is attached to a target organ of a subject, an indicator of the condition of the target organ can be calculated with high accuracy during the procedure.

[0107] While several embodiments have been described, these embodiments are presented as examples only and are not intended to limit the scope of the invention. These embodiments can be implemented in a variety of other forms, and various omissions, substitutions, modifications, and combinations of embodiments are possible without departing from the spirit of the invention. These embodiments and their variations are included in the scope and spirit of the invention, as well as in the claims and their equivalents. [Explanation of Symbols]

[0108] 20 Mitral valve 21 Anterior leaflet 22 Posterior tip 30 devices 31 shafts 60 device area 61, 61a~61d, 62, 62a~62d, 63, 63a~63d Three-dimensional coordinates 70 Annular region Coordinates 71, 72, 211~214, 221~224 73 Anterior commissure coordinates 74 Posterior commissure coordinates 75 Valve tip region 76,610 midpoint 90,90a~90d Three-dimensional images 91, 91a~91d Mesh images 92 Segmentation Images 93 images 100 Medical Image Processing Equipment 110 NW Interfaces 120 Memory circuit 130 Input Interfaces 140 displays 150 Processing Circuits 151 Acquisition function 152 Target organ region identification function 153 Device Area Identification Function 154. Function to identify target coordinates 155 Focus Direction Identification Function 156 Average coordinate identification function 157 Change Calculation Function 158 Display Control Function 159 Reception function 200, 200a~200d Mitral valve region 201 Anterior leaflet region 202 Posterior cusp region 210,210a~210g average coordinates 230 Valve annular surface 301-303 Key Features 650, 650a~650d, 651 Coordinates of interest A1,A2,Ba,Bb,Bc distance Change in C D Direction of focus t0~t6 time

Claims

1. An acquisition unit that acquires multiple images of the target organ in chronological order, A target organ region identification unit identifies the target organ region in which the target organ is depicted from each of the aforementioned multiple images, A device region identification unit identifies a device region in which a device attached to the target organ is depicted from each of the aforementioned multiple images, An indicator identification unit identifies an indicator relating to the state of the target organ to which the device is attached, based on the identified target organ region and the device region, from the change in the relative position of the device with respect to the target organ in a specific direction of interest, A medical information processing device equipped with [a specific feature].

2. The aforementioned multiple images are multiple three-dimensional images of the target organ taken in chronological order. A focus coordinate identification unit identifies a focus coordinate, which is the three-dimensional coordinate of a point included in the device region identified by the device region identification unit, from each of the aforementioned multiple images. The system further comprises a focus direction identification unit that identifies the longitudinal axis direction of the device in three-dimensional space as the focus direction based on the device region identified by the device region identification unit from each of the plurality of images, The indicator identification unit calculates the change in distance between the coordinate of interest and the target organ region in the direction of interest as the indicator. The medical information processing device according to claim 1.

3. The acquisition unit acquires the multiple images of the target organ taken in chronological order at multiple different points in time, The system further comprises an average coordinate identification unit that identifies the average coordinates of the target organ at each of the multiple different time points, based on the target organ region identified from each of the multiple images by the target organ region identification unit, The indicator identification unit identifies the maximum position and the minimum position relative to the direction of interest from among the average coordinates at each of the multiple different time points, and calculates the absolute value of the difference between the first distance between the coordinate of interest and the maximum position in the direction of interest, and the second distance between the coordinate of interest and the minimum position in the direction of interest, as the amount of change. The medical information processing device according to claim 2.

4. The aforementioned direction of interest identification unit identifies the first principal component obtained from principal component analysis of the coordinate information of each pixel included in the device region identified from each of the plurality of images as the direction of interest. The medical information processing device according to claim 2.

5. The device region identification unit identifies the device region using three feature points corresponding to the three endpoints of the device. The aforementioned coordinate identification unit identifies the centroid of the three feature points as the coordinate of interest, The aforementioned direction of interest identification unit identifies the direction of interest as the extension direction of the line segment connecting the midpoint of the three-dimensional coordinates of two feature points corresponding to the two endpoints in the short-side direction of the device among the three feature points, and the coordinate of interest, from the coordinate of interest toward the midpoint. The medical information processing device according to claim 2.

6. The system further includes a display control unit that displays the calculated index together with images showing the maximum and minimum positions of the coordinates of interest on the display unit, or displays a graph showing the time-series changes of the calculated index on the display unit. The medical information processing device according to claim 3.

7. The indicator identification unit corrects the indicator according to the size or number of the devices. The medical information processing device according to claim 1.

8. The aforementioned target organ region identification unit identifies a reference plane in the target organ, A focus coordinate identification unit identifies a focus coordinate, which is the three-dimensional coordinate of a point included in the device region identified by the device region identification unit, from each of the aforementioned multiple images. The system further comprises a unit for identifying the direction of interest, which identifies the normal direction of the reference plane as the direction of interest, The indicator identification unit identifies the amount of change in the distance between the reference plane and the coordinate of interest in the direction of interest as an indicator related to the state of the target organ. The medical information processing device according to claim 1.

9. The organ in question is the mitral valve. The device grasps the tips of the anterior and posterior leaflets of the mitral valve, The aforementioned index is a numerical value that indirectly indicates the tension of the mitral valve when it is grasped by the device. A medical information processing device according to any one of claims 1 to 8.

10. The acquisition step involves obtaining multiple images of the target organ taken in chronological order, A target organ region identification step involves identifying the target organ region from each of the aforementioned multiple images in which the target organ is depicted, A device region identification step involves identifying a device region from each of the aforementioned multiple images in which a device attached to the target organ is depicted. An indicator identification step, based on the identified target organ region and the device region, to identify an indicator relating to the state of the target organ to which the device is attached, from the change in the relative position of the device with respect to the target organ in a specific direction of interest; A medical information processing method including [the specified term].

Images