Medical information processing apparatus, method and program

Abstract

To improve consistency between the shape of valve leaflets perceived by a user in a medical image and the shape of valve leaflets extracted from the medical image and / or measurement results of the shape.SOLUTION: A medical information processing apparatus includes an acquisition unit, a measurement unit, a calculation unit, and a correction unit. The acquisition unit acquires a first shape of a heart valve composed of a plurality of valve leaflets. The measurement unit acquires measurement information between the plurality of valve leaflets at a predetermined position of the first shape. The calculation unit calculates at least one of the correction amount and direction for correcting the predetermined position based on the measurement information. The correction unit corrects the predetermined position of the first shape based on at least one of the correction amount and direction.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The embodiments disclosed in this specification and the drawings relate to a medical information processing apparatus, method, and program.

Background Art

[0002] In the medical field, there is a technique for facilitating the understanding of the structure of a heart valve by extracting a region of a heart valve composed of a plurality of valve tips in a three-dimensional image obtained by imaging with various medical imaging diagnostic apparatuses (modalities), and displaying to a user the shape of the valve tip and / or measurement values related to the shape from the region of the heart valve. For example, a technique for extracting a region of a heart valve in a three-dimensional image is known.

Prior Art Documents

Non-Patent Documents

[0003]

Non-Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] One of the problems to be solved by the embodiments disclosed in this specification and the drawings is to improve the consistency between the shape of the valve tip extracted from a medical image and / or the measurement results related to the shape, and the shape of the valve tip recognized by a user in the medical image. However, the problems to be solved by the embodiments disclosed in this specification and the drawings are not limited to the above problems. It is also possible to position, as other problems, the problems corresponding to the respective effects of each configuration shown in the embodiments described later.

Means for Solving the Problem

[0005] The medical information processing apparatus according to the embodiment includes an acquisition unit, a measurement unit, a calculation unit, and a correction unit. The acquisition unit acquires a first shape of a heart valve composed of a plurality of valve leaflets. The measurement unit acquires measurement information between a plurality of valve leaflets at a predetermined position of the first shape. The calculation unit calculates at least one of a correction amount or a direction for correcting the predetermined position based on the measurement information. The correction unit corrects the predetermined position of the first shape based on at least one of the correction amount or the direction.

Brief Description of the Drawings

[0006]

Figure 1

Figure 2

Figure 3

Figure 4A

Figure 4B

Figure 5

Figure 6

Modes for Carrying Out the Invention

[0007] Hereinafter, embodiments of a medical information processing apparatus, method, and program will be described in detail with reference to the drawings. Note that the medical information processing apparatus, method, and program according to the present application are not limited to the embodiments shown below. Although a plurality of features are described in the embodiments, not all of these plurality of features are essential, and the plurality of features may be arbitrarily combined. Further, in the accompanying drawings, the same or similar configurations are denoted by the same reference numerals, and redundant descriptions are omitted.

[0008] As described above, a technique for extracting the region of a heart valve in a three-dimensional image is known. However, for example, due to an error or the like in the extraction of the region of the heart valve, a minute gap is generated between a plurality of valve tips extracted from the three-dimensional image, and there may be a discrepancy between the shape of the displayed valve tip and / or the measurement result regarding the shape and the shape of the valve tip recognized by the user in the three-dimensional image. That is, there may be a discrepancy between the shape of the valve tip obtained by automatic extraction and the shape assumed by the user who observes the three-dimensional image in his / her mind. Therefore, an object of the present application is to provide a medical information processing apparatus, method, and program capable of improving the consistency between the shape of the valve tip extracted from a medical image and / or the measurement result regarding the shape and the shape of the valve tip recognized by the user in the medical image.

[0009] (First Embodiment) The medical information processing apparatus according to the present embodiment extracts a region of a heart valve composed of a plurality of valve tips in a medical image, and corrects the shape of at least one valve tip so that the plurality of valve tips are in contact with each other when the plurality of valve tips extracted from the region of the heart valve are close to each other. Then, the medical information processing apparatus performs display control to cause the display unit to display the corrected shape of the valve tip and / or the measurement result related to the shape. By correcting the shape of the heart valve by the method described in the present embodiment, in the heart valve recognized by the user that the plurality of valve tips are in contact with each other in the medical image, the shape of the valve tip extracted from the medical image and / or the measurement related to the shape The consistency with the shape of the valve tip recognized by the user in the medical image can be improved. Hereinafter, the configuration and processing of the present embodiment will be described with reference to FIG. In the following embodiments, the case where the medical image is a three-dimensional image will be described.

[0010] FIG. 1 is a diagram showing a configuration of a medical information processing system 10 including a medical information processing apparatus 100 according to the first embodiment. The medical information processing system 10 includes, as its functional configuration, a medical information processing apparatus 100, a network 120, and a data server 130. The medical information processing apparatus 100 is communicably connected to the data server 130 via the network 120. The network 120 includes, for example, a LAN (Local Area Network) or a WAN (Wide Area Network). Note that various other devices (for example, a medical image diagnostic apparatus (modality) that captures a medical image) and systems may be connected to the network 120 shown in FIG.

[0011] The data server 130 is an image storage and communication system (PACS: Picture Archiving and Communication Systems) that holds and manages medical images and information associated with the medical images. For example, it stores medical images in a format compliant with DICOM (Digital Imaging and Communications in Medicine). The medical information processing device 100 can acquire medical images held in the data server 130 via the network 120. The data server 130 receives and stores medical images captured by modalities, and transmits the medical images to each device in response to requests from devices connected to the network 120. Further, the data server 130 includes a database capable of storing various data associated with the received medical images together with the medical images.

[0012] In the present embodiment, a CT image captured by an X-ray CT (Computed Tomography) device will be described as an example of a medical image, but other modalities may be used to capture the medical image. Modalities include, in addition to the X-ray CT device, for example, an MRI (Magnetic Resonance Imaging) device, a SPECT (Single Photon Emission Computed Tomography) device, a PET (Positron Emission computed Tomography) device, a transthoracic echocardiogram device, a transesophageal echocardiogram device, an X-ray diagnostic device, etc. The medical information processing device 100 according to the present embodiment is applicable to medical images acquired by various modalities. The medical images referred to here also include three-dimensional images pseudo-reconstructed from a plurality of two-dimensional images.

[0013] The medical information processing device 100 is a device that performs image processing according to this embodiment. Specifically, the medical information processing device 100 receives a medical image from the data server 130 via the network 120, and performs various image processes using the medical image. Further, the medical information processing device 100 is a device that causes a display 150 to display a medical image, the result of the image processing according to the embodiment, etc., and functions as a reading terminal device operated by a user such as a doctor. For example, the medical information processing device 100 is realized by a computer device such as a server or a workstation.

[0014] The medical information processing device 100 includes a communication IF (Interface) 111, a ROM (Read Only Memory) 112, a RAM (Random Access Memory) 113, a storage circuit 114, and a processing circuit 115. Further, the medical information processing device 100 is connected to an input IF (Interface) 140 and a display 150.

[0015] The communication IF 111 is configured by a LAN card or the like, and realizes communication between an external device (for example, the data server 130) and the medical information processing device 100. The ROM 112 is configured by a non-volatile memory or the like, and stores various programs. The RAM 113 is configured by a volatile memory or the like, and temporarily stores various information as data. The storage circuit 114 is configured by an HDD (Hard Disk Drive) or the like, and stores various information as data.

[0016] The processing circuit 115 controls the entire medical information processing device 100. For example, the processing circuit 115 performs various processes according to an input operation received from a user via the input IF 140. Further, for example, the processing circuit 115 controls the transmission and reception of data with an external device via the communication IF 111. Further, for example, the processing circuit 115 controls the display of a medical image or the like on the display 150. Further, the processing circuit 115 performs various image processes on the medical image.

[0017] The input IF 140 receives input operations of various instructions and various information from the user. Specifically, the input IF 140 is connected to the processing circuit 115, converts the input operations received from the user into electrical signals, and transmits them to the processing circuit 115. For example, the input IF 140 is realized by a trackball, a switch button, a mouse, a keyboard, a touch pad that performs an input operation by touching an operation surface, a touch screen in which a display screen and a touch pad are integrated, a non-contact input interface using an optical sensor, a voice input interface, and the like. Note that in this specification, the input IF 140 is not limited to only those having physical operation components such as a mouse and a keyboard. For example, an electrical signal processing circuit that receives an electrical signal corresponding to an input operation from an external input device provided separately from the apparatus and transmits this electrical signal to the processing circuit is also included in the example of the input IF 140.

[0018] For example, a medical image to be processed is input to the medical information processing apparatus 100 according to an instruction of a user who operates the input IF 140. Note that the selection of the medical image may not be based on an instruction of the user. For example, the processing circuit 115 of the medical information processing apparatus 100 may automatically select a medical image to be processed based on a predetermined rule.

[0019] The display 150 displays various information and various data. Specifically, the display 150 is connected to the processing circuit 115 and displays various information and various data received from the processing circuit 115. For example, the display 150 displays a medical image, the shape of a heart valve generated by the processing circuit 115, and the like. The display 150 is realized by, for example, a liquid crystal display, a CRT (Cathode Ray Tube) display, a touch panel, or the like.

[0020] FIG. 2 is a diagram showing a functional configuration of the processing circuit 115. As shown in FIG. 2, the processing circuit 115 includes, as its functional configuration, an input image acquisition function 210, an initial shape acquisition function 220, a measurement information acquisition function 230, a corrected shape acquisition function 240, and a display control function 250. Here, the initial shape acquisition function 220 is an example of an acquisition unit. The measurement information acquisition function 230 is an example of a measurement unit. The corrected shape acquisition function 240 is an example of a calculation unit and a correction unit.

[0021] The input image acquisition function 210 acquires an input image to be processed from the data server 130 via the communication IF 111 and the network 120. Here, in the present embodiment, the input image is a three-dimensional CT image including a heart valve. That is, the input image acquisition function 210 acquires a three-dimensional CT image (volume data) including a heart valve from the data server 130 via the communication IF 111. Note that the input image acquisition function 210 can also acquire a plurality of volume data obtained by imaging in three dimensions in the time direction. The processing by the input image acquisition function 210 will be described in detail later.

[0022] The initial shape acquisition function 220 acquires a first shape of the heart valve composed of a plurality of valve tips. Specifically, the initial shape acquisition function 220 acquires the first shape from a medical image of the heart valve. For example, the initial shape acquisition function 220 acquires an initial shape of the heart valve composed of a plurality of valve tips from the input image (three-dimensional CT image). The processing by the initial shape acquisition function 220 will be described in detail later.

[0023] The measurement information acquisition function 230 acquires measurement information between a plurality of valve tips at a predetermined position of the first shape. Specifically, the measurement information acquisition function 230 acquires measurement information between a plurality of valve tips from the initial shape of the heart valve. Here, the measurement information is, for example, a distance at a predetermined position between a plurality of valve tips or an area of a region formed by a predetermined position of a plurality of valve tips. The processing by the measurement information acquisition function 230 will be described in detail later.

[0024] The correction shape acquisition function 240 calculates at least one of a correction amount or a direction for correcting a predetermined position based on the measurement information. Then, the correction shape acquisition function 240 corrects the predetermined position of the first shape based on at least one of the correction amount or the direction. Specifically, the correction shape acquisition function 240 acquires a second shape obtained by correcting the predetermined position of the first shape in the medical image. That is, the correction shape acquisition function 240 acquires a correction shape obtained by correcting the initial shape based on the measurement information. The processing by the correction shape acquisition function 240 will be described in detail later.

[0025] The display control function 250 generates an image to be displayed on the display 150 and the shape of the heart valve, and performs display control for displaying on the display 150. The processing by the display control function 250 will be described in detail later.

[0026] The above-described processing circuit 115 is realized by, for example, a processor. In that case, each of the above-described processing functions is stored in the ROM 112 or the storage circuit 114 in the form of a program executable by a computer. Then, the processing circuit 115 reads and executes each program stored in the ROM 112 or the storage circuit 114 using the RAM 113 as a work area, thereby realizing the functions corresponding to the respective programs. In other words, the processing circuit 115 has each of the processing functions shown in FIG. 2 when each program is read.

[0027] Note that the processing circuit 115 may be configured by combining a plurality of independent processors, and each processing function may be realized by each processor executing a program. Further, each processing function included in the processing circuit 115 may be appropriately distributed or integrated into a single or a plurality of processing circuits and realized. Further, each processing function included in the processing circuit 115 may be realized by a combination of hardware such as a circuit and software. Here, an example in which a program corresponding to each processing function is stored in a single ROM 112 or the storage circuit 114 has been described, but the embodiment is not limited to this. For example, programs corresponding to each processing function may be distributed and stored in a plurality of ROMs 112 and storage circuits 114, and the processing circuit 115 may be configured to read and execute each program from the plurality of ROMs 112 and storage circuits 114. Note that a part of each processing function included in the processing circuit 115 may be realized by a cloud computer connected to the medical information processing apparatus 100 via the network 120. For example, an arithmetic device located at a location different from the medical information processing apparatus 100 may be communicably connected to the medical information processing apparatus 100 via the network 120, and the medical information processing apparatus 100 and the arithmetic device may transmit and receive data, whereby the functions of the components of the medical information processing apparatus 100 or the processing circuit 115 may be realized.

[0028] Next, after explaining the processing procedure by the medical information processing apparatus 100 with reference to FIG. 3, details of each process will be described. FIG. 3 is a flowchart showing an example of the overall processing procedure of the medical information processing apparatus 100 according to the first embodiment. In the present embodiment, a process of obtaining a corrected shape obtained by correcting the shape of the mitral valve will be described by taking as an example a CT image in which the mitral valve of a subject is depicted. However, the present embodiment is applicable to images obtained by other modalities and other heart valves (aortic valve and tricuspid valve). Further, the present invention is not necessarily limited to heart valves, and is also applicable to a site formed of a plurality of planar structures such as a fossa ovalis formed by the primary septum and the secondary septum of the heart.

[0029] For example, as shown in FIG. 3, in this embodiment, the input image acquisition function 210 acquires an input image (3D CT image) of the subject from the data server 130 (step S310). For example, the input image acquisition function 210 acquires a 3D CT image including the morphological information of the anatomical structure of the mitral valve in response to the acquisition operation of the 3D CT image via the input IF140. This process is realized, for example, by the processing circuit 115 calling and executing a program corresponding to the input image acquisition function 210 from the ROM 112 or the storage circuit 114.

[0030] Subsequently, the initial shape acquisition function 220 acquires the initial shape of the mitral valve from the acquired 3D CT image (step S320). This process is realized, for example, by the processing circuit 115 calling and executing a program corresponding to the initial shape acquisition function 220 from the ROM 112 or the storage circuit 114.

[0031] Subsequently, the measurement information acquisition function 230 acquires measurement information between a plurality of valve tips based on the initial shape (step S330). This process is realized, for example, by the processing circuit 115 calling and executing a program corresponding to the measurement information acquisition function 230 from the ROM 112 or the storage circuit 114.

[0032] Subsequently, the corrected shape acquisition function 240 acquires a corrected shape obtained by correcting the initial shape of the mitral valve based on the measurement information (step S340). This process is realized, for example, by the processing circuit 115 calling and executing a program corresponding to the corrected shape acquisition function 240 from the ROM 112 or the storage circuit 114.

[0033] Subsequently, the display control function 250 causes the display 150 to display an image showing the corrected shape of the mitral valve (step S350). This process is realized, for example, by the processing circuit 115 calling and executing a program corresponding to the display control function 250 from the ROM 112 or the storage circuit 114.

[0034] Details of each process executed by the medical information processing apparatus 100 will be described below.

[0035] (Input Image Acquisition Process) As described in step S310 of FIG. 3, when the user instructs to acquire a medical image via the input IF140, the input image acquisition function 210 acquires the medical image specified by the user from the data server 130 as the input image. Then, the input image acquisition function 210 outputs the acquired input image to the initial shape acquisition function 220 and the display control function 250.

[0036] Note that the input image acquisition process in step S310 may be started by the user's instruction via the input IF140 as described above, or may be automatically started. In such a case, for example, the input image acquisition function 210 monitors the data server 130 and automatically acquires the medical image every time a new medical image is stored.

[0037] Here, the input image acquisition function 210 may determine the newly stored medical image based on a preset acquisition condition, and execute the acquisition process when the medical image satisfies the acquisition condition. For example, the acquisition condition capable of determining the state of the medical image is stored in the storage circuit 114, and the input image acquisition function 210 determines the newly stored volume data based on the acquisition condition stored in the storage circuit 114.

[0038] For example, the storage circuit 114 stores, as the acquisition condition, "acquire a medical image captured by an imaging protocol targeting the heart valve", "acquire an enlarged and reconstructed medical image", or a combination thereof. The input image acquisition function 210 acquires the volume data that satisfies the above-described acquisition condition.

[0039] (Initial Shape Acquisition Process) As described in step S320 of FIG. 3, the initial shape acquisition function 220 acquires the initial shape of the mitral valve from the input image (3D CT image) acquired in step S310. Specifically, the initial shape acquisition function 220 acquires the coordinate information of all or part of the pixels indicating the mitral valve in the 3D CT image. Here, for the acquisition of the initial shape by the initial shape acquisition function 220, any method may be used as long as it is a method of extracting the mitral valve region based on the pixels at the positions corresponding to the mitral valve region on the image.

[0040] For example, the initial shape acquisition function 220 can acquire the region specified by the user via the input IF140 as the mitral valve region. Also, for example, the initial shape acquisition function 220 can acquire the mitral valve region based on the information on the anatomical structure of the mitral valve depicted in the 3D CT image by using known segmentation processing. Note that examples of the above segmentation processing include Otsu's binarization method based on CT values, region growing method, snake method, graph cut method, mean shift method, and the like.

[0041] Also, for example, the initial shape acquisition function 220 may identify the mitral valve region by using a learned model of the target region (mitral valve region) constructed based on the learning data prepared in advance using machine learning techniques (including deep learning). In addition, the initial shape of the mitral valve may be acquired using the method described in Non-Patent Document 1. Also, when acquiring the initial shape of the heart valve, it may be acquired separately for each valve tip. For example, since the mitral valve is composed of two valve tips, the anterior tip and the posterior tip, the initial shape may be acquired separately for each.

[0042] In addition, the initial shape acquisition function 220 can also acquire, as the initial shape of the mitral valve, the information indicating the above-described mitral valve region as a point cloud. For example, the initial shape acquisition function 220 can also acquire, as the initial shape of the mitral valve, the mesh information representing the above-described mitral valve region as a mesh. FIGS. 4A and 4B are diagrams schematically illustrating the mesh information of the mitral valve according to the first embodiment. For example, as shown in FIG. 4A, the initial shape acquisition function 220 acquires, as the initial shape of the mitral valve in the input image, the mesh information constituted by the three-dimensional coordinates of the lattice points of 19×9 points for the shape of the anterior cusp 400 of the mitral valve, and the mesh information constituted by the three-dimensional coordinates of the lattice points of 25×9 points for the shape of the posterior cusp 410.

[0043] Here, indexes are assigned to the lattice points in the mesh information, and the three-dimensional coordinates of a predetermined lattice point can be specified by designating the index. The index of the lattice point is a label capable of identifying each lattice point, and for example, numerical values or symbols can be used.

[0044] For example, the mesh information shown in FIG. 4 can be given the identifiers shown in FIG. 4B. That is, as shown in FIG. 4B, for the mesh information represented by the lattice point group of 19 columns and 9 rows for the anterior cusp and the lattice point group of 25 columns and 9 rows for the posterior cusp, with one end in the row direction at the origin at the boundary between the anterior cusp and the posterior cusp, the row direction coordinate being "x" and the column direction coordinate being "y", identifiers (x, y) can be assigned to each lattice point. In this case, the identifier (8, 0) indicates the anterior commissure, and the identifier (8, 18) indicates the posterior commissure. Also, the outermost part located at the anterior cusp and the posterior cusp (the position where the x coordinate is "0" in FIG. 4B) is referred to as the annulus part. Also, the innermost part located at the anterior cusp and the posterior cusp (the position where the x coordinate is "8" in FIG. 4B) is referred to as the valve tip part.

[0045] Note that the mesh information shown in FIGS. 4A and 4B is merely an example, and the specific configuration (the number of lattice points, arrangement, array, etc.) of the lattice point group data is not particularly limited and may be changed as appropriate.

[0046] When the initial shape is taken as mesh information consisting of a predetermined number of grid points, the initial shape acquisition function 220 can estimate and acquire the mesh information from the input image by a known image analysis technique. For example, as an example of a method based on machine learning, the initial shape acquisition function 220 can estimate and acquire the initial shape (mesh information) of the mitral valve from the input image using DenseNet, which is a type of CNN (Convolutional Neural Network). Alternatively, the initial shape acquisition function 220 can also estimate the shape of the mitral valve specific to the subject using a statistical shape model (mesh information) of the mitral valve.

[0047] Also, as described above, the initial shape acquisition function 220 can also acquire the mesh information by extracting the mitral valve region from the input image by a known image processing technique such as the region expansion method and converting the extracted mitral valve region into a mesh. Further, the user may manually create the mesh information using a tool (not shown) via the input IF140, or the configuration may be such that the mesh information stored in advance in the data server is read out and acquired.

[0048] When the initial shape of the mitral valve is acquired as described above, the initial shape acquisition function 220 outputs the acquired initial shape to the measurement information acquisition function 230 and the corrected shape acquisition function 240.

[0049] (Acquisition process of measurement information) As described in step S330 of FIG. 3, the measurement information acquisition function 230 acquires the measurement information between a plurality of valve tips based on the initial shape acquired in step S320. Here, the measurement information acquisition function 230 can acquire the measurement information based on the initial shape acquired from the mitral valve region or the mesh information representing the mitral valve region as a mesh. In this embodiment, as an example of the measurement information, the case of acquiring the distance between the anterior tip and the posterior tip at each grid point of the mesh information of the mitral valve will be described.

[0050] FIG. 5 is a diagram schematically explaining an example of measurement information according to the first embodiment. Here, FIG. 5 is a diagram showing the shape of the mitral valve shown in FIG. 4A on an arbitrary plane (YZ plane) passing through an arbitrary lattice point 501 of the anterior cusp included in the input image (3D CT image). That is, FIG. 5 shows the shape of the mitral valve on the YZ plane passing through an arbitrary lattice point 501 of the anterior cusp 400 in FIG. 4A.

[0051] For example, in FIG. 5, an arbitrary lattice point 501 of the anterior cusp, the anterior cusp 500 which is the region of the anterior cusp 400 on the YZ plane 520 (the region of the anterior cusp in the mesh information), and the posterior cusp 510 which is the region of the posterior cusp 410 on the YZ plane 520 (the region of the posterior cusp in the mesh information) are shown. Also, in FIG. 5, a region 530 which is the region of the anterior cusp of the YZ plane (the region formed by all the pixels showing the anterior cusp on the CT image) and the region of the posterior cusp of the YZ plane (the region formed by all the pixels showing the posterior cusp on the CT image) are shown. Since the anterior cusp 500 represents the region when the mitral valve is represented by a mesh (points and lines), as shown in FIG. 5, it has a different thickness from the region 530. Similarly, since the posterior cusp 510 represents the region when the mitral valve is represented by a mesh, as shown in FIG. 5, it has a different thickness from the region 540.

[0052] The measurement information acquisition function 230 calculates, as measurement information, the distance between the anterior cusp 500 and the posterior cusp 510 shown in FIG. 5. For example, as shown by the broken line 550 in FIG. 5, the measurement information acquisition function 230 calculates the shortest distance from the lattice point 501 on the anterior cusp 500 to the posterior cusp 510 on the YZ plane 520. That is, the measurement information acquisition function 230 calculates the distance from the lattice point 501 on the anterior cusp 500 to the position 511 on the posterior cusp 510.

[0053] The measurement information acquisition function 230 calculates the distance (the shortest distance) to the posterior cusp in the same manner as above for all the lattice points constituting the anterior cusp 400. Further, the measurement information acquisition function 230 calculates the distance (the shortest distance) to the anterior cusp in the same manner as above for all the lattice points of the posterior cusp 410. Then, the measurement information acquisition function 230 outputs the acquired measurement information to the corrected shape acquisition function 240.

[0054] In addition, in FIG. 5, for the sake of simplicity of explanation, the case where the shortest distance on the YZ plane 520 is calculated as measurement information has been described, but this is merely an example, and the embodiment is not limited thereto. For example, when calculating the distance from the lattice point 501 on the anterior cusp to the posterior cusp, the shortest three-dimensional distance based on the three-dimensional shape of the posterior cusp may be calculated. That is, the measurement information acquisition function 230 can also identify the position on the posterior cusp closest to the lattice point 501 on the anterior cusp and calculate the distance from the lattice point 501 to the said position.

[0055] In addition, the measurement information acquisition function 230 can also identify, for example, the lattice point of the posterior cusp closest to the lattice point 501 and calculate the distance between the lattice point 501 and the identified lattice point. Further, the measurement information acquisition function 230 can also calculate, for example, the distance from the lattice point 501 to the line segment connecting the lattice point on the posterior cusp identified by an arbitrary or predetermined method and the adjacent lattice point on the posterior cusp. Further, the measurement information acquisition function 230 can also calculate, for example, the distance from the lattice point 501 to the plane formed by three or more lattice points identified by an arbitrary or predetermined method on the posterior cusp.

[0056] Note that the lattice points for calculating the measurement information may be not all lattice points but some predetermined lattice points. For example, when acquiring measurement information, it may be limited to the lattice points near the valve orifice or at the valve tip where the distance between the anterior cusp and the posterior cusp tends to be short. Thereby, the process of calculating the measurement information can be speeded up.

[0057] In addition, the measurement information does not have to be a distance as long as it represents the positional relationship between a plurality of valve cusps. For example, it may be the area or perimeter at a predetermined position between a plurality of valve cusps. In such a case, the measurement information acquisition function 230 calculates, for example, the area or perimeter of the plane (triangular plane) formed by any one lattice point of the anterior cusp and any two lattice points of the posterior cusp as the measurement information. Here, the smaller the calculated area or perimeter is, the closer the lattice point of the anterior cusp is to the posterior cusp in the positional relationship.

[0058] In the above example, the case of calculating the distance from a lattice point of one of the anterior and posterior leaflets to an arbitrary position of the other leaflet as measurement information was described. However, the embodiment is not limited to this, and the distance from an arbitrary position (a point on a line segment or a plane) of one of the anterior and posterior leaflets to an arbitrary position of the other leaflet may be calculated as measurement information. That is, when calculating the measurement information, it is not necessary to use the lattice points in the mesh information.

[0059] In addition, in the processing procedure of the medical information processing apparatus 100, the processing may proceed to step S340 after the acquisition of the measurement information is completed at each predetermined lattice point (all lattice points, or a predetermined part of the lattice points), or the acquisition of the measurement information may be terminated when one or more lattice points in a positional relationship where the measurement information is less than a predetermined threshold value are detected, and the processing may proceed to step S340. Also, in the processing procedure of the medical information processing apparatus 100, when there are no lattice points in a positional relationship where the measurement information is less than a predetermined threshold value, the correction processing in step S340 may be skipped and the processing may proceed to step S350.

[0060] Here, a predetermined value may be used as the threshold value for comparison with the measurement information. Alternatively, the threshold value may be determined based on the initial shape of the mitral valve acquired in S320. In this case, for example, the threshold value may be determined based on the thickness of the leaflet tip calculated from the initial shape. The thickness of the leaflet tip is calculated, for example, from the mitral valve region (the region formed by the pixels indicating the mitral valve) when acquiring the initial shape. For example, the threshold value may be set such that it increases as the thickness of the mitral valve increases, or the threshold value may be set such that it decreases as the thickness of the mitral valve increases.

[0061] Also, the threshold value may vary depending on the position on the leaflet tip. For example, the threshold value in the vicinity of the commissure where the distance between the leaflet tips is close may be set to be smaller than the threshold values at other positions. Also, a user may be able to set an arbitrary threshold value using a GUI (not shown).

[0062] In this embodiment, a method for obtaining measurement information has been described using the geometric positional relationships with line segments and planes represented by pairs of them based on the positions of lattice points that constitute the mesh information forming the anterior tip and the posterior tip. However, the embodiment is not limited to this. For example, based on the mesh information, the positions where each of the anterior tip and the posterior tip exists may be represented in the form of a label image, and measurement information may be obtained based on the amount of background pixels in the label image. More specifically, a label image is generated based on the mesh information, where the pixel value (label) at the position where the anterior tip exists is 1, the pixel value at the position where the posterior tip exists is 2, and the pixel value at other positions is 0 as the background. Then, measurement information on the anterior tip and the posterior tip can be obtained based on the number of pixels of the background label existing between the pixels of the label of the anterior tip and the pixels of the label of the posterior tip.

[0063] (Acquisition Process of Corrected Shape) As described in step S340 of FIG. 3, the corrected shape acquisition function 240 acquires a corrected shape obtained by correcting the initial shape based on the measurement information acquired in step S330. Specifically, the corrected shape acquisition function 240 compares each measurement information acquired in step S330 with a threshold value, and when the measurement information is less than the threshold value, it executes a correction process. For example, the corrected shape acquisition function 240 corrects the initial shape indicated by a point cloud (for example, mesh information) based on the measurement information to acquire a corrected shape indicated by the point cloud (for example, mesh information). Then, the acquired corrected shape is output to the display control function 250.

[0064] For example, in FIG. 5, when the distance (the length of the dashed line 550) between the lattice point 501 of the anterior tip 500 and the position 511 on the posterior tip at the shortest distance from the lattice point 501 is less than the threshold value, the corrected shape acquisition function 240 corrects the shape of the anterior tip 500 by moving the lattice point 501 in the direction of the position 511. That is, the corrected shape acquisition function 240 performs a correction that reduces the distance between a plurality of valve tips (for example, a correction that makes the distance zero).

[0065] Similarly, the corrected shape acquisition function 240 executes a correction process of moving a lattice point in the direction of the position on the corresponding posterior cusp for lattice points where the measurement information (shortest distance) is less than the threshold value at the anterior cusp. Also, the corrected shape acquisition function 240 similarly executes a correction process of moving a lattice point in the direction of the position on the corresponding anterior cusp for lattice points where the measurement information (shortest distance) is less than the threshold value at the posterior cusp. In this way, the corrected shape acquisition function 240 acquires a corrected shape by correcting the initial shape based on the correction amount (shortest distance) and direction (shortest direction) calculated by the measurement information acquisition function 230.

[0066] Note that the direction in which the lattice point is moved is not limited to the shortest direction described above, and for example, the normal direction of the lattice point may be used. For example, the corrected shape acquisition function 240 executes a correction process of moving the lattice point 501 by the correction amount (shortest distance) calculated by the measurement information acquisition function 230 in the direction approaching the posterior cusp in the normal direction of the lattice point 501. That is, the corrected shape acquisition function 240 can also perform a correction process of correcting the initial shape by using only the correction amount (shortest distance) from the measurement information and using a direction not based on the measurement information for the correction direction.

[0067] In the above example, the case where only lattice points with measurement information less than the threshold value are corrected has been described, but the embodiment is not limited to this, and lattice points other than lattice points with measurement information less than the threshold value may be corrected together with lattice points with measurement information less than the threshold value. For example, the corrected shape acquisition function 240 can execute a correction process of moving each lattice point with all lattice points involved in the calculation of the measurement information as correction targets.

[0068] Also, for example, the corrected shape acquisition function 240 can also use lattice points not involved in the calculation of the measurement information as correction targets. For example, the corrected shape acquisition function 240 can execute a correction process of moving each lattice point with lattice points existing within an arbitrary distance from the lattice point 501 as correction targets. Note that the arbitrary distance can be set as appropriate.

[0069] Here, the correction method and correction amount for each grid point may be determined according to the distance from the grid point where the measurement information is less than the threshold value (for example, grid point 501). For example, when the movement amount of grid point 501 is "d", the movement amount of the grid points existing within an arbitrary distance from grid point 501 is set to "wd" (where w is 0 or more and less than 1). Here, "w" may be set based on the distance from grid point 501. For example, "w" is set to be inversely proportional to the distance from grid point 501, and the value becomes smaller as the distance from grid point 501 increases.

[0070] Alternatively, "w" may be set according to the positional relationship with grid point 501. For example, the grid points in the 4-neighborhood positional relationship with grid point 501 may be set to "w = 0.5", and the grid points among the grid points in the 8-neighborhood positional relationship with grid point 501 that are not in the 4-neighborhood positional relationship may be set to "w = 0.25" for the movement amount. Note that the grid points in the above-mentioned 4-neighborhood positional relationship are the four grid points above, below, left, and right of a predetermined grid point (grid point 501), and the grid points among the grid points in the 8-neighborhood positional relationship that are not in the 4-neighborhood positional relationship are the four grid points in the diagonal positions of a predetermined grid point (grid point 501).

[0071] The correction shape acquisition function 240 acquires the correction shape by moving each surrounding grid point by the set movement amount in the same direction as the movement direction of grid point 501. Thereby, it is possible to suppress the steep shape change caused by moving only one grid point. Note that the movement amount "d" of the grid point may be the same value as the distance between grid point 501 and position 511, or may be a different value. The movement amount "d" may be set to a smaller "d'" (for example, 0.5×d) than the distance between grid point 501 and position 511, and the movement amount of each surrounding grid point may be set to "wd'". According to this, it is possible to suppress the steep shape change caused by the large movement of the grid point in one correction process.

[0072] Note that the above-described movement amount "d" may be the same as the threshold value compared with the measurement information, or may be a value obtained by multiplying the threshold value by a predetermined value. That is, the correction shape acquisition function 240 performs a correction process for correcting the initial shape by using only the correction direction (the shortest direction) from the measurement information and using a movement amount not based on the measurement information for the correction amount.

[0073] Here, when the shape is corrected by the above-described method and the shapes of the front tip 400 and the rear tip 410 change, the measurement information at a predetermined position may become larger than the threshold value. Therefore, the process of acquiring the measurement information in step S330 may be executed again with the first corrected shape obtained by correcting the shape by the above-described method as the initial shape. Then, in step S340, a second corrected shape obtained by further correcting the first corrected shape based on the newly acquired measurement information may be obtained. That is, steps S330 and S340 may be configured to be repeatedly executed so that the measurement information is within the threshold value at any position in step S330. Thereby, the front tip and the rear tip with measurement information less than the threshold value can be obtained at any position. Alternatively, the upper limit number of times of repeatedly executing steps S330 and S340 may be set in advance, and when the upper limit number of times is reached, the correction process may be terminated and the process may proceed to step S350. When steps S330 and S340 are repeatedly executed, the lattice points used for acquiring the measurement information in step S330 may be limited to the lattice points that have moved in step S340 and the surrounding lattice points. Thereby, the processing can be speeded up.

[0074] In order to suppress the distortion of the overall shape of the mitral valve due to the above-described correction, the correction shape acquisition function 240 can move the positions of a plurality of lattice points so that the anatomical measurement values of the mitral valve do not change. For example, when the lattice points to be corrected are near the valve orifice, the correction shape acquisition function 240 moves the corresponding lattice points near the valve annulus by approximately the same amount to obtain a correction shape such that the valve length does not change before and after correction. Note that the correction shape acquisition function 240 can obtain a correction shape in which the shape is corrected so that not only the valve length but also the valve annulus length, the valve orifice length, the valve orifice area, and the valve tip area do not change. For the same reason, the correction shape acquisition function 240 can also correct the positions of the lattice points so that the positions of anatomical sites (for example, the commissure located at the boundary between a plurality of valve tips) do not change.

[0075] In the above-described embodiment, the case where the measurement information and the threshold value are compared for all the lattice points for which the measurement information has been obtained, and the lattice points whose measurement information is less than the threshold value are corrected has been described. However, the embodiment is not limited to this, and it may also be the case where the lattice points to be processed are selected from among a plurality of lattice points. For example, when the measurement information at two or more lattice points is obtained in step S330, one lattice point may be selected from among them and the above-described correction process may be executed for the selected one lattice point, or the correction process may be executed for any number of lattice points among the lattice points for which the measurement information has been obtained in step S330.

[0076] For example, when the correction shape acquisition function 240 obtains the measurement information at two or more lattice points in step S330, it can determine which lattice point (and the order thereof) to correct according to a predetermined condition (such as the order based on the index of the lattice point defined by the mesh information), and can sequentially correct each of the determined lattice points, or can correct them in order from the lattice points with smaller measurement information.

[0077] Note that the order of performing the correction is not limited to the above example, and it may be determined by other methods. For example, in the initial shape obtained in step S320, the correction may be performed in order from the lattice point with the lowest reliability. That is, the measurement information acquisition function 230 determines a predetermined position (lattice point) in the first shape based on the reliability of the point cloud. The corrected shape acquisition function 240 performs correction with the predetermined position (lattice point) determined by the measurement information acquisition function 230 as the correction target.

[0078] Here, as an example of the method for calculating the reliability, the smoothness of the shape can be utilized. For example, since the mitral valve and other heart valves are locally smooth curved surfaces, lattice points whose positions change steeply compared to surrounding lattice points can be regarded as having low reliability. Therefore, the measurement information acquisition function 230 determines lattice points whose positions change steeply compared to surrounding lattice points as lattice points to be preferentially corrected. Thereby, while eliminating minute gaps between valve tips, the positions of lattice points with low reliability can be corrected, and unnatural deformation of the corrected shape can be suppressed.

[0079] In the above-described embodiment, the case where distance is used as the measurement information has been described. However, the embodiment is not limited to this, and the case where area is used as the measurement information may also be applicable. As described above, the measurement information acquisition function 230 can calculate, for example, the area or perimeter of a surface (triangular surface) formed by any one lattice point of the anterior cusp and any two lattice points of the posterior cusp as the measurement information. In this case, when the measurement information (area or perimeter) is less than the threshold value, the corrected shape acquisition function 240 performs correction to move the lattice points forming the surface.

[0080] Here, the corrected shape acquisition function 240 performs correction to reduce the area of the region (triangular surface) formed by a plurality of valve tips (correction to approximate the area to 0 or correction to make the area 0). That is, the corrected shape acquisition function 240 moves the three lattice points forming the triangular surface so that the area of the triangular surface becomes smaller. For example, the corrected shape acquisition function 240 performs correction to move all of the three lattice points forming the triangular surface, or one or two of the three lattice points.

[0081] At this time, the correction shape acquisition function 240 can also perform a determination process for determining which of the three lattice points to move. For example, when the correction shape acquisition function 240 performs a correction to make the area zero, it can identify the lattice point with the smallest movement amount and set the identified lattice point as the target of movement. Further, the correction shape acquisition function 240 can also set all the lattice points as the targets of movement and move all the lattice points by the same amount as much as possible.

[0082] In the above example, the case where the area or perimeter of the surface formed by the three lattice points is used as the measurement information has been described. However, the embodiment is not limited to this, and the area or perimeter of the surface formed by four or more lattice points may be used as the measurement information. In such a case, since it is often the case that not all of the four or more lattice points are on one surface, the measurement information acquisition function 230 identifies one surface by performing approximation processing using the least squares method or the like on the four or more lattice points, and calculates the area or perimeter of the identified surface as the measurement information.

[0083] (Result display process) As described in step S350 of FIG. 3, the display control function 250 performs control to display the input image (CT image) acquired in step S310 and the correction shape acquired in step S340 on the display 150. Specifically, the display control function 250 generates an image to be displayed on the display 150 based on the input image and the correction shape, and performs control to display the generated image on the display 150.

[0084] For example, the display control function 250 performs control to generate an image obtained by CG-rendering mesh information, which is the corrected shape of the mitral valve, and display it on the display 150. Also, the display control function 250 has the same functions as a general medical image viewer. In response to user input, it selects a 2D slice image from the input image, and for the selected slice image, it can also perform control to generate an image in which the corrected shape of the mitral valve at the position corresponding to the slice image is superimposed and display it on the display 150. Further, the display control function 250 can also perform control to display on the display 150 measured values (such as valve orifice area, etc.) measured from the corrected shape.

[0085] Note that the display control function 250 can also perform control to save the corrected shape of the mitral valve calculated in the process of step S340 in the data server 130 via the storage circuit 114 of the medical information processing apparatus 100 or the network 120. Thereby, it is possible to display the projection image on any other medical image viewer, or load and use it in any other surgical support software, etc. In this case, the display of the image in step S350 does not necessarily have to be performed. Also, the display and storage of the corrected shape are not necessarily required, and a configuration may be adopted in which an analysis process using the acquired corrected shape is performed. For example, a configuration may be adopted in which measured values related to the shape of the valve (valve length, valve orifice area) and attribute values related to the state of the valve (deviation of the valve, insufficiency of closure) are calculated or estimated based on the corrected shape and then displayed or stored.

[0086] According to the present embodiment, in the heart valve that can be visually recognized when the user touches it, it is possible to improve the consistency between the shape of the valve tip extracted from the medical image and / or the measurement result related to the shape, and the shape of the valve tip recognized by the user in the medical image.

[0087] (Modification Example 1: Variations of the Input Image) In the above-described embodiment, an example in which a three-dimensional CT image is used as the input image has been described, but the embodiment is not limited thereto. For example, the input image may be a three-dimensional ultrasonic image (such as a transthoracic ultrasound or a transesophageal ultrasound), an MRI image, a PET image, a SPECT image, or the like. Further, a time-series image may be used as the input image. For example, it may be a time-series three-dimensional CT image (4D CT image). In this case, the above-described respective processes can be executed for the images of each time phase of the time-series three-dimensional CT image to obtain the corrected shape of the mitral valve corresponding to each time phase. Then, the result can be displayed as an animation.

[0088] Further, the input image does not necessarily have to be a three-dimensional image. For example, it may be a two-dimensional image such as a long-axis image or a short-axis image of the mitral valve.

[0089] Also, when the input image is a time-series image, by utilizing the fact that the cardiac valve continuously displaces with time, the discontinuity of the position compared with the previous and subsequent time phases may be utilized for the reliability of the grid points calculated in step S340. For example, for the grid points of a predetermined time phase, by comparing the coordinates of the grid points of the same index of the surrounding time phases (for example, the previous and subsequent time phases) and the coordinates obtained by time-averaging them, if the coordinates of the grid points of the predetermined time phase are far from any of the coordinates, the grid points (the grid points of the predetermined time phase) may be at a position deviated compared with the grid points of other time phases.

[0090] To avoid this, for example, a method can be considered in which a value inversely proportional to the sum of the distances between the coordinates of the grid points of the same index of the surrounding time phases or the coordinates obtained by time-averaging them and the coordinates of the grid points of the predetermined time phase is used as the reliability of the grid points of the predetermined time phase. For example, the correction shape acquisition function 240 performs correction processing based on the reliability calculated by the above-described calculation for each grid point of a predetermined time phase.

[0091] Also, when correcting the lattice points, the correction may be performed by moving the lattice points to the average position of the lattice points at the same index in the surrounding time phases. In such a case, for example, the measurement information acquisition function 230 calculates the measurement information before moving the lattice points at a predetermined time phase to the average position of the lattice points at the same index in the surrounding time phases, and the measurement information after moving the lattice points at a predetermined time phase to the average position, respectively. Then, when the measurement information after the movement is smaller than the measurement information before the movement, the correction shape acquisition function 240 executes a correction to move the lattice points at a predetermined time phase to the average position of the lattice points at the same index in the surrounding time phases, thereby obtaining the correction shape.

[0092] Also, the direction for correcting the lattice points may be determined based on two directions: the direction when moving the lattice points to the average position of the lattice points at the same index in the surrounding time phases and the direction in which the measurement information becomes smaller. In such a case, for example, the correction shape acquisition function 240 determines, as the correction direction, the direction obtained by taking the weighted average of the direction when moving the lattice points at a predetermined time phase to the average position of the lattice points at the same index in the surrounding time phases and the direction in which the measurement information becomes smaller. The weighting can be set as appropriate. In this way, by performing the correction using the average position of the lattice points at the same index in the surrounding time phases, the position of the lattice points at the deviated position can be corrected, and a more plausible correction shape can be obtained as the shape of the actual valve tip.

[0093] The correction using the average position of the lattice points at the same index in the surrounding time phases is not limited to the above example, and it may be performed by other methods. For example, after specifying the correction amount and the correction direction based on the information between the time phases, correcting the lattice points according to the specified correction amount and correction direction to obtain a first correction shape, then specifying the correction amount and the correction direction based on the measurement information in the first correction shape, and obtaining a second correction shape by correcting the lattice points of the first correction shape according to the specified correction amount and correction direction.

[0094] That is, the corrected shape acquisition function 240 determines lattice points included in the image of a predetermined time phase and lattice points of the same index included in the images of surrounding time phases in the input images of multiple time phases (for example, 4D CT images) acquired by the input image acquisition function 210. Then, the corrected shape acquisition function 240 calculates the direction and the amount of movement when moving the lattice points of the predetermined time phase to the average position of the lattice points of the same index in the surrounding time phases, and performs correction to move the lattice points in the image of the predetermined time phase according to the calculated direction and amount of movement, thereby acquiring the first corrected shape of the heart valve in the predetermined time phase.

[0095] The measurement information acquisition function 230 calculates measurement information for each lattice point in the first corrected shape of the heart valve in a predetermined time phase. The corrected shape acquisition function 240 compares the calculated measurement information with a threshold value, and executes correction processing for lattice points where the measurement information is less than the threshold value, thereby acquiring the second corrected shape of the heart valve in the predetermined time phase.

[0096] The measurement information acquisition function 230 and the corrected shape acquisition function 240 perform the above-described processing for each of the input images of multiple time phases, thereby acquiring the second corrected shape for each time phase in the multiple time phases. Note that the correction based on the information between time phases and the correction based on the measurement information may each be executed once, or may be repeatedly executed multiple times. For example, the correction based on the information between time phases may be executed again for the second corrected shape of each time phase acquired by the corrected shape acquisition function 240, and then the correction based on the measurement information may be executed again to acquire the final corrected shape. Note that the number of repetitions can be set as appropriate.

[0097] Note that the medical information processing apparatus 100 has a processing system that inputs 2D images, a processing system that inputs 3D images, and a processing system that inputs time-series images, and can also switch and execute the driving processing system according to the actually input images.

[0098] (Modification Example 2: Variation of the valve tip) In the above-described embodiment, the mitral valve composed of two valve leaflets, i.e., the anterior leaflet and the posterior leaflet, was described as an example. However, the embodiment is not limited thereto. For example, it may be the aortic valve, tricuspid valve, or pulmonary valve composed of three valve leaflets. In this case, two valve leaflets may be selected as the objects of processing from the three valve leaflets, and the same processing as in the above-described embodiment may be executed, or all three valve leaflets may be the objects of processing. When all three valve leaflets are the objects of processing, for example, in step S330, measurement information can be acquired and corrected in the same manner as in the above-described embodiment between the lattice points of the mesh of any one valve leaflet and each of the other two valve leaflets. Also, the plurality of valve leaflets are not necessarily the valves of the same heart. For example, it is also applicable to a combination of the anterior leaflet of the mitral valve and the left coronary cusp of the aortic valve. Thereby, the medical information processing apparatus 100 can correct the positional relationship of different heart valves.

[0099] (Modification Example 3: Variation of Selecting the Phase to be Corrected) In the above-described embodiment, the case of performing processing on medical images of an arbitrary phase was described. That is, the case of performing the same correction without distinguishing the phase was described. However, the embodiment is not limited thereto, and processing may be performed according to the phase. For example, the processing may be switched based on the cardiac phase.

[0100] In such a case, for example, when the input image acquisition function 210 acquires the input image collected by electrocardiogram-synchronized imaging, it acquires the information of the cardiac phase together to specify the phase of the acquired input image. Then, the initial shape acquisition function 220, the measurement information acquisition function 230, and the corrected shape acquisition function 240 switch the processing based on the phase specified by the input image acquisition function 210.

[0101] For example, the initial shape acquisition function 220, the measurement information acquisition function 230, and the corrected shape acquisition function 240 can execute each of the above-described processes with only the input image corresponding to the closed phase (the systolic phase for the mitral valve and the diastolic phase for the aortic valve), which is the phase when the heart valve is closed, as the object of the correction process.

[0102] Also, for example, the initial shape acquisition function 220, the measurement information acquisition function 230, and the corrected shape acquisition function 240 can change the threshold value, correction amount, and correction target part between the closed phase and the open phase. For example, regarding the correction for the heart valve in the closed phase, the threshold value and correction amount for comparison with the measurement information are set to be larger, and the corrected shape acquisition function 240 executes a correction process based on the set threshold value and correction amount. That is, in this correction, the cardiac phase is more reliable than the estimated result of the shape, and in the closed phase, the correction is performed so that the heart valve closes as much as possible.

[0103] Also, for example, in the correction for the heart valve in the open phase, it is set to exclude the vicinity of the center of the valve tip on the valve orifice side (the part where the distance between the valve tips is far) from the correction target. As a result, the measurement information acquisition function 230 acquires measurement information for grid points other than the vicinity of the center of the valve tip on the valve orifice side, and the corrected shape acquisition function 240 executes a correction process for grid points other than the vicinity of the center of the valve tip on the valve orifice side based on the acquired measurement information.

[0104] By switching the process based on the cardiac phase as described above, the efficiency of the correction process (speed-up) can be improved and the process can be stabilized.

[0105] In addition, the medical information processing device 100 can acquire cardiac phase information based on, for example, an input image in addition to the information of the electrocardiogram-synchronized imaging described above. For example, the initial shape acquisition function 220 estimates and acquires the cardiac phase based on the initial shape of the heart valve acquired from the input image.

[0106] (Modification Example 4: Switching of Processing Based on Disease Information) In the above-described Modification 3, the case of switching the process based on the cardiac phase has been described. However, the medical information processing apparatus 100 can also switch the process based on the disease information of the subject. In such a case, for example, the input image acquisition function 210 further acquires the medical information of the subject corresponding to the acquired input image. Then, the initial shape acquisition function 220, the measurement information acquisition function 230, and the corrected shape acquisition function 240 switch the process based on the medical information of the subject acquired by the input image acquisition function 210. Note that the medical information of the subject is acquired from a RIS (Radiology Information System) or a HIS (Hospital Information System) (not shown) via the network 120.

[0107] For example, the initial shape acquisition function 220, the measurement information acquisition function 230, and the corrected shape acquisition function 240 determine whether the subject (patient) has valvular insufficiency based on the medical information of the subject acquired by the input image acquisition function 210, and switch the process based on the determination result. For example, when the subject has valvular insufficiency, the initial shape acquisition function 220, the measurement information acquisition function 230, and the corrected shape acquisition function 240 do not perform the process.

[0108] Also, for example, when the subject has valvular insufficiency, the initial shape acquisition function 220, the measurement information acquisition function 230, and the corrected shape acquisition function 240 can change the threshold value, the correction amount, and the correction target site. For example, when the abnormal site (which cusp or local position) is included in the medical information, the initial shape acquisition function 220, the measurement information acquisition function 230, and the corrected shape acquisition function 240 do not perform the correction process only on the abnormal site. That is, the initial shape acquisition function 220, the measurement information acquisition function 230, and the corrected shape acquisition function 240 execute the correction process for the lattice points other than the abnormal site.

[0109] Further, for example, the threshold value and the correction amount are set such that they become smaller as they are closer to the abnormal site, and when the abnormal site is included in the medical information, the correction shape acquisition function 240 executes a correction process based on the set threshold value and correction amount. That is, in this correction, the parts closer to the abnormal site are controlled so as not to be corrected much.

[0110] Note that the medical information processing apparatus 100 can also acquire the medical information of the subject based on, for example, an input image, in addition to the external apparatuses described above. For example, the initial shape acquisition function 220 acquires the initial shape of the heart valve from input images of a plurality of time phases, and based on the temporal shape change of the acquired initial shape of the heart valve, estimates whether the subject has valvular insufficiency and the abnormal site.

[0111] By switching the process based on the medical information of the subject as described above, appropriate correction can be performed based on the presence or absence of a disease and the possibility thereof.

[0112] (Modification 5: Variations of display) In the above-described embodiment, the case of displaying the corrected shape has been described, but the embodiment is not limited thereto. For example, the display control function 250 can perform various measurements on the corrected shape (which may be different from or the same as S330) between step S340 and step S350 and display the measurement results.

[0113] Further, the display control function 250 can display an image showing the initial shape and the measured value for the corrected shape in the display in step S350. Also, the display control function 250 can switch and display an image showing the initial shape and an image showing the corrected shape.

[0114] As described above, according to the first embodiment, the initial shape acquisition function 220 acquires the first shape of the heart valve composed of a plurality of valve leaflets. The measurement information acquisition function 230 acquires measurement information between the plurality of valve leaflets at a predetermined position of the first shape. The corrected shape acquisition function 240 calculates at least one of a correction amount or a direction for correcting a predetermined position based on the measurement information, and corrects the predetermined position of the first shape based on at least one of the correction amount or the direction. Therefore, the medical information processing apparatus 100 according to the first embodiment can automatically correct the shape of the heart valve extracted based on the measurement information between the valve leaflets, and improve the consistency between the shape of the automatically extracted valve leaflet and / or the measurement result regarding the shape and the shape of the valve leaflet recognized by the user.

[0115] Also, according to the first embodiment, the initial shape acquisition function 220 acquires the first shape from a medical image obtained by imaging the heart valve. The corrected shape acquisition function 240 acquires a second shape obtained by correcting a predetermined position of the first shape in the medical image. Therefore, the medical information processing apparatus 100 according to the first embodiment can improve the consistency between the shape of the valve leaflet extracted from the medical image and / or the measurement result regarding the shape and the shape of the valve leaflet recognized by the user in the medical image.

[0116] Also, according to the first embodiment, the measurement information is the distance between a plurality of valve leaflets, and the corrected shape acquisition function 240 performs correction to reduce the distance between the plurality of valve leaflets. Further, the measurement information is the area of the region formed by a plurality of valve leaflets, and the corrected shape acquisition function 240 performs correction to reduce the area of the region formed by the plurality of valve leaflets. Therefore, the medical information processing apparatus 100 according to the first embodiment can appropriately perform correction to eliminate minute gaps generated between a plurality of valve leaflets in the automatic extraction of the heart valve.

[0117] Also, according to the first embodiment, the first shape (initial shape) and the second shape (corrected shape) are formed from a point cloud. The point cloud is mesh information including the connection relationship between each point. Therefore, the medical information processing apparatus 100 according to the first embodiment enables appropriate correction processing to be easily performed.

[0118] Also, according to the first embodiment, the measurement information acquisition function 230 determines a predetermined position in the first shape based on the reliability of the point cloud. Therefore, the medical information processing apparatus 100 according to the first embodiment can suppress unnatural deformation of the corrected shape.

[0119] Also, according to the first embodiment, the initial shape acquisition function 220 acquires the first shape of the heart valve from medical images of a plurality of time phases respectively. The corrected shape acquisition function 240 acquires a third shape (first corrected shape) obtained by correcting the first shape of a predetermined time phase included in the plurality of time phases based on the first shape (initial shape) of the heart valve for each of the plurality of time phases. The measurement information acquisition function 230 acquires measurement information at a predetermined position of the third shape (first corrected shape). The corrected shape acquisition function 240 calculates at least one of a correction amount or a direction for correcting the predetermined position based on the measurement information, and acquires a fourth shape (second corrected shape) obtained by correcting the predetermined position of the third shape (first corrected shape) based on at least one of the correction amount or the direction. Therefore, the medical information processing apparatus 100 according to the first embodiment can acquire a more plausible corrected shape as the actual shape of the valve tip.

[0120] (Second Embodiment) In the above-described first embodiment, the configuration for calculating the correction amount and direction for correcting the lattice points from the initial shape has been described. In the second embodiment, a configuration for acquiring a corrected shape obtained by correcting the shape based on the input image in addition to the initial shape will be described.

[0121] The configuration of the medical information processing system 10 according to the second embodiment is the same as that of the first embodiment described with reference to FIG. 1. FIG. 6 is a diagram showing the functional configuration of the processing circuit 115 of the medical information processing apparatus 100 according to the second embodiment. Note that the flowchart showing the overall processing procedure performed by the medical information processing apparatus 100 in the present embodiment is the same as that of the first embodiment described with reference to FIG. 3. In the following description, only the configuration of the parts different from those of the first embodiment will be described.

[0122] (Acquisition process of input image) In step S310 according to the second embodiment, when the user instructs to acquire a medical image via the input IF140, the input image acquisition function 210 acquires, as an input image, the image (image data) specified by the user from the data server 130. Then, the input image acquisition function 210 outputs the acquired input image to the initial shape acquisition function 220, the measurement information acquisition function 230, the corrected shape acquisition function 240, and the display control function 250.

[0123] (Acquisition process of measurement information) In the first embodiment, the measurement information was acquired using only the initial shape, but in the second embodiment, the measurement information acquisition function 230 can also acquire the measurement information using the input image. For example, the threshold value for comparison with the measurement information may be determined based on the thickness of the valve tip calculated from the input image. For example, the measurement information acquisition function 230 can calculate the thicknesses of the anterior tip 530 and the posterior tip 540 in FIG. 5, and determine the threshold value based on the calculated thicknesses.

[0124] Further, for example, if there are a certain number or more of pixel values that do not correspond to the mitral valve (for example, pixel values of the imaged atrium) between any positions from the lattice points of the anterior leaflet to the posterior leaflet, it may be determined that the lattice points of the anterior leaflet are separated from the posterior leaflet and excluded from the target of the correction process in step S340. That is, the measurement information acquisition function 230 determines whether there are a certain number or more of pixel values that do not correspond to the mitral valve between any positions from the lattice points of the anterior leaflet to the posterior leaflet in the initial shape acquired by the initial shape acquisition function 220. If there are a certain number or more of pixel values that do not correspond to the mitral valve, the determined lattice points of the anterior leaflet are excluded from the correction target, and the measurement information at the lattice points is not acquired.

[0125] (Acquisition process of corrected shape) In step S340 according to the second embodiment, the corrected shape acquisition function 240 acquires a corrected shape obtained by correcting the initial shape based on the input image acquired in step S310 and the measurement information acquired in step S330. Then, the corrected shape acquisition function 240 outputs the acquired corrected shape to the display control function 250.

[0126] Here, in the first embodiment, in the acquisition process of the corrected shape, the correction amount and direction for correcting the lattice points from the initial shape were calculated. However, in the second embodiment, the corrected shape acquisition function 240 determines the lattice points to be corrected using the pixel values of the input image, and calculates the correction amount and direction for correction. Specifically, when the input image is a contrast CT image and the initial shape of the mitral valve has been acquired, the corrected shape acquisition function 240 calculates the correction amount and direction for correcting the lattice points by utilizing the fact that the pixel values of the mitral valve in the contrast CT image have lower pixel values than the surrounding cardiac cavity region.

[0127] For example, among the plurality of grid points where the measurement information is less than the threshold value in step S330, the correction shape acquisition function 240 selects, as the grid points to be corrected, those grid points whose pixel values in the CT image at the coordinates of the grid points are higher than the pixel values corresponding to the mitral valve because they are likely to be far from the position of the mitral valve (due to low reliability). Then, the correction shape acquisition function 240 corrects the grid points based on the gradient of the pixel values around the selected grid points in the direction (vector) in which the pixel values decrease.

[0128] For example, the correction shape acquisition function 240 can perform correction to move the grid points in the direction in which the pixel values decrease by the correction amount obtained in the same manner as in the first embodiment. That is, the correction shape acquisition function 240 can perform a correction process to correct the initial shape by using only the correction amount (the correction amount obtained in the same manner as in the first embodiment) from the measurement information and using a direction not based on the measurement information (the direction in which the pixel values decrease) for the correction direction.

[0129] Also, for example, the correction shape acquisition function 240 can perform correction to move the grid points in the correction direction obtained in the same manner as in the first embodiment by the correction amount determined based on the gradient of the surrounding pixel values. That is, the correction shape acquisition function 240 can perform a correction process to correct the initial shape by using only the correction direction (the correction direction obtained in the same manner as in the first embodiment) from the measurement information and using a direction not based on the measurement information (the correction amount based on the gradient of the pixel values) for the correction amount. Note that, for example, the correction amount based on the gradient of the pixel values may be the distance from the position (pixel) of the grid point to the pixel having the pixel value corresponding to the mitral valve.

[0130] Also, the correction direction may be determined by combining the direction obtained by the same method as in the first embodiment and the direction based on the gradient of the pixel values. For example, the lattice points can be corrected by moving them in the direction obtained by taking the weighted average of the direction for correcting the lattice points calculated by the same method as in the first embodiment and the direction calculated using the pixel value gradient. Here, as the weight, a predetermined value determined in advance may be used, or it may be determined based on the pixel values. Also, the weight may be determined based on the degree of measurement information (closeness between a plurality of valve tips) described in step S330. For example, when the gradient of the pixel values in the direction calculated by the same method as in the first embodiment indicates that the pixel values are increasing (away from the pixel values corresponding to the mitral valve), there is a possibility that the position of the lattice points will move away from the position of the mitral valve on the image due to the movement of the lattice points by the correction process. Therefore, in this case, the weight related to the direction calculated by the same method as in the first embodiment is set small and averaged. Also, when determining the weight based on the degree of measurement information, for example, when the distance between a plurality of valve tips is small, the weight related to the direction calculated by the same method as in the first embodiment may be made small. Thereby, it is possible to obtain a corrected shape that improves the consistency between the shape of the valve tip and / or the measurement result regarding the shape and the shape of the valve tip recognized by the user, and suppresses the deviation in position from the valve of the heart shown in the input image and display it.

[0131] According to the present embodiment, in the heart valve visually recognized when the user is in contact, it is possible to obtain and display a corrected shape that improves the consistency between the shape of the valve tip and / or the measurement result regarding the shape and the shape of the valve tip recognized by the user, and suppresses the deviation in position from the valve of the heart shown in the input image.

[0132] (Modification 1: Variations of the input image) In the above-described embodiments, an example has been described in which a CT image is used as an input image and the reliability is calculated based on the magnitude of the pixel values thereof. However, the embodiments are not limited thereto. For example, an image showing the region of a heart valve obtained by a known technique from a CT image or an image captured by another modality may be used as the input image, and a method based on the pixel values of the image may also be used. For example, as an example of a technique based on machine learning, a UNet, which is a type of CNN, can be used to obtain the region of the heart valve from a CT image, and the obtained image can be used as the input image. The image showing the region of the heart valve may be a binary label in which the foreground pixels showing the region of the heart valve are stored as 255 and the other background pixels are stored as 0, or a likelihood map in which the likelihood of the presence of the heart valve is stored in the pixel value of each pixel. In the case of the binary label, an image having a gradient of pixel values can be obtained by performing a smoothing process. When these images are input, in step S340, by correcting the lattice points in the direction toward the pixel value of the foreground pixel or the pixel value having a high likelihood based on the gradient of the pixel values, it is possible to obtain a corrected shape that suppresses the deviation from the position of the mitral valve on the image while improving the consistency with the shape visible to the user.

[0133] (Other Embodiments) In addition, in the above-described embodiments, an example has been described in which the acquisition unit, measurement unit, calculation unit, and correction unit in this specification are realized by the initial shape acquisition function, measurement information acquisition function, and corrected shape acquisition function of the processing circuit, respectively. However, the embodiments are not limited thereto. For example, the acquisition unit, measurement unit, calculation unit, and correction unit in this specification may be realized not only by the initial shape acquisition function, measurement information acquisition function, and corrected shape acquisition function described in the embodiments, but also by only hardware, only software, or a combination of hardware and software.

[0134] Also, the term "processor" used in the description of the above-described embodiments means, for example, a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), or a circuit such as an application specific integrated circuit (ASIC), a programmable logic device (e.g., a simple programmable logic device (SPLD), a complex programmable logic device (CPLD), and a field programmable gate array (FPGA)). Here, instead of storing a program in a storage circuit, the program may be directly incorporated into the circuit of the processor. In this case, the processor realizes its function by reading and executing the program incorporated in the circuit. Also, each processor of the present embodiment is not limited to being configured as a single circuit for each processor, and may be configured as one processor by combining a plurality of independent circuits so as to realize its function.

[0135] Here, the medical information processing program executed by the processor is provided by being pre - incorporated into a ROM (Read Only Memory), a memory circuit, etc. Note that this medical information processing program may be recorded and provided on a non - transitory computer - readable storage medium such as a CD (Compact Disk) - ROM, an FD (Flexible Disk), a CD - R (Recordable), a DVD (Digital Versatile Disk) in a file in a form installable or executable on these devices. Also, this medical information processing program may be stored on a computer connected to a network such as the Internet and provided or distributed by being downloaded via the network. For example, this medical information processing program is composed of modules including each of the above - described processing functions. As actual hardware, the CPU reads the medical information processing program from a storage medium such as a ROM and executes it, whereby each module is loaded onto the main memory device and generated on the main memory device.

[0136] Also, in the above - described embodiments and modified examples, each component of each illustrated device is conceptually functional and does not necessarily have to be physically configured as shown in the drawings. That is, the specific form of the distribution or integration of each device is not limited to that shown in the drawings, and all or part of it can be functionally or physically distributed or integrated in any unit according to various loads, usage situations, etc. Further, each processing function performed by each device can be realized in whole or in any part by a CPU and a program analyzed and executed by the CPU, or can be realized as hardware by wired logic.

[0137] In addition, among the respective processes described in the above-described embodiments and modifications, all or part of the processes described as being automatically performed can be manually performed, or all or part of the processes described as being manually performed can be automatically performed by a known method. Additionally, regarding the processing procedures, control procedures, specific names, and information including various data and parameters shown in the above documents and drawings, they can be arbitrarily changed unless otherwise specified.

[0138] According to at least one of the embodiments described above, it is possible to improve the consistency between the shape of the valve tip extracted from the medical image and / or the measurement results regarding the shape, and the shape of the valve tip recognized by the user in the medical image.

[0139] Although several embodiments have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the gist of the invention. These embodiments and their modifications are included in the scope and gist of the invention, as well as in the invention described in the claims and its equivalent scope.

Description of Reference Numerals

[0140] 100 Medical information processing device 210 Input image acquisition function 220 Initial shape acquisition function 230 Measurement information acquisition function 240 Corrected shape acquisition function 250 Display control function

Claims

1. An acquisition unit that acquires a first shape of a heart valve composed of a plurality of valve leaflets; a measurement unit that acquires measurement information between a plurality of valve cusps at a predetermined position of the first shape; a calculation unit that calculates at least one of a correction amount and a direction for correcting the predetermined position based on the measurement information; a correction unit that corrects the predetermined position of the first shape based on at least one of the correction amount or the direction; A medical information processing device comprising:

2. The acquisition unit acquires the first shape from a medical image of the heart valve, The medical image processing apparatus according to claim 1 , wherein the correction unit acquires a second shape by correcting the predetermined position of the first shape in the medical image.

3. The medical information processing device according to claim 1 , wherein the measurement information is distances between the plurality of valve cusps.

4. The medical image processing device according to claim 3 , wherein the correction unit performs a correction such that a distance between the plurality of valve cusps becomes smaller.

5. The medical information processing device according to claim 1 , wherein the measurement information is an area of ​​a region formed by the plurality of valve cusps.

6. The medical image processing device according to claim 5 , wherein the correction unit performs a correction such that an area of ​​a region formed by the plurality of valve cusps becomes smaller.

7. The medical image processing device according to claim 2 , wherein the first shape and the second shape are formed from a cloud of points.

8. The medical image processing apparatus according to claim 7 , wherein the point cloud is mesh information including a connection relationship between each point.

9. The medical image processing apparatus according to claim 7 , wherein the measurement unit determines the predetermined position in the first shape based on a reliability of the point cloud.

10. The medical image processing apparatus according to claim 2 , wherein the correction unit corrects the first shape based on the measurement information and a gradient of pixel values ​​of the medical image.

11. The acquisition unit acquires a first shape of the cardiac valve from medical images of a plurality of time phases, the correction unit obtains a third shape by correcting a first shape of a predetermined time phase included in the multiple time phases based on a first shape of the cardiac valve in each of the multiple time phases; the measurement unit acquires the measurement information at a predetermined position of the third shape; the calculation unit calculates at least one of a correction amount or a direction for correcting the predetermined position based on the measurement information; The medical image processing apparatus according to claim 2 , wherein the correction unit acquires a fourth shape by correcting the predetermined position of the third shape based on at least one of the correction amount and the direction.

12. Obtaining a first shape of a heart valve comprising a plurality of leaflets; obtaining measurement information between a plurality of leaflets at predetermined positions of the first shape; Calculating at least one of a correction amount and a direction for correcting the predetermined position based on the measurement information; correcting the predetermined position of the first shape based on at least one of the correction amount or the direction; A medical information processing method comprising:

13. Obtaining a first shape of a heart valve comprising a plurality of leaflets; obtaining measurement information between a plurality of leaflets at predetermined positions of the first shape; Calculating at least one of a correction amount and a direction for correcting the predetermined position based on the measurement information; correcting the predetermined position of the first shape based on at least one of the correction amount or the direction; A medical information processing program that causes a computer to execute each process.

Images