Information processing device, information processing method and program

Abstract

To provide an information processing technique for acquiring three-dimensional posture information of a partition wall part of the heart from image information.SOLUTION: An information processing device includes: image acquisition means that acquires an image in which a partition wall part separating a pair of lumina constituting the heart and a reference portion located at a position different from the partition wall part are captured; first shape acquisition means that acquires information on a first shape of the partition wall part on the basis of the image; orientation acquisition means that acquires information on a reference orientation of the partition wall part on the basis of the first shape; position acquisition means that acquires first position information on the partition wall part and second position information on the reference portion on the basis of the image; and position and posture acquisition means that acquires information on the position and posture of the partition wall part on the basis of the first position information, the reference orientation of the partition wall part, and the second position information.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present invention relates to an information processing device, an information processing method, and a program. [Background technology]

[0002] In the medical field, there is a technology that acquires the position and shape of the septum between multiple cardiac chambers from medical images captured by various modalities, and displays the shape and provides the measured values ​​of the shape to assist in understanding the structure and condition of the heart. Patent Document 1 discloses a technology that acquires three-dimensional position and orientation information of the ventricular septum, which is a thin structure, based on multiple two-dimensional X-ray images captured at different imaging positions and position information (X-ray irradiation direction) of an X-ray detector, and uses the information for surgical support. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2022-186038 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, with the technique disclosed in Patent Document 1, it is difficult to obtain information on the three-dimensional position and orientation of the septum of the heart only from the image information without using position information from the X-ray detector.

[0005] The present invention has been made in consideration of the above problems, and has an object to provide an information processing technique for acquiring three-dimensional orientation information of the septum of the heart from image information. [Means for solving the problem]

[0006] A first aspect of the present invention is an information processing device characterized by having an image acquisition means for acquiring an image of a partition separating a pair of cavities that constitute the heart and a reference site located at a position different from the partition, a first shape acquisition means for acquiring information regarding a first shape of the partition based on the image, an orientation acquisition means for acquiring information regarding a reference orientation of the partition based on the first shape, a position acquisition means for acquiring first position information regarding the partition and second position information regarding the reference site based on the image, and a position and orientation acquisition means for acquiring information regarding the position and orientation of the partition based on the first position information, the reference orientation of the partition, and the second position information.

[0007] A second aspect of the present invention is an information processing device comprising: an image acquisition means for acquiring an image of a partition separating a pair of cavities that constitute a subject's heart; a shape acquisition means for acquiring information regarding a shape of the partition based on the image; an orientation acquisition means for acquiring information regarding a reference orientation of the partition based on the shape of the partition and for acquiring information regarding a reference orientation of the subject based on the image; a position acquisition means for acquiring positional information regarding the partition based on the image; and a position and orientation acquisition means for acquiring information regarding the position and orientation of the partition based on the positional information regarding the partition, the reference orientation of the partition, and the reference orientation of the subject. Effect of the Invention

[0008] According to the present invention, three-dimensional orientation information of the septum of the heart can be obtained from image information. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram illustrating an example of a configuration of an information processing system. [Diagram 2] 2 is a block diagram illustrating a functional configuration of a control unit of the information processing device. FIG. [Diagram 3] 11 is a flowchart illustrating a process of an information processing device. [Figure 4]4A to 4C are diagrams illustrating the process of acquiring posture information of the fossa ovalis in the first embodiment. [Diagram 5] 13A to 13C are diagrams illustrating the process of acquiring posture information of the fossa ovalis in the second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. Note that the present invention is not limited to the following embodiment. A plurality of features described in the embodiment can be arbitrarily combined.

[0011] First Embodiment The information processing device according to the present invention acquires information on the three-dimensional position and orientation of the septum, which is a thin structure of the heart, from a three-dimensional image of the heart, based on the positional relationship between the septum and surrounding anatomical structures different from the septum. The septum is a wall portion that separates a pair of adjacent cavities among a plurality of cavities that constitute the heart. For example, when the right atrium and the left atrium are considered as a pair of cavities, the atrial septum that separates them and the fossa ovalis located at the center of the atrial septum are examples of the septum.

[0012] The information processing device supports the user in understanding the structure and state of the septum of the heart by presenting to the user the shape of the septum whose position and orientation have been converted based on the acquired information. The information processing device acquires the position and orientation of the septum, and can present to the user a plurality of different images of the septum with the septum aligned in position and orientation. That is, the septum is displayed uniformly in the same position and orientation in any image, improving visibility of the septum and allowing the user to easily understand the orientation of the septum. The configuration and processing of the information processing device according to the first embodiment will be described below with reference to Figs. 1 to 3.

[0013] 1 is a diagram illustrating a configuration of an information processing system 10 including an information processing device 100 according to the first embodiment. The information processing system 10 includes the information processing device 100, a network 120, and a data server 130. The information processing device 100 is communicatively connected to the data server 130 via the network 120. The network 120 includes, for example, a local area network (LAN) and a wide area network (WAN).

[0014] The data server 130 is, for example, a picture archiving and communication system (PACS) that holds and manages medical images and information associated with the medical images. The information processing device 100 can acquire medical images held in the data server 130 via the network 120. The data server 130 receives and stores images captured by a medical imaging device (modality), and transmits the images to each device in response to a request from the device connected to the network 120. The data server 130 also includes a database that can store various data associated with the received images together with the images received from the modality.

[0015] In the following description, the three-dimensional image is an image captured by an X-ray CT (Computed Tomography) device, but the three-dimensional image may be captured by other modalities. In addition to the X-ray CT device, the modality may include, for example, an MRI (Magnetic Resonance Imaging) device and a SPECT (Single Photon Emission Computed Tomography) device. In addition, the modality may include a PET (Positron Emission Tomography) device, an ultrasonography (Ultrasonography) device, and the like. The information processing device 100 may include an ultrasound diagnostic device, etc. The three-dimensional image may be an image that has been subjected to pre-processing such as resolution conversion and smoothing, or may be an image derived from a plurality of images captured by a plurality of modalities that have been subjected to pre-processing such as image fusion and image difference. The processing by the information processing device 100 is applicable to three-dimensional images acquired by various modalities.

[0016] The information processing device 100 is a device that performs image processing, and generates a projection image of a three-dimensional image and displays it on a display unit 150. The information processing device 100 is used, for example, as a terminal device for reading images operated by a user such as a doctor. The information processing device 100 includes a communication IF (Interface) 111 (communication unit), a ROM (Read Only Memory) 112, a RAM (Random Access Memory) 113, a storage unit 114, and a control unit 115. The information processing device 100 is connected to an instruction unit 140 and a display unit 150.

[0017] The communication IF 111 is, for example, a LAN card, and realizes communication with an external device (for example, a data server 130, etc.) via the network 120. The ROM 112 is, for example, a non-volatile memory, and stores various programs. The RAM 113 is, for example, a volatile memory, and temporarily stores various information as data. The storage unit 114 is, for example, a HDD (Hard Disk Drive), and stores various information as data.

[0018] The instruction unit 140 includes a GUI (Graphical User Interface) such as a keyboard, a mouse, and a touch panel. The instruction unit 140 inputs instructions from a user (e.g., a doctor) to the information processing device 100. An image to be processed is input to the information processing device 100 in response to an instruction from a user who operates the instruction unit 140, for example. Note that the image to be processed is not limited to being selected based on an instruction from a user, and may be automatically selected by the control unit 115 based on a predetermined rule, for example.

[0019] 2 is a diagram illustrating an example of a functional configuration of the control unit 115 of the information processing device 100. The control unit 115 is, for example, a CPU (Central Processing Unit), and controls the entire processing by the information processing device 100. The control unit 115 includes, as its functional configuration, an input image acquisition unit 210, an initial shape acquisition unit 220, a position acquisition unit 230, a position and orientation acquisition unit 240, a transformed image acquisition unit 245, a detailed shape acquisition unit 250, and a display control unit 260.

[0020] The input image acquisition unit 210 acquires an input image to be processed from the data server 130 via the communication IF 111 (communication unit) and the network 120. The input image is an image of a partition wall separating a pair of cavities that constitute the heart, and a reference site located at a different position from the partition wall. The reference site is an anatomical structure that exists around the partition wall.

[0021] The initial shape acquisition unit 220 acquires information on the initial shape (first shape) of the septum of the heart based on the input image. The position acquisition unit 230 acquires, based on the input image, first position information (first position information) on the septum and second position information (second position information) on a reference site located at a position different from the septum.

[0022] The position and orientation acquisition unit 240 acquires information on the reference orientation of the partition based on the initial shape. The position and orientation acquisition unit 240 acquires information on the position and orientation of the partition based on the first position information, the reference orientation of the partition, and the second position information (position and orientation information). The converted image acquisition unit 245 acquires a converted image obtained by converting the input image based on the position and orientation information of the partition.

[0023] The detailed shape acquisition unit 250 acquires information about a detailed shape (second shape) of the partition portion, which is different from the initial shape of the partition portion, based on the converted image converted by the converted image acquisition unit 245. The display control unit 260 generates an image to be displayed on the display unit 150 and the shape of the septum of the heart, and performs display control for displaying the image on the display unit 150 .

[0024] The display unit 150 includes devices such as an LCD (Liquid Crystal Display) and a CRT (Cathode Ray Tube) display. The display unit 150 displays images and various information to the user. Specifically, the display unit 150 displays an input image and a projected image acquired from the information processing device 100.

[0025] Each component of the information processing device 100 functions according to a computer program. For example, the control unit 115 (CPU) loads a computer program stored in the ROM 112 or the storage unit 114 into the RAM 113 as a work area and executes the program, thereby realizing the function of each component. Note that some or all of the functions of the components of the information processing device 100 may be realized using a dedicated circuit. Also, some of the functions of the components of the control unit 115 may be realized using a cloud computer.

[0026] For example, a calculation device located at a different location from the information processing device 100 is communicably connected to the information processing device 100 via the network 120, and the information processing device 100 transmits and receives data to and from the calculation device. The control unit 115 of the information processing device 100 can realize the functions of each component by using the data received from the calculation device.

[0027] The processing of the information processing device 100 shown in FIG. 1 will be described with reference to FIG. 3. FIG. 3 is a flowchart illustrating the processing of the information processing device 100. As a specific example, a process of acquiring and displaying a detailed shape of the fossa ovalis present in the atrial septum based on a CT image of the heart will be described. Note that the processing of the information processing device 100 shown in FIG. 3 can also be applied to images acquired by modalities other than a CT device. The processing of the information processing device 100 can also be applied to other partitions of the heart (for example, the ventricular septum, which is a partition separating the right ventricle from the left ventricle).

[0028] (S310: Acquire input image) In step S310, the input image acquisition unit 210 (image acquisition means) acquires, in response to a user's instruction, a cardiac CT image (input image) of the subject's heart and a cardiac label image from the data server 130. The user can instruct acquisition of an input image via the instruction unit 140 and specify the input image to be acquired.

[0029] The input image is a three-dimensional image in which the fossa ovalis as the septum and the valve cusps constituting the cardiac valve as the reference site are captured. The cardiac labeled image is an image in the same image coordinate system as the input image, and is an image showing the region of the structure adjacent to the septum. When the septum is the fossa ovalis, the cardiac labeled image is an image in which a label value unique to each structure is stored in pixels showing structures such as the left atrium and the right atrium. For example, the cardiac labeled image is an image in which the label value of the pixels included in the left atrium region is 1, the label value of the pixels included in the right atrium region is 2, and the label value of the pixels not included in the heart is 0. The input image acquisition unit 210 outputs the acquired cardiac CT image and cardiac labeled image to the initial shape acquisition unit 220, the converted image acquisition unit 245, and the display control unit 260.

[0030] Note that the cardiac labeled image is not limited to being acquired from the data server 130, but may be acquired by known image processing techniques, etc. For example, the input image acquisition unit 210 can extract a cardiac region from a cardiac CT image to acquire a cardiac labeled image. Specifically, the input image acquisition unit 210 can estimate and acquire a cardiac labeled image from a cardiac CT image using UNet, which is a type of CNN, as an example of a method based on machine learning.

[0031] Furthermore, the cardiac labeled image is not limited to being composed of one image, but may be composed of multiple images. For example, the cardiac labeled image includes a labeled image in which a label value of the left atrium is stored and a labeled image in which a label value of the right atrium is stored, and these labeled images are acquired separately. When the labeled image of the left atrium and the labeled image of the right atrium are acquired separately, the label value of the left atrium and the label value of the right atrium may be the same value. If the labeled image of the left atrium and the labeled image of the right atrium are separate images, the region of the left atrium and the region of the right atrium can be distinguished even if the label value of the left atrium and the label value of the right atrium are not different label values. Furthermore, the cardiac labeled image may include regions of structures (e.g., the left ventricle and the mitral valve) other than the structures adjacent to the septum.

[0032] (S320: Obtaining the initial shape) In step S320, the initial shape acquisition unit 220 (first shape acquisition means) acquires an initial shape (first shape) of the cardiac septum region based on the cardiac CT image and the cardiac labeled image acquired in step S310. The initial shape acquisition unit 220 outputs the acquired initial shape to the position acquisition unit 230 and the position and orientation acquisition unit 240.

[0033] The initial shape acquisition unit 220 acquires a fossa ovalis mask image showing the region of the fossa ovalis included in the cardiac CT image as the initial shape of the cardiac septum. The fossa ovalis mask image is a binary image in the same image coordinate system as the cardiac CT image, in which a foreground pixel value (e.g., 1) is stored in pixels included in the region of the fossa ovalis and a background pixel value (e.g., 0) is stored in regions other than the fossa ovalis.

[0034] As an example of a method for acquiring a fossa ovalis mask image, a method using a cardiac label image will be described. The fossa ovalis is a region between the left atrium and the right atrium that is relatively thin in thickness. The initial shape acquisition unit 220 calculates the distance (thickness) between the contours of the left atrium and the contours of the right atrium, and acquires a region whose thickness is within a threshold as the region of the fossa ovalis (foreground region of the fossa ovalis mask image). The acquired region of the fossa ovalis corresponds to the first shape. In this way, the initial shape acquisition unit 220 can acquire information about the first shape based on the thickness of the septum.

[0035] As another method of acquiring the fossa ovalis mask image, the initial shape acquisition unit 220 may acquire the region of the fossa ovalis by fitting a statistical shape model to the cardiac CT image of the subject. For example, the initial shape acquisition unit 220 aligns the statistical shape model representing the statistical tendency of the shapes and positions of the left atrium, right atrium, and fossa ovalis calculated from the cardiac CT image of another case with the left atrium and right atrium of the cardiac label image of the subject. The initial shape acquisition unit 220 may set the region obtained by projecting the region of the fossa ovalis included in the statistical shape model onto the image coordinate system of the subject based on the alignment result as the foreground region of the fossa ovalis mask image. In this way, the initial shape acquisition unit 220 can acquire information about the region of the fossa ovalis (first shape) based on the statistical shape model representing the statistical tendency of the shape of the septum.

[0036] The initial shape acquisition unit 220 may acquire a fossa ovalis mask image based on both the region of the first fossa ovalis acquired by a technique using the thickness between the left atrium and the right atrium and the region of the second fossa ovalis acquired by a technique using a statistical shape model. For example, the initial shape acquisition unit 220 may determine the intersection region where the region of the first fossa ovalis and the region of the second fossa ovalis overlap as the foreground region of the fossa ovalis mask image, or may determine the union region of the region of the first fossa ovalis and the region of the second fossa ovalis as the foreground region of the fossa ovalis mask image.

[0037] Furthermore, the fossa ovalis mask image showing the initial shape of the cardiac septum may be obtained from the cardiac CT image using a known image analysis technique, without using the cardiac labeled image. For example, the initial shape acquisition unit 220 may estimate and acquire the area of ​​the fossa ovalis from the cardiac CT image using UNet, which is a type of CNN, as an example of a machine learning based technique. In this case, the initial shape acquisition unit 220 provides the cardiac labeled image to different input channels of UNet to estimate the area of ​​the fossa ovalis. It may be estimated.

[0038] In either case, the initial shape acquisition unit 220 may directly acquire the fossa ovalis mask image, which is a binary image, by UNet. Alternatively, the initial shape acquisition unit 220 may acquire an image in which a value representing the fossa ovalis-likeness (probability of the existence of the fossa ovalis) is stored in the pixel value of each pixel, and then perform threshold processing to acquire the fossa ovalis mask image, which is a binary image. The initial shape acquisition unit 220 can also acquire a binary fossa ovalis mask image by applying a statistical shape model to the image in which a value representing the fossa ovalis-likeness is stored in the pixel value of each pixel.

[0039] Furthermore, the initial shape acquisition unit 220 may acquire a fossa ovalis mask image that is manually created by the user using a tool (not shown) via the instruction unit 140, or may acquire the fossa ovalis mask image by reading it out from a fossa ovalis mask image that is stored in advance in the data server 130. When the fossa ovalis mask image is acquired without using a heart-labeled image, the input image acquisition unit 210 does not need to acquire a heart-labeled image in step S310.

[0040] (S330: Acquiring location information) In step S330, the position acquisition section 230 (position acquisition means) acquires first position information (first position information) regarding the partition based on the initial shape acquired in step S320. The position acquisition section 230 also acquires second position information (second position information) regarding a surrounding anatomical structure (reference site) that is in a position different from the partition. The position acquisition section 230 outputs the acquired first position information to the detailed shape acquisition section 250, and outputs the first position information and the second position information to the position and orientation acquisition section 240.

[0041] The position acquiring unit 230 acquires three-dimensional coordinates indicating the center of gravity of a region of the septum (e.g., the fossa ovalis) as an example of the first position information, and acquires three-dimensional coordinates indicating the center of gravity of a region of a reference site (e.g., the annulus of the mitral valve) as an example of the second position information.

[0042] The first position is not limited to the center of gravity of the fossa ovalis region, and may be, for example, the center of a bounding box including the fossa ovalis region. The first position information may be directly acquired from a cardiac CT image, not from the fossa ovalis region. For example, the position acquisition unit 230 may estimate the center of gravity of the fossa ovalis region from the cardiac CT image using DenseNet, which is a type of CNN as an example of a machine learning based technique, and acquire the estimated center of gravity as the first position information.

[0043] In addition, the second position information is not limited to the center of gravity of the mitral valve annulus region, and may be the three-dimensional coordinates of an anatomical feature such as the anterior commissure of the mitral valve. The second position information is not limited to the mitral valve, and may be the three-dimensional coordinates of an anatomical feature related to the aortic valve, or may be the three-dimensional coordinates of the tip (apex) of the left ventricle. The second position information can be acquired by a known image processing technique. For example, the position acquisition unit 230 may estimate the center of gravity of the mitral valve annulus region from a cardiac CT image using a machine learning-based technique similar to the first position information, and acquire the estimated center of gravity as the second position information. The second position information is used to acquire position and orientation information of the fossa ovalis in step S340 described later. It is desirable that the second position is a position whose relative positional relationship with the fossa ovalis does not change substantially between different input images.

[0044] (S340: Acquisition of position and orientation information) In step S340, the position and orientation acquisition unit 240 acquires the position and orientation information of the partition. The position and orientation acquisition unit 240 acquires the first position information acquired in step S330 as the position information of the partition. The position and orientation acquisition unit 240 acquires the first position information acquired in step S330 as the position information of the partition. Based on the initial shape acquired in step S320 and the first position information and the second position information acquired in step S330, , acquires orientation information of the partition portion. The orientation information of the partition portion is represented by three mutually perpendicular axes. The position and orientation acquisition section 240 outputs the acquired position and orientation information to the converted image acquisition section 245 and the display control section 260.

[0045] Specifically, in the process shown in Fig. 3, the position and orientation information is the position information and orientation information of the fossa ovalis. The position information of the fossa ovalis is information of a first position that is the center of gravity position of the fossa ovalis area. The orientation information of the fossa ovalis is information of three axes perpendicular to each other and is acquired from a cardiac CT image.

[0046] The process of acquiring posture information of the fossa ovalis in the first embodiment will be described with reference to Fig. 4. Fig. 4(a) is a diagram showing the posture of the subject's heart relative to the image coordinates (XYZ axes) of a cardiac CT image. A shaded area 430 between the left atrium 400 and the right atrium 410 in Fig. 4(a) is an example of the initial shape of the fossa ovalis acquired in step S320. Fig. 4(b) shows a first position 440 which is the position of the center of gravity of the area of ​​the fossa ovalis acquired in step S330, and a second position 450 which is the position of the center of gravity of the area of ​​the annulus of the mitral valve 420.

[0047] The position and orientation acquisition unit 240 (position and orientation acquisition means) acquires position and orientation information of the fossa ovalis (partition) based on information of the first position 440 (first position information), information of the second position 450 (second position information), and the reference orientation of the fossa ovalis (partition).

[0048] First, the position and orientation acquisition unit 240 (orientation acquisition means) acquires the reference orientation of the fossa ovalis. The reference orientation is, for example, the orientation of the normal to the initial shape of the fossa ovalis, and is the direction from the right atrium to the left atrium. In the example of FIG. 4(b), the reference orientation is indicated by the vector u.

[0049] The normal of the initial shape of the fossa ovalis can be obtained by a known technique. For example, when the initial shape is obtained using a fossa ovalis mask image representing the area of ​​the fossa ovalis, the position and orientation acquisition unit 240 performs principal component analysis on a set of pixel positions included in the area of ​​the initial shape. The position and orientation acquisition unit 240 calculates the eigenvector of the first principal component and the eigenvector of the second principal component, and can obtain the cross product of these as the normal of the initial shape of the fossa ovalis.

[0050] The position and orientation acquisition unit 240 may also acquire normals at multiple local locations using the same method as described above, and use a representative value (e.g., average vector) of the acquired multiple normals as the normal of the initial shape. The reference orientation may be a unit vector in the direction from the right atrium side to the left atrium side of the normal of the acquired initial shape. The reference orientation is one of the three axes representing the orientation information acquired in step S340.

[0051] Next, the position and orientation acquisition unit 240 acquires information that determines rotation around an axis that is the reference orientation. A septum such as the fossa ovalis has few anatomical shape features and is thin, so it is difficult to determine an axis perpendicular to the reference orientation of the fossa ovalis using the shape features of the fossa ovalis. Therefore, the position and orientation acquisition unit 240 acquires an axis perpendicular to the reference orientation of the fossa ovalis using information on a site other than the fossa ovalis.

[0052] Specifically, as shown in Fig. 4(b), the position and orientation acquisition unit 240 acquires a vector v that passes through the second position 450 and is orthogonal to the vector u of the reference orientation as the second axis of the three axes representing the orientation information. The vector v determines the rotation around the axis of the reference orientation based on the second position 450. The position and orientation acquisition unit 240 acquires a vector w that is the cross product of the vector u and the vector v as the third axis of the three axes representing the orientation information.

[0053] In this way, the position and orientation acquisition unit 240 acquires three orthogonal axes as orientation information of the fossa ovalis. By using information on the annulus of the mitral valve, it is possible to acquire an axis representing similar orientation information of the fossa ovalis even between images of different subjects. Through the above process, in step S340, position and orientation acquisition section 240 can acquire position information and orientation information of the fossa ovalis, that is, position and orientation information of the septum.

[0054] (S345: Obtaining converted image) In step S345, the converted image acquisition unit 245 acquires a converted image in which the position and orientation of the cardiac CT image are converted based on the position and orientation information of the septum acquired in step S340. The converted image acquisition unit 245 outputs the acquired converted image to the detailed shape acquisition unit 250 and the display control unit 260.

[0055] The transformed image acquisition unit 245 first transforms the orientation of the cardiac CT image so that the vectors u, v, and w, which are the orientation information of the fossa ovalis acquired in step S340, become the image coordinate axes of the transformed image. Then, the transformed image acquisition unit 245 generates a transformed image by performing a cutout process so that the first position (the center of gravity position of the fossa ovalis) acquired in step S340 is the image center and a bounding box including both the left atrium and the right atrium becomes the image range.

[0056] This allows the converted image acquisition unit 245 to acquire converted images in which the position and orientation of the fossa ovalis are consistent within various cardiac CT images, such as cardiac CT images of different subjects and cardiac CT images of the same subject at different times. The range of the cutout process may be a range that is approximately consistent between different images, and is not limited to a bounding box that exactly surrounds both the left atrium and the right atrium, and may be a range obtained by multiplying the size of the bounding box in each direction of the image coordinate axis by a constant. In addition, the bounding box is not limited to a bounding box that exactly surrounds the combination of the left atrium and the right atrium, and may be, for example, a bounding box that exactly surrounds both the mitral valve annulus and the right atrium acquired as the second position information.

[0057] When not only the cardiac CT image but also the cardiac labeled image is used to obtain the detailed shape (second shape) in step S350, the converted image obtaining unit 245 also generates a converted image of the cardiac labeled image in the same manner.

[0058] (S350: Acquisition of detailed shape) In step S350, the detailed shape acquisition unit 250 acquires the detailed shape of the partition wall based on the converted image acquired in step S345. The detailed shape acquisition unit 250 outputs the acquired detailed shape to the display control unit 260.

[0059] The detailed shape acquisition unit 250 can acquire a detailed shape (second shape) indicating the area of ​​the fossa ovalis from the converted image using UNet, similar to the case where the area of ​​the fossa ovalis is estimated from the cardiac CT image. In this case, the detailed shape acquisition unit 250 may estimate the detailed shape of the fossa ovalis by providing the converted image of the cardiac label image to a different input channel of UNet. In either case, the detailed shape acquisition unit 250 may directly acquire a fossa ovalis mask image, which is a binary image, as the detailed shape using UNet. Furthermore, the detailed shape acquisition unit 250 may acquire an image in which a value representing the fossa ovalis-likeness (probability of the existence of the fossa ovalis) is stored in the pixel value of each pixel, and then perform threshold processing to acquire a fossa ovalis mask image, which is a binary image, as the detailed shape.

[0060] The UNet for acquiring the detailed shape is a model that has previously learned the relationship between the image and the detailed shape using a large amount of training data. The training data used for learning the UNet model is preferably a converted image obtained by applying the same method as the processes in steps S310 to S345 to other cases, and the position and orientation of the fossa ovalis are roughly aligned. In other words, the training data is obtained by using an image in the same form as the converted image input to the UNet in step S350. It is desirable to study.

[0061] Generally, a method for augmenting training data is known in which data generated by transforming (translating and rotating) the original training data to improve inference accuracy is added to the learning data. It is preferable to determine the range of each of the translation and rotation parameters used to augment the training data based on the expected variation in position and orientation of the unknown data to be inferred.

[0062] In step S350, the transformed image used for inferring the detailed shape is expected to have the position and orientation of the fossa ovalis roughly aligned, so the range of parameters for enhancing the training data can be made smaller. A model that uses transformed images as training data can increase the density without reducing the number of parameter samples, compared to a case where training data with a larger variation in orientation and a larger range of translation and rotation parameters than the transformed image is used. That is, a trained model using transformed images can learn with data with a distribution closer to the variation in orientation expected for unknown data, without increasing the number of training data used for learning. In addition, a trained model using transformed images can improve the accuracy of inference without increasing the time required for learning and the size of the trained model, compared to a process of acquiring an initial shape.

[0063] The method of acquiring the detailed shape is not limited to machine learning, and may be a method using a statistical shape model, as in the acquisition of the initial shape in step S320. In this case, it is preferable that the data used to generate the statistical shape model is also a converted image acquired by applying the processing method from step S310 to step S345 to another case. By using data of a converted image in which the position and orientation of the fossa ovalis are roughly consistent, the variation in the position and orientation of the fossa ovalis between data is reduced, and it is possible to generate a statistical shape model that represents only the difference in the shape of the fossa ovalis between data.

[0064] The position and orientation of the fossa ovalis are consistent between the statistical shape model and the transformed image to be processed. Therefore, it is possible to more precisely fit the statistical shape model to the transformed image to acquire the detailed shape of the fossa ovalis, compared to the case where the statistical shape model is fitted to the cardiac CT image of the subject to acquire the initial shape in step S320. Therefore, the detailed shape acquisition unit 250 can acquire the detailed shape with higher accuracy.

[0065] Furthermore, the detailed shape acquisition unit 250 may acquire a detailed shape that is manually created by the user of the converted image using a tool (not shown) via the instruction unit 140. Even for different subjects, the orientation of the fossa ovalis in the converted image is consistent, so that the user can efficiently grasp the structure of the fossa ovalis and create a detailed shape.

[0066] The method for acquiring the initial shape in step S320 and the method for acquiring the detailed shape in step S350 may be the same or different. For example, in step S320, the initial shape acquisition unit 220 may acquire the initial shape using a method for fitting a statistical shape model to a cardiac CT image, and in step S350, the detailed shape acquisition unit 250 may acquire the detailed shape using machine learning.

[0067] The bounding box indicating the image range of the transformed image is not limited to a range including both the left atrium and the right atrium. The image range of the transformed image may include the fossa ovalis and may be an anatomically equivalent range between different cardiac CT images. For example, the bounding box may be a range whose edge is a plane passing through the centroid position of the left atrium and the centroid position of the right atrium, or may be a range set based on the mitral valve or other cardiac valve as a reference site.

[0068] The information processing device 100 may replace the initial shape acquired in step S320 with the detailed shape acquired in step S350, and execute the processes from step S330 to S350 again. By using the detailed shape with improved accuracy as the initial shape, the accuracy of the acquired reference orientation and first position information also increases, and it is expected that the accuracy of the detailed shape acquired again in step S350 will improve.

[0069] Furthermore, information processing device 100 may execute the processes from steps S330 to S350 again, with the AND region (sum region) of the initial shape and the detailed shape as the detailed shape. A region included in both regions acquired by two different methods has a higher probability of indicating the fossa ovalis compared to a region acquired by either method. According to the method of using the sum region of the initial shape and the detailed shape as the detailed shape, information processing device 100 can remove outliers such as erroneous extractions included in the detailed shape acquired in step S350.

[0070] Furthermore, when the detailed shape is inferred by CNN, there is a possibility that a plurality of independent regions including outliers may be acquired. In this case, the detailed shape acquisition unit 250 may remove the outliers by, for example, comparing the centroid position of each independent region with the centroid position of the initial shape and acquiring the region having the centroid position closer to the centroid position of the initial shape as the detailed shape.

[0071] Furthermore, if it can be determined that the inference of the detailed shape by the CNN has failed, the detailed shape acquisition unit 250 may discard the detailed shape acquired by the CNN and use the initial shape as the detailed shape. For example, the detailed shape acquisition unit 250 may determine that the inference by the CNN has failed when it detects that a region representing the fossa ovalis has not been acquired anywhere on the converted image. The detailed shape acquisition unit 250 may determine that the inference by the CNN has failed when the region acquired by the CNN has a size, volume, or shape that is different from that of a typical fossa ovalis.

[0072] Further, the detailed shape acquisition unit 250 may acquire the detailed shape not as a region but as other shape information. For example, the detailed shape may be a mesh (third shape). The mesh is represented by a plurality of lattice points and line segments connecting the lattice points, and represents the shape of the fossa ovalis. The detailed shape acquisition unit 250 can acquire the mesh by converting the detailed shape (second shape). The detailed shape acquisition unit 250 may acquire the mesh representing the shape of the fossa ovalis from the converted image by a known image analysis technique. The detailed shape acquisition unit 250 can estimate and acquire the mesh representing the shape of the fossa ovalis from the converted image, for example, by using DenseNet, which is a type of CNN as an example of a technique based on machine learning. In this way, information on the mesh representing the shape of the fossa ovalis is acquired based on the detailed shape or the converted image.

[0073] The detailed shape acquisition unit 250 may determine the positions and indexes of the mesh lattice points based on the attitude information acquired in step S340. For example, the detailed shape acquisition unit 250 places at least one of the mesh lattice points on a plane formed by the vector v and the vector w that passes through the first position, and sets a reference index (for example, 0) for that lattice point. The indexes for the other lattice points are set based on the reference index. The indexes for the other lattice points can be set based on, for example, the distance from the reference index or the orientation with respect to the vector u. Specifically, the indexes for the other lattice points may be consecutive numbers set in order of the distance from the reference index to the nearest one.

[0074] The detailed shape acquisition unit 250 sets the same index to lattice points indicating corresponding positions between meshes acquired from each of a plurality of different cardiac CT images. By setting the same index to corresponding lattice points between a plurality of meshes, the user can easily associate and observe the meshes of the different cardiac CT images. The detailed shape acquisition unit 250 uses the information of the set index as information about the mesh. The result is stored in the storage unit 114 or the like.

[0075] (S360:Display) In step S360, the display control unit 260 controls the display unit 150 to display the cardiac CT image, the initial shape, the converted image, the detailed shape, and the like acquired in steps S310, S320, S345, and S350, respectively.

[0076] The display control unit 260 displays the initial shape based on, for example, the position and orientation information of the fossa ovalis acquired in step S340. When displaying a plurality of different cardiac CT images, the display control unit 260 can display the initial shape of each fossa ovalis by unifying the position and orientation of the fossa ovalis between the cardiac CT images by using the position and orientation information of the fossa ovalis acquired from each cardiac CT image.

[0077] The display control unit 260 generates an image to be displayed on the display unit 150 based on the cardiac CT image and the detailed shape, and displays the generated image on the display unit 150. For example, the display control unit 260 generates an image by performing CG rendering of the area acquired as the detailed shape of the fossa ovalis, and displays it on the display unit 150. The display control unit 260 generates and displays an image based on the position and orientation information of the fossa ovalis acquired in step S340, thereby making it possible to uniformly display the fossa ovalis in a similar position and orientation between different cardiac CT images. This allows the user to easily compare and understand the fossa ovalis between different cardiac CT images.

[0078] The display control unit 260 has the same functions as a general medical image viewer. The display control unit 260 selects a 2D (two-dimensional) slice image from the cardiac CT image in response to a user input, generates an image in which a detailed shape of the fossa ovalis corresponding to the selected slice image is superimposed on the slice image, and displays the image on the display unit 150.

[0079] The display control unit 260 may display a detailed shape or a transformed image based on the position and orientation information of the fossa ovalis acquired in step S340. The display control unit 260 can display transformed images in which a plurality of different cardiac CT images are transformed based on the position and orientation information of the respective fossa ovalis, with the transformed images being unified in position and orientation. Furthermore, the display control unit 260 can display a detailed shape with the position and orientation of each fossa ovalis being unified based on the position and orientation information of the fossa ovalis.

[0080] The display control unit 260 may display a mesh or a transformed image representing the shape of the fossa ovalis based on the position and orientation information of the fossa ovalis acquired in step S340. The display control unit 260 can display transformed images in which a plurality of different cardiac CT images are transformed based on the position and orientation information of the respective fossa ovalis, thereby unifying the positions and orientations of the fossa ovalis between the transformed images. Furthermore, the display control unit 260 can display a mesh representing the shape of the fossa ovalis by unifying the positions and orientations of the respective fossa ovalis based on the position and orientation information of the fossa ovalis.

[0081] When the detailed shape acquired in step S350 is a mesh, the display control unit 260 may display the meshes acquired from different cardiac CT images in association with the positions and indexes of the mesh lattice points. The display control unit 260 can acquire information on the index of each lattice point from information on the mesh stored in the storage unit 114 or the like. By displaying the meshes acquired from different cardiac CT images in association with the positions and indexes of the lattice points, the user can easily compare and observe the meshes obtained from different cardiac CT images.

[0082] When associating the positions and indices of lattice points between different meshes, the display control unit 260 may draw an index corresponding to each lattice point of the mesh. The index is The indexes may be displayed at positions that do not interfere with the visibility of the detailed shapes, for example, around the corresponding grid points. By displaying the indexes of each grid point, the user can more clearly grasp the correspondence of the detailed shapes of the fossa ovalis between different cardiac CT images.

[0083] Furthermore, the display control unit 260 may change the drawing mode of the mesh according to the index of the grid point of the mesh. Specifically, the display control unit 260 draws the mesh by changing the color and width of the lines and points representing the mesh according to the index value of the grid point.

[0084] The detailed shape acquisition unit 250 may store the detailed shape information acquired in step S350 in the storage unit 114 of the information processing device 100, or in the data server 130 via the network 120. The stored detailed shape information can be used to display the shape of the fossa ovalis in any other medical image viewer, or can be read and used in any other surgery support software, etc. That is, the display process in step S360 is performed in an external device other than the information processing device 100.

[0085] Information on the detailed shape of the fossa ovalis may be used in the analysis process without being displayed or stored. The information processing device 100 may acquire or estimate information on, for example, measurement values ​​(thickness, diameter, etc.) related to the shape of the fossa ovalis and attribute values ​​(patent foramen ovale, atrial septal defect, etc.) related to the state of the fossa ovalis based on the information on the detailed shape. The information processing device 100 may display or store the results of the analysis process.

[0086] 3, it is not necessary to perform the acquisition of a converted image in step S345 and the acquisition of a detailed shape in step S350. It is sufficient for the information processing device 100 to convert the cardiac CT image using the position and orientation information acquired in step S340 and display an image in which the fossa ovalis is unified in a similar position and orientation.

[0087] Furthermore, the information processing device 100 can perform various analysis processes using the acquired position and orientation information. As the analysis process, for example, the information processing device 100 can calculate a relative positional relationship (distance and angle) between the fossa ovalis and the center of gravity of the mitral valve annulus based on the position and orientation information of the fossa ovalis.

[0088] Furthermore, the information processing device 100 may store or display the converted image acquired in step S345, or may analyze it with any other medical image processing software. The information processing device 100 may analyze the converted image to acquire and display measurement values ​​related to the shape of the fossa ovalis and attribute values ​​related to the state of the fossa ovalis.

[0089] According to the first embodiment, the information processing device 100 can accurately obtain three-dimensional orientation information of the cardiac septum from image information. Therefore, the information processing device 100 can obtain and display images in which the orientation of the septum is consistent between different cardiac CT images.

[0090] (Modification 1 of the First Embodiment: Variations of Input Images) 3 is a three-dimensional cardiac CT image in the first embodiment, but is not limited to this. The input image may be, for example, a three-dimensional ultrasound image (transcutaneous ultrasound image, transesophageal ultrasound image, etc.), or an image captured by an MRI device, a PET device, a SPECT device, or the like.

[0091] The input images may be time-series images, for example, time-series three-dimensional CT images (four-dimensional CT images). The information processing device 100 executes the processes described in FIG. 3 for the images of the time-series three-dimensional CT images at each time phase, thereby obtaining the position of the fossa ovalis corresponding to each time phase. The information processing device 100 can acquire a detailed shape of the fossa ovalis corresponding to each time phase and display the result as an animation.

[0092] According to the first modification of the first embodiment, the information processing device 100 can display an image in which the position and orientation of the septum (fossa ovalis) are aligned for various input images.

[0093] (Modification 2 of the First Embodiment: Variations of the Partition Wall) In the first embodiment, the septum of the heart, which is the object of acquiring the position and orientation information, is the fossa ovalis (atrial septum) between the left and right atria, but is not limited to this. The septum may be, for example, the ventricular septum between the left and right ventricles, or a wall between any cardiac chamber and blood vessel of the heart.

[0094] Furthermore, the object for acquiring position and orientation information is not limited to the septum of the heart, but may be other parts of the human body (for example, between the lobes of the lungs or liver) or may be other than parts of the human body, such as industrial parts.

[0095] According to the second variant of the first embodiment, the information processing device 100 can accurately acquire information regarding the position and posture of an object other than the partition portion by using information regarding the positions of other reference portions whose relative positional relationship with the object does not change substantially.

[0096] <Second embodiment> In the first embodiment, the information processing device 100 acquires posture information by using a reference orientation of the partition and second position information of a surrounding anatomical structure different from the partition. In the second embodiment, the information processing device 100 acquires posture information by using a reference orientation of the subject (e.g., a direction of the body axis) instead of the second position information.

[0097] The configuration of the information processing system 10 according to the second embodiment is similar to that of the first embodiment described in FIG. 1. Moreover, the functional configuration of the control unit 115 of the information processing device 100 according to the second embodiment is similar to that of the first embodiment described in FIG. 2. Moreover, the processing flow of the information processing device 100 according to the second embodiment is similar to that of the first embodiment described in FIG. 3. Below, the processing of steps S330 and S340 of the second embodiment that differs from that of the first embodiment will be described.

[0098] (S330: Acquiring location information) In step S330, the position acquisition unit 230 acquires first position information on the partition and second position information on the reference portion in the first embodiment, but in the second embodiment, it is sufficient to acquire position information on the partition. The position acquisition unit 230 can acquire position information on the partition in a similar manner to the first position information in the first embodiment. The position acquisition unit 230 does not need to acquire position information on the reference portion. The position acquisition unit 230 outputs the acquired first position information to the detailed shape acquisition unit 250.

[0099] (S340: Acquisition of position and orientation information) In step S340, the position and orientation acquisition section 240 acquires position and orientation information of the fossa ovalis. The position and orientation information of the fossa ovalis includes first position information and orientation information represented by three mutually orthogonal axes. The position and orientation acquisition section 240 can acquire orientation information of the partition based on the initial shape acquired in step S320, the first position information acquired in step S330, and the reference orientation of the subject's body axis. The position and orientation acquisition section 240 outputs the acquired position and orientation information to the converted image acquisition section 245.

[0100] In the first embodiment, the position and orientation acquisition unit 240 detects an anatomical structure (a reference structure) different from the septum. In contrast to this, in the second embodiment, the position and orientation acquisition section 240 acquires the orientation information of the septum using the image coordinate axes of the cardiac CT image indicating the body axis direction of the subject instead of the second position information.

[0101] The process of acquiring orientation information of the fossa ovalis in the second embodiment will be described with reference to Fig. 5. The position and orientation acquisition unit 240 acquires a vector u indicating the reference orientation of the fossa ovalis in the same manner as in the first embodiment. The position and orientation acquisition unit 240 acquires a vector v' that is orthogonal to the vector u by taking the cross product of the vector u and the Z axis (reference orientation of the subject), which is approximately the same direction as the body axis direction from the head to the feet, among the image coordinate axes (XYZ axes) of the cardiac CT image.

[0102] Since the cardiac CT image is captured in a posture in which the body axis of the subject is parallel to the bed of the CT apparatus, the Z axis of the image coordinate axis is approximately parallel to the body axis. Therefore, the position and orientation acquisition unit 240 can use the Z axis as an approximate body axis. In the case of a healthy subject, the fossa ovalis takes a similar posture with respect to the body axis of the subject between different subjects, so the position and orientation acquisition unit 240 can use the body axis of the subject to calculate the posture information of the fossa ovalis, similar to the second position information used in the first embodiment. The position and orientation acquisition unit 240 can acquire three mutually orthogonal axes of vector u, vector v', and vector w' which is the cross product of vector u and vector v', as posture information.

[0103] Although the position and orientation acquisition unit 240 uses the image coordinate axis as the subject's body axis, the body axis may be acquired from the image using a known image processing technique. For example, the position and orientation acquisition unit 240 may extract a vertebral body from the image and set the direction along the center line of the vertebral body as the body axis. In the case of an organ that moves during imaging, such as the heart, motion artifacts occur in the image obtained by imaging the subject, and it may be difficult to accurately acquire position information of the anatomical structure of the organ. On the other hand, in the second embodiment, the reference direction of the subject is acquired based on the image coordinate axis or a part that moves relatively little, such as the vertebral body. Therefore, the position and orientation acquisition unit 240 is less susceptible to motion artifacts caused by the subject's movement, and can stably acquire the reference direction of the subject and accurately acquire the orientation information of the fossa ovalis.

[0104] In the above-described second embodiment, the information processing device 100 acquires position and orientation information of the partition based on position information on the partition, the reference orientation of the partition, and the reference orientation of the subject. The information processing device 100 can stably acquire position and orientation information of the partition by using the direction of the subject's body axis as the reference orientation of the subject.

[0105] The above-described embodiment (including the modified examples) is merely an example, and the present invention can be modified and changed in various ways within the scope of the gist of the present invention. The present invention also includes configurations obtained by appropriately combining the configurations of the above-described embodiment.

[0106] <Other embodiments> The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions.

[0107] The disclosure of this embodiment includes the following configuration, method, and program. (Configuration 1) an image acquisition means for acquiring an image of a partition wall separating a pair of cavities constituting the heart and a reference site located at a position different from the partition wall; a first shape acquisition means for acquiring information regarding a first shape of the partition portion based on the image; an orientation acquisition means for acquiring information regarding a reference orientation of the partition portion based on the first shape; a position acquisition means for acquiring first position information regarding the partition portion and second position information regarding the reference portion based on the image; a position and orientation acquisition means for acquiring information regarding a position and orientation of the partition portion based on the first position information, a reference orientation of the partition portion, and the second position information; 13. An information processing device comprising: (Configuration 2) The image is a three-dimensional image, and is an image capturing the fossa ovalis as the septum portion and the valve cusps constituting the heart valve as the reference portion. 2. The information processing device according to configuration 1. (Configuration 3) The first shape is the shape of the partition region. 3. The information processing device according to configuration 1 or 2. (Configuration 4) The reference orientation is the orientation of the normal to the first shape. 4. The information processing device according to any one of configurations 1 to 3. (Configuration 5) The first position information is coordinates indicating a center of gravity of the partition region, and the second position information is coordinates indicating a center of gravity of the reference portion region. 5. The information processing device according to any one of configurations 1 to 4. (Configuration 6) The information regarding the position and orientation of the partition includes the first position information and orientation information represented by three mutually orthogonal axes. 6. The information processing device according to any one of configurations 1 to 5. (Configuration 7) The image processing device further includes a converted image acquiring means for acquiring a converted image obtained by converting the position and the orientation of the image based on information regarding the position and the orientation of the partition portion. 7. The information processing device according to any one of configurations 1 to 6. (Configuration 8) The display control means further includes a display control means for displaying the first shape based on information regarding the position and attitude of the partition portion. 8. The information processing device according to any one of configurations 1 to 7. (Configuration 9) The method further includes a second shape acquisition means for acquiring information about a second shape of the partition portion, which is different from the first shape of the partition portion, based on the converted image. 8. The information processing device according to configuration 7. (Configuration 10) The display control means further includes a display control means for displaying the second shape or the converted image based on information regarding the position and orientation of the partition portion. 10. The information processing device according to configuration 9. (Configuration 11) The second shape acquisition means acquires information regarding a third shape of the partition wall based on the second shape or the converted image. 10. The information processing device according to configuration 9. (Configuration 12) The display control means further includes a display control means for displaying the third shape or the converted image based on information regarding the position and attitude of the partition portion. 12. The information processing device according to configuration 11. (Configuration 13) The third shape is a mesh that represents the shape of the partition portion. 13. The information processing device according to configuration 12. (Configuration 14) The display control means controls the display of the indexes set for each of the plurality of grid points of the mesh. 14. The information processing device according to configuration 13. (Configuration 15) The second shape acquisition means sets the same index to the lattice points indicating corresponding positions between the third shapes acquired from each of the plurality of images. 15. The information processing device according to configuration 14. (Configuration 16) The first shape acquisition means acquires information about the first shape based on a thickness of the partition portion. 16. The information processing device according to any one of configurations 1 to 15. (Configuration 17) The first shape acquisition means acquires information about the first shape based on a statistical shape model that represents a statistical tendency of the shape of the partition portion. 16. The information processing device according to any one of configurations 1 to 15. (Configuration 18) The image includes an image captured by at least one of a CT (Computed Tomography) device, an MRI (Magnetic Resonance Imaging) device, a PET (Positron Emission Tomography) device, and a SPECT (Single Photon Emission Computed Tomography) device. 18. The information processing device according to any one of configurations 1 to 17. (Configuration 19) an image acquiring means for acquiring an image of a partition wall separating a pair of cavities constituting the heart of a subject; a shape acquisition means for acquiring information regarding a shape of the partition portion based on the image; an orientation acquisition means for acquiring information on a reference orientation of the partition based on a shape of the partition and acquiring information on a reference orientation of the subject based on the image; a position acquisition means for acquiring position information regarding the partition wall based on the image; a position and orientation acquiring means for acquiring information regarding a position and orientation of the partition portion based on position information regarding the partition portion, a reference orientation of the partition portion, and a reference orientation of the subject; 13. An information processing device comprising: (Method 1) acquiring an image of a partition wall separating a pair of cavities constituting the heart and a reference site located at a position different from the partition wall; acquiring information about a first shape of the partition portion based on the image; acquiring information regarding a reference orientation of the partition portion based on the first shape; acquiring first position information regarding the partition portion and second position information regarding the reference portion based on the image; acquiring information about a position and attitude of the partition portion based on the first position information, a reference orientation of the partition portion, and the second position information; 13. An information processing method comprising: (Method 2) acquiring an image of a partition wall separating a pair of cavities constituting a heart of a subject; acquiring information about a shape of the partition wall based on the image; The information on the reference orientation of the partition wall is obtained based on the shape of the partition wall, and acquiring information on a reference orientation of the object by using the acquiring position information regarding the partition wall based on the image; acquiring information about a position and orientation of the partition based on position information about the partition, a reference orientation of the partition, and a reference orientation of the subject; 13. An information processing method comprising: (program) A program for causing a computer to execute each step of the information processing method according to the first or second method. [Explanation of symbols]

[0108] 100: information processing device, 115: control unit 210: input image acquisition unit, 220: initial shape acquisition unit, 230: position information acquisition unit, 240: attitude information acquisition unit

Claims

1. an image acquisition means for acquiring an image of a partition wall separating a pair of cavities constituting the heart and a reference site located at a position different from the partition wall; a first shape acquisition means for acquiring information regarding a first shape of the partition portion based on the image; an orientation acquisition means for acquiring information regarding a reference orientation of the partition portion based on the first shape; a position acquisition unit that acquires first position information regarding the partition portion and second position information regarding the reference portion based on the image; a position and orientation acquiring means for acquiring information regarding a position and orientation of the partition portion based on the first position information, a reference orientation of the partition portion, and the second position information; 13. An information processing device comprising:

2. The image is a three-dimensional image, and is an image in which the fossa ovalis as the septum and the valve cusps constituting the heart valve as the reference portion are captured.

2. The information processing apparatus according to claim 1,

3. The first shape is a shape of the partition region.

3. The information processing apparatus according to claim 1, wherein the information processing apparatus is a computer.

4. The reference orientation is the orientation of the normal to the first shape.

3. The information processing apparatus according to claim 1, wherein the information processing apparatus is a computer.

5. The first position information is coordinates indicating a center of gravity of the partition region, and the second position information is coordinates indicating a center of gravity of the reference portion region.

3. The information processing apparatus according to claim 1, wherein the information processing apparatus is a computer.

6. The information on the position and attitude of the partition includes the first position information and attitude information represented by three mutually orthogonal axes.

3. The information processing apparatus according to claim 1, wherein the information processing apparatus is a computer.

7. The image processing device further includes a converted image acquiring means for acquiring a converted image obtained by converting the position and the orientation of the image based on information regarding the position and the orientation of the partition portion.

3. The information processing apparatus according to claim 1, wherein the information processing apparatus is a computer.

8. The display control means further includes a display control means for displaying the first shape based on information regarding the position and orientation of the partition portion.

3. The information processing apparatus according to claim 1, wherein the information processing apparatus is a computer.

9. The method further includes: acquiring information about a second shape of the partition portion, which is different from the first shape of the partition portion, based on the converted image.

8. The information processing apparatus according to claim 7,

10. The display control means further includes a display control means for displaying the second shape or the converted image based on information regarding the position and orientation of the partition portion.

10. The information processing apparatus according to claim 9,

11. The second shape acquisition means acquires information regarding a third shape of the partition wall based on the second shape or the converted image.

10. The information processing apparatus according to claim 9,

12. The display control means further includes a display control means for displaying the third shape or the converted image based on information regarding the position and orientation of the partition portion.

12. The information processing apparatus according to claim 11 .

13. The third shape is a mesh that represents the shape of the partition portion.

13. The information processing apparatus according to claim 12.

14. The display control means controls the display of the indexes set for each of the plurality of grid points of the mesh.

14. The information processing apparatus according to claim 13,

15. The second shape acquisition means sets the same index to the lattice points indicating corresponding positions between the third shapes acquired from each of the plurality of images.

15. The information processing apparatus according to claim 14,

16. The first shape acquisition means acquires information about the first shape based on a thickness of the partition portion.

3. The information processing apparatus according to claim 1, wherein the information processing apparatus is a computer.

17. The first shape acquisition means acquires information about the first shape based on a statistical shape model that represents a statistical tendency of the shape of the partition portion.

3. The information processing apparatus according to claim 1, wherein the information processing apparatus is a computer.

18. The image includes an image captured by at least one of a CT (Computed Tomography) device, an MRI (Magnetic Resonance Imaging) device, a PET (Positron Emission Tomography) device, and a SPECT (Single Photon Emission Computed Tomography) device.

3. The information processing apparatus according to claim 1, wherein the information processing apparatus is a computer.

19. an image acquiring means for acquiring an image of a partition wall separating a pair of cavities constituting the heart of a subject; a shape acquisition means for acquiring information regarding a shape of the partition portion based on the image; an orientation acquisition means for acquiring information on a reference orientation of the partition based on a shape of the partition and acquiring information on a reference orientation of the subject based on the image; a position acquisition means for acquiring position information regarding the partition wall based on the image; a position and orientation acquiring means for acquiring information regarding a position and orientation of the partition portion based on position information regarding the partition portion, a reference orientation of the partition portion, and a reference orientation of the subject; 13. An information processing device comprising:

20. acquiring an image of a partition wall separating a pair of cavities constituting the heart and a reference site located at a position different from the partition wall; acquiring information about a first shape of the partition portion based on the image; acquiring information regarding a reference orientation of the partition portion based on the first shape; acquiring first position information regarding the partition portion and second position information regarding the reference portion based on the image; acquiring information about a position and an attitude of the partition portion based on the first position information, a reference orientation of the partition portion, and the second position information; 13. An information processing method comprising:

21. acquiring an image of a partition wall separating a pair of cavities constituting a heart of a subject; acquiring information about a shape of the partition wall based on the image; acquiring information on a reference orientation of the partition based on a shape of the partition, and acquiring information on a reference orientation of the subject based on the image; acquiring position information regarding the partition wall based on the image; acquiring information about a position and orientation of the partition based on position information about the partition, a reference orientation of the partition, and a reference orientation of the subject; 13. An information processing method comprising:

22. A program for causing a computer to execute each step of the information processing method according to claim 20 or 21.

Images