Information processing device, information processing method, and program

Abstract

To improve consistency of shapes of a plurality of valve cusps extracted from a three-dimensional image.SOLUTION: An information processing device includes a crossing information acquisition part for acquiring crossing information of a plurality of planar structures of an object in a three-dimensional image obtained by imaging the object, and a correction part for determining at least one of the plurality of planar structures as a correction object on the basis of the crossing information, and correcting the shape of the planar structure of the correction object.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to an information processing apparatus, an information processing method, and a program.

Background Art

[0002] In the medical field, there is a technique for assisting in grasping the structure and state of a heart valve by extracting a heart valve composed of a plurality of valve leaflets from a three-dimensional image obtained by imaging with various modalities and presenting the shape of the valve leaflets and measurement values of the shape to a user. For example, Non-Patent Document 1 discloses a technique for extracting a heart valve from a three-dimensional image.

Prior Art Documents

Non-Patent Documents

[0003]

Non-Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, due to errors and the like in the valve leaflets extracted from the three-dimensional image, a shape of the valve leaflets in a state different from the actual state where a plurality of valve leaflets cross each other may be extracted. That is, there is a problem that a shape inconsistency occurs between a plurality of valve leaflets.

[0005] The present invention has been made in view of the above problems, and an object thereof is to provide an information processing technology capable of improving the shape consistency between a plurality of valve leaflets extracted from a three-dimensional image.

Means for Solving the Problems

[0006] An information processing apparatus according to one aspect of the present invention has the following configuration. That is, the information processing apparatus includes intersection information acquisition means for acquiring intersection information of a plurality of planar structures of the object in a three-dimensional image obtained by imaging the object, correction means for determining at least one of the plurality of planar structures as a correction target based on the intersection information and correcting the shape of the planar structure to be corrected.

Effect of the Invention

[0007] According to the present invention, it is possible to improve the shape consistency among a plurality of valve tips extracted from a three-dimensional image.

Brief Description of the Drawings

[0008]

Figure 1

Figure 2

Figure 3A

Figure 3B

Figure 3C

Figure 3D

Figure 3E

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Embodiments for Carrying Out the Invention

[0009] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the invention according to the claims. Although a plurality of features are described in the embodiments, not all of these plurality of features are essential to the invention, and the plurality of features may be arbitrarily combined. Further, in the accompanying drawings, the same or similar configurations are denoted by the same reference numerals, and redundant descriptions are omitted.

[0010] <First Embodiment> The information processing apparatus according to the present embodiment extracts a heart valve composed of a plurality of valve tips (planar structures) from a three-dimensional image, and presents the shape of the valve and / or the measured value of the shape to the user, thereby assisting in grasping the structure and state of the heart valve. When the plurality of extracted valve tips intersect each other, this information processing apparatus corrects the shape of the valve tip to obtain a shape of the heart valve with improved consistency without intersection. By displaying the shape of the heart valve with improved consistency in the present embodiment, the user can visually recognize the structure of the heart valve more accurately. Hereinafter, the configuration and processing of the present embodiment will be described with reference to FIG. 1.

[0011] FIG. 1 is a diagram showing the configuration of an information processing system 10 including an information processing apparatus 100 according to the first embodiment. The information processing system 10 includes, as its functional configuration, an information processing apparatus 100, a network 120, and a data server 130. The information processing apparatus 100 is communicably connected to the data server 130 via the network 120. The network 120 includes, for example, a LAN (Local Area Network) or a WAN (Wide Area Network).

[0012] The data server 130 is a Picture Archiving and Communication Systems (PACS) that holds and manages medical images and information associated with the medical images. The information processing device 100 can acquire the medical images held in the data server 130 via the network 120. The data server 130 receives and stores the images captured by a medical imaging device (modality), and transmits the images to each device in response to requests from the devices connected to the network 120. Further, it includes a database capable of storing various data associated with the image together with the image received from the medical imaging device.

[0013] In the present embodiment, the three-dimensional image will be described by taking as an example the image captured by an X-ray CT device, but the three-dimensional image may be captured by other modalities. The modalities include, in addition to the X-ray CT device, for example, an MRI (Magnetic Resonance Imaging) device, a SPECT (Single Photon Emission Computed Tomography) device, a PET (Positron Emission Tomography) device, an ultrasonic diagnostic device, etc., and the information processing device 100 according to the present embodiment is applicable to the three-dimensional images acquired by various modalities.

[0014] The information processing device 100 is a device that performs the image processing according to the embodiment. Further, the information processing device 100 is a device that generates a projection image of the three-dimensional image and causes the display unit 150 to display it, and functions as a reading terminal device operated by a user such as a doctor. The information processing device 100 includes a communication IF (Interface) 111 (communication unit), a ROM (Read Only Memory) 112, a RAM (Random Access Memory) 113, a storage unit 114, and a control unit 115. The information processing device 100 is connected to the instruction unit 140 and the display unit 150.

[0015] The communication IF 111 (communication unit) is composed of, for example, a LAN card and realizes communication between the external device (such as the data server 130) and the information processing apparatus 100. The ROM 112 is composed of a non-volatile memory or the like and stores various programs. The RAM 113 is composed of a volatile memory or the like and temporarily stores various information as data. The storage unit 114 is composed of a storage medium including a storage device such as an HDD (Hard Disk Drive), an SSD (Solid State Drive), or an optical disk device, and stores various information as data.

[0016] The instruction unit 140 is composed of a GUI (Graphical User Interface) such as a keyboard, a mouse, or a touch panel, and inputs an instruction from the user (such as a doctor) to the information processing apparatus 100. The image to be processed is input to the information processing apparatus 100 according to the instruction of the user who operates the instruction unit 140. Note that the selection of the image does not necessarily need to be based on the instruction of the user. For example, the control unit 115 of the information processing apparatus 100 may automatically select the image to be processed based on a predetermined rule.

[0017] FIG. 2 is a diagram showing the functional configuration of the control unit 115. The control unit 115 is composed of a processor such as a CPU (Central Processing Unit) and comprehensively controls the processing in the information processing apparatus 100. In addition to the CPU, the control unit 115 may be composed of a processor such as a GPU (Graphics Processing Unit) that executes a predetermined program, or hardware such as an FPGA (Field Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit). As its functional configuration, the control unit 115 includes an input image acquisition unit 210, an initial shape acquisition unit 220, an intersection information acquisition unit 230, a corrected shape acquisition unit 240, and a display control unit 250.

[0018] The input image acquisition unit 210 acquires an input image to be processed from the data server 130 via the communication IF 111 (communication unit) and the network 120. The initial shape acquisition unit 220 acquires the initial shape of an object (for example, a heart valve) extracted from the input image. The intersection information acquisition unit 230 acquires intersection information of a plurality of valve tips from the initial shape. The intersection information is, for example, a determination result as to whether they intersect or information indicating the intersection position. The corrected shape acquisition unit 240 acquires a corrected shape obtained by correcting the initial shape based on the intersection information. The display control unit 250 generates an image to be displayed on the display unit 150 and the shape of the heart valve, and performs display control to cause the display unit 150 to display them.

[0019] The display unit 150 is constituted by an arbitrary device such as a liquid crystal display (LCD), a CRT (Cathode Ray Tube), a plasma display panel, or an organic EL panel, and displays an image and various information to the user. Specifically, it displays the input image and the projection image acquired from the information processing apparatus 100.

[0020] Each component of the information processing apparatus 100 functions according to a computer program. For example, the control unit 115 (CPU) reads and executes a computer program stored in the ROM 112 or the storage unit 114 or the like with the RAM 113 as a work area, thereby realizing the functions of each component. Note that some or all of the functions of the components of the information processing apparatus 100 may be realized by using dedicated circuits. Also, some of the functions of the components of the control unit 115 may be realized by using a cloud computer.

[0021] For example, an arithmetic unit located at a location different from the information processing apparatus 100 is communicably connected to the information processing apparatus 100 via the network 120, and the information processing apparatus 100 and the arithmetic unit perform data transmission and reception, whereby the functions of the components of the information processing apparatus 100 or the control unit 115 may be realized.

[0022] Next, with reference to FIG. 3A, an example of the processing of the information processing apparatus 100 in FIG. 1 will be described. FIG. 3A is a flowchart showing an example of the processing procedure of the information processing apparatus 100. In the present embodiment, taking the CT image of the mitral valve of the subject as an example, the process of obtaining the corrected shape in which the shape of the mitral valve is corrected will be described. However, the present embodiment is also applicable to images obtained by other modalities and other valves of the heart (aortic valve and tricuspid valve). Further, it is not necessarily limited to valves, and it is also applicable to a part formed from a plurality of planar structures such as the fossa ovalis of the heart.

[0023] (S310: Acquisition of Input Image) In step S310, when the user instructs the acquisition of a three-dimensional image via the instruction unit 140, the input image acquisition unit 210 acquires, from the data server 130, the image (image data) designated by the user as the input image. Then, the acquired input image is output to the initial shape acquisition unit 220 and the display control unit 250.

[0024] (S320: Acquisition of Initial Shape) In step S320, the initial shape acquisition unit 220 acquires the initial shape of the mitral valve of the heart based on the input image acquired in step S310. Then, the acquired initial shape is output to the intersection information acquisition unit 230 and the corrected shape acquisition unit 240.

[0025] In the present embodiment, the initial shape acquisition unit 220 acquires, as the initial shape of the mitral valve, mesh information composed of three-dimensional coordinates of lattice points of m1×n1 points (for example, 19×9 points) indicating the shape of the anterior cusp of the mitral valve, and mesh information composed of three-dimensional coordinates of lattice points of m2×n2 points (for example, 25×9 points) indicating the shape of the posterior cusp.

[0026] Each lattice point in the mesh information is assigned an index, and by specifying the index, the three-dimensional coordinates of a predetermined lattice point can be identified. The index of a lattice point is a label that can identify each lattice point, and for example, numerical values or symbols can be used. The initial shape is mesh information composed of a predetermined number of lattice points and can be obtained by estimating it from the input image using known image analysis techniques. For example, as an example of a method based on machine learning, the initial shape of the mitral valve may be estimated and obtained from the input image using DenseNet (Densely Connected Convolutional Networks), which is a type of convolutional neural network (CNN). Alternatively, the shape of the mitral valve specific to the subject may be estimated using a statistical shape model (mesh information) of the mitral valve, or mesh information may be obtained by converting the region of the mitral valve extracted from the input image into a mesh by using known image processing techniques such as the region expansion method. Further, the user may manually create mesh information using a tool (not shown) via the instruction unit 140, or the configuration may be such that the mesh information stored in advance in the data server is read out and obtained.

[0027] (S330: Acquisition of intersection information) In step S330, the intersection information acquisition unit 230 acquires the intersection information of a plurality of planar structures (for example, a plurality of valve tips) in a three-dimensional image obtained by imaging an object (for example, a heart valve) having a plurality of planar structures. Specifically, the intersection information acquisition unit 230 acquires the intersection information of a plurality of planar structures (a plurality of valve tips, for example, the anterior tip and the posterior tip of the mitral valve) in the object in the three-dimensional image obtained by imaging the object. Here, the intersection information acquisition unit 230 acquires the intersection information using the mesh information composed of a plurality of points having the coordinate information of the plurality of planar structures. That is, the intersection information acquisition unit 230 acquires the intersection information of a plurality of valve tips (anterior tip and posterior tip) based on the initial shape acquired in step S320. Then, the acquired intersection information is output to the corrected shape acquisition unit 240.

[0028] In this embodiment, as an example of the intersection information, a determination result as to whether the anterior tip and the posterior tip of the mitral valve intersect, three-dimensional coordinates of a position (intersection point) where the anterior tip and the posterior tip intersect, and indices of lattice points around the intersection point are acquired.

[0029] FIG. 8 is a diagram schematically illustrating contact, opening, and intersection of the mitral valve. As an example, the anterior tip 810 and the posterior tip 820 of the mitral valve 800 are schematically shown. 8A of FIG. 8 is a diagram schematically showing the mitral valve 800 as viewed from the left atrium side (above in the Z direction), and schematically shows a state in which the anterior tip 810 and the posterior tip 820 are closed (contact state). In the coordinate system of 8A of FIG. 8, the X axis indicates the horizontal axis direction, the Y axis indicates the vertical axis direction, and the Z axis indicates the direction perpendicular to the paper surface.

[0030] 8B of FIG. 8 schematically shows a state in which the mitral valve 800 in a state where the anterior tip 810 and the posterior tip 820 are closed (contact state: 8A of FIG. 8) is viewed from the side (YZ plane). In 8B of FIG. 8, the lower end side of the valve leaf in the Z direction is assumed to indicate the valve orifice side of the mitral valve 800, and the upper end side of the valve leaf in the Z direction is assumed to indicate the annulus side of the mitral valve 800. In 8B of FIG. 8, the anterior tip 810 and the posterior tip 820 are shown in a state of being in contact at the valve orifice.

[0031] 8C of FIG. 8 is a diagram schematically showing the mitral valve 800 as viewed from the annulus side (above in the Z direction), and schematically shows a state in which the anterior tip 810 and the posterior tip 820 are open.

[0032] 8D of FIG. 8 schematically shows a state in which the mitral valve 800 in a state where the anterior tip 810 and the posterior tip 820 are open (8C of FIG. 8) is viewed from the side (YZ plane). In 8D of FIG. 8, the lower end side in the Z direction indicates the valve orifice side of the mitral valve 800, and the upper end side in the Z direction indicates the annulus side of the mitral valve 800. As shown in 8C and 8D of FIG. 8, in a state where the anterior tip 810 and the posterior tip 820 are open, no intersection occurs between the anterior tip 810 and the posterior tip 820.

[0033] 8E of FIG. 8 and 8F of FIG. 8 schematically show a state in which the anterior cusp 810 and the posterior cusp 820 intersect. Although the intersection of a plurality of valve cusps (anterior cusp 810, posterior cusp 820) cannot occur in the actual mitral valve 800, when there are errors in the shape information of a plurality of valve cusps (anterior cusp 810, posterior cusp 820) extracted by image recognition technology of three-dimensional images or the like, a state in which a plurality of valve cusps (anterior cusp 810, posterior cusp 820) intersect, which cannot occur in the actual mitral valve 800, may occur.

[0034] In the information processing apparatus 100 of the present embodiment, based on the initial shape acquired in step S320, the intersection information acquisition unit 230 determines whether an intersection occurs in a plurality of valve cusps (anterior cusp 810, posterior cusp 820). When an intersection occurs, the intersection information acquisition unit 230 acquires the intersection information and outputs the acquired intersection information to the corrected shape acquisition unit 240. The corrected shape acquisition unit 240, which will be described later, corrects the positions of the lattice points around the intersection to improve the shape consistency among a plurality of valve cusps (anterior cusp 810, posterior cusp 820).

[0035] 8E of FIG. 8 is a diagram schematically showing the mitral valve 800 as viewed from the valve annulus side (above in the Z direction), and schematically shows a state in which the anterior cusp 810 and the posterior cusp 820 intersect at the intersection points 830 and 840.

[0036] 8F of FIG. 8 schematically shows the mitral valve 800 in a state where the anterior cusp 810 and the posterior cusp 820 intersect (8E of FIG. 8) as viewed from the side (YZ plane). In 8F of FIG. 8, the lower end side in the Z direction indicates the valve orifice side of the mitral valve 800, and the upper end side in the Z direction indicates the valve annulus side of the mitral valve 800. As shown in 8F of FIG. 8, the anterior cusp 810 and the posterior cusp 820 intersect at the intersection point 850. For example, the lattice point 811 on the valve orifice side of the posterior cusp 820 is located at a position where it penetrates the anterior cusp 810 through the intersection point 850.

[0037] As an example of the correction process, the correction shape acquisition unit 240 described below corrects the position of the grid points on one side of the grid points on the valve port side of the front tip 810 and the grid points on the valve port side of the rear tip 820 so that they match the position of the intersection point 850 where an intersection occurs. In order to prevent the shape of the valve tip from being unnaturally deformed by the correction, the correction shape acquisition unit 240 compares the distances between the intersection point 850 and the grid points (801, 811) at the tip on the valve port side, respectively, and determines the grid point with the shorter distance (for example, the grid point 811 on the valve port side of the rear tip 820) as the correction target. The correction shape acquisition unit 240 moves the position of the grid point 811 in the direction of the arrow 860 so that the position of the grid point 811 at the tip on the valve port side of the rear tip 820 coincides with the position of the intersection point 850. By correcting the position of the grid point 811, the grid point at the tip on the valve port side of the rear tip 820 comes into contact with the front tip 810, and the intersecting state is eliminated. The correction shape acquisition unit 240 performs the same correction process for other intersection points, and by correcting the positions of the grid points, it becomes possible to improve the shape consistency among a plurality of valve tips (front tip 810, rear tip 820). A specific example of the correction process will be described in detail later.

[0038] Indices ID1 to ID3 for specifying three-dimensional coordinates are assigned to the intersection points 830, 840, and 850 shown in 8E of FIG. 8 and 8F of FIG. 8. The positions of the intersection points are calculated based on the initial shape, and are respectively (x 1 , y 1 , z 1 ), (x 2 , y 2 , z 2 ), (x 3 , y 3 , z 3 ). For example, index ID1 is assigned to the three-dimensional coordinate information (x 1 , y 1 , z 1 ), index ID2 is assigned to the three-dimensional coordinate information (x 2 , y 2 , z 2 ), and index ID3 is assigned to the three-dimensional coordinate information ID3 (x 3 , y 3 , z 3) is assigned an index ID3. By specifying index IDs 1 to 3, the three-dimensional coordinate information of each intersection point 830 to 850 can be identified.

[0039] An example of the process of acquiring intersection information will be specifically described with reference to FIGS. 4 and 5. The front tip 400 and the rear tip 410 of the mitral valve shown in FIG. 4 are mesh information as the initial shape of the mitral valve acquired in step S320, and show an example where the front tip 400 and the rear tip 410 intersect. 4A, 4B, and 4C in FIG. 4 are diagrams visualizing the mesh information as the initial shape of the mitral valve from different directions.

[0040] 4A in FIG. 4 is a view of the XY plane looking at the mitral valve from the valve orifice side (downward in the Z direction). At the portion indicated by the broken line 450 in 4A of FIG. 4, the front tip 400 and the rear tip 410 intersect. The portion indicated by the dashed-dotted line 460 in 4A of FIG. 4 shows the portion where the rear tip 410 penetrates from the intersecting portion toward the front tip 400 side. When the front tip 400 and the rear tip 410, which are planar structures, intersect, the intersecting portion becomes a line (linear) in the XY plane, as shown by the broken line 450 for example. The lattice points 411, 412, and 413 on the dashed-dotted line 460 are lattice points at the tip of the rear tip 410 on the valve orifice side, and the lattice points 411, 412, and 413 at the tip of the rear tip 410 on the valve orifice side are in a state of penetrating from the intersecting portion toward the front tip 400 side. The portion where the rear tip 410 penetrates from the intersecting portion toward the front tip 400 side corresponds to a portion equivalent to the lattice point 811 at the tip of the rear tip 820 on the valve orifice side as shown in the schematic diagram of 8F in FIG. 8, taking one lattice point 411 as an example.

[0041] 4B in FIG. 4 is a view of the XY plane looking at the mitral valve from the valve annulus side (upward in the Z direction). In the XY plane looking at the mitral valve from the valve annulus side, the front tip 400 and the rear tip 410 appear to be in contact (closed), but when looking at the mitral valve from the valve orifice side (downward in the Z direction), an intersecting state as shown in 4A of FIG. 4 occurs. In 4B of FIG. 4, near the lattice points 404, 405, and 406 on the valve annulus side of the front tip 400 and the lattice points 414, 415, and 416 on the valve annulus side of the rear tip 410, which are closer to the valve annulus side than the tip of the front tip 400 on the valve orifice side, the respective valve tips are in a state of contact.

[0042] 4C in FIG. 4 is a view of the mitral valve in a state where the anterior leaflet 400 and the posterior leaflet 410 intersect, as seen from the side (YZ plane). The lattice point 411 on the orifice side of the posterior leaflet 410 has penetrated from the intersection point toward the anterior leaflet 400 side. In 4C of FIG. 4, the portion where the posterior leaflet 410 penetrates from the intersecting portion toward the anterior leaflet 400 side (for example, the lattice point 411) corresponds to the portion of the lattice point 811 on the orifice side of the posterior leaflet 820 shown in 8F of FIG. 8.

[0043] FIG. 5 is a view showing an example of a portion where the anterior leaflet 400 and the posterior leaflet 410 of the mitral valve intersect, as seen from the orifice side (downward in the Z direction). FIG. 5 shows the details of the surface of the anterior leaflet 400 composed of the lattice points 401 to 403 on the orifice side of the anterior leaflet 400 and the lattice points 404 to 406 on the annulus side of the anterior leaflet 400, and the surface of the posterior leaflet 410 composed of the lattice points 411 to 413 on the orifice side of the posterior leaflet 410 and the lattice points 414 to 416 on the annulus side of the posterior leaflet 410. In FIG. 5, examples of the intersection points where the anterior leaflet 400 and the posterior leaflet 410 intersect are indicated by intersection points 501 to 504.

[0044] First, the intersection information acquisition unit 230 performs a determination process on whether the anterior leaflet 400 and the posterior leaflet 410 of the mitral valve intersect. Hereinafter, the details of the intersection determination process will be described. FIG. 3B is a view showing an example of the flow of the intersection determination process.

[0045] (Process flow of intersection determination) In step S1, based on the initial shape acquired in step S320, the intersection information acquisition unit 230 acquires the position information of each lattice point of the anterior leaflet 400 and the posterior leaflet 410.

[0046] Next, the intersection information acquisition unit 230 acquires, as intersection information, a position (intersection) where a line segment selected using multiple points of one of the planar structures (e.g., the anterior tip 400) and a surface selected using multiple points of the other planar structure (e.g., the posterior tip 410) intersect. The intersection information acquisition unit 230 also acquires, as intersection information, at least one point (lattice point) located around the intersection position (intersection). Specifically, in order to determine whether the anterior tip 400 and the posterior tip 410, which are planar structures, intersect, the intersection information acquisition unit 230 obtains a line segment consisting of two lattice points and a surface consisting of three lattice points, and determines whether the line segment and the surface intersect.

[0047] In order to obtain a line segment, the intersection information acquisition unit 230 obtains a line segment from two lattice points selected from either the anterior cusp 400 or the posterior cusp 410. In addition, the intersection information acquisition unit 230 obtains a surface from three lattice points selected from the other of the anterior cusp 400 and the posterior cusp 410, and determines whether the line segment and the surface intersect. In the flowchart of FIG. 3B, as an example of processing, the flow of processing is described as selecting two lattice points from the anterior cusp 400 (S2) and selecting three lattice points from the posterior cusp 410 (S3). However, two lattice points may be selected from the posterior cusp 410 (S2) and three lattice points may be selected from the anterior cusp 400 (S3).

[0048] In step S2, the intersection information acquisition unit 230 selects two adjacent points (e.g., lattice points 402 and 405) from among the lattice points (intersections of the lines in FIG. 4) of the mesh forming the anterior cusp 400 shown in FIG. 4. Note that the number of lattice points to be selected is not limited to one pair, and multiple combinations of the lattice points of the two selected points may be selected.

[0049] Next, in step S3, the intersection information acquisition unit 230 selects three adjacent points (for example, lattice points 411, 414, and 415) among the lattice points of the mesh that forms the trailing edge 410. Note that the three selected lattice points are not limited to one set, and a plurality of sets of combinations of the three selected lattice points may be selected. At this time, since the closer the adjacent lattice points are to each other, the closer the line segments or surfaces can approximate the shape of the actual valve tip surface, it is desirable to select lattice points adjacent to each other.

[0050] In step S4, the intersection information acquisition unit 230 determines whether or not a line segment composed of two lattice points on the selected leading edge 400 intersects with a plane composed of three lattice points on the trailing edge 410. The determination of intersection can be made by a known technique from the equation of the straight line passing through the two lattice points and the equation of the plane passing through the three lattice points. Note that when any of the two lattice points constituting the line segment is a lattice point on the valve port, it may be determined that they are in contact rather than intersecting.

[0051] In the determination process of step S4, when the line segment composed of two lattice points intersects with the plane specified by the three lattice points (S4 - YES), the process proceeds to step S5. Then, in step S5, the intersection information acquisition unit 230 acquires information (three - dimensional coordinate information) regarding the intersection point as intersection information. Specifically, the intersection information acquisition unit 230 acquires the three - dimensional coordinates of the intersection point and the indices of the lattice points around the intersection point. For example, when the line segment composed of lattice points 402 and 405 intersects with the plane composed of lattice points 411, 414, and 415 at the intersection point 502, the three - dimensional coordinates of the intersection point 502 are acquired. Further, the indices of the lattice points 402, 405, 411, 414, and 415 located around the intersection point 502 are acquired. By acquiring the indices of the lattice points, the intersection information acquisition unit 230 can specify the three - dimensional coordinate information of the lattice points 402, 405, 411, 414, and 415 located around the intersection point 502.

[0052] On the other hand, in the determination process of step S4, when the intersection information acquisition unit 230 determines that the line segment and the plane do not intersect by the determination process of step S4 (S4-NO), the process proceeds to step S6.

[0053] In step S6, the intersection information acquisition unit 230 determines whether the intersection determination has been completed for the combination of planes specified by the selected three lattice points. If the intersection determination has not been completed (S6-NO), the process proceeds to step S8.

[0054] In step S8, the intersection information acquisition unit 230 selects the next three lattice points from the combinations of the plurality of sets of lattice points selected in step S3, returns the process to step S4, and repeats the same process (S4, S5).

[0055] In the determination process of step S6, when the intersection determination has been completed for the combination of the selected three lattice points (S6-YES), the intersection information acquisition unit 230 proceeds the process to step S7.

[0056] In step S7, the intersection information acquisition unit 230 determines whether the intersection determination has been completed for the combination of the selected two lattice points. If the intersection determination has not been completed (S7-NO), the process proceeds to step S9.

[0057] In step S9, the intersection information acquisition unit 230 selects the next two lattice points from the combinations of the plurality of sets of lattice points selected in step S2, returns the process to step S3, and repeats the same process (S3, S4, S5).

[0058] In the determination process of step S7, when the intersection determination has been completed for the combination of the selected two lattice points (S7-YES), the intersection information acquisition unit 230 ends the process of FIG. 3B.

[0059] Then, the intersection information acquisition unit 230 similarly performs intersection determination processing for all combinations regarding the combinations of two points of the rear tip 410 and three points of the front tip 400 determined in advance. In this case, the intersection information acquisition unit 230 may select two lattice points from the rear tip 410 (S2) and three lattice points from the front tip 400 (S3).

[0060] Then, the intersection information acquisition unit 230 acquires, as intersection information, the three-dimensional coordinates of the intersection point and the indexes of the lattice points around the intersection point, and advances the process to step S340. By switching the selection of the lattice points specifying the line segment and the selection of the lattice points specifying the plane at the front tip 400 and the rear tip 410 and performing intersection determination respectively, the accuracy of the intersection determination can be improved.

[0061] In the present embodiment, as the intersection determination process, it has been described to determine whether or not there is an intersection between the line segment forming the front tip 400 and the plane forming the rear tip 410, and to determine whether or not there is an intersection between the line segment forming the rear tip 410 and the plane forming the front tip 400. Note that the intersection determination process is not limited to this example, and the determination of intersection may be made only between the line segment specified by two lattice points at one of the valve tips of the front tip 400 and the rear tip 410 and the plane specified by three lattice points at the other valve tip.

[0062] In this embodiment, determination of whether or not there is an intersection was performed for all combinations of two lattice points of the anterior cusp 400 and three lattice points of the posterior cusp 410 that were determined in advance. However, the intersection determination process may be terminated when one or more intersections are obtained. Similarly, in the combination of two lattice points of the posterior cusp 410 and three lattice points of the anterior cusp 400, the intersection determination process may also be terminated when one or more intersections are obtained. If at least one intersection is obtained without performing intersection determination for all combinations of lattice points, it becomes possible to correct the positions of the lattice points related to this intersection. Then, in the next step S340, the positions of the lattice points may be corrected, and intersection determination may be performed again for the shape based on the corrected lattice points. Thereby, it becomes possible to shorten the time required for the intersection determination process and the correction of the positions of the lattice points.

[0063] Also, since the distance between the anterior cusp and the posterior cusp near the valve annulus of the mitral valve tends to be greater than that near the valve orifice, the lattice points near the valve annulus may not be subjected to the process of determining intersections, and only the lattice points near the valve orifice may be targeted for intersection determination. For example, the intersection information acquisition unit 230 may acquire intersection information in the region specified by the region setting information among a plurality of planar structures (a plurality of valve cusps). Specifically, in order to specify the region near the valve annulus and the region near the valve orifice of the mitral valve, a quantitative distance may be set to specify the positions near the valve annulus and the valve orifice. For example, according to the first region setting information, a predetermined range (first range) may be specified as the region near the valve annulus from one end side (the upper end side in the Z direction) of the valve cusp, which is a planar structure, and the lattice points located in the predetermined range may be excluded from the targets of intersection determination. Also, according to the second region setting information, a predetermined range (second range) may be specified as the region near the valve orifice from the other end side (the lower end side in the Z direction) of the valve cusp, and the lattice points located in the predetermined range may be targeted for intersection determination.

[0064] Note that regardless of the setting of the quantitative distance, the length of the valve cusp in the Z direction (for example, the distance L from the valve orifice to the valve annulus) may be set and divided at the position that is half of this distance L. For example, the upper half may be set as the region near the valve annulus, and the lower half may be set as the region near the valve orifice.

[0065] The intersection information acquisition unit 230 may acquire intersection information in a region specified based on a parameter for dividing a plurality of planar structures (a plurality of valve leaflets) into a predetermined length. For example, the intersection information acquisition unit 230 may acquire intersection information in a region specified based on the product of the lengths of the plurality of planar structures and the parameter. Specifically, a parameter m (0 < m < 1) for dividing the distance L in the Z direction into a region near the valve annulus and a region near the valve orifice may be set. For example, the region specified by the distance L×m in the Z direction may be regarded as the region near the valve orifice. Then, the region obtained by removing the region specified as the region near the valve orifice (the distance L×m in the Z direction) from the distance L in the Z direction may be regarded as the region near the valve annulus. The parameter m can be arbitrarily set, and by changing the setting of the parameter m, the region to be the object of intersection determination can be arbitrarily adjusted.

[0066] Further, the parameter m is not limited to the mitral valve only, and different parameters may be set according to other valve types (for example, the aortic valve, tricuspid valve, pulmonary valve, etc. composed of three valve leaflets).

[0067] Then, the intersection information acquisition unit 230 acquires, as intersection information, the three-dimensional coordinates of at least one intersection point and the index of the grid points around the intersection point, and proceeds to step S340. According to this, the process of acquiring intersection information can be executed at high speed.

[0068] In this embodiment, a method for specifying the presence or absence of intersection and the intersection position using the geometric positional relationship between line segments and planes represented by combinations thereof based on the positions of the grid points constituting the mesh information forming the anterior leaflet 400 and the posterior leaflet 410 has been described. However, the intersection determination method in this embodiment is not limited to this example, and the presence or absence of intersection between the anterior leaflet 400 and the posterior leaflet 410 may be determined using the pixel values of the mask image.

[0069] For example, based on the mesh information forming the front tip 400 and the rear tip 410, the positions where the front tip 400 and the rear tip 410 are respectively located may be represented in the form of a mask image, and the intersection may be determined based on the overlap of the foreground pixels in the mask image. More specifically, a mask image representing the region of the front tip 400 and a mask image representing the region of the rear tip 410 are generated. As the mask image of the front tip 400, a binary mask image is generated based on the mesh information, where the pixel value at the position of the pixel indicating the presence of the front tip 400 is set to "1" (foreground pixel), and the pixel values at other positions are set to "0" (background pixel). In the mask image of the front tip 400, the pixel value "0" is set in the region of the rear tip 410.

[0070] Similarly for the mask image of the rear tip 410, a binary mask image is generated based on the mesh information, where the pixel value at the position of the pixel indicating the presence of the rear tip 410 is set to "1" (foreground pixel), and the pixel values at other positions are set to "0" (background pixel). In the mask image of the rear tip 410, the pixel value "0" is set in the region of the front tip 400.

[0071] Then, the intersection information acquisition unit 230 acquires intersection information at positions where the position of the pixel indicating the presence of one planar structure (front tip 400) and the position of the pixel indicating the presence of the other planar structure (rear tip 410) overlap in the object. That is, the intersection information acquisition unit 230 compares the pixel values in the mask image of the front tip 400 and the mask image of the rear tip 410, and determines whether there is a position where the pixel values of both mask images are "1" at the same position. In both mask images, the pixel values at the same position should be set to either "0" or "1", but at the position where the pixel values at the same position in both mask images are "1", it is the position where the front tip 400 and the rear tip 410 intersect.

[0072] For example, in 8F of FIG. 8, in the mask image of the anterior cusp 810, the pixel value at the position where the anterior cusp 810 exists is "1", and the pixel value at the position where the posterior cusp 820 exists is "0". Also, in the mask image of the posterior cusp 820, the pixel value at the position where the posterior cusp 820 exists is "1", and the pixel value at the position where the anterior cusp 810 exists is "0". In both mask images, the pixel value at the intersection 850 is "1".

[0073] The intersection information acquisition unit 230 acquires intersection information at a position where the pixel value "1" indicating the position where one planar structure (for example, the anterior cusp 400) exists in the object overlaps with the pixel value "1" indicating the position where the other planar structure (for example, the posterior cusp 410) exists. That is, the intersection information acquisition unit 230 compares the pixel values in the mask image of the anterior cusp 400 and the mask image of the posterior cusp 410, and determines whether there is a position where the pixel value becomes "1" at a common position. The intersection information acquisition unit 230 specifies the position where the pixel value becomes "1" at the common position as the intersection point. The intersection information acquisition unit 230 determines the presence or absence of an intersection based on the overlap of the foreground pixels in the mask image, acquires the three-dimensional coordinates of the intersection point and the index of the grid points around the intersection point, and proceeds to step S340 for processing.

[0074] (S340: Acquisition of corrected shape) In step S340, the corrected shape acquisition unit 240 acquires a corrected shape obtained by correcting the initial shape based on the intersection information acquired in step S330. Then, the acquired corrected shape is output to the display control unit 250. When the planar structure to be corrected intersects with another planar structure, the corrected shape acquisition unit 240 corrects the planar structure to be corrected to a shape that does not intersect with the other planar structure. For example, when an intersection occurs between one planar structure (anterior cusp 400) and the other planar structure (posterior cusp 410) based on the intersection information, the corrected shape acquisition unit 240 corrects the position of the end (tip) of either one of the planar structures to match the position of the intersection point.

[0075] In the present embodiment, the correction shape acquisition unit 240 determines which of the lattice points of the anterior cusp 400 and the posterior cusp 410 of the mitral valve is to be corrected as the cusp to be corrected.

[0076] At this time, in order to prevent the shape of the cusp from being unnaturally deformed by the correction, based on the intersection information acquired in step S330, the correction is performed so as to keep the amount of change in the shape due to the correction low. For example, when the three-dimensional coordinates of the intersections 501 to 504 and the indices of the lattice points around each intersection (both ends of the line segment passing through the intersection) as shown in FIG. 5 are acquired, the correction shape acquisition unit 240 calculates the distance from each intersection to the lattice point on the valve orifice side of the mitral valve among the lattice points at both ends of the line segment passing through the intersection, and corrects the position of the lattice point with the minimum distance.

[0077] In FIG. 5, the lattice points on the valve orifice side (the tip of the valve orifice) of the posterior cusp 410 are lattice points 411, 412, and 413 (hereinafter also referred to as the lattice points of the valve orifice of the posterior cusp 410). Lattice points 414, 415, and 416 are lattice points connected from the lattice points 411, 412, and 413 on the valve orifice side of the posterior cusp 410 to the annulus side (left atrium side), and are not lattice points on the valve orifice side.

[0078] Also, the lattice points on the valve orifice side (the tip of the valve orifice) of the anterior cusp 400 are lattice points 401, 402, and 403 (hereinafter also referred to as the lattice points of the valve orifice of the anterior cusp 400). Lattice points 404, 405, and 406 are lattice points connected from the lattice points 401, 402, and 403 on the valve orifice side of the anterior cusp 400 to the annulus side (left atrium side), and are not lattice points on the valve orifice side. In the following description, the lattice points on the valve orifice side are also referred to as the lattice points at the tip of the valve orifice.

[0079] The correction shape acquisition unit 240 acquires a first distance D1 between a first intersection point 503 included in the intersection information and a point 412 on the valve port side among the line segments passing through the first intersection point 503. Further, the correction shape acquisition unit 240 acquires a second distance D2 between a second intersection point 501 included in the intersection information and a point 411 on the valve port side among the line segments passing through the second intersection point 501. The points 411 and 412 are points at the end of the other planar structure (the rear tip 410). Also, the correction shape acquisition unit 240 acquires a third distance between a third intersection point 502 included in the intersection information and a point 402 at the end of one planar structure (the front tip 400) among the line segments passing through the third intersection point. Further, the correction shape acquisition unit 240 acquires a fourth distance between a fourth intersection point 504 included in the intersection information and a point 403 at the end of one planar structure (the front tip 400) among the line segments passing through the fourth intersection point 504.

[0080] Then, the correction shape acquisition unit 240 may determine, among the first distance to the fourth distance, for example, by comparing the first distance and the third distance, or by comparing the second distance and the fourth distance, the planar structure with the shorter distance as the object of correction. The correction shape acquisition unit 240 may determine, by comparing the first distance and the third distance, the planar structure with the shorter distance as the object of the correction, or alternatively, the correction shape acquisition unit 240 may determine, by comparing the second distance and the fourth distance, the planar structure with the shorter distance as the object of the correction. Examples of the combination of distances are not limited to this example. For example, the first distance and the fourth distance may be compared, or the second distance and the third distance may be compared.

[0081] The correction shape acquisition unit 240 determines, among a plurality of points 411 and 412 at the end of the planar structure to be corrected (for example, the rear tip 410), the point at the end with the shorter distance (for example, 412) as the object of correction by comparing the distances. The correction shape acquisition unit 240 corrects, for example, when the first distance is the shortest distance, the position of the point (for example, 412) at the end of the other planar structure to match the position of the first intersection point 503. Although an example of acquiring the first distance to the fourth distance has been described, it is not limited to this example. At least, distances obtained by any one combination of two distances, that is, the first distance and the third distance, or the second distance and the fourth distance, may be acquired.

[0082] Taking the intersection point 503 shown in FIG. 5 as an example, a specific example of the distance from the intersection point will be described. Among the grid points 412 and 415 at both ends of the line segment passing through the intersection point 503, the grid point on the valve port side is the grid point 412. The correction shape acquisition unit 240 calculates the distance D1 between the grid point 412 on the valve port side and the intersection point 503.

[0083] Similarly for the other intersection point 501, among the grid points 411 and 414 at both ends of the line segment passing through the intersection point 501, the grid point on the valve port side is the grid point 411. The correction shape acquisition unit 240 calculates the distance D2 between the grid point 411 on the valve port side and the intersection point 501.

[0084] The correction shape acquisition unit 240 similarly calculates the distance D3 between the other intersection point 502 and the grid point 402, and the distance D4 between the intersection point 504 and the grid point 403. For example, when the distance D1 between the grid point 412 and the intersection point 503 is the smallest, the correction shape acquisition unit 240 determines to correct the posterior cusp 410 and also determines to correct the grid point 412 of the posterior cusp 410. The correction shape acquisition unit 240 corrects from the grid point where the distance between the intersection point and the grid point on the valve port side is the smallest in order to prevent the shape of the valve cusp from being unnaturally deformed by the correction.

[0085] FIG. 6 is a diagram schematically illustrating the correction process of the valve cusp of the mitral valve in the first embodiment. Using FIG. 6, the process of correcting the initial shape will be described. The correction shape acquisition unit 240 moves the grid point 412 of the posterior cusp 410 determined as the correction target to the position of the intersection point 503 to correct the shape. In FIG. 6, the corrected grid point after correcting the position of the grid point 412 is shown as the grid point 612. By correcting the grid point 412 (the corrected grid point 612) by the correction shape acquisition unit 240, the grid point 612 at the tip of the valve port in the posterior cusp 410 comes into contact with the anterior cusp 400, and a corrected shape in which the intersection occurring at the intersection point 503 is eliminated can be obtained.

[0086] Then, by performing the same correction at other intersection points, a corrected shape in which the intersections at each intersection point are eliminated can be obtained.

[0087] On the other hand, by performing the correction by the above processing, the shapes of the cusp 400 and the cusp 410 may change, and new intersections other than the intersections obtained in step S330 may occur. After the correction shape acquisition unit 240 corrects the position of the grid point on the valve port side (for example, the grid point 412 described above) with the minimum distance from the intersection to obtain the first correction shape, the intersection information acquisition unit 230 may use the first correction shape corrected by the correction shape acquisition unit 240 as the initial shape and execute again the process of obtaining the intersection information in step S330.

[0088] FIG. 3C is a diagram showing an example of the flow of the correction shape acquisition process in step S340. First, in step S11, the correction shape acquisition unit 240 acquires a correction shape (for example, the first correction shape) by the above-described correction process.

[0089] Then, in step S12, the intersection information acquisition unit 230 executes again the process of acquiring intersection information with the correction shape (for example, the first correction shape) as the initial shape. Here, the process of acquiring intersection information is based on the process described in step S330.

[0090] In step S13, the correction shape acquisition unit 240 determines whether intersection information has been acquired in step S12. If intersection information has been acquired (S13 - YES), the process returns to step S11. Then, in step S11, the correction shape acquisition unit 240 acquires a second correction shape obtained by further correcting the first correction shape based on the newly acquired intersection information by the intersection information acquisition unit 230, and the processes of steps S11 to S13 may be repeatedly executed.

[0091] If intersection information has not been acquired in the determination process of step S13 (S13 - NO), the correction shape acquisition unit 240 ends the process.

[0092] That is, steps S330 and S340 may be configured to be repeatedly executed until a determination result is obtained in step S330 that no line segment intersects the plane (S13-NO). In this way, the front tip and the rear tip of the mitral valve where no intersection occurs at any position can be obtained.

[0093] Alternatively, the maximum number of times of repeatedly executing steps S330 and S340 may be set in advance, and when the maximum number of times is reached, the correction process may be terminated, and the process may proceed to step S350.

[0094] When the intersection information acquisition unit 230 acquires the intersection information again, the points used for selecting the line segment specified by two points and the plane specified by three points are limited to the points (lattice points) moved by correction and the points (lattice points) located around the point, and the intersection information is acquired. That is, when the intersection information acquisition process is repeatedly executed, the lattice points forming the line segment and the plane used for the intersection determination in step S330 may be limited to the lattice points moved in step S340 and the lattice points around them. This enables the acceleration of the process in step S340.

[0095] In the present embodiment, an example in which one lattice point is corrected has been described. However, two or more lattice points including the lattice point on the valve orifice side with the minimum distance from the intersection point and the lattice points around it may be used as the lattice points to be corrected. The correction shape acquisition unit 240 corrects the position of the point (for example, lattice point 412) at the end of the planar structure (rear tip 410) to be corrected by the first movement amount D1, and corrects the nearby points (for example, lattice points 411, 413, 414, 415, and 416) located around the point (lattice point 412) at the end of the planar structure (rear tip 410) in the same direction as the movement direction of the point (lattice point 412) by the second movement amount (w×D1) set based on the first movement amount D1 and a predetermined weight coefficient w for the first movement amount.

[0096] For example, in FIG. 5, when the lattice point 412 is to be moved as the correction target based on the first movement amount D1, the positions of the neighboring points (lattice points 411, 413, 414, 415, and 416) located around the lattice point 412 to be corrected may be corrected by moving them with a second movement amount (w×D1) set based on different weight coefficients according to the distance between the lattice point 412 and the neighboring points.

[0097] At this time, for each surrounding lattice point located around the lattice point 412 to be corrected, the correction shape acquisition unit 240 can determine the movement amount according to the distance from the lattice point 412. For example, when the movement amount (correction amount) of the lattice point 412 is D1 (the distance between the lattice point 412 and the intersection point 503), the correction shape acquisition unit 240 can set the movement amount (second movement amount) of the surrounding lattice points based on the weight coefficient (w) of the movement amount and the movement amount (first movement amount D1) of the lattice point to be corrected (w×D1). Here, the weight coefficient w of the movement amount is, for example, a coefficient of 0 or more and less than 1, and as the weight coefficient w of the movement amount, a value based on the distance between the lattice point 412 and the surrounding lattice points (for example, a weight coefficient inversely proportional to the distance) may be set.

[0098] Alternatively, the correction shape acquisition unit 240 may set the weight coefficient w of the movement amount according to the connection relationship between the lattice point 412 to be corrected and the surrounding lattice points.

[0099] Here, FIG. 9 is a diagram schematically explaining the connection relationship of lattice points. Taking the arrangement of 3×3 lattice points as an example, the lattice points 901 to 904 located in the vertical and horizontal directions with respect to the target lattice point 900 indicate lattice points in a 4-neighborhood connection relationship. The 4-neighborhood connection relationship means that among the plurality of lattice points existing around the lattice point 900, it is a lattice point connected to the target lattice point 900 via one line segment. For example, the lattice point 900 is connected to the lattice point 901 via the line segment 921, and the lattice point 900 is connected to the lattice point 902 via the line segment 922. Also, the lattice point 900 is connected to the lattice point 903 via the line segment 923, and the lattice point 900 is connected to the lattice point 904 via the line segment 924. The lattice points 901 to 904 are lattice points in a 4-neighborhood connection relationship.

[0100] In addition, the lattice points in the connection relationship near 8 refer to the lattice point 900 of interest and the lattice points connected by a plurality of line segments via other lattice points. For example, the lattice point 911 is connected by a plurality of line segments 932, 921 (931, 923) via other lattice points 901 (903). Among the eight lattice points existing around the lattice point 900, the lattice points 911 to 914 excluding the connection relationship of the 4-neighborhood are lattice points in the 8-neighborhood connection relationship with respect to the lattice point 900.

[0101] Regarding the lattice points (lattice points 411, 413, 415) in the 4-neighborhood connection relationship with the lattice point 412, the correction shape acquisition unit 240 may set the weight coefficient of the movement amount to w = 0.5, and regarding the lattice points in the 8-neighborhood connection relationship excluding the lattice points in the 4-neighborhood connection relationship, set the weight coefficient of the movement amount to w = 0.25.

[0102] Then, the correction shape acquisition unit 240 may move each surrounding lattice point by the set movement amount in the same direction as the movement direction of the lattice point 412 to be corrected with respect to the movement direction of the surrounding lattice points. According to this, it is possible to suppress a sharp change in shape caused by moving only the lattice points to be corrected.

[0103] In addition, it is not necessarily required that the movement amount of the lattice point to be corrected is D1. The movement amount of the lattice point to be corrected may be set to D1' smaller than D1 (for example, 0.5 × D1), and the movement amount of each surrounding lattice point may be set based on the weight coefficient w of the movement amount and the movement amount D1' of the lattice point to be corrected (w × D1'). According to this, it is possible to suppress a sharp change in shape caused by a large movement of the lattice point in one correction process.

[0104] In addition, in order to suppress the distortion of the overall shape of the mitral valve due to correction, the positions of a plurality of lattice points may be moved so that the anatomical measurement values of the mitral valve do not change. For example, when the lattice points to be corrected are near the valve orifice, the lattice points near the opposing valve annulus may also be moved by about the same amount so that the valve length does not change before and after correction. The shape may be corrected so that not only the valve length but also the valve annulus length, the valve orifice length, the valve orifice area, and the valve tip area do not change. For the same reason, the positions of the lattice points may be corrected so that the positional relationship of anatomical parts (for example, the commissure located at the boundary of a plurality of valve tips) does not change.

[0105] In the present embodiment, among the lattice points at both ends of the line segment passing through each intersection point and the intersection point, the distances of the lattice points on the valve orifice side of the mitral valve (for example, the distances D1 to D4 in FIG. 5) are calculated, and the lattice point with the minimum distance is determined as the lattice point to be corrected (for example, the lattice point 412 of the distance D1), but it is not necessarily limited to this. For example, among the lattice points on the valve orifice side of the mitral valve at both ends of the line segment passing through each intersection point, the lattice point with the lowest reliability may be used as the lattice point to be corrected.

[0106] As an example of the method for calculating the reliability, the smoothness of the shape can be utilized. Since the mitral valve and other heart valves are locally smooth curved surfaces, a lattice point whose position changes steeply compared to the surrounding lattice points may be regarded as having low reliability and determined as the lattice point to be corrected. For example, when the position information of the points at the end portions (the tip portions of each valve tip) of a plurality of planar structures (for example, the anterior leaflet 400 and the posterior leaflet 410) of the correction shape acquisition unit 240 differs from the position information of the nearby points located around the points at the end portions by exceeding a threshold value indicating the reference of the change in the position information, the position of the points at the end portions (the tip portions of each valve tip) is corrected so as to be within the range of the threshold value. For example, the correction shape acquisition unit 240 may determine, by comparing the position information (three-dimensional coordinate information) of the tip portion of each valve tip with the position information (three-dimensional coordinate information) of the surrounding lattice points, that a lattice point whose change in position information (difference in position information) changes by exceeding the threshold value is a lattice point with low reliability and determine it as the lattice point to be corrected.

[0107] As a result, it is possible to correct the positions of lattice points with low reliability while eliminating intersections, and it is possible to suppress unnatural deformation of the corrected shape.

[0108] In addition, in the explanation using FIG. 5, for each intersection point, the distance between the lattice points on the cusp side of the mitral valve among the lattice points at both ends of the line segment passing through the intersection point is calculated, and the process of correcting the position of the lattice point with the minimum distance is described. However, the present invention is not limited to this process. For example, as the lattice points to be corrected, two lattice points constituting a line segment with the minimum distance from the intersection point may be determined as the lattice points to be corrected.

[0109] When an intersection occurs between one planar structure (anterior cusp 400) and the other planar structure (posterior cusp 410) based on the intersection information, the correction shape acquisition unit 240 may change the position of the intersection point (for example, intersection point 503) by moving lattice points (for example, lattice points 412, 415). In this case, the correction shape acquisition unit 240 corrects the non-correction target cusp tip so as to match the position of the lattice point to be corrected.

[0110] For example, the lattice points 412 and 415 at both ends of the line segment passing through the intersection point 503 in FIG. 5 may be selected as correction targets. The line segment composed of the lattice points 412 and 415 intersects with the plane formed by the four points of the lattice points 402, 403, 405, and 406 on the anterior cusp 400 at the intersection point 503. The line segment composed of the lattice points 412 and 415 may be translated in parallel to the position of the tip of the anterior cusp 400 (for example, the position of the line segment of the lattice points 402 and 403 constituting the valve orifice of the anterior cusp 400). As a result, the intersection state at the intersection point 503 can be eliminated by the parallel translation of the line segment composed of the lattice points 412 and 415.

[0111] Three or more lattice points that form a surface with the minimum distance from the intersection point may be determined as lattice points to be corrected. For example, for the intersection point 503 in FIG. 5, lattice points 402 and 403 that form the line segment on the valve port side among the lattice points of the anterior cusp 400 existing around the intersection point 503 may be determined as lattice points to be corrected. In this case, the positions of lattice points 402 and 403 are corrected so that the line segment formed by lattice points 402 and 403 passes through the intersection point 503. As a result, the tip of the valve port of the anterior cusp 400 comes into contact with the posterior cusp 410 at the position of the intersection point 503, and the intersection at the intersection point 503 can be eliminated. When correcting three or more lattice points that form a surface, the intersection can be similarly eliminated by moving the lattice points so that the surface contacts the intersection point.

[0112] In the above description, the case where one of the lattice points forming the line segment passing through the intersection point is a lattice point on the valve port has been described as an example. However, even when none of the lattice points forming the line segment passing through the intersection point is a lattice point on the valve port, it can be corrected in the same manner. For example, when neither of the two lattice points (the first lattice point on the valve ring side and the second lattice point on the valve port side) forming the line segment of the posterior cusp 410 passing through the intersection point on the anterior cusp 400 is a lattice point on the valve port of the posterior cusp 410, the correction shape acquisition unit 240 moves the second lattice point on the valve port side to the position of the intersection point among the two lattice points in the same manner as in the above example to correct the shape.

[0113] The correction shape acquisition unit 240 can set the movement amount (w×D1) of the position of each lattice point based on a predetermined weight coefficient or a connection relationship (for example, a 4-neighborhood connection relationship, an 8-neighborhood connection relationship) based on the positions of lattice points on the valve port.

[0114] In such a case, simply moving the second lattice point of the posterior cusp 410 to the position of the intersection point on the anterior cusp 400 does not necessarily eliminate the intersection. Therefore, the intersection information acquisition unit 230 re-executes the process of acquiring intersection information with the corrected shape as the initial shape. The correction shape acquisition unit 240 determines whether intersection information has been acquired in the re-executed intersection information acquisition process. If intersection information has been acquired, a corrected shape obtained by further correcting the corrected shape based on the newly acquired intersection information may be obtained, and the intersection information acquisition process and the correction shape acquisition process may be repeatedly executed.

[0115] At this time, when correcting the third grid point on the valve ring side that connects to the second grid point of the posterior cusp 410, in step S340, the correction shape acquisition unit 240 moves the third grid point to the intersection position (the position of the second grid point), and for the second grid point, it may be further moved by the movement amount (w×D1) as described above to correct the positions of each grid point.

[0116] By repeating this process, the grid points on the valve orifice of the posterior cusp 410 are finally moved to the intersection position, and the grid points on the valve orifice of the posterior cusp 410 come into contact with the anterior cusp 400, and the intersection can be eliminated.

[0117] Also, in step S330, the intersection information acquisition unit 230 may acquire the index of the grid point on the valve orifice closest to the intersection, and the correction shape acquisition unit 240 may correct it so as to move the grid point on the valve orifice to the intersection position. According to this, it is possible to reduce the number of times of repeating steps S330 and S340, and the processing can be speeded up.

[0118] (S350: Display of Image) In step S350, the display control unit 250 performs control to display the input image acquired in step S310 and the correction shape acquired in step S340 on the display unit 150. The display control unit 250 generates an image to be displayed on the display unit 150 based on the input image and the correction shape, and performs control to display the generated image on the display unit 150. For example, it performs control to generate an image obtained by CG rendering the mesh information, which is the correction shape of the mitral valve, and display it on the display unit 150. Also, the display control unit 250 has the same functions as a general medical image viewer, and in response to user input, selects a two-dimensional (2D) slice image from the input image, and for the selected slice image, generates an image in which the correction shape of the mitral valve at the position corresponding to the slice image is superimposed and performs control to display it on the display unit 150.

[0119] What is displayed in this processing step is not limited to the corrected shape, and the initial shape before correction may be displayed. For example, according to user input, the corrected shape and the initial shape may be switched and displayed. In this case, it is desirable to enable the user to recognize whether the displayed shape is the corrected shape or the initial shape. For example, character information indicating whether the displayed shape is the corrected shape or the initial shape may be displayed simultaneously, or the display colors of each shape may be changed. According to this, the user can confirm whether the correction process in this embodiment has been appropriately performed, or the user can confirm the magnitude of the change due to the correction. In addition to switching and displaying the corrected shape and the initial shape, the corrected shape and the initial shape may be superimposed and displayed simultaneously, and similar effects can be expected.

[0120] Note that control may be performed to save the corrected shape obtained in step S340 in the data server 130 via the storage unit 114 of the information processing apparatus 100 or the network 120.

[0121] Thereby, the projection image can be displayed on any other medical image viewer, or can be read and used in any other surgical support software, etc. In this case, the display of the image in step S350 does not necessarily have to be performed. Also, the display and storage of the corrected shape are not necessarily required, and a configuration may be adopted in which an analysis process using the obtained shape is performed. For example, a configuration may be adopted in which measurement values related to the shape of the valve (valve length and valve orifice area) and attribute values related to the state of the valve (valve deviation and insufficiency) are calculated or estimated based on the corrected shape and then displayed or stored.

[0122] As described above, according to this embodiment, it is possible to obtain and display a shape with improved consistency among a plurality of valve tips of the heart extracted from the three-dimensional image, and the user can visually recognize the structure of the heart valve more accurately.

[0123] (Modification Example 1-1: Variation of Input Image) In the above-described embodiments, an example in which a three-dimensional CT image is used as the input image has been described. However, the embodiments of the present invention are not limited to this. For example, the input image may be a three-dimensional ultrasonic image (such as percutaneous ultrasound, transesophageal ultrasound, etc.), or may be an image obtained by an MRI device, a PET device, a SPECT device, an ultrasonic diagnostic device, or the like. Further, a time-series image may be used as the input image. For example, it may be a time-series three-dimensional CT image (four-dimensional CT image). In this case, each of the above-described processes can be executed on the images of each time phase of the time-series three-dimensional CT image to obtain the corrected shape of the mitral valve corresponding to each time phase. Then, the result can be displayed as an animation.

[0124] Further, the input image does not necessarily have to be a three-dimensional image. For example, it may be a two-dimensional image such as a long-axis image or a short-axis image of the mitral valve. For example, the information processing apparatus 100 can also obtain a two-dimensional corrected shape of the mitral valve, which is the shape (cross-sectional shape) of the mitral valve on the two-dimensional image.

[0125] That is, for the two-dimensional image acquired by the input image acquisition unit 210, the initial shape acquisition unit 220 executes a process of acquiring a two-dimensional initial shape (mesh information composed of two-dimensional coordinates). The intersection information acquisition unit 230 acquires intersection information based on the acquired two-dimensional initial shape, and the corrected shape acquisition unit 240 acquires a two-dimensional corrected shape based on the two-dimensional initial shape and the intersection information. Then, the display control unit 250 performs control to display the two-dimensional input image and the corrected shape on the display unit 150.

[0126] In addition, when the input image is a time-series image, the discontinuity in the position compared with the previous and subsequent time phases may be utilized for the reliability calculated in step S340 by taking advantage of the fact that the heart valve continuously displaces over time. For example, at the lattice points of a predetermined time phase, by comparing the coordinates of the lattice points of the same index of the surrounding time phases (for example, the previous and subsequent time phases) and the coordinates obtained by time-averaging them, if the distance from any of the coordinates is far, it is possible that the lattice point is at a position deviated from the lattice points of other time phases. To avoid this, for example, a value inversely proportional to the sum of the distances from each coordinate of the lattice points of the same index of the surrounding time phases or the coordinates obtained by time-averaging them can be used as the reliability.

[0127] Also, when correcting the lattice points, the lattice points may be corrected by moving them to the average position of the lattice points of the same index of the surrounding time phases. According to these methods, by correcting the positions of the lattice points at the deviated positions, a more plausible corrected shape can be obtained as the shape of the actual valve.

[0128] Note that the information processing apparatus 100 is also equipped with a processing system that takes a two-dimensional image as input, a processing system that takes a three-dimensional image as input, and a processing system that takes a time-series image as input, and can also switch and execute the processing system to be driven according to the actually input image.

[0129] (Modification Example 1-2: Variation of Valve Tips) In the above-described embodiment, the mitral valve composed of two valve tips, the anterior tip 400 and the posterior tip 410, has been described as an example, but the embodiment is not limited to this. For example, it may be an aortic valve, a tricuspid valve, or a pulmonary valve composed of three valve tips. In this case, two valve tips may be selected as the objects of processing from the three valve tips, and the same processing as in the above-described embodiment may be executed, or all three valve tips may be the objects of processing.

[0130] When all three valve leaflets are to be processed, for example, in step S330, the intersection information acquisition unit 230 may acquire intersection information between a line segment composed of two lattice points in the mesh information of any one valve leaflet and each surface composed of three lattice points in the mesh information of the other two valve leaflets based on the same method as in the above-described embodiment. Further, the corrected shape acquisition unit 240 can correct the positions of the lattice points based on the acquired intersection information.

[0131] Also, the plurality of valve leaflets are not necessarily the same heart valve, and for example, it can be applied to a combination of the anterior leaflet of the mitral valve and the left coronary cusp of the aortic valve.

[0132] Further, the embodiment is not limited to the case of targeting the heart valve, and it can also be applied to parts other than the valve, such as the fossa ovalis formed by the primary septum and the secondary septum of the heart, which are formed from a plurality of planar structures.

[0133] Also, the embodiment is not limited to the case of targeting parts of the heart, and it may target other parts of the human body (for example, the pleura and the diaphragm), or may be outside the parts of the human body (for example, industrial members). Further, the embodiment is not limited to the case of targeting thin structures such as valve leaflets, and it can also be applied to obtaining the surface shapes of a plurality of adjacent parts such as organs like the liver and blood vessels as mesh information.

[0134] <Second Embodiment> In the first embodiment, the configuration for calculating the movement amount and direction for correcting the lattice points from the initial shape has been described. In this embodiment, in addition to the initial shape, a configuration for obtaining a corrected shape obtained by correcting the shape based on the image information of the input image will be described. The shape information in the initial shape may include cases where the position of the mitral valve is correctly acquired and cases where the correct position of the mitral valve is not acquired. In this embodiment, the image information (pixel values) of the input image is used to determine whether the position of the mitral valve is correctly acquired, and the lattice points corresponding to the pixels where the position of the mitral valve is not correctly acquired are selected as the lattice points to be corrected.

[0135] The configuration of the information processing system 10 according to this embodiment is the same as that of the first embodiment described with reference to FIG. 1. FIG. 7 is a diagram showing the functional configuration of the control unit 115 according to this embodiment. Also, the flowchart showing the overall processing procedure performed by the information processing apparatus 100 in this embodiment is the same as that of the first embodiment described with reference to FIG. 3A. In the following description, only the configuration of the parts different from those of the first embodiment will be described.

[0136] (S310: Acquisition of input image) In step S310, when the user instructs to acquire a three-dimensional image via the instruction unit 140, the input image acquisition unit 210 acquires, from the data server 130, the image (image data) designated by the user as the input image. Then, the acquired input image is output to the initial shape acquisition unit 220, the corrected shape acquisition unit 240, and the display control unit 250.

[0137] (S340: Acquisition of corrected shape) FIG. 3D is a diagram showing an example of the flow of the corrected shape acquisition process in the second embodiment. In step S21, the corrected shape acquisition unit 240 acquires the position information (three-dimensional coordinate information) of the points (lattice points) obtained as intersection information and the image information (for example, pixel values (pixel values of the lattice points)) at the positions corresponding to the position information of the lattice points. The corrected shape acquisition unit 240 acquires the image information (pixel values) corresponding to the coordinate information of the lattice points indicated by the index acquired as intersection information based on the three-dimensional coordinate information and the image information (pixel values) of the lattice points.

[0138] In step S22, the corrected shape acquisition unit 240 compares the image information (pixel values) of the lattice points with the reference information of the plurality of planar structures (reference image information (pixel values), for example, the image information (pixel values) of the cardiac cavity region around the mitral valve). For example, when the input image is a contrast CT image and the initial shape of the mitral valve has been acquired, the corrected shape acquisition unit 240 utilizes the fact that the image information (pixel values) of the mitral valve in the contrast CT image has lower image information (pixel values) than the surrounding cardiac cavity region to determine whether the position of the lattice point has been correctly acquired as a position on the mitral valve.

[0139] Of the indices of the grid points acquired as intersection information, when the image information (pixel value) of the CT image at the coordinates of the grid point indicated by the index is smaller than the reference information (reference pixel value) (S22 - NO), the correction shape acquisition unit 240 determines that the position of the grid point has been correctly acquired as a grid point on the mitral valve (S25).

[0140] On the other hand, in the determination process of step S22, when the image information (pixel value) of the CT image at the coordinates of the grid point indicated by the index among the indices of the grid points acquired as intersection information is larger than the reference information (S22 - YES), since there is a possibility of being away from the position of the mitral valve, the correction shape acquisition unit 240 selects this grid point as a point (grid point) to be corrected (S23).

[0141] In step S24, the correction shape acquisition unit 240 corrects the position of the grid point based on the direction (vector) in which the image information (pixel value) of the point (grid point) decreases, based on the gradient of the image information (pixel value) around the point (grid point) selected in step S23. At this time, the grid point may be moved and corrected in the direction obtained by taking the weighted average of the direction for correcting the grid point calculated by the same method as in the first embodiment and the direction calculated using the gradient of the image information (pixel value).

[0142] Here, a predetermined value determined in advance may be used as the weight, or it may be determined based on the image information (pixel value). For example, when the gradient of the image information (pixel value) in the direction calculated by the same method as in the first embodiment indicates that the image information (pixel value) increases (away from the pixel value corresponding to the mitral valve), there is a possibility that the position of the grid point moves away from the position of the mitral valve on the image due to the movement of the grid point by the correction process. Therefore, in this case, the weight related to the direction calculated by the same method as in the first embodiment may be set small and the average may be taken.

[0143] In step S25, for the lattice points determined to be correctly acquired as lattice points on the mitral valve, in step S26, the correction shape acquisition unit 240 determines whether to select them as correction targets based on the positions of the lattice points constituting the mesh information, in the same manner as the explanation process in the first embodiment.

[0144] If not selected as a correction target (S26-NO), the process of this step ends. If selected as a correction target (S26-YES), the process proceeds to step S24.

[0145] Then, the correction shape acquisition unit 240 acquires a corrected shape obtained by correcting the initial shape based on the input image acquired in step S310 and the intersection information acquired in step S330. The correction shape acquisition unit 240 corrects the positions of the lattice points selected as correction targets based on the amount of movement and the direction of movement obtained by the same method as the process described in the first embodiment. Then, the correction shape acquisition unit 240 outputs the acquired correction shape to the display control unit 250.

[0146] Regarding the setting of the weight in the correction process, in addition to the intersection information (presence or absence of intersection, position of intersection) described in step S330, the intersection information acquisition unit 230 also calculates the amount of intersection, and in this step, the correction shape acquisition unit 240 may determine the weight based on the amount of intersection.

[0147] FIG. 3E is a diagram showing an example of the flow of the correction shape acquisition process in the second embodiment. When determining the weight based on the intersection amount, the distance from the intersection point to the tip of the valve tip may be used as the intersection amount. The intersection amount is not limited to the distance, and the number of intersection points where intersections occur between a plurality of valve tips may be used as the intersection amount. When the intersection amount (including the distance or the number of intersection points) is smaller than the threshold value, the correction shape acquisition unit 240 sets a weight coefficient based on the intersection amount and corrects the positions of the nearby points (the correction shape acquisition process of the first embodiment). Further, when the intersection amount is larger than the threshold value, the correction shape acquisition unit 240 performs a correction process based on the image information. The correction shape acquisition unit 240 corrects the position of the point (grid point) based on the comparison between the image information of the point (grid point) and the reference information of the plurality of planar structures (the correction shape acquisition process of the second embodiment).

[0148] In step S31, when the intersection amount (including the distance or the number of intersection points) is smaller than a predetermined threshold value (S31 - YES), the correction shape acquisition unit 240 proceeds to step S32 with the process.

[0149] In step 32, the correction shape acquisition unit 240 sets a weight coefficient based on the intersection amount. When the intersection amount is small and the degree of shape inconsistency due to the intersection is low, the weight w may be set smaller according to the intersection amount compared to the weight w described in the first embodiment.

[0150] The correction shape acquisition unit 240 changes the setting of the weight w according to the intersection amount. For example, when the intersection amount is smaller than the threshold value, the correction shape acquisition unit 240 sets it smaller than the weight w of the movement amount (for example, 0.5) for the grid points in the 4-neighborhood connection relationship. Similarly, the correction shape acquisition unit 240 sets it smaller than the weight coefficient w of the movement amount (for example, 0.25) for the grid points in the 8-neighborhood connection relationship. The case where the intersection amount is shorter than the threshold value means that the intersection is in a minor state. In such a case, the shape inconsistency can be resolved by slightly moving the grid point (the correction shape acquisition process of the first embodiment).

[0151] On the other hand, when the intersection amount is greater than the threshold value (S31 - NO), the correction shape acquisition unit 240 proceeds to step S33. When there is a shape inconsistency such that the intersection amount exceeds the threshold value, in step S33, the correction shape acquisition unit 240 performs the correction shape acquisition process (S21 to S26) of the second embodiment described with reference to FIG. 3D. That is, the correction shape acquisition unit 240 corrects the position of the point (grid point) based on the comparison between the image information at the position corresponding to the position information of the point (grid point) and the reference information. The correction shape acquisition unit 240 corrects the position of the grid point based on the gradient of the image information (pixel value) around the grid point and based on the direction (vector) in which the image information (pixel value) of the grid point decreases.

[0152] When there is a shape inconsistency such that the intersection amount exceeds the threshold value, by performing the correction shape acquisition process (correction process) of the second embodiment based on the gradient of the image information (pixel value), it is possible to suppress deviation from the actual shape of the mitral valve, and thereby obtain a corrected shape that suppresses the deviation from the position of the mitral valve on the image while eliminating the intersection.

[0153] As described above, according to the present embodiment, it is possible to obtain and display a corrected shape in which the consistency is improved among a plurality of valve tips of the heart extracted from the three - dimensional image and the deviation in position from the heart valve shown in the input image is suppressed, and the user can visually recognize the structure of the heart valve more accurately.

[0154] (Modification Example 2 - 1: Variation of the input image) In the second embodiment described above, an example in which a CT image is used as the input image and the reliability is calculated based on the magnitude of its pixel value has been described, but the present embodiment is not limited to this. For example, an image showing the region of the heart valve obtained by a known technique from a CT image or an image captured by another modality may be used as the input image, and a method based on the pixel value of the image may be used. For example, as an example of a method based on machine learning, a UNet, which is a type of CNN, can be used to obtain the region of the heart valve from a CT image, and the obtained image can be used as the input image.

[0155] An image showing the region of the heart valve may be a binary label in which foreground pixels showing the region of the heart valve are stored as 255 and other background pixels are stored as 0, or a likelihood map in which the likelihood of the presence of the heart valve is stored in the pixel value of each pixel. In the case of a binary label, an image having a gradient of pixel values can be obtained by performing a smoothing process. When these images are input, in step S340, by correcting the grid points in the direction toward the pixel value of the foreground pixels or the pixel value with high likelihood based on the gradient of the pixel values, a corrected shape can be obtained that suppresses the deviation from the position of the mitral valve on the image while eliminating intersections.

[0156] The disclosure of the specification includes the following information processing apparatus, information processing method, and program. (Item 1) An intersection information acquisition means for acquiring intersection information of a plurality of planar structures of the object in a three-dimensional image obtained by imaging the object, Based on the intersection information, at least one of the plurality of planar structures is determined as a correction target, and a correction means for correcting the shape of the planar structure to be corrected, characterized in that the information processing apparatus has the correction means. (Item 2) The information processing apparatus according to Item 1, characterized in that the intersection information acquisition means acquires the intersection information using mesh information composed of a plurality of points having coordinate information of the plurality of planar structures. (Item 3) The information processing apparatus according to Item 2, characterized in that the intersection information acquisition means acquires, as the intersection information, an intersection point between a line segment selected using the plurality of points of any one of the plurality of planar structures and a plane selected using the plurality of points of the other planar structure. (Item 4) The information processing apparatus according to Item 3, characterized in that the intersection information acquisition means further acquires at least one point located around the intersection point as the intersection information. (Item 5) The information processing apparatus according to Item 1, characterized in that the intersection information acquisition means acquires the intersection information in a region specified by region setting information among the plurality of planar structures. (Item 6) The intersection information acquisition means acquires the intersection information in a region specified based on the product of the lengths of the plurality of planar structures and a parameter for dividing the plurality of planar structures into a predetermined length. The information processing apparatus according to item 1, characterized in that. (Item 7) The intersection information acquisition means acquires the intersection information at a position where the position of a pixel indicating the presence of any one of the plurality of planar structures and the position of a pixel indicating the presence of the other planar structure overlap in the object. The information processing apparatus according to item 1, characterized in that. (Item 8) The correction means corrects the planar structure to be corrected into a shape that does not intersect with the other planar structure when the planar structure to be corrected intersects with the other planar structure. The information processing apparatus according to item 1 or 2, characterized in that. (Item 9) When an intersection occurs between the one planar structure and the other planar structure based on the intersection information, the correction means corrects the position of the end of any one of the planar structures so as to match the position of the intersection point. The information processing apparatus according to item 3, characterized in that. (Item 10) The correction means a first distance between a first intersection point included in the intersection information, a line segment passing through the first intersection point, and an end point of the other planar structure; a second distance between a second intersection point included in the intersection information, a line segment passing through the second intersection point, and an end point of the other planar structure; a third distance between a third intersection point included in the intersection information, a line segment passing through the third intersection point, and an end point of the one planar structure; a fourth distance between a fourth intersection point included in the intersection information, a line segment passing through the fourth intersection point, and an end point of the one planar structure, and obtains The correction means determines the planar structure with the shorter distance as the object of the correction by comparing the first distance and the third distance, or determines the planar structure with the shorter distance as the object of the correction by comparing the second distance and the fourth distance. The information processing apparatus according to item 9, characterized in that. (Item 11) The correction means determines, by comparing the distances among a plurality of points at the end of the planar structure to be corrected, a point at the end with a shorter distance as the object to be corrected, according to the information processing apparatus described in Item 10. (Item 12) The correction means corrects, by comparing the distances, the position of the point at the end of the other planar structure to match the position of the first intersection point when the first distance is the shortest distance, according to the information processing apparatus described in Item 10. (Item 13) The correction means corrects the position of the point at the end by a first movement amount, and corrects a neighboring point located around the point at the end by a second movement amount set based on a predetermined weighting coefficient with respect to the first movement amount in the same direction as the movement direction of the point at the end, according to the information processing apparatus described in Item 12. (Item 14) The correction means corrects the position of the neighboring point by the second movement amount set based on different weighting coefficients according to the distance between the point at the end and the neighboring point as the predetermined weighting coefficient, according to the information processing apparatus described in Item 13. (Item 15) The correction means corrects the position of the point at the end so that it falls within the range of the threshold value when the position information of the points at the ends of the plurality of planar structures differs from the position information of the neighboring points located around the points at the ends by exceeding a threshold value indicating a reference for the change in the position information, according to the information processing apparatus described in Item 1. (Item 16) When an intersection occurs between the one planar structure and the other planar structure based on the intersection information, the correction means determines the intersection point and a plurality of points constituting a line segment passing through the intersection point as objects to be corrected, and corrects the intersection point and the plurality of points to match the position of the end of either one of the one planar structure and the other planar structure, according to the information processing apparatus described in Item 3. (Item 17) The correction means acquires the position information of the points obtained as the intersection information and the image information at the position corresponding to the position information of the points, Compare the image information at the position corresponding to the position information of the point with the reference information of the plurality of planar structures, The information processing apparatus according to item 13, wherein when the image information is larger than the reference information, the position of the point is corrected in a direction in which the image information of the point becomes lower based on the gradient of the image information around the point. (Item 18) When the intersection amount between one of the plurality of planar structures and the other planar structure is smaller than a threshold value, the correction means sets a weight coefficient based on the intersection amount and corrects the position of the point in the vicinity, The information processing apparatus according to item 17, wherein when the intersection amount is larger than the threshold value, the position of the point is corrected based on the comparison between the image information and the reference information. (Item 19) The intersection information acquisition means re-executes the process of acquiring the intersection information, with the first corrected shape of the plurality of planar structures corrected by the correction means as the initial shape, The information processing apparatus according to item 12, wherein the correction means acquires a second corrected shape obtained by further correcting the first corrected shape based on the intersection information newly acquired by the intersection information acquisition means. (Item 20) When the intersection information acquisition means re-acquires the intersection information, the information processing apparatus according to item 19, wherein the points used for the selection of the line segment and the plane are limited to the points moved by the correction and the points located around the point, and the intersection information is acquired. (Item 21) The information processing apparatus according to item 1, further comprising display control means for causing a display means to display the shapes of the plurality of planar structures corrected by the correction means. (Item 22) The object is a heart valve, The information processing apparatus according to item 1, wherein the plurality of planar structures are a plurality of valve leaflets constituting the heart valve. (Item 23) A step of acquiring intersection information of a plurality of planar structures of an object in a three-dimensional image obtained by imaging the object, Based on the intersection information, determining at least one of the plurality of planar structures as a correction target, and correcting the shape of the planar structure to be corrected; An information processing method characterized by having the above. (Item 24) A program for causing a computer to execute the information processing method described in Item 23.

[0157] (Other Embodiments) The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. Further, it can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

[0158] The invention is not limited to the above embodiments, and various changes and modifications are possible without departing from the spirit and scope of the invention. Therefore, claims are attached to disclose the scope of the invention.

Explanation of Reference Numerals

[0159] 10: Information processing system, 100: Information processing device, 111: Communication IF, 115: Control unit, 120: Network, 130: Data server, 140: Instruction unit, 150: Display unit, 210: Input image acquisition unit, 220: Initial shape acquisition unit, 230: Intersection information acquisition unit, 240: Correction shape acquisition unit (correction unit), 250: Display control unit, 400: Front tip (one planar structure), 410: Rear tip (the other planar structure), 401, 402, 403: Lattice points of the valve port of the front tip (lattice points on the valve port side), 411, 412, 413: Lattice points of the valve port of the rear tip (lattice points on the valve port side)

Claims

1. An intersection information acquisition means for acquiring intersection information of a plurality of planar structures of an object in a three-dimensional image obtained by capturing the object; a correction means for determining at least one of the plurality of planar structures as a correction target based on the intersection information, and correcting a shape of the planar structure to be corrected; 13. An information processing device comprising:

2. 2. The information processing apparatus according to claim 1, wherein the intersection information acquisition means acquires the intersection information by using mesh information constituted by a plurality of points having coordinate information of the plurality of planar structures.

3. The information processing device according to claim 2, characterized in that the intersection information acquisition means acquires as the intersection information an intersection between a line segment selected using the multiple points of one of the multiple planar structures and a surface selected using the multiple points of the other planar structure.

4. 4. The information processing apparatus according to claim 3, wherein the intersection information acquisition means further acquires at least one point located around the intersection as the intersection information.

5. The information processing apparatus according to claim 1 , wherein the intersection information acquisition means acquires the intersection information for a region of the plurality of planar structures that is specified by region setting information.

6. The information processing device according to claim 1, characterized in that the intersection information acquisition means acquires the intersection information in an area specified based on the product of the lengths of the plurality of planar structures and a parameter that divides the plurality of planar structures into predetermined lengths.

7. The information processing device according to claim 1, characterized in that the intersection information acquisition means acquires the intersection information at a position where a pixel position indicating the presence of one of the plurality of planar structures overlaps with a pixel position indicating the presence of the other planar structure.

8. The information processing device according to claim 1 or 2, characterized in that, when the planar structure to be corrected intersects with another planar structure, the correction means corrects the planar structure to be corrected into a shape that does not intersect with the other planar structures.

9. The information processing device according to claim 3, characterized in that the correction means corrects the position of one of the planar structures to match the position of the intersection point when an intersection occurs between the one planar structure and the other planar structure based on the intersection information.

10. The correction means is a first distance between a first intersection point included in the intersection information, a line segment passing through the first intersection point, and an end point of the other planar structure; a second distance between a second intersection point included in the intersection information, a line segment passing through the second intersection point, and an end point of the other planar structure; and a third intersection point included in the intersection information, a third distance between a line segment passing through the third intersection point and an end point of the one planar structure; and obtain a fourth intersection point included in the intersection information and a fourth distance between a line segment passing through the fourth intersection point and an end point of the one planar structure; The correction means is By comparing the first distance with the third distance, a planar structure having a shorter distance is determined as a target for the correction, or The information processing apparatus according to claim 9 , further comprising: determining, by comparing the second distance with the fourth distance, a planar structure having a shorter distance as a target for the correction.

11. The information processing apparatus according to claim 10 , characterized in that the correction means determines, by comparing the distances, a point at an end portion having a shorter distance from among a plurality of points at an end portion of the planar structure to be corrected, as the point to be corrected.

12. The information processing device according to claim 10, characterized in that the correction means corrects the position of the end point of the other planar structure to align with the position of the first intersection when the first distance is the shortest distance based on the comparison of the distances.

13. The correction means corrects the position of the end point by a first movement amount, The information processing device according to claim 12, characterized in that nearby points located around the end point are corrected in the same direction as the movement direction of the end point by a second movement amount set based on a predetermined weighting coefficient for the first movement amount.

14. The information processing device according to claim 13, characterized in that the correction means corrects the position of the nearby point by the second movement amount set based on a weighting coefficient that differs depending on the distance between the end point and the nearby point as the predetermined weighting coefficient.

15. The information processing device described in claim 1, characterized in that when position information of an end point of the multiple planar structures differs from position information of nearby points located around the end point by more than a threshold indicating a standard for change in position information, the correction means corrects the position of the end point to be within the range of the threshold.

16. the correction means determines, when an intersection occurs between the one planar structure and the other planar structure based on the intersection information, the intersection point and a plurality of points constituting a line segment passing through the intersection point as correction targets; 4. The information processing apparatus according to claim 3, wherein the intersection and the plurality of points are corrected to coincide with the position of an end of one of the planar structures.

17. The correction means acquires position information of the point obtained as the intersection information and image information of a position corresponding to the position information of the point, comparing image information at a position corresponding to the position information of the point with reference information of the plurality of planar structures; 14. The information processing apparatus according to claim 13, wherein, when the image information is larger than the reference information, the position of the point is corrected in a direction in which the image information of the point becomes lower based on a gradient of the image information around the point.

18. the correction means, when an amount of intersection between any one of the plurality of planar structures and another planar structure is smaller than a threshold value, sets a weighting coefficient based on the amount of intersection to correct the position of the nearby point; 18. The information processing apparatus according to claim 17, wherein, when the amount of intersection is greater than the threshold value, the position of the point is corrected based on a comparison between the image information and the reference information.

19. the intersection information acquisition means executes again the process of acquiring the intersection information by using the first corrected shape of the plurality of planar structures corrected by the correction means as an initial shape; 13 . The information processing apparatus according to claim 12 , wherein the correction means acquires a second corrected shape by further correcting the first corrected shape based on the intersection information newly acquired by the intersection information acquisition means.

20. The information processing device according to claim 19, characterized in that, when the intersection information acquisition means re-acquires the intersection information, the intersection information acquisition means acquires the intersection information by limiting the points used to select the line segments and the faces to the points moved by the correction and points located around those points.

21. 2. The information processing apparatus according to claim 1, further comprising a display control means for displaying on a display means the shapes of the plurality of planar structures corrected by the correction means.

22. the object is a heart valve; The information processing apparatus according to claim 1 , wherein the plurality of planar structures are a plurality of valve cusps that constitute the heart valve.

23. acquiring intersection information of a plurality of planar structures of an object in a three-dimensional image obtained by capturing the object; determining at least one of the plurality of planar structures as a correction target based on the intersection information, and correcting a shape of the planar structure to be corrected; 13. An information processing method comprising:

24. A program for causing a computer to execute the information processing method according to claim 23.

Images