Medical device, image processing method, computer program, and computer program product

Abstract

This medical device comprises: a three-dimensional image data acquisition unit that acquires three-dimensional image data of a target blood vessel obtained by imaging the target blood vessel with a tomographic imaging device and first orientation information for specifying the orientation of the target blood vessel; a two-dimensional image data acquisition unit that acquires two-dimensional image data of the target blood vessel captured by an X-ray imaging device; an estimation unit that estimates second orientation information for specifying the orientation of the target blood vessel in the X-ray imaging device; and a generation unit that uses the first orientation information and the second orientation information to generate, from the three-dimensional image data, a display image in which the direction of the displayed target blood vessel is corrected, generates a two-dimensional image of the target blood vessel from the two-dimensional image data, and generates a superimposed image in which the target blood vessel displayed in the display image and the target blood vessel displayed in the two-dimensional image are superimposed.

Description

Medical device, image processing method, computer program, and computer program product

[0001] The present disclosure relates to a technique for displaying images captured by a tomography imaging device such as a CT (Computed Tomography) device or an MRI (Magnetic Resonance Imaging) device, and a technique for displaying images captured by an X-ray imaging device, and more particularly to a technique for displaying an image captured by the tomography imaging device and an image captured by the X-ray imaging device in a superimposed manner.

[0002] Patent Document 1 describes a technique for displaying a blood vessel image captured by a CT device and an X-ray fluoroscopic image captured by an X-ray device in an overlapping manner.

[0003] JP 2019-76192 A

[0004] There is room for improvement in the technology for more accurately superimposing and displaying an image captured by a tomography imaging apparatus and an X-ray fluoroscopic image captured by an X-ray imaging apparatus.

[0005] The present disclosure has been made to solve at least part of the above-mentioned problems, and can be realized in the following aspects.

[0006] One form of the present disclosure is a medical device comprising: a three-dimensional image data acquisition unit that acquires three-dimensional image data of a target blood vessel obtained by imaging the target blood vessel with a tomography imaging device and first orientation information for identifying the orientation of the target blood vessel; a two-dimensional image data acquisition unit that acquires two-dimensional image data of the target blood vessel imaged with an X-ray imaging device; an estimation unit that estimates second orientation information for identifying the orientation of the target blood vessel in the X-ray imaging device; and a generation unit that uses the first orientation information and the second orientation information to generate a display image from the three-dimensional image data in which the orientation of the target blood vessel to be displayed is corrected, generates a two-dimensional image of the target blood vessel from the two-dimensional image data, and generates a superimposed image in which the target blood vessel represented in the display image and the target blood vessel represented in the two-dimensional image are superimposed.

[0007] The present disclosure can be realized in various forms, for example, in the form of a computer program product, such as a CT (Computed Tomography) device, an MRI (Magnetic Resonance Imaging) device, an X-ray imaging device, a system including these, a computer program for realizing the functions of the system, a server device for distributing the computer program, a non-transitory storage medium storing the computer program, etc. A computer program product refers to a program as a tradeable product. The product can generally exist in any form, such as paper or a computer-readable data medium.

[0008] 1 is an explanatory diagram illustrating a medical device and an X-ray imaging device; 2 is an explanatory diagram illustrating a CT device; 3 is an explanatory diagram illustrating a medical device; 4 is an explanatory diagram illustrating an image processing method; 5 is an explanatory diagram illustrating an image for display; 6 is an explanatory diagram illustrating a first two-dimensional image; 7 is an explanatory diagram illustrating a second two-dimensional image; 8 is an explanatory diagram illustrating a method for estimating second posture information; 9 is an explanatory diagram illustrating a method for inputting to a first two-dimensional image; 10 is an explanatory diagram illustrating a superimposed image; 11 is an explanatory diagram illustrating a first estimation method of correction information; 12 is an explanatory diagram illustrating a method for deriving a rotation axis vector; 13 is an explanatory diagram illustrating a method for deriving a rotation angle; 14 is an explanatory diagram illustrating a second estimation method of correction information; 15 is an explanatory diagram illustrating a method for deriving a rotation axis vector; 16 is an explanatory diagram illustrating a relationship between a rotation axis vector and a plane including a vascular axis vector; 17 is an explanatory diagram illustrating a state in which a vascular axis vector is projected onto the same plane.

[0009] 1 is an explanatory diagram illustrating a medical device 1 and an X-ray imaging device 10. The medical device 1 includes a computer 40, a display device 46, and an input device 47a.

[0010] <X-ray Imaging Device 10> The X-ray imaging device 10 includes a C-arm 11 equipped with an X-ray generator 12 and an X-ray detector 13, and a bed 14. The X-ray generator 12 and the X-ray detector 13 face each other. The X-ray detector 13 has a function of receiving X-ray fluoroscopy information and converting it into a video signal. The C-arm 11 is movable by being fixed to a cart or the like (not shown). The C-arm 11 can rotate around a tabletop 15 to adjust the positions of the X-ray generator 12 and the X-ray detector 13 relative to the patient 200. The X-ray imaging device 10 receives commands from a two-dimensional image data acquisition unit 32 (FIG. 3) included in the medical device 1 and acquires first two-dimensional image data and second two-dimensional image data. The X-ray imaging device 10 is used to acquire first two-dimensional image data in step S3 ( FIG. 4 ) and second two-dimensional image data in step S4 ( FIG. 4 ) during the process of acquiring the superimposed image Img3 ( FIG. 10 ). The procedure for acquiring two-dimensional image data using the X-ray imaging device 10 in digital X-ray imaging using an FPD (Flat Panel Detector) as the X-ray detection device 13 will be described. The C-arm 11 is moved and rotated so that the target blood vessel 100 of the patient 200, who is the subject of imaging, is positioned between the X-ray generator 12 and the X-ray detection device 13. X-rays are emitted from the X-ray generator 12 toward the target blood vessel 100, and the X-rays that have passed through the target blood vessel 100 are detected by the X-ray detection device 13. The X-ray fluoroscopic information of the target blood vessel 100 is converted into a digital signal by the X-ray detection device 13 and stored as two-dimensional image data in a storage medium. A generating unit 35 (FIG. 3), which will be described later, generates two-dimensional images (Img21, Img22) that are X-ray fluoroscopic images based on the two-dimensional image data. The generated two-dimensional images (Img21, Img22) are displayed on a display device 46. The target blood vessel 100 refers to the blood vessel that is the subject of examination or treatment. In analog X-ray photography using film in the X-ray detection device 13, the film is read by an image scanner and converted into data, which can be treated as two-dimensional image data.

[0011] In the coordinate system shown in Figure 1, the X axis is the width direction of the human body, the Y axis is the thickness direction of the human body, and the Z axis is the height direction of the human body. The X axis increases from the patient's right arm to the left arm, the Y axis increases from the patient's chest to the back, and the Z axis increases from the patient's feet to the head. Hereinafter, the coordinate system in which the Z axis is directed toward the head of the human body as described above will be referred to as a universal coordinate system.

[0012] FIG. 2 is an explanatory diagram illustrating a CT device 20. The CT device 20 is an example of a tomographic imaging device. The CT device 20 includes a CT gantry 21 and a bed 23. The CT gantry 21 includes a cavity 22 that can accommodate a top plate 24 of the bed 23, an X-ray source (not shown) that can rotate around the cavity 22, and an X-ray detector (not shown). The bed 23 has a function for moving the top plate 24 in the longitudinal direction of the bed 23, the transverse direction of the bed 23, and the height direction of the bed 23, and this function allows the top plate 24 to move into the cavity 22. The procedure for acquiring three-dimensional image data using the CT device 20 will be described. First, a patient 200 is placed in the cavity 22 of the CT gantry 21 by moving the top plate 24 of the bed 23. An X-ray generating source within the CT gantry 21 is driven to irradiate the patient 200 with X-rays, and the X-rays that pass through the patient 200 are detected by an X-ray detector. The tomographic image data detected by the X-ray detector is converted into a digital signal and sent to a computer 40 (FIG. 3). The tomographic image data sent to the computer 40 is subjected to image reconstruction processing by a CPU 30 (FIG. 3) installed in the computer 40, and three-dimensional image data is generated. The three-dimensional image data is a collection of data constructed from voxels that store values ​​related to the X-ray absorption coefficient of the target blood vessel 100. If the tomographic imaging device is a CT device 20, the value related to the X-ray absorption coefficient may be a CT value or a brightness value proportional to the CT value. As described below, a generation unit 35 generates a display image Img1 based on the three-dimensional image data. The coordinate axes shown in FIG. 2 represent the coordinate axes of the common coordinate system described above.

[0013] <Medical Device 1> Figure 3 is an explanatory diagram illustrating the medical device 1. The medical device 1 includes a central processing unit (CPU) 30, a read-only memory (ROM) 41, a random access memory (RAM) 42, a hard disk drive (HDD) 43, and a communication control device 44. The medical device 1 includes a three-dimensional image data acquisition unit 31, a two-dimensional image data acquisition unit 32, a position information acquisition unit 33, an estimation unit 34, and a generation unit 35. The CPU 30 is interconnected with the ROM 41, RAM 42, HDD 43, and communication control device 44 of a computer 40 via a bus 45 serving as a common signal transmission path. The medical device 1 is connected to the X-ray imaging device 10 and the CT device 20 via a network 48 or cables (not shown). The medical device 1 is connected to a display device 46 and an input device 47 via a cable or the like. The medical device 1 acquires three-dimensional image data of a target blood vessel 100 ( FIG. 2 ) using a three-dimensional image data acquisition unit 31, and generates a display image Img1 from the three-dimensional image data using a generation unit 35. The medical device 1 acquires two-dimensional image data of the target blood vessel 100 using a two-dimensional image data acquisition unit 32, and generates two-dimensional images (Img21, Img22) ( FIGS. 6 and 7 ) from the two-dimensional image data using a generation unit 35. The medical device 1 estimates correction information for superimposing the orientation of the target blood vessel 100 ( FIG. 2 ) when the patient 200 ( FIG. 2 ) is imaged using a CT device 20 ( FIG. 2 ) with the orientation of the target blood vessel 100 ( FIG. 1 ) when the patient 200 ( FIG. 1 ) is imaged using an X-ray imaging device 10 ( FIG. 1 ), and generates a display image Img1 from the three-dimensional image data based on the correction information. The medical device 1 generates a superimposed image Img3 (FIG. 10) by superimposing the display image Img1 and the first two-dimensional image Img21.

[0014] Generally, a patient 200 ( FIG. 2 ) may be imaged by a CT device 20 ( FIG. 2 ) and then imaged by an X-ray imaging device 10 ( FIG. 1 ) at another location. Therefore, the posture of the patient 200 when imaged by the CT device 20 often differs from the posture of the patient 200 when imaged by the X-ray imaging device 10. Therefore, the posture of the target blood vessel 100 of the patient 200 when imaged by the CT device 20 ( FIG. 2 ) often differs from the posture of the target blood vessel 100 of the patient 200 when imaged by the X-ray imaging device 10. The medical device 1 estimates correction information for superimposing the posture of the target blood vessel 100 when the patient 200 ( FIG. 2 ) is imaged by the CT device 20 with the posture of the target blood vessel 100 when the patient 200 ( FIG. 1 ) is imaged by the X-ray imaging device 10 ( FIG. 1 ). The medical device 1 generates a display image Img1 from the three-dimensional image data based on the correction information. The medical device 1 generates a superimposed image Img3 (FIG. 10) by superimposing the display image Img1 and the first two-dimensional image Img21.

[0015] The three-dimensional image data acquisition unit 31 acquires tomographic image data of a patient 200 ( FIG. 2 ) including a target blood vessel 100 ( FIG. 2 ) using a tomographic imaging device, and generates three-dimensional image data of the target blood vessel 100 from the acquired tomographic image data. Here, a CT device 20 is used as an example of a tomographic imaging device. The tomographic imaging device is not limited to the CT device 20, and other tomographic imaging devices such as an MRI may be used. The three-dimensional image data acquisition unit 31 acquires first orientation information. The first orientation information is information for identifying the orientation of the target blood vessel 100 when the patient 200 is imaged by the CT device 20. For example, information for identifying the orientation of the target blood vessel 100 when the patient 200 is imaged using the CT device 20 for examining the target blood vessel 100 on the day before treatment of the target blood vessel 100 can be used as the first orientation information. In this embodiment, information regarding the orientation of the characteristic portion 101 ( FIG. 5 ) and information regarding the orientation of the characteristic portion 102 ( FIG. 5 ) are used as information for identifying the orientation of the target blood vessel 100. Therefore, the first orientation information can be described as information for identifying the orientations of the characteristic portions 101 and 102 included in the target blood vessel 100. The first orientation information includes a blood vessel axis vector Va1 ( FIG. 5 ) representing the orientation of the characteristic portion 101, which is part of the target blood vessel 100, and a blood vessel axis vector Vb1 ( FIG. 5 ) representing the orientation of the characteristic portion 102, which is a portion different from the characteristic portion 101. The first orientation information may be automatically identified based on coordinate information of the voxels of the characteristic portions (101, 102) in the acquired three-dimensional image data. When automatically identifying the first orientation information by image processing, for example, the value of the X-ray absorption coefficient of each voxel in the three-dimensional image data is first referenced, and the target blood vessel 100 is extracted using the difference between the X-ray absorption coefficient of the target blood vessel 100 and the X-ray absorption coefficient of the surrounding area of ​​the target blood vessel 100. The extension direction of the target blood vessel 100 is identified from the coordinates of the voxels of the extracted target blood vessel 100, and the extension direction of the target blood vessel 100 is treated as the first orientation information. Alternatively, the first orientation information may be specified based on the coordinates of a straight line generated by a user of the medical device 1 ( FIG. 1 ) along characteristic portions (101, 102) on an image generated based on three-dimensional image data of the target blood vessel 100. The three-dimensional image data acquisition unit 31 may acquire the three-dimensional image data ( FIG. 5 ) and the first orientation information prepared in advance from outside the medical device 1.

[0016] The two-dimensional image data acquisition unit 32 uses the X-ray imaging device 10 (FIG. 1) to acquire two-dimensional image data for generating an X-ray fluoroscopic image of the target blood vessel 100 (FIG. 6). The two-dimensional image data acquisition unit 32 sends a command to the X-ray imaging device 10 to acquire first two-dimensional image data and second two-dimensional image data.

[0017] The position information acquisition unit 33 acquires, as first imaging position information, information on the imaging position of the X-ray imaging device 10 ( FIG. 1 ) when the first two-dimensional image data is acquired. The imaging position information can be, for example, information on the position of the X-ray detection device 13 ( FIG. 1 ) in three-dimensional space and its orientation relative to the imaging target. The position information acquisition unit 33 acquires, as second imaging position information, information on the imaging position of the X-ray imaging device 10 when the second two-dimensional image data is acquired. The position information acquisition unit 33 acquires, as first position information, position information of the target blood vessel 100 ( FIG. 6 ) in the first two-dimensional image Img21 generated from the first two-dimensional image data. The position information of the target blood vessel 100 in the first two-dimensional image Img21 is information on the position (i.e., display position) of the target blood vessel 100 on the image represented in the first two-dimensional image Img21. The position information acquisition unit 33 acquires, as second position information, position information of the target blood vessel 100 in the second two-dimensional image Img22 generated from the second two-dimensional image data. The generation unit 35 may generate two-dimensional images (Img21, Img22) to acquire the first position information or the second position information. An example of a method for acquiring the first position information and the second position information will be described later with reference to FIG. 9 .

[0018] The estimation unit 34 estimates second orientation information using the information on the first imaging position, the information on the second imaging position, the first position information, and the second position information acquired by the position information acquisition unit 33. The second orientation information is information for identifying the orientation of the target blood vessel 100 when the patient 200 is imaged by the X-ray imaging device 10. In this embodiment, information on the orientation of the characteristic portion 101 and information on the orientation of the characteristic portion 102 are used as information for identifying the orientation of the target blood vessel 100. Therefore, the second orientation information can be rephrased as information for identifying the orientations of the characteristic portions 101 and 102 included in the target blood vessel 100. The second orientation information includes a blood vessel axis vector Va2 ( FIG. 8 ) representing the orientation of the characteristic portion 101 ( FIG. 6 ) in the two-dimensional images (Img21, Img22) and a blood vessel axis vector Vb2 ( FIG. 12 ) representing the orientation of the characteristic portion 102. A method by which the estimation unit 34 estimates the second orientation information will be described later.

[0019] The generation unit 35 generates a display image Img1 ( FIG. 5 ) and generates a superimposed image Img3 ( FIG. 10 ) of the display image Img1 and the first two-dimensional image Img21 ( FIG. 6 ). The generation unit 35 generates the display image Img1 based on the three-dimensional image data acquired by the three-dimensional image data acquisition unit 31. In this embodiment, the display image Img1 is a two-dimensional image generated based on the three-dimensional image data of the target blood vessel 100. The generation unit 35 renders the three-dimensional image data using a method such as surface rendering or volume rendering to generate the display image Img1. The generation unit 35 estimates correction information for more accurately superimposing the display image Img1 of the target blood vessel 100 ( FIG. 5 ) on the target blood vessel 100 represented in the first two-dimensional image Img21 based on the first orientation information and the second orientation information. The correction information is a vector representation of a correction amount for reducing the deviation in the orientation of the target blood vessel 100 in the first two-dimensional image Img21 ( FIG. 6 ) and the target blood vessel 100 displayed in the display image Img1. The generation unit 35 generates a display image Img1 from the three-dimensional image data, in which the orientation of the target blood vessel (100) displayed in the display image Img1 has been adjusted, based on the acquired correction information. The generation unit 35 adjusts the size of each image so that the scale of the display image Img1 and the scale of the first two-dimensional image Img21 are the same. The generation unit 35 superimposes the display image Img1 and the first two-dimensional image Img21 so that the positions of the target blood vessel 100 in the display image Img1 and the target blood vessel 100 in the first two-dimensional image Img21 are aligned, thereby generating a superimposed image Img3 (10). The generation unit 35 generates two-dimensional images (Img21, Img22) based on the two-dimensional image data acquired by the two-dimensional image data acquisition unit 32. In this embodiment, the generation unit 35 generates a first two-dimensional image Img21 based on the first two-dimensional image data acquired by the two-dimensional image data acquisition unit 32, and generates a second two-dimensional image Img22 based on the second two-dimensional image data acquired by the two-dimensional image data acquisition unit 32.

[0020] The step of aligning the positions of the target blood vessel 100 ( FIG. 5 ) in the display image Img1 ( FIG. 5 ) and the target blood vessel 100 ( FIG. 6 ) in the first two-dimensional image Img21 ( FIG. 6 ) to generate the superimposed image Img3 ( FIG. 10 ) may be automatically processed by the generation unit 35. When the step of generating the superimposed image Img3 is automatically processed, the target blood vessel 100 in the display image Img1 and the target blood vessel 100 in the first two-dimensional image Img21 are extracted by image processing, and coordinate information on the image associated with the pixel displaying the target blood vessel 100 is obtained. The superimposed image Img3 can be generated by superimposing the display image Img1 and the first two-dimensional image Img21 using the coordinate information of the target blood vessel 100 in the display image Img1 and the coordinate information of the target blood vessel 100 in the first two-dimensional image Img21.

[0021] <Hardware Details> The ROM 41 is a non-volatile storage device that stores an IPL (initial program loading) that reads the OS (operating system) from the HDD 43, expands it into the RAM 42, and starts it up, as well as a BIOS (basic input / output system) that controls peripheral devices such as an input device 47.

[0022] The RAM 42 is a volatile storage device used as a work memory for the CPU 30 and for temporary storage.

[0023] The HDD 43 is a storage device that stores programs necessary for the medical device 1 to perform its functions, including a three-dimensional image acquisition program, a two-dimensional image acquisition program, a position information acquisition program, a posture estimation program, and a superimposed image generation program.

[0024] The communication control device 44 is a communication interface for connecting to a network 48 wirelessly or via a wire, and controls communication in accordance with a communication standard.

[0025] The network 48 may be a public network such as the Internet, a private network such as a local area network (LAN) or a wide area network (WAN), or any combination thereof.

[0026] <Program Details> Computer programs for causing the medical device 1 to function include a three-dimensional image data acquisition program, a two-dimensional image data acquisition program, a position information acquisition program, a posture estimation program, and a superimposed image generation program. The CPU 30 is a processor for executing the computer programs for causing the medical device 1 to function. When a user of the medical device 1 operates the input device 47 and inputs commands to execute various programs, the CPU 30 loads the programs stored in the HDD 43 into the RAM 42 and executes them. Commands are sent from the CPU 30 to the X-ray imaging device 10 and the CT device 20 via the network 48, and the X-ray imaging device 10 and the CT device 20 perform operations based on the commands.

[0027] The CPU 30 executes a three-dimensional image data acquisition program to realize the functions of a three-dimensional image data acquisition unit 31. The three-dimensional image data acquisition unit 31 operates the CT device 20 (FIG. 2), which acquires tomographic image data of a patient 200 (FIG. 2). The three-dimensional image data acquisition unit 31 acquires three-dimensional image data of a target blood vessel 100 (FIG. 5) based on the acquired tomographic image data of the patient 200. The three-dimensional image data acquisition unit 31 acquires first orientation information for specifying the orientation of the target blood vessel 100 when the three-dimensional image data of the target blood vessel 100 is acquired.

[0028] The CPU 30 executes a two-dimensional image data acquisition program to realize the function of the two-dimensional image data acquisition unit 32. The two-dimensional image data acquisition unit 32 operates the X-ray imaging device 10 (FIG. 1), which images the target blood vessel 100 (FIG. 6) from a first imaging position and acquires first two-dimensional image data. The X-ray imaging device 10 images the target blood vessel 100 from a second imaging position and acquires second two-dimensional image data.

[0029] The CPU 30 executes a position information acquisition program to realize the function of the position information acquisition unit 33. The position information acquisition unit 33 acquires a first imaging position of the X-ray detection device 13 ( FIG. 1 ) when the first two-dimensional image data is captured. The position information acquisition unit 33 acquires a second imaging position of the X-ray detection device 13 when the second two-dimensional image data is captured. The position information acquisition unit 33 acquires first position information of the target blood vessel 100 in the first two-dimensional image Img21 generated from the first two-dimensional image data and second position information of the target blood vessel 100 in the second two-dimensional image Img22 generated from the second two-dimensional image data.

[0030] The CPU 30 executes the posture estimation program to realize the function of the estimation unit 34. The estimation unit 34 uses the information on the first imaging position, the information on the second imaging position, the first position information, and the second position information to identify the posture of the target blood vessel 100 ( FIG. 6 ) at the time of capturing the first two-dimensional image Img21 ( FIG. 6 ) and the second two-dimensional image Img22 ( FIG. 7 ), as second posture information.

[0031] The CPU 30 executes the superimposed image generation program to implement the functions of the generation unit 35. The generation unit 35 generates a display image Img1 based on the three-dimensional image data acquired by the three-dimensional image data acquisition unit 31. The generation unit 35 acquires two-dimensional image data of the target blood vessel 100 using the two-dimensional image data acquisition unit 32, and generates two-dimensional images (Img21, Img22) (FIGS. 6 and 7) from the two-dimensional image data. The generation unit 35 superimposes the position and orientation of the target blood vessel 100 in the display image Img1 with the position and orientation of the target blood vessel 100 in the first two-dimensional image Img21 to generate a superimposed image Img3 (FIG. 10) of the display image Img1 and the first two-dimensional image Img21.

[0032] <Details of Image Data Acquisition Method, Posture Information Acquisition Method, and Image Processing Method> Figure 4 is an explanatory diagram illustrating an example of an image processing method. The image processing method in this embodiment includes the following steps S1 to S11. (S1) In step S1, the three-dimensional image data acquisition unit 31 (Figure 3) acquires three-dimensional image data of the target blood vessel 100 (Figure 5) from tomographic image data of the patient 200 (Figure 2) acquired by the CT device 20 (Figure 2). (S2) In step S2, the three-dimensional image data acquisition unit 31 acquires first posture information for identifying the posture of the target blood vessel 100 at the time of acquisition of the three-dimensional image data. (S3) In step S3, the two-dimensional image data acquisition unit 32 (Figure 3) images the target blood vessel 100 from a first imaging position using the X-ray imaging device 10 (Figure 1) and acquires first two-dimensional image data. (S4) In step S4, the two-dimensional image data acquisition unit 31 images the target blood vessel 100 from a second imaging position using the X-ray imaging device 10, and acquires second two-dimensional image data. (S5) In step S5, the position information acquisition unit 33 ( FIG. 3 ) acquires information on a first imaging position of the X-ray imaging device 10 and first position information on the target blood vessel 100. (S6) In step S6, the position information acquisition unit 33 acquires information on a second imaging position of the X-ray imaging device 10 and second position information on the target blood vessel 100. (S7) In step S7, the estimation unit 34 ( FIG. 3 ) estimates second orientation information for specifying the orientation of the target blood vessel 100 when the patient 200 was imaged by the X-ray imaging device 10, using the information on the first imaging position and the first position information, and the information on the second imaging position and the second position information. (S8) In step S8, the generator 35 (FIG. 3) uses the first orientation information and the second orientation information of the target blood vessel 100 to estimate correction information for superimposing the orientation of the target blood vessel 100 in the first two-dimensional image Img21 on the orientation of the target blood vessel 100 when the three-dimensional image data was acquired. (S9) In step S9, the generator 35 (FIG. 3) generates a display image Img1 from the three-dimensional image data using the correction information. (S10) In step S10, the generator 35 generates a first two-dimensional image Img21 (FIG. 6) from the first two-dimensional image data.(S11) In step S11, the generation unit 35 generates a superimposed image Img3 (FIG. 10) by superimposing the display image Img1 and the first two-dimensional image Img21.

[0033] FIG. 5 is an explanatory diagram illustrating a display image Img1. The generation unit 35 (FIG. 3) generates the display image Img1 based on three-dimensional image data of the patient 200 (FIG. 2) acquired by the CT device 20 (FIG. 2) and correction information. The three-dimensional image Img0 shown in FIG. 5 is a three-dimensional image of the target blood vessel 100 constructed based on the three-dimensional image data in a virtual three-dimensional space. The display image Img1 can be expressed as a two-dimensional image projected when the three-dimensional image Img0 is observed from a specific viewpoint (e.g., the direction indicated by the arrow labeled "View Angle" in FIG. 5). Of the target blood vessel 100, a portion of the blood vessel branch is referred to as a characteristic portion 101, and another portion of the blood vessel branch is referred to as a characteristic portion 102. The first orientation information of the target blood vessel 100 described above includes orientation information of the characteristic portion 101 and orientation information of the characteristic portion 102 when the patient 200 was imaged by the CT device 20. The orientation information of the characteristic portion 101 includes a blood vessel axis vector Va1 extending along the axis of the characteristic portion 101. The orientation information of the characteristic portion 102 includes a blood vessel axis vector Vb1 extending along the axis of the second characteristic portion 102. When generating the display image Img1, the three-dimensional image of the target blood vessel 100 is rotated in three-dimensional space so that the orientations of the vectors Va1 and Vb1 are aligned with the orientations of the vectors Va2 and Vb2, and a two-dimensional image of the three-dimensional image Img0 obtained when these are aligned is generated as the display image Img1 when observed from a specific viewpoint. For example, when generating the display image Img1 to be superimposed on the first two-dimensional image Img21 ( FIG. 6 ), the display image Img1 is generated, depicting the target blood vessel 100 such that the orientations of the vectors Va1 and Vb1 are aligned with the orientations of the vectors Va2 and Vb2 ( FIG. 8 ) as viewed from the first two-dimensional image Img21. The correction information is information for aligning the directions of vectors Va1 and Vb1 with the directions of vectors Va2 and Vb2. In this embodiment, the characteristic portions (101, 102) are set to two blood vessels extending in different directions at the bifurcation of the target blood vessel 100 as described above, but the method for setting the characteristic portions (101, 102) is not limited to the method shown in this embodiment. The characteristic portions (101, 102) may be identified automatically by the medical device 1 or manually by a user of the medical device 1 ( FIG. 1 ).When performed manually, an image generated based on three-dimensional image data of the target blood vessel 100 may be displayed on the display device 46 (Figure 3), and the user of the medical device 1 may select characteristic areas (101, 102) using the input device 47 (Figure 3).

[0034] 6 is an explanatory diagram illustrating a first two-dimensional image Img21. The first two-dimensional image Img21 is an X-ray fluoroscopic image of the target blood vessel 100 captured with the X-ray detection device 13 ( FIG. 1 ) positioned at the first imaging position. The first two-dimensional image Img21 is generated based on the first two-dimensional image data. The first imaging position of the X-ray detection device 13 can be represented by the position and orientation of the X-ray detection device 13 relative to the imaging target, and the X-ray detection device 13 in this embodiment can be represented as LAO 90° and CRA 0°. The case where the X-ray detection device 13 is disposed on the left side of the body of the patient 200 is called LAO (Left Anterior Oblique view), and the case where the X-ray detection device 13 is disposed on the right side of the body of the patient 200 is called RAO (Right Anterior Oblique view). The case where the X-ray detection device 13 is disposed on the head side of the imaging region of the patient 200 is called CRA (CRAnial), and the case where the X-ray detection device 13 is disposed on the leg side of the imaging region of the patient 200 is called CAU (CAUdal). The imaging position of the X-ray detection device 13 can be specified by the above-mentioned four positions (LAO, RAO, CRA, CAU) and the orientation with respect to the imaging target. The Z1 axis shown in Figure 6 is a projection image of the Z axis (Figure 1) of the universal coordinate system projected onto the first two-dimensional image Img21. The Z1 axis is perpendicular to a vector A1 (not shown) directed from the X-ray generator 12 to the X-ray detector 13 when the first two-dimensional image Img21 is captured. The Z1 axis can be obtained by determining a vector H1 (not shown) perpendicular to the vector A1 and the Z1 axis, and then rotating the vector A1 by 90 degrees around the vector H1. The vector H1 can be obtained by the cross product of the vector A1 and the Z axis.

[0035] The position information acquisition unit 33 ( FIG. 3 ) acquires, as first position information, position information of the characteristic portion 101 and position information of the characteristic portion 102 in the first two-dimensional image Img21. The position information of the characteristic portion 101 includes information of a plane Sx1 that passes through the characteristic portion 101. The plane Sx1 is a plane that includes a straight line Lx1 that runs along the characteristic portion 101 in the first two-dimensional image Img21 and extends in the same direction as the above-mentioned vector A1. The position information of the characteristic portion 102 includes information of a plane Sx2 that runs through the characteristic portion 102. The plane Sx2 is defined as a plane that includes a straight line Lx2 that runs along the characteristic portion 102 in the first two-dimensional image Img21 and extends in the same direction as the vector A1.

[0036] FIG. 7 is an explanatory diagram illustrating a second two-dimensional image Img22. The second two-dimensional image Img22 is an X-ray fluoroscopic image of the target blood vessel 100 captured when the X-ray detection device 13 ( FIG. 1 ) is positioned at the second imaging position. The second two-dimensional image Img22 is generated based on the second two-dimensional image data. In this embodiment, the second imaging position of the X-ray detection device 13 is LAO 30°, CRA 30°. The Z2 axis shown in FIG. 7 is a projection image of the Z axis ( FIG. 1 ) of the common coordinate system projected onto the second two-dimensional image Img22. The Z2 axis is perpendicular to a vector A2 (not shown) extending from the X-ray generator 12 of the X-ray imaging device 10 toward the X-ray detection device 13 when the second two-dimensional image Img22 is captured. The Z2 axis can be determined by calculating vector A2 and vector H2 (not shown) perpendicular to the Z2 axis, and then rotating vector A2 by 90 degrees around vector H2. Vector H2 can be determined by the cross product of vector A2 and the Z axis.

[0037] The position information acquisition unit 33 ( FIG. 3 ) acquires, as second position information, position information of the characteristic portion 101 and position information of the characteristic portion 102 in the second two-dimensional image Img22. The position information of the characteristic portion 101 includes information on a plane Sy1 that passes through the characteristic portion 101. The plane Sy1 is a plane that includes a straight line Ly1 that runs along the characteristic portion 101 in the second two-dimensional image Img22 and extends in the same direction as the above-mentioned vector A2. The position information of the characteristic portion 102 includes information on a plane Sy2 that runs through the characteristic portion 102. The plane Sy2 is defined as a plane that includes a straight line Ly2 that runs along the characteristic portion 102 in the second two-dimensional image Img22 and extends in the same direction as the vector A2.

[0038] FIG. 8 is an explanatory diagram illustrating a method for estimating second orientation information. FIG. 8 shows two planes (Sx1, Sy1) passing through the characteristic portion 101 ( FIG. 6 ) and a vascular axis vector Va2. The estimation unit 34 ( FIG. 3 ) identifies a straight line defined by the intersection of the planes Sx1 and Sy1 passing through the characteristic portion 101 as the vascular axis vector Va2 along the characteristic portion 101. Similarly, the estimation unit 34 identifies a straight line defined by the intersection of the planes Sx2 ( FIG. 6 ) and Sy2 ( FIG. 7 ) passing through the characteristic portion 102 ( FIG. 6 ) as the vascular axis vector Vb2 ( FIG. 12 ) along the characteristic portion 102. The second orientation information includes the vascular axis vector Va2 and the vascular axis vector Vb2, and is information representing the orientation of the target blood vessel 100 when the patient 200 ( FIG. 1 ) is imaged with the X-ray imaging device 10 ( FIG. 1 ).

[0039] FIG. 9 is an explanatory diagram illustrating a method for inputting position information of the target blood vessel 100 into the first two-dimensional image Img21. FIG. 9 shows the first two-dimensional image Img21 displayed on the display device 46 and an input device 47b. The display device 46 is a monitor electrically connected to the medical device 1 (FIG. 1) and displays data and images acquired by the X-ray imaging device 10 (FIG. 1) or the CT device 20 (FIG. 2). The input device 47b is a mouse electrically connected to the medical device 1. A user of the medical device 1 can manually input straight lines (Lx1, Lx2, Ly1, Ly2) (FIGS. 6 and 7) along the characteristic portions (101, 102) into the medical device 1 to assist in obtaining a plane (Sx1, Sy1) (FIGS. 6 and 7) passing through the characteristic portions (101, 102). For example, the user moves the pointer Pt, which is linked to the input device 47b, along the characteristic portion 101 in the first two-dimensional image Img21 displayed on the display device 46 to draw a straight line Lx1, and similarly draws a straight line Lx2 along the characteristic portion 102. The position information acquisition unit 33 defines a plane Sx1 from the line Lx1 and defines a plane Sx2 from the line Lx2. Similarly, the user uses the input device 47b to draw a straight line Ly1 along the characteristic portion 101 and a straight line Ly2 along the characteristic portion 102 in the second two-dimensional image Img22 displayed on the display device 46. The position information acquisition unit 33 defines a plane Sy1 from the line Ly1 and defines a plane Sy2 from the line Ly2. Furthermore, if the display device 46 is a touch panel display and has a function for identifying the position of a user's finger or a touch pen touching the display, the user's finger or touch pen may be moved along the characteristic portions (101, 102) in the two-dimensional images (Img21, Img22) to draw a straight line (Lx1, Lx2, Ly1, Ly2). As described above, the vascular axis vector (Va2, Vb2) (FIGS. 8 and 12), which is the second posture information, may be estimated from the straight lines (Lx1, Lx2, Ly1, Ly2) specified by the user. By specifying the straight lines (Lx1, Lx2, Ly1, Ly2) passing through the characteristic portions (101, 102) by the user of the medical device 1 (FIG. 1), a superimposed image Img3 closer to the user's expectations can be generated.

[0040] FIG. 10 is an explanatory diagram illustrating a superimposed image Img3. FIG. 10 shows the superimposed image Img3 obtained by superimposing the display image Img1 ( FIG. 5 ) and the first two-dimensional image Img21 ( FIG. 6 ). In FIG. 10 , the target blood vessel 100 in the display image Img1 is indicated by a solid line, and the target blood vessel 100 in the first two-dimensional image Img21 is indicated by a dotted line. Based on the vascular axis vector (Va1, Vb1) ( FIG. 5 ), which is the first orientation information, and the vascular axis vector (Va2, Vb2) ( FIGS. 8 and 12 ), which is the second orientation information, correction information is estimated for superimposing the orientation of the target blood vessel 100 when the patient 200 ( FIG. 2 ) is imaged by the CT device 20 ( FIG. 2 ) and the orientation of the target blood vessel 100 when the target blood vessel 100 is imaged by the X-ray imaging device 10 ( FIG. 1 ). The display image Img1 is generated from the three-dimensional image data based on the estimated correction information.

[0041] The medical device 1 ( FIG. 1 ) can generate a superimposed image Img3 ( FIG. 10 ) of the display image Img1 ( FIG. 5 ) and the first two-dimensional image Img21 ( FIG. 6 ) using the above-described technology. The imaging of the patient 200 ( FIG. 2 ) using the CT device 20 ( FIG. 2 ) may be performed on the day before the treatment of the target blood vessel 100. Due to such circumstances, the position of the patient 200 when the patient 200 ( FIG. 1 ) is imaged using the X-ray imaging device 10 ( FIG. 1 ) on the day of treatment does not exactly match the position of the patient 200 when the patient 200 is imaged using the CT device 20. Therefore, even if an attempt is made to superimpose an X-ray fluoroscopic image taken during treatment on a CT image prepared in advance, there may be a discrepancy in the posture of the target blood vessel 100 between the images, making it difficult to superimpose the positions of the blood vessels in the respective images. According to the medical device 1 of this embodiment, the degree of deviation between the orientation of the target blood vessel 100 when the patient 200 is imaged by the CT device 20 and the orientation of the target blood vessel 100 when the patient 200 ( FIG. 1 ) is imaged by the X-ray imaging device 10 ( FIG. 1 ), due to differences in the patient 200's position during imaging, can be estimated as correction information, and a display image Img1 can be generated based on the correction information. As a result, the orientation of the target blood vessel 100 in the display image Img1 approaches the orientation of the target blood vessel 100 in the first two-dimensional image Img21, allowing for the generation of a superimposed image Img3 in which the target blood vessels displayed in each image are more accurately superimposed. The user can proceed with treatment by checking the target blood vessel 100 in the superimposed image Img3. This allows the user to reduce the frequency of injecting contrast medium to acquire X-ray fluoroscopic images during surgery to understand the course of the blood vessel, thereby enabling more minimally invasive treatment.

[0042] The medical device 1 (FIG. 1) uses the first posture information and the second posture information to align the posture of the target blood vessel 100 (FIG. 2) when the patient 200 (FIG. 2) is imaged by the CT device 20 (FIG. 2) with the posture of the target blood vessel 100 (FIG. 1) when the patient 200 (FIG. 1) is imaged by the X-ray imaging device 10 (FIG. 1), thereby enabling more accurate alignment of the posture of each target blood vessel 100.

[0043] By using the information on the first imaging position and the information on the second imaging position of the X-ray imaging device 10 (Figure 1), as well as the first position information of the target blood vessel 100 representing the position of the target blood vessel 100 (Figure 6) in the first two-dimensional image Img21 (Figure 6) and the second position information representing the position of the target blood vessel 100 (Figure 7) in the second two-dimensional image Img22 (Figure 7), it becomes easy to identify the posture of the target blood vessel 100 when the target blood vessel 100 is imaged by the X-ray imaging device 10.

[0044] The medical device 1 (Fig. 1) uses the vector (Va1, Vb1) (Fig. 5) included in the first posture information and the vector (Va2, Vb2) (Figs. 8, 12) included in the second posture information to align the orientation of the target blood vessel 100 (Fig. 2) when the patient 200 (Fig. 2) is imaged by the CT device 20 (Fig. 2) with the orientation of the target blood vessel 100 when the target blood vessel 100 (Fig. 1) is imaged by the X-ray imaging device 10 (Fig. 1), thereby enabling the posture of each target blood vessel 100 to be aligned more accurately.

[0045] <Correction Method> Fig. 11 is an explanatory diagram illustrating a first estimation method of correction information. The first estimation method is a method of aligning the orientation of the vascular axis vector (Va1, Vb1) (Fig. 12) with the orientation of the vascular axis vector (Va2, Vb2) (Fig. 12) by rotation about one rotation axis vector C (Fig. 12). The first estimation method includes the following steps S1a to S5a. (S1a) In step S1a, the generator 35 (Fig. 3) rotates the vascular axis vector (Va1, Vb1) (Fig. 12) by an angle θa (Fig. 13) to assume, as a rotation axis vector C, an axis that serves as the center of rotation when aligning the orientation of the vascular axis vector Va1 with the orientation of the vascular axis vector Va2 (Fig. 12) and the orientation of the vascular axis vector Vb1 with the orientation of the vascular axis vector Vb2 (Fig. 12). (S2a) In step S2a, the generator 35 calculates the rotation axis vector C from the cross product of the vector (Va2-Va1) and the vector (Vb2-Vb1). (S3a) In step S3a, the generator 35 calculates the rotation angle θa from the vector d1 (FIG. 13) extending perpendicularly from the vascular axis vector Va1 toward the rotation axis vector C and the vector d2 (FIG. 13) extending perpendicularly from the vascular axis vector Vb1 toward the rotation axis vector C. (S4a) In step S4a, the generator 35 aligns the orientation of the rotation axis vector C with the orientation of the X-ray detection device 13 (FIG. 1) when the target blood vessel 100 was imaged by the X-ray imaging device 10. (S5a) In step S5a, the generator 35 rotates the vascular axis vector (Va1, Vb1) by the angle θa around the rotation axis vector C. The vascular axis vector Va1, vascular axis vector Vb1, vascular axis vector Va2, vascular axis vector Vb2, and rotation axis vector C only require direction, not magnitude, and are therefore treated as unit vectors.

[0046] Fig. 12 is an explanatory diagram illustrating a method for deriving the rotation axis vector C. The generator 35 (Fig. 3) rotates the vascular axis vector (Va1, Vb1) (Fig. 12) by an angle θa (Fig. 13) to assume, as the rotation axis vector C, an axis that serves as the center of rotation when aligning the orientation of the vascular axis vector Va1 with the orientation of the vascular axis vector Va2 (Fig. 12) and the orientation of the vascular axis vector Vb1 with the orientation of the vascular axis vector Vb2 (Fig. 12).

[0047] The generation unit 35 (FIG. 3) calculates the rotation axis vector C. The rotation axis vector C can be calculated using the cross product of the vectors (Va2-Va1) and (Vb2-Vb1), as shown in the following equation (1). The vectors (Va2-Va1) and (Vb2-Vb1) are perpendicular to the rotation axis vector C. Because the cross product has the property of being a vector perpendicular to any two vectors, it is possible to calculate a vector perpendicular to the two vectors using the cross product. By calculating the cross product of the vectors (Va2-Va1) and (Vb2-Vb1), it is possible to calculate the vector C, which is perpendicular to the vectors (Va2-Va1) and (Vb2-Vb1). Here, angle ω (not shown) is the angle formed by vector (Va2-Va1) and vector (Vb2-Vb1), and angle ω can be calculated using the dot product of vector (Va2-Va1) and vector (Vb2-Vb1) as shown in the following equation (2). The rotation axis vector C can be obtained by substituting the angle ω obtained by the above equation (2) into the above equation (1).

[0048] Fig. 13 is an explanatory diagram illustrating a method for deriving the angle θa. The generator 35 (Fig. 3) calculates the angle θa. The angle θa is equal to the angle formed by a vector d1 extending perpendicularly from the vascular axis vector Va1 toward the rotation axis vector C and a vector d2 extending perpendicularly from the vascular axis vector Va2 toward the rotation axis vector C. Therefore, the angle θa can be calculated based on the dot product of the vector d1 and the vector d2. Fig. 13 illustrates a plane Sv perpendicular to the rotation axis vector C. where: where, Then, the numerator of the above formula (3) is If the angle formed by the blood vessel axis vector Va1 and the rotation axis vector C is defined as angle Δ, then and the angle between the vascular axis vector Va1 and the rotation axis vector C is the same as the angle between the vascular axis vector Va2 and the rotation axis vector C, so Therefore, the denominator of the above formula (3) is where, Therefore, Therefore, the above equation (3) can be expressed as the following equation (4), and the angle θa can be found from the blood vessel axis vector Va1, the blood vessel axis vector Va2, and the rotation axis vector C.

[0049] The generator 35 (FIG. 3) aligns the orientation of the rotation axis vector C with the orientation of the X-ray detection device 13 (FIG. 1) when the target blood vessel 100 is imaged by the X-ray imaging device 10. As an example, assume that the rotation axis vector C does not coincide with any of the X-axis, Y-axis, or Z-axis of the universal coordinate system (FIG. 1), and explain the procedure for aligning the rotation axis vector C with the Y-axis. First, the rotation axis vector C is rotated by an angle α around the X-axis using the rotation matrix Rx(α) of the following equation (5), and the rotation axis vector C is moved onto the XY plane. The rotation angle α (not shown) around the X-axis when the rotation axis vector C is moved on the XY plane is equal to the angle between the unit vector Ey (not shown) in the Y-axis direction and the vector Cyz (0, cy, cz) (not shown), which has the same y and z components as the rotation axis vector C (cx, cy, cz) and exists on the YZ plane, and can be derived as follows: Here, when cz>0, it becomes Rx(α), and when cz<0, it becomes Rx(−α).

[0050] The rotation axis vector C rotated around the X axis by a rotation angle α is represented as the rotation axis vector Rx(α)C. The rotation axis vector Rx(α)C is then rotated around the Z axis by an angle β (not shown) using the rotation matrix Rz(β) in the following equation (6) to align it with the Y axis. The angle β around the Z axis is the angle between the rotation axis vector Rx(α)C and the unit vector Ey in the Y axis direction, so it can be derived as follows: As described above, by rotating the rotation axis vector C by angle α around the X axis using the rotation matrix Rx(α), and then further rotating the rotation axis vector Rx(α)C by angle β around the Z axis, it is possible to align the rotation axis vector C with the Y axis. By aligning the Y axis with the orientation of the X-ray detection device 13 ( FIG. 1 ) when the target blood vessel 100 is imaged by the X-ray imaging device 10, it is possible to align the orientation of the rotation axis vector C with the orientation of the X-ray detection device 13 ( FIG. 1 ).

[0051] The generator 35 (FIG. 3) rotates the vascular axis vector (Va1, Vb1) by an angle θa around the rotation axis vector C (Y axis) using the rotation matrix Ry(θa) of the following equation (7), where θa is expressed by the above equation (4). As a result, the direction of the blood vessel axis vector (Va1, Vb1) and the direction of the blood vessel axis vector (Va2, Vb2) can be aligned.

[0052] FIG. 14 is an explanatory diagram illustrating a second estimation method for correction information. The second estimation method aligns the orientation of the vascular axis vector (Va1, Vb1) with the orientation of the vascular axis vector (Va2, Vb2) by rotating the vascular axis vectors around two rotation axis vectors (C1, C2). The second estimation method includes the following steps S1b to S8b. (S1b) In step S1b, the generator 35 (FIG. 3) rotates a plane St1 (FIG. 15) containing the vascular axis vector Va1 (FIG. 15) and the vascular axis vector Vb1 (FIG. 15) by an angle θb (FIG. 15) to assume that the axis serving as the center of rotation when the plane St1 (FIG. 15) is superimposed on a plane St2 (FIG. 15) containing the vascular axis vector Va2 (FIG. 15) and the vascular axis vector Vb2 (FIG. 15) is the rotation axis vector C1 (FIG. 15). In FIG. 14, the rotation axis vector C1 is represented as the first rotation axis vector, and the angle θb is represented as the first rotation angle. (S2b) In step S2b, the generator 35 calculates the rotation axis vector C1 from the cross product of vector n1 (FIG. 15) and vector n2 (FIG. 15). Vector n1 is the normal vector of plane St1, and vector n2 is the normal vector of plane St2. (S3b) In step S3b, the generator 35 calculates the angle θb from the dot product of vector n1 and vector n2. (S4b) In step S4b, the generator 35 rotates the vascular axis vector (Va1, Vb1) by the angle θb around the rotation axis vector C1. The vascular axis vector (Va1, Vb1) projected onto plane St2 by this rotation is called the vascular axis vector (Va1', Vb1') (FIG. 16). (S5b) In step S5b, the generator 35 rotates the vascular axis vector (Va1', Vb1') by an angle Φ (not shown) and assumes that the axis serving as the center of rotation when superimposing it on the vascular axis vector (Va2, Vb2) is a rotation axis vector C2 ( FIG. 17 ). The angle Φ is the average value of the angle Φa between Va1' and Va2, which will be described later, and the angle Φb between Vb1' and Vb2, which will be described later. In FIG. 14 , the rotation axis vector C2 (normal vector n2) is represented as a second rotation axis vector, and the angle Φ is represented as a second rotation angle. (S6b) In step S6b, the generator 35 calculates the rotation axis vector C2. Since the rotation axis vector C2 is a vector perpendicular to the plane St2, the normal vector n2 can be treated as the rotation axis vector C2.(S7b) In step S7b, the generator 35 calculates the angle Φa between Va1' and Va2 by calculating the dot product of Va1' and Va2. The generator 35 calculates the angle Φb between Vb1' and Vb2 by calculating the dot product of Vb1' and Vb2. The generator 35 (FIG. 3) calculates the angle Φ as the average value of the angles Φa and Φb. (S8b) In step S8b, the generator 35 rotates the vascular axis vector (Va1', Vb1') by the angle Φ around the rotation axis vector C2.

[0053] Fig. 15 is an explanatory diagram illustrating a method for deriving the rotation axis vector C1. The generator 35 (Fig. 3) rotates the plane St1 containing the vascular axis vector Va1 (Fig. 11) and the vascular axis vector Vb1 (Fig. 11) by an angle θb (Fig. 12) to superimpose it on the plane St2 containing the vascular axis vector Va2 (Fig. 12) and the vascular axis vector Vb2 (Fig. 12), and assumes that the axis serving as the center of rotation is the rotation axis vector C1 (Fig. 11).

[0054] The generator 35 (FIG. 3) calculates the rotation axis vector C1. The rotation axis vector C1 can be calculated from the cross product of vector n1 and vector n2 as shown in the following equation (8). Vector n1 is the normal vector of plane St1, and vector n2 is the normal vector of plane St2. The vector n1 can be obtained from the cross product of the blood vessel axis vector Va1 and the blood vessel axis vector Vb1 as shown in the following equation (9). Vector n2 can be obtained from the cross product of blood vessel axis vector Va2 and blood vessel axis vector Vb2 as shown in the following equation (10). Here, the angle θb is the angle formed by the vectors n1 and n2, and the angle θb can be determined using the inner product of the vectors n1 and n2 as shown in the following equation (11). The rotation axis vector C1 can be determined by substituting the angle θb determined by the above equation (11) into the above equation (8).

[0055] 16 is an explanatory diagram showing the relationship between the rotation axis vector C1 and a plane (St1, St2) including the vascular axis vectors (Va1, Vb1, Va2, Vb2). The generator 35 (FIG. 3) rotates the vascular axis vector (Va1, Vb1) by an angle θb around the rotation axis vector C1. By rotating the vascular axis vector (Va1, Vb1) by the angle θb, the vascular axis vector (Va1, Vb1) is projected onto the plane St2. The vascular axis vector (Va1, Vb1) projected onto the plane St2 is called the vascular axis vector (Va1', Vb1').

[0056] 17 is an explanatory diagram illustrating the state in which vascular axis vectors (Va1, Vb1, Va2, Vb2) are projected onto the same plane. The generator 35 (FIG. 3) rotates the vascular axis vectors (Va1', Vb1') by an angle Φ (not shown) to align the orientations of the vascular axis vectors Va1' and Va2, and the orientations of the vascular axis vectors Vb1' and Vb2. The rotation axis vector C2 is assumed to be the axis around which the rotation is centered when aligning the orientations of the vascular axis vectors Va1' and Va2, and the orientations of the vascular axis vectors Vb1' and Vb2. Because the rotation axis vector C2 is perpendicular to the plane St2, the normal vector n2 can be treated as the rotation axis vector C2.

[0057] The angle Φa formed by Va1' and Va2 can be determined by the inner product of Va1' and Va2. The angle Φb formed by Vb1' and Vb2 can be determined from the inner product of Vb1' and Vb2. The angle Φ is calculated as the average value of the angles Φa and Φb. The generator 35 rotates the vascular axis vector (Va1', Vb1') by the angle Φ around the rotation axis vector C2. As a result, the orientations of the vascular axis vectors Va1' and Va2, and the orientations of the vascular axis vectors Vb1' and Vb2 can be aligned.

[0058] <Modifications> The present disclosure is not limited to the above-described embodiment, and can be implemented in various forms without departing from the gist thereof. For example, the following modifications are also possible.

[0059] <Modification 1> The display image Img1 ( FIG. 5 ), the first two-dimensional image Img21 ( FIG. 6 ), or the second two-dimensional image Img22 ( FIG. 7 ) is not limited to a blood vessel image of the patient 200 ( FIG. 1 ), but may be an image of a biological lumen, such as the lymphatic system, biliary system, urinary system, respiratory system, digestive system, secretory glands, or reproductive organs, captured at different times of the patient 200 ( FIG. 1 ). The medical device 1 ( FIG. 1 ), image processing method, and program for image processing of the present disclosure can be used for the images of the aforementioned biological lumen.

[0060] <Modification 2> The computer program may include programs other than the three-dimensional image data acquisition program, the two-dimensional image data acquisition program, the position information acquisition program, the posture estimation program, and the weight image generation program.

[0061] <Modification 3> The image processing method shown in Fig. 4 may omit some steps, or may include other steps not included in Fig. 4. The order of the image processing method shown in Fig. 4 is not limited to the order shown in Fig. 4, and may be changed.

[0062] <Variation 4> During treatment, the body of the patient 200 ( FIG. 1 ) is constantly moving due to factors such as heartbeat and changes in thoracic volume caused by breathing. Therefore, the position and orientation of the target blood vessel 100 ( FIG. 2 ) when the patient 200 ( FIG. 2 ) is imaged by the CT device 20 ( FIG. 2 ) and the position and orientation of the target blood vessel 100 ( FIG. 1 ) when the patient 200 ( FIG. 1 ) is imaged by the X-ray imaging device 10 ( FIG. 1 ) are affected by heartbeat and breathing. In such a case, the cardiac pulsation cycle when the patient 200 is imaged by the CT device 20 may be recorded, and when the target blood vessel 100 is imaged by the X-ray imaging device 10, the image may be captured in accordance with the cardiac pulsation cycle when the patient 200 was imaged by the CT device 20. This makes it easier to align the position and orientation of the target blood vessel 100 when the patient 200 is imaged by the CT device 20 with the position and orientation of the target blood vessel 100 when the patient 200 is imaged by the X-ray imaging device 10.

[0063] <Variation 5> The medical device 1 ( FIG. 1 ) may generate a superimposed image by superimposing the display image Img1 on the second two-dimensional image Img22 ( FIG. 7 ), instead of the superimposed image Img3 ( FIG. 10 ) of the display image Img1 ( FIG. 5 ) and the first two-dimensional image Img21 ( FIG. 6 ). When superimposing the display image Img1 on the second two-dimensional image Img22, the orientation of the vector (Va1, Vb1) ( FIG. 5 ) included in the first posture information is aligned with the orientation of the vector (Va2, Vb2) ( FIGS. 8 and 12 ) included in the second posture information. The display image Img1 is generated using the view angle ( FIG. 5 ) for the three-dimensional image shown in FIG. 5 as the imaging direction (the position and orientation of the X-ray detection device 13 relative to the imaging target) of the X-ray detection device 13 ( FIG. 1 ) when the second two-dimensional image Img22 ( FIG. 7 ) was captured. A superimposed image is generated by superimposing the display image Img1 and the second two-dimensional image Img22 so that the positions of the target blood vessels 100 are aligned. In a similar manner, the medical device 1 can generate a superimposed image by superimposing the display image Img1 on a two-dimensional image of the target blood vessels 100 captured by the X-ray imaging device 1 ( FIG. 1 ) other than the first two-dimensional image Img21 and the second two-dimensional image Img22.

[0064] <Variation 6> The functions realized by the components described in this specification may be implemented in circuitry or processing circuitry, including general-purpose processors, application-specific processors, integrated circuits, ASICs (Application Specific Integrated Circuits), CPUs (Central Processing Units), conventional circuits, and / or combinations thereof, programmed to realize the described functions. A processor includes transistors and other circuits and is considered to be circuitry or processing circuitry. A processor may also be a programmed processor that executes a program stored in a memory.

[0065] <Variation 7> In this specification, a circuit, a unit, or a means is hardware that is programmed to realize or executes a described function. The hardware may be any hardware disclosed in this specification or any hardware known to be programmed to realize or execute the described function.

[0066] <Variation 8> When the hardware is a processor that is considered to be a type of circuitry, the circuitry, means, or unit is a combination of hardware and software used to configure the hardware and / or processor.

Claims

1. A medical device (1), comprising: a three-dimensional image data acquisition unit (31) that acquires three-dimensional image data of a target blood vessel (100) obtained by imaging the target blood vessel (100) with a tomography device and first orientation information for identifying the orientation of the target blood vessel (100); a two-dimensional image data acquisition unit (32) that acquires two-dimensional image data of the target blood vessel (100) imaged with an X-ray imaging device (10); and an estimation unit (34) that estimates second orientation information for identifying the orientation of the target blood vessel (100) in the X-ray imaging device (10); and using the first orientation information and the second orientation information, generates a display image (Img1) from the three-dimensional image data in which the orientation of the target blood vessel (100) to be displayed is corrected; and generates two-dimensional images (Img21, Img22) of the target blood vessel (100) from the two-dimensional image data. a generation unit (35) that generates a superimposed image (Img3) by superimposing the target blood vessel (100) represented in the display image (Img1) and the target blood vessel (100) represented in the two-dimensional images (Img21, Img22).

2. A medical device (1) according to claim 1, wherein the first orientation information is information for identifying the orientation of the target blood vessel (100) when the three-dimensional image data is acquired by the tomography imaging device.

3. A medical device (1) according to claim 1 or claim 2, wherein the first orientation information is information relating to two or more vectors (Va1, Vb1) for specifying the orientation of the target blood vessel (100) in the tomography imaging device, the two or more vectors (Va1, Vb1) following the shapes of characteristic portions (101, 102) of the target blood vessel (100), and the second orientation information is information relating to two or more vectors (Va2, Vb2) for specifying the orientation of the target blood vessel (100) in the X-ray imaging device (10), the two or more vectors (Va2, Vb2) following the shapes of the characteristic portions (101, 102).

4. A medical device (1) according to any one of claims 1 to 3, wherein the two-dimensional image data acquisition unit (32) acquires first two-dimensional image data obtained by imaging the target blood vessel (100) from a first imaging position using the X-ray imaging device (10), and second two-dimensional image data obtained by imaging the target blood vessel (100) from a second imaging position different from the first imaging position using the X-ray imaging device (10).

5. A medical device (1) according to claim 4, wherein the generation unit (35) generates a first two-dimensional image (Img21) from the first two-dimensional image data and generates a second two-dimensional image (Img22) from the second two-dimensional image data, and the medical device (1) further comprises a position information acquisition unit (33) that acquires information on the first imaging position and information on the second imaging position of the X-ray imaging device (10), as well as first position information representing the position of the target blood vessel (100) in the first two-dimensional image (Img21) and second position information representing the position of the target blood vessel (100) in the second two-dimensional image (Img22).

6. A medical device (1) according to claim 5, wherein the position information acquisition unit (33) comprises a display device (46) for displaying the first two-dimensional image (Img21) and the second two-dimensional image (Img22), and an input unit (47) for inputting the first position information and the second position information, and causes the display device (46) to display linear images corresponding to the first position information and the second position information, respectively, input by the input unit (47).

7. A medical device (1) according to claim 5 or claim 6, wherein the estimation unit (34) estimates the second posture information using information on the first imaging position and information on the second imaging position, as well as the first position information and the second position information.

8. An image processing method, comprising: acquiring three-dimensional image data of a target blood vessel (100) obtained by imaging the target blood vessel (100) with a tomography imaging device and first orientation information for identifying the orientation of the target blood vessel (100); acquiring two-dimensional image data of the target blood vessel (100) imaged with an X-ray imaging device (10); estimating second orientation information for identifying the orientation of the target blood vessel (100) in the X-ray imaging device (10); generating a display image (Img1) from the three-dimensional image data, in which the orientation of the target blood vessel (100) to be displayed is corrected, using the first orientation information and the second orientation information; generating two-dimensional images (Img21, Img22) of the target blood vessel (100) from the two-dimensional image data; An image processing method for generating a superimposed image (Img3) by superimposing the target blood vessel (100) shown in the display image (Img1) and the target blood vessel (100) shown in the two-dimensional images (Img21, Img22).

9. An image processing method according to claim 8, wherein the first orientation information is information for specifying the orientation of the target blood vessel (100) when the three-dimensional image data was acquired by the tomography device, and the method comprises the steps of: acquiring first two-dimensional image data obtained by imaging the target blood vessel (100) from a first imaging position using the X-ray imaging device (10); and acquiring second two-dimensional image data obtained by imaging the target blood vessel (100) from a second imaging position different from the first imaging position using the X-ray imaging device (10); generating a first two-dimensional image (Img21) from the first two-dimensional image data; and generating a second two-dimensional image (Img22) from the second two-dimensional image data; an image processing method for acquiring information on the first imaging position and information on the second imaging position of the X-ray imaging device (10), as well as first position information representing the position of the target blood vessel (100) in the first two-dimensional image (Img21) and second position information representing the position of the target blood vessel (100) in the second two-dimensional image (Img22); and estimating the second posture information using the information on the first imaging position and the information on the second imaging position, as well as the first position information and the second position information.

10. A computer program, which provides a computer (40) with the following functions: acquiring three-dimensional image data of a target blood vessel (100) obtained by imaging the target blood vessel (100) with a tomography device, and first orientation information for identifying the orientation of the target blood vessel (100); acquiring two-dimensional image data of the target blood vessel (100) imaged with an X-ray imaging device (10); and estimating second orientation information for identifying the orientation of the target blood vessel (100) in the X-ray imaging device (10); generating a display image (Img1) from the three-dimensional image data, in which the orientation of the target blood vessel (100) to be displayed is corrected, using the first orientation information and the second orientation information; and generating two-dimensional images (Img21, Img22) of the target blood vessel (100) from the two-dimensional image data; A computer program for realizing a function of generating a superimposed image (Img3) by superimposing the target blood vessel (100) shown in the display image (Img1) and the target blood vessel (100) shown in the two-dimensional images (Img21, Img22).

11. A computer program according to claim 10, wherein the first orientation information is information for specifying the orientation of the target blood vessel (100) when the three-dimensional image data was acquired by the tomography device, and the computer (40) is provided with the functions of acquiring first two-dimensional image data obtained by imaging the target blood vessel (100) from a first imaging position using the X-ray imaging device (10) and second two-dimensional image data obtained by imaging the target blood vessel (100) from a second imaging position different from the first imaging position using the X-ray imaging device (10), generating a first two-dimensional image (Img21) from the first two-dimensional image data, and generating a second two-dimensional image (Img22) from the second two-dimensional image data, A computer program for realizing a function of acquiring information on the first imaging position and information on the second imaging position of the X-ray imaging device (10), as well as first position information representing the position of the target blood vessel (100) in the first two-dimensional image (Img21) and second position information representing the position of the target blood vessel (100) in the second two-dimensional image (Img22), and a function of estimating the second posture information using the information on the first imaging position and the information on the second imaging position, as well as the first position information and the second position information.

12. A computer program product which, when a program according to claim 10 or claim 11 is executed by a processor, implements the functionality according to claim 10 or claim 11.

Images