Surgical assistance device, surgical assistance method, and computer program

Abstract

A surgical assistance device (1) is provided with: a true lumen information acquisition unit (14) that acquires three-dimensional position information of a true lumen present in a target blood vessel; a true lumen image generation unit (15) that acquires an angiographic image captured of the target blood vessel from the flat panel detector (21) disposed at the first imaging position, and generates a true lumen image of the true lumen indicating a position and orientation corresponding to the angiographic image using position information of the first imaging position and three-dimensional position information of the true lumen; and an image synthesis unit (16) that generates a synthesized image in which the angiography image and the true cavity image are synthesized, and outputs the synthesized image.

Description

Technical Field

[0001] The present invention relates to techniques for assisting surgery. Background Art

[0002] In recent years, IVUS (Intra Vascular Ultrasound) and FPD (Flat Panel Detector) have been used in blood vessel photography for inspection and treatment. IVUS is a device that has an ultra-small ultrasonic transducer at the front end and acquires ultrasonic images of the inside of blood vessels. FPD has an X-ray tube device and an X-ray plane detector, and is a device that acquires X-ray images of blood vessels. The X-ray image acquired by the FPD is also called an "angiography image." For example, Patent Document 1 describes a device that aligns a two-dimensional X-ray image of an area of ​​interest with three-dimensional ultrasonic data, determines the spatial relationship between a portion of an interventional device and a target location, and displays the spatial relationship. For example, Patent Document 2 describes an X-ray diagnostic device that generates a diagram representing the angle information of an FPD arm.

[0003] Prior art literature

[0004] Patent Literature

[0005] Patent Document 1: Japanese Patent No. 6770655

[0006] Patent Document 2: Japanese Patent Application Publication No. 2013-233413 Summary of the invention

[0007] Problems to be solved by the invention

[0008] Here, there are cases where the blood vessels are blocked by obstructions, such as chronic total occlusion (CTO). In this case, the blood vessels are reopened by performing surgeries such as removing the obstructions in the blood vessels or placing stents on the sides of the obstructions. In such surgeries, it is important for the operator to correctly grasp the position of the true lumen in the blood vessels from the viewpoints of improving the accuracy of the surgery, shortening the time required for the surgery, and reducing the burden on the patient. In this regard, in blood vessels where CTO has occurred, since the contrast agent does not flow into the true lumen as the target, there is a problem that the image of the true lumen is sometimes not shown in the angiographic image. In addition, even if the contrast agent flows into the true lumen as the target, the image of the true lumen is shown on the angiographic image only for the moment when the contrast agent passes through, but multiple injections of contrast agent in order to confirm the position of the true lumen will increase the burden on the patient, which is not a preferred issue.

[0009] In this regard, in the device described in Patent Document 1, the interventional device can be navigated to the target location (specific point of interest), but when the operator cannot grasp the position of the true lumen, there is a problem that the target location cannot be determined. In addition, when the techniques to be adopted according to various situations such as the state of the true lumen and the state of the blockage (including the method of advancing the device until the blood vessel is reopened) involve many aspects, the device described in Patent Document 1 has a problem that it is difficult to apply such a flexible technique. In addition, in the device described in Patent Document 2, there is no consideration for the problem that the true lumen is sometimes not visible in the angiographic image, and the problem that it is not desirable to inject contrast agent multiple times to confirm the position of the true lumen.

[0010] The present invention has been made to solve at least a part of the above-mentioned problems, and an object of the present invention is to display an image of the true lumen of a blood vessel in an image (angiographic image) of an FPD.

[0011] Means for solving problems

[0012] The present invention has been made to solve at least a part of the above-mentioned problems, and can be implemented as the following aspects.

[0013] (1) According to one embodiment of the present invention, a surgical assisting device is provided. The surgical assisting device comprises: a true lumen information acquiring unit that acquires three-dimensional position information of a true lumen existing in a target blood vessel; a true lumen image generating unit that acquires an angiographic image of the target blood vessel from an FPD (flat panel detector) disposed at a first imaging position, and generates a true lumen image of the true lumen that indicates a position and posture corresponding to the angiographic image using the position information of the first imaging position and the three-dimensional position information of the true lumen; and an image synthesizing unit that generates a synthesized image obtained by synthesizing the angiographic image and the true lumen image, and outputs the synthesized image.

[0014] According to this structure, the true lumen image generation unit can generate a true lumen image of the true lumen representing the position and posture corresponding to the angiographic image using the position information of the first shooting position of the angiographic image and the three-dimensional position information of the true lumen obtained by the true lumen information acquisition unit. That is, the true lumen image generation unit can generate a true lumen image representing the image of the true lumen based on the three-dimensional position information of the true lumen even when the contrast agent does not flow into the true lumen as the target or when the contrast agent does not flow. In addition, the image synthesis unit generates a synthetic image obtained by synthesizing the angiographic image at any first shooting position and the true lumen image representing the image of the true lumen, and outputs the synthetic image, so that the image of the true lumen of the blood vessel can be displayed in the image (angiographic image) of the FPD. Therefore, by confirming the synthetic image, the operator can confirm the positional relationship between the medical device on the angiographic image and the true lumen on the true lumen image while advancing the operation. As a result, the operator can correctly grasp the position of the true lumen in the target blood vessel, so that the accuracy of the operation can be improved, the time required for the operation can be shortened, and the burden on the patient can be reduced.

[0015] (2) In the surgical assisting device of the above aspect, the three-dimensional position information of the true lumen includes information related to the width of the true lumen, and the true lumen image generating unit generates a true lumen image representing the true lumen having a width corresponding to the three-dimensional position information of the true lumen.

[0016] According to this configuration, the true lumen image generation unit generates a true lumen image representing the true lumen having a width corresponding to the three-dimensional position information of the true lumen, so the operator can check the width of the true lumen while checking the synthesized image. As a result, it is possible to further improve the accuracy of the surgery, shorten the time required for the surgery, and reduce the burden on the patient.

[0017] (3) In the surgical assisting device of the above-mentioned method, when the FPD is moved to a second shooting position different from the first shooting position and the target blood vessel is shot from the FPD, the true lumen image generating unit obtains the angiography image shot at the second shooting position again, uses the position information of the second shooting position and the three-dimensional position information of the true lumen, and regenerates a true lumen image of the true lumen representing the position and posture corresponding to the angiography image obtained again, and the image synthesis unit regenerates a synthesized image by synthesizing the angiography image obtained again and the true lumen image generated again, and outputs the synthesized image.

[0018] According to this structure, when the FPD is moved to a second shooting position different from the first shooting position and shooting is performed based on the FPD, the true lumen image generation unit regenerates a true lumen image corresponding to the angiography image at the second shooting position, and the image synthesis unit regenerates a synthesized image synthesized by synthesizing the angiography image acquired again and the true lumen image generated again, and outputs the synthesized image. That is, the true lumen image generation unit and the image synthesis unit can follow the movement of the shooting position of the FPD and display a synthesized image including the moved true lumen image. As a result, the convenience of the surgical auxiliary device can be improved, and the accuracy of the surgery can be further improved, the time required for the surgery can be shortened, and the burden on the patient can be reduced.

[0019] (4) In the surgical assisting device of the above aspect, the surgical assisting device further comprises: an angiographic image acquisition unit that acquires a first angiographic image captured by the FPD arranged at a first position and a second angiographic image captured by the FPD arranged at a second position different from the first position; and an ultrasonic image acquisition unit that acquires an ultrasonic image of the inside of the target blood vessel captured by an ultrasonic sensor, the first angiographic image including: the ultrasonic sensor arranged at a first marked position in the target blood vessel; and a medical device different from the ultrasonic sensor and arranged at a second marked position in the target blood vessel, the second angiographic image including the ultrasonic sensor arranged at the first marked position in the target blood vessel, the ultrasonic image being an image captured in a state where the ultrasonic sensor is arranged at the first marked position, the ultrasonic image including the target blood vessel and the medical device arranged at the second marked position in the target blood vessel, and the true lumen information acquisition unit acquires three-dimensional position information of the true lumen using the position information of the first position, the first angiographic image, the position information of the second position, the second angiographic image, and the ultrasonic image.

[0020] According to this configuration, the true lumen information acquisition unit can acquire the three-dimensional position information of the true lumen using the position information of the first position acquired by the first angiographic image, the first angiographic image, the position information of the second position acquired by the second angiographic image, the second angiographic image, and the ultrasonic image. Specifically, the true lumen information acquisition unit can acquire the three-dimensional position information of the ultrasonic sensor using the position information of the first position and the first angiographic image, the position information of the second position, and the second angiographic image. Furthermore, the true lumen information acquisition unit can acquire the three-dimensional position information of the true lumen using the three-dimensional position information of the ultrasonic sensor, the position information of the first position and the first angiographic image, and the ultrasonic image of the true lumen showing the target blood vessel.

[0021] (5) In the surgical assisting device of the above-mentioned method, the true cavity information acquisition unit performs the following processing: using the image of the ultrasonic sensor contained in the first angiography image and the second angiography image, the position of the ultrasonic sensor is acquired; using the image of the medical device contained in the first angiography image and the ultrasonic image, the positional relationship between the first angiography image and the ultrasonic image is associated; acquiring the position information of the true cavity from the ultrasonic image; and using the acquired position of the ultrasonic sensor and the position information of the true cavity in the ultrasonic image obtained by the ultrasonic sensor, the three-dimensional position information of the true cavity is acquired.

[0022] According to this configuration, the true lumen information acquisition unit can acquire the three-dimensional position information of the ultrasonic sensor using the image of the ultrasonic sensor included in the first angiographic image and the second angiographic image. In addition, the true lumen information acquisition unit can associate the positional relationship between the first angiographic image and the ultrasonic image using the image of the medical device included in the first angiographic image and the ultrasonic image, acquire the position information of the true lumen from the ultrasonic image, and acquire the three-dimensional position information of the true lumen using the acquired position of the ultrasonic sensor and the position information of the true lumen in the ultrasonic image obtained by the ultrasonic sensor.

[0023] (6) In the surgical assisting device of the above-mentioned method, the surgical assisting device includes a vascular angiography image acquisition unit, which acquires a first vascular angiography image captured by the FPD arranged at a first position and a second vascular angiography image captured by the FPD arranged at a second position different from the first position, the first vascular angiography image including the true lumen of the target blood vessel and a medical device arranged at a first marked position in the target blood vessel, and the second vascular angiography image including the true lumen of the target blood vessel and the medical device arranged at the first marked position in the target blood vessel, and the true lumen information acquisition unit uses the images of the medical device and the true lumen contained in the first vascular angiography image and the second vascular angiography image to acquire three-dimensional position information of the true lumen.

[0024] According to this configuration, the true lumen information acquisition unit can acquire the three-dimensional position information of the true lumen using the image of the medical device and the image of the true lumen included in the first angiographic image and the second angiographic image.

[0025] The present invention is made to solve at least a part of the above-mentioned problems, and can be implemented in the following ways. For example, it can be implemented in the form of an information processing device that outputs a composite image, an information processing device that outputs a recommended range of FPD shooting positions together with a composite image, an FPD (flat panel detector) that outputs a composite image, an FPD that outputs a recommended range of FPD shooting positions together with a composite image, a system including these devices, a computer program that realizes the functions of these devices and the system, a server device for distributing the computer program, a non-transitory storage medium storing the computer program, etc. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 It is an explanatory diagram illustrating the configuration of a surgery support system.

[0027] Figure 2 It is a diagram for explaining the imaging position of the first FPD.

[0028] Figure 3 This is a diagram illustrating the target blood vessels and the equipment used.

[0029] Figure 4 This is a flowchart showing an example of synthesized image output processing.

[0030] Figure 5 This is a flowchart showing an example of synthesized image output processing.

[0031] Figure 6 A diagram showing a screen used in the composite image output process.

[0032] Figure 7 It is a diagram for explaining steps S5 and S7 of the synthesized image output process.

[0033] Figure 8 This is a diagram for explaining step S8 of the synthesized image output process.

[0034] Fig. 9 It is a diagram for explaining steps S12 to S17 of the synthesized image output process.

[0035] Fig.10 It is a diagram for explaining step S18 of the synthesized image output process.

[0036] Fig.11 is a diagram illustrating the calculation of vector length.

[0037] Fig.12 It is a diagram for explaining steps S19 to S21 of the synthesized image output process.

[0038] Fig.13 This is a diagram showing an example of an ultrasonic image displayed on the canvas after step S21 of the composite image output process.

[0039] Fig.14 This is a diagram for explaining step S22 of the synthesized image output process.

[0040] Fig.15 This is a diagram for explaining step S23 of the synthesized image output process.

[0041] Fig.16 It is a diagram for explaining steps S24 to S28 of the synthesized image output process.

[0042] Fig.17 This is a diagram for explaining step S30 of the synthesized image output process.

[0043] Fig.18 This is a diagram for explaining step S30 of the synthesized image output process.

[0044] Fig.19 It is a diagram showing an example of a synthesized image.

[0045] Fig. 20 It is a diagram for explaining a modified example of the true cavity image.

[0046] Fig.21 It is a diagram for explaining a modified example of the true cavity image.

[0047] Fig. 22 It is an explanatory diagram illustrating the configuration of a surgery support system according to the second embodiment.

[0048] Fig.23 This is a flowchart showing an example of the synthesized image output process according to the second embodiment.

[0049] Fig.24 It is a diagram showing the synthesized image output process according to the second embodiment.

[0050] Fig.25 It is a diagram for explaining step S18A of the synthesized image output process according to the second embodiment.

[0051] Fig.26 It is an explanatory diagram illustrating the configuration of a surgery support system according to a third embodiment. DETAILED DESCRIPTION

[0052] <First Embodiment>

[0053] Figure 11 is an explanatory diagram illustrating the structure of a surgery support system 1. The surgery support system 1 is a system for assisting examination and treatment. The surgery support system 1 includes a surgery support device 10, a blood vessel imaging device 20 having an FPD (Flat Panel Detector), a display device 30, a workbench 40, and an operation unit 50. The surgery support system 1 of this embodiment is provided with the surgery support device 10 described later, and can generate a true lumen image of a true lumen showing a position and a posture corresponding to the angiography image for an image captured by the FPD of an object blood vessel (hereinafter also referred to as an "angiography image"), and display a composite image synthesized by the angiography image and the true lumen image. In addition, the "object blood vessel" refers to a blood vessel that is the object of examination or treatment, but the surgery support system 1 is not limited to the blood vessel system, and can also be used in a biological lumen such as a lymphatic system, a biliary system, a urinary system, a respiratory system, a digestive organ system, a secretory gland, and a reproductive organ.

[0054] exist Figure 1 In the figure, mutually orthogonal XYZ axes are shown. The X axis corresponds to the width direction of the blood vessel imaging device 20, the Y axis corresponds to the height direction of the blood vessel imaging device 20, and the Z axis corresponds to the depth direction of the blood vessel imaging device 20. In the following description, the patient ( Figure 1 The direction in which the head 92 of the human body 90 is located is referred to as the "Z-axis direction", or "Z" for short. The three-dimensional space formed by the three-dimensional coordinates (XYZ coordinates) composed of the X, Y, and Z axes is referred to as the XYZ three-dimensional space. In addition, the origin O of the XYZ three-dimensional space (XYZ coordinates) is set to the position of the heart 91 of the human body 90.

[0055] In the synthetic image output process described later, the surgical assisting device 10 generates a true lumen image of the true lumen representing the position and posture corresponding to the angiographic image captured by the FPD, and outputs a synthetic image synthesized by the angiographic image and the true lumen image. The surgical assisting device 10 is configured to include a CPU, a ROM, and a RAM. The CPU executes a computer program stored in the ROM to realize the functions of the main control unit 11, the angiographic image acquisition unit 12, the ultrasonic image acquisition unit 13, the true lumen information acquisition unit 14, the true lumen image generation unit 15, and the image synthesis unit 16. The surgical assisting device 10 is electrically connected to the control unit 29, the display device 30, and the operation unit 50 of the angiographic device 20, respectively.

[0056] The main control unit 11 transmits and receives information with the control unit 29, the display device 30, and the operation unit 50 of the angiography device 20, and controls the entire surgery support device 10. The main control unit 11 also controls the entire synthetic image output process described later.

[0057] In the composite image output process, the angiographic image acquisition unit 12 acquires a first angiographic image and a second angiographic image from the vascular imaging device 20. The "first angiographic image" refers to an angiographic image captured by placing the FPD at an arbitrary imaging position. The imaging position of the FPD when the first image is acquired is also referred to as the "first position". The "second angiographic image" refers to an angiographic image captured by placing the FPD at an arbitrary imaging position different from the first position. The imaging position of the FPD when the second image is acquired is also referred to as the "second position". The details of the first and second angiographic images and the first and second positions will be described later. The process (step) performed by the angiographic image acquisition unit 12 is also referred to as an angiographic image acquisition process (step).

[0058] In the synthetic image output process, the ultrasonic image acquisition unit 13 obtains the image from the imaging sensor 300 ( Figure 3 ) obtains an ultrasonic image of the inside of the target blood vessel captured by the imaging sensor 300. The details will be described later. The process (step) performed by the ultrasonic image acquisition unit 13 is also referred to as an ultrasonic image acquisition process (step).

[0059] In the synthetic image output process, the true lumen information acquisition unit 14 uses the position information of the first position, the first angiographic image, the position information of the second position, the second angiographic image, and the ultrasonic image to acquire the three-dimensional position information (position information in the XYZ three-dimensional space) of the true lumen existing in the target blood vessel. The details will be described later. The process (step) performed by the true lumen information acquisition unit 14 is also referred to as a true lumen information acquisition process (step).

[0060] In the synthetic image output processing, the true lumen image generation unit 15 generates a true lumen image of the true lumen representing the position and posture corresponding to the angiography image captured by the FPD configured at an arbitrary shooting position (hereinafter also referred to as the "first shooting position"). In addition, when the FPD is moved to an arbitrary shooting position different from the first shooting position (hereinafter also referred to as the "second shooting position") and the FPD is shot based on the FPD, the true lumen image generation unit 15 regenerates a true lumen image of the true lumen representing the position and posture corresponding to the angiography image captured at the second shooting position. Here, the first shooting position and the second shooting position refer to arbitrary positions different from the above-mentioned first position and second position, that is, arbitrary positions at which the operator intends to confirm the target blood vessels and equipment. However, the first shooting position and the second shooting position may also be the same as the above-mentioned first position and second position. The details will be described later. The process (step) performed by the true lumen image generation unit 15 is also called a true lumen image generation process (step).

[0061] In the synthetic image output process, the image synthesis unit 16 generates a synthetic image synthesized by synthesizing the angiography image captured by the FPD arranged at the first imaging position and the true lumen image generated by the true lumen image generation unit 15, and displays it on the display device 30. In addition, the image synthesis unit 16 regenerates a synthetic image synthesized by synthesizing the angiography image captured by the FPD arranged at the second imaging position and the true lumen image regenerated by the true lumen image generation unit 15, and displays it on the display device 30. The details will be described later. The process (step) performed by the image synthesis unit 16 is also called an image synthesis process (step).

[0062] The angiography device 20 includes an FPD, acquires X-rays that have passed through the human body, and converts them into digital signals to acquire images (angiography images). The angiography device 20 includes a first FPD 21, a first X-ray tube device 22, a first C-arm 23, a first support unit 24, a second FPD 25, a second X-ray tube device 26, a second C-arm 27, a second support unit 28, and a control unit 29.

[0063] The first FPD 21 includes an X-ray plane detector, which converts X-rays incident from the first X-ray tube device 22 into electrical signals, performs A / D (analog / digital) conversion, and generates an X-ray image. The first X-ray tube device 22 receives a high voltage output from an X-ray high voltage device (not shown) and irradiates an X-ray beam. Figure 1 As shown by the thick dotted line extending in the Y-axis direction, the X-ray beam irradiated from the first X-ray tube device 22 is incident on the first FPD21 through the human body 90. The first C-arm 23 is a C-shaped arm (support) that fixes the first FPD21 and the first X-ray tube device 22 in relative positions. The first support part 24 supports the first C-arm 23 so as to be rotatable. That is, the first FPD21 and the first X-ray tube device 22 can be moved to any shooting position around the human body 90 lying on the bed 41 while being fixed in relative positions by the first C-arm 23. Hereinafter, the first FPD21 and the first X-ray tube device 22 fixed to the first C-arm 23 will also be referred to as the "first FPD21".

[0064] The structure of the second FPD 25 is the same as that of the first FPD 21. The structure of the second X-ray tube device 26 is the same as that of the first X-ray tube device 22. Figure 1As shown by the thick dotted line extending in the X-axis direction, the X-ray beam irradiated from the second X-ray tube device 26 enters the second FPD25 through the human body 90. The second C-arm 27 is a C-shaped arm (support) that fixes the second FPD25 and the second X-ray tube device 26 at relative positions. The second support 28 supports the second C-arm 27 so that it can rotate. That is, the second FPD25 and the second X-ray tube device 26 can move to any shooting position around the human body 90 while being fixed at relative positions by the second C-arm 27. Hereinafter, the second FPD25 and the second X-ray tube device 26 fixed to the second C-arm 27 will also be referred to as the "second FPD25".

[0065] The second FPD 25 is generally arranged along the normal direction of the first FPD 21. Figure 1 As shown, when the first FPD 21 is set to the shooting position in the front direction (vertical direction of the human body 90, longitudinal direction of the human body 90) of the human body 90, the second FPD 25 is set to the shooting position in the horizontal direction (lateral direction of the human body 90) of the human body 90. In addition, the blood vessel imaging device 20 is sometimes simply referred to as "FPD", "FPD device", etc.

[0066] The control unit 29 is configured to include a CPU, a ROM, and a RAM, and the CPU executes a computer program stored in the ROM to control the entirety of the angiography device 20. The control unit 29 is electrically connected to the first FPD 21, the second FPD 25, the first support unit 24, the second support unit 28, the display device 30, the table 40, and the operation unit 50. The control unit 29 causes the display device 30 to display the X-ray image generated by the first FPD 21 and the second FPD 25. In addition, the control unit 29 drives the first support unit 24 to rotate the first C-arm 23 and drives the second support unit 28 to rotate the second C-arm 27 in accordance with the operation from the operation unit 50. Furthermore, the control unit 29 changes the height of the bed 41 by extending and retracting the telescopic unit 42 and changes the position of the bed 41 by moving the table 40 in the Z-axis direction in accordance with the operation from the operation unit 50.

[0067] The display device 30 is connected to the control unit 29 of the surgery support device 10 and the vascular imaging device 20, and functions as an output interface for the surgery support device 10 and the vascular imaging device 20. The display device 30 includes a monitor 31 and an arm 32. The monitor 31 is a "display unit" composed of a well-known unit such as a liquid crystal display, smart glasses, and a projector. The arm 32 supports and fixes the monitor 31.

[0068] The workbench 40 is a table for a human body 90 to lie down and is located near the first FPD 21 and the second FPD 25. The workbench 40 has a bed 41, a telescopic part 42, and a leg part 43. The bed 41 has a mattress for the human body 90 to lie down. The bed 41 is supported on the workbench 40 in a state that it can move in the Z-axis direction. The telescopic part 42 is configured to be able to change the height of the bed 41 by telescoping in the Y-axis direction. The leg part 43 supports the bed 41 and the telescopic part 42. Figure 1 As shown by the middle dotted line, the human body 90 lies on the bed 41 with the head 92 placed on the side close to the first FPD 21 and the second FPD 25 and the feet 93 placed on the side away from the first FPD 21 and the second FPD 25. In this way, the image of the target blood vessel in the heart 91 can be easily obtained through the first FPD 21 and the second FPD 25.

[0069] The operation unit 50 is connected to the control unit 29 of the surgery support device 10 and the blood vessel imaging device 20, and functions as an input interface for the surgery support device 10 and the blood vessel imaging device 20. The operation unit 50 is an "input unit" composed of well-known units such as a touch panel, an operation button, an operation lever, an operation switch, a keyboard, a mouse, a sound input unit, and a foot switch. In the example shown in the figure, the operation unit 50 is fixed to the workbench 40.

[0070] Figure 2 It is a diagram for explaining the imaging position of the first FPD 21 . Figure 2 (A) is a diagram illustrating LAO. Figure 2 (B) is a diagram illustrating RAO. Figure 2 As shown in (A), the situation where the first FPD 21 is located on the left side of the human body 90 is called LAO (Left Anterior Oblique view). Figure 2 As shown in (B), the situation where the first FPD 21 is located on the right side of the human body 90 is called RAO (Right Anterior Oblique view). Figure 2 (C) is a diagram illustrating CRA. Figure 2 (D) is a diagram illustrating CAU. Figure 2 As shown in (C), the situation where the first FPD 21 is located in the upper direction of the human body 90 is called CRA (CRAnial). Figure 2 As shown in (D), the case where the first FPD 21 is positioned in the lower direction of the human body 90 is called CAU (CAUdal). That is, the "imaging position of the first FPD 21" is determined by a combination of the left-right position (A1) and the up-down position (A2) shown below.

[0071] (A1) The difference between LAO and RAO, the angle θ1 with the center O of the human body at 90 degrees,

[0072] (A2) The difference between CRA and CAU, and the angle θ2 with the center O of the human body 90.

[0073] Here, the center O of the human body 90 is the position of the heart 91 of the human body 90 (the position of the origin O in the XYZ three-dimensional space). For example, "the imaging position of the first FPD 21 is (RAO28 CRA5)" means that the first FPD 21 is located at a position 28 degrees to the right of the human body 90 and 5 degrees to the upper side of the human body 90.

[0074] Figure 3 1 is a diagram illustrating a target blood vessel 100 and equipment to be used. Figure 3 (A) is a diagram showing a longitudinal section of a target blood vessel 100. Figure 3 (B) is viewed from above Figure 3 FIG. 1 is a diagram of a portion of the target blood vessel 100 surrounded by a dotted rectangle in FIG. Figure 3 (A) shows a target blood vessel 100, a CTO 101 generated in the target blood vessel 100, a false lumen 102 formed in the intima or subintima of the target blood vessel 100, and a true lumen 103. The false lumen 102 refers to all dissociated lumens other than the true lumen 103 formed by the medical device.

[0075] Here, if Figure 3 As shown in (B), the true cavity 103 is not necessarily limited to extending in a straight line, and sometimes it is winding. Figure 3 In the example shown in (B), in order to guide the guide wire 500 to the true cavity 103, the operator needs to make the front end of the guide wire 500 face the lower side of the paper. However, in FPD, only a perspective image in a certain cross section can be obtained, so it is not possible to obtain such a cross section according to the shooting position of the FPD. Figure 3 A portion of the true lumen 103 winding as shown in (A) is included in the angiographic image. In this regard, the surgery support device 10 of this embodiment generates and displays a true lumen image showing an image of the true lumen by a synthetic image output process described later, thereby solving such a problem.

[0076] In the surgical assistance system 1, use Figure 3The imaging sensor 300 and the guide wire 500 shown in (A) are shown in FIG. The imaging sensor 300 is an ultrasonic sensor for obtaining an ultrasonic image of the interior of the target blood vessel 100. The imaging sensor 300 has a long strip-shaped appearance and has a transducer 301 at the front end. The transducer 301 is an ultrasonic probe (also called an ultrasonic vibrator, a piezoelectric body, an ultrasonic transceiver element, or an ultrasonic element) that transmits ultrasonic waves to biological tissues and receives ultrasonic waves propagated and reflected in biological tissues. The imaging sensor 300 moves back and forth in the lumen of the sensor catheter 200 and obtains an ultrasonic image of the interior of the target blood vessel 100 located around the transducer 301. Specifically, the transducer 301 obtains an ultrasonic image of the interior of the target blood vessel 100 in a direction perpendicular to the transducer axis (in a 360° circumferential direction of the transducer axis) while rotating around the central axis of the transducer 301 extending in the long side direction of the image sensor 300 (hereinafter also referred to as the transducer axis).

[0077] The guide wire 500 is a medical device having a long strip shape. The guide wire 500 may also be a plasma guide wire that has an electrode at the front end and uses a plasma flow to ablate biological tissue. In this case, the wire catheter 400 may also be a structure having another electrode at the front end. In addition, the guide wire 500 may be a penetration guide wire that has a pointed portion at the front end and uses the pointed portion to penetrate biological tissue, or may be a transmission guide wire that does not have a pointed portion. The guide wire 500 is accommodated in the lumen of the wire catheter 400, and the front end of the guide wire 500 protrudes to the outside from the front end 401 of the wire catheter 400.

[0078] Figure 4 and Figure 5 This is a flowchart showing an example of synthetic image output processing. The synthetic image output processing can be started at any opportunity, such as when the surgical assisting device 10 is powered on, when a predetermined application is started, when the vascular imaging device 20 is powered on, etc. In the following description, the case where an angiographic image is obtained using the first FPD21 is exemplified. However, an angiographic image can also be obtained using the second FPD25. In this case, in the following description, the portion recorded as "first FPD21" can be replaced with "second FPD25" for processing. In addition, in Figure 4 as well as Figure 5 In the flowchart of FIG. 1 , for the convenience of illustration, the imaging sensor 300 is simply referred to as “sensor” and the angiography image is simply referred to as “image”.

[0079] Figure 6 A diagram for explaining a screen used in the composite image output process. Figure 6 (A) is a diagram showing the structure of the operation screen OS. Figure 6(B) is a diagram showing an example of the first angiography image V1. In the synthetic image output process described later, the true lumen information acquisition unit 14 guides the operator in the operation while advancing the process. In the example of this embodiment, the true lumen information acquisition unit 14 causes the display device 30 to display Figure 6 The operation screen OS as shown in (A) of FIG. 1 is used to display guidance for various operations. The operation screen OS has an operation button display area A1 where buttons for performing various operations are configured, a canvas A2, and a guidance display area A3 for displaying various guidance messages. The canvas A2 is an area for displaying the angiography image sequentially acquired by the first FPD 21 and the ultrasonic image acquired by the imaging sensor 300. In addition, the true cavity information acquisition unit 14 enters each step while updating the angiography image displayed on the canvas A2 of the operation screen OS during the synthetic image output process.

[0080] The operation screen OS is just an example, and various changes can be made. For example, the guidance display area A3 of the operation screen OS can be omitted, and guidance based on sound can be performed. For example, the guidance display area A3 of the operation screen OS can be omitted, and a button with an item name attached (for example, a button recorded as "first mark configuration" in the case of step S5) can be arranged in the operation button display area A1 instead of guidance.

[0081] In step S1, the true cavity information acquisition unit 14 provides guidance to prepare for the first FPD 21 to take pictures. The operator follows the guidance to prepare for the first FPD 21 to take pictures. Specifically, Figure 1 As shown, a human body 90 is made to lie on a bed 41, and the power of the blood vessel imaging device 20 is turned on.

[0082] In step S2, the true cavity information acquisition unit 14 guides the imaging sensor 300 to prepare for photographing. The operator prepares for photographing by the imaging sensor 300 according to the guidance. Figure 3 As shown in FIG. 1A , the imaging sensor 300 and the guide wire 500 are inserted into a blood vessel of a human body 90 , and the transducer 301 of the imaging sensor 300 and the distal end of the guide wire 500 are delivered so as to be located near the CTO 101 of the target blood vessel 100 .

[0083] In step S3, the true lumen information acquisition unit 14 guides the operator to move the first FPD 21 to the first position to capture an X-ray image. The operator moves the first FPD 21 to the first position according to the guidance, captures an X-ray image of the target blood vessel 100, and thereby acquires a first angiography image V1. The first position may be any position (RAO XX CRA XX: X is an arbitrary natural number). In step S3, the true lumen information acquisition unit 14 may also automatically move the first FPD 21 to the first position to capture. The angiography image acquisition unit 12 acquires the captured first angiography image V1 from the blood vessel capturing device 20. Figure 6 As shown in (B), the first angiography image V1 includes an image of the imaging sensor 300, an image of the guide wire 500, and an image of the target blood vessel 100 through which the imaging sensor 300 and the guide wire 500 are inserted (not shown). Figure 6 In (B), for the sake of distinction, the imaging sensor 300 is shown in dashed lines, and the guide wire 500 is shown in solid lines. In the example of the present embodiment, the first angiography image V1 includes the image of the target blood vessel 100. However, the first angiography image V1 only needs to include the image of the imaging sensor 300 and the image of the guide wire 500, and may not include the image of the target blood vessel 100. In addition, the “image of the target blood vessel 100” refers to the image of the contour of the target blood vessel 100.

[0084] In step S4, the true lumen information acquisition unit 14 displays the first angiographic image V1 on the canvas A2 and adjusts the position of the first angiographic image V1 so that the transducer 301 (see FIG. 1 ) in the image of the imaging sensor 300 is Figure 3 ) is located at the center of canvas A2.

[0085] Figure 7 This is a diagram for explaining steps S5 and S7 of the synthesized image output process. Figure 7 (A) is a diagram showing the state of the canvas A2 in step S5. Figure 7 (B) is a diagram showing the state of the canvas A2 in step S7. Figure 7 As shown, the square or rectangular canvas A2 has a vertex located at the upper left of the paper as the origin Oc, and has a two-dimensional coordinate system consisting of an Xc axis extending to the right of the paper and a Yc axis extending to the bottom of the paper, namely, an XcYc coordinate system. The two-dimensional space formed by the XcYc coordinate system is called an XcYc two-dimensional space. In addition, in the XcYc two-dimensional space, the negative direction of the Yc axis (the direction on the paper) is the head 92 of the human body 90 (refer to Figure 1 ) is located. In step S5, the true cavity information acquisition unit 14 guides by configuring a first marker a1 on the image of the transducer 301 of the imaging sensor 300 in the first angiography image V1 displayed on the canvas A2. Figure 7 As shown in (A), the operator configures the first mark a1 on the first angiography image V1 on the canvas A2 according to the guidance. The configuration of the first mark a1 can be achieved by, for example, clicking or tapping a desired position on the image of the canvas A2. In addition, on the first angiography image V1, when the image of the transducer 301 is located at the first mark a1 of the XcYc coordinates in the canvas A2, the actual position (position in the XYZ coordinates) of the transducer 301 in the target blood vessel 100 is set to P1.

[0086] In step S6, the true cavity information acquisition unit 14 guides the operator to acquire the ultrasonic image IV1 using the imaging sensor 300 while maintaining the position of the imaging sensor 300. According to the guidance, the operator does not move the imaging sensor 300 from the Figure 7 The ultrasonic image acquiring unit 13 acquires the ultrasonic image IV1 captured by the imaging sensor 300 and stores the ultrasonic image IV1 in a storage unit inside the surgery assisting device 10. That is, the ultrasonic image acquiring unit 13 acquires the ultrasonic image IV1 when the transducer 301 is located at P1 in the target blood vessel 100.

[0087] That is, the first angiographic image V1 acquired in step S3 includes the target blood vessel 100, the image of the imaging sensor 300 arranged at the first marking position (first marker a1) in the target blood vessel 100, and the guidewire 500 arranged at the second marking position (an arbitrary position different from the first marker a1) in the target blood vessel 100. In addition, the ultrasonic image IV1 acquired in step S6 includes the target blood vessel 100, and the guidewire 500 arranged at the second marking position (an arbitrary position different from the first marker a1) in the target blood vessel 100.

[0088] In step S7, the true cavity information acquisition unit 14 guides the imaging sensor 300 by placing the second marker a2 on the transducer 301 after advancing the imaging sensor 300 within a range that is considered a straight line. Here, "straight line" means that the trajectory of the transducer 301 when moving in the sensor catheter 200 is a straight line. Figure 7 As shown in (B), the operator advances the imaging sensor 300 to the range (distance) regarded as a straight line according to the guidance, and then configures the second mark a2 on the first angiographic image V1 on the canvas A2. In addition, when the image of the transducer 301 is located at the second mark a2 of the XcYc coordinates in the canvas A2 on the first angiographic image V1, the actual position (position in the XYZ coordinates) of the transducer 301 in the target blood vessel 100 is set to Pe.

[0089] Figure 8This is a diagram for explaining step S8 of the synthesized image output process. Figure 8 (A) and (B) are diagrams for explaining calculation of BNV described later. Figure 8 (C) is a diagram showing the relationship between a first angiographic image V1 at a first position and a second angiographic image V2 at a second position described later. Figure 8 (D) is a diagram showing a first angiography image V1 at a first position. Figure 8 (E) is a diagram showing a second angiography image V2 at a second position.

[0090] In step S8, the true lumen information acquisition unit 14 calculates the BNV using the first marker a1, the second marker a2, and the first position in the first angiographic image V1 on the canvas A2. Figure 8 As shown in (A), the true cavity information acquisition unit 14 uses the vector (the vector representing the trajectory of the transducer 301 regarded as a straight line) with the first mark a1 as the starting point and the second mark a2 as the end point as the first axis vector Ie′ of the imaging sensor 300, and uses the vector representing the first FPD 21 disposed at the first position relative to the heart 91 (refer to Figure 1 ) is taken as the first view vector Vw1, and the BNV with respect to the plane W including the first principal axis vector Ie′ and the first view vector Vw1 is calculated. In addition, the plane W is a plane including the first principal axis vector Ie′, so it can be said to be a plane in the XYZ three-dimensional space on which the track from P1 (equivalent to the first mark a1) to Pe (equivalent to the second mark a2) of the transducer 301 is placed. Here, as Figure 8 As shown in (A), "BNV" ​​refers to the shooting direction perpendicular to the plane W. That is, BNV refers to the shooting direction that can obtain the second angiographic image V2 perpendicular to the first angiographic image V1. The position of the first FPD21 with the shooting direction BNV is set as the "second position", and the shooting direction (=BNV) of the first FPD21 arranged at the second position relative to the heart 91 is set as the second view. The vector representing the second view is set as the second view vector Vw2.

[0091] The true cavity information acquisition unit 14 obtains the numerical value RL indicating LAO or RAO as the position information of the first FPD 21 at the first position (also referred to as “position information of the first position”). val , CC represents the numerical value of CRA or CAU val , and the slope δ of the first principal axis vector Ie′ with respect to the Yc′ axis parallel to the Yc axis in canvas A2 (refer to Figure 8(B)) is substituted into the following formula (1) to calculate ψ, θ, and δ. In addition, in formula (1), "ψ" and "θ" are variables used to display the first view vector Vw1 in polar coordinates. In addition, "CW" means "clockwise" and "CCW" means "counterclockwise". Next, the true cavity information acquisition unit 14 substitutes the calculated ψ, θ, and δ into formula (2) to calculate the orthogonal coordinates of the second view vector Vw2. Next, the true cavity information acquisition unit 14 uses formula (3) to transform the orthogonal coordinates (x, y, z) of the second view vector Vw2 into polar coordinates (r, θ, ψ). Next, the true cavity information acquisition unit 14 calculates the numerical value RL representing LAO or RAO at the second position based on the polar coordinates (r, θ, ψ) of the second view vector Vw2. val and CC, which represents the numerical value of CRA or CAU val .

[0092] [Formula 1]

[0093]

[0094] [Formula 2]

[0095]

[0096] [Formula 3]

[0097]

[0098] Consider that the imaging sensor 300 is in the subject's blood vessel 100 Figure 8 In this case, in step S7, the imaging sensor 300 is advanced within the range regarded as a straight line, so it can be said that the first angiographic image V1 is an image obtained from the direction regarded as a straight line by the imaging sensor 300, that is, from the first view at the first position. Therefore, in the first angiographic image V1, as shown in FIG. Figure 8 As shown in the images with diagonal hatching in (C) and (D) of FIG. 1 , the imaging sensor 300 is captured as a straight line. On the other hand, the second view (BNV) at the second position is perpendicular to the first view, so the second angiographic image V2 obtained in the second view at the second position is perpendicular to the first angiographic image V1, and can be said to be an image obtained from the direction in which the imaging sensor 300 (the track of the transducer 301) looks curved. Therefore, in the second angiographic image V2, as shown in FIG. Figure 8 As shown in the images with dotted shadows in (C) and (E), the imaging sensor 300 is captured in a curved shape.

[0099] In step S9, the true cavity information acquisition unit 14 guides the transducer 301 of the imaging sensor 300 to be pulled back to the position of the first mark a1. The operator pulls the transducer 301 back to the position of the first mark a1 according to the guidance. That is, the operator pulls the transducer 301 in the target blood vessel 100 from the position Pe in the XYZ coordinates to P1.

[0100] In step S10, the true lumen information acquisition unit 14 guides the first FPD 21 to move to the second position that becomes the BNV calculated in step S8 to capture an X-ray image. The operator moves the first FPD 21 to the second position according to the guidance, captures an X-ray image of the target blood vessel 100, and thereby acquires a second angiographic image V2. In step S10, the true lumen information acquisition unit 14 may also automatically move the first FPD 21 to the second position to capture. The angiographic image acquisition unit 12 acquires the captured second angiographic image V2 from the blood vessel capturing device 20. In the second angiographic image V2, as shown in Fig. 9 As described later in the first angiography image V1, the second angiography image V2 includes an image of the imaging sensor 300, an image of the guidewire 500, and an image of the target blood vessel 100 through which the imaging sensor 300 and the guidewire 500 are inserted (not shown). In the example of the present embodiment, the second angiography image V2 includes the image of the target blood vessel 100. However, the second angiography image V2 only needs to include the image of the imaging sensor 300, and may not include the image of the target blood vessel 100 or the image of the guidewire 500.

[0101] In step S11 , the true lumen information acquisition unit 14 displays the second angiographic image V2 on the canvas A2 , and adjusts the position of the second angiographic image V2 so that the image of the transducer 301 in the imaging sensor 300 is located at the center of the canvas A2 .

[0102] Fig. 9 This is a diagram for explaining steps S12 to S17 of the synthesized image output process. Fig. 9 An example of a second angiographic image V2 on canvas A2 is shown. In step S12, the true lumen information acquisition unit 14 guides in a manner of configuring a first marker b1 on the image of the transducer 301 of the imaging sensor 300 in the second angiographic image V2 displayed on canvas A2. According to the guidance, the operator configures the first marker b1 on the second angiographic image V2 on canvas A2. The configuration of the first marker b1 can be achieved, for example, by clicking or tapping the position of the intended operation on the image of canvas A2. In addition, in step S9, the transducer 301 is pulled back to the position of the first marker a1, so the actual position of the first marker b1 in the XYZ three-dimensional space is the same as the position of the first marker a1 marked in step S5 ( Fig. 9: b1=a1). That is, on the second angiographic image V2, when the image of the transducer 301 is located at the first mark b1 of the XcYc coordinates in the canvas A2, the actual position (position in the XYZ coordinates) of the transducer 301 in the target blood vessel 100 is P1.

[0103] In step S13, the true lumen information acquisition unit 14 substitutes 2 for the variable n used in the synthetic image output processing. n is a natural number. In step S14, the true lumen information acquisition unit 14 guides the imaging sensor 300 by advancing it to an arbitrary distance and configuring the second mark b2 on the transducer 301. After the operator advances the imaging sensor 300 according to the guidance, the second mark b2 is configured on the second angiography image V2 on the canvas A2. In addition, on the second angiography image V2, when the image of the transducer 301 is located at the second mark b2 of the XcYc coordinates in the canvas A2, the actual position (position in the XYZ coordinates) of the transducer 301 in the target blood vessel 100 is set to P2.

[0104] In step S15, the true lumen information acquisition unit 14 guides the imaging sensor 300 to acquire the ultrasonic image IV2 while maintaining the position of the imaging sensor 300. The operator acquires the ultrasonic image IV2 from the imaging sensor 300 without moving the imaging sensor 300 according to the guidance. The ultrasonic image acquisition unit 13 acquires the captured ultrasonic image IV2 from the imaging sensor 300, and stores the ultrasonic image IV2 in the storage unit inside the surgical assistance device 10. That is, the ultrasonic image acquisition unit 13 acquires the ultrasonic image IV2 when the transducer 301 is located at P2 in the target blood vessel 100. In step S16, the true lumen information acquisition unit 14 adds 1 to the variable n.

[0105] In step S17, the true lumen information acquisition unit 14 determines whether the placement of the mark on the second angiography image V2 (step S14) and the acquisition of the ultrasonic image at the position of the mark (step S15) have completed the target number of marks. The target number of marks can be arbitrarily determined. Fig. 9 In the case of n=4. When the variable n reaches the target number of marks (step S17: Yes), the true cavity information acquisition unit 14 transfers the process to step S18. When the variable n does not reach the target number of marks (step S17: No), the true cavity information acquisition unit 14 transfers the process to step S14 and repeats the above process. Fig. 9As shown in FIG. 1 , the variable n is incremented in the manner of n=3 and n=4, and the n-th marker bn on the second angiographic image V2 is arranged (step S14), and the ultrasonic image IVn corresponding to the position of each n-th marker bn is acquired (step S15). In addition, on the second angiographic image V2, when the image of the transducer 301 is located at the n-th marker bn of the XcYc coordinates in the canvas A2, the actual position (position in the XYZ coordinates) of the transducer 301 in the target blood vessel 100 is set to Pn. In addition, the range in which the imaging sensor 300 is advanced on the second angiographic image V2 is the range in which the imaging sensor 300 is regarded as a straight line on the first angiographic image V1, that is, from the first marker a1 to the second marker a2 (from the position P1 to Pe in the target blood vessel 100).

[0106] Fig.10 This is a diagram for explaining step S18 of the synthesized image output process. Fig.10 (A) is a diagram showing each point on the first angiography image V1 displayed on the canvas A2. Fig.10 (B) is a diagram showing each point on the second angiography image V2 displayed on the canvas A2, that is, Fig.10 (A) Fig.10 The upper side of the paper of (B) is the Z axis of the XYZ coordinates ( Figure 1 : the direction in which the head 92 of the human body 90 is located). That is, the negative direction of the Yc axis of the canvas A2 is Figure 1 The direction of the head 92 of the human body 90 in FIG. Fig.10 In (A) and (B), the Yc′ axis is a straight line parallel to the Yc axis.

[0107] In step S18, the true lumen information acquisition unit 14 calculates the following (B1) and (B2) using the XcYc coordinates of the first marker a1 and the second marker a2 in the first angiography image V1 displayed on the canvas A2, and the XcYc coordinates of the first marker b1 to the nth marker bn in the second angiography image V2.

[0108] (B1) Position vectors P2 to Pn of the transducer 301 in the XYZ three-dimensional space: The position vectors P2 to Pn of the transducer 301 are vectors extending from the position P1 serving as a reference point (starting point) to the positions P2 to Pn, respectively, in the target blood vessel 100 .

[0109] (B2) Transducer axis vectors T1 to Tn of the transducer 301 of the imaging sensor 300: The transducer axis vectors T1 to Tn are vectors of the transducer axis (the central axis of the transducer 301 extending along the long side direction of the imaging sensor 300) when the transducer 301 is located at positions P1 to Pn in the target blood vessel 100. Here, the transducer axis vectors when the transducer 301 is located at positions P1 to Pn can be said to be tangent vectors at positions P1 to Pn on the trajectory of the transducer 301.

[0110] Here, if Figure 8 As described in FIG. 1 , the first angiography image V1 and the second angiography image V2 displayed on the canvas A2 are images captured by the imaging sensor 300 from different angles. Fig.10 The first marker b1 to the nth marker bn (n=4 in the example shown in the figure) in the second angiographic image V2 shown in FIG. Fig.10 Therefore, the orientation of the "(B1) position vector P2-Pn of the transducer 301" can be determined using the slope δ of the first principal axis vector Ie' with respect to the Yc axis (see Fig.10 (A)) and the slope of the second principal axis vectors P2′ to Pn′ of the transducer 301 on the second angiographic image V2 with respect to the Yc axis. In addition, the first principal axis vector Ie′ is a vector extending from the first mark a1 to the second mark a2 on the first angiographic image V1. In addition, the second principal axis vector P2′ is a vector extending from the first mark b1 to the second mark b2 on the second angiographic image V2, and the second principal axis vector Pn′ is a vector extending from the first mark b1 to the nth mark bn.

[0111] In addition, Fig.10 In the first angiography image V1 shown in (A), it can be said that the tangent vector T′ on the track of the transducer 301 passing through the first marker a1 and the second marker a2 is located on the first principal axis vector Ie′. Therefore, the orientation of the "(B2) transducer axis vectors T1~Tn of the transducer 301 of the imaging sensor 300" can be calculated using the slope δ of the principal axis vector Ie′ in the first angiography image V1 relative to the Yc axis and the slope of the tangent vectors T1′~Tn′ in the second angiography image V2 relative to the Yc axis. In addition, the tangent vector T1′ in the second angiography image V2 is the tangent vector at the first marker b1 on the track of the transducer 301 extending from the first marker b1 to the nth marker bn. In addition, similarly, the tangent vector Tn′ is the tangent vector at the nth marker bn on the track of the transducer 301.

[0112] International application PCT / JP2021 / 034980 discloses the details of the following method: using a first angiographic image V1 and a second angiographic image V2 captured at different first and second positions, respectively, the orientation of a vector of an object located on a straight line where a plane defined by two vectors, a first view vector Vw1 indicating the shooting direction of the first FPD 21 at the first position and a first principal axis vector Ie′ indicating the trajectory of the transducer 301 projected on the first angiographic image V1, intersects with a plane defined by a second view vector Vw2 indicating the shooting direction of the first FPD 21 at the second position and second principal axis vectors P2′ to Pn′ of the transducer 301 projected on the second angiographic image V2. In international application PCT / JP2021 / 034980, a blood vessel existing plane H2 observed from the first position and a blood vessel existing plane S observed from the second position are used to define a straight line where the H2 plane intersects the S plane, and a “blood vessel axis vector” is calculated as the straight line. In the example of the present embodiment, for example, the plane defined by the first view vector Vw1 indicating the shooting direction of the first FPD 21 at the first position and the first principal axis vector Ie′ indicating the trajectory of the transducer 301 projected on the first angiographic image V1 corresponds to the H2 plane, and the plane defined by the second view vector Vw2 indicating the shooting direction of the first FPD 21 at the second position and the second principal axis vectors P2′ to Pn′ projected on the second angiographic image V2 corresponds to the S plane, and the position vectors P2 to Pn calculated in (B1) above may be calculated as vectors equivalent to the blood vessel axis vectors. In addition, the transducer axis vectors T1 to Tn calculated in (B2) above may be calculated in the same manner as vectors equivalent to the blood vessel axis vectors.

[0113] Fig.11 301 is a diagram for explaining the calculation of the lengths of the position vectors P2 to Pn of the transducer 301. Specifically, Fig.11 1 is a diagram for explaining the calculation of the length of the position vector P2 of the transducer 301 when the transducer 301 moves from the starting point (reference point) P1 to the end point P2 in the target blood vessel 100. The length of the position vector Pn can also be calculated in the same manner. Fig.11 , the vector Vw2 is a second view vector Vw2 indicating the shooting direction when the first FPD 21 is in the second position (refer to Figure 8(A)). In addition, b1 is the first mark b1 on the second angiography image V2, which is the position of the transducer 301 on the second angiography image V2 when the transducer 301 is located at the starting point P1. Similarly, b2 is the second mark b2 in the second angiography image V2, which is the position of the transducer 301 on the second angiography image V2 when the transducer 301 is located at the end point P2. Vector P2′ is the second principal axis vector P2′ of the transducer 301 on the second angiography image V2, and is the orthogonal projection vector of the position vector P2 of the transducer 301 onto the second angiography image V2. In addition, θ is the angle between the second view vector Vw2 and the position vector P2. In step S18, the true cavity information acquisition unit 14 uses the following formula (4) to calculate the length of the above-mentioned "(B1) position vector P2~Pn of the transducer 301". If the second view vector Vw2 and the position vector P2 are set as unit vectors, θ can be calculated by the first formula of formula (4) based on the inner product formula of vectors. In addition, since the length of the second principal axis vector P2′ can be calculated based on the coordinates of b1 and b2 in the XcYc coordinates of the canvas A2, the length of the position vector P2 can be derived by the second formula of formula (4). In addition, in the first formula of formula (4), "Vw2×P2" represents the inner product of the second view vector Vw2 and the position vector P2, and in the second formula of formula (4), "P2" and "P2′" represent the lengths of the position vector P2 and the second principal axis vector P2′, respectively.

[0114] [Formula 4]

[0115]

[0116] Fig.12 This is a diagram for explaining steps S19 to S21 of the synthesized image output process. Fig.12 (A) is a diagram showing an example of angiographic image Vα when the first FPD 21 is arranged at position α. Fig.12 (B) is a diagram showing the relationship among the first FPD 21 , the imaging sensor 300 , and the guide wire 500 when the first FPD 21 is arranged at the position α.

[0117] In step S19, the true cavity information acquisition unit 14 guides the operator to pull the transducer 301 of the imaging sensor 300 back to the position of the first mark b1 (= a1). According to the guidance, the operator pulls the transducer 301 back to the position of the first mark b1. In addition, the true cavity information acquisition unit 14 guides the operator to find the position α of the first FPD 21 of the angiography image Vα in which the transducer 301 of the imaging sensor 300 overlaps with the guide wire 500 (in other words, the transducer 301 of the imaging sensor 300 crosses the guide wire 500). Fig.12As shown in (A), the operator moves the first FPD 21 to the position α where the imaging sensor 300 and the guide wire 500 appear to overlap according to the guidance. That is, the angiographic image Vα is an angiographic image captured by the first FPD 21 arranged at the position α where the transducer 301 and the guide wire 500 appear to overlap when the transducer 301 is located at P1 in the target blood vessel 100.

[0118] In step S20 , the true lumen information acquisition unit 14 displays the ultrasonic image IV1 (an ultrasonic image when the transducer 301 is located at the first mark b1 on the second angiographic image V2 , that is, an ultrasonic image when the transducer 301 is located at P1 in the target blood vessel 100 ) on the canvas A2 .

[0119] In step S21, the true cavity information acquisition unit 14 performs an azimuth calibration process (a process of associating the direction from the transducer 301 to the guide wire 500 in the XYZ three-dimensional space with the direction from the transducer 301 to the guide wire 500 in the ultrasound image IV1 displayed in the XcYc two-dimensional space of the canvas A2). Specifically, the true cavity information acquisition unit 14 acquires the position α of the first FPD 21 in step S19. Here, Fig.12 As shown in (B), the first FPD 21 arranged at the position α is positioned relative to the heart 91 (refer to Figure 1 ) is set as the view vector Vwα. The true lumen information acquisition unit 14 calculates the view vector Vwα based on the acquired position α of the first FPD 21. The true lumen information acquisition unit 14 calculates the rotation axis R (r1, r2, r3) using the outer product of the transducer axis vector T1 of the imaging sensor 300 at the position P1 in the target blood vessel 100 calculated in (B2) above and the view vector Vwα as shown in formula (5).

[0120] [Formula 5]

[0121]

[0122] Then, as shown in formula (6), the true cavity information acquisition unit 14 uses the rotation axis R obtained by formula (5) as the axis to calculate the vector CV1 that rotates the transducer axis vector T1 of the transducer 301 calculated by the above (B2) by 90 degrees. That is, the vector CV1 is a vector extending from the transducer 301 toward the guide wire 500 when the transducer 301 is located at position P1 in the XYZ three-dimensional space, and is a vector perpendicular to the transducer axis vector T1. In addition, formula (7) is a formula that expresses the Rodriguez's rotation formula shown in formula (6) in a matrix.

[0123] [Formula 6]

[0124] CV1=Rn(90)T1···(6)

[0125] [Formula 7]

[0126]

[0127] Fig.13 FIG. 4 is a diagram showing an example of an ultrasonic image IV1 displayed on the canvas A2 in step S20. Fig.13 In FIG. 1 , the ultrasonic image IV1 is an ultrasonic image of the interior of the target blood vessel 100 in a direction perpendicular to the transducer axis T1 (the 360° circumferential direction of the transducer axis T1) when the transducer 301 is located at the first mark b1 of the second angiographic image V2. Fig.13 As shown, the ultrasonic image IV1 includes an image of the guide wire 500 (a relatively white portion compared to the surroundings), an image of the true cavity 103 (a relatively dark portion compared to the surroundings), and an image of the portion where the transducer 301 is located (a relatively dark portion compared to the surroundings located near the center of the ultrasonic image IV1). Fig.13 In the drawing, an arrow CV is displayed from the center of the ultrasonic image IV1 (i.e., the center of the transducer 301) toward the center of the image of the guide wire 500. The extending direction of the arrow CV in the XcYc two-dimensional space of the canvas A2 is equivalent to the direction of the vector CV1 in the calculated XYZ three-dimensional space. In step S21, the direction from the transducer 301 to the guide wire 500 in the XYZ three-dimensional space is associated with the direction from the transducer 301 to the guide wire 500 in the ultrasonic image IV1 displayed in the XcYc two-dimensional space of the canvas A2.

[0128] Fig.14 2 is a diagram for explaining step S22 of the synthetic image output process. In step S22, the true cavity information acquisition unit 14 performs a size calibration process (a process of associating the number of pixels of the ultrasonic image IV1 displayed on the canvas A2 with the actual size of the ultrasonic image IV1). Fig.14 As shown in FIG. 1 , a scale SC indicating the actual size of the target blood vessel 100 is added to the ultrasonic image IV1. In the present embodiment, the interval between adjacent scales is 1 mm. The true lumen information acquisition unit 14 draws a line segment from the center of the image of the target blood vessel 100 to a scale located outside the contour of the image of the target blood vessel 100, and measures how many pixels ( Fig.14 The true cavity information acquisition unit 14 calculates the number of pixels per actual size of 1 mm on the canvas A2 by calculating the measured number of pixels / the length of the line segment.

[0129] Fig.15 This is a diagram for explaining step S23 of the synthesized image output process. Fig.15An example of an ultrasonic image IV1 displayed on the canvas A2 in step S23 is shown. In step S23, the true cavity information acquisition unit 14 calculates the true cavity vector S1 in the XYZ three-dimensional space (a vector perpendicular to the transducer axis T1 and extending from the transducer 301 to the true cavity 103). Specifically, the true cavity information acquisition unit 14 guides the center of the ultrasonic image IV1 (i.e., the center of the transducer 301 of the imaging sensor 300) in such a way that an arrow CV is drawn toward the center of the image of the guide wire 500 of the ultrasonic image IV1. Fig.15 As shown, the image of the portion where the transducer 301 is located is located near the center of the ultrasonic image IV1 and appears relatively dark compared to the surroundings. In addition, the image of the guidewire 103 appears relatively white compared to the surroundings, so that the operator who observes the ultrasonic image IV1 can grasp the positions of the transducer 301 and the guidewire 500. According to the guidance, the operator draws an arrow CV from the center of the ultrasonic image IV1 toward the center of the image of the guidewire 500. The drawing of the arrow CV can be achieved, for example, by clicking or tapping the center of the transducer 301 and the center of the image of the guidewire 500 on the ultrasonic image IV1 of the canvas A2. The vector represented by the arrow CV, that is, the vector extending from the center of the transducer 301 to the center of the guidewire 500 in the XcYc coordinates is called the vector cv. In addition, the true cavity information acquisition unit 14 guides in a manner that draws an arrow S from the center of the ultrasonic image IV1 (that is, the center of the image of the transducer 301 of the imaging sensor 300) toward the center of the image of the true cavity 103 of the ultrasonic image IV1. As shown in FIG. Fig.15 As shown, on the ultrasonic image IV1, the true cavity 103 is relatively darker than the surroundings, so the operator who observes the ultrasonic image IV1 can grasp the position of the true cavity 103. According to the guidance, the operator draws an arrow S from the center of the ultrasonic image IV1 toward the center of the image of the true cavity 103. Drawing of the arrow S can be achieved by, for example, clicking or tapping the center of the transducer 301 and the center of the image of the true cavity 103 on the image of the canvas A2. The vector indicated by the arrow S, that is, the vector extending from the center of the transducer 301 to the center of the true cavity 103 in the XcYc coordinates, is set to vector s. In addition, the angle formed by the vector cv and the vector s is set to θ.

[0130] Then, the true cavity information acquisition unit 14 calculates the angle θ formed by the vector cv and the vector s according to the vector cv and the vector s in the XcYc coordinates of the canvas A2 by the formula of the inner product of the vectors. Then, as shown in formula (8), the true cavity information acquisition unit 14 uses the transducer axis vector T1 (r1, r2, r3) of the imaging sensor 300 calculated in the above (B2) as the rotation axis, and rotates the vector CV1 calculated in step S21 by θ degrees, thereby calculating the direction of the true cavity vector S1 in the XYZ three-dimensional space. In addition, formula (9) is a formula that expresses the rotation formula of Rodrigues shown in formula (8) in a matrix.

[0131] [Formula 8]

[0132]

[0133] [Formula 9]

[0134]

[0135] In addition, the true cavity information acquisition unit 14 acquires the number of pixels a of the arrow S drawn in the XcYc coordinates of the canvas A2 and the number of pixels c corresponding to the width of the image of the true cavity 103 of the ultrasonic image IV1. In addition, the number of pixels c of the image of the true cavity 103 can be acquired automatically by performing image analysis on the ultrasonic image IV1, or the width can be designated by the operator. Then, the true cavity information acquisition unit 14 calculates the actual length S of the true cavity vector S1 by substituting the acquired number of pixels a and the result b of step S22 (the number of pixels b on the canvas A2 with an actual size of 1 mm) into equation (10). length (mm). In addition, the true lumen information acquisition unit 14 calculates the actual width S of the true lumen corresponding to the true lumen vector S1 by substituting the acquired number of pixels c and the result b of step S22 into equation (11). width (mm).

[0136] [Formula 10]

[0137]

[0138] [Formula 11]

[0139]

[0140] Fig.16 This is a diagram for explaining steps S24 to S28 of the synthetic image output process. In step S24, the true cavity information acquisition unit 14 substitutes 2 into the variable n used in the synthetic image output process. In step S25, the true cavity information acquisition unit 14 displays the ultrasonic image IV2 stored in the storage unit (that is, the ultrasonic image acquired when n=2 in step S15) on the canvas A2.

[0141] In step S26, the true cavity information acquisition unit 14 calculates the true cavity vector Sn. Specifically, the true cavity information acquisition unit 14 guides the image of the true cavity 103 of the ultrasonic image IV2 displayed on the canvas A2 by drawing an arrow S from the center of the ultrasonic image IV2 toward the ultrasonic image IV2. Then, the true cavity information acquisition unit 14 calculates the angle θd (θd) formed by the transducer axis vector T1 of the imaging sensor 300 calculated in (B2) above and the transducer axis vector T2 of the imaging sensor 300. Fig.16 ). The true cavity information acquisition unit 14 calculates the vector CV2 by rotating the vector CV1 calculated in step S21 by the same amount as the calculated angle θd. In addition, the rotation axis when the vector CV1 is rotated by θd is calculated by the outer product of the transducer axis vectors T1 and T2. Thereafter, the same as step S23. Specifically, the true cavity information acquisition unit 14 obtains the angle θ formed by the arrow CV and the arrow S depicted in the ultrasonic image IV2. As shown in formula (8), the true cavity information acquisition unit 14 uses the transducer axis vector T2 of the imaging sensor 300 as the rotation axis and rotates the vector CV2 by θ degrees, thereby calculating the direction of the true cavity vector S2. In addition, the true cavity information acquisition unit 14 obtains the number of pixels a of the arrow S depicted in the ultrasonic image IV2 and the number of pixels c corresponding to the width of the image of the true cavity 103 of the ultrasonic image IV2. The true cavity information acquisition unit 14 calculates the actual length S of the true cavity vector S2 by substituting the number of pixels a and the number of pixels c into formulas (10) and (11), respectively. length (mm), the actual width S of the true cavity corresponding to the true cavity vector S2 width (mm). Similarly, for n=3 and later, the true cavity information acquisition unit 14 can calculate the vector CVn using the difference θd between the transducer axis vector T1 and the transducer axis vector Tn. In step S27 , the true cavity information acquisition unit 14 adds 1 to the variable n.

[0142] In step S28, the true lumen information acquisition unit 14 determines whether the calculation of the true lumen vector Sn (step S26) has ended at the target number of marks specified in steps S14 to S17. When the variable n reaches the target number of marks (step S28: Yes), the true lumen information acquisition unit 14 transfers the process to step S29. When the variable n does not reach the target number of marks (step S28: No), the true lumen information acquisition unit 14 transfers the process to step S25 and repeats the above-mentioned process. As a result, the variable n is incremented as n=3 and n=4, and the ultrasonic image IVn is displayed on the canvas A2 (step S25), and the calculation of the true lumen vector Sn is performed (step S26).

[0143] In step S29, the true cavity information acquisition unit 14 stores the three-dimensional position information (position information in the XYZ three-dimensional space) of the true cavity 103 acquired in steps S23 to S28 in the storage unit inside the surgical assisting device 10. That is, in the example of the present embodiment, the three-dimensional position information of the true cavity 103 includes the directions of the true cavity vectors S1 to Sn in the XYZ three-dimensional space, the lengths S of the true cavity vectors S1 to Sn, and the position information of the true cavity vectors S1 to Sn. length (mm), and the actual size S of the true cavity corresponding to the true cavity vectors S1 to Sn. width (mm). Among them, the actual size of the true cavity S width This corresponds to "information related to the width of the true lumen". In addition, the actual size S of the true lumen may be substituted in the three-dimensional position information of the true lumen. width The number of pixels c of the image of the true lumen corresponding to the true lumen vectors S1 to Sn is included. In this case, the number of pixels c is equivalent to "information related to the width of the true lumen". In this way, even if the image of the true lumen 103 is not included in the first angiography image V1 and the second angiography image V2, the true lumen information acquisition unit 14 can acquire the three-dimensional position information of the true lumen 103 based on the information of the true lumen 103 acquired from the ultrasonic image IV1.

[0144] Fig.17 as well as Fig.18 This is a diagram for explaining step S30 of the synthesized image output process. Fig.17 (A) is a diagram showing the true cavity vector Sn in the XYZ three-dimensional space, the true cavity image VY obtained by projecting (orthogonally projecting) the true cavity vector Sn from the shooting direction of the first FPD21 arranged at the shooting position A (the direction of the hollow arrow from the upper side to the lower side on the paper) to the projection plane VY′, and the orthogonal projection vector Spn of the true cavity vector Sn on the true cavity image VY. Fig.17 (B) and (C) are diagrams illustrating the calculation of the orthogonal projection vector Spn of the true cavity vector Sn. Fig.18 (A) is a diagram showing an example of an angiographic image VX at an arbitrary FPD position. Fig.18 (B) is a diagram showing an example of a true lumen image VY corresponding to the angiographic image VX.

[0145] In step S30, the image synthesis unit 16 displays the true lumen image superimposed on the image captured at an arbitrary FPD position (angiography image). Specifically, the true lumen image generation unit 15 and the image synthesis unit 16 perform the processing described in (C1) to (C4) below.

[0146] (C1) The true lumen image generation unit 15 uses the first FPD 21 (see FIG. 1 ) disposed at an arbitrary imaging position A. Fig.17(A)), angiographic image VX of the target blood vessel 100 is acquired. The imaging position A corresponds to the "first imaging position".

[0147] (C2) The true cavity image generation unit 15 obtains the position information of the imaging position A from the first FPD 21. In addition, the true cavity image generation unit 15 obtains the position information of the imaging position A from the surgery support device 10 (see Figure 1 ) has a storage unit to obtain the three-dimensional position information of the true cavity.

[0148] (C3) The true lumen image generation unit 15 uses the position information of the imaging position A and the three-dimensional position information of the true lumen to generate a true lumen image VY of the true lumen indicating the position and posture corresponding to the angiographic image VX of the imaging position A. The method of generating the true lumen image VY will be described later in (D1) to (D7).

[0149] (C4) The image synthesis unit 16 generates a synthesized image V by synthesizing the angiographic image VX and the true lumen image VY, and displays the synthesized image V on the canvas A2.

[0150] like Fig.18 As shown, the angiography image VX includes an image of the imaging sensor 300, an image of the guide wire 500, and an image of the target blood vessel 100 (not shown) observed from an arbitrary imaging position A. In addition, the true lumen image VY includes an image of the true lumen 103 at a position and posture corresponding to the angiography image VX (in other words, when observed from the imaging position A where the angiography image VX is obtained). The position of the true lumen is synonymous with the coordinates of the image of the true lumen 103 on the true lumen image VY. The posture of the true lumen is synonymous with the orientation of the image of the true lumen 103 on the true lumen image VY.

[0151] The true lumen image generation unit 15 generates a true lumen image VY through the following procedure (D1) to (D7).

[0152] (D1) Fig.17 As shown in (A), a vector extending perpendicularly from the imaging position A to the projection plane VY′ (hereinafter, the true lumen image VY will be described as the projection plane VY′) is set as a view vector VnA (that is, the view vector VnA is a vector representing the first FPD 21 arranged at the imaging position A relative to the heart 91 (refer to Figure 1 ) in the shooting direction). The true cavity image generation unit 15 specifies the view vector VnA in the XYZ three-dimensional space using, for example, the first equation of equation (12), and calculates the second equation of equation (12) (vector a1 and vector a2) representing the true cavity image VY as a plane orthogonal to the view vector VnA based on the first equation of equation (12).

[0153] [Formula 12]

[0154]

[0155] (D2) The true cavity image generation unit 15 sets the true cavity image VY defined by the vector a1 and the vector a2 calculated in step (D1) as a matrix A, and calculates the projection matrix P for calculating the orthogonal projection vector Spn of the true cavity vector Sn included in the three-dimensional position information of the true cavity 103 onto the true cavity image VY by using equation (13). In equation (13), the matrix A is T Represents the transposed matrix of matrix A.

[0156] [Formula 13]

[0157] P=A(A T A) -1 A T ···(13)

[0158] (D3) The true lumen image generation unit 15 calculates the true lumen vector Sn and the Z axis ( Figure 1 : the direction where the head 92 of the human body 90 is located), and the orthogonal projection vectors Spn and Zp to the true cavity image VY are calculated by formula (14). In addition, in formula (14), Ze refers to the unit vector representing the Z axis in the XYZ three-dimensional space. In addition, the orthogonal projection vectors Spn and Zp calculated by formula (14) are vectors on the true cavity image VY represented by XYZ coordinates.

[0159] [Formula 14]

[0160]

[0161] (D4) The true cavity image generation unit 15 transforms the orthogonal projection vector Spn calculated in step (D3) into a two-dimensional coordinate on the true cavity image VY, and calculates the direction of the orthogonal projection vector Spn on the true cavity image VY. Specifically, Fig.17 As shown in (B), the true cavity image generation unit 15 first calculates the angle θp formed by the orthogonal projection vector Spn of the true cavity vector Sn onto the true cavity image VY and the orthogonal projection vector Zp of the Ze vector representing the Z axis onto the true cavity image VY. The orthogonal projection vector Spn and the orthogonal projection vector Zp are calculated by equation (14), so θp is calculated by, for example, the formula of the inner product of the vectors. Here, it is assumed that the orthogonal projection vector Spn and the orthogonal projection vector Zp are displayed on the canvas A2 with XcYc coordinates (that is, as shown in FIG. Fig.17 As shown in (B), assuming that the true cavity image VY is displayed on canvas A2). Therefore, the vector Zp can be said to be in the same direction as the Z axis in the XYZ three-dimensional space, and is therefore parallel to the Yc axis. However, the direction of the vector Zp is the same as the Z axis (vector Ze), which is the direction in which the head 92 of the human body 90 is located, that is, in Fig.17In (B), it is the direction from the lower side of the paper toward the upper side, so it is the opposite direction to the Yc axis. Next, the true cavity image generation unit 15 sets the unit vector of the orthogonal projection vector Spn in the XcYc coordinates to the orthogonal projection unit vector Spn′(x′, y′), and calculates it by formula (15) or formula (16). It can be said that the orthogonal projection unit vector Spn′(x′, y′) represents the direction of the orthogonal projection vector Spn on the true cavity image VY. In addition, formula (15) is the case when the orthogonal projection unit vector Spn' is in the clockwise direction (CW) when viewed from the orthogonal projection vector Zp, and formula (16) is the case when the orthogonal projection unit vector Spn' is in the counterclockwise direction (CCW) when viewed from the orthogonal projection vector Zp.

[0162] [Formula 15]

[0163] x'=sin(θp),y'=-cos(θp)···(15)

[0164] [Formula 16]

[0165] x'=-sin(θp), y'=-cos(θp)···(16)

[0166] (D5) The true lumen image generation unit 15 calculates the angle between the true lumen vector Sn and the view vector VnA in order to calculate the length of the orthogonal projection vector Spn of the true lumen vector Sn. Specifically, Fig.17 As shown in (c), the angle between the true cavity vector Sn and the view vector VnA is set to θ, and based on the formula of the outer product of vectors, the true cavity vector Sn and the view vector VnA are used to calculate sinθ through equation (17). In equation (17), VnA×Sn means the outer product of the view vector VnA and the true cavity vector Sn.

[0167] [Formula 17]

[0168]

[0169] (D6) If the length of the true cavity vector Sn included in the three-dimensional position information of the true cavity is set to Snlength (mm), the true cavity image generation unit 15 uses sinθ calculated in step (D5) to calculate the length Spnlength of the orthogonal projection vector Spn of the true cavity vector Sn through equation (18) (refer to Fig.17 (c)). Next, the true cavity image generation unit 15 calculates the orthogonal projection vector Spn (x, y) of the true cavity vector Sn in the XcYc coordinates of the canvas A2 through equation (15) or equation (16), equation (18) and equation (19). It can be said that the orthogonal projection vector Spn (x, y) represents the direction and length of the orthogonal projection vector Spn on the true cavity image VY.

[0170] [Formula 18]

[0171] Spn length =Sn length ×sinθ···(18)

[0172] [Formula 19]

[0173]

[0174] (D7) The true lumen image generator 15 generates a true lumen image VY using the orthogonal projection vector Spn (x, y) obtained in step (D6).

[0175] Fig.19 This is a diagram showing an example of a synthesized image. Fig.19 (A) is a diagram showing an example of a composite image V at an arbitrary imaging position A. Fig.19 (B) is a diagram showing an example of a composite image Vb at a shooting position B different from the shooting position A. By performing the processing described in the sequence (C1) to (C4), the image Vb is displayed on the canvas A2. Fig.19 The composite image V shown in (A) is a composite image V of Fig.18 The angiography image VX shown in the figure is superimposed with the image of the true lumen image VY. Here, in the true lumen image VY of the present embodiment, the image of the true lumen 103 is at a position and posture corresponding to the angiography image VX of the imaging position A, and has a width corresponding to the three-dimensional position information of the true lumen. When generating such a true lumen image VY, the true lumen image generating unit 15 further uses the actual size S of the true lumen included in the three-dimensional position information of the true lumen in sequence (D7). width (mm), the width of the image of the true cavity 103 can be changed.

[0176] Here, the case where the operator moves the first FPD 21 from the imaging position A to the imaging position B different from A is described. In addition, the imaging position B is equivalent to the "second imaging position". In this case, the true cavity image generation unit 15 and the image synthesis unit 16 use the position information of the changed imaging position B (that is, the position information of the second imaging position) to repeatedly perform the above-mentioned sequence (C1) to (C4). Fig.19 As shown in (B), the composite image Vb obtained by synthesizing the angiography image VX including the image of the imaging sensor 300, the guide wire 500, and the target blood vessel 100 observed from different angles and the true lumen image VY including the image of the true lumen 103 corresponding to the angle is displayed again on the canvas A2. Fig.19 In the example of (B), the image of the true cavity 103 included in the true cavity image VY has an actual size S of the true cavity. widthIn this way, the operator can intuitively grasp the position, posture, thickness of the true cavity 103 and the positional relationship between the medical equipment (imaging sensor 300, guide wire 500) and the true cavity 103.

[0177] Fig. 20 as well as Fig.21 This is a diagram for explaining a modified example of the true cavity image VY. Fig. 20 (A) is a diagram showing a synthesized image Vc using the true lumen image VY according to the first modification. Fig. 20 (B) is a diagram showing a synthesized image Vd using the true cavity image VY according to the second modification. Fig.21 FIG. 4 is a diagram showing a composite image Ve using a true cavity image VY according to the third modification example. Fig. 20 In the example of (A), the image of the true cavity 103 included in the true cavity image VY has an actual size S of the true cavity. width (mm) corresponding to the width of the line segment. In this way, it is possible to prevent the visibility of the front end of the medical device (imaging sensor 300, guide wire 500) from being reduced due to the image of the true cavity 103. Fig. 20 In the example of (A), the image of the true cavity 103 included in the true cavity image VY has an actual size S of the true cavity. width In this way, it is possible to suppress the visibility of the front end of the medical device (imaging sensor 300, guide wire 500) being reduced due to the image of the true lumen 103, and it is possible to easily grasp the inclination of the true lumen 103 based on the inclination of the true lumen cross section. Fig.21 In the example, the image of the true lumen 103 included in the true lumen image VY is a line segment having a position and posture corresponding to the angiographic image VX of the imaging position A and having no width. In this way, the processing load in the true lumen image generator 15 can be reduced.

[0178] As described above, according to the surgical assistance device 10 of the first embodiment, the true lumen image generation unit 15 can generate a true lumen image VY of the true lumen indicating the position and posture corresponding to the angiographic image VX, using the position information of the first imaging position A at which the angiographic image VX is acquired and the three-dimensional position information of the true lumen acquired by the true lumen information acquisition unit 14. That is, the true lumen image generation unit 15 can generate the true lumen image VY indicating the image of the true lumen 103 based on the three-dimensional position information of the true lumen, even when the contrast agent does not flow into the true lumen 103 as the target or when the contrast agent does not flow. In addition, the image synthesis unit 16 generates a synthesized image V synthesized by synthesizing the angiographic image VX at an arbitrary first imaging position A and the true lumen image VY indicating the image of the true lumen 103, and outputs the synthesized image V, so that the image of the FPD (angiographic image VX) can display the image of the true lumen 103 of the target blood vessel 100. Therefore, the operator can perform the operation while confirming the positional relationship between the medical devices 300, 500 on the angiographic image VX and the true lumen 103 on the true lumen image VY by confirming the synthesized image V. As a result, the operator can accurately grasp the position of the true lumen 103 in the target blood vessel 100, thereby improving the accuracy of the operation, shortening the time required for the operation, and reducing the burden on the patient.

[0179] In addition, according to the surgical assisting device 10 of the first embodiment, the true lumen image generating unit 15 generates the true lumen image VY representing the true lumen 103 having a width corresponding to the three-dimensional position information of the true lumen, so the operator can perform the surgery while checking the width of the true lumen 103 by checking the synthetic image V. As a result, it is possible to further improve the accuracy of the surgery, shorten the time required for the surgery, and reduce the burden on the patient.

[0180] Furthermore, according to the surgical assisting device 10 of the first embodiment, when the first FPD 21 moves to the second imaging position B different from the first imaging position A and the first FPD 21 performs imaging, the true lumen image generation unit 15 regenerates the true lumen image VY corresponding to the angiography image VX at the second imaging position B, and the image synthesis unit 16 regenerates the synthesized image Vb synthesized by synthesizing the angiography image VX acquired again and the true lumen image VY generated again, and outputs the synthesized image Vb. That is, the true lumen image generation unit 15 and the image synthesis unit 16 can follow the movement of the imaging position of the first FPD 21 and display the synthesized image Vb including the moved true lumen image VY. As a result, the convenience of the surgical assisting device 10 can be improved, and the accuracy of the surgery can be further improved, the time required for the surgery can be shortened, and the burden on the patient can be reduced.

[0181] Furthermore, according to the surgical assisting device 10 of the first embodiment, the true lumen information acquiring unit 14 can acquire the three-dimensional position information ( 3D position information ) of the true lumen by using the position information of the first position acquired from the first angiographic image, the first angiographic image, the position information of the second position acquired from the second angiographic image, the second angiographic image, and the ultrasonic image. Figure 4 , 5 :Steps S3 to S29). Specifically, the true cavity information acquisition unit 14 can use the position information of the first position and the first angiography image V1, the position information of the second position and the second angiography image V2 to acquire the three-dimensional position information ( Figure 5 :Step S18). Then, the true cavity information acquisition unit 14 can use the three-dimensional position information of the imaging sensor 300, the position information of the first position, the first angiography image V1, and the ultrasonic image IVn of the true cavity 103 of the target blood vessel 100 to acquire the three-dimensional position information of the true cavity ( Figure 5 : Steps S23, S26).

[0182] Furthermore, according to the surgical assisting device 10 of the first embodiment, the true cavity information acquiring unit 14 can acquire the three-dimensional position information ( Figure 5 : Step S18). In addition, the true lumen information acquisition unit 14 can associate the positional relationship between the first angiography image V1 and the ultrasonic image IV1 using the image of the medical device (specifically, the guide wire 500) included in the first angiography image V1 and the ultrasonic image IV1 ( Figure 5 : Step S21), obtain the position information of the true cavity 103 from the ultrasonic image IVn ( Figure 5 : Steps S23, S26), using the acquired position of the imaging sensor 300 and the position information of the true cavity 103 in the ultrasonic image IVn obtained by the imaging sensor 300, the three-dimensional position information of the true cavity is acquired (step S29).

[0183] <Second Embodiment>

[0184] Fig. 22 This is an explanatory diagram illustrating the structure of a surgical assisting system 1A according to a second embodiment. The surgical assisting system 1A according to the second embodiment is different from the first embodiment in that the three-dimensional position information of the true cavity is obtained without using an imaging sensor 300. In the structure of the first embodiment, the surgical assisting system 1A has a surgical assisting device 10A instead of the surgical assisting device 10. The surgical assisting device 10A does not have an ultrasonic image acquisition unit 13, and has a true cavity information acquisition unit 14A instead of the true cavity information acquisition unit 14.

[0185] Since the imaging sensor 300 is not used in the surgery assisting system 1A, Figure 3 4 is a state in which the line catheter 400 and the guidewire 500 are inserted into the target blood vessel 100, but the sensor catheter 200 and the imaging sensor 300 are not inserted. In the synthetic image output process, the true lumen information acquisition unit 14A acquires the three-dimensional position information of the true lumen using the image of the guidewire 500 and the image of the true lumen 103 included in the first angiographic image and the second angiographic image.

[0186] Fig.23 This is a flowchart showing an example of the synthesized image output process according to the second embodiment. Fig.24 It is a diagram for explaining the synthesized image output processing according to the second embodiment. Fig.24 (A) is a diagram for explaining steps S3A to S7A. Fig.24 (B) is a diagram for explaining steps S10 to S17. Fig.23 In step S1, the true cavity information acquisition unit 14A guides the preparation for imaging of the first FPD 21. Figure 4 The same as step S1.

[0187] In step S3A, the true cavity information acquisition unit 14A guides the operator to move the first FPD 21 to the first position to capture an X-ray image. The operator moves the first FPD 21 to the first position to capture an X-ray image according to the guidance, thereby acquiring a first angiographic image V1. The first position may be any position (RAOXX CRAXX: X is an arbitrary natural number). The angiographic image acquisition unit 12 acquires the captured first angiographic image V1 from the angiographic device 20. Fig.24 As shown in FIG. 1A , the first angiographic image V1 includes an image of the distal end portion 401 of the wire catheter 400 , an image of the guidewire 500 inserted into the wire catheter 400 , an image of the target blood vessel 100 (not shown), and an image of the true lumen 103 .

[0188] In step S4A, the true lumen information acquisition unit 14A displays the first angiographic image V1 on the canvas A2 and adjusts the position of the first angiographic image V1 so that the image of the distal end portion 401 of the wire catheter 400 is located at the center of the canvas A2. In the second embodiment, the distal end portion 401 of the wire catheter 400 is treated as the "reference point of the reference device".

[0189] In step S5A, the true lumen information acquisition unit 14A guides the user by placing a first marker a1 at the end (in other words, the end on the proximal side) of the image of the true lumen 103 in the first angiography image V1 displayed on the canvas A2. Fig.24As shown in (A) of FIG. 2 , the operator places the first marker a1 on the first angiographic image V1 on the canvas A2 according to the guidance.

[0190] In step S7A, the true lumen information acquisition unit 14A guides the user to arrange the second marker a2 at any position in the range in which the image of the true lumen 103 in the first angiography image V1 displayed on the canvas A2 is considered to extend linearly with respect to the first marker a1. Fig.24 As shown in (A) of FIG. 2 , the operator places the second marker a2 on the image of the true lumen 103 appearing in the first angiographic image V1 on the canvas A2 according to the guidance.

[0191] In step S8A, the true cavity information acquisition unit 14A calculates BNV using the first marker, the second marker, and the first position. Figure 4 The process is the same as step S8, except that the “first axis vector of the imaging sensor 300” in step S8 is replaced by the “central axis vector of the true lumen 103 in the first angiography image V1”.

[0192] In step S10, the true lumen information acquisition unit 14A guides the first FPD 21 to move to the BNV (second position) calculated in step S8A to capture an X-ray image. The operator moves the first FPD 21 to the BNV (second position) according to the guidance to capture an X-ray image, thereby acquiring the second angiographic image V2. The angiographic image acquisition unit 12 acquires the captured second angiographic image V2 from the angiographic device 20. Fig.24 As shown in (B), the second angiography image V2 includes an image of the front end portion 401 of the wire catheter 400, an image of the guide wire 500 inserted into the wire catheter 400, an image of the target blood vessel 100 (not shown), and an image of the true lumen 103, all of which are captured from a different direction from the first angiography image V1.

[0193] In step S11A, the true lumen information acquisition unit 14A displays the second angiographic image V2 on the canvas A2 and adjusts the position of the second angiographic image V2 so that the image of the distal end portion 401 of the wire catheter 400 is located at the center of the canvas A2.

[0194] In step S12A, the true lumen information acquisition unit 14A guides the user so as to arrange the first marker b1 at the end (in other words, the end on the proximal side) of the image of the true lumen 103 in the second angiographic image V2 displayed on the canvas A2. Fig.24 As shown in (B) of FIG. 2 , the operator places the first marker b1 on the image of the true lumen 103 shown in the second angiographic image V2 on the canvas A2 according to the guidance.

[0195] In step S13, the true lumen information acquisition unit 14A substitutes 2 into the variable n used in the synthetic image output process. n is a natural number. In step S14A, the true lumen information acquisition unit 14A guides the operator to arrange the second marker b2 at an arbitrary position away from the end of the image of the true lumen 103 in the distant direction. According to the guidance, the operator arranges the second marker b2 for the image of the true lumen 103 reflected in the second angiography image V2 on the canvas A2. In step S16, the true lumen information acquisition unit 14A adds 1 to the variable n.

[0196] In step S17, the true lumen information acquisition unit 14A determines whether the placement of the marks on the second angiography image V2 (step S14A) has reached the target number of marks. If the variable n reaches the target number of marks (step S17: Yes), the true lumen information acquisition unit 14A transfers the process to step S18A. If the variable n does not reach the target number of marks (step S17: No), the true lumen information acquisition unit 14A transfers the process to step S14A and repeats the above process.

[0197] In step S18A, the true lumen information acquisition unit 14A calculates the following (E1) and (E2) using the coordinates of the first marker a1 and the second marker a2 in the first angiographic image V1 and the coordinates of the first marker b1 to the nth marker bn in the second angiographic image V2.

[0198] (E1) True cavity vector S2~Sn: True cavity vector S2~Sn is equivalent to Figure 4 as well as Figure 5 The true cavity vectors S1~Sn obtained in .

[0199] (E2) Vector P from the reference point of the reference device (the distal end 401 of the wire catheter 400) to the end of the true lumen 103.

[0200] Fig.25 This is a diagram for explaining step S18A of the synthesized image output process according to the second embodiment. Fig.25 (A) is a diagram showing each point on the first angiography image V1, Fig.25 (B) is a diagram showing each point on the second angiography image V2. Fig.25 (A) and (B) are the Z axis ( Figure 1 : The direction in which the head 92 of the human body 90 is located) is located on the upper side of the paper. The first angiographic image V1 and the second angiographic image V2 of the second embodiment are images of the true lumen 103 captured from different angles, so it can be said that Fig.25 The first marker b1 to the nth marker bn (n=4 in the example shown in the figure) in the second angiographic image V2 shown in FIG. Fig.25On the vector S′ of the first angiography image V1 shown in (A). Therefore, the direction of the “(E1) true lumen vector S2~Sn” can be calculated using the slope of the vector S′ and the slope of the vectors S2′~Sn′. In addition, the vector S′ is a vector extending from the first marker a1 to the second marker a2. In addition, the vector S2′ is a vector extending from the first marker b1 to the second marker b2, and the vector Sn′ is a vector extending from the first marker b1 to the nth marker bn. The details of the method for calculating the direction of the “(E1) true lumen vector S2~Sn” are as follows. Figure 5 The same as step S18, the "position vectors P2 to Pn of the transducer 301" in step S18 are replaced with "true cavity vectors S2 to Sn" to perform the processing. In addition, the true cavity information acquisition unit 14A uses Figure 5 The length of "(E1) true cavity vector S2~Sn" is calculated using equation (4) described in step S18.

[0201] The direction of "(E2) vector P from the reference point of the reference device to the end of the true lumen 103" can be calculated using the slope of the vector P' in the first angiography image V1 and the slope of the vector P' in the second angiography image V2. In addition, the vector P' in the first angiography image V1 is a vector extending from the reference point of the reference device in the first angiography image V1 (the front end 401 of the wire catheter 400) to the first marker a1. In addition, the vector P' in the second angiography image V2 is a vector extending from the reference point of the reference device in the second angiography image V2 to the first marker b1 (= a1). Regarding the calculation of the direction of "(E2) vector P from the reference point of the reference device to the end of the true lumen 103", it is possible to apply Figure 5 The direction of the "position vector P2 to Pn of the transducer 301" in step S18 can be calculated by the calculation method. The length of "(E2) vector P from the reference point of the reference device to the end of the true cavity 103" can be calculated by using Figure 5 It is calculated using the formula (4) described in step S18.

[0202] In step S29, the true lumen information acquisition unit 14A stores the three-dimensional position information of the true lumen obtained in step S18A in the storage unit inside the surgical assisting device 10. That is, in the example of the present embodiment, the three-dimensional position information of the true lumen includes the direction of the true lumen vectors S2 to Sn and the length of the true lumen vectors S2 to Sn. In step S30, the image synthesis unit 16 displays the true lumen superimposed on the captured image (angiography image) at any FPD position. Figure 5 The same as step S30.

[0203] In this way, when the first angiographic image V1 and the second angiographic image V2 include the image of the true lumen 103, as described in the second embodiment, the three-dimensional position information of the true lumen can be obtained without using the imaging sensor 300. In such a second embodiment, the same effect as the first embodiment can be achieved. In addition, according to the surgical assistance device 10A of the second embodiment, the true lumen information acquisition unit 14A can obtain the three-dimensional position information of the true lumen using the image of the medical device and the image of the true lumen included in the first angiographic image and the second angiographic image.

[0204] <Third Embodiment>

[0205] Fig.26 : is an explanatory diagram illustrating the structure of a surgery support system 1B according to the third embodiment. In the surgery support system 1B according to the third embodiment, the surgery support device 10B is connected to a network, and acquires the first angiography image V1, the second angiography image V2, the ultrasonic image IV1, and the ultrasonic image IVn from an external device via the network, respectively, to acquire three-dimensional position information of the true lumen. Then, the surgery support device 10B acquires an angiography image VX arranged at an arbitrary imaging position, and generates a true lumen image VY including an image of the true lumen 103 at a position and posture corresponding to the angiography image VX. The surgery support device 10B generates a composite image V synthesized by synthesizing the angiography image VX and the true lumen image VY, and outputs the generated composite image V to an external device via the network. In addition, the surgery support device 10B may acquire the images V1, V2, IV1, and IVn from a storage medium such as a USB memory, instead of an external device connected via the network, and output the composite image V to the storage medium.

[0206] In this way, the structure of the surgery support system 1B can be changed in various ways, and it is also possible not to have the blood vessel imaging device 20, the display device 30, the workbench 40, and the operation unit 50. In other words, the surgery support system 10B can also be configured as an information processing device or a server. In such a third embodiment, the same effect as the first embodiment described above can also be achieved. In addition, according to the surgery support system 1B of the third embodiment, it is possible to realize a service of providing a synthetic image to an external device connected via a network.

[0207] <Variations of this embodiment>

[0208] The present invention is not limited to the above-mentioned embodiments, and can be implemented in various ways within the scope of the main purpose thereof. For example, a part of the structure implemented by hardware can be replaced by software, and conversely, a part of the structure implemented by software can be replaced by hardware. In addition, for example, the following deformations can also be performed.

[0209] [Modification 1]

[0210] In the above-mentioned first to third embodiments, the structures of the surgical support systems 1, 1A, and 1B are illustrated. However, the structure of the surgical support system 1 can be changed in various ways. For example, the display device 30 can also be a monitor or a touch panel built into the surgical support device 10, 10A, and 10B. For example, the blood vessel imaging device 20 can also be a structure having a single FPD (in other words, a structure without the second FPD 25). For example, the surgical support system 1 can also have other medical devices not shown in the figure (for example, a CT device, an MRI device), etc. In this case, Figure 6 The operation screen OS described in (A) may also include images acquired by other medical devices.

[0211] [Modification 2]

[0212] In the first to third embodiments, the structures of the surgical assisting devices 10, 10A, and 10B are shown. However, the structure of the surgical assisting device 10 can be modified in various ways. For example, the functions of the functional units of the surgical assisting device 10 can be realized by cooperation of a plurality of devices connected via a network.

[0213] [Variation 3]

[0214] In the first to third embodiments described above, an example of the sequence of the synthesized image output process is shown. Figure 4 , Figure 5 , Fig.23 The order of the synthesized image output process described in the above is only an example, and various changes can be made. For example, the execution order of each step can be changed, at least a part of the steps can be omitted, and other steps not described can be executed.

[0215] For example, the three-dimensional position information of the true cavity may not be specified by the direction and length of the true cavity vectors S1 to Sn. In this case, the three-dimensional position information of the true cavity may be specified by any information such as a set of point column coordinates forming the outer edge of the true cavity, a set of coordinates of feature points in the outer edge of the true cavity, etc.

[0216] For example, when the first FPD 21 moves to the shooting position B (second shooting position) different from the shooting position A (first shooting position), the regeneration of the true cavity image VY and the re-output of the synthesized image Vb may be omitted. In addition, for example, when the first FPD 21 moves to the shooting position B (second shooting position) different from the shooting position A (first shooting position), the image synthesis unit 16 may output to the canvas A2 in a manner that allows visual recognition of both the synthesized image V at the first shooting position and the synthesized image Vb at the second shooting position. The manner in which both can be visually recognized can be appropriately determined by referring to the history of the arrangement display, etc.

[0217] For example, the true lumen image generator 15 may set the image of the true lumen 103 included in the true lumen image VY to be semi-transparent, or may set at least any one of the hue / chroma / brightness of the image of the true lumen 103 to a hue / chroma / brightness that is easily recognizable with the imaging sensor 300 and the guide wire 500. In this way, the image of the true lumen 103 can be displayed without impairing the visibility of the medical equipment (imaging sensor 300, guide wire 500). This adjustment may be automatically performed by the true lumen image generator 15 performing image analysis on the angiography image VX, or may be changed according to the designated content obtained from the operator.

[0218] For example, the image synthesis unit 16 may be configured to switch between displaying / hiding the angiographic image XV and displaying / hiding the true lumen image VY in the synthesized image V. In this way, the usability of the surgical support apparatuses 10 , 10A, 10B can be further improved.

[0219] For example, in Figure 4 and Figure 5 In the order of synthetic image output processing, the processing of step S19 may also be included in step S3. Specifically, the "true cavity information acquisition unit 14 guides the first FPD21 to move to the first position to capture an X-ray image" in step S3 may also be set to "the true cavity information acquisition unit 14 guides the first FPD21 to move to the first position (equivalent to the position α of step S19) such that a first angiography image V1 (equivalent to the angiography image Vα of step S19) in which the transducer 301 of the imaging sensor 300 overlaps with the guide wire 500 (in other words, the transducer 301 of the imaging sensor 300 crosses the guide wire 500) is captured to capture an X-ray image". The operator moves the first FPD21 to the first position in accordance with the guidance to obtain the first angiography image V1 in which the transducer 301 and the guide wire 500 overlap, and obtains the first angiography image V1 in which the transducer 301 and the guide wire 500 overlap. In this case, in step S21, the true cavity information acquisition unit 14 performs the orientation calibration process using the first position of the first FPD 21. By changing the order of the synthetic image output process as described above, step S19 can be omitted, and the synthetic image output process can be made more efficient.

[0220] [Variation 4]

[0221] The structures of the surgical assisting devices 10, 10A, and 10B of the first to third embodiments and the structures of the modifications 1 to 3 may be appropriately combined. For example, the main control unit 11 may be configured to switch and execute the synthetic image output processing ( Figure 4 , Figure 5 ) and the synthesized image output processing described in the second embodiment ( Fig.23 The main control unit 11 may switch according to an instruction from the operator, or may switch automatically by performing image analysis on the first angiographic image V1 and the second angiographic image V2. For example, in the case of automatic switching, the main control unit 11 may perform the synthetic image output processing described in the second embodiment when the first angiographic image V1 and the second angiographic image V2 contain an image of the true lumen 103 ( Fig.23 ), the synthetic image output process described in the first embodiment is performed without including the image of the true cavity 103 ( Figure 4 , Figure 5 ).

[0222] The present invention has been described above based on the embodiments and variations, but the embodiments of the above-mentioned embodiments are intended to facilitate understanding of the present invention and do not limit the present invention. The present invention can be changed and improved within the scope of the subject matter and claims, and its equivalents are included in the present invention. In addition, if its technical features are not described as essential features in this specification, they can be appropriately deleted.

[0223] Description of Reference Numerals

[0224] 1.A, 1B…Surgery Assist System

[0225] 10, 10A, 10B…Surgical assist devices

[0226] 11…Main control unit

[0227] 12…Angiography image acquisition unit

[0228] 13…Ultrasonic image acquisition unit

[0229] 14, 14A…True cavity information acquisition unit

[0230] 15…True cavity image generation unit

[0231] 16…Image synthesis unit

[0232] 20…Vascular imaging device

[0233] 22…First X-ray tube device

[0234] 23…The first C-arm

[0235] 24…First support portion

[0236] 26…Second X-ray tube device

[0237] 27…Second C-arm

[0238] 28 ... second supporting portion

[0239] 29…Control Department

[0240] 30…Display device

[0241] 31…Monitor

[0242] 32…arm

[0243] 40…Workbench

[0244] 41…bed

[0245] 42…Telescopic part

[0246] 43…Legs

[0247] 50…Operation Department

[0248] 90…Human body

[0249] 91…Heart

[0250] 92…Head

[0251] 93…feet

[0252] 100…Target blood vessels

[0253] 102…False cavity

[0254] 103…True cavity

[0255] 200...Sensor tube

[0256] 300…Imaging Sensor

[0257] 301…Transducer

[0258] 400…Wire conduit

[0259] 401…Front end

[0260] 500…Guidewire.

Claims

1. A surgical assisting device, characterized in that: have: a true lumen information acquisition unit that acquires three-dimensional position information of a true lumen existing in a target blood vessel; a true lumen image generating unit that acquires an angiographic image of the target blood vessel from an FPD disposed at a first imaging position, and generates a true lumen image of the true lumen indicating a position and a posture corresponding to the angiographic image using position information of the first imaging position and three-dimensional position information of the true lumen, wherein the FPD is a flat panel detector; and An image synthesis unit generates a synthesized image by synthesizing the angiographic image and the true lumen image, and outputs the synthesized image.

2. The surgical assisting device according to claim 1, characterized in that: The three-dimensional position information of the true lumen includes information related to the width of the true lumen. The true lumen image generation unit generates a true lumen image representing the true lumen having a width corresponding to three-dimensional positional information of the true lumen.

3. The surgical assisting device according to claim 1 or 2, characterized in that: The true lumen image generating unit reacquires the angiography image captured at the second imaging position when the FPD is moved to a second imaging position different from the first imaging position and the target blood vessel is imaged from the FPD, and regenerates the true lumen image of the true lumen showing the position and posture corresponding to the reacquired angiography image using the position information of the second imaging position and the three-dimensional position information of the true lumen, The image synthesis unit regenerates a synthesized image by synthesizing the reacquired angiographic image and the regenerated true lumen image, and outputs the synthesized image.

4. The surgical assisting device according to any one of claims 1 to 3, characterized in that: The surgical assisting device further comprises: an angiographic image acquisition unit that acquires a first angiographic image captured by the FPD disposed at a first position and a second angiographic image captured by the FPD disposed at a second position different from the first position; as well as an ultrasonic image acquisition unit that acquires an ultrasonic image of the interior of the target blood vessel captured by an ultrasonic sensor, The first angiographic image comprises: the ultrasonic sensor disposed at a first marked position within a blood vessel of the subject; and a medical device different from the ultrasonic sensor and disposed at a second marked position in the subject's blood vessel, The second angiographic image includes the ultrasonic sensor disposed at the first marking position in the target blood vessel, The ultrasonic image is an image captured when the ultrasonic sensor is arranged at the first marking position. The ultrasonic image includes the target blood vessel and the medical device disposed at the second marked position in the target blood vessel, The true lumen information acquisition unit acquires three-dimensional position information of the true lumen using the position information of the first position, the first angiographic image, the position information of the second position, the second angiographic image, and the ultrasonic image.

5. The surgical assisting device according to claim 4, characterized in that: The true cavity information acquisition unit performs the following processing: acquiring the position of the ultrasonic sensor using the image of the ultrasonic sensor included in the first angiographic image and the second angiographic image; using the image of the medical device included in the first angiographic image and the ultrasonic image, associating the positional relationship between the first angiographic image and the ultrasonic image; Acquire the position information of the true cavity from the ultrasonic image; and Three-dimensional position information of the true lumen is acquired using the acquired position of the ultrasonic sensor and the position information of the true lumen in the ultrasonic image obtained by the ultrasonic sensor.

6. The surgical assisting device according to any one of claims 1 to 3, characterized in that: The surgical assisting device further includes an angiographic image acquisition unit that acquires a first angiographic image captured by the FPD disposed at a first position and a second angiographic image captured by the FPD disposed at a second position different from the first position. The first angiographic image includes the true lumen of the target blood vessel and a medical device disposed at a first marked position in the target blood vessel, the second angiographic image includes the true lumen of the target blood vessel and the medical device disposed at the first marked position in the target blood vessel, The true lumen information acquisition unit acquires three-dimensional position information of the true lumen using the image of the medical device and the image of the true lumen included in the first angiographic image and the second angiographic image.

7. A surgical assistance method, characterized in that: The following processes are available: a true lumen information acquisition step of acquiring three-dimensional position information of a true lumen existing in the target blood vessel; a true lumen image generating step of acquiring a blood vessel imaging image of the target blood vessel from an FPD disposed at a first imaging position, and generating a true lumen image of the true lumen indicating a position and a posture corresponding to the blood vessel imaging image using position information of the first imaging position and three-dimensional position information of the true lumen, wherein the FPD is a flat panel detector; and The image synthesis step generates a synthesized image by synthesizing the angiographic image and the true lumen image, and outputs the synthesized image.

8. The surgical assisting method according to claim 7, characterized in that: The three-dimensional position information of the true lumen includes information related to the width of the true lumen. In the true lumen image generation step, a true lumen image representing the true lumen having a width corresponding to the three-dimensional positional information of the true lumen is generated.

9. The surgical assisting method according to claim 7 or 8, characterized in that: In the true lumen image generation step, when the FPD is moved to a second imaging position different from the first imaging position and the target blood vessel is imaged from the FPD, the angiography image captured at the second imaging position is acquired again, and a true lumen image of the true lumen representing the position and posture corresponding to the angiography image acquired again is generated again using position information of the second imaging position and three-dimensional position information of the true lumen, In the image synthesis step, a synthesized image is generated again by synthesizing the angiographic image acquired again and the true lumen image generated again, and the synthesized image is output.

10. The surgical assisting method according to any one of claims 7 to 9, characterized in that: The surgical assistance method further comprises the following steps: an angiographic image acquisition step of acquiring a first angiographic image captured by the FPD disposed at a first position and a second angiographic image captured by the FPD disposed at a second position different from the first position; and an ultrasonic image acquisition step of acquiring an ultrasonic image of the interior of the target blood vessel captured by an ultrasonic sensor, The first angiographic image comprises: the ultrasonic sensor disposed at a first marked position within a blood vessel of the subject; and a medical device different from the ultrasonic sensor and disposed at a second marked position in the subject's blood vessel, The second angiographic image includes the ultrasonic sensor disposed at the first marking position in the target blood vessel, The ultrasonic image is an image captured when the ultrasonic sensor is arranged at the first marking position. The ultrasonic image includes the target blood vessel and the medical device disposed at the second marked position in the target blood vessel, In the true lumen information acquisition step, three-dimensional position information of the true lumen is acquired using the position information of the first position, the first angiographic image, the position information of the second position, the second angiographic image, and the ultrasonic image.

11. The surgical assisting method according to claim 10, characterized in that: The following processing is performed in the true cavity information acquisition step: acquiring the position of the ultrasonic sensor using the image of the ultrasonic sensor included in the first angiographic image and the second angiographic image; using the image of the medical device included in the first angiographic image and the ultrasonic image, associating the positional relationship between the first angiographic image and the ultrasonic image; Acquire the position information of the true cavity from the ultrasonic image; and Three-dimensional position information of the true lumen is acquired using the acquired position of the ultrasonic sensor and the position information of the true lumen in the ultrasonic image obtained by the ultrasonic sensor.

12. The surgical assisting method according to any one of claims 7 to 9, characterized in that: The surgery assisting method further comprises a vascular angiography image acquisition step, in which a first vascular angiography image captured by the FPD arranged at a first position and a second vascular angiography image captured by the FPD arranged at a second position different from the first position are acquired. The first angiographic image includes the true lumen of the target blood vessel and a medical device disposed at a first marked position in the target blood vessel, the second angiographic image includes the true lumen of the target blood vessel and the medical device disposed at the first marked position in the target blood vessel, In the true lumen information acquisition step, three-dimensional position information of the true lumen is acquired using the image of the medical device and the image of the true lumen included in the first angiographic image and the second angiographic image.

13. A computer program, characterized in that The following steps are required: a true lumen information obtaining step of obtaining three-dimensional position information of a true lumen existing in the target blood vessel; a true lumen image generating step of acquiring an angiographic image of the target blood vessel from an FPD disposed at a first imaging position, and generating a true lumen image of the true lumen indicating a position and a posture corresponding to the angiographic image using position information of the first imaging position and three-dimensional position information of the true lumen, wherein the FPD is a flat panel detector; and The image synthesis step generates a synthesized image by synthesizing the angiographic image and the true lumen image, and outputs the synthesized image.

14. The computer program according to claim 13, characterized in that The three-dimensional position information of the true lumen includes information related to the width of the true lumen. In the true lumen image generating step, a true lumen image representing the true lumen having a width corresponding to the three-dimensional positional information of the true lumen is generated.

15. The computer program according to claim 13 or 14, characterized in that In the true lumen image generation step, when the FPD is moved to a second imaging position different from the first imaging position and the target blood vessel is imaged from the FPD, the angiographic image captured at the second imaging position is acquired again, and a true lumen image of the true lumen representing the position and posture corresponding to the angiographic image acquired again is generated again using position information of the second imaging position and three-dimensional position information of the true lumen, In the image synthesis step, a synthesized image is generated again by synthesizing the angiography image acquired again and the true lumen image generated again, and the synthesized image is output.

16. The computer program according to any one of claims 13 to 15, characterized in that The computer program further comprises the following steps: an angiographic image acquisition step of acquiring a first angiographic image captured by the FPD disposed at a first position and a second angiographic image captured by the FPD disposed at a second position different from the first position; and an ultrasonic image acquisition step of acquiring an ultrasonic image of the interior of the target blood vessel captured by an ultrasonic sensor, The first angiographic image comprises: the ultrasonic sensor disposed at a first marked position within a blood vessel of the subject; and a medical device different from the ultrasonic sensor and disposed at a second marked position in the subject's blood vessel, The second angiographic image includes the ultrasonic sensor disposed at the first marking position in the target blood vessel, The ultrasonic image is an image captured when the ultrasonic sensor is arranged at the first marking position. The ultrasonic image includes the target blood vessel and the medical device disposed at the second marked position in the target blood vessel, In the true lumen information acquisition step, three-dimensional position information of the true lumen is acquired using the position information of the first position, the first angiographic image, the position information of the second position, the second angiographic image, and the ultrasonic image.

17. The computer program according to claim 16, characterized in that In the true cavity information acquisition step, the following processing is performed: acquiring the position of the ultrasonic sensor using the image of the ultrasonic sensor included in the first angiographic image and the second angiographic image; using the image of the medical device included in the first angiographic image and the ultrasonic image, associating the positional relationship between the first angiographic image and the ultrasonic image; Acquire the position information of the true cavity from the ultrasonic image; and Three-dimensional position information of the true lumen is acquired using the acquired position of the ultrasonic sensor and the position information of the true lumen in the ultrasonic image obtained by the ultrasonic sensor.

18. The computer program according to any one of claims 13 to 15, characterized in that The computer program further comprises an angiographic image acquisition step, in which a first angiographic image captured by the FPD disposed at a first position and a second angiographic image captured by the FPD disposed at a second position different from the first position are acquired. The first angiographic image includes the true lumen of the target blood vessel and a medical device disposed at a first marked position in the target blood vessel, the second angiographic image includes the true lumen of the target blood vessel and the medical device disposed at the first marked position in the target blood vessel, In the true lumen information acquisition step, three-dimensional position information of the true lumen is acquired using the image of the medical device and the image of the true lumen included in the first angiographic image and the second angiographic image.

Images