Surgical operation support device, surgical operation support method and computer program

Abstract

To display an image of a true lumen of a blood vessel of an image of an FPD (angiography image).SOLUTION: A surgical operation support device comprises: a true lumen information acquisition unit which acquires three-dimensional position information of a true lumen existing in an object blood vessel; a true lumen image generation unit which acquires an angiography image captured for the object blood vessel from an FPD (flat panel detector) arranged at a first imaging position and generates a true lumen image expressing a true lumen of a position and posture corresponding to the angiography image by using the position information of the first imaging position and the three-dimensional position information of the true lumen; and an image synthesis unit which generates a synthetic image obtained by synthesizing the angiography image with the true lumen image to output the synthetic image.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a technique for assisting surgery. [Background technology]

[0002] In recent years, IVUS (IntraVascular Ultrasound) and FPD (Flat Panel Detector) have been used in vascular imaging for examinations and treatments. IVUS is a device with an ultra-small ultrasound transducer at its tip that acquires ultrasound images of the inside of blood vessels. FPD is a device with an X-ray tube and an X-ray planar detector that acquires X-ray images of blood vessels. X-ray images acquired by FPD are also called "angiographic images." For example, Patent Document 1 describes an apparatus that aligns a two-dimensional X-ray image of a region of interest with three-dimensional ultrasound data to determine the spatial relationship between a part of an interventional device and a target location and displays the spatial relationship. For example, Patent Document 2 describes an X-ray diagnostic apparatus that generates a diagram showing angle information of an FPD arm. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 6770655 [Patent Document 2] Japanese Patent Application Laid-Open No. 2013-233413 Summary of the Invention [Problem to be solved by the invention]

[0004] In some cases, the blood vessel may become blocked by an obstruction, such as a chronic total occlusion (CTO). In such cases, the blood vessel is recanalized by removing the obstruction or by placing a stent laterally. In such procedures, it is important for the surgeon to accurately determine the location of the true lumen within the blood vessel, from the perspectives of improving the accuracy of the procedure, shortening the procedure time, and reducing the burden on the patient. In a CTO-affected blood vessel, the true lumen may not be visible on an angiogram because the contrast agent does not flow into the target true lumen. Even if the contrast agent does flow into the target true lumen, the true lumen appears on an angiogram only for a moment while the contrast agent is passing through. Therefore, multiple injections of contrast agent to confirm the true lumen location increase the burden on the patient, which is undesirable.

[0005] In this regard, the device described in Patent Document 1 is capable of navigating an interventional device to a target location (a specific point of interest), but has the problem that it is not possible to determine the target location if the surgeon is unable to grasp the location of the true lumen. Furthermore, while a wide variety of procedures (approaches including how to advance the device until the blood vessel is recanalized) should be adopted depending on individual cases, such as the state of the true lumen and the state of the obstruction, the device described in Patent Document 1 has the problem that it is difficult to apply such flexible procedures. Furthermore, the device described in Patent Document 2 does not take into consideration the problems that an image of the true lumen may not appear on an angiogram, and that it is undesirable to inject contrast agent multiple times to confirm the location of the true lumen.

[0006] The present invention has been made to solve at least part of the above-mentioned problems, and has an object to display an image of the true lumen of a blood vessel in an image (angiogram) of an FPD. [Means for solving the problem]

[0007] The present invention has been made to solve at least part of the above-mentioned problems, and can be realized in the following aspects.

[0008] (1) According to one aspect of the present invention, there is provided a surgery support device comprising: a true lumen information acquisition unit that acquires three-dimensional positional information of a true lumen present in a target blood vessel; a true lumen image generation unit that acquires an angioimage of the target blood vessel from an FPD (flat panel detector) positioned at a first imaging position and generates a true lumen image representing the true lumen at a position and orientation corresponding to the angioimage, using the positional information of the first imaging position and the three-dimensional positional information of the true lumen; and an image synthesis unit that generates a composite image by synthesizing the angioimage and the true lumen image and outputs the composite image.

[0009] According to this configuration, the true lumen image generating unit can generate a true lumen image showing the true lumen at a position and orientation corresponding to the angioimage using position information of the first imaging position where the angioimage was acquired and three-dimensional position information of the true lumen acquired by the true lumen information acquiring unit. That is, even when the contrast agent does not flow into the target true lumen or when no contrast agent is being injected, the true lumen image generating unit can generate a true lumen image showing an image of the true lumen based on the three-dimensional position information of the true lumen. Furthermore, the image combining unit generates a composite image by combining the angioimage at any first imaging position with the true lumen image showing an image of the true lumen and outputs the composite image, thereby displaying an image of the true lumen of the blood vessel on the FPD image (angioimage). Therefore, by checking the composite image, the surgeon can proceed with the procedure while confirming the positional relationship between the medical device on the angioimage and the true lumen on the true lumen image. As a result, the surgeon can correctly grasp the position of the true lumen within the target blood vessel, which improves the accuracy of the procedure, shortens the time required for the procedure, and reduces the burden on the patient.

[0010] (2) In the above-described surgical support device, the three-dimensional position information of the true lumen includes information regarding the width of the true lumen, and the true lumen image generation unit may generate a true lumen image representing the true lumen having a width according to the three-dimensional position information of the true lumen. With this configuration, the true lumen image generator generates a true lumen image that shows the true lumen having a width corresponding to the three-dimensional position information of the true lumen, so the surgeon can proceed with the procedure while checking the composite image, thereby further improving the accuracy of the procedure, shortening the time required for the procedure, and reducing the burden on the patient.

[0011] (3) In the above-described surgical support device, when the FPD is moved to a second imaging position different from the first imaging position and an image of the target blood vessel is taken from the FPD, the true lumen image generation unit may reacquire the angioimage taken at the second imaging position, and regenerate a true lumen image representing the true lumen at a position and orientation corresponding to the reacquired angioimage using position information of the second imaging position and three-dimensional position information of the true lumen, and the image synthesis unit may regenerate a composite image by synthesizing the reacquired angioimage and the regenerated true lumen image, and output the composite image. According to this configuration, when the FPD is moved to a second imaging position different from the first imaging position and imaging is performed using the FPD, the true lumen image generation unit regenerates a true lumen image corresponding to the angioimage at the second imaging position, and the image synthesis unit regenerates a composite image by synthesizing the reacquired angioimage and the regenerated true lumen image, and outputs the composite image. That is, the true lumen image generation unit and the image synthesis unit can display a composite image including the true lumen image after the movement, following the movement of the FPD imaging position. As a result, the convenience of the surgery support device can be improved, and the accuracy of the procedure can be further improved, the time required for the procedure can be shortened, and the burden on the patient can be reduced.

[0012] (4) The surgery assistance device of the above aspect further includes an angio-image acquisition unit that acquires a first angio-image captured by the FPD placed at a first position and a second angio-image captured by the FPD placed at a second position different from the first position, and an ultrasound image acquisition unit that acquires an ultrasound image of the inside of the target blood vessel captured by an ultrasound sensor, wherein the first angio-image includes the ultrasound sensor placed at a first landmark position in the target blood vessel and a medical device different from the ultrasound sensor that is placed at a second landmark position in the target blood vessel. In rare cases, the second angio-image may include the ultrasonic sensor placed at the first landmark position within the target blood vessel, the ultrasonic image may be an image captured with the ultrasonic sensor placed at the first landmark position, the ultrasonic image may include the target blood vessel and the medical device placed at the second landmark position within the target blood vessel, and the true lumen information acquisition unit may acquire three-dimensional position information of the true lumen using position information of the first position, the first angio-image, position information of the second position, the second angio-image, and the ultrasonic image. According to this configuration, the true lumen information acquisition unit can acquire three-dimensional position information of the true lumen using position information of the first position where the first angioimage was acquired, the first angioimage, position information of the second position where the second angioimage was acquired, the second angioimage, and the ultrasound image. Specifically, the true lumen information acquisition unit can acquire three-dimensional position information of the ultrasound sensor using position information of the first position and the first angioimage, and position information of the second position and the second angioimage. Then, the true lumen information acquisition unit can acquire three-dimensional position information of the true lumen using the three-dimensional position information of the ultrasound sensor, position information of the first position and the first angioimage, and the ultrasound image showing the true lumen of the target blood vessel.

[0013] (5) In the above-described surgical support device, the true lumen information acquisition unit may acquire the position of the ultrasonic sensor using an image of the ultrasonic sensor included in the first angio image and the second angio image, associate the positional relationship between the first angio image and the ultrasonic image using an image of the medical device included in the first angio image and the ultrasonic image, acquire positional information of the true lumen from the ultrasonic image, and acquire three-dimensional positional information of the true lumen using the acquired position of the ultrasonic sensor and the positional information of the true lumen in the ultrasonic image captured by the ultrasonic sensor. According to this configuration, the true lumen information acquisition unit can acquire three-dimensional position information of the ultrasonic sensor using the image of the ultrasonic sensor included in the first angioimage and the second angioimage. Also, the true lumen information acquisition unit can associate the positional relationship between the first angioimage and the ultrasonic image using the image of the medical device included in the first angioimage and the ultrasonic image, acquire position information of the true lumen from the ultrasonic image, and acquire three-dimensional position information of the true lumen using the acquired position of the ultrasonic sensor and the position information of the true lumen in the ultrasonic image captured by the ultrasonic sensor.

[0014] (6) The above-described surgical support device may further include an angio-image acquisition unit that acquires a first angio-image captured by the FPD placed at a first position and a second angio-image captured by the FPD placed at a second position different from the first position, wherein the first angio-image includes the true lumen of the target blood vessel and a medical device placed at a first landmark position within the target blood vessel, and the second angio-image includes the true lumen of the target blood vessel and the medical device placed at the first landmark position within the target blood vessel, and the true lumen information acquisition unit may acquire three-dimensional position information of the true lumen using an image of the medical device and an image of the true lumen included in the first angio-image and the second angio-image. According to this configuration, the true lumen information acquisition section can acquire three-dimensional position information of the true lumen using the image of the medical device and the image of the true lumen contained in the first angioimage and the second angioimage.

[0015] The present invention has been made to solve at least some of the problems described above, and can be realized in the following forms: For example, the present invention can be realized in the form of an information processing device that outputs a composite image, an information processing device that outputs a recommended FPD imaging position range together with the composite image, an FPD (flat panel detector) that outputs the composite image, an FPD that outputs a recommended FPD imaging position range together with the composite image, a system including these devices, a computer program that realizes the functions of these devices and systems, a server device for distributing the computer program, a non-transitory storage medium that stores the computer program, etc. [Brief explanation of the drawings]

[0016] [Figure 1] FIG. 1 is an explanatory diagram illustrating the configuration of a surgery assistance system. [Figure 2] FIG. 10 is a diagram illustrating the imaging position of the first FPD. [Figure 3] FIG. 1 is a diagram illustrating a target blood vessel and a device used. [Figure 4] 10 is a flowchart illustrating an example of a composite image output process. [Figure 5] 10 is a flowchart illustrating an example of a composite image output process. [Figure 6] 10A and 10B are diagrams illustrating screens used in the composite image output process. [Figure 7] FIG. 10 is a diagram illustrating steps S5 and S7 of the composite image output process. [Figure 8] FIG. 10 is a diagram illustrating step S8 of the composite image output process. [Figure 9] FIG. 10 is a diagram illustrating steps S12 to S17 of the composite image output process. [Figure 10] FIG. 10 is a diagram illustrating step S18 of the composite image output process. [Figure 11] FIG. 10 is a diagram illustrating calculation of the length of a vector. [Figure 12] FIG. 10 is a diagram illustrating steps S19 to S21 of the composite image output process. [Figure 13] FIG. 10 is a diagram showing an example of an ultrasound image displayed on the canvas after step S21 of the composite image output processing. [Figure 14] FIG. 10 is a diagram illustrating step S22 of the composite image output process. [Figure 15] FIG. 10 is a diagram illustrating step S23 of the composite image output process. [Figure 16] FIG. 10 is a diagram illustrating steps S24 to S28 of the composite image output process. [Figure 17] FIG. 10 is a diagram illustrating step S30 of the composite image output process. [Figure 18] FIG. 10 is a diagram illustrating step S30 of the composite image output process. [Figure 19] FIG. 10 is a diagram illustrating an example of a composite image. [Figure 20] FIG. 10 is a diagram illustrating a modified example of the true lumen image. [Figure 21] FIG. 10 is a diagram illustrating a modified example of the true lumen image. [Figure 22] FIG. 10 is an explanatory diagram illustrating the configuration of a surgery support system according to a second embodiment. [Figure 23] 10 is a flowchart illustrating an example of a composite image output process according to the second embodiment. [Figure 24] 10A to 10C are diagrams illustrating a composite image output process according to the second embodiment. [Figure 25] FIG. 10 is a diagram illustrating step S18A of the composite image output process according to the second embodiment. [Figure 26] FIG. 10 is an explanatory diagram illustrating the configuration of a surgery support system according to a third embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0017] First Embodiment FIG. 1 is an explanatory diagram illustrating the configuration of a surgery support system 1. The surgery support system 1 is a system for supporting examinations and treatments. The surgery support system 1 includes a surgery support device 10, a blood vessel imaging device 20 having an FPD (Flat Panel Detector), a display device 30, a table 40, and an operation unit 50. By including the surgery support device 10 described below, the surgery support system 1 of this embodiment can generate a true lumen image representing the true lumen at a position and orientation corresponding to an image (hereinafter referred to as an "angiogram") captured of a target blood vessel by an FPD, and display a composite image combining the angioimage and the true lumen image. Note that the "target blood vessel" refers to a blood vessel that is the subject of examination or treatment. However, the surgery support system 1 is not limited to the vascular system, and may also be used in biological lumens such as the lymphatic system, biliary system, urinary system, respiratory system, digestive system, secretory glands, and reproductive organs.

[0018] FIG. 1 illustrates mutually orthogonal X, Y, and Z axes. The X axis corresponds to the width direction of the blood vessel imaging device 20, the Y axis corresponds to the height direction of the blood vessel imaging device 20, and the Z axis corresponds to the depth direction of the blood vessel imaging device 20. In the following explanation, the direction in which the head 92 of the patient (human body 90 in FIG. 1) is located will be simply referred to as the "Z-axis direction" and represented simply as "Z." The three-dimensional space formed by three-dimensional coordinates (XYZ coordinates) consisting of these X, Y, and Z axes will be referred to as the XYZ three-dimensional space. Note that the origin O of the XYZ three-dimensional space (XYZ coordinates) is set to the position of the heart 91 of the human body 90.

[0019] In a composite image output process described below, the surgery support device 10 generates a true lumen image showing the true lumen at a position and orientation corresponding to the angioimage captured by the FPD, and outputs a composite image obtained by combining the angioimage and the true lumen image. The surgery support device 10 includes a CPU, ROM, and RAM, and the CPU executes a computer program stored in the ROM to realize the functions of a main control unit 11, an angioimage acquisition unit 12, an ultrasound image acquisition unit 13, a true lumen information acquisition unit 14, a true lumen image generation unit 15, and an image synthesis unit 16. The surgery support device 10 is electrically connected to the control unit 29, display device 30, and operation unit 50 of the vascular imaging device 20.

[0020] The main control unit 11 transmits and receives information to and from the control unit 29 of the blood vessel imaging device 20, the display device 30, and the operation unit 50, and also controls the entire surgery assistance device 10. The main control unit 11 also controls the entire composite image output process, which will be described later.

[0021] In the composite image output process, the angio-image acquisition unit 12 acquires a first angio-image and a second angio-image from the blood vessel imaging device 20. The "first angio-image" is an angio-image acquired by placing the FPD at an arbitrary imaging position. The imaging position of the FPD when the first image is acquired is also referred to as the "first position." The "second angio-image" is an angio-image acquired by placing the FPD at an arbitrary imaging position different from the first position. The imaging position of the FPD when the second image is acquired is also referred to as the "second position." The first and second angio-images and the first and second positions will be described in detail later. The process (step) executed by the angio-image acquisition unit 12 is also referred to as an angio-image acquisition process (step).

[0022] In the composite image output process, the ultrasound image acquisition unit 13 acquires, from the imaging sensor 300 (FIG. 3), an ultrasound image of the inside of the target blood vessel captured by the imaging sensor 300. Details will be described later. The process (step) executed by the ultrasound image acquisition unit 13 is also referred to as an ultrasound image acquisition process (step).

[0023] In the composite image output process, the true lumen information acquisition unit 14 acquires three-dimensional position information (position information in XYZ three-dimensional space) of the true lumen present in the target blood vessel using position information of the first position, the first angioimage, position information of the second position, the second angioimage, and the ultrasound image. Details will be described later. The process (step) executed by the true lumen information acquisition unit 14 is also referred to as a true lumen information acquisition process (step).

[0024] In the composite image output process, the true lumen image generating unit 15 generates a true lumen image representing the true lumen at a position and orientation corresponding to an angioimage captured by an FPD placed at an arbitrary imaging position (hereinafter also referred to as the "first imaging position"). Furthermore, when the FPD is moved to an arbitrary imaging position (hereinafter also referred to as the "second imaging position") different from the first imaging position and imaging is performed by the FPD, the true lumen image generating unit 15 regenerates a true lumen image representing the true lumen at a position and orientation corresponding to the angioimage captured at the second imaging position. Here, the first imaging position and the second imaging position refer to arbitrary positions different from the first position and the second position described above, i.e., arbitrary positions at which the surgeon intends to confirm the target blood vessel and device. However, the first imaging position and the second imaging position may be the same as the first position and the second position described above. Details will be described later. The process (step) executed by the true lumen image generating unit 15 is also referred to as a true lumen image generating process (step).

[0025] In the composite image output process, the image synthesis unit 16 generates a composite image by synthesizing an angioimage captured by the FPD placed at the first imaging position with the true lumen image generated by the true lumen image generation unit 15, and displays the generated composite image on the display device 30. The image synthesis unit 16 also regenerates a composite image by synthesizing an angioimage captured by the FPD placed at the second imaging position with the true lumen image regenerated by the true lumen image generation unit 15, and displays the generated composite image on the display device 30. Details will be described later. The process (step) executed by the image synthesis unit 16 is also referred to as an image synthesis process (step).

[0026] The blood vessel imaging device 20 has an FPD and acquires an image (angiogram) by acquiring X-rays that have passed through the human body and converting them into digital signals. The blood vessel imaging device 20 has a first FPD 21, a first X-ray tube device 22, a first C-arm 23, a first support unit 24, a second FPD 25, a second X-ray tube device 26, a second C-arm 27, a second support unit 28, and a control unit 29.

[0027] The first FPD 21 includes an X-ray flat panel detector, converts X-rays received from the first X-ray tube device 22 into electrical signals, and performs A / D (analog / digital) conversion to generate an X-ray image. The first X-ray tube device 22 receives a high voltage output from an X-ray high voltage device (not shown) and emits an X-ray beam. As indicated by the thick dashed line extending in the Y-axis direction in FIG. 1, the X-ray beam emitted from the first X-ray tube device 22 passes through a human body 90 and enters the first FPD 21. The first C-arm 23 is a C-shaped arm (support) that fixes the first FPD 21 and the first X-ray tube device 22 in opposing positions. The first support unit 24 rotatably supports the first C-arm 23. That is, the first FPD 21 and the first X-ray tube device 22 can be moved to any imaging position around the human body 90 lying on the bed 41 while being fixed at opposing positions by the first C-arm 23. Hereinafter, the first FPD 21 and the first X-ray tube device 22 fixed to the first C-arm 23 will also be simply referred to as the "first FPD 21."

[0028] The second FPD 25 has a configuration similar to that of the first FPD 21. The second X-ray tube device 26 has a configuration similar to that of the first X-ray tube device 22. As indicated by the thick dashed line extending in the X-axis direction in FIG. 1 , the X-ray beam emitted from the second X-ray tube device 26 enters the second FPD 25 through the human body 90. The second C-arm 27 is a C-shaped arm (support) that fixes the second FPD 25 and the second X-ray tube device 26 in opposing positions. The second support unit 28 rotatably supports the second C-arm 27. That is, the second FPD 25 and the second X-ray tube device 26 can move to any imaging position around the human body 90 while being fixed in opposing positions by the second C-arm 27. Hereinafter, the second FPD 25 and the second X-ray tube device 26 fixed to the second C-arm 27 will also be simply referred to as the "second FPD 25."

[0029] The second FPD 25 is generally disposed in the normal direction of the first FPD 21. For example, as shown in Fig. 1, when the first FPD 21 is positioned to image in the front direction of the human body 90 (the vertical direction of the human body 90, the longitudinal direction of the human body 90), the second FPD 25 is positioned to image in the horizontal direction of the human body 90 (the lateral direction of the human body 90). The blood vessel imaging device 20 may also be simply referred to as an "FPD" or "FPD device."

[0030] The control unit 29 includes a CPU, a ROM, and a RAM. The CPU executes a computer program stored in the ROM to control the entire vascular imaging device 20. The control unit 29 is electrically connected to the first FPD 21, the second FPD 25, the first support unit 24, the second support unit 28, the display device 30, the table 40, and the operation unit 50. The control unit 29 displays X-ray images generated by the first FPD 21 and the second FPD 25 on the display device 30. In addition, in accordance with an operation from the operation unit 50, the control unit 29 drives the first support unit 24 to rotate the first C-arm 23 and drives the second support unit 28 to rotate the second C-arm 27. In addition, in accordance with an operation from the operation unit 50, the control unit 29 extends and retracts the extension unit 42 to change the height of the bed 41 and moves the table 40 in the Z-axis direction to change the position of the bed 41.

[0031] The display device 30 is connected to the surgery assistance device 10 and the control unit 29 of the blood vessel imaging device 20, and functions as an output interface for the surgery assistance device 10 and the blood vessel imaging device 20. The display device 30 has a monitor 31 and an arm 32. The monitor 31 is a "display unit" configured by well-known means such as a liquid crystal display, smart glasses, or a projector. The arm 32 supports and fixes the monitor 31.

[0032] The table 40 is a platform on which the human body 90 lies and is positioned near the first FPD 21 and the second FPD 25. The table 40 has a bed 41, an extension unit 42, and legs 43. The bed 41 is equipped with a mattress on which the human body 90 lies. The bed 41 is supported by the table 40 so as to be movable in the Z-axis direction. The extension unit 42 is configured to extend and retract in the Y-axis direction, thereby changing the height of the bed 41. The legs 43 support the bed 41 and the extension unit 42. As shown by the dashed lines in FIG. 1 , the human body 90 lies face up on the bed 41 with its head 92 placed closer to the first FPD 21 and the second FPD 25 and its feet 93 placed farther from the first FPD 21 and the second FPD 25. This makes it easier to acquire images of the target blood vessels in the heart 91 using the first FPD 21 and the second FPD 25.

[0033] The operation unit 50 is connected to the surgery assistance device 10 and the control unit 29 of the vascular imaging device 20, and functions as an input interface for the surgery assistance device 10 and the vascular imaging device 20. The operation unit 50 is an "input unit" configured with well-known means such as a touch panel, operation buttons, operation levers, operation switches, a keyboard, a mouse, a voice input unit, a foot switch, etc. In the illustrated example, the operation unit 50 is fixed to the table 40.

[0034] FIG. 2 is a diagram illustrating the imaging position of the first FPD 21. FIG. 2(A) is a diagram illustrating LAO, and FIG. 2(B) is a diagram illustrating RAO. As shown in FIG. 2(A), the case where the first FPD 21 is positioned on the left side of the human body 90 is called LAO (Left Anterior Oblique view). As shown in FIG. 2(B), the case where the first FPD 21 is positioned on the right side of the human body 90 is called RAO (Right Anterior Oblique view). FIG. 2(C) is a diagram illustrating CRA, and FIG. 2(D) is a diagram illustrating CAU. As shown in FIG. 2(C), the case where the first FPD 21 is positioned on the upper side of the human body 90 is called CRA (CRAnial). As shown in FIG. 2(D), the case where the first FPD 21 is positioned on the lower side of the human body 90 is called CAU (CAUdal). That is, the "imaging position of the first FPD 21" is specified by a combination of the left-right position (A1) and the up-down position (A2) shown below. (A1) LAO or RAO and the angle θ1 from the center O of the human body 90, (A2) CRA or CAU and the angle θ2 from the center O of the human body 90. Here, the center O of the human body 90 is the position of the heart 91 of the human body 90 (the position of the origin O in the XYZ three-dimensional space). For example, "the imaging position of the first FPD 21 is (RAO28 CRA5)" means that the first FPD 21 is at a position 28 degrees to the right of the human body 90 and at a position 5 degrees above the human body 90.

[0035] Figure 3 is a diagram illustrating a target blood vessel 100 and the device used. Figure 3(A) is a diagram showing a longitudinal section of the target blood vessel 100, and Figure 3(B) is a diagram showing a portion of the target blood vessel 100 surrounded by a dashed rectangle in Figure 3(A) as viewed from above. Figure 3(A) shows the target blood vessel 100, a CTO 101 that has occurred in the target blood vessel 100, a false lumen 102 formed in the intima or subintima of the target blood vessel 100, and a true lumen 103. Note that the false lumen 102 refers to all dissected cavities other than the true lumen 103 formed by the medical device.

[0036] Here, as shown in FIG. 3(B), the true lumen 103 does not necessarily extend linearly, but may meander. For example, in the example shown in FIG. 3(B), in order to guide the guidewire 500 into the true lumen 103, the surgeon needs to point the distal end of the guidewire 500 downward in the plane of the drawing. However, since an FPD can only obtain a fluoroscopic image of a certain cross section, depending on the imaging position of the FPD, it may not be possible to capture a portion of the meandering true lumen 103 in the angiographic image, as shown in FIG. 3(A). In this regard, the surgery support device 10 of this embodiment can solve this problem by generating and displaying a true lumen image representing an image of the true lumen by a composite image output process described below.

[0037] The surgery assistance system 1 uses an imaging sensor 300 and a guidewire 500 shown in FIG. 3(A). The imaging sensor 300 is an ultrasonic sensor that acquires ultrasonic images of the inside of the target blood vessel 100. The imaging sensor 300 has an elongated outer shape and has a transducer 301 at its tip. The transducer 301 is an ultrasonic probe (also called an ultrasonic vibrator, piezoelectric element, ultrasonic transmitting / receiving element, or ultrasonic element) that emits ultrasonic waves toward biological tissue and receives the ultrasonic waves that propagate through the biological tissue and are reflected. The imaging sensor 300 acquires ultrasonic images of the inside of the target blood vessel 100 around the transducer 301 while moving back and forth within the lumen of the sensor catheter 200. Specifically, the transducer 301 rotates around the central axis of the transducer 301 (hereinafter also referred to as the transducer axis) extending in the longitudinal direction of the image sensor 300, and acquires ultrasound images of the inside of the target blood vessel 100 in a direction perpendicular to the transducer axis (in the 360° circumferential direction of the transducer axis).

[0038] The guidewire 500 is a medical device having an elongated outer shape. The guidewire 500 may be a plasma guidewire that has an electrode at its tip and ablates biological tissue using a plasma flow. In this case, the wire catheter 400 may be configured to have the other electrode at its tip. The guidewire 500 may also be a penetration guidewire that has a pointed portion at its tip and penetrates biological tissue using the pointed portion, or may be a delivery guidewire that does not have a pointed portion. The guidewire 500 is housed within the lumen of the wire catheter 400, and the tip of the guidewire 500 protrudes to the outside from the tip portion 401 of the wire catheter 400.

[0039] 4 and 5 are flowcharts showing an example of composite image output processing. The composite image output processing can be started at any time, such as when the surgery support device 10 is powered on, a predetermined application is started, or the vascular imaging device 20 is powered on. In the following description, an example is given of an angio-image being obtained using the first FPD 21. However, angio-images may also be obtained using the second FPD 25. In this case, the "first FPD 21" in the following description may be replaced with the "second FPD 25." Note that, for convenience of illustration, in the flowcharts of FIGS. 4 and 5, the imaging sensor 300 is simply referred to as the "sensor," and the angio-image is simply referred to as the "image."

[0040] FIG. 6 is a diagram illustrating a screen used in the composite image output process. FIG. 6(A) is a diagram illustrating the configuration of the operation screen OS, and FIG. 6(B) is a diagram illustrating an example of a first angio-image V1. In the composite image output process described below, the true lumen information acquisition unit 14 proceeds with the process while providing operation guidance to the surgeon. In this embodiment, the true lumen information acquisition unit 14 displays an operation screen OS as shown in FIG. 6(A) on the display device 30 and displays various operation guidance using this operation screen OS. The operation screen OS has an operation button display area A1 in which buttons for various operations are arranged, a canvas A2, and a guidance display area A3 in which various guidance messages are displayed. The canvas A2 is an area for displaying angio-images sequentially acquired by the first FPD 21 and ultrasound images acquired by the imaging sensor 300. In the composite image output process, the true lumen information acquisition unit 14 proceeds with each step while updating the angio-image displayed on the canvas A2 of the operation screen OS.

[0041] The operation screen OS is merely an example, and various modifications are possible. For example, the guidance display area A3 of the operation screen OS may be omitted, and guidance may be provided by voice. For example, the guidance display area A3 of the operation screen OS may be omitted, and a button with an item name (for example, a button labeled "Place first landmark" in the case of step S5) may be placed in the operation button display area A1 to provide guidance instead.

[0042] In step S1, the true lumen information acquisition unit 14 provides guidance to prepare for imaging using the first FPD 21. Following the guidance, the surgeon prepares for imaging using the first FPD 21. Specifically, as shown in FIG. 1, the surgeon lies the human body 90 on the bed 41 and turns on the power of the blood vessel imaging device 20.

[0043] In step S2, the true lumen information acquisition unit 14 provides guidance to prepare for imaging by the imaging sensor 300. Following the guidance, the surgeon prepares for imaging by the imaging sensor 300. Specifically, as shown in FIG. 3(A), the imaging sensor 300 and the guide wire 500 are inserted into a blood vessel in the human body 90, and the guide wire 500 is delivered so that the transducer 301 of the imaging sensor 300 and the distal end of the guide wire 500 are positioned near the CTO 101 in the target blood vessel 100.

[0044] In step S3, the true lumen information acquisition unit 14 guides the surgeon to move the first FPD 21 to a first position and capture an X-ray image. Following the guidance, the surgeon moves the first FPD 21 to the first position and captures an X-ray image of the target blood vessel 100, thereby acquiring a first angio image V1. The first position can be any position (RAO XX CRA XX: X is any natural number). In step S3, the true lumen information acquisition unit 14 may automatically move the first FPD 21 to the first position and perform imaging. The angio image acquisition unit 12 acquires the captured first angio image V1 from the blood vessel imaging device 20. As shown in FIG. 6(B), the first angio image V1 includes an image of the imaging sensor 300, an image of the guidewire 500, and an image (not shown) of the target blood vessel 100 through which the imaging sensor 300 and the guidewire 500 are inserted. In FIG. 6(B), for the sake of distinction, the imaging sensor 300 is shown with a dashed line and the guidewire 500 is shown with a solid line. In the example of this embodiment, the first angio image V1 includes an image of the target blood vessel 100. However, it is sufficient for the first angio image V1 to include an image of the imaging sensor 300 and an image of the guidewire 500, and it is not necessary for the first angio image V1 to include an image of the target blood vessel 100. Note that the "image of the target blood vessel 100" means an image of the contour of the target blood vessel 100.

[0045] In step S4, the true lumen information acquisition unit 14 displays the first angio image V1 on the canvas A2 and adjusts the position of the first angio image V1 so that the image of the transducer 301 (see Figure 3) among the images of the imaging sensor 300 is located at the center of the canvas A2.

[0046] FIG. 7 is a diagram illustrating steps S5 and S7 of the composite image output process. FIG. 7(A) is a diagram illustrating the state of the canvas A2 in step S5, and FIG. 7(B) is a diagram illustrating the state of the canvas A2 in step S7. As shown in FIG. 7, the square or rectangular canvas A2 has an XcYc coordinate system, which is a two-dimensional coordinate system consisting of an Xc axis extending to the right of the page and a Yc axis extending downward on the page, with the origin Oc at the vertex at the top left of the page. The two-dimensional space formed by the XcYc coordinate system is called an XcYc two-dimensional space. In the XcYc two-dimensional space, the negative direction of the Yc axis (upward on the page) corresponds to the direction in which the head 92 of the human body 90 (see FIG. 1) is located. In step S5, the true lumen information acquisition unit 14 guides the user to place a first mark a1 on the image of the transducer 301 of the imaging sensor 300 in the first angioimage V1 displayed on the canvas A2. Following the guidance, the surgeon places the first mark a1 on the first angio image V1 on the canvas A2, as shown in Figure 7(A). The placement of the first mark a1 can be achieved, for example, by clicking or tapping on the intended position on the image of the canvas A2. Note that when the image of the transducer 301 on the first angio image V1 is at the first mark a1 in the XcYc coordinates on the canvas A2, the actual position of the transducer 301 in the target blood vessel 100 (position in the XYZ coordinates) is defined as P1.

[0047] In step S6, the true lumen information acquisition unit 14 provides guidance to acquire an ultrasound image IV1 using the imaging sensor 300 while maintaining the position of the imaging sensor 300. Following the guidance, the surgeon acquires the ultrasound image IV1 from the imaging sensor 300 without moving the imaging sensor 300 from the position shown in FIG. 7(A). The ultrasound image acquisition unit 13 acquires the captured ultrasound image IV1 from the imaging sensor 300 and stores the ultrasound image IV1 in a storage unit inside the surgery assistance device 10. That is, the ultrasound image acquisition unit 13 acquires the ultrasound image IV1 when the transducer 301 is located at P1 within the target blood vessel 100.

[0048] That is, the first angio-image V1 acquired in step S3 includes the target blood vessel 100, an image of the imaging sensor 300 placed at a first landmark position (first landmark a1) in the target blood vessel 100, and the guidewire 500 placed at a second landmark position (any position different from the first landmark a1) in the target blood vessel 100. Furthermore, the ultrasound image IV1 acquired in step S6 includes the target blood vessel 100 and the guidewire 500 placed at a second landmark position (any position different from the first landmark a1) in the target blood vessel 100.

[0049] In step S7, the true lumen information acquisition unit 14 guides the surgeon to move the imaging sensor 300 within a range that can be considered a straight line and then place a second mark a2 on the transducer 301. Here, "straight line" means that the trajectory of the transducer 301 is a straight line when moved inside the sensor catheter 200. Following the guidance, the surgeon moves the imaging sensor 300 within a range (distance) that can be considered a straight line, as shown in FIG. 7(B), and then places the second mark a2 on the first angio image V1 on the canvas A2. Note that when the image of the transducer 301 on the first angio image V1 is at the second mark a2 in the XcYc coordinates on the canvas A2, the actual position (position in the XYZ coordinates) of the transducer 301 within the target blood vessel 100 is defined as Pe.

[0050] Fig. 8 is a diagram illustrating step S8 of the composite image output process. Figs. 8(A) and 8(B) are diagrams illustrating calculation of BNV, which will be described later. Fig. 8(C) is a diagram illustrating the relationship between a first angio-image V1 at a first position and a second angio-image V2 at a second position, which will be described later. Fig. 8(D) is a diagram illustrating the first angio-image V1 at the first position. Fig. 8(E) is a diagram illustrating the second angio-image V2 at the second position.

[0051] In step S8, the true lumen information acquisition unit 14 calculates the BNV using the first mark a1, the second mark a2, and the first position in the first angioimage V1 on the canvas A2. Specifically, as shown in Fig. 8(A), the true lumen information acquisition unit 14 defines a vector having the first mark a1 as its start point and the second mark a2 as its end point (a vector representing the trajectory of the transducer 301 that can be regarded as a straight line) as the first shaft axis vector Ie' of the imaging sensor 300, defines a vector representing a first view, which is the imaging direction of the first FPD 21 arranged at the first position with respect to the heart 91 (see Fig. 1), as the first view vector Vw1, and calculates the BNV for a plane W including the first shaft axis vector Ie' and the first view vector Vw1. Furthermore, since the plane W is a plane including the first shaft axis vector Ie', it can be said that it is a plane on which the trajectory of the transducer 301 from P1 (corresponding to the first mark a1) to Pe (corresponding to the second mark a2) in the XYZ three-dimensional space is located. Here, "BNV" ​​means an imaging direction perpendicular to the plane W, as shown in FIG. 8(A). That is, BNV means an imaging direction that can acquire a second angio-image V2 perpendicular to the first angio-image V1. The position of the first FPD 21 whose imaging direction is BNV is defined as a "second position," and the imaging direction (=BNV) of the first FPD 21 arranged at the second position with respect to the heart 91 is defined as a second view. A vector representing the second view is defined as a second view vector Vw2.

[0052] The true lumen information acquisition unit 14 acquires position information of the first FPD 21 at the first position (also simply referred to as "position information at the first position"), which is LAO or a numerical value RL representing RAO. val and a numeric value CC representing CRA or CAU. valThen, on canvas A2, the inclination δ (see FIG. 8B) of the first shaft axis vector Ie′ with respect to the Yc′ axis parallel to the Yc axis is substituted into the following equation (1) to calculate ψ, θ, and δ. In equation (1), “ψ” and “θ” are variables for expressing the first view vector Vw1 in polar coordinates. “CW” means “clockwise” and “CCW” means “counterclockwise.” Next, the true lumen information acquisition unit 14 substitutes the calculated ψ, θ, and δ into equation (2) to calculate the Cartesian coordinates of the second view vector Vw2. Next, the true lumen information acquisition unit 14 converts the Cartesian coordinates (x, y, z) of the second view vector Vw2 into polar coordinates (r, θ, ψ) using equation (3). Next, the true lumen information acquisition unit 14 calculates a numerical value RL representing the LAO or RAO at the second position from the polar coordinates (r, θ, ψ) of the second view vector Vw2. val and a numeric value CC representing CRA or CAU. val Calculate.

[0053]

number

number

number

[0054] Consider a case where the imaging sensor 300 is positioned within the target blood vessel 100 as shown in FIG. 8(C). In this case, in step S7, the imaging sensor 300 is advanced within a range that can be regarded as a straight line. Therefore, the first angio image V1 can be said to be an image acquired from a direction in which the imaging sensor 300 appears straight, i.e., from the first view at the first position. Therefore, the imaging sensor 300 is captured in a straight line in the first angio image V1, as shown by the hatched images in FIGS. 8(C) and (D). On the other hand, the second view (BNV) at the second position is perpendicular to the first view. Therefore, the second angio image V2 obtained from the second view at the second position is perpendicular to the first angio image V1, and can be said to be an image acquired from a direction in which the imaging sensor 300 (the trajectory of the transducer 301) appears curved. Therefore, in the second angioimage V2, the imaging sensor 300 is captured in a curved shape, as shown by the dot-hatched images in FIGS. 8(C) and 8(E).

[0055] In step S9, the true lumen information acquisition unit 14 provides guidance to return the transducer 301 of the imaging sensor 300 to the position of the first mark a1. Following the guidance, the surgeon returns the transducer 301 to the position of the first mark a1. That is, the surgeon returns the transducer 301 in the target blood vessel 100 from position Pe to P1 in the XYZ coordinate system.

[0056] In step S10, the true lumen information acquisition unit 14 guides the surgeon to move the first FPD 21 to a second position corresponding to the BNV calculated in step S8 and capture an X-ray image. Following the guidance, the surgeon moves the first FPD 21 to the second position and captures an X-ray image of the target blood vessel 100, thereby acquiring a second angio-image V2. In step S10, the true lumen information acquisition unit 14 may automatically move the first FPD 21 to the second position and perform imaging. The angio-image acquisition unit 12 acquires the captured second angio-image V2 from the blood vessel imaging device 20. As will be described later with reference to FIG. 9 , the second angio-image V2 includes an image of the imaging sensor 300, an image of the guidewire 500, and an image (not shown) of the target blood vessel 100 through which the imaging sensor 300 and the guidewire 500 are inserted, all captured from a different direction from the first angio-image V1. In the example of this embodiment, the second angio-image V2 includes an image of the target blood vessel 100. However, it is sufficient that the second angio-image V2 contains an image of the imaging sensor 300, and it is not necessary that the image of the target blood vessel 100 or the image of the guidewire 500 is contained.

[0057] In step S11, the true lumen information acquisition unit 14 displays the second angio image V2 on the canvas A2 and adjusts the position of the second angio image V2 so that the image of the transducer 301 of the imaging sensor 300 is positioned at the center of the canvas A2.

[0058] FIG. 9 is a diagram illustrating steps S12 to S17 of the composite image output process. FIG. 9 shows an example of a second angioimage V2 on the canvas A2. In step S12, the true lumen information acquisition unit 14 guides the surgeon to place a first mark b1 on the image of the transducer 301 of the imaging sensor 300 in the second angioimage V2 displayed on the canvas A2. Following the guidance, the surgeon places the first mark b1 on the second angioimage V2 on the canvas A2. The placement of the first mark b1 can be achieved, for example, by clicking or tapping the intended position on the image of the canvas A2. Note that because the transducer 301 is returned to the position of the first mark a1 in step S9, the actual position of the first mark b1 in the XYZ three-dimensional space is the same as the position of the first mark a1 placed in step S5 (FIG. 9: b1 = a1). That is, when the image of the transducer 301 on the second angio image V2 is at the first mark b1 of the XcYc coordinates on the canvas A2, the actual position (position in the XYZ coordinates) of the transducer 301 in the target blood vessel 100 is P1.

[0059] In step S13, the true lumen information acquisition unit 14 assigns 2 to the variable n used in the composite image output process. n is a natural number. In step S14, the true lumen information acquisition unit 14 guides the imaging sensor 300 to move an arbitrary distance and place a second mark b2 on the transducer 301. After following the guidance, the surgeon moves the imaging sensor 300 and then places the second mark b2 on the second angio image V2 on the canvas A2. Note that when the image of the transducer 301 on the second angio image V2 is located at the second mark b2 in the XcYc coordinates on the canvas A2, the actual position (position in the XYZ coordinates) of the transducer 301 within the target blood vessel 100 is designated as P2.

[0060] In step S15, the true lumen information acquisition unit 14 provides guidance to acquire an ultrasound image IV2 using the imaging sensor 300 while maintaining the position of the imaging sensor 300. Following the guidance, the surgeon acquires the ultrasound image IV2 from the imaging sensor 300 without moving the imaging sensor 300. The ultrasound image acquisition unit 13 acquires the captured ultrasound image IV2 from the imaging sensor 300 and stores the ultrasound image IV2 in a memory unit inside the surgery assistance device 10. That is, the ultrasound image acquisition unit 13 acquires the ultrasound image IV2 when the transducer 301 is positioned at P2 within the target blood vessel 100. In step S16, the true lumen information acquisition unit 14 adds 1 to the variable n.

[0061] In step S17, the true lumen information acquisition unit 14 determines whether the placement of landmarks on the second angioimage V2 (step S14) and the acquisition of ultrasound images at the locations of the landmarks (step S15) have been completed for the desired number of landmarks. The desired number of landmarks can be determined arbitrarily, and in the case of FIG. 9, n = 4. If the variable n has reached the desired number of landmarks (step S17: YES), the true lumen information acquisition unit 14 transitions the process to step S18. If the variable n has not yet reached the desired number of landmarks (step S17: NO), the true lumen information acquisition unit 14 transitions the process to step S14 and repeats the above-described process. As a result, as shown in FIG. 9, the variable n is incremented, such as n = 3, n = 4, and so on, and an n-th landmark bn is placed on the second angioimage V2 (step S14), and an ultrasound image IVn corresponding to the location of each n-th landmark bn is acquired (step S15). When the image of the transducer 301 on the second angio image V2 is at the n-th mark bn of the XcYc coordinates on the canvas A2, the actual position (position in the XYZ coordinates) of the transducer 301 in the target blood vessel 100 is defined as Pn. The range in which the imaging sensor 300 is advanced on the second angio image V2 is the range in which the imaging sensor 300 can be regarded as a straight line on the first angio image V1, that is, from the first mark a1 to the second mark a2 (in the target blood vessel 100, this is from position P1 to Pe).

[0062] 10A and 10B are diagrams illustrating step S18 of the composite image output process. Fig. 10A is a diagram showing points on a first angio image V1 displayed on a canvas A2, and Fig. 10B is a diagram showing points on a second angio image V2 displayed on a canvas A2. In Figs. 10A and 10B, the upper side of the paper faces the Z axis of the XYZ coordinate system (Fig. 1: the direction in which the head 92 of the human body 90 is located). That is, the negative direction of the Yc axis of the canvas A2 is the direction in which the head 92 of the human body 90 in Fig. 1 is located. In Figs. 10A and 10B, the Yc' axis is a straight line parallel to the Yc axis.

[0063] In step S18, the true lumen information acquisition unit 14 calculates the following (B1) and (B2) using the coordinates in the XcYc coordinate system of the first mark a1 and the second mark a2 in the first angio image V1 displayed on the canvas A2, and the coordinates in the XcYc coordinate system of the first mark b1 to the nth mark bn in the second angio image V2. (B1) Position vectors P2 to Pn of the transducer 301 in the XYZ three-dimensional space: The position vectors P2 to Pn of the transducer 301 are vectors that extend from a position P1 that is a reference point (starting point) within the target blood vessel 100 to the positions P2 to Pn, respectively. (B2) Transducer axis vectors T1 to Tn of the transducer 301 of the imaging sensor 300: The transducer axis vectors T1 to Tn are vectors of the transducer axis (the central axis of the transducer 301 extending in the longitudinal direction of the imaging sensor 300) when the transducer 301 is located at positions P1 to Pn within the target blood vessel 100. Here, the transducer axis vector when the transducer 301 is located at positions P1 to Pn can be said to be a tangent vector at positions P1 to Pn on the trajectory of the transducer 301.

[0064] 8, the first angio-image V1 and the second angio-image V2 displayed on the canvas A2 are images captured by the imaging sensor 300 from different angles, and therefore the first mark b1 to the n-th mark bn (n=4 in the illustrated example) in the second angio-image V2 shown in Fig. 10(B) can be said to be on the first shaft axis vector Ie' of the image sensor 300 of the first angio-image V1 shown in Fig. 10(A). Therefore, the direction of the "(B1) position vectors P2 to Pn of the transducer 301" can be calculated using the tilt δ of the first shaft axis vector Ie' with respect to the Yc axis (see Fig. 10(A)) and the tilt of the second shaft axis vectors P2' to Pn' of the transducer 301 on the second angio-image V2 with respect to the Yc axis. The first shaft axis vector Ie' is a vector extending from the first mark a1 to the second mark a2 on the first angio image V1. The second shaft axis vector P2' is a vector extending from the first mark b1 to the second mark b2 on the second angio image V2, and the second shaft axis vector Pn' is a vector extending from the first mark b1 to the n-th mark bn.

[0065] 10(A), the tangent vector T' on the trajectory of the transducer 301 that passes through the first landmark a1 and the second landmark a2 can be said to be on the first shaft axis vector Ie'. Therefore, the direction of the "(B2) transducer axis vectors T1 to Tn of the transducer 301 of the imaging sensor 300" can be calculated using the tilt δ of the shaft axis vector Ie' in the first angio image V1 with respect to the Yc axis and the tilts of the tangent vectors T1' to Tn' in the second angio image V2 with respect to the Yc axis. Note that the tangent vector T1' in the second angio image V2 is the tangent vector at the first landmark b1 on the trajectory of the transducer 301 that extends from the first landmark b1 to the nth landmark bn. Similarly, the tangent vector Tn' is the tangent vector at the nth landmark bn on the trajectory of the transducer 301.

[0066] Details of a method for using first and second angio images V1 and V2 taken at different first and second positions to determine the vector direction of an object located on a straight line where a plane defined by two vectors, a first view vector Vw1 representing the imaging direction of the first FPD21 at the first position and a first shaft axis vector Ie' representing the trajectory of the transducer 301 shown on the first angio image V1, intersects with a plane defined by a second view vector Vw2 representing the imaging direction of the first FPD21 at the second position and second shaft axis vectors P2' to Pn' of the transducer 301 shown on the second angio image V2, are disclosed in International Application PCT / JP2021 / 034980. In International Application PCT / JP2021 / 034980, a blood vessel existence plane H2 seen from a first position and a blood vessel existence plane S seen from a second position are used to define a line where the H2 plane and the S plane intersect, and a "vascular axis vector" is calculated as the line. In the example of this embodiment, for example, a plane defined by two vectors, a first view vector Vw1 representing the imaging direction of the first FPD 21 at the first position and a first shaft axis vector Ie' representing the trajectory of the transducer 301 shown on the first angio image V1, corresponds to the H2 plane, a plane defined by a second view vector Vw2 representing the imaging direction of the first FPD 21 at the second position and second shaft axis vectors P2' to Pn' shown on the second angio image V2 corresponds to the S plane, and the position vectors P2 to Pn calculated in (B1) above may be calculated as equivalents to the blood vessel axis vector. Moreover, the transducer axis vectors T1 to Tn calculated in (B2) above can also be calculated in the same way as they correspond to the blood vessel axis vectors.

[0067] 11 is a diagram for explaining calculation of the lengths of the position vectors P2 to Pn of the transducer 301. Specifically, FIG. 11 is a diagram for explaining, as an example, calculation of the length of the position vector P2 of the transducer 301 when the transducer 301 moves from the start point (reference point) P1 to the end point P2 within the target blood vessel 100. The length of the position vector Pn can be calculated in a similar manner. In FIG. 11, vector Vw2 is the second view vector Vw2 (see FIG. 8(A)) that represents the imaging direction when the first FPD 21 is at the second position. Furthermore, b1 is the first landmark b1 on the second angio image V2, and is the position of the transducer 301 on the second angio image V2 when the transducer 301 is located at the start point P1. Similarly, b2 is the second landmark b2 on the second angio image V2, and is the position of the transducer 301 on the second angio image V2 when the transducer 301 is located at the end point P2. Vector P2' is the second shaft axis vector P2' of the transducer 301 on the second angio image V2, and is also the orthogonal projection vector of the position vector P2 of the transducer 301 onto the second angio image V2. Furthermore, θ is the angle formed between the second view vector Vw2 and the position vector P2. In step S18, the true lumen information acquisition unit 14 calculates the length of the above-mentioned "(B1) position vectors P2 to Pn of the transducer 301" using the following equation (4). If the second view vector Vw2 and the position vector P2 are unit vectors, θ can be calculated using the first equation of equation (4) based on the formula for the dot product of vectors. Furthermore, since the length of the second shaft axis vector P2' can be calculated from the coordinates of b1 and b2 in the XcYc coordinate system of canvas A2, the length of the position vector P2 can be derived from the second equation of equation (4). Note that in the first equation of equation (4), "Vw2 × P2" represents the dot product of the second view vector Vw2 and the position vector P2, and in the second equation of equation (4), "P2" and "P2'" represent the lengths of the position vector P2 and the second shaft axis vector P2', respectively.

[0068]

number

[0069] 12A and 12B are diagrams illustrating steps S19 to S21 of the composite image output process. Fig. 12A is a diagram illustrating an example of an angio-image Vα when the first FPD 21 is placed at position α. ​​Fig. 12B is a diagram illustrating the relationship between the first FPD 21, the imaging sensor 300, and the guidewire 500 when the first FPD 21 is placed at position α.

[0070] In step S19, the true lumen information acquisition unit 14 provides guidance to return the transducer 301 of the imaging sensor 300 to the position of the first mark b1 (= a1). Following the guidance, the surgeon returns the transducer 301 to the position of the first mark b1. The true lumen information acquisition unit 14 also provides guidance to search for a position α of the first FPD 21 where the transducer 301 of the imaging sensor 300 and the guidewire 500 overlap (in other words, the transducer 301 of the imaging sensor 300 and the guidewire 500 intersect) and an angioimage Vα can be obtained. Following the guidance, the surgeon moves the first FPD 21 to the position α where the imaging sensor 300 and the guidewire 500 appear to overlap, as shown in FIG. 12(A). That is, the angio image Vα is an angio image captured by the first FPD 21 positioned at position α where the transducer 301 and the guide wire 500 appear to overlap when the transducer 301 is positioned at P1 within the target blood vessel 100.

[0071] In step S20, the true lumen information acquisition unit 14 displays the ultrasound image IV1 (the ultrasound image when the transducer 301 is located at the first landmark b1 on the second angio image V2, i.e., the ultrasound image when the transducer 301 is located at P1 within the target blood vessel 100) on the canvas A2.

[0072] In step S21, the true lumen information acquisition unit 14 performs an orientation calibration process (a process of associating the direction from the transducer 301 to the guidewire 500 in the XYZ three-dimensional space with the direction from the transducer 301 to the guidewire 500 in the ultrasound image IV1 displayed in the XcYc two-dimensional space of the canvas A2). Specifically, the true lumen information acquisition unit 14 acquires the position α of the first FPD 21 in step S19. Here, as shown in FIG. 12(B), a vector representing the imaging direction of the first FPD 21 placed at the position α with respect to the heart 91 (see FIG. 1) is defined as a view vector Vwα. The true lumen information acquisition unit 14 calculates the view vector Vwα from the acquired position α of the first FPD 21. The true lumen information acquisition unit 14 calculates the rotation axis R (r1, r2, r3) using the cross product of the transducer axis vector T1 of the imaging sensor 300 at position P1 in the target blood vessel 100 calculated in (B2) above and the view vector Vwα, as shown in equation (5).

[0073]

number

[0074] Thereafter, the true lumen information acquiring unit 14 calculates a vector CV1 by rotating the transducer axis vector T1 of the transducer 301 calculated in (B2) above by 90 degrees around the rotation axis R obtained by equation (5), as shown in equation (6). That is, when the transducer 301 is at position P1 in the XYZ three-dimensional space, the vector CV1 is a vector that extends from the transducer 301 toward the guidewire 500 and is perpendicular to the transducer axis vector T1. Note that equation (7) is a matrix representation of the Rodrigues rotation formula shown in equation (6).

[0075]

number

number

[0076] FIG. 13 is a diagram showing an example of an ultrasound image IV1 displayed on the canvas A2 in step S20. In FIG. 13, the ultrasound image IV1 is an ultrasound image of the inside of the target blood vessel 100 in a direction perpendicular to the transducer axis T1 (the entire 360° circumference direction of the transducer axis T1) when the transducer 301 is located at the first mark b1 of the second angioimage V2. As shown in FIG. 13, the ultrasound image IV1 includes an image of the guidewire 500 (a portion that appears whiter than its surroundings), an image 103 of the true lumen (a portion that appears darker than its surroundings), and an image of the portion where the transducer 301 is located (a portion located near the center of the ultrasound image IV1 that appears darker than its surroundings). Also shown in FIG. 13 is an arrow CV pointing from the center of the ultrasound image IV1 (i.e., the center of the transducer 301) to the center of the image of the guidewire 500. The extension direction of the arrow CV in the XcYc two-dimensional space of the canvas A2 corresponds to the direction of the calculated vector CV1 in the XYZ three-dimensional space. In this way, in step S21, the direction from the transducer 301 to the guidewire 500 in the XYZ three-dimensional space is associated with the direction from the transducer 301 to the guidewire 500 in the ultrasound image IV1 displayed in the XcYc two-dimensional space of the canvas A2.

[0077] FIG. 14 is a diagram illustrating step S22 of the composite image output process. In step S22, the true lumen information acquisition unit 14 performs size calibration processing (processing for correlating the number of pixels of the ultrasound image IV1 displayed on the canvas A2 with the actual dimensions of the ultrasound image IV1). As shown in FIG. 14, the ultrasound image IV1 is marked with graduations SC that represent the actual dimensions of the target blood vessel 100. In this embodiment, the interval between adjacent graduations is 1 mm. The true lumen information acquisition unit 14 draws a line segment from the center of the image of the target blood vessel 100 to a graduation that is outside the contour of the image of the target blood vessel 100, and measures how many pixels this line segment is on the ultrasound image IV1 of the canvas A2 (FIG. 14: x pixels). The true lumen information acquisition unit 14 calculates the number of pixels on the canvas A2 per 1 mm of actual size by dividing the measured number of pixels by the length of the line segment.

[0078] FIG. 15 is a diagram illustrating step S23 of the composite image output processing. FIG. 15 shows an example of an ultrasound image IV1 displayed on the canvas A2 in step S23. In step S23, the true lumen information acquisition unit 14 calculates a true lumen vector S1 in the XYZ three-dimensional space (a vector perpendicular to the transducer axis T1 and extending from the transducer 301 to the true lumen 103). Specifically, the true lumen information acquisition unit 14 guides the operator to draw an arrow CV from the center of the ultrasound image IV1 (i.e., the center of the transducer 301 of the imaging sensor 300) toward the center of the image of the guidewire 500 in the ultrasound image IV1. As shown in FIG. 15, the image of the portion where the transducer 301 is located is located near the center of the ultrasound image IV1 and appears relatively dark compared to its surroundings. Furthermore, the image of the guidewire 103 appears relatively white compared to its surroundings, allowing the operator viewing the ultrasound image IV1 to grasp the positions of the transducer 301 and the guidewire 500. Following the guidance, the surgeon draws an arrow CV from the center of the ultrasound image IV1 toward the center of the image of the guidewire 500. Drawing the arrow CV can be achieved, for example, by clicking or tapping the center of the transducer 301 and the center of the image of the guidewire 500 on the ultrasound image IV1 on the canvas A2. The vector indicated by the arrow CV, i.e., the vector extending from the center of the transducer 301 to the center of the guidewire 500 in the XcYc coordinate system, is defined as vector cv. The true lumen information acquisition unit 14 also guides the surgeon to draw an arrow S from the center of the ultrasound image IV1 (i.e., the center of the image of the transducer 301 of the imaging sensor 300) toward the center of the image of the true lumen 103 on the ultrasound image IV1. As shown in FIG. 15 , the true lumen 103 appears darker on the ultrasound image IV1 than its surroundings, allowing the surgeon looking at the ultrasound image IV1 to grasp the position of the true lumen 103. Following the guidance, the surgeon draws an arrow S from the center of the ultrasound image IV1 toward the center of the image of the true lumen 103. The arrow S can be drawn by, for example, clicking or tapping the center of the transducer 301 and the center of the image of the true lumen 103 on the image of the canvas A2.The vector indicated by the arrow S, that is, the vector extending from the center of the transducer 301 to the center of the true lumen 103 in the XcYc coordinate system, is defined as vector s. The angle formed between vector cv and vector s is defined as θ.

[0079] Thereafter, the true lumen information acquisition unit 14 calculates the angle θ between vectors cv and s in the XcYc coordinates of canvas A2 using the vector dot product formula. Then, as shown in equation (8), the true lumen information acquisition unit 14 calculates the orientation of true lumen vector S1 in the XYZ three-dimensional space by rotating vector CV1 calculated in step S21 by θ degrees around the transducer axis vector T1 (r1, r2, r3) of the imaging sensor 300 calculated in (B2) above as the rotation axis. Note that equation (9) is a matrix representation of the Rodrigues rotation formula shown in equation (8).

[0080]

number

number

[0081] Furthermore, the true lumen information acquisition unit 14 acquires the number of pixels a of the arrow S drawn on the XcYc coordinates of the canvas A2, and the number of pixels c corresponding to the width of the image of the true lumen 103 in the ultrasound image IV1. The number of pixels c of the image of the true lumen 103 may be acquired automatically by image analysis of the ultrasound image IV1, or may be acquired by accepting a width specification from the surgeon. Thereafter, the true lumen information acquisition unit 14 substitutes the acquired number of pixels a and the result b of step S22 (the number of pixels b on the canvas A2 per 1 mm of actual size) into equation (10) to calculate the actual length S of the true lumen vector S1. length Furthermore, the true lumen information acquiring unit 14 calculates the actual width S of the true lumen at the portion corresponding to the true lumen vector S1 by substituting the number of acquired pixels c and the result b of step S22 into equation (11). width Calculate (mm).

[0082]

number

number

[0083] 16 is a diagram illustrating steps S24 to S28 of the composite image output process. In step S24, the true lumen information acquisition unit 14 assigns 2 to the variable n used in the composite image output process. In step S25, the true lumen information acquisition unit 14 displays the ultrasound image IV2 stored in the storage unit (i.e., the ultrasound image acquired when n=2 in step S15) on the canvas A2.

[0084] In step S26, the true lumen information acquisition unit 14 calculates the true lumen vector Sn. Specifically, the true lumen information acquisition unit 14 guides the user to draw an arrow S from the center of the ultrasound image IV2 displayed on the canvas A2 toward the image of the true lumen 103 in the ultrasound image IV2. Then, the true lumen information acquisition unit 14 calculates the angle θd ( FIG. 16 ) between the transducer axis vector T1 of the imaging sensor 300 calculated in (B2) above and the transducer axis vector T2 of the imaging sensor 300. The true lumen information acquisition unit 14 calculates the vector CV2 by rotating the vector CV1 calculated in step S21 by the calculated angle θd. Note that the rotation axis when rotating the vector CV1 by θd is calculated as the cross product of the transducer axis vectors T1 and T2. The process thereafter is the same as in step S23. Specifically, the true lumen information acquisition unit 14 acquires the angle θ between the arrow CV and the arrow S drawn in the ultrasound image IV2. As shown in equation (8), the true lumen information acquisition unit 14 calculates the direction of the true lumen vector S2 by rotating the vector CV2 by θ degrees around the transducer axis vector T2 of the imaging sensor 300 as the rotation axis. The true lumen information acquisition unit 14 also acquires the number of pixels a of the arrow S drawn in the ultrasound image IV2 and the number of pixels c corresponding to the width of the image of the true lumen 103 in the ultrasound image IV2. The true lumen information acquisition unit 14 calculates the actual length S of the true lumen vector S2 by substituting the number of pixels a and the number of pixels c into equations (10) and (11), respectively. length (mm) and the actual width S of the true lumen corresponding to the true lumen vector S2 width (mm). Similarly, for n=3 and onward, the true lumen information acquiring unit 14 can calculate the vector CVn using the difference θd between the transducer axis vector T1 and the transducer axis vector Tn. In step S27, the true lumen information acquiring unit 14 adds 1 to the variable n.

[0085] In step S28, the true lumen information acquisition unit 14 determines whether the calculation of the true lumen vector Sn (step S26) has been completed for the target number of landmarks specified in steps S14 to S17. If the variable n has reached the target number of landmarks (step S28: YES), the true lumen information acquisition unit 14 transitions the process to step S29. If the variable n has not reached the target number of landmarks (step S28: NO), the true lumen information acquisition unit 14 transitions the process to step S25 and repeats the above-described process. As a result, the variable n is incremented, for example, n = 3, n = 4, and so on, and an ultrasound image IVn is displayed on the canvas A2 (step S25), and the true lumen vector Sn is calculated (step S26).

[0086] In step S29, the true lumen information acquisition unit 14 stores the three-dimensional position information (position information in the XYZ three-dimensional space) of the true lumen 103 acquired in steps S23 to S28 in a storage unit inside the surgery support device 10. That is, in this embodiment, the three-dimensional position information of the true lumen 103 includes the directions of the true lumen vectors S1 to Sn in the XYZ three-dimensional space and the lengths S length (mm) and the actual size S of the true lumen corresponding to the true lumen vectors S1 to Sn. width (mm) and the actual size of the true lumen S width corresponds to "information about the width of the true lumen." The three-dimensional position information of the true lumen includes the actual size S width Alternatively, the number of pixels c of the image of the true lumen in the portion corresponding to the true lumen vectors S1 to Sn may be included. In this case, the number of pixels c corresponds to "information relating to the width of the true lumen." In this way, even if the first angioimage V1 or the second angioimage V2 does not include an image of the true lumen 103, the true lumen information acquisition unit 14 can acquire three-dimensional position information of the true lumen 103 based on the information of the true lumen 103 acquired from the ultrasound image IV1.

[0087] 17 and 18 are diagrams illustrating step S30 of the composite image output processing. FIG. 17(A) illustrates a true lumen vector Sn in an XYZ three-dimensional space, a true lumen image VY obtained by projecting (orthogonally projecting) the true lumen vector Sn onto a projection plane VY' from the imaging direction of the first FPD 21 placed at imaging position A (the direction of the white arrow pointing from the top to the bottom of the page), and the orthogonal projection vector Spn of the true lumen vector Sn on the true lumen image VY. FIGS. 17(B) and 17(C) are diagrams illustrating the calculation of the orthogonal projection vector Spn of the true lumen vector Sn. FIG. 18(A) illustrates an example of an angioimage VX at an arbitrary FPD position. FIG. 18(B) illustrates an example of a true lumen image VY corresponding to the angioimage VX.

[0088] In step S30, the image composition unit 16 superimposes the true lumen image on the captured image (angiographic image) at an arbitrary FPD position. Specifically, the true lumen image generation unit 15 and the image composition unit 16 perform the processes described in the following (C1) to (C4). (C1) The true lumen image generating unit 15 acquires an angio-image VX of the target blood vessel 100 by the first FPD 21 (see FIG. 17(A)) placed at an arbitrary imaging position A. The imaging position A corresponds to the "first imaging position." (C2) The true lumen image generating unit 15 acquires position information of the imaging position A from the first FPD 21. The true lumen image generating unit 15 also acquires three-dimensional position information of the true lumen from a storage unit included in the surgery support device 10 (see FIG. 1). (C3) Using the position information of imaging position A and the three-dimensional position information of the true lumen, the true lumen image generating unit 15 generates a true lumen image VY that represents the true lumen at a position and orientation corresponding to the angioimage VX at imaging position A. The method for generating the true lumen image VY will be described later in (D1) to (D7). (C4) The image synthesis unit 16 generates a synthetic image V by synthesizing the angio image VX and the true lumen image VY, and displays the synthetic image V on the canvas A2.

[0089] 18, the angio image VX includes an image of the imaging sensor 300 viewed from an arbitrary imaging position A, an image of the guidewire 500, and an image of the target blood vessel 100 (not shown). The true lumen image VY also includes an image of the true lumen 103 at a position and orientation corresponding to the angio image VX (in other words, when viewed from the imaging position A where the angio image VX was acquired). The position of the true lumen is synonymous with the coordinates of the image of the true lumen 103 on the true lumen image VY. The orientation of the true lumen is synonymous with the orientation of the image of the true lumen 103 on the true lumen image VY.

[0090] The true lumen image generating unit 15 generates the true lumen image VY according to the following procedures (D1) to (D7). 17(A), a vector extending perpendicularly from imaging position A to projection plane VY' (hereinafter, true lumen image VY will be described as projection plane VY') is defined as view vector VnA (i.e., view vector VnA is a vector representing the imaging direction of first FPD 21 placed at imaging position A with respect to heart 91 (see FIG. 1)). The true lumen image generator 15, for example, defines view vector VnA in XYZ three-dimensional space by the first equation of equation (12), and calculates, from the first equation of equation (12), the second equation (vectors a1 and a2) of equation (12) that represents true lumen image VY, which is a plane orthogonal to view vector VnA.

[0091]

number

[0092] (D2) The true lumen image generating unit 15 defines the true lumen image VY defined by the vector a1 and vector a2 calculated in step (D1) as a matrix A, and calculates a projection matrix P for calculating an orthogonal projection vector Spn onto the true lumen image VY of the true lumen vector Sn included in the three-dimensional position information of the true lumen 103 using equation (13). Note that in equation (13), the matrix A T means the transpose of matrix A.

[0093]

number

[0094] (D3) The true lumen image generator 15 calculates the orthogonal projection vectors Spn and Zp onto the true lumen image VY using equation (14) for the true lumen vector Sn and the Z axis in the XYZ three-dimensional space (FIG. 1: the direction of the head 92 of the human body 90). Note that in equation (14), Ze represents the unit vector representing the Z axis in the XYZ three-dimensional space. Furthermore, the orthogonal projection vectors Spn and Zp calculated using equation (14) are vectors on the true lumen image VY expressed in XYZ coordinates.

[0095]

number

[0096] (D4) The true lumen image generating unit 15 converts the orthogonal projection vector Spn calculated in step (D3) into two-dimensional coordinates on the true lumen image VY and calculates the orientation of the orthogonal projection vector Spn on the true lumen image VY. Specifically, as shown in FIG. 17(B), the true lumen image generating unit 15 first calculates the angle θp formed by the orthogonal projection vector Spn of the true lumen vector Sn onto the true lumen image VY and the orthogonal projection vector Zp of the Ze vector representing the Z axis onto the true lumen image VY. Since the orthogonal projection vector Spn and the orthogonal projection vector Zp have been calculated using equation (14), θp can be calculated, for example, using the formula for the dot product of the vectors. Now, suppose that the orthogonal projection vector Spn and the orthogonal projection vector Zp are displayed on a canvas A2 having XcYc coordinates (i.e., suppose that the true lumen image VY is displayed on the canvas A2 as shown in FIG. 17(B)). In this case, vector Zp can be said to have the same direction as the Z axis in the XYZ three-dimensional space, and therefore is parallel to the Yc axis. However, like the Z axis (vector Ze), the direction of vector Zp is the direction in which the head 92 of the human body 90 is located, that is, the direction from the bottom to the top of the page in FIG. 17(B), and therefore is the exact opposite direction to the Yc axis. Next, the true lumen image generating unit 15 defines the unit vector of the orthogonal projection vector Spn in the XcYc coordinate system as the orthogonal projection unit vector Spn′(x′, y′) and calculates it using equation (15) or equation (16). The orthogonal projection unit vector Spn′(x′, y′) can be said to represent the direction of the orthogonal projection vector Spn on the true lumen image VY. Note that equation (15) applies when the orthogonal projection unit vector Spn' is in the clockwise direction (CW) when viewed from the orthogonal projection vector Zp, and equation (16) applies when the orthogonal projection unit vector Spn' is in the counterclockwise direction (CCW) when viewed from the orthogonal projection vector Zp.

[0097]

number

number

[0098] (D5) The true lumen image generation unit 15 calculates the angle between the true lumen vector Sn and the view vector VnA to calculate the length of the orthogonal projection vector Spn of the true lumen vector Sn. Specifically, as shown in FIG. 17(c), the angle between the true lumen vector Sn and the view vector VnA is defined as θ, and sin θ is calculated using the true lumen vector Sn and the view vector VnA according to equation (17) based on the formula for the cross product of vectors. Note that in equation (17), VnA × Sn means the cross product of the view vector VnA and the true lumen vector Sn.

[0099]

number

[0100] (D6) The true lumen image generating unit 15 calculates the length of the true lumen vector Sn included in the three-dimensional position information of the true lumen as Sn length (mm), then, using sinθ calculated in step (D5), the length Spn of the orthogonal projection vector Spn of the true lumen vector Sn is length is calculated using equation (18) (see FIG. 17(c)). Next, the true lumen image generation unit 15 calculates the orthogonal projection vector Spn(x, y) of the true lumen vector Sn in the XcYc coordinates of the canvas A2 using equations (15), (16), (18), and (19). It can be said that the orthogonal projection vector Spn(x, y) represents the direction and length of the orthogonal projection vector Spn on the true lumen image VY.

[0101]

number

number

[0102] (D7) The true lumen image generating unit 15 generates a true lumen image VY using the orthogonal projection vector Spn(x, y) obtained in step (D6).

[0103] FIG. 19 is a diagram showing an example of a composite image. FIG. 19(A) is a diagram showing an example of a composite image V at an arbitrary imaging position A. FIG. 19(B) is a diagram showing an example of a composite image Vb at an imaging position B different from imaging position A. By performing the processes described in steps (C1) to (C4), the composite image V shown in FIG. 19(A) is displayed on canvas A2. The composite image V is an image in which a true lumen image VY is superimposed on an angio image VX shown in FIG. 18. Here, in the true lumen image VY of this embodiment, the image of the true lumen 103 is at a position and orientation corresponding to the angio image VX at imaging position A, and further has a width according to the three-dimensional position information of the true lumen. When generating such a true lumen image VY, the true lumen image generator 15 further calculates in step (D7) the actual size S of the true lumen included in the three-dimensional position information of the true lumen. width (mm) to change the width of the image of the true lumen 103.

[0104] Here, a case will be described in which the surgeon moves the first FPD 21 from imaging position A to imaging position B different from A. Note that imaging position B corresponds to the "second imaging position." In such a case, the true lumen image generating unit 15 and the image combining unit 16 repeatedly execute the above-described steps (C1) to (C4) using the position information of the changed imaging position B (i.e., the position information of the second imaging position). As a result, as shown in FIG. 19(B), a combined image Vb is redisplayed on the canvas A2, combining an angio image VX including images of the imaging sensor 300, guidewire 500, and target blood vessel 100 viewed from a different angle with a true lumen image VY including an image of the true lumen 103 corresponding to that angle. In the example of FIG. 19(B), the image of the true lumen 103 included in the true lumen image VY is the same as the actual size S of the true lumen. width This is a polygon image having a width corresponding to the width (mm) of the true lumen 103. In this way, the surgeon can intuitively grasp the position, posture, and thickness of the true lumen 103, as well as the positional relationship between the medical device (the imaging sensor 300 or the guide wire 500) and the true lumen 103.

[0105] 20 and 21 are diagrams illustrating modified examples of the true lumen image VY. FIG. 20(A) is a diagram showing a composite image Vc using the true lumen image VY of a first modified example. FIG. 20(B) is a diagram showing a composite image Vd using the true lumen image VY of a second modified example. FIG. 21 is a diagram showing a composite image Ve using the true lumen image VY of a third modified example. In the example of FIG. 20(A), the image of the true lumen 103 included in the true lumen image VY is the actual size S of the true lumen. width In this way, it is possible to prevent the image of the true lumen 103 from reducing the visibility of the distal end of the medical device (the imaging sensor 300 or the guide wire 500). In the example of FIG. 20(A), the image of the true lumen 103 included in the true lumen image VY is a set of line segments having a width corresponding to the actual size S of the true lumen. width 21 , the image of the true lumen 103 included in the true lumen image VY is a line segment having a position and orientation corresponding to the angio image VX at imaging position A, but has no width. In this way, the processing load on the true lumen image generation unit 15 can be reduced.

[0106] As described above, according to the surgery support device 10 of the first embodiment, the true lumen image generation unit 15 can generate a true lumen image VY representing the true lumen at a position and orientation corresponding to the angioimage VX, using position information of the first imaging position A at which the angioimage VX was acquired and three-dimensional position information of the true lumen acquired by the true lumen information acquisition unit 14. That is, even when the contrast agent does not flow into the target true lumen 103 or when no contrast agent is being injected, the true lumen image generation unit 15 can generate a true lumen image VY representing an image of the true lumen 103 based on the three-dimensional position information of the true lumen. Furthermore, the image synthesis unit 16 generates a synthesized image V by synthesizing the angioimage VX at an arbitrary first imaging position A with the true lumen image VY representing an image of the true lumen 103, and outputs the synthesized image V, thereby displaying an image of the true lumen 103 of the target blood vessel 100 in the image (angioimage VX) of the FPD. Therefore, the surgeon can proceed with the procedure while checking the positional relationship between the medical devices 300, 500 on the angio image VX and the true lumen 103 on the true lumen image VY by checking the composite image V. As a result, the surgeon can correctly grasp the position of the true lumen 103 within the target blood vessel 100, which can improve the accuracy of the procedure, shorten the time required for the procedure, and reduce the burden on the patient.

[0107] Furthermore, according to the surgery support device 10 of the first embodiment, the true lumen image generator 15 generates a true lumen image VY representing the true lumen 103 having a width corresponding to the three-dimensional position information of the true lumen, and therefore the surgeon can proceed with the procedure while checking the width of the true lumen 103 by viewing the composite image V. As a result, it is possible to further improve the accuracy of the procedure, shorten the time required for the procedure, and reduce the burden on the patient.

[0108] Furthermore, according to the surgery support device 10 of the first embodiment, when the first FPD 21 is moved to a second imaging position B different from the first imaging position A and imaging is performed by the first FPD 21, the true lumen image generation unit 15 regenerates a true lumen image VY corresponding to the angioimage VX at the second imaging position B, and the image composition unit 16 regenerates a composite image Vb by combining the reacquired angioimage VX and the regenerated true lumen image VY, and outputs the composite image Vb. That is, the true lumen image generation unit 15 and the image composition unit 16 can display a composite image Vb including the true lumen image VY after the movement of the imaging position of the first FPD 21. As a result, the convenience of the surgery support device 10 can be improved, and the accuracy of the procedure can be further improved, the time required for the procedure can be shortened, and the burden on the patient can be reduced.

[0109] Furthermore, according to the surgery support device 10 of the first embodiment, the true lumen information acquisition unit 14 can acquire three-dimensional position information of the true lumen using position information of the first position where the first angioimage was acquired, the first angioimage, position information of the second position where the second angioimage was acquired, the second angioimage, and the ultrasound image (FIGS. 4 and 5: steps S3 to S29). Specifically, the true lumen information acquisition unit 14 can acquire three-dimensional position information of the imaging sensor 300 using the position information of the first position and the first angioimage V1, and the position information of the second position and the second angioimage V2 (FIG. 5: step S18). Then, the true lumen information acquisition unit 14 can acquire three-dimensional position information of the true lumen using the three-dimensional position information of the imaging sensor 300, the position information of the first position, the first angio image V1, and the ultrasound image IVn showing the true lumen 103 of the target blood vessel 100 (Figure 5: steps S23, S26).

[0110] Furthermore, according to the surgery support device 10 of the first embodiment, the true lumen information acquisition unit 14 can acquire three-dimensional position information of the ultrasonic sensor using images of the imaging sensor 300 included in the first angio image V1 and the second angio image V2 (FIG. 5: step S18). The true lumen information acquisition unit 14 also associates the positional relationship between the first angio image V1 and the ultrasonic image IV1 using images of a medical device (specifically, the guidewire 500) included in the first angio image V1 and the ultrasonic image IV1 (FIG. 5: step S21), acquires position information of the true lumen 103 from the ultrasonic image IVn (FIG. 5: steps S23 and S26), and acquires three-dimensional position information of the true lumen using the acquired position of the imaging sensor 300 and the position information of the true lumen 103 in the ultrasonic image IVn acquired by the imaging sensor 300 (step S29).

[0111] Second Embodiment 22 is an explanatory diagram illustrating the configuration of a surgery support system 1A of the second embodiment. The surgery support system 1A of the second embodiment differs from the first embodiment in that it acquires three-dimensional position information of the true cavity without using an imaging sensor 300. The surgery support system 1A has a surgery support device 10A instead of the surgery support device 10 in the configuration of the first embodiment. The surgery support device 10A does not have an ultrasound image acquisition unit 13, and has a true cavity information acquisition unit 14A instead of the true cavity information acquisition unit 14.

[0112] Since the surgery assistance system 1A does not use the imaging sensor 300, the wire catheter 400 and the guidewire 500 are inserted into the target blood vessel 100 described in Fig. 3, while the sensor catheter 200 and the imaging sensor 300 are not. In the composite image output process, the true lumen information acquisition unit 14A acquires three-dimensional position information of the true lumen using the image of the guidewire 500 and the image of the true lumen 103 included in the first angioimage and the second angioimage.

[0113] Fig. 23 is a flowchart showing an example of composite image output processing according to the second embodiment. Fig. 24 is a diagram illustrating composite image output processing according to the second embodiment. Fig. 24(A) is a diagram illustrating steps S3A to S7A. Fig. 24(B) is a diagram illustrating steps S10 to S17. In step S1 of Fig. 23, the true lumen information acquisition unit 14A guides the user to prepare for imaging by the first FPD 21. The details are the same as in step S1 of Fig. 4.

[0114] In step S3A, the true lumen information acquisition unit 14A guides the surgeon to move the first FPD 21 to a first position and capture an X-ray image. Following the guidance, the surgeon moves the first FPD 21 to the first position and captures an X-ray image, thereby acquiring a first angio image V1. The first position can be any position (RAOXX CRAXX: X is any natural number). The angio image acquisition unit 12 acquires the captured first angio image V1 from the blood vessel imaging device 20. As shown in FIG. 24(A), the first angio image V1 includes an image of the tip 401 of the wire catheter 400, an image of the guidewire 500 inserted into the wire catheter 400, an image of the target blood vessel 100 (not shown), and an image of the true lumen 103.

[0115] In step S4A, the true lumen information acquisition unit 14A displays the first angio image V1 on the canvas A2 and adjusts the position of the first angio image V1 so that the image of the tip 401 of the wire catheter 400 is positioned at the center of the canvas A2. In the second embodiment, the processing proceeds with the tip 401 of the wire catheter 400 as the "reference point of the reference device."

[0116] In step S5A, the true lumen information acquisition unit 14A guides the surgeon to place a first mark a1 on the edge (in other words, the proximal end) of the image of the true lumen 103 in the first angio image V1 displayed on the canvas A2. Following the guidance, the surgeon places the first mark a1 on the first angio image V1 on the canvas A2, as shown in FIG. 24(A).

[0117] In step S7A, the true lumen information acquisition unit 14A guides the surgeon to place the second mark a2 at any position within the range of the first angioimage V1 displayed on the canvas A2 where the image of the true lumen 103 can be considered to extend linearly with the first mark a1 as the reference. Following the guidance, the surgeon places the second mark a2 on the image of the true lumen 103 that appears in the first angioimage V1 on the canvas A2, as shown in FIG. 24(A).

[0118] In step S8A, the true lumen information acquisition unit 14A calculates the BNV using the first landmark, the second landmark, and the first position. Details are the same as in step S8 in Fig. 4, and the processing can be performed by replacing the "first shaft axis vector of the imaging sensor 300" in step S8 with the "central axis vector of the true lumen 103 in the first angio image V1."

[0119] In step S10, the true lumen information acquisition unit 14A guides the surgeon to move the first FPD 21 to the BNV (second position) calculated in step S8A and capture an X-ray image. Following the guidance, the surgeon moves the first FPD 21 to the BNV (second position) and captures an X-ray image, thereby acquiring a second angio image V2. The angio image acquisition unit 12 acquires the captured second angio image V2 from the blood vessel imaging device 20. As shown in FIG. 24(B), the second angio image V2 includes an image of the tip 401 of the wire catheter 400, an image of the guidewire 500 inserted into the wire catheter 400, an image of the target blood vessel 100 (not shown), and an image of the true lumen 103, all captured from a different orientation from the first angio image V1.

[0120] In step S11A, the true lumen information acquisition unit 14A displays the second angio image V2 on the canvas A2 and adjusts the position of the second angio image V2 so that the image of the tip 401 of the wire catheter 400 is positioned at the center of the canvas A2.

[0121] In step S12A, the true lumen information acquisition unit 14A provides guidance to place a first mark b1 on the edge (in other words, the proximal end) of the image of the true lumen 103 in the second angioimage V2 displayed on the canvas A2. Following the guidance, the surgeon places the first mark b1 on the image of the true lumen 103 shown in the second angioimage V2 on the canvas A2, as shown in FIG. 24(B).

[0122] In step S13, the true lumen information acquisition unit 14A assigns 2 to the variable n used in the composite image output process. n is a natural number. In step S14A, the true lumen information acquisition unit 14A guides the surgeon to place a second mark b2 at any position distal to the stump of the image of the true lumen 103. Following the guidance, the surgeon places the second mark b2 on the image of the true lumen 103 appearing in the second angio image V2 on the canvas A2. In step S16, the true lumen information acquisition unit 14A adds 1 to the variable n.

[0123] In step S17, the true lumen information acquisition unit 14A determines whether the placement of markers on the second angioimage V2 (step S14A) has been completed for the desired number of markers. If the variable n has reached the desired number of markers (step S17: YES), the true lumen information acquisition unit 14A transitions the process to step S18A. If the variable n has not yet reached the desired number of markers (step S17: NO), the true lumen information acquisition unit 14A transitions the process to step S14A and repeats the above-described process.

[0124] In step S18A, the true lumen information acquisition unit 14A calculates the following (E1) and (E2) using the coordinates of the first landmark a1 and the second landmark a2 in the first angio image V1 and the coordinates of the first landmark b1 to the nth landmark bn in the second angio image V2. (E1) True lumen vectors S2 to Sn: The true lumen vectors S2 to Sn correspond to the true lumen vectors S1 to Sn obtained in FIGS. (E2) Vector P from the reference point of the reference device (tip 401 of the wire catheter 400) to the edge of the true lumen 103.

[0125] FIG. 25 is a diagram illustrating step S18A of the composite image output process of the second embodiment. FIG. 25(A) is a diagram illustrating points on a first angio image V1, and FIG. 25(B) is a diagram illustrating points on a second angio image V2. In FIGS. 25(A) and 25(B), the Z axis (the direction in which the head 92 of the human body 90 in FIG. 1 is located) is oriented toward the top of the paper. The first angio image V1 and the second angio image V2 of the second embodiment are images of the true lumen 103 captured from different angles. Therefore, the first landmark b1 to the n-th landmark bn (n=4 in the illustrated example) in the second angio image V2 shown in FIG. 25(B) can be said to be located on the vector S′ of the first angio image V1 shown in FIG. 25(A). Therefore, the direction of the “(E1) true lumen vectors S2 to Sn” can be calculated using the gradient of the vector S′ and the gradient of the vectors S2′ to Sn′. Note that vector S' is a vector extending from the first mark a1 to the second mark a2. Vector S2' is a vector extending from the first mark b1 to the second mark b2, and vector Sn' is a vector extending from the first mark b1 to the n-th mark bn. Details of the method for calculating the direction of the "(E1) true lumen vectors S2 to Sn" are the same as in step S18 of FIG. 5, and processing can be performed by replacing "position vectors P2 to Pn of transducer 301" in step S18 with "true lumen vectors S2 to Sn." Furthermore, the true lumen information acquisition unit 14A calculates the length of the "(E1) true lumen vectors S2 to Sn" using equation (4) described in step S18 of FIG.

[0126] The direction of "(E2) vector P from the reference point of the reference device to the edge of the true lumen 103" can be calculated using the gradient of vector P' in the first angio image V1 and the gradient of vector P' in the second angio image V2. Note that vector P' in the first angio image V1 is a vector extending from the reference point of the reference device (the distal end 401 of the wire catheter 400) in the first angio image V1 to the first mark a1. Also, vector P' in the second angio image V2 is a vector extending from the reference point of the reference device in the second angio image V2 to the first mark b1 (= a1). The direction of "(E2) vector P from the reference point of the reference device to the edge of the true lumen 103" can be calculated by applying the method for calculating the direction of "position vectors P2 to Pn of transducer 301" in step S18 of FIG. 5. The length of "(E2) vector P from the reference point of the reference device to the stump of the true lumen 103" can be calculated using equation (4) described in step S18 of FIG.

[0127] In step S29, the true lumen information acquisition unit 14A stores the three-dimensional position information of the true lumen acquired in step S18A in a memory unit inside the surgery support device 10. That is, in this embodiment, the three-dimensional position information of the true lumen includes the directions and lengths of the true lumen vectors S2 to Sn. In step S30, the image synthesis unit 16 superimposes the true lumen on an image (angiogram) captured at an arbitrary FPD position. The details are the same as in step S30 in FIG. 5.

[0128] In this way, when the first angio-image V1 and the second angio-image V2 include an image of the true lumen 103, it is possible to acquire three-dimensional position information of the true lumen without using the imaging sensor 300, as described in the second embodiment. In this second embodiment, the same effects as those of the first embodiment can be achieved. Furthermore, according to the surgery support device 10A of the second embodiment, the true lumen information acquisition unit 14A can acquire three-dimensional position information of the true lumen using the image of the medical device and the image of the true lumen included in the first angio-image and the second angio-image.

[0129] <Third embodiment> FIG. 26 is an explanatory diagram illustrating the configuration of a surgery support system 1B according to a third embodiment. In the surgery support system 1B according to the third embodiment, a surgery support device 10B is connected to a network. The surgery support device 10B acquires a first angio-image V1, a second angio-image V2, an ultrasound image IV1, and an ultrasound image IVn from an external device via the network to acquire three-dimensional position information of the true lumen. The surgery support device 10B then acquires an angio-image VX positioned at an arbitrary imaging position and generates a true lumen image VY including an image of the true lumen 103 at a position and orientation corresponding to the angio-image VX. The surgery support device 10B generates a composite image V by combining the angio-image VX and the true lumen image VY and outputs the generated composite image V to an external device via the network. The surgery support device 10B may acquire images V1, V2, IV1, and IVn from a storage medium such as a USB memory stick instead of an external device connected via the network and output the composite image V to the storage medium.

[0130] As described above, the configuration of the surgery assistance system 1B can be modified in various ways, and it does not have to include the blood vessel imaging device 20, the display device 30, the table 40, and the operation unit 50. In other words, the surgery assistance system 10B may be configured as an information processing device or a server. This third embodiment can also achieve the same effects as the first embodiment. Furthermore, the surgery assistance system 1B of the third embodiment can provide a service of providing a composite image to an external device connected via a network.

[0131] <Modification of this embodiment> The present invention is not limited to the above-described embodiment, and can be embodied in various forms without departing from the spirit of the present invention. For example, a part of the configuration realized by hardware may be replaced by software, and conversely, a part of the configuration realized by software may be replaced by hardware. In addition, for example, the following modifications are also possible.

[0132] [Variation 1] In the first to third embodiments, the configurations of the surgery assistance systems 1, 1A, and 1B have been illustrated. However, the configuration of the surgery assistance system 1 can be modified in various ways. For example, the display device 30 may be a monitor or touch panel built into the surgery assistance devices 10, 10A, and 10B. For example, the blood vessel imaging device 20 may be configured to have a single FPD (in other words, a configuration without the second FPD 25). For example, the surgery assistance system 1 may include other medical devices (e.g., a CT device, an MRI device), etc., not shown. In this case, the operation screen OS described in FIG. 6(A) may include images acquired by the other medical devices.

[0133] [Variation 2] In the first to third embodiments, the configurations of the surgery support devices 10, 10A, and 10B are shown. However, the configuration of the surgery support device 10 can be modified in various ways. For example, the functions of each functional unit included in the surgery support device 10 may be realized by cooperation between multiple devices connected via a network.

[0134] [Variation 3] In the first to third embodiments described above, an example of the procedure for the composite image output process has been shown. However, the procedure for the composite image output process described in Figures 4, 5, and 23 is merely an example, and various modifications are possible. For example, the order in which the steps are performed may be changed, at least some steps may be omitted, or other steps not described may be performed.

[0135] For example, the three-dimensional position information of the true lumen does not have to be defined by the direction and length of the true lumen vectors S1 to Sn. In this case, the three-dimensional position information of the true lumen can be defined by any information, such as a set of coordinates of a sequence of points forming the outer edge of the true lumen or a set of coordinates of characteristic points on the outer edge of the true lumen.

[0136] For example, when the first FPD 21 is moved to an imaging position B (second imaging position) different from the imaging position A (first imaging position), the regeneration of the true lumen image VY and the re-output of the composite image Vb may be omitted. Furthermore, when the first FPD 21 is moved to an imaging position B (second imaging position) different from the imaging position A (first imaging position), the image composition unit 16 may output the composite image V at the first imaging position and the composite image Vb at the second imaging position on the canvas A2 in a manner in which both can be viewed. The manner in which both can be viewed can be determined appropriately by, for example, displaying them side by side or allowing reference to a history.

[0137] For example, the true lumen image generating unit 15 may make the image of the true lumen 103 included in the true lumen image VY semi-transparent, or may set at least one of the hue, saturation, and brightness of the image of the true lumen 103 to a hue, saturation, and brightness that makes it easy to distinguish from the imaging sensor 300 and the guidewire 500. In this way, the image of the true lumen 103 can be displayed without impairing the visibility of the medical devices (imaging sensor 300, guidewire 500). This adjustment may be performed automatically by the true lumen image generating unit 15 through image analysis of the angioimage VX, or may be changed in accordance with instructions received from the surgeon.

[0138] For example, the image composition unit 16 may be able to switch between displaying and hiding the angio image XV and the true lumen image VY in the composite image V. This can further improve the usability of the surgery support devices 10, 10A, and 10B.

[0139] 4 and 5, the processing of step S19 may be included in step S3. Specifically, in step S3, "the true lumen information acquisition unit 14 moves the first FPD 21 to a first position to guide the first FPD 21 to capture an X-ray image" may be changed to "the true lumen information acquisition unit 14 moves the first FPD 21 to a first position (corresponding to position α in step S19) where a first angio image V1 (corresponding to angio image Vα in step S19) is obtained, where the transducer 301 of the imaging sensor 300 overlaps with the guidewire 500 (in other words, the transducer 301 of the imaging sensor 300 intersects with the guidewire 500). Following the guidance, the surgeon moves the first FPD 21 to a first position where a first angioimage V1 where the transducer 301 and the guidewire 500 overlap can be obtained, and acquires the first angioimage V1 where the transducer 301 and the guidewire 500 overlap. In this case, in step S21, the true lumen information acquisition unit 14 performs orientation calibration processing using the first position of the first FPD 21. By changing the procedure of the composite image output processing as described above, step S19 can be omitted, and the composite image output processing can be made more efficient.

[0140] [Variation 4] The configurations of the surgery support devices 10, 10A, and 10B of the first to third embodiments and the configurations of the first to third modifications may be combined as appropriate. For example, the main controller 11 may be configured to switch between the composite image output process (FIGS. 4 and 5) described in the first embodiment and the composite image output process (FIG. 23) described in the second embodiment. The main controller 11 may perform the switching in response to an instruction from the surgeon, or may perform the switching automatically by image analysis of the first angio image V1 and the second angio image V2. For example, in the case of automatic switching, the main controller 11 may perform the composite image output process (FIG. 23) described in the second embodiment when the first angio image V1 and the second angio image V2 include an image of the true lumen 103, and perform the composite image output process (FIGS. 4 and 5) described in the first embodiment when the image of the true lumen 103 is not included.

[0141] This aspect has been described above based on embodiments and modifications. However, the above-described embodiments are intended to facilitate understanding of this aspect and are not intended to limit this aspect. This aspect may be modified or improved without departing from the spirit and scope of the claims, and equivalents thereof are included in this aspect. Furthermore, if a technical feature is not described as essential in this specification, it may be deleted as appropriate. [Explanation of symbols]

[0142] 1, 1A, 1B...Surgery support system 10,10A,10B…Surgical support device 11...Main control unit 12...Angio image acquisition unit 13...Ultrasound image acquisition unit 14, 14A... True lumen information acquisition section 15...True lumen image generation unit 16...Image synthesis unit 20...vascular imaging device 22...1st X-ray tube device 23...1st C-arm 24...First support part 26…Second X-ray tube device 27...Second C-arm 28…Second support part 29...Control unit 30…Display device 31...Monitor 32...Arm 40...table 41...Bed 42...Extendable part 43...legs 50...Operation unit 90...Human body 91...Heart 92…head 93...feet 100...Target blood vessel 102…false lumen 103…True cavity 200...Sensor catheter 300...imaging sensor 301...Transducer 400...wire catheter 401...Tip 500...Guidewire

Claims

1. A surgical support device, comprising: a true lumen information acquiring unit for acquiring three-dimensional position information of a true lumen existing in a target blood vessel; a true lumen image generating unit that acquires an angio image of the target blood vessel from an FPD (flat panel detector) disposed at a first imaging position, and generates a true lumen image representing the true lumen at a position and orientation corresponding to the angio image, using position information of the first imaging position and three-dimensional position information of the true lumen; an image synthesis unit that generates a synthetic image by synthesizing the angio image and the true lumen image and outputs the synthetic image; A surgical support device comprising:

2. The surgery support device according to claim 1, The three-dimensional position information of the true lumen includes information regarding a width of the true lumen, The true lumen image generating unit generates a true lumen image representing the true lumen having a width according to three-dimensional position information of the true lumen.

3. The surgery support device according to claim 1 or 2, when the FPD is moved to a second imaging position different from the first imaging position and an image of the target blood vessel is captured from the FPD, the true lumen image generating unit reacquires the angio-image captured at the second imaging position, and regenerates a true lumen image representing the true lumen at a position and orientation corresponding to the reacquired angio-image using position information of the second imaging position and three-dimensional position information of the true lumen; The image synthesis unit regenerates a synthetic image by synthesizing the reacquired angio image and the regenerated true lumen image, and outputs the synthetic image.

4. The surgery support device according to claim 1 or 2, further comprising: an angio-image acquiring unit that acquires a first angio-image captured by the FPD disposed at a first position and a second angio-image captured by the FPD disposed at a second position different from the first position; an ultrasonic image acquisition unit for acquiring an ultrasonic image of the inside of the target blood vessel captured by an ultrasonic sensor; Equipped with The first angio image includes: The ultrasonic sensor is disposed at a first landmark position in the target blood vessel; a medical device different from the ultrasonic sensor that is disposed at a second landmark position in the target blood vessel; The second angio image includes: The ultrasonic sensor is disposed at the first landmark position in the target blood vessel, The ultrasonic image is an image captured in a state where the ultrasonic sensor is placed at the first mark position, and the ultrasonic image includes: The target blood vessel; the medical device positioned at the second landmark location within the target blood vessel; A surgical support device, wherein the true lumen information acquisition unit acquires three-dimensional position information of the true lumen using position information of the first position, the first angio image, position information of the second position, the second angio image, and the ultrasound image.

5. The surgery support device according to claim 4, The true lumen information acquisition unit acquiring a position of the ultrasonic sensor using an image of the ultrasonic sensor included in the first angio-image and the second angio-image; Associating a positional relationship between the first angio-image and the ultrasound image using images of the medical device included in the first angio-image and the ultrasound image; acquiring position information of the true cavity from the ultrasound image; A surgical support device that acquires three-dimensional position information of the true cavity using the acquired position of the ultrasonic sensor and position information of the true cavity in the ultrasonic image captured by the ultrasonic sensor.

6. The surgery support device according to claim 1 or 2, further comprising: an angio-image acquiring unit that acquires a first angio-image captured by the FPD disposed at a first position and a second angio-image captured by the FPD disposed at a second position different from the first position; Equipped with The first angio image includes: The true lumen of the target blood vessel; a medical device positioned at a first landmark location within the target blood vessel; The second angio image includes: The true lumen of the target blood vessel; the medical device positioned at the first landmark location within the target blood vessel; The true lumen information acquisition unit A surgical support device that obtains three-dimensional position information of the true cavity using an image of the medical device and an image of the true cavity contained in the first angio image and the second angio image.

7. A surgical assistance method, comprising: a true lumen information acquiring step of acquiring three-dimensional position information of a true lumen existing in a target blood vessel; a true lumen image generating step of acquiring an angio-image of the target blood vessel from an FPD (flat panel detector) arranged at a first imaging position, and generating a true lumen image showing the true lumen at a position and orientation corresponding to the angio-image, using position information of the first imaging position and three-dimensional position information of the true lumen; an image synthesis step of generating a synthetic image by synthesizing the angio image and the true lumen image and outputting the synthetic image; A surgical assistance method comprising:

8. The surgical support method according to claim 7, The three-dimensional position information of the true lumen includes information regarding a width of the true lumen, In the true lumen image generating step, a true lumen image is generated that represents the true lumen and has a width according to three-dimensional position information of the true lumen.

9. The surgical support method according to claim 7 or 8, in the true lumen image generating step, when the FPD is moved to a second imaging position different from the first imaging position and an image of the target blood vessel is captured from the FPD, the angio-image captured at the second imaging position is acquired again, and a true lumen image representing the true lumen at a position and orientation corresponding to the reacquired angio-image is generated using position information of the second imaging position and three-dimensional position information of the true lumen; In the image synthesis step, a synthetic image is regenerated by synthesizing the reacquired angio image and the regenerated true lumen image, and the synthetic image is output.

10. The surgical support method according to claim 7 or 8, further comprising: an angio-image acquiring step of acquiring a first angio-image captured by the FPD disposed at a first position and a second angio-image captured by the FPD disposed at a second position different from the first position; an ultrasonic image acquiring step of acquiring an ultrasonic image of the inside of the target blood vessel captured by an ultrasonic sensor; Equipped with The first angio image includes: The ultrasonic sensor is disposed at a first landmark position in the target blood vessel; a medical device different from the ultrasonic sensor that is disposed at a second landmark position in the target blood vessel; The second angio image includes: The ultrasonic sensor is disposed at the first landmark position in the target blood vessel, The ultrasonic image is an image captured in a state where the ultrasonic sensor is placed at the first mark position, and the ultrasonic image includes: The target blood vessel; the medical device positioned at the second landmark location within the target blood vessel; A surgical support method in which, in the true lumen information acquisition process, three-dimensional position information of the true lumen is acquired using position information of the first position, the first angio image, position information of the second position, the second angio image, and the ultrasound image.

11. The surgical support method according to claim 10, In the true lumen information acquiring step, acquiring a position of the ultrasonic sensor using an image of the ultrasonic sensor included in the first angio-image and the second angio-image; Associating a positional relationship between the first angio-image and the ultrasound image using images of the medical device included in the first angio-image and the ultrasound image; acquiring position information of the true cavity from the ultrasound image; A surgical support method for acquiring three-dimensional position information of the true cavity using the acquired position of the ultrasonic sensor and position information of the true cavity in the ultrasonic image captured by the ultrasonic sensor.

12. The surgical support method according to claim 7 or 8, further comprising: an angio-image acquiring step of acquiring a first angio-image captured by the FPD disposed at a first position and a second angio-image captured by the FPD disposed at a second position different from the first position; Equipped with The first angio image includes: The true lumen of the target blood vessel; a medical device positioned at a first landmark location within the target blood vessel; The second angio image includes: The true lumen of the target blood vessel; the medical device positioned at the first landmark location within the target blood vessel; In the true lumen information acquiring step, A surgical support method for obtaining three-dimensional position information of the true cavity using an image of the medical device and an image of the true cavity contained in the first angio image and the second angio image.

13. A computer program comprising: a true lumen information acquiring step of acquiring three-dimensional position information of a true lumen existing in a target blood vessel; a true lumen image generating step of acquiring an angio-image of the target blood vessel from an FPD (flat panel detector) arranged at a first imaging position, and generating a true lumen image showing the true lumen at a position and orientation corresponding to the angio-image, using position information of the first imaging position and three-dimensional position information of the true lumen; an image synthesis step of generating a synthetic image by synthesizing the angio image and the true lumen image and outputting the synthetic image; A computer program comprising:

14. 14. A computer program according to claim 13, comprising: The three-dimensional position information of the true lumen includes information regarding a width of the true lumen, The computer program includes a step of generating a true lumen image that represents the true lumen and has a width according to three-dimensional position information of the true lumen.

15. A computer program according to claim 13 or claim 14, comprising: in the true lumen image generating step, when the FPD is moved to a second imaging position different from the first imaging position and an image of the target blood vessel is captured from the FPD, the angio-image captured at the second imaging position is acquired again, and a true lumen image representing the true lumen at a position and orientation corresponding to the reacquired angio-image is generated using position information of the second imaging position and three-dimensional position information of the true lumen; The image synthesis step regenerates a synthetic image by synthesizing the reacquired angio image and the regenerated true lumen image, and outputs the synthetic image.

16. A computer program according to claim 13 or claim 14, further comprising: an angio-image acquiring step of acquiring a first angio-image captured by the FPD disposed at a first position and a second angio-image captured by the FPD disposed at a second position different from the first position; an ultrasonic image acquiring step of acquiring an ultrasonic image of the inside of the target blood vessel captured by an ultrasonic sensor; Equipped with The first angio image includes: The ultrasonic sensor is disposed at a first landmark position in the target blood vessel; a medical device different from the ultrasonic sensor that is disposed at a second landmark position in the target blood vessel; The second angio image includes: The ultrasonic sensor is disposed at the first landmark position in the target blood vessel, The ultrasonic image is an image captured in a state where the ultrasonic sensor is placed at the first mark position, and the ultrasonic image includes: The target blood vessel; the medical device positioned at the second landmark location within the target blood vessel; A computer program for acquiring three-dimensional position information of the true lumen using position information of the first position, the first angio-image, position information of the second position, the second angio-image, and the ultrasound image in the true lumen information acquisition step.

17. 17. A computer program according to claim 16, comprising: In the true lumen information acquisition step, acquiring a position of the ultrasonic sensor using an image of the ultrasonic sensor included in the first angio-image and the second angio-image; Associating a positional relationship between the first angio-image and the ultrasound image using images of the medical device included in the first angio-image and the ultrasound image; acquiring position information of the true cavity from the ultrasound image; A computer program that acquires three-dimensional position information of the true lumen using the acquired position of the ultrasonic sensor and position information of the true lumen in the ultrasonic image captured by the ultrasonic sensor.

18. A computer program according to claim 13 or claim 14, further comprising: an angio-image acquiring step of acquiring a first angio-image captured by the FPD disposed at a first position and a second angio-image captured by the FPD disposed at a second position different from the first position; Equipped with The first angio image includes: The true lumen of the target blood vessel; a medical device positioned at a first landmark location within the target blood vessel; The second angio image includes: The true lumen of the target blood vessel; the medical device positioned at the first landmark location within the target blood vessel; In the true lumen information acquisition step, A computer program for obtaining three-dimensional position information of the true lumen using an image of the medical device and an image of the true lumen contained in the first angio image and the second angio image.

Images