Surgical assistance device, angiography device, surgical assistance system, control method therefor, and computer program

Abstract

A surgical assistance device is provided with: an angiographic image acquisition unit that sequentially acquires angiographic images indicating a target blood vessel into which a medical device is inserted for each predetermined interval, with the time interval corresponding to the beat cycle of the heart being set as the predetermined interval; and an image correction unit that sequentially corrects the angiographic images sequentially acquired by the angiographic image acquisition unit, and generates a corrected angiographic image in which the angiographic image to be corrected is corrected. The position of the specifying section of the medical device included in the angiographic image to be corrected approaches the position of the specifying section of the medical device included in the angiographic image acquired temporally prior to the angiographic image to be corrected.

Description

Technical Field

[0001] The present invention relates to a technique for assisting surgery. Background Art

[0002] In recent years, FPD (Flat Panel Detector) has been used in blood vessel photography for inspection and treatment. FPD has an X-ray tube device and an X-ray plane detector, and is a device for acquiring X-ray images of blood vessels. The X-ray image acquired by FPD is also called an "angiography image", and the photography device equipped with FPD is also called an "angiography device". For example, Patent Document 1 describes the following: an added correction image is generated by adding a correction image and at least one correction image generated before the correction image, and the images are displayed sequentially, thereby performing real-time fixed dynamic image display control on a stent that moves due to pulsation.

[0003] Prior art literature

[0004] Patent Literature

[0005] Patent Document 1: Japanese Patent Application Publication No. 2015-165942 Summary of the invention

[0006] Problems to be solved by the invention

[0007] Among them, there are cases where the blood vessels are blocked by occlusive objects, such as chronic total occlusion (CTO). In such cases, after the medical device is moved from the true lumen to the false lumen, the occlusive object in the blood vessel is removed by the subintimal method, etc., in which the medical device is moved from the false lumen to the true lumen again, thereby reopening the blood vessel. When performing such an operation, the operator visually recognizes the angiographic image captured by the angiographic device, confirms the position and direction of the front end of the medical device in the blood vessel, and performs the operation.

[0008] However, the patient's body is constantly moving due to the pulsation of the heart, the volume change of the thorax accompanying breathing, and the like. Furthermore, since the medical device is inserted into the patient's body, the front end of the medical device on the angiographic image is not stationary, but moves continuously with the movement of the patient's body (in other words, the position of the front end of the medical device on the angiographic image fluctuates). Such jittering of the front end of the medical device on the angiographic image makes it difficult for the operator to grasp the position and orientation of the front end of the medical device in the blood vessel. Such a problem is particularly significant when the CTO is generated in the coronary artery of the heart which is strongly affected by the pulsation. In this regard, a technology is described in Patent Document 1, which uses an additive correction image to perform fixed dynamic image display control of a stent that moves due to the pulsation in real time, but there is still room for improvement in the technology described in Patent Document 1.

[0009] In addition, such a problem is not limited to the reopening of CTO, but is common to all the inspections and treatments of the lymphatic system, biliary system, urinary system, respiratory system, digestive system, secretory glands and reproductive organs, in which a medical device is inserted into a lumen of a living body and the medical device is referenced on angiographic images while performing surgery. In addition, such a problem is not limited to the front end of the medical device, but is common to all the technologies that can reduce the jitter of the position of the designated part (a certain part) of the medical device on the angiographic image.

[0010] An object of the present invention is to reduce the jitter of the position of a designated part of a medical device on an angiographic image.

[0011] Solutions to Solve Problems

[0012] The present invention has been made to solve at least a part of the above-mentioned problems, and can be implemented as the following aspects.

[0013] (1) According to one embodiment of the present invention, there is provided a surgical assistance device. The surgical assistance device comprises: an angiographic image acquisition unit that sets a time interval corresponding to a heart beat cycle as a predetermined interval and sequentially acquires angiographic images representing a target blood vessel in which a medical device is inserted at each predetermined interval; and an image correction unit that sequentially corrects the angiographic images sequentially acquired by the angiographic image acquisition unit and generates a corrected angiographic image obtained by correcting the correction target angiographic image so that a position of a designated portion of the medical device included in the correction target angiographic image approaches a position of the designated portion of the medical device included in the angiographic image acquired earlier in time than the correction target angiographic image.

[0014] According to this structure, the angiographic image acquisition unit sets the time interval corresponding to the heart's pulsation cycle as a predetermined interval, and sequentially acquires angiographic images at each predetermined interval. The heart regularly repeats expansion and contraction (hereinafter also referred to as "expansion and contraction") according to the pulsation cycle, so by acquiring angiographic images at each predetermined interval corresponding to the pulsation cycle, it is possible to acquire angiographic images with equal (uniform) expansion and contraction states of the heart. As a result, the angiographic image acquisition unit can reduce the jitter of the position of the medical device designation portion on the angiographic image caused by the pulsation. In addition, the patient's body is constantly moving due to reasons such as the volume change of the thorax accompanying breathing in addition to the pulsation of the heart. According to this structure, the image correction unit generates a corrected angiographic image after correcting the angiographic image of the correction object, so that the position of the medical device designation portion included in the angiographic image of the correction object in the angiographic images sequentially acquired by the angiographic image acquisition unit is close to the position of the medical device designation portion included in the angiographic image acquired earlier in time than the angiographic image of the correction object. Therefore, the image correction unit can reduce the jitter of the position of the medical device designation unit on the angiographic image caused mainly by factors other than the pulsation of the heart. In this way, the surgical assistance device of this embodiment separately (individually) reduces the jitter caused by the pulsation by the angiographic image acquisition unit and reduces the jitter caused by factors other than the pulsation by the image correction unit, thereby reducing the jitter of the position of the medical device designation unit on the angiographic image with high precision. As a result, according to the surgical assistance device of this embodiment, the operator can correctly grasp the position of the medical device designation unit in the target blood vessel, thereby achieving improved surgical accuracy, shortened surgical time, and reduced patient burden.

[0015] (2) In the surgical assisting device of the above aspect, the angiographic image acquiring unit may sequentially acquire the angiographic images with a time interval between two temporally consecutive electrocardiographic data previously acquired by an electrocardiograph being set to a time interval corresponding to a beat cycle of the heart.

[0016] According to this configuration, the predetermined interval is calculated based on the time interval between two temporally consecutive electrocardiogram data acquired in the past by the electrocardiogram measurement device. Therefore, the angiographic image acquisition unit can acquire an angiographic image with reduced jitter due to pulsation with high accuracy based on the heart pulsation cycle of each patient.

[0017] (3) In the surgical assisting device of the above aspect, the angiographic image acquiring unit may sequentially acquire the angiographic images with a time interval between two temporally consecutive electrocardiographic data just acquired by the electrocardiograph being set to a time interval corresponding to a beat cycle of the heart.

[0018] According to this configuration, the predetermined interval is calculated based on the time interval of two temporally consecutive electrocardiogram data just acquired by the electrocardiogram measuring device. Therefore, the angiographic image acquisition unit can acquire an angiographic image with reduced jitter caused by pulsation with high accuracy based on the pulsation cycle of each patient's heart, that is, the pulsation cycle immediately before the current pulsation cycle.

[0019] (4) In the surgical assisting device of the above-mentioned method, it may also be that the electrocardiogram data is electrocardiogram waveform data, and when the data acquired relatively later of the two consecutive electrocardiogram waveform data is taken as the nth electrocardiogram waveform data and the data acquired relatively earlier is taken as the n-1th electrocardiogram waveform data, the angiography image acquisition unit sequentially acquires the angiography images with the time interval between the time point tn-1 at which a specific waveform appears in the n-1th electrocardiogram waveform data and the time point tn at which the specific waveform appears in the nth electrocardiogram waveform data being set to a time interval corresponding to the beating cycle of the heart.

[0020] According to this configuration, the predetermined interval is calculated based on the time interval between the time point tn-1 at which the specific waveform appears in the n-1th electrocardiogram waveform data and the time point tn at which the specific waveform appears in the nth electrocardiogram waveform data in two temporally continuous electrocardiogram waveform data acquired by the electrocardiogram measurement device. Therefore, by defining the specific waveform as the time when the expansion and contraction state of the heart is suitable for visually confirming the expansion and contraction state of the medical device (for example, the specific waveform = R wave), the angiographic image acquisition unit can acquire an angiographic image when the expansion and contraction state of the heart is suitable for visually confirming the expansion and contraction state of the medical device. As a result, the visibility of the medical device in the angiographic image can be improved.

[0021] (5) In the surgical assisting device of the above-mentioned method, the image correction unit may correct the angiography image of the correction object so that the position of the designated part of the medical device included in the angiography image of the correction object is close to the position of the designated part of the medical device included in the corrected angiography image after the correction of the angiography image acquired immediately before the angiography image of the correction object.

[0022] According to this configuration, the angiographic image to be corrected is corrected using the position of the designated portion of the medical device included in the corrected angiographic image obtained by correcting the angiographic image immediately before the angiographic image to be corrected. Therefore, the positional shift of the designated portion of the medical device in the latest correction result (corrected angiographic image K′ for the angiographic image K to be corrected) and the immediately preceding correction result (corrected angiographic image K-1′ for the angiographic image K-1 obtained immediately before the angiographic image to be corrected) can be reduced, and a more natural corrected angiographic image can be provided.

[0023] (6) In the above-mentioned surgical assisting device, it may also be provided with: an electrocardiogram information acquiring unit, which acquires electrocardiogram data from an electrocardiogram measuring device; and a target image acquiring unit, which extracts the angiography image of each of the predetermined intervals from continuous angiography images representing the target blood vessel continuously photographed at intervals shorter than the predetermined interval, through an FPD (flat panel detector).

[0024] According to this configuration, the surgical assisting device further includes a target image acquisition unit that extracts a predetermined interval of angiographic images corresponding to a heart beat cycle from the continuous angiographic images showing the target blood vessel continuously captured by the FPD. In this way, the angiographic device including the FPD only needs to continuously supply the angiographic images of the target blood vessel to the surgical assisting device (in other words, supply the continuous angiographic images), thereby improving the scalability of the system composed of the surgical assisting device and the angiographic device.

[0025] (7) In the above-mentioned surgical assisting device, it may also be provided with: a true lumen information acquiring unit, which acquires the three-dimensional position information of the true lumen existing in the target blood vessel; a true lumen image generating unit, which generates a true lumen image representing the true lumen; and an image synthesizing unit, which generates a synthesized image by synthesizing the corrected angiography image and the true lumen image, and outputs the synthesized image, wherein the true lumen image generating unit acquires from the image correction unit the corrected angiography image of the angiography image acquired by photographing the target blood vessel using an FPD (flat panel detector) configured at an arbitrary photographing position, that is, the corrected angiography image of the angiography image at each of the predetermined intervals, and uses the position information of the arbitrary photographing position and the three-dimensional position information of the true lumen to generate a true lumen image of the true lumen representing the position and posture corresponding to the corrected angiography image.

[0026] According to this configuration, the true lumen image generation unit can generate a true lumen image of the true lumen representing the position and posture corresponding to the corrected angiographic image using the position information of an arbitrary shooting position of the acquired angiographic image and the three-dimensional position information of the true lumen acquired by the true lumen information acquisition unit. That is, the true lumen image generation unit can generate a true lumen image representing the image of the true lumen based on the three-dimensional position information of the true lumen even when the contrast agent does not flow into the target true lumen or when the contrast agent does not flow through the blood vessel. In addition, the image synthesis unit generates a synthesized image synthesized by synthesizing the corrected angiographic image at an arbitrary shooting position and the true lumen image representing the image of the true lumen, and outputs the synthesized image, so that the image of the true lumen of the blood vessel can be displayed on the corrected angiographic image. Therefore, by confirming the synthesized image, the operator can confirm the positional relationship between the medical device on the corrected angiographic image and the true lumen on the true lumen image while advancing the operation. As a result, the operator can correctly grasp the position of the true lumen in the target blood vessel, thereby achieving improved accuracy of the operation, shortened time required for the operation, and reduced burden on the patient.

[0027] (8) In the surgical assisting device of the above-mentioned method, it may also be that the angiographic image acquisition unit sequentially acquires the angiographic images of each of the predetermined intervals, that is, the first angiographic image acquired by photographing the FPD arranged at a first position, and the second angiographic image acquired by photographing the FPD arranged at a second position different from the first position, the image correction unit sequentially generates the corrected angiographic image for the first angiographic image, that is, the first corrected angiographic image, and the corrected angiographic image for the second angiographic image, that is, the second corrected angiographic image, and the true lumen information acquisition unit uses an ultrasonic image, the position information of the first position, the first corrected angiographic image, the position information of the second position, and the second corrected angiographic image to acquire the three-dimensional position information of the true lumen, the ultrasonic image being an image acquired by photographing the inside of the target blood vessel by an ultrasonic sensor, that is, an ultrasonic image acquired at each of the predetermined intervals.

[0028] According to this configuration, the true lumen information acquisition unit acquires the three-dimensional position information of the true lumen using the first and second corrected angiography images that achieve reduction in jitter caused by the pulsation of the heart and reduction in jitter caused by factors other than the pulsation of the heart (such as changes in the volume of the thorax accompanying breathing). Therefore, compared with the case where the angiography images that do not achieve jitter reduction are used, the three-dimensional position information of the true lumen can be acquired with high accuracy. In addition, since the shooting intervals of the ultrasonic image and the first and second corrected angiography images can be synchronized with a predetermined interval, the three-dimensional position information of the true lumen can be acquired with higher accuracy.

[0029] (9) According to one embodiment of the present invention, there is provided an angiography apparatus comprising: an FPD (flat panel detector) having an X-ray tube device and an X-ray plane detector; and a target image acquisition unit that causes the FPD to capture a target blood vessel in which a medical device is inserted at predetermined intervals corresponding to a heart beat cycle, thereby acquiring an angiography image representing the target blood vessel, and outputting the acquired image.

[0030] According to this structure, the object image acquisition unit sets the time interval corresponding to the heart's pulsation cycle as a predetermined interval, photographs the object blood vessel in which the medical device is inserted at each predetermined interval, thereby acquiring an angiographic image representing the object blood vessel, and outputs the acquired image. The heart regularly expands and contracts repeatedly according to the pulsation cycle, so by acquiring an angiographic image at each predetermined interval corresponding to the pulsation cycle, it is possible to acquire an angiographic image with an equal (uniform) expansion and contraction state of the heart. As a result, the object image acquisition unit can reduce the jitter of the position of the medical device designated part on the angiographic image caused by pulsation. As a result, according to the angiographic device of this embodiment, the operator can correctly grasp the position of the medical device designated part in the object blood vessel, thereby achieving improved surgical accuracy, shortened surgical time, and reduced burden on patients.

[0031] (10) In the above-mentioned method, the angiography device may also include: an electrocardiogram information acquisition unit, which acquires electrocardiogram data from an electrocardiogram measurement device, and the object image acquisition unit sets the time interval between two temporally consecutive electrocardiogram data acquired by the electrocardiogram information acquisition unit in the past to a time interval corresponding to the heart's beating cycle, i.e., the predetermined interval.

[0032] According to this configuration, the predetermined interval is calculated based on the time interval between two temporally consecutive electrocardiogram data acquired in the past by the electrocardiograph. Therefore, the target image acquisition unit can output an angiographic image with reduced jitter due to pulsation with high accuracy according to the heart pulsation cycle of each patient.

[0033] (11) In the above-mentioned method, the angiography device may also be provided with an electrocardiogram information acquisition unit, which acquires electrocardiogram data from an electrocardiogram measurement device, and the object image acquisition unit sets the time interval between two temporally consecutive electrocardiogram data just acquired by the electrocardiogram information acquisition unit to a time interval corresponding to the heart's beating cycle, i.e., the predetermined interval.

[0034] According to this configuration, the predetermined interval is calculated based on the time interval of two temporally consecutive electrocardiogram data just acquired by the electrocardiogram measuring device. Therefore, the target image acquisition unit can output an angiography image with reduced jitter caused by pulsation with high accuracy based on the pulsation cycle of each patient's heart, that is, the pulsation cycle immediately before the current pulsation cycle.

[0035] (12) In the above-mentioned angiography device, the electrocardiogram data may be electrocardiogram waveform data, and when the data acquired relatively later of the two consecutive electrocardiogram waveform data is taken as the nth electrocardiogram waveform data and the data acquired relatively earlier is taken as the n-1th electrocardiogram waveform data, the object image acquisition unit sets the time interval between the time point tn-1 at which a specific waveform appears in the n-1th electrocardiogram waveform data and the time point tn at which the specific waveform appears in the nth electrocardiogram waveform data to be the time interval corresponding to the heart's beat cycle, i.e., the predetermined interval.

[0036] According to this configuration, the predetermined interval is calculated based on the time interval between the time point tn-1 at which the specific waveform appears in the n-1th electrocardiogram waveform data and the time point tn at which the specific waveform appears in the nth electrocardiogram waveform data in two temporally continuous electrocardiogram waveform data acquired by the electrocardiogram measurement device. Therefore, by defining the specific waveform as the time when the expansion and contraction state of the heart is suitable for visually confirming the expansion and contraction state of the medical device (for example, the specific waveform = R wave), the target image acquisition unit can output the angiographic image when the expansion and contraction state of the heart is suitable for visually confirming the expansion and contraction state of the medical device. As a result, the visibility of the medical device in the angiographic image can be improved.

[0037] (13) According to one aspect of the present invention, there is provided a surgery support system. The surgery support system comprises a surgery support device of the above aspect and an angiography device of the above aspect, wherein the target image acquisition unit of the angiography device sequentially sends the angiography images at each predetermined interval to the surgery support device, the angiography image acquisition unit of the surgery support device sequentially acquires the angiography images at each predetermined interval from the angiography device, and the image correction unit of the surgery support device sets the latest angiography image among the plurality of angiography images acquired by the angiography image acquisition unit as the angiography image to be corrected.

[0038] According to this configuration, the target image acquisition unit of the angiography device sets the time interval corresponding to the heart beat cycle as the predetermined interval, and sequentially transmits the angiography images of each predetermined interval to the surgery support device. The angiography image acquisition unit of the surgery support device sequentially acquires the angiography images of each predetermined interval from the angiography device, thereby distributing the processing load in the system to the angiography device and the surgery support device, and suppressing the occurrence of processing delay caused by the increase in processing load. In addition, the image correction unit of the surgery support device sets the latest angiography image among the plurality of angiography images acquired by the angiography image acquisition unit as the angiography image to be corrected, thereby being able to output the corrected angiography image for the latest angiography image without delay.

[0039] (14) In the surgical assistance system of the above-mentioned method, it may also be that the surgical assistance device further includes: a true lumen information acquisition unit, which acquires three-dimensional position information of the true lumen existing in the target blood vessel; a true lumen image generation unit, which generates a true lumen image representing the true lumen; and an image synthesis unit, which generates a synthesized image by synthesizing the corrected angiography image and the true lumen image, and outputs the synthesized image, wherein the true lumen image generation unit acquires from the image correction unit the corrected angiography image of the angiography image acquired by photographing the target blood vessel using the FPD configured at an arbitrary photographing position, that is, the corrected angiography image of the angiography image at each predetermined interval, and uses the position information of the arbitrary photographing position and the three-dimensional position information of the true lumen to generate a true lumen image of the true lumen representing the position and posture corresponding to the corrected angiography image.

[0040] According to this structure, the operator can check the synthetic image and perform the operation while checking the positional relationship between the medical device on the corrected angiography image and the true lumen on the true lumen image. As a result, the operator can correctly grasp the position of the true lumen in the target blood vessel, thereby improving the accuracy of the operation, shortening the time required for the operation, and reducing the burden on the patient.

[0041] (15) In the surgical assistance system of the above-mentioned method, it may also be that the angiographic image acquisition unit sequentially acquires the angiographic images of each of the predetermined intervals, that is, the first angiographic image acquired by photographing the FPD configured at a first position, and the second angiographic image acquired by photographing the FPD configured at a second position different from the first position; the image correction unit sequentially generates the corrected angiographic image for the first angiographic image, that is, the first corrected angiographic image, and the corrected angiographic image for the second angiographic image, that is, the second corrected angiographic image; the true cavity information acquisition unit uses an ultrasonic image, the position information of the first position, the first corrected angiographic image, the position information of the second position, and the second corrected angiographic image to acquire the three-dimensional position information of the true cavity; the ultrasonic image is an image acquired by photographing the inside of the target blood vessel by an ultrasonic sensor, that is, an ultrasonic image acquired at each of the predetermined intervals.

[0042] According to this structure, the three-dimensional position information of the true cavity can be obtained with high accuracy compared to the case where angiographic images without jitter reduction are used. In addition, since the shooting intervals of the ultrasonic image and the first and second corrected angiographic images can be synchronized at a predetermined interval, the three-dimensional position information of the true cavity can be obtained with higher accuracy.

[0043] The present invention is made to solve at least a part of the above-mentioned problems, and can be implemented in the following ways. For example, it can be implemented in the form of an angiography device, a surgical assisting device, a server device or a robot that realizes the functions of these devices, a system including these devices, a computer program that realizes the functions of these devices and systems, a server device for distributing the computer program, a non-temporary storage medium storing the computer program, etc. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 It is an explanatory diagram illustrating the configuration of a surgery support system.

[0045] Figure 2 This is a flowchart showing an example of a processing procedure of the first processing executed by the target image acquisition unit.

[0046] Figure 3 This is a diagram for explaining the first process and the second process.

[0047] Figure 4 : is a flowchart showing an example of the processing procedure of the second process executed by the surgery support device.

[0048] Figure 5 It is a figure explaining the second process.

[0049] Figure 6 It is a figure explaining the second process.

[0050] Figure 7 It is an explanatory diagram illustrating the configuration of a surgery support system according to the second embodiment.

[0051] Figure 8 It is a diagram for explaining the angiographic image acquisition unit according to the second embodiment.

[0052] Fig. 9 This is a diagram for explaining an example of a method in which the true cavity information acquisition unit acquires three-dimensional position information of the true cavity.

[0053] Fig.10 It is a diagram for explaining a true lumen image generating unit and an image synthesizing unit.

[0054] Fig.11 It is an explanatory diagram illustrating the configuration of a surgery support system according to a third embodiment.

[0055] Fig.12 This is a flowchart showing an example of the processing procedure of the first processing in the third embodiment.

[0056] Fig.13 It is an explanatory diagram illustrating the configuration of a surgery support system according to a fourth embodiment.

[0057] Fig.14 This is a flowchart showing an example of the processing procedure of the first processing in the fourth embodiment.

[0058] Fig.15 It is a diagram for explaining the first process and the second process of the fourth embodiment. DETAILED DESCRIPTION

[0059] <First Embodiment>

[0060] Figure 1 1 is an explanatory diagram illustrating the structure of a surgery support system 1. The surgery support system 1 is a system for assisting examination and treatment. The surgery support system 1 includes a surgery support device 10, an angiography device 20 having an FPD (Flat Panel Detector), a display device 30, a workbench 40, an operation unit 50, and an electrocardiogram measurement device 60. Hereinafter, the image of the target blood vessel captured by the FPD is also referred to as an "angiography image."

[0061] The surgical support system 1 of the present embodiment can obtain an angiographic image in which the jitter of the position of the designated part of the medical device on the angiographic image caused by the pulsation of the heart is reduced, through the "first processing" described later. In addition, the surgical support system 1 can generate a corrected angiographic image in which the jitter of the position of the designated part of the medical device on the angiographic image caused mainly by factors other than the pulsation of the heart is reduced, through the "second processing" described later. In addition, factors other than the pulsation of the heart are a general term for factors other than pulsation, such as the patient's body movement corresponding to the volume change of the thoracic cavity accompanying breathing, the patient's body movement corresponding to the swallowing action, etc. In the second processing, as described above, although the jitter caused by the main reason other than the pulsation is reduced, the jitter caused by the pulsation (that is, the jitter caused by the pulsation remaining after the first processing) can also be reduced.

[0062] In addition, "medical equipment" refers to any equipment used in surgery, such as a guide wire, a through guide wire, a plasma guide wire, a catheter, and an imaging sensor. "Medical equipment designated portion" refers to a specific part of a medical device. The medical equipment designated portion can be determined arbitrarily, for example, it can be set to a part of the medical equipment that is important for inspection and treatment (such as the front end), or it can be set to a characteristic part of the medical equipment (for example, a protrusion, a part with a mark, etc.). The medical equipment designated portion is also referred to as the "designated portion". In the following description, a catheter is exemplified as a medical device, and the front end piece of the catheter is exemplified as a designated portion. In addition, "target blood vessel" refers to a blood vessel that is the object of inspection or treatment, but the surgical support system 1 is not limited to the vascular system, and can also be used in the lumen of a biological body such as the lymphatic system, biliary system, urinary system, respiratory system, digestive organ system, secretory glands and reproductive organs.

[0063] exist Figure 1 In the figure, mutually orthogonal XYZ axes are shown. The X axis corresponds to the width direction of the angiography device 20, the Y axis corresponds to the height direction of the angiography device 20, and the Z axis corresponds to the depth direction of the angiography device 20. In the following description, the patient ( Figure 1 The direction in which the head 92 of the human body 90 is located is referred to as the "Z-axis direction", or "Z" for short. The three-dimensional space formed by the three-dimensional coordinates (XYZ coordinates) composed of the X, Y, and Z axes is referred to as the XYZ three-dimensional space. In addition, the origin O of the XYZ three-dimensional space (XYZ coordinates) is set to the position of the heart 91 of the human body 90.

[0064] The surgical assisting device 10 is a device that generates and outputs a corrected angiographic image by executing the second processing described later. A "corrected angiographic image" refers to an image in which the jitter of the position of the designated part of the medical device on the angiographic image caused mainly by factors other than the pulsation of the heart is reduced. The surgical assisting device 10 is configured to include a CPU, a ROM, and a RAM. The CPU executes a computer program stored in the ROM or the RAM to realize the functions of the main control unit 11, the angiographic image acquisition unit 12, the image correction unit 13, and the display control unit 14. In addition, the surgical assisting device 10 has a storage unit not shown in the figure. The storage unit is composed of a hard disk, a flash memory, a memory card, etc. The surgical assisting device 10 is electrically connected to the control device 29, the display device 30, and the operating unit 50 of the angiographic device 20, respectively.

[0065] The main control unit 11 transmits and receives information with the control device 29 , the display device 30 , and the operation unit 50 of the angiography apparatus 20 , and controls the entire surgery support apparatus 10 .

[0066] The angiographic image acquisition unit 12 sets a time interval corresponding to the heart beat cycle as a predetermined interval, and sequentially acquires angiographic images representing the target blood vessel in which the medical device is inserted (in other words, angiographic images obtained by the first processing) for each predetermined interval. The process (step) performed by the angiographic image acquisition unit 12 is also referred to as an angiographic image acquisition process (step).

[0067] The image correction unit 13 performs a "second process" of sequentially correcting the angiographic images sequentially acquired by the angiographic image acquisition unit 12. The second process is the process indicated by the following a1 and a2. The details will be described later.

[0068] The angiographic image K to be corrected is corrected so that (a1) the position of the designated part of the medical device included in the angiographic image K to be corrected is

[0069] (a2) The position of the designated part of the medical device included in the angiographic image Kx (x is a natural number) acquired earlier in time than the angiographic image K to be corrected is approached to generate a corrected angiographic image K'. The process (step) executed by the image corrector 13 is also referred to as an image correction process (step).

[0070] The display control unit 14 outputs the corrected angiographic image K' generated by the image correcting unit 13 to the display device 30. The process (step) executed by the display control unit 14 is also referred to as a display control process (step).

[0071] The angiography device 20 has an FPD, acquires X-rays that have passed through the human body, and converts them into digital signals, thereby acquiring an image (angiography image). The angiography device 20 has a first FPD 21, a first X-ray tube device 22, a first C-shaped arm 23, a first support unit 24, a second FPD 25, a second X-ray tube device 26, a second C-shaped arm 27, a second support unit 28, and a control device 29.

[0072] The first FPD 21 includes an X-ray plane detector, which converts X-rays incident from the first X-ray tube device 22 into electrical signals, performs A / D (analog / digital) conversion, and generates an X-ray image. The first X-ray tube device 22 receives a high voltage output from an X-ray high voltage device (not shown) and irradiates an X-ray beam. Figure 1 As shown by the thick dotted line extending in the Y-axis direction, the X-ray beam irradiated from the first X-ray tube device 22 enters the first FPD21 through the human body 90. The first C-arm 23 is a C-shaped arm (support) that fixes the first FPD21 and the first X-ray tube device 22 in opposing positions. The first support part 24 rotatably supports the first C-arm 23. That is, the first FPD21 and the first X-ray tube device 22 can move to any shooting position around the human body 90 lying on the bed 41 in a state where they are fixed in opposing positions by the first C-arm 23. Hereinafter, the first FPD21 and the first X-ray tube device 22 fixed to the first C-arm 23 will also be referred to as the "first FPD21".

[0073] The structure of the second FPD 25 is the same as that of the first FPD 21. The structure of the second X-ray tube device 26 is the same as that of the first X-ray tube device 22. Figure 1 As shown by the thick dotted line extending in the X-axis direction, the X-ray beam irradiated from the second X-ray tube device 26 enters the second FPD25 through the human body 90. The second C-arm 27 is a C-shaped arm (support) that fixes the second FPD25 and the second X-ray tube device 26 at opposing positions. The second support 28 rotatably supports the second C-arm 27. That is, the second FPD25 and the second X-ray tube device 26 can be moved to any imaging position around the human body 90 while being fixed at opposing positions by the second C-arm 27. Hereinafter, the second FPD25 and the second X-ray tube device 26 fixed to the second C-arm 27 will also be referred to as the "second FPD25".

[0074] The second FPD 25 is usually arranged in the normal direction of the first FPD 21. Figure 1As shown, when the first FPD 21 is located at a shooting position in the front direction of the human body 90 (the vertical direction of the human body 90 or the longitudinal direction of the human body 90), the second FPD 25 is located at a shooting position in the horizontal direction of the human body 90 (the transverse direction of the human body 90). In addition, the angiography device 20 is sometimes also referred to as "FPD", "FPD device", etc.

[0075] The control device 29 is configured to include a CPU, a ROM, and a RAM, and the CPU executes a computer program stored in the ROM and the RAM to control the entire angiography device 20. The control device 29 is electrically connected to the surgery support device 10, the first FPD 21, the second FPD 25, the first support portion 24, the second support portion 28, the display device 30, the table 40, the operation portion 50, and the electrocardiogram measurement device 60. The control device 29 transmits and receives information to and from the surgery support device 10, the display device 30, the table 40, the operation portion 50, and the electrocardiogram measurement device 60. The control device 29 drives the first support portion 24 to rotate the first C-shaped arm 23 and drives the second support portion 28 to rotate the second C-shaped arm 27 according to the operation from the operation portion 50. In addition, the control device 29 changes the height of the bed 41 by extending and retracting the telescopic portion 42 and changes the position of the bed 41 by moving the table 40 in the Z-axis direction according to the operation from the operation portion 50.

[0076] The control device 29 also realizes the functions of the target image acquisition unit 291 and the electrocardiogram information acquisition unit 292 by having the CPU execute computer programs stored in the ROM and RAM.

[0077] The target image acquisition unit 291 performs a "first process" in which a time interval corresponding to the heart beat cycle is set to a predetermined interval, the first FPD 21 (or the second FPD 25) images the target blood vessel at a predetermined interval, and an angiographic image of the target blood vessel acquired by the imaging is output to the surgery support device 10. The details of the first process will be described later.

[0078] The electrocardiogram information acquisition unit 292 acquires electrocardiogram data measured by the electrocardiogram measurement device 60 from the electrocardiogram measurement device 60. The "electrocardiogram data" of this embodiment includes both information shown in b1 and b2 below. In addition, the electrocardiogram information acquisition unit 292 may acquire only one of the electrocardiogram waveform data of b1 and the trigger signal of b2 from the electrocardiogram measurement device 60.

[0079] (b1) ECG waveform data including P wave, Q wave, R wave, S wave, T wave, and U wave,

[0080] (b2) A trigger signal that turns on (ON) when any specific waveform (e.g., R wave) in the electrocardiogram waveform data appears and turns off (OFF) when other waveforms appear. The electrocardiogram measurement device 60 may be configured to change the specific waveform that turns on (ON).

[0081] The display device 30 is connected to the control device 29 of the surgical assisting device 10 and the angiographic device 20, and functions as an output interface for the surgical assisting device 10 and the angiographic device 20. The display device 30 includes a monitor 31 and an arm 32. The monitor 31 is a "display unit" composed of a known component such as a liquid crystal display, smart glasses, or a projector. The arm 32 supports and fixes the monitor 31.

[0082] The workbench 40 is a table for allowing the human body 90 to lie down near the first FPD 21 and the second FPD 25. The workbench 40 includes a bed 41, a telescopic portion 42, and a leg portion 43. The bed 41 includes a mattress for allowing the human body 90 to lie down. The bed 41 is supported on the workbench 40 in a state where it can move in the Z-axis direction. The telescopic portion 42 is configured to be able to change the height of the bed 41 by telescoping in the Y-axis direction. The leg portion 43 supports the bed 41 and the telescopic portion 42. Figure 1 As shown by the dotted line in FIG, the human body 90 lies upward on the bed 41 with the head 92 placed on the side close to the first FPD 21 and the second FPD 25 and the feet 93 placed on the side away from the first FPD 21 and the second FPD 25. In this way, the image of the target blood vessel in the heart 91 can be easily obtained through the first FPD 21 and the second FPD 25.

[0083] The operation unit 50 is connected to the control device 29 of the surgery support device 10 and the angiography device 20, and functions as an input interface for the surgery support device 10 and the angiography device 20. The operation unit 50 is an "input unit" composed of well-known components such as a touch panel, an operation button, an operation lever, an operation switch, a keyboard, a mouse, a sound input unit, and a foot switch. In the example shown in the figure, the operation unit 50 is fixed to the workbench 40.

[0084] The electrocardiogram measuring device 60 is a device that collects and measures the electrocardiogram data after amplifying the minute electromotive force generated by the movement of the heart 91 through electrodes attached to the human body 90. As described above, the electrocardiogram measuring device 60 sends both the electrocardiogram waveform data of b1 and the trigger signal of b2 to the angiography device 20.

[0085] Figure 22 is a flowchart showing an example of the processing steps of the first processing executed by the target image acquisition unit 291. The first processing is a processing for setting time intervals corresponding to the heart beat cycle as predetermined intervals, imaging the target blood vessel in which the medical device is inserted at each predetermined interval, and acquiring an angiographic image showing the target blood vessel. Figure 2 The first process shown is started at an arbitrary trigger. The arbitrary trigger may be, for example, the power supply of the angiography device 20 is turned on, or the predetermined application provided by the control device 29 is started, or the second process in the surgery assisting device 10 (at Figure 4 In the following description, the FPD (the first FPD 21 or the second FPD 25) that acquires an angiographic image by imaging a target blood vessel is simply referred to as an FPD.

[0086] Figure 3 This is a diagram for explaining the first process and the second process. Figure 3 The trigger signal S2 of b2 in the electrocardiogram data acquired by the controller 29 from the electrocardiogram measuring device 60 is shown in time series in the uppermost part of FIG. Figure 3 Under the trigger signal S2, the electrocardiogram waveform data S1 of b1 in the electrocardiogram data obtained by the controller 29 from the electrocardiogram measuring device 60 is shown in time series. The moment when the trigger signal S2 becomes 1, that is, the moment when the R wave appears in the electrocardiogram waveform data S1 is also called "Wn time point (n is a natural number)". Figure 3 In the figure, each Wn time point is marked with a dotted line extending in the vertical direction of the paper, and the time point that is the same as each Wn time point is called "time point tn (n is a natural number)". Figure 3 Below the electrocardiogram waveform data S1, the calculation of the predetermined interval in the control device 29 and the time t at which the control device 29 causes the FPD to image the subject blood vessel and acquire the angiographic image are illustrated along the time series. Figure 3 At time t, angiographic images Vn (n is a natural number) acquired by photographing the FPD are shown along a time series diagram. Figure 3 Below the angiographic image Vn of FIG. 1 , a corrected angiographic image Vn′ after the second process described later and corrected by the surgery support device 10 is shown along the time series. Figure 3 , the case where the interval between Wn and Wn-1 (in other words, the interval between time point tn and time point tn-1) is constant is illustrated. However, in the actual human body 90, there are cases where the interval between Wn and Wn-1 deviates slightly.

[0087] Below, use Figure 2 and Figure 3 The first process is described below. Figure 2In step S100, the object image acquisition unit 291 uses one of the trigger signal S2 in the electrocardiogram data and the electrocardiogram waveform data S1 to detect the W1 time point and the W2 time point. In addition, when the electrocardiogram waveform data S1 is used to detect the W1 and W2 time points, the object image acquisition unit 291 can detect the specific waveform (R wave) by performing pattern matching and peak detection processing on the electrocardiogram waveform data S1. If the trigger signal S2 is used, the processing load of the object image acquisition unit 291 (the processing load for detecting the specific waveform) can be reduced. After detecting the W1 and W2 time points, the object image acquisition unit 291 obtains Δt2 (Δt2=t2-t1) obtained by subtracting the time point t1 corresponding to W1 from the time point t2 corresponding to W2.

[0088] In step S102, the target image acquisition unit 291 causes the FPD to image the target blood vessel at time t (t=t2+Δt2) after Δt2 obtained in step S100 from time point t2 corresponding to W2, thereby acquiring an angiography image V1 of the target blood vessel. In step S104, the target image acquisition unit 291 transmits the acquired angiography image V1 to the surgery support device 10.

[0089] In step S106, the target image acquisition unit 291 substitutes 3 into a variable n and substitutes 2 into a variable m (m is a natural number), and transfers the process to step S108. Steps S100 to S106 are initial processes.

[0090] In step S108, the target image acquisition unit 291 detects the time point Wn using the trigger signal S2 in the electrocardiogram data and the electrocardiogram waveform data S1. The target image acquisition unit 291 obtains Δtn (Δtn=tn-tn-1) by subtracting the time point tn-1 corresponding to Wn from the time point tn corresponding to Wn.

[0091] For example, in the case of n=3, tn=t3, tn-1=t2. Therefore, Δt3=t3-t2. That is, tn in tn and tn-1 is the time point when the trigger signal S2=1 (on) in the electrocardiogram data acquired relatively later in two consecutive electrocardiogram data, and is the time point when a specific waveform appears in the electrocardiogram waveform data S1 in the electrocardiogram data acquired relatively later. On the other hand, tn-1 is the time point when the trigger signal S2=1 (on) in the electrocardiogram data acquired relatively earlier in two consecutive electrocardiogram data, and is the time point when a specific waveform appears in the electrocardiogram waveform data S1 in the electrocardiogram data acquired relatively earlier.

[0092] In step S110, the object image acquisition unit 291 determines whether the absolute value of Δtn |Δtn=tn-tn-1| obtained in step S108 is greater than a predetermined value. The predetermined value is a threshold value for determining whether the heart 91 of the human body 90 has not developed arrhythmia or bradycardia. An appropriate value is predetermined to determine the presence or absence of arrhythmia or bradycardia, and is stored in the control device 29. In addition, in step S110, a first threshold value for determining the presence or absence of arrhythmia and a second threshold value for determining the presence or absence of bradycardia may be compared. In addition, in step S110, it may be determined whether the absolute value of Δtn |Δtn=tn-tn-1| obtained in step S108 is less than a third threshold value for determining the presence or absence of tachycardia.

[0093] When the absolute value of Δtn is greater than the predetermined value (step S110: Yes), in step S112, the object image acquisition unit 291 sends a warning to the surgery support device 10. The surgery support device 10 that receives the warning prompts the operator to pay attention by displaying a warning message on the display device 30. On the other hand, when the absolute value of Δtn is less than the predetermined value (step S110: No), the object image acquisition unit 291 transfers the process to step S114.

[0094] In step S114, the target image acquisition unit 291 causes the FPD to image the target blood vessel at time t (t=tn+Δtn) after Δtn obtained in step S108 from the time point tn corresponding to Wn, thereby acquiring the angiographic image Vm of the target blood vessel. However, the target image acquisition unit 291 sends an operation command to the FPD slightly before time t in consideration of the preparation time required for the start-up of the FPD, in order to correctly perform the imaging of the FPD at time t (i.e., the X-ray irradiation of the first X-ray tube device 22, and the X-ray detection and conversion of the first FPD 21). This point is also the same in the above-mentioned step S102.

[0095] In step S116 , the target image acquisition unit 291 transmits the acquired angiographic image Vm to the surgery support device 10 .

[0096] In step S118, the target image acquisition unit 291 adds 1 to the variable n and the variable m, respectively, and transfers the process to step S108 to repeat the above process. Figure 3 The angiographic images V1, V2, V3, ..., Vm shown are sequentially transmitted to the surgery support device 10. The surgery support device 10 uses the angiographic images V1, V2, V3, ..., Vm sequentially acquired from the angiographic device 20 to perform the second process. Figure 2The processing is repeated until a predetermined termination condition is satisfied. The termination condition may be, for example, power off of the angiography apparatus 20 or termination of an application program started at the start of the processing.

[0097] In this way, Figure 3 In the example of FIG. 1 , the target image acquisition unit 291 of the angiography device 20 acquires the angiography images V1, V2, V3, ..., Vm by causing the FPD to image the target blood vessels (in other words, the angiography images V1, V2, V3, ..., Vm acquired by the angiography image acquisition unit 12 of the surgery support device 10) by setting the time interval Δtn (Δtn=tn-tn-1) of two temporally consecutive electrocardiogram data acquired by the electrocardiogram measurement device 60 immediately before to a predetermined interval corresponding to the heart's pulsation cycle, and acquiring the angiography images at each predetermined interval. The heart 91 of the human body 90 regularly expands and contracts repeatedly according to the pulsation cycle. Therefore, the target image acquisition unit 291 of the present embodiment predicts that the pulsation cycle of the heart 91 is in the same phase as that at the time point tn-1 (in other words, the expansion and contraction state of the heart 91 is equal to that at the time point tn-1) after Δtn from the time point tn, and acquires the angiography image Vm by causing the FPD to acquire the angiography image Vm. In the present embodiment, since the specific waveform is the R wave that appears at the end of the diastolic period, the target image acquisition unit 291 can acquire the angiographic image Vm of the heart 91 that captures the state in which diastolic period ends, the movement slows down, and the changes (movements) of the blood vessels are small.

[0098] In the above example, the target image acquisition unit 291 causes the FPD to image the target blood vessel at each calculated time t (t=tn+Δtn) to acquire the angiography image Vm (still image). Therefore, compared with the case where the FPD is caused to image the target blood vessel at intervals shorter than Δtn to acquire a dynamic image of the angiography image, the time for irradiating the human body 90 with X-rays can be shortened, and the radiation dose of the human body 90 can be reduced.

[0099] Figure 4 : is a flowchart showing an example of the processing procedure of the second processing executed by the surgery support device 10. As described above, the second processing is the processing shown in the following a1 and a2.

[0100] The angiographic image K to be corrected is corrected so that (a1) the position of the designated part of the medical device included in the angiographic image K to be corrected is

[0101] (a2) The position of the designated part of the medical device included in an angiographic image Kx (x is a natural number) acquired earlier in time than the angiographic image K to be corrected is approached to generate a corrected angiographic image K′.

[0102] Figure 4The second process shown is started at an arbitrary trigger. The arbitrary trigger may be, for example, when the surgical assisting device 10 is powered on, when a predetermined application provided by the surgical assisting device 10 is started, or when the first process in the angiography device 20 (in Figure 2 The process starts in conjunction with the start of the process described later in the text.

[0103] Figure 5 as well as Figure 6 This is a diagram for explaining the second process. Figure 4 In step S200, the angiographic image acquisition unit 12 acquires an angiographic image V1 ( Figure 2 : The image sent in step S104). Figure 5 (A) shows an example of angiographic image V1.

[0104] In step S202, the display control unit 14 displays the angiography image V1 acquired in step S200 on the display device 30. Thereafter, the image correction unit 13 guides the operator to select a designated portion of the medical device (a designated portion: a front end portion, a protruding portion, a marked portion, etc.). Guidance may be provided by displaying text or an image containing a message urging selection on the display device 30, or by outputting a message urging selection by voice from the display device 30. The operator follows the guidance and selects the designated portion of the medical device shown in the angiography image V1 displayed on the display device 30. Figure 5 As shown in the dashed rectangle in (A), selection can be performed by performing range selection on a portion corresponding to a designated portion of the medical device in the angiographic image V1 displayed on the display device 30. Figure 5 In the example of (A), the "front end piece" of the medical device MD (for example, a catheter) is exemplified as the designated part FP. The selection may also be performed by other known means such as tapping or clicking.

[0105] After selecting the designated part FP of the medical device shown in the angiography image V1, the image correction unit 13 calculates the coordinates x and y of the center of gravity of the designated part FP on the angiography image V1, and sets the calculated coordinates x and y of the center of gravity as the center C (x, y) of the designated part FP. The image correction unit 13 extracts an image of a predetermined range (for example, a region corresponding to the designated part FP) including the selected designated part FP from the angiography image V1. Figure 5 (A) and stored in a storage unit (not shown) together with the coordinate information of the center C (x, y) of the designated part FP. The image stored here will also be referred to as the "designated part image" hereinafter. In addition, steps S200 and S202 are initial processing.

[0106] In step S204, the angiographic image acquisition unit 12 acquires an angiographic image Vm ( Figure 2 : The image sent in step S116). Figure 5 (B) shows an example of angiographic image Vm (m=2) which is an example of angiographic image Vm acquired later in time than the angiographic image V1.

[0107] In step S206, the image correction unit 13 uses the angiography image Vm acquired in step S204 to perform template matching on the designated part of the medical device. Specifically, the image correction unit 13 matches the image of the medical device shown in the angiography image Vm with the image of the designated part FP of the medical device included in the designated part image, such as Figure 5 As shown in (B), the specified part FPm (m=2) of the medical device shown in the angiographic image Vm is detected. Furthermore, the image correction unit 13 calculates the center Cm (xm, ym: m=2) of the specified part FPm on the angiographic image Vm, that is, the coordinates xm, ym of the centroid position.

[0108] In step S208, the image correction unit 13 performs image correction. Image correction is performed according to a1 and a2 above. Figure 5 as well as Figure 6 A specific example of image correction is described. Figure 6 (A) shows an example of angiographic image Vm (m=2) which is an example of angiographic image Vm acquired later in time than the angiographic image V1. Figure 6 (B) shows an example of angiographic image Vm (m=3) which is an example of angiographic image Vm acquired later in time than the angiographic image V1.

[0109] right Figure 5 Specific examples of (A) and (B) are described below.

[0110] The correction is performed by parallel translation of the angiographic image V2 to be corrected so that (a1) in step S208 executed for the first time, the image correction unit 13 moves the position x2, y2 of the designated part FP2 of the medical device MD included in the angiographic image V2 to be corrected.

[0111] Approach (a2) the position x, y of the designated part FP of the medical device MD included in the angiography image V1 acquired earlier in time than the angiography image V2 to be corrected, so as to generate a corrected angiography image V2'. Here, ""parallel movement of the image so that the position x2, y2 approaches the position x, y" means approaching the coordinates (x2, y2) to the coordinates (x, y), that is, parallel movement of the upper left end C0 (0, 0) of the image so that the difference between x2 and x becomes smaller and the difference between y2 and y becomes smaller. As a result, as the display range of the image in the angiography image before and after correction changes, the position of the medical device MD in the image changes.

[0112] right Figure 6 Specific examples of (A) and (B) are described below.

[0113] The correction is performed by parallel shifting the range of the angiographic image V3 to be corrected so that (a1) in step S208 executed for the second time, the image correcting unit 13 moves the position x3, y3 of the designated part FP3 of the medical device MD included in the angiographic image V3 to be corrected.

[0114] The corrected angiographic image V3' is generated by approaching (a2') the positions x2 and y2 of the designated part FP2 of the medical device MD included in the corrected angiographic image V2' corrected to the angiographic image V2 obtained immediately before the correction target angiographic image V2.

[0115] Thus, in the first (initial) image correction, the image correction unit 13 corrects the angiographic image to be corrected using the position of the designated portion FP in the angiographic image V1 serving as the reference. In the second and subsequent image corrections, the image correction unit 13 corrects the angiographic image to be corrected using the position of the designated portion FP in the "corrected angiographic image after correction" of the angiographic image acquired immediately before the angiographic image to be corrected, as described in a2' above.

[0116] return Figure 4 Continue with the description. Figure 4 In step S210, the display control unit 14 displays (outputs) the corrected angiographic image Vm′ corrected in step S208 on the display device 30. Thereafter, the display control unit 14 transfers the process to step S204 and repeats the above process. Thus, in the display device 30, after the angiographic image V1 which becomes the reference, the angiographic image acquisition unit 12 sequentially acquires the corrected angiographic images V2′, V3′, ..., Vm′ from the angiographic device 20 and sequentially corrected by the image correction unit 13, and outputs them to the display device 30 (refer to FIG. 1 ). Figure 3: bottom row). The operator can grasp the position of the designated part of the medical device and advance the operation by referring to the corrected angiographic images V2', V3', ..., Vm' output to the display device 30.

[0117] Figure 4 The processing is repeated until a predetermined termination condition is satisfied. The termination condition may be, for example, the power supply of the surgery support device 10 is turned off, or the application program started at the start of the processing is terminated.

[0118] As described above, according to the surgical assisting device 10 of the first embodiment, the angiographic image acquisition unit 12 sequentially acquires angiographic images Vm at predetermined intervals Δtn corresponding to the pulsation cycle of the heart 91. The heart 91 regularly repeats expansion and contraction (expansion and contraction) according to the pulsation cycle, and therefore, by acquiring angiographic images Vm at predetermined intervals Δtn corresponding to the pulsation cycle, it is possible to acquire angiographic images Vm having an equal (uniform) expansion and contraction state of the heart 91. By acquiring such angiographic images Vm, the angiographic image acquisition unit 12 can reduce the jitter of the position of the medical device designation portion on the angiographic image caused by the pulsation. In addition, the patient's body (human body 90) is constantly moving due to the volume change of the thorax accompanying breathing, in addition to the pulsation of the heart 91. According to the surgical assistance device 10 of the first embodiment, the image correction unit 13 generates a corrected angiographic image V2′ that corrects the angiographic image V2 to be corrected so that the position of the designated portion FP2 of the medical device included in the angiographic image V2 to be corrected in the angiographic images Vm sequentially acquired by the angiographic image acquisition unit 12 is close to the position of the designated portion FP of the medical device included in the angiographic image V1 to be corrected that is acquired earlier in time than the angiographic image V2 to be corrected ( Figure 5 ). Therefore, the image correction unit 13 can reduce the jitter of the position of the medical device designated part on the angiographic image caused mainly by factors other than the pulsation of the heart 91. In this way, the surgical assistance device 10 of the first embodiment separately (individually) performs the reduction of the jitter caused by the pulsation based on the angiographic image acquisition unit 12 and the reduction of the jitter caused by major factors other than the pulsation based on the image correction unit 13, thereby being able to reduce the jitter of the position of the medical device designated part on the angiographic image with high precision. As a result, according to the surgical assistance device 10 of the first embodiment, the operator can correctly grasp the position of the medical device designated part in the target blood vessel, thereby being able to achieve improved surgical accuracy, shortened surgical time, and reduced burden on patients.

[0119] In addition, according to the surgical assistance device 10 of the first embodiment, the image correction unit 13 corrects the angiographic image V3 to be corrected using the position of the designation portion FP2 of the medical device included in the corrected angiographic image V2′ obtained by correcting the angiographic image V2 acquired immediately before the angiographic image V3 to be corrected ( Figure 6 ). Therefore, the position movement of the designated part of the medical device in the latest correction result (corrected angiography image K′=V3′ for the angiography image K=V3 of the correction object) and the immediately previous correction result (corrected angiography image K-1′=V2′ for the angiography image K-1=V2 acquired immediately before the angiography image of the correction object) can be reduced, and a more natural corrected angiography image can be provided.

[0120] Furthermore, according to the angiography apparatus 20 of the first embodiment, the target image acquisition unit 291 causes the FPD to capture the target blood vessel in which the medical device is inserted at each predetermined interval Δtn corresponding to the pulsation cycle of the heart 91, thereby acquiring an angiography image Vm showing the target blood vessel, and outputs the acquired image to the surgery support apparatus 10. The heart 91 regularly expands and contracts repeatedly according to the pulsation cycle, and therefore, by acquiring the angiography image Vm at each predetermined interval Δtn corresponding to the pulsation cycle, it is possible to acquire an angiography image Vm having an equal (uniform) expansion and contraction state of the heart 91. As a result, the target image acquisition unit 291 can reduce the jitter of the position of the medical device designated portion on the angiography image caused by pulsation. As a result, according to the angiography apparatus 20 of the first embodiment, the operator can accurately grasp the position of the medical device designated portion in the target blood vessel, thereby achieving improved surgical accuracy, shortened surgical time, and reduced burden on the patient.

[0121] Furthermore, according to the angiography apparatus 20 of the first embodiment, the predetermined interval Δtn ( Figure 3 ). Therefore, the object image acquisition unit 291 can acquire and output the angiography image Vm with high precision in which the jitter caused by the pulsation is reduced according to the pulsation cycle of each patient's heart 91, that is, the pulsation cycle close to the current pulsation cycle. Similarly, the angiography image acquisition unit 12 of the surgical assistance device 10 can acquire the angiography image Vm with high precision in which the jitter caused by the pulsation is reduced.

[0122] Furthermore, according to the angiography apparatus 20 of the first embodiment, the predetermined interval Δtn ( tn-tn-1 ) is calculated based on the time interval tn-tn-1 between the time point tn-1 at which the specific waveform appears in the n-1th electrocardiogram waveform data and the time point tn at which the specific waveform appears in the nth electrocardiogram waveform data, in two temporally consecutive electrocardiogram waveform data S1 acquired by the electrocardiogram measurement device 60. Figure 3 ). Therefore, by defining the specific waveform as the expansion and contraction state of the heart 91 being suitable for visual confirmation of the medical device (for example, specific waveform = R wave), the object image acquisition unit 291 can acquire and output the angiographic image Vm when the expansion and contraction state of the heart 91 is suitable for visual confirmation of the medical device. As a result, the visibility of the medical device in the angiographic image can be improved. Similarly, the angiographic image acquisition unit 12 of the surgical assistance device 10 can acquire the angiographic image Vm when the expansion and contraction state of the heart 91 is suitable for visual confirmation of the medical device.

[0123] Furthermore, according to the surgery support system 1 of the first embodiment, the target image acquisition unit 291 of the angiography device 20 sequentially transmits the angiography images Vm at predetermined intervals Δtn corresponding to the beat cycle of the heart 91 to the surgery support device 10 ( Figure 2 : Step S116), the angiographic image acquisition unit 12 of the surgery support device 10 sequentially acquires angiographic images Vm ( ) at each predetermined interval Δtn from the angiographic device. Figure 4 :Step S204). Therefore, the processing load in the surgery support system 1 can be distributed to the angiography device 20 and the surgery support device 10, and the occurrence of processing delay caused by the increase in processing load can be suppressed. In addition, the image correction unit 13 of the surgery support device 10 sets the latest angiography image among the multiple angiography images acquired by the angiography image acquisition unit 12 as the angiography image to be corrected ( Figure 4 : Step S204), so that the corrected angiographic image Vm′ ( Figure 4 :Step S210).

[0124] <Second Embodiment>

[0125] Figure 7This is an explanatory diagram illustrating the structure of a surgical assistance system 1A according to a second embodiment. In the second embodiment, a structure is described in which a true lumen image representing the true lumen in a target blood vessel having a position and posture corresponding to the corrected angiography image is generated for a corrected angiography image of a target blood vessel, and a synthetic image obtained by synthesizing the corrected angiography image and the true lumen image is displayed. The surgical assistance system 1A according to the second embodiment includes a surgical assistance device 10A instead of the surgical assistance device 10. In the structure described in the first embodiment, the surgical assistance device 10A includes an angiography image acquisition unit 12A instead of the angiography image acquisition unit 12, an image correction unit 13A instead of the image correction unit 13, and a true lumen information acquisition unit 15, a true lumen image generation unit 16, and an image synthesis unit 17.

[0126] Figure 8 This is a diagram for explaining the angiographic image acquisition unit 12A according to the second embodiment. Figure 2 In the first processing described in the above, angiographic images at each predetermined interval Δtn are acquired, namely, a first angiographic image Va captured by the first FPD 21 disposed at a first position and a second angiographic image Vb captured by the first FPD 21 disposed at a second position different from the first position. The first angiographic image Va and the second angiographic image Vb are similar to the angiographic images V1 and V2 of the first embodiment in that they are images in which the beating cycle of the heart 91 is in the same phase, that is, images in which the expansion and contraction states of the heart 91 are equal.

[0127] On the other hand, the positions of the first FPD 21 of the captured images Va and Vb of the first angiographic image Va and the second angiographic image Vb are different, respectively. Figure 8 As shown, the second angiography image Vb( Figure 8 : dotted shadow) is configured in the first angiography image Va ( Figure 8 : oblique line shadow) The image captured by the first FPD21 in the vertical shooting direction. Figure 8 In the figure, the shooting direction of the first FPD21 configured at the first position relative to the heart 91, i.e., the vector representing the first view, is represented as the first view vector Vw1, and the shooting direction of the first FPD21 configured at the second position relative to the heart 91, i.e., the vector representing the second view (=BNV) is represented as the second view vector Vw2.

[0128] The image correction unit 13A performs Figure 4The second processing described in , generates a first corrected angiographic image Va′ which is a corrected angiographic image for the first angiographic image Va and a second corrected angiographic image Vb′ which is a corrected angiographic image for the second angiographic image Vb, and sends them to the true lumen information acquisition unit 15 .

[0129] The true lumen information acquisition unit 15 uses the corrected angiographic images (specifically, the first corrected angiographic image Va' and the second corrected angiographic image Vb') acquired from the image correction unit 13A to acquire three-dimensional position information of the true lumen existing in the target blood vessel. In addition, the process (step) performed by the true lumen information acquisition unit 15 is also referred to as a true lumen information acquisition process (step).

[0130] Fig. 9 This is a diagram for explaining an example of a method in which the true cavity information acquisition unit 15 acquires the three-dimensional position information of the true cavity. Fig. 9 The example in which an imaging sensor 300 (ultrasonic sensor 300) as medical equipment and a guidewire catheter 400 into which a guidewire 500 is inserted is shown in the longitudinal section of the target blood vessel BV where CTO is generated and the false lumen generated under the intima of the target blood vessel BV. In this example, the true lumen information acquisition unit 15 uses the first position ( Figure 8 ), the first corrected angiographic image Va′, the second position ( Figure 8 ), the second corrected angiography image Vb′, and the ultrasonic image to obtain the three-dimensional position information of the true lumen. The first corrected angiography image Va′ and the second corrected angiography image Vb′ include the image of the imaging sensor 300. The ultrasonic image includes the image of the guidewire 500 and the image of the true lumen.

[0131] The true cavity information acquisition unit 15 acquires the three-dimensional position information of the true cavity through steps c1 to c11 shown below.

[0132] (c1) Using the position information of the first position and the first corrected angiographic image Va′ at the first position and the position information of the second position and the second corrected angiographic image Vb′ at the second position, the position vector of the transducer 301 of the imaging sensor 300 and the axial vector T1 of the transducer 301 are obtained.

[0133] (c2) The position α of the first FPD 21 at which the corrected angiographic image Vα′ where the transducer 301 of the imaging sensor 300 overlaps (intersects) with the guide wire 500 is obtained is determined, and the view vector Vwα is calculated based on the position α. ​​The corrected angiographic image Vα′ is generated by the angiographic image acquisition unit 12A for each angiographic image Vα at a predetermined interval Δtn obtained by the first processing, and the image correction unit 13A generates the corrected angiographic image Vα′ by the image correction unit 13A. Figure 4The second process described in generates a corrected angiographic image.

[0134] (c3) The rotation axis R is calculated based on the outer product of the axial vector T1 of the transducer 301 calculated in step c1 and the view vector Vwα calculated in step c2.

[0135] (c4) A vector CV1 is calculated by rotating the axial vector T1 of the transducer 301 calculated in step c1 by 90 degrees, with the rotation axis R calculated in step c3 as the axis.

[0136] (c5) The imaging sensor 300 acquires an ultrasonic image of the inside of the target blood vessel in a direction perpendicular to the axial vector T1 of the transducer 301 (a 360° full circumferential direction of T1). At this time, the ultrasonic image is acquired at the same time t (time t at each time interval corresponding to the heart beat cycle, i.e., the predetermined time interval) as when the first angiographic image Va, the second angiographic image Vb, and the angiographic image Vα are captured.

[0137] (c6) Using the ultrasonic image obtained in step c5, correlation is performed in the direction from the transducer 301 to the guide wire 500.

[0138] (c7) The number of pixels of the ultrasonic image is associated with the actual size.

[0139] (c8) Using the image of the guidewire 500 and the image of the true lumen shown in the ultrasonic image, the true lumen vector S1 (a vector perpendicular to the transducer axis T1 and extending from the transducer 301 to the true lumen) in the XYZ three-dimensional space is calculated.

[0140] (c9) Using the true cavity vector S1 and the associated dimensions, calculate the actual width and length of the true cavity.

[0141] (c10) The above-mentioned processing is repeated for the target label number i (i is a natural number) to calculate the true cavity vectors S1 to Si in the XYZ three-dimensional space, as well as the width and length.

[0142] (c11) The direction of the obtained true cavity vectors S1~Si, the length (mm) of the true cavity vectors S1~Si, and the actual size (mm) of the true cavity width of the part corresponding to the true cavity vectors S1~Si are stored as the three-dimensional position information of the true cavity (position information in the XYZ three-dimensional space).

[0143] Another example of a method for the true lumen information acquisition unit 15 to acquire the three-dimensional position information of the true lumen is described. In this example, a method is described in which the true lumen information acquisition unit 15 acquires the three-dimensional position information of the true lumen by using images of the true lumen captured from different angles, namely, a first corrected angiography image Va′ and a second corrected angiography image Vb′ without using the imaging sensor 300. The first corrected angiography image Va′ includes an image of the front end of the guidewire catheter 400 and an image of the true lumen. The second corrected angiography image Vb′ includes an image of the front end of the guidewire catheter 400 and an image of the true lumen. The true lumen information acquisition unit 15 uses the images of the true lumen respectively projected in the first corrected angiography image Va′ and the second corrected angiography image Vb′ to obtain the true lumen vectors S1 to Si.

[0144] Fig.10 It is a diagram for explaining the true lumen image generating unit 16 and the image synthesizing unit 17 . Fig.10 (A) shows an example of the corrected angiographic image Vx' acquired from the image correcting unit 13A. Fig.10 (B) shows an example of the true lumen image Vy generated by the true lumen image generating unit 16 . Fig.10 (C) shows an example of the synthesized image Vx′+Vy generated by the image synthesizing unit 17 .

[0145] The true lumen image generation unit 16 generates a true lumen image representing the true lumen. Specifically, first, the true lumen image generation unit 16 obtains from the image correction unit 13A a corrected angiographic image Vx′ of the angiographic image Vx obtained by photographing the target blood vessel BV from the first FPD 21 disposed at an arbitrary photographing position A, that is, the angiographic image Vx at each predetermined interval Δtn. Fig.10 (A) shows an example of a corrected angiography image Vx′ acquired by the true lumen image generation unit 16. The imaging position A is an arbitrary position different from the first position, the second position, and the position α described above. Then, the true lumen image generation unit 16 uses the orthogonal projection vector obtained based on the position information of the arbitrary imaging position A and the three-dimensional position information of the true lumen acquired by the true lumen information acquisition unit 15 to generate a true lumen image Vy of the true lumen representing the position and posture corresponding to the corrected angiography image Vx′. Fig.10 As shown in (B), the true lumen image Vy includes an image of the true lumen TC at a position and posture corresponding to the corrected angiographic image Vx′ (in other words, when viewed from the imaging position A). In addition, the process (step) performed by the true lumen image generation unit 16 is also referred to as a true lumen image generation process (step).

[0146] The image synthesis unit 17 generates Fig.10 (A) shows the corrected angiographic image Vx′ and Fig.10The synthesized image Vx′+Vy after synthesizing the real cavity image Vy shown in (B) is output to the display device 30. Fig.10 As shown in (C), the composite image Vx'+Vy is an image obtained by superimposing the true lumen image Vy on the corrected angiography image Vx'. The process (step) executed by the image composite unit 17 is also referred to as an image composite process (step).

[0147] In addition, details of the processes in the true lumen information acquisition unit 15 , the true lumen image generation unit 16 , and the image synthesis unit 17 are disclosed in International Application No. PCT / JP2022 / 28524.

[0148] In this way, further processing can be performed using the corrected angiographic images Va′, Vb′, and Vx′ corrected by the image correction unit 13A. As described in the first embodiment, the corrected angiographic images Va′, Vb′, and Vx′ achieve reduction in jitter caused by pulsation and reduction in jitter caused by factors other than pulsation. By performing further processing using such corrected angiographic images Va′, Vb′, and Vx′, the accuracy of subsequent processing can be improved. According to the surgical support device 10A and the surgical support system 1A of the second embodiment as described above, the same effects as those of the first embodiment can be achieved.

[0149] Furthermore, according to the surgical assisting device 10A of the second embodiment, the true lumen image generating unit 16 can generate a true lumen image Vy of the true lumen TC representing the position and posture corresponding to the corrected angiographic image Vx′, using the position information of the arbitrary imaging position A where the angiographic image Vx is acquired and the three-dimensional position information of the true lumen acquired by the true lumen information acquiring unit 15. That is, the true lumen image generating unit 16 can generate a true lumen image Vy representing the image TC of the true lumen based on the three-dimensional position information of the true lumen, even when the contrast agent has not flowed into the target true lumen or when the contrast agent has not flowed through the blood vessel. In addition, the image synthesizing unit 17 generates a synthesized image Vx′+Vy synthesized by the corrected angiographic image Vx′ at the arbitrary imaging position A and the true lumen image Vy representing the image TC of the true lumen, and outputs the synthesized image Vx′+Vy, so that the image Vy of the true lumen of the blood vessel can be displayed on the corrected angiographic image Vx′. Therefore, by checking the composite image Vx′+Vy, the operator can perform the operation while checking the positional relationship between the medical devices 300, 500 on the corrected angiography image Vx′ and the true lumen TC on the true lumen image Vy. As a result, the operator can accurately grasp the position of the true lumen in the target blood vessel, thereby improving the accuracy of the operation, shortening the time required for the operation, and reducing the burden on the patient.

[0150] Furthermore, according to the surgical assisting device 10A of the second embodiment, the true lumen information acquiring unit 15 acquires the three-dimensional position information of the true lumen using the first and second corrected angiographic images Va′ and Vb′ that achieve the reduction of the jitter caused by the pulsation of the heart 91 and the reduction of the jitter caused by factors other than the pulsation of the heart 91 (such as the volume change of the thoracic cavity accompanying breathing). Therefore, compared with the case where the angiographic images that do not achieve the reduction of the jitter are used, the three-dimensional position information of the true lumen can be acquired with high accuracy. In addition, since the shooting interval of the ultrasonic image and the first and second corrected angiographic images Va′ and Vb′ can be synchronized with the predetermined interval Δtn, the three-dimensional position information of the true lumen can be acquired with higher accuracy.

[0151] <Third Embodiment>

[0152] Fig.11 This is an explanatory diagram illustrating the structure of a surgical assisting system 1B according to a third embodiment. In the third embodiment, a structure is described in which a surgical assisting device 10B performs a first process in addition to a second process. The surgical assisting system 1B according to the third embodiment includes a surgical assisting device 10B instead of the surgical assisting device 10, and includes an angiography device 20B instead of the angiography device 20. In the structure described in the first embodiment, the surgical assisting device 10B includes an angiography image acquisition unit 12B instead of the angiography image acquisition unit 12, and also includes an object image acquisition unit 18 and an electrocardiogram information acquisition unit 19. The object image acquisition unit 18 performs the first process (in Fig.12 The surgery support device 10B is electrically connected to the electrocardiogram measurement device 60, and the electrocardiogram information acquisition unit 19 has the same function as the electrocardiogram information acquisition unit 292 of the first embodiment.

[0153] The angiography device 20B includes a control device 29B in place of the control device 29 in the configuration described in the first embodiment. The angiography device 20B is not connected to the electrocardiogram measurement device 60. The control device 29B of the angiography device 20B does not have the functions of the target image acquisition unit 291 and the electrocardiogram information acquisition unit 292 described in the first embodiment, and causes the FPD to continuously photograph the target blood vessel at an interval shorter than the predetermined interval Δtn described in the first embodiment, and transmits the continuous angiography images acquired by the photographing to the surgery support device 10B. In addition, the "continuous angiography images" refer to dynamic images or still images that are smoothly continuous at intervals shorter than the predetermined interval Δtn described in the first embodiment.

[0154] Fig.12: is a flowchart showing an example of the processing procedure of the first processing in the third embodiment. The first processing of the third embodiment is executed in the surgical support device 10B. Fig.12 The first process shown is started at an arbitrary trigger, which may be, for example, when the surgical support device 10B is powered on or when a predetermined application provided by the surgical support device 10B is activated.

[0155] In step S100B, the target image acquisition unit 18 detects the time point W1 and the time point W2 using one of the trigger signal S2 and the electrocardiogram waveform data S1 in the electrocardiogram data acquired by the electrocardiogram information acquisition unit 19, and obtains Δt2. Figure 2 The same as step S100 of . In step S102B, the target image acquisition unit 18 extracts the angiography image V1 at time t (t=t2+Δt2) after Δt2 from time point t2 from the continuous angiography images received from the angiography device 20B. In step S106B, the target image acquisition unit 18 substitutes 3 into the variable n and 2 into the variable m.

[0156] In step S108B, the target image acquisition unit 18 detects the time point Wn using one of the trigger signal S2 and the electrocardiogram waveform data S1 in the electrocardiogram data acquired by the electrocardiogram information acquisition unit 19, and obtains Δtn. Figure 2 In step S110B, the object image acquisition unit 18 determines whether the absolute value of Δtn is greater than a predetermined value. If it is greater than the predetermined value (step S110B: Yes), the display device 30 outputs a warning in step S112B. If it is less than the predetermined value (step S110B: No), the process is transferred to step S114B. Figure 2 Steps S110 to S112 are the same.

[0157] In step S114B, the target image acquisition unit 18 extracts the angiography image Vm at time t (t=tn+Δtn) after time tn from the continuous angiography images received from the angiography device 20B. In step S120, the angiography image acquisition unit 12 acquires the angiography image Vm extracted by the target image acquisition unit 18 and executes the Figure 4 Thus, in the third embodiment, the second process is executed as a subroutine of the first process. In step S118B, the object image acquisition unit 18 adds 1 to each of the variables n and m, transfers the process to step S108B, and repeats the above process. Fig.12The processing is repeated until a predetermined termination condition is satisfied. The termination condition may be, for example, power off of the surgery support device 10B or the termination of an application program started at the start of the processing.

[0158] In this way, the structure of the surgery support system 1B can be changed in various ways. The angiography device 20B can also be configured to continuously capture only the target blood vessel to obtain continuous angiography images. In the surgery support system 10B, the first process of sequentially acquiring angiography images Vm at predetermined intervals Δtn corresponding to the beat cycle of the heart 91 can also be performed. In the surgery support system 1B of the third embodiment as described above, the same effects as those of the first embodiment can be achieved. Furthermore, according to the surgery support system 1B of the third embodiment, the surgery support system 10B further includes a target image acquisition unit 18 that extracts angiography images Vm at predetermined intervals Δtn corresponding to the beat cycle of the heart 91 from the angiography images (continuous angiography images) representing the target blood vessel continuously captured by the FPD. In this way, the angiography device 20B equipped with an FPD only needs to continuously supply angiography images of the target blood vessels to the surgery support device 10B (in other words, supply continuous angiography images), thereby improving the scalability of the surgery support system 1B composed of the surgery support device 10B and the angiography device 20B.

[0159] In addition, Fig.12 In steps S102B and S114B, the target image acquisition unit 18 may send an "FPD shooting command" to the control device 29B of the angiography device 20B instead of extracting the angiography images V1 and Vm at time t from the continuous angiography images. In order to accurately perform FPD shooting at time t (i.e., X-ray irradiation by the first X-ray tube device 22 and X-ray detection and conversion by the first FPD 21), it is preferable to consider the preparation time required for sending and receiving commands and starting the FPD in advance and send the FPD shooting command slightly before time t. The control device 29B that receives the FPD shooting command drives the FPD to shoot the target blood vessels at time t to obtain angiography images V1 and Vm, and sends the acquired angiography images V1 and Vm to the surgery support device 10B. In this way, a system having the same functions as the third embodiment described above can also be constructed.

[0160] <Fourth Embodiment>

[0161] Fig.13This is an explanatory diagram illustrating the structure of a surgery support system 1C according to a fourth embodiment. In the fourth embodiment, a structure in which the processing contents of the first process and the second process are partially different from those of the first embodiment is described. The surgery support system 1C according to the fourth embodiment includes a surgery support device 10C instead of the surgery support device 10, and includes an angiography device 20C instead of the angiography device 20. In the structure described in the first embodiment, the surgery support device 10C includes an image correction unit 13C instead of the image correction unit 13. In the structure described in the first embodiment, the angiography device 20C includes an object image acquisition unit 291C instead of the object image acquisition unit 291.

[0162] Fig.14 This is a flowchart showing an example of the processing procedure of the first processing in the fourth embodiment. Fig.15 These are diagrams for explaining the first process and the second process of the fourth embodiment. Fig.15 The diagrams and content of each stage Figure 3 Same as the instructions in . Fig.14 As shown in step S108C, the object image acquisition unit 291C only detects Wn (=tn) and does not calculate Δtn. Figure 2 Furthermore, in step S114C, the target image acquisition unit 291C causes the FPD to image the target blood vessel at time t (t=tn+Δt2) after Δt2 obtained in step S100 from time point tn corresponding to Wn, thereby acquiring the angiographic image Vm.

[0163] That is, in the fourth embodiment, the object image acquisition unit 291C does not use "two temporally continuous electrocardiogram data just acquired" as described in the first embodiment, but sequentially acquires angiographic images Vm at predetermined intervals Δt2 (time intervals corresponding to the heart's beat cycle) calculated using "two temporally continuous electrocardiogram data acquired at a certain time point in the past". In this way, the processing load in the control device 29C can be reduced compared to the case where Δtn is calculated each time using the previous electrocardiogram data. In addition, the object image acquisition unit 291C may update the reference time interval ( Δtn ) at each predetermined cycle (for example, every 10 cycles). Fig.14 , Fig.15 The updated time interval is used to acquire the angiography image Vm from the time of the update to the time of the next update.

[0164] like Fig.15 As shown, the image correction unit 13C performs the image correction in the second processing ( Figure 3: In step S208), each time according to the first embodiment Figure 5 The order of the processes a1 and a2 described in (A) and (B) generates the corrected angiographic image Vm'.

[0165] (a1) Specifically, the image correction unit 13C performs parallel translation of the angiographic image Vm to be corrected so that the positions x2 and y2 of the designated part FP2 of the medical device included in the angiographic image Vm to be corrected are the same as those in FIG.

[0166] (a2) The position x, y of the designated part FP of the medical device included in the single angiography image V1 acquired earlier in time than the angiography image Vm to be corrected is approached to generate the corrected angiography image Vm'. In other words, the image correction unit 13C does not perform the first embodiment. Figure 6 Processing a2' described in (A) and (B).

[0167] Thus, the contents of the first process and the second process can be variously changed. In addition to the above-mentioned variations, in the first process and the second process, the execution order of each step to be executed can be changed, at least a part of the steps can be omitted, and other steps not described can be executed. According to the above-mentioned fourth embodiment of the surgical assisting device 10C and the angiographic device 20C, the same effects as those of the above-mentioned first embodiment can be achieved. In addition, according to the angiographic device 20C of the fourth embodiment, the predetermined interval Δt2 is calculated based on the time interval of two temporally continuous electrocardiogram data acquired in the past by the electrocardiogram measurement device 60. Therefore, the target image acquisition unit 291C can capture the angiographic image Vm with high precision and reduced jitter caused by the pulsation according to the pulsation cycle of each patient's heart 91, and output the angiographic image Vm. Similarly, the angiographic image acquisition unit 12 of the surgical assisting device 10C can acquire the angiographic image Vm with high precision and reduced jitter caused by the pulsation.

[0168] <Variations of this embodiment>

[0169] The present invention is not limited to the above-mentioned embodiments, and can be implemented in various ways within the scope of the main purpose thereof. For example, a part of the structure implemented by hardware can be replaced by software, and conversely, a part of the structure implemented by software can be replaced by hardware. In addition, for example, the following deformations can also be performed.

[0170] [Modification 1]

[0171] In the first to fourth embodiments described above, the structures of the surgical support systems 1, 1A to 1C are illustrated. However, the structure of the surgical support system 1 can be changed in various ways. For example, the display device 30 can also be a monitor or a touch panel built into the surgical support device 10, 10A to 10C. For example, the angiography device 20, 20B, 20C can be a structure with a single FPD (in other words, a structure without the second FPD 25). For example, the surgical support system 1 can also have other medical devices not shown in the figure (for example, a CT device, an MRI device), etc.

[0172] [Modification 2]

[0173] In the first to fourth embodiments, the structures of the surgical support devices 10 and 10A to 10C are shown. However, the structure of the surgical support device 10 can be modified in various ways. For example, the functions of the functional units of the surgical support device 10 can be realized by cooperation of a plurality of devices connected via a network.

[0174] [Variation 3]

[0175] In the first to fourth embodiments, an example of the steps of the first process and the second process is shown. However, the order of the first process and the second process is only an example, and various changes can be made. For example, the execution order of each step can be changed, at least a part of the steps can be omitted, and other steps not described can be executed.

[0176] For example, the predetermined interval Δtn (Δt2 in the case of the fourth embodiment) that determines the time t in the first process may be an average time interval (or a time interval obtained by statistics such as a mode time interval) calculated from three or more electrocardiogram data that are continuous in time just acquired or acquired in the past. In addition, the predetermined interval Δtn (Δt2 in the case of the fourth embodiment) may be calculated from two or more electrocardiogram data that are not continuous in time just acquired or acquired in the past.

[0177] For example, in Figure 4In step S206 of the second process shown, the image of the designated part of the medical device included in the designated part image just obtained may be used to perform template matching. Specifically, in step S206 executed for the first time, the image correction unit 13 matches the image of the medical device included in the angiography image V2 with the designated part image generated using the angiography image V1, and detects the designated part FP2 of the medical device reflected in the angiography image V2. Thereafter, the image correction unit 13 extracts an image of a predetermined range including the designated part FP2 in the angiography image V2, and updates the designated part image in the storage unit together with the coordinate information of the center C2 (x2, y2) of the designated part FP2. In step S206 executed for the second time, the image correction unit 13 matches the image of the medical device included in the angiography image V3 with the designated part image generated using the angiography image V2, and detects the designated part FP3 of the medical device reflected in the angiography image V3. After that, the image correction unit 13 extracts an image of a predetermined range including the designated portion FP3 in the angiography image V3, and updates the designated portion image in the storage unit together with the coordinate information of the center C3 (x3, y3) of the designated portion FP3. By repeatedly performing this process, the image correction unit 13 can perform template matching using the image of the designated portion of the medical device included in the designated portion image just now. In this way, the image correction unit 13 detects the designated portion of the angiography image Vm to be corrected using the image of the designated portion that appears in the angiography image just now that is closest in time to the angiography image Vm to be corrected, thereby improving the detection accuracy of the designated portion.

[0178] For example, in Figure 4 In step S206 of the second process shown, template matching may be performed using a designated part pattern instead of a designated part image. The designated part pattern is a data set including a plurality of images obtained by photographing designated parts of a plurality of medical devices used in an inspection or treatment operation from various angles, and is prepared in advance and stored in a storage unit. In this case, in step S202, the image correction unit 13 reads the designated part pattern, compares the designated part FP of the medical device selected in step S202 with the images of the designated parts of various medical devices included in the designated part pattern, and determines the one actually used in the operation. Then, in step S206, the image correction unit 13 detects the designated part FPm of the medical device reflected in the angiography image Vm by matching the image of the medical device reflected in the angiography image Vm with one of the data determined in step S202.

[0179] exist Figure 4In step S210 of the second process shown, the angiographic image V1 used as a reference is not corrected at all, is used in the image correction a1 of step S208, and is output to the display device 30. However, the image correction unit 13 may correct the angiographic image V1 by parallel shifting the designated portion FP of the angiographic image V1 so that the designated portion FP is located at the center of the image V1 (the corrected angiographic image V1 is also referred to as "corrected angiographic image V1a"). In this case, the image correction unit 13 uses the corrected angiographic image V1a as a reference in the image correction a1 performed in step S208. In addition, the display control unit 14 causes the display device 30 to output the corrected angiographic image V1a in step S210. In this way, for the angiographic images V1a to Vm', the designated portions FP to FPm of the image displayed on the display device 30 can be located at the center of the screen, so that the operator can easily confirm the position of the designated portions FP to FPm.

[0180] For example, in Figure 4 In step S210 of the second process shown in the figure, the display control unit 14 may cause the display device 30 to display only a portion (specific region image R) of the angiographic images V1 to Vm. In this case, the display control unit 14 trims (cuts) a portion (specific region image R) from the angiographic images V1 to Vm and then displays it on the display device 30. The specific region image R is selected together with the selection of the designation unit FP in step S202 of the second process, for example. In the case of causing the display device 30 to display only the specific region image R, in step S208, the image correction unit 13 may parallel shift the trimmed range (cut range) of the image instead of parallel shifting the image.

[0181] [Variation 4]

[0182] The structures of the surgical assisting devices 10, 10A to 10C of the first to fourth embodiments, the structures of the angiography devices 20, 20B, and 20C, and the structures of the above-mentioned modifications 1 to 3 may be appropriately combined. For example, the surgical assisting device 10 may be configured to be capable of outputting the synthesized image described in the second embodiment and executing the first process described in the third embodiment. For example, in the surgical assisting system 1C that performs the first process and the second process described in the fourth embodiment, the output of the synthesized image described in the second embodiment may also be performed. For example, in the surgical assisting systems 1A to 1C described in the second to fourth embodiments, the modifications of step S206, step S208, and step S210 described in modification 3 may also be adopted.

[0183] The above is an explanation of the present invention based on the implementation mode and the modification example, but the implementation mode of the above-mentioned method is to facilitate the understanding of the present invention and does not limit the present invention. The present invention can be changed and improved within the scope of its main purpose and technical solution, and its equivalent is included in the present invention. In addition, if its technical features are not described as necessary features in this specification, they can be appropriately deleted.

[0184] Explanation of symbols

[0185] 1. 1A to 1C—surgical assistance system, 10. 10A to 10C—surgical assistance device, 11—main control unit, 12. 12A, 12B—angiography image acquisition unit, 13. 13A, 13C—image correction unit, 14—display control unit, 15—true cavity information acquisition unit, 16—true cavity image generation unit, 17—image synthesis unit, 18—object image acquisition unit, 19—ECG information acquisition unit, 20. 20B, 20C—angiography device, 21—first FPD, 22—first X-ray tube device, 23—first C-arm, 24—first support unit, 25— Second FPD, 26—second X-ray tube device, 27—second C-arm, 28—second support part, 29, 29B, 29C—control device, 30—display device, 31—monitor, 32—arm, 40—workbench, 41—bed, 42—telescopic part, 43—leg, 50—operating part, 60—ECG measuring device, 90—human body, 91—heart, 92—head, 93—foot, 291, 291C—object image acquisition part, 292—ECG information acquisition part, 300—shooting sensor, 301—transducer, 400—guidewire catheter, 500—guidewire.

Claims

1. A surgical assisting device, characterized in that: have: an angiographic image acquisition unit that sets a time interval corresponding to a heart beat cycle as a predetermined interval and sequentially acquires an angiographic image representing a target blood vessel in which the medical device is inserted at each predetermined interval; and An image correction unit sequentially corrects the angiography images sequentially acquired by the angiography image acquisition unit, and generates a corrected angiography image obtained by correcting the angiography image of the correction object, so that the position of the designated part of the medical device included in the angiography image of the correction object is close to the position of the designated part of the medical device included in the angiography image acquired earlier in time than the angiography image of the correction object.

2. The surgical assisting device according to claim 1, characterized in that: The angiographic image acquisition unit sequentially acquires the angiographic images with a time interval between two temporally consecutive electrocardiographic data acquired in the past by an electrocardiograph being set to a time interval corresponding to a beating cycle of the heart.

3. The surgical assisting device according to claim 1, characterized in that: The angiographic image acquisition unit sequentially acquires the angiographic images with a time interval between two temporally consecutive electrocardiographic data just acquired by the electrocardiograph being a time interval corresponding to a beating cycle of the heart.

4. The surgical assisting device according to claim 2 or 3, characterized in that: The electrocardiogram data is electrocardiogram waveform data. When the data acquired relatively later in the two consecutive electrocardiogram waveform data is taken as the nth electrocardiogram waveform data and the data acquired relatively earlier is taken as the n-1th electrocardiogram waveform data, The angiographic image acquisition unit sequentially acquires the angiographic images with the time interval between the time point tn-1 at which a specific waveform appears in the n-1th electrocardiographic waveform data and the time point tn at which the specific waveform appears in the nth electrocardiographic waveform data being set to a time interval corresponding to the heart beat cycle.

5. The surgical assisting device according to any one of claims 1 to 4, characterized in that: The image correction unit corrects the angiography image of the correction object so that the position of the designated part of the medical device included in the angiography image of the correction object is close to the position of the designated part of the medical device included in the corrected angiography image after correction of the angiography image acquired immediately before the angiography image of the correction object.

6. The surgical assisting device according to any one of claims 1 to 5, characterized in that: Also available: an electrocardiogram information acquisition unit that acquires electrocardiogram data from an electrocardiogram measurement device; and The target image acquisition unit extracts the angiographic image for each predetermined interval from the continuous angiographic images showing the target blood vessel continuously imaged at intervals shorter than the predetermined interval using a flat panel detector (FPD).

7. The surgical assisting device according to any one of claims 1 to 6, characterized in that: Also available: a true lumen information acquisition unit configured to acquire three-dimensional position information of a true lumen existing in the target blood vessel; a true lumen image generating unit configured to generate a true lumen image representing the true lumen; as well as an image synthesis unit that generates a synthesized image by synthesizing the corrected angiography image and the true lumen image, and outputs the synthesized image, The true lumen image generation unit obtains from the image correction unit the corrected angiographic image for each of the predetermined intervals of the angiographic image acquired by imaging the target blood vessel using an FPD, i.e., a flat panel detector, arranged at an arbitrary imaging position, The true lumen image generating unit generates a true lumen image of the true lumen indicating a position and a posture corresponding to the corrected angiographic image, using the position information of the arbitrary imaging position and the three-dimensional position information of the true lumen.

8. The surgical assisting device according to claim 7, characterized in that: The angiographic image acquisition unit sequentially acquires the angiographic images at each of the predetermined intervals, that is, a first angiographic image acquired by photographing the FPD disposed at a first position and a second angiographic image acquired by photographing the FPD disposed at a second position different from the first position. the image correction unit sequentially generates a first corrected angiographic image which is the corrected angiographic image for the first angiographic image and a second corrected angiographic image which is the corrected angiographic image for the second angiographic image, The true lumen information acquisition unit acquires the three-dimensional position information of the true lumen using an ultrasonic image, the position information of the first position, the first corrected angiography image, the position information of the second position, and the second corrected angiography image, wherein the ultrasonic image is an image acquired by photographing the interior of the object blood vessel with an ultrasonic sensor, that is, an ultrasonic image acquired at each predetermined interval.

9. A control method for a surgical assisting device, characterized in that: have: an angiographic image acquisition step of setting a time interval corresponding to a heart beat cycle as a predetermined interval and sequentially acquiring an angiographic image representing a target blood vessel in which the medical device is inserted at each predetermined interval; and An image correction step sequentially corrects the angiography images sequentially acquired by the angiography image acquisition step, and generates a corrected angiography image obtained by correcting the angiography image of the correction object, so that the position of the designated part of the medical device included in the angiography image of the correction object is close to the position of the designated part of the medical device included in the angiography image acquired earlier in time than the angiography image of the correction object.

10. A computer program executed in a surgical assisting device, the computer program comprising: an angiographic image acquisition step of setting a time interval corresponding to a heart beat cycle as a predetermined interval, and sequentially acquiring an angiographic image representing a target blood vessel in which the medical device is inserted at each predetermined interval; and An image correction step is performed to sequentially correct the angiography images sequentially acquired by the angiography image acquisition step, and generate a corrected angiography image obtained by correcting the angiography image of the correction object, so that the position of the designated part of the medical device included in the angiography image of the correction object is close to the position of the designated part of the medical device included in the angiography image acquired earlier in time than the angiography image of the correction object.

11. A vascular angiography device, characterized in that: have: FPD is a flat panel detector, which has an X-ray tube device and an X-ray flat panel detector; and The target image acquisition unit causes the FPD to image a target blood vessel in which the medical device is inserted at predetermined intervals corresponding to a heart beat cycle, thereby acquiring an angiographic image showing the target blood vessel, and outputs the acquired image.

12. The angiography device according to claim 11, characterized in that: The device further comprises an electrocardiogram information acquiring unit for acquiring electrocardiogram data from an electrocardiogram measuring device. The target image acquisition unit sets the time interval between two temporally continuous electrocardiogram data acquired in the past by the electrocardiogram information acquisition unit as the predetermined interval which is a time interval corresponding to the heart beat cycle.

13. The angiography device according to claim 11, characterized in that: The device further comprises an electrocardiogram information acquiring unit for acquiring electrocardiogram data from an electrocardiogram measuring device. The target image acquisition unit sets the time interval between two temporally continuous electrocardiogram data just acquired by the electrocardiogram information acquisition unit as the predetermined interval which is a time interval corresponding to the heart beat cycle.

14. The angiography device according to claim 12 or 13, characterized in that: The electrocardiogram data is electrocardiogram waveform data. When the data acquired relatively later in the two consecutive electrocardiogram waveform data is taken as the nth electrocardiogram waveform data and the data acquired relatively earlier is taken as the n-1th electrocardiogram waveform data, The target image acquisition unit sets the time interval between the time point tn-1 at which a specific waveform appears in the n-1th electrocardiogram waveform data and the time point tn at which the specific waveform appears in the nth electrocardiogram waveform data as the predetermined interval which is a time interval corresponding to the heart beat cycle.

15. A method for controlling an angiography device, characterized in that: The method includes a target image acquisition step of acquiring an angiographic image showing the target blood vessel by imaging the target blood vessel into which the medical device is inserted at predetermined intervals corresponding to a heart beat cycle, and outputting the acquired image.

16. A computer program executed in an angiography apparatus, the computer program being characterized in that: The method includes a target image acquisition step of acquiring an angiographic image showing the target blood vessel by photographing the target blood vessel into which the medical device is inserted at predetermined intervals corresponding to the heart beat cycle, and outputting the acquired image.

17. A surgical assistance system, characterized in that: have: The surgical assisting device according to any one of claims 1 to 5; and The angiography device according to any one of claims 11 to 14, The target image acquisition unit of the angiography device sequentially transmits the angiography images at each predetermined interval to the surgery support device, The angiographic image acquisition unit of the surgery assisting device sequentially acquires the angiographic image at each predetermined interval from the angiographic device, The image correction unit of the surgery support device sets the latest angiographic image among the plurality of angiographic images acquired by the angiographic image acquisition unit as the angiographic image to be corrected.

18. The surgical assisting system according to claim 17, characterized in that: The surgical assisting device further comprises: a true lumen information acquisition unit configured to acquire three-dimensional position information of a true lumen existing in the target blood vessel; a true lumen image generating unit configured to generate a true lumen image representing the true lumen; as well as an image synthesis unit that generates a synthesized image by synthesizing the corrected angiography image and the true lumen image, and outputs the synthesized image, The true lumen image generation unit acquires from the image correction unit the corrected angiographic image for each of the predetermined intervals of the angiographic image acquired by imaging the target blood vessel using the FPD disposed at an arbitrary imaging position, The true lumen image generating unit generates a true lumen image of the true lumen indicating a position and a posture corresponding to the corrected angiographic image, using the position information of the arbitrary imaging position and the three-dimensional position information of the true lumen.

19. The surgical assisting system according to claim 18, characterized in that: The angiographic image acquisition unit sequentially acquires the angiographic images at each of the predetermined intervals, that is, a first angiographic image acquired by photographing the FPD disposed at a first position and a second angiographic image acquired by photographing the FPD disposed at a second position different from the first position. the image correction unit sequentially generates a first corrected angiographic image which is the corrected angiographic image for the first angiographic image and a second corrected angiographic image which is the corrected angiographic image for the second angiographic image, The true lumen information acquisition unit acquires the three-dimensional position information of the true lumen using an ultrasonic image, the position information of the first position, the first corrected angiography image, the position information of the second position, and the second corrected angiography image, wherein the ultrasonic image is an image acquired by photographing the interior of the object blood vessel with an ultrasonic sensor, that is, an ultrasonic image acquired at each predetermined interval.

20. A control method for a surgery support system, comprising: a surgery support device according to any one of claims 1 to 5; and a vascular angiography device according to any one of claims 11 to 14, wherein the control method for the surgery support system is characterized in that: The angiography device sequentially sends the angiography images at each predetermined interval to the surgery assisting device, The surgery assisting device sequentially acquires the angiographic images at each predetermined interval from the angiographic device, The surgery support device sets the latest angiographic image among the plurality of acquired angiographic images as the angiographic image to be corrected.

21. A computer program executed in a surgery support system, the surgery support system comprising: a surgery support device according to any one of claims 1 to 5; and an angiography device according to any one of claims 11 to 14, wherein the computer program is characterized in that: The angiography device sequentially sends the angiography images at each predetermined interval to the surgery assisting device, The surgery assisting device sequentially acquires the angiographic images at each predetermined interval from the angiographic device, The surgery support device sets the latest angiographic image among the plurality of acquired angiographic images as the angiographic image to be corrected.

Images