Endoscopic vascular sampling using temperature control and augmented reality displays

Abstract

To provide a vessel harvesting system capable of removing a target vessel from a patient for use as a bypass.SOLUTION: An elongated harvesting instrument is inserted into a body along a path of a target vessel which includes at least one side branch. The harvesting instrument includes a cutter for applying thermal energy to sever and cauterize the side branch. An endoscope camera captures visible-light images from a distal tip of the instrument within an anatomical tunnel around the target vessel. A thermal camera captures thermograms coinciding with the visible-light images to characterize a temperature present at respective surfaces in the tunnel. An image processor (e.g., an electronic controller) renders a video stream including the visible-light images and an overlay depicting the temperatures present on at least some of the respective surfaces when applying the thermal energy. A display presenting the video stream and overlay to a user may be an augmented-reality display.SELECTED DRAWING: Figure 7

Description

Technical Field

[0001] Cross - reference to related applications Not applicable. Description of research funded by federal government Not applicable.

[0002] The present invention generally relates to devices and methods for endoscopic harvesting of blood vessels within a patient's limb using thermal energy applied to cut and cauterize tissue and branches, and more particularly to achieving temperature management within an endoscopic vessel harvesting system that can have an augmented reality device worn by a person performing the harvesting procedure.

Background Art

[0003] In relation to coronary artery bypass grafting (CABG), a blood vessel or vascular section, such as an artery or vein, is "harvested" (i.e., removed) from its natural location within a patient's body for use elsewhere in the body. In CABG surgery, for example, this blood vessel is used to form a bypass between an arterial blood source and a coronary artery to be bypassed. Suitable sources for vascular conduits to be used as bypass grafts include, in particular, the saphenous vein in the leg and the radial artery in the arm.

[0004] To avoid the disadvantages and potential complications of harvesting via serial incisions, endoscopic surgical techniques have been developed for subcutaneous harvesting of venous sections (e.g., saphenous veins). One such minimally invasive technique employs a small incision to locate the desired vessel and introduce one or more endoscopic harvesting devices. Initial dissection is performed by introducing dissection instruments through the incision to create working space and separate the vessel from the surrounding tissue. Cutting instruments are then introduced into the working space to separate the vessel from the connective tissue and its collateral branches. The collateral branches can be cut and cauterized using the cutting instruments.

[0005] In one typical procedure, the endoscopic entry point is located near the midpoint of the vessel to be harvested, where the dissection and cutting of the branch advances bidirectionally along the vessel from the entry point. To remove the desired section of the vessel, a second small incision or puncture is made at one end of this desired section, and this vessel section is ligated. A third small incision is made at the other end of the thus ligated vessel section, thereby allowing the complete removal of this desired vessel section through the first incision. Alternatively, if the length of the endoscopic device is sufficient to obtain the desired length of vessel while working in only one direction along the vessel from the entry point, only the first two incisions may be necessary.

[0006] An example of a commercially available product for performing the endoscopic venous sampling described above is the VirtuoSaph Plus® endoscopic vascular sampling system from Terumo Cardiovascular Systems Corporation of Ann Arbor, Michigan. This type of endoscopic vascular sampling system is also shown in U.S. Patent Nos. 7,331,971 and 8,048,100, as well as U.S. Patent Application Publication Nos. 2010 / 0292533 and 2012 / 0035606, which are incorporated herein by reference in their entirety.

[0007] Dissection tools typically comprise a longitudinal rod made of stainless steel or plastic, with a tip at one end and a surgeon's handle at the other. The rod may have a PTFE coating to reduce sliding resistance. The tip tapers to a non-sharp end and is made of clear plastic. Dissection progresses along the outer circumference of the harvested vessel to separate it from the surrounding tissue and expose its side branches, so that the vessel can be cut using a cutting tool. In the VirtuoSaph® Plus system, the cutting tool for cutting and cauterizing branches has the form of a V-cutter, where the V-shaped tip is extendable from the distal end of the unit to guide the branch to be cut into a longitudinal slit. An electrode adjacent to the slit is electrically excited with a high-frequency voltage (e.g., bipolar electrosurgical energy) to cauterize and decompose the branch by coagulation. Furthermore, the V-Keeper extends from the distal end to capture the blood vessel and guide the tool along it.

[0008] An internal endoscopic view is provided to the user via an optical system having a camera and video display. The camera may be installed inside the distal tip of the collection device. Alternatively, a lens and optical fiber mounted inside the collection device can transmit images to a camera located at the remote end of an optical fiber outside the collection device or located inside the handle of the device. The view is illuminated by a light source, such as an LED, installed at the tip of the collection device (dissection instrument or cutter), or by a remote source that inputs light into an optical fiber extending through the collection device to emit light from the tip. [Prior art documents] [Patent Documents]

[0009] [Patent Document 1] U.S. Patent No. 7,331,971 [Patent Document 2] U.S. Patent No. 8,048,100 [Patent Document 3] U.S. Patent Application Publication No. 2010 / 0292533 [Patent Document 4] U.S. Patent Application Publication No. 2012 / 0035606 [Overview of the Initiative] [Problems that the invention aims to solve]

[0010] The endoscopic camera view during the dissection or cutting phase is displayed on a computer monitor. Significant training may be required for the user to become proficient in appropriately adjusting their movements with dissection / cutting instruments while looking at the monitor and taking their eyes off the patient.

[0011] Thermal energy applied to a lateral branch or to the connective tissue surrounding the target vessel has the potential to extend to the target vessel, especially if the lateral branch is severed at a point excessively close to the target vessel. Thermal damage to the target vessel can accumulate during the harvesting procedure, and this may go undetected until the target vessel is finally removed from the body and examined. [Means for solving the problem]

[0012] In one aspect of the present invention, a vascular sampling system comprises an elongated sampling instrument for insertion into the body along the path of a target vascular, wherein the target vascular is connected to at least one side branch, and the sampling instrument has a cutter for applying thermal energy to sever and cauterize the side branch. An endoscope camera is mounted on the sampling instrument to capture a visible light image from the distal tip of the instrument within a dissection tunnel around the target vascular. A thermal camera is mounted on the sampling instrument to capture a thermogram (i.e., a thermal image) corresponding to the visible light image to characterize the temperature present on each surface within the tunnel. An image processor (e.g., a control device) displays a video stream containing the visible light image and an overlay depicting the temperature present on at least some of the surfaces when thermal energy is applied. A display presents the video stream and the overlay to the user. The display may be, for example, a computer monitor or an augmented reality display. [Brief explanation of the drawing]

[0013] [Figure 1] This is an external view showing the saphenous vein taken from the leg. [Figure 2] This is a side view showing a conventional anatomical instrument unit. [Figure 3] This is a side view showing a conventional cutting unit. [Figure 4] This is a plan view showing another conventional non-sharp dissection instrument equipped with an endoscope. [Figure 5] This is a partial cross-sectional view showing the anatomy of blood vessels. [Figure 6] This is an endoscopic camera view showing the V-keeper and V-cutter of the sampling device unit deployed within the working tunnel around the target blood vessel. [Figure 7] This is a block diagram showing one embodiment of a vascular sampling system using augmented reality. [Figure 8] This is a schematic diagram illustrating an augmented reality display with streaming endoscopic video images overlaid on a real-world view. [Figure 9] A diagram showing an endoscopic visible light view having an overlay display depicting the temperature of a selected surface obtained using a thermal imager. [Figure 10] A perspective view showing a display end of a sampling instrument having cutting jaws for cutting and having a visible light camera and a temperature camera. [Figure 11] A diagram showing another endoscopic visible light view having an overlay display depicting the temperature of a selected surface obtained using a thermal imager. [Figure 12] A diagram showing an endoscopic visible light view having an additional overlay display indicating the optimal location for cutting a side branch. [Figure 13] A diagram showing another endoscopic visible light view having an additional overlay display superimposed on the optimal location for cutting a side branch. [Figure 14] A schematic diagram showing an augmented reality display having a continue / abort (Go / No-Go) identification indicator in the "abort (No)" state, indicating a cutter guide instruction. [Figure 15] A schematic diagram showing an augmented reality display having a continue / abort (Go / No-Go) identification indicator in the "continue (Go)" state. [Figure 16] A graph showing the accumulated thermal energy at a specific portion of a target vasculature. [Figure 17] A schematic diagram showing a display of a visible light endoscopic view and an indicator showing the accumulated thermal damage to a target vasculature. [Figure 18] A schematic diagram showing an augmented reality display having an endoscopic image and a text overlay display showing the accumulated thermal damage. [Figure 19] A flowchart showing a method of guiding a cutter to an optimal cutting location on a side branch and monitoring thermal damage to a target vasculature. [Figure 20] A schematic diagram showing an augmented reality display representing a warning message generated when the accumulated thermal damage exceeds a threshold. [Modes for carrying out the invention]

[0014] This invention relates to the jointly filed U.S. Patent Application No. (0320PRO), titled Augmented-Reality Endoscopic Vessel Harvesting, which is incorporated herein by reference in its entirety.

[0015] Referring to Figure 1, patient 10 has a saphenous vein 11 in a lower limb 12. An incision 13 is made just above the vein 11, and tissue is separated from the incision 13 to access the vein. Endoscopic instruments are inserted through the incision 13 to separate the vein 11 from the connective tissue and then to sever and cauterize the side branches extending from the vein 11. A second incision or puncture wound 14 is made at a second location on the limb 12, so that a second end of the vein 11 may be severed. The vein 11 is then drawn out through one of the incisions. The entry point and / or the second incision or puncture wound may be located at various locations along the vein 11, as shown in Figure 15.

[0016] Figure 2 shows a known dissection instrument unit 16 for endoscopic dissection of a saphenous vein or other blood vessel, which is inserted through the initial incision and pressed into the fat along the direction of the blood vessel to separate it from adjacent tissue. The dissection instrument unit 16 has a handle 18 connected to a longitudinal rod 19 having a dissection instrument tip 17 at its distal end. A receiver 20 at the end of the handle 18 receives an endoscope and optical cable (not shown) extending through the rod 19 to the dissection instrument tip 17, and the dissection instrument tip 17 is transparent to allow visualization of the blood vessel and surrounding tissue. An inhalation tube 21 is part of an inhalation gas channel that passes through the handle 12 and extends to a discharge port located within or near the tip 17. The tube 21 is connected to a source of CO2 or other inhalation gas to fill cavities adjacent to the blood vessel when cavities are formed.

[0017] Following an initial blunt dissection around the blood vessel, a harvesting device cutting unit 22, as shown in Figure 3, is used subcutaneously to grasp the dissected blood vessel and to sever any branches or connective tissue connected to the blood vessel. The harvesting device 22 has an elongated sleeve member 24 and a handle 23 connected to an endoscope receiver 25. At the distal end of the sleeve 24, there is a vascular keeper (V-keeper) 26 for holding the dissected blood vessel and a vascular cutter (V-cutter) 27 for severing branches. The V-keeper 26 is operated by a V-keeper button 28 located on the handle 23. The V-cutter 27 is extended or retracted by operating a V-cutter extension button 29 located on the handle 23. An inhalation device tube 30 is adapted to be connected to an inhalation source to deliver gas to the distal end of the sleeve 24 via a gas channel extending between the handle 23 at the proximal end and the discharge port at the distal end. A bipolar cord or integrated bipolar cord 31 is connected to a source of high-frequency voltage and has conductors for supplying voltage to electrodes on a V-cutter 27 for cutting and cauterizing lateral branches and connective tissue.

[0018] In some embodiments, cutting and cauterization can be achieved using jaws similar to those of ordinary scissors instead of a V-cutter. The jaws may have electrodes or other excitation devices on their inner surface, which are pressed against the side branches to be cut.

[0019] Figures 4 and 5 show another vascular sampling system comprising an endoscope unit 32 for performing intracellular monitoring of a patient, a dissection instrument unit 36 ​​for dissecting blood vessels within the body, and a trocar 40 for assisting in the insertion of the endoscope 32 and the dissection instrument unit 36 ​​into the body. The optical system is shown as a rigid endoscope 32 having an elongated rod-shaped insertion portion 33. The proximal end of the insertion portion 33 is connected to an end adapter 34 for transmitting endoscopic images. A light guide port 35 protrudes from the end adapter 34 to be connected to a light guide cable that supplies illumination light to the endoscope 32. In other embodiments, the optical system may employ a camera and LED light source mounted at the distal end of the endoscope 32, connected via an electrical cable for power supply, and a video image processor.

[0020] The dissection instrument unit 36 ​​has a tubular main body portion with a hollow longitudinal rod 37 into which the endoscope 32 is inserted. The endoscope 32 is inserted into or removed from the longitudinal rod 37 through the handle portion 38. The material of the longitudinal rod 37 may be a fluoropolymer. The most suitable material for forming the outer surface of the longitudinal rod 37 is polytetrafluoroethylene (PTFE). By using a fluoropolymer, the friction generated by moving the rod 37 through connective tissue is reduced, thereby reducing the force required to perform the dissection.

[0021] A non-sharp dissection instrument tip 39 is positioned at the distal end of the longitudinal rod 37. The tip 39 has a conical shape and includes a transparent synthetic resin material to assist in visualizing the direction along the tip 39 using the endoscope 32. A trocar 40 guides the dissection instrument unit 36 ​​into the incision site. The outer surface of the trocar 40 has projections for engaging with biological tissue and a retaining portion 41 for holding the trocar 40 against biological tissue 43 (e.g., the patient's skin). Because the insertion direction of the dissection instrument 36 is aligned with the direction of the target blood vessel 45 being dissected, in order to dissect peripheral tissue 46 from the blood vessel 45 (to create a working tunnel 44), the surgeon will gradually insert the dissection instrument while viewing the endoscopic image on a display 48 connected to the endoscope 32 by a cable 47.

[0022] After dissecting the working tunnel along the target vessel, the dissection instruments may be removed, and cutting instruments may be inserted into the working tunnel to separate the target vessel from any side branches and any undissected connective tissue. Figure 6 is an endoscopic view (in other words, field of view) as seen during vascular harvesting, where the target vessel 45 (e.g., saphenous vein) is held within the V-keeper 26. Side branches 50 extend from the vessel 45 into the previously made tunnel during blunt dissection. The V-cutter 27 is positioned to extend toward the side branches 50 to cauterize and separate them in order to prepare the section of the vessel 45 for removal. Because side branches such as side branches 50 extend radially away from the vessel 45, the harvesting device must be rotated around the vessel 45 to directly access all different side branches along the length of the vessel 45 being harvested.

[0023] Figure 7 shows a first embodiment of a vascular sampling system utilizing augmented reality. A patient 51 has an incision 52 into which a sampling instrument (e.g., a dissection instrument or cutter) 53 is inserted. The instrument 53 is connected to an inhalation gas source 54, a power supply 55, and a light source 56. Camera images from the instrument 53 are sent (coupled) to a control device (e.g., an image processor) 57, and the processed images may be sent to a conventional display 58 (such as a computer monitor) and / or an augmented reality display 60.

[0024] Power 55 may be supplied to the instrument 53 under the control of a switch 59 (e.g., a foot pedal switch) when the cutter of the cutting instrument is positioned to cut / caute a lateral branch or connective tissue, or to spot-caute, for example, the wall of a tunnel. Other control selectors, such as a switch on the handle of the instrument 53, may also be used by the user to excite the cutting electrode on the cutter at a desired moment. The magnitude of the energy applied by the cutter for cutting and cauterization can be controlled by the duration for which the switch 59 is activated, by controlling the magnitude of the applied voltage, or, for example, by changing the modulation pattern of the applied voltage.

[0025] Augmented reality (AR) is an interactive experience of the real world environment in which objects present in the real world are enhanced by computer-generated perceptual information, sometimes through multiple types of senses, including sight, hearing, touch, somatosensory, and olfactory. Specifically, an augmented reality display may have eyewear comprising an open view field for viewing physical objects and an augmented viewing portion configured to represent glyphs and / or video content (e.g., streaming video images).

[0026] The augmented reality display 60 may consist of a head-mounted display, sometimes referred to as “smart glasses.” For example, the display 60 may take the form of glasses, a visor, an open area, or a face shield worn on the head or face by a user (e.g., a surgical technician or a physician’s assistant) during a harvesting procedure. The display 60 may have a view field through which a user can see physical objects within their field of vision, and may be sometimes referred to as “non-obstructive” or “non-obstructive head-up display (HUD).” For example, there may be a clear portion of glass, plastic, or a similar transparent material through which light emitted from a physical object passes into the user’s eye. In some embodiments, the display 60 may have a solid or opaque portion that completely or partially obstructs the user’s view, and may be sometimes referred to as “obstructive” or “obstructive HUD.” This view field may have one or more screens (e.g., light-emitting diode screens, i.e., LED screens) along with one or more cameras that capture video data from the user’s viewpoint. Therefore, the video is displayed on the screen, thereby providing the user with a view field that is similar to a clear view of the physical environment.

[0027] In another embodiment, the display 60 may have a retinal projection device configured to project an image directly onto the wearer's eyes. In some cases, the retinal projection device may have a clear portion of glass, plastic, or similar transparent material through which light emitted from a physical object passes into the user's eyes. In some cases, the display 60 with the retinal projection device may have one or more cameras that capture video data of the user's viewpoint. The video is then represented and projected onto the user's eyes, thereby providing the user with a view field similar to a clear view of the physical environment. In some implementations, the display 60 may be configured to take into account the user's visual impairment. For example, the retinal projection device may be configured to provide projected images to users with corneal opacity or cataracts in a manner that is clear to such users.

[0028] In yet another embodiment, the display 60 may have a half-mirror portion made of glass, plastic, or a similar transparent material, through which light emitted from a physical object passes to the user's eyes, during which time light is emitted onto the half-mirror view field to represent glyphs and the like.

[0029] The augmented reality display 60 is configured to represent glyphs (e.g., characters, symbols, colored overlays, etc.) and to represent video within a view field. For example, a light emitter may emit light into a transparent view field, resulting in the user seeing a reflection of light. In another embodiment, if a screen is used to display video from the user's viewpoint, glyphs and video may be displayed superimposed on the viewpoint video. In either case, the display 60 displays the glyphs and video as overlays superimposed on a view of a physical object.

[0030] The display 60 may have other feature components. For example, it may include a microphone or earphone for connecting to an internal communication device, a mobile phone, or other telecommunications device. This may allow the surgeon to communicate with people in the same facility or with people further away via the microphone or earphone.

[0031] As will be discussed in more detail later, many different types of glyphs and video images may be displayed to the user. A selector (not shown) may allow the user to generate screen update commands to modify the content of the display 60 (e.g., changing the characteristics of the displayed items by selecting a different glyph, scrolling through and examining the patient's monitored physiological parameters, selecting a different image source, or zooming in on a region of an image). Because it is desirable for the user (e.g., the wearer of the display 60) to maintain their handgrip over the collection instrument, the selector may be configured to receive commands while the user continues to hold the instrument. The selector may consist of a manual control device (e.g., an electric push-button switch or toggle) installed within its gripping area on the instrument. Alternatively, the selector may consist of a hands-free device that senses other actions by the user. For example, the selector may have an eye-tracking camera that detects specific eye movements of the user designated to trigger a corresponding update command. Alternatively, the selector may have 1) a microphone and voice recognition system for enabling the user to generate screen update commands via voice commands, 2) a motion sensor that responds to predetermined movements of the user, or 3) a foot pedal (e.g., connected to an image processor 57 via a Bluetooth® connection) equipped with one or more switches for generating desired update commands.

[0032] Figure 8 shows one embodiment of the view field 61 on and through an augmented reality display as it may be seen by a user (e.g., a wearer of a head-mounted display). The transparent portion of the view field 61 without an overlay display provides an actual live view of the scene around the wearer, including the patient's limbs 62 and endoscopic instruments 63. The overlay display 64 provides a copy of the instantaneous endoscopic image received by the image processor / control device from the endoscopic camera (e.g., displayed on the side of the view field).

[0033] In addition to visible light endoscopic images for guiding and using the sampling instrument, the present invention employs a thermal camera connected to the sampling instrument (e.g., carried together with the endoscope), so that the image data includes a temperature overlay display based on a thermogram captured by the thermal camera. The temperature overlay display allows the user to monitor heating of the target vessel and its side branches and to better manage the total heat load on various regions of the target vessel to maintain its health. Figure 9 shows an exemplary display view (e.g., as it may be seen on a computer monitor) combining a colored temperature overlay display 70 with an endoscopic visible light image (including the V-keeper 26, V-cutter 27, target vessel 45, and side branches 50). The temperature overlay display 70 can symbolize lower temperatures with cool colors (e.g., from blue to green as the temperature increases) and higher temperatures with warm colors (e.g., from yellow to red as the temperature increases). To show a visual correspondence between color and temperature, a color key 71 may be displayed as part of the overlay display. As an alternative to coloring, the temperature overlay can also be generated as an area of ​​speckles or dots of the corresponding color. Preferably, the temperature overlay can be limited to only the relevant surfaces in the visible light view, such as the walls of the side branches, target vessels, and tunnels. Limiting the temperature overlay 70 to only important locations can be achieved by 1) utilizing pattern recognition of the visible light image to identify the location of adjacent portions of the side branches 50 and target vessels 45, or by 2) representing the temperature only when it exceeds a predetermined threshold (e.g., 40°C) between normal body temperature and a temperature that would only be present in response to, for example, the use of cutting / cauterizing electrodes.

[0034] Figure 10 shows a sampling instrument in the form of a cutting tool 72 having an elongated body 73. A visible light imager (in other words, an imaging device) 74 and a thermal imager 75 are located at the distal end of the body 73. A cutter 76 may be able to slide longitudinally within the body 73 and may have an indirect joint for positioning a pair of jaws 77 in a fixed position over the side branch to be cut / cauterized. The jaws 77 may have bipolar electrodes on their inward-facing surfaces for applying heat to the side branch compressed between the jaws 77. The tool 72 may further have a V-keeper 78.

[0035] The visible light imager 74 may comprise a rigid rod (e.g., glass or optical fiber) having a molded distal end or carrying a lens for providing an endoscopic view. The proximal end of the rigid rod (not shown) may have an eyepiece and / or optical connection to a camera for capturing images transmitted to an image processor. Light from an optical port (not shown) may be emitted from the distal end of the rod to illuminate the tunnel with visible light. Alternatively, the imager 74 may comprise an electronic image sensor (e.g., a CCD camera) and an LED for illuminating the tunnel. The imager 74 may consist of a stereoscopic view (either an electronic 3D imager or a rigid 3D endoscope) for obtaining depth information that can help identify target vessels and / or side branches.

[0036] The thermal imager 75 may include a passive, thermo-sensitive array configured to detect surface temperature as a thermograph overlapping with a visible light image. The pixel size (e.g., resolution) in the thermograph does not need to be as fine as that of visible light pixels. This resolution only needs to be sufficient to identify areas that are overheated and to depict coarse temperature gradients.

[0037] Figure 11 shows another displayed video frame with a temperature overlay display, in which a cutter 76 with jaws 77 is used to cut a lateral branch 50. For clarity, the jaws 77 are shown open near the cutting area of ​​the lateral branch 50, during which the temperature overlay display 80 and color key 81 correspond to temperature data observed using the jaws, which are activated and compressed against the lateral branch 50 (for example, the maximum temperature occurs at the contact point within the lateral branch 50, and the hot area spreads through the lateral branch 50 and reaches the target vessel 45). By depicting the temperature data, the user can gain an awareness of the heating obtained from the cutting / cauterizing operation and thus take appropriate action.

[0038] In some embodiments of the present invention, automatic image analysis is used to identify the optimal location for cutting a side branch and to inform the user of the optimal location. For example, the diameter and length of a side branch in a tunnel may be characterized by image analysis (e.g., pattern recognition and depth detection). The dimensions of the side branch to be cut may be used to estimate the energy level (e.g., the amount of heating required) that will be sufficient to cut and cauterize the side branch. Based on the energy level and the estimated heat flow model, the minimum required distance between the cutting location and the main target vessel may be determined. Using this minimum distance, a cutting indicator overlay display 82 may be added to the view field, as shown in Figure 12, to provide guidance to the user. The cutting overlay display 82 may include a character indicator 83 that identifies the overlay display as relating to the cutting zone and a pointer 84 that points to a spatial location on the side branch 50 that will allow for a suitable cut without significantly affecting the main target vessel. The overlay display 82 may also display a temperature overlay display 80 simultaneously. Figure 13 shows an alternative embodiment in which a Go / No-Go overlay display 85 is superimposed on the side branch 50 to highlight suitable locations for cutting, as well as locations where cutting is undesirable (for example, due to the possibility of overheating the target vessel). The overlay display 85 may have color tinting, such as green zones (where the cutting operation is "Go") and red zones (where the cutting operation is "No Go"). A color key 86 is generated on the display to inform the user which colored area corresponds to a suitable cutting area. In addition, the image processor / control device may be equipped with an acoustic transducer (e.g., a speaker) to generate an acoustic signal that can be used to indicate a Go or No Go state. For example, a warning sound may be generated if the cutter 77 is positioned too close to the target vessel 45 on the side branch.

[0039] To assist the user in applying an appropriate amount of thermal energy sufficient to perform cutting / cauterization without using significantly excessive energy, a diameter indicator 87 may be displayed on the display according to the estimated diameter of the side branch calculated by the image processor / control device. The user can learn from experience how much energy (e.g., duration for exciting the cutting electrode) may be required for side branches of various diameters. Alternatively, a recommended amount of thermal energy to be applied (e.g., target energy) may be determined by the control device and displayed as an energy indicator 88. The optimal target energy for detaching the side branch may be displayed, for example, as the duration for exciting the cutter. In some embodiments, the amount of energy applied may be automatically controlled when the user initiates a cutting command to detach a side branch (e.g., by pressing down a foot pedal or other switch).

[0040] The camera views and overlay displays in Figures 11-13 may be displayed on a computer monitor or an augmented reality (AR) display. Figures 14 and 15 show one embodiment of a view field 90 of an AR display having a video overlay display 91 (streaming a live endoscopic view from a visible light camera). To assist the user in guiding the cutting tool (e.g., a jaw or V-cutter) to the optimal / preferred cutting location during the harvesting procedure, a flag 92 is represented in the view field 90, having a no-go state (Figure 14) when the cutter is not in a preferred location and a go state (Figure 15) when the cutter is in a preferred location. When in the no-go state, the view field 90 may have an overlay display 93 of guidance instructions having arrows or glyphs corresponding to the direction of movement of the jaw or V-cutter, which is the cutting tool, to reach the preferred location. The arrows may indicate, for example, the movement to be made within a reference frame of the visible light image. By following the superimposed guidance display, the user can obtain the "go" state when they reach a point where the heating of the target blood vessel is maintained below the heat threshold.

[0041] In some embodiments of the present invention, thermal exposure to a target blood vessel is accumulated using a thermal model. For example, the high temperature occurring at each location on the target blood vessel may be integrated over time, and this integral is represented as a parameter display indicator. The magnitude displayed on the parameter display corresponds to the accumulated thermal exposure, and as a result, the user can monitor and avoid harmful exposure to the target blood vessel.

[0042] Figure 16 is a graph showing the thermal model, where trace 93 plots the measured temperature of a spot monitored on the target vessel over a certain period of time. When trace 93 exceeds the nominal temperature threshold (representing the temperature at which vessel damage may begin) starting at time t0, the control unit can begin calculating an integral corresponding to the area 94, which is proportional to the excess thermal energy absorbed by the target vessel. i The integral value, proportional to the area 94, can be used to determine the appearance of the parameter bar graph 95, as shown in Figure 17, where the height of the colored bar indicates the integral value. Any scale may be shown to provide the user with a reference point when determining the severity of the accumulated heat exposure. Alternatively, as shown in Figure 18, the view field 96 may have a warning indicator 97 in the form of a text message when thermal damage occurs. The indicator 97 may further have data on the percentage of thermal damage, which can be dynamically updated as more heat accumulates in the target blood vessel.

[0043] Figure 19 shows a flowchart of a method according to one embodiment of the present invention, in which a side branch (e.g., integrated with the target vessel and the branching point of the side branch) is identified in step 100. This identification may be performed automatically by pattern recognition, or may be performed or assisted by manual indication by the user. The diameter and length of the side branch are calculated in step 101. In step 102, a suitable (e.g., optimal) placement of the cutting tool and an optimal energy level are determined according to the diameter and length of the side branch.

[0044] In step 103, the relative location of the preferred placement and the current position of the cutting tool may be compared, and guidance instructions are presented to the user to achieve the preferred placement (e.g., cutting indicator, marker, and movement pointer). Furthermore, the calculated optimal energy may be displayed to the user. As cutting / cautering energy begins to be applied, in step 104, the temperature rise of the target vascular is monitored. Compression and / or indentation of the cutting tool and the supply of thermal energy are continued in step 105 for an optimal duration which may be controlled automatically or manually. In step 106, a check is performed to determine whether the accumulated heat above the nominal temperature exceeds the damage threshold. If it does not, the process returns to step 104 to continue monitoring, or to step 100 when the cutting operation at that point is complete. If the accumulated heat exceeds the damage threshold, a warning is issued in step 107.

[0045] As shown in Figure 20, a warning may be presented on the augmented reality display view field 110 as a warning indicator 111 that notifies the user that the damage limit has been exceeded. [Explanation of Symbols]

[0046] 10 patients 11 Saphenous vein 12 Lower Limbs 13. Incision site 14 Puncture wound 16 Dissection Instrument Unit 17 Dissection instrument tip 18 handle 19 Longitudinal rods 20 receivers 21 Inhalation tube 22 Sampling device cutting unit 23 Handle 24. Long, slender sleeve component 25 Endoscope receiver 26 V-Keeper 27 V-Cutter 28 V-Key Buttons 29 V-Cutter Extension Button 30 Inhalation device tube 31 Bipolar Code 32 Endoscopy Units 33 Insertion part 34 End adapters 35 light guide ports 36 Dissection Instrument Unit 37 Longitudinal rods 38 Handle section 39 Non-sharp tips of dissection instruments 40 Trocar 41 Holding part 43. Living tissue 44 Work Tunnel 45 Target Vessel 46 Peripheral tissues 47 Cables 48 displays 50 side branches 51 patients 52 Incision site 53 Collection equipment 54. Source of inhaled gas, gas 55 Power supply, electric power 56 Light source, light 57 Image processor, control unit 58 Conventional displays 59 switches 60 Augmented Reality Display 61 Viewfield 62 Limbs 63 Endoscopic Instruments 64 Overlay Display 65 Overlay Display 70 Temperature Overlay Display 71 Color Keys 72 Cutting Tools 73 Long and slender body 74 Visible Light Imager 75 Thermal Imager 76 cutters 77 Jaw area, cutter 78 V-Keeper 80 Temperature Overlay Display 81 Color Keys 82 Cutting indicator overlay display 83 character indicator, disconnection zone 84 pointers 85. Continue / Cancel Overlay Display 86 color keys, cutting area 87 Diameter indicator, branch 88 Energy Indicators, Optimal Energy 90 Viewfield 91 Video Overlay Display 92 Flag, Cancelled, Continue 93 Overlay display and tracing of guidance instructions 94 area 95-parameter bar graph 96 Viewfield 97 Warning indicator, heat damage 110 Augmented Reality Display View Field 111 Warning indicator, limit exceeded.

Claims

1. A vascular sampling system, An elongated sampling instrument for insertion into the body along the pathway of a target vessel, wherein the target vessel is connected to at least one side branch, and the sampling instrument has a cutter for applying thermal energy to sever and cauterize the side branch; An endoscope camera installed on the sampling device captures a visible light image from the distal tip of the sampling device within the dissection tunnel surrounding the target blood vessel, A thermal camera installed in the sampling device captures a thermogram that matches the visible light image in order to characterize the temperature present on each surface within the dissection tunnel, An image processor that represents a video stream including the visible light image, and an overlay display that depicts the temperature present on at least some of the surfaces when the thermal energy is applied, A display that presents the video stream and the overlaid display to the user. A vascular sampling system equipped with the following features.

2. The system according to claim 1, wherein the superimposed display includes a colored display portion having multiple colors corresponding to each temperature class.

3. The system according to claim 1, wherein the image processor identifies the target vessel and the side branches, and the superimposed display depicts the temperature present in at least a portion of the target vessel and at least a portion of the side branches.

4. The system according to claim 3, wherein the image processor uses image analysis to identify the target vessel and the side branches.

5. The system according to claim 1, wherein the image processor identifies the target vessel and the side branches, and the image processor represents a cutting indicator for guiding the cutter to a suitable location for cutting the identified side branches in order to limit the thermal load applied to the target vessel.

6. The system according to claim 5, wherein the cutting indicator includes a marker that points to a spatial location on the side branch.

7. The system according to claim 5, wherein the cutting indicator includes a highlight region superimposed on the location of the preferred location in the visible light image.

8. The system according to claim 5, wherein the cutting indicator includes a guide indicator corresponding to the direction of movement of the cutter to reach the preferred location.

9. The system according to claim 5, wherein the cutting indicator includes a flag having a stop (no-go) state when the cutter is not in the preferred location and a continue (go) state when the cutter is in the preferred location.

10. The system according to claim 5, wherein the image processor identifies the preferred location for detaching the identified side branch, so that the distance from the preferred location to the target vessel keeps the heating of the target vessel below a thermal threshold.

11. The system according to claim 1, wherein the image processor accumulates the thermal exposure received by the target blood vessel, and the image processor displays a parameter display indicator corresponding to the accumulated thermal exposure.

12. The system according to claim 11, wherein the image processor displays a warning indicator if the accumulated heat exposure is greater than a damage threshold.

13. The system according to claim 1, wherein the image processor identifies the target vessel and the side branches, and the image processor represents a diameter indicator according to the estimated diameter of the side branches.

14. The system according to claim 1, wherein the image processor identifies the target vessel and the side branches, the image processor determines a target energy for detaching the side branches according to the estimated diameter of the side branches, and the image processor represents an energy indicator according to the target energy.

15. The system according to claim 1, wherein the display comprises an augmented reality display worn by the user.

Images