Systems and methods for using detectable radiation in surgery - Patents.com

Abstract

The surgical camera system includes a camera having at least one sensor configured to capture image data including a first wavelength range and a second wavelength range. An excitation light source emits excitation emissions at an excitation wavelength. A controller is in communication with the at least one sensor of the camera. The controller is configured to process the image data from the at least one sensor and detect at least one fluorescent portion of the image data in response to fluorescent emissions produced by the fluorescent agent in the second wavelength range. The controller is further configured to generate enhanced image data revealing the at least one fluorescent portion of the surgical instrument in the image data.

Description

[Technical field]

[0001] FIELD OF THE DISCLOSURE The present disclosure relates generally to surgical visualization systems, and more particularly to devices and methods that utilize detectable radiation in surgery. [Background technology]

[0002] Generally speaking, during endoscopic surgery, the surgical field is cluttered with different anatomical structures and surgical instruments, as well as fluids that may impede the surgeon's view of the associated anatomical structures and surgical instruments. It is often difficult to ascertain the location of the surgical instruments relative to the different anatomical structures and to properly position the surgical instruments in the surgical field. The present disclosure provides various systems and methods for improving visualization of procedures implemented in a surgical environment. Summary of the Invention

[0003] In various embodiments, the present disclosure provides a surgical instrument that includes a fluorescent agent. The fluorescent agent can be incorporated into a surgical tool or instrument to help distinguish the instrument, or a portion of the instrument, from its surroundings in the surgical field. In general, the fluorescent agent can be excited in response to receiving an excitation emission of radiation over a range of excitation wavelengths. In response to this excitation emission, the fluorescent agent emits a fluorescent emission of radiation in a known wavelength band that is detectable in image data captured by the surgical camera. In response to detecting the fluorescent emission, the camera can respond in a number of ways to improve visualization, detection, and / or identification of the surgical instrument associated with the fluorescent agent. In some cases, the excitation emission and / or the fluorescent emission can correspond to a wavelength of light capable of penetrating biological tissue. In such cases, the fluorescent emission can be detected by the camera system to identify the location or presence of the surgical instrument through the biological tissue. Once identified, a display controller of the camera system can overlay or provide a visual indication of the location of the fluorescent portion of the surgical instrument in the image data to improve visualization during surgery. These and other features are described in the detailed description below.

[0004] These and other features, objects and advantages of the present disclosure will become apparent when the following description of the present disclosure is read in conjunction with the accompanying drawings.

[0005] The features, aspects, and advantages of the present disclosure will become better understood with reference to the following description, appended claims, and accompanying drawings. [Brief description of the drawings]

[0006] [Figure 1] FIG. 1 is a representative diagram of a surgical environment demonstrating a camera system for improved intraoperative visualization. [Figure 2A] FIG. 1 is a simplified diagram of a camera configured to excite a fluorescent agent and identify the fluorescent emissions produced in a surgical field. [Figure 2B] FIG. 1 is a simplified diagram demonstrating a surgical instrument illuminated with visible light. [Figure 2C] FIG. 2C is a simplified diagram illustrating the surgical instrument of FIG. 2B enhanced to highlight fluorescent areas. [Diagram 3] FIG. 1 is a simplified cutaway view demonstrating a surgical instrument including surgical sutures and an anchor including a fluorescent agent. [Figure 4] FIG. 4 is a representative diagram illustrating the suture and suture anchor of FIG. 3 enhanced by a camera system. [Figure 5A] FIG. 1 is a side view of a shaver including a plurality of fluorescent markings configured to identify an orientation. [Figure 5B] FIG. 2 is a side view of a surgical probe demonstrating a number of graduated markings identifying the dimensions of the surgical probe. [Figure 6] 1 is a representative diagram demonstrating enhanced image data captured by a surgical camera in a patient's cavity. [Figure 7] FIG. 1 is a perspective view of an arthroscopic procedure performed on a patient's shoulder. [Figure 8] 1 is a representative view demonstrating enhanced image data within a patient's shoulder cavity. [Figure 9] 1 is a representative view demonstrating enhanced image data within a patient's shoulder cavity. [Figure 10A] 1 is a perspective view illustrating a shoulder surgical procedure. [Figure 10B] FIG. 13 is a representative diagram demonstrating multiple sutures reinforced with distinctive colors or patterns for improved visualization. [Figure 11] 1 is a flow chart demonstrating a method of object or surgical instrument detection in a surgical field. [Figure 12] 1 is a flow chart illustrating a method for providing enhanced display of surgical image data. [Figure 13] FIG. 2 is a modified block diagram illustrating a surgical camera system and display according to the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0007] In the following description of the preferred embodiments, reference is made to the accompanying drawings, which show specific embodiments that may be practiced. Wherever possible, the same reference numbers will be used throughout the drawings to refer to the same or like parts. It is to be understood that other embodiments may be utilized and structural and functional changes may be made without departing from the scope of the present disclosure.

[0008] Referring to FIG. 1, a simplified representation of a camera system 10 is shown illustrating a typical surgical environment 12. As shown, the camera system 10 is implemented in combination with one or more surgical instruments 14, such as a surgical tool 14a or a shaver, in association with a control console 16. In operation, a camera or endoscope 18 of the camera system 10 can capture image data in the visible light range (e.g., 400 nm to 650 nm) as well as the near infrared range (e.g., 650 nm to 900 nm). This image data can be communicated to a display controller 20 configured to generate enhanced image data. This enhanced image data can highlight or visibly define one or more fluorescent portions 22 of the surgical instruments 14 to aid in the visualization of one or more of the surgical instruments 14 presented on a display device 24. In this configuration, the camera system 10 can provide improved visualization and enhanced viewing of the fluorescent portion 22 of the surgical instrument 14 to improve visibility, detection, and identification of the surgical instrument 14 when implemented at the surgical site 26 of the patient 28.

[0009] 2A-2C are simplified diagrams illustrating the operation of the camera system 10 to identify fluorescent emissions 32 output from a fluorescent portion of an exemplary surgical instrument 14. Referring now to FIG. 1 and FIG. 2A-2C, in various embodiments, the fluorescent portion 22 of the surgical instrument 14 may include a fluorescent agent implemented in a coating, insert, or embedded structure that may be excited to emit fluorescent emissions 32 in response to receiving excitation emissions 34. As illustrated in FIG. 2A, the excitation emissions 34 may be output from a first light source 36 and may correspond to emissions of light outside the visible spectrum. Additionally, visible light emissions 38 may be output from a second light source 40. The excitation emissions may include a wavelength or range of wavelengths configured to activate and excite the fluorescent agent incorporated in the fluorescent portion 22. In various examples, the excitation emissions 34 may include wavelengths within the near infrared range, which may correspond to wavelengths ranging from approximately 600 nm to 900 nm. The first light source 36 may correspond to a laser emitter module configured to output an emission over a range of 650 nm to 680 nm. In some cases, the first light source 36 may output an excitation emission 34 in a wavelength range of approximately 740 nm to 780 nm. A particular excitation wavelength associated with the first light source 36 and the excitation emission 34 may be selected to effectively activate the fluorescent agent of the fluorescent portion 22 such that the generated fluorescent emission 32 may be captured by one or more image sensors 42 of the camera system 10. In this manner, the camera system 10 may detect the presence or location of the surgical instrument 14 in response to detecting the fluorescent emission 32 in the image data.

[0010] As previously discussed, the camera system 10 may be configured to capture image data associated with the visible light emissions 38 as well as the fluorescent emissions 32. Once captured, the system 10 may enhance the image data representing the visible light with one or more overlays or graphics to generate enhanced image data that highlights and / or identifies a portion of the field of view 44 corresponding to the surgical instrument 14. To provide the enhanced image data, the camera controller 46 may be configured to selectively control each of the first light source 36 and the second light source 40, as well as process image data received from the first image sensor 42a and the second image sensor 42b. In a standard operating mode, the camera controller 46 may activate the visible light emissions 38 output from the second light source 40 to illuminate the surgical site 26 with wavelengths of light in the visible range (e.g., 400 nm to 650 nm). Reflections from the visible light emissions 38 may be captured by the second image sensor 42b, which may correspond to a visible light image sensor. Such operation can provide illumination of the surgical site 26 in visible wavelengths of light such that the camera controller 46 can output image data to the display controller 20 that reveals visible characteristics of the surgical site 26. An example of a revealed surgical instrument 14 illuminated by a visible light emission 38 and captured by a second image sensor 42b is shown in FIG. 2B. To represent the surgical instrument 14, only a simplified representative body is revealed in FIG. 2B, but the fluorescent portion 22 is depicted as being generally indistinguishable from the depicted surface cross-hatched portion illuminated by the visible light emission 38.

[0011] To generate enhanced image data, the camera controller 46 can activate the first light source 36 to output an excitation emission 34. In response to the excitation emission 34, the fluorescent agent of the fluorescent portion 22 can be excited to output a fluorescent emission 32. Concurrent with the activation of the first light source 36, the camera controller 46 can also activate the second light source 40 to illuminate the surgical site 26 with a visible light emission 38. As a result, the fluorescent emission 32 and the visible light emission 38 can be captured within the field of view 44 of each of the image sensors 42. The first image sensor 42a can correspond to a near-infrared image sensor configured to capture wavelengths of light within the near-infrared range (e.g., 650 nm to 900 nm), while the second image sensor 42b can be configured to capture reflected visible light emission 38. As shown in the figure, each of the image sensors 42 can include one or more optical filters designated as a first optical filter 52a and a second optical filter 52b. In operation, the optical filters 52a, 52b can filter the combined wavelengths of the fluorescent emissions 32 and the visible light emissions 38 within the field of view 44 to improve the fidelity of detection of the corresponding wavelengths detected by each of the image sensors 42a, 42b. In this manner, the camera controller 46 can process the image data recorded by each of the image sensors 42a, 42b to detect and distinguish between the fluorescent emissions 32 and the visible light emissions 38 in the field of view 44 that are representative of the surgical site 26.

[0012] Although generally described as optical filters 52, the first filter 52a and the second filter 52b may correspond to one or more high-pass, low-pass, and / or band-pass filters configured to transmit light over a range associated with the corresponding detection range of the image sensors 42a, 42b. For example, the first optical filter 52a may correspond to a band-pass filter configured to pass a range of near-infrared wavelengths from approximately 800 nm to 850 nm. In this configuration, the first optical filter 52a may be selected to have a center frequency of approximately 825 nm, which optical filter may effectively pass the wavelengths of light associated with the fluorescent emission 32 to the first image sensor 42a. In such a case, the fluorescent emission 32 may correspond to emission from a fluorescent agent in the form of an indocyanine green (ICG) dye. Thus, the fluorescent emission 32 output from the fluorescent unit 22 may pass through the first optical filter 52a within the band-pass range, such that the associated light from the fluorescent emission 32 is captured and identified by the camera controller 46. Similarly, the visible light emission 38 and corresponding light reflected from the surgical site 26 can be passed through a second optical filter 52b, which can be configured to pass wavelengths of light in the visible range (e.g., 400 nm to 650 nm). In this manner, the camera system 10 can actively detect the fluorescent emission 32 and generate an overlay, graphic, or other visual enhancement to augment the image data illuminated by the visible light emission 38 within the field of view 44.

[0013] In addition to the first and second optical filters 52a and 52b, the camera system 10 may further include additional filters, which may include one or more dichroic filters or mirrors configured to separate the fluorescent emission 32 from the visible emission 38. Such filters, generally referred to as optical filters 52, may be incorporated into the endoscope or camera 60, which may include the image sensor 42, the light sources 36, 40, and the camera controller 46, as well as the optical filter 52, in an integrated package. For example, the camera 60 may include each of the light sources 36, 40, the image sensor 42, the filter 52, and the camera controller 46, in a compact endoscope similar to that described below with reference to Figs. 3, 7, etc. In this manner, the camera system 10 may be implemented in a simple to operate package well suited for operation in the surgical environment 12. Although ICG is described in various examples of the present disclosure, other fluorescers, including methylene blue (MB), fluorescer, and protoporphyrin IX (PpIX), may be similarly implemented with the camera system 10.

[0014] Using image data associated with the visible light emissions 38 detected independently of the fluorescent emissions 32, the camera system 10 can provide an enhancement of the fluorescent portions 22 in the image data. In this manner, one or more colors, patterns, or other visual enhancements or overlays 62 can be superimposed or overlaid on the image data to generate enhanced image data for presentation on the display device 24. As shown in FIG. 2C, the location of the fluorescent portions 22 in the image data is highlighted by the overlay 62 such that the fluorescent portions 22 are clearly distinguishable from the remainder of the surgical instruments 14 as well as the local environment within the surgical site 26. As will be described in further detail throughout this application, the enhanced image data can be implemented in a variety of ways to provide improved visualization of the surgical site 26 and aid in identifying the presence, location, orientation, and / or dimensions of the various surgical instruments 14.

[0015] Generally, with reference to Figures 1, 2A, 2B, and 2C, the implementation and operation of the camera system 10 will be described in further detail. In general, ICG, fluorescein, PpIX, and methylene blue may correspond to dyes used in medical diagnosis. ICG has very low toxicity and high absorptivity in the wavelength range of about 600 nm to about 900 nm, and a peak absorptivity of about 780 nm. ICG emits fluorescence at a wavelength of about 830 nm. Furthermore, fluorescent agents such as ICG that emit near-infrared radiation may be detectable through biological tissue. As used herein, the terms "radiation" and "light" are used interchangeably. Another example of a fluorescent agent, PpIX, may be excited over the blue range (e.g., 405 nm) with a corresponding peak fluorescence of approximately 635 nm. MB is excited over the red to NIR color range (e.g., 600 nm) with a corresponding peak fluorescence of approximately 650 nm. Fluorescein has a peak absorption of approximately 490 nm with a fluorescence emission of approximately 520 nm. The gap between the absorption and emission ranges of each of the fluorescent agents is called the Stokes shift and can be used to distinguish between the wavelengths associated with the excitation emission 34 and the generated fluorescence emission 32.

[0016] In various examples, the fluorescent agent may be coated or used as a main part (e.g., embedded in the material or structure) of the surgical instrument 14. In some cases, the fluorescent agent may be incorporated into the fluorescent portion 22 of the surgical instrument 14 during manufacturing. For example, a plastic surgical instrument may have a fluorescent dye mixed into the plastic during manufacturing. Additionally, light-tight packaging may be used to protect the fluorescent dye from light until the surgical instrument 14 is ready for use. For example, but not limited to, a surgical instrument 14 such as a sponge, suture, pin, screw, plate, surgical tool, or implant may be coated with a fluorescent material. As used herein, the term "surgical tool" may include, but is not limited to, a biter, grasper, retriever, pick, punch, hook, probe, dissector, retractor, or scissors. A surgical instrument 14 may have a fluorescent agent coated on a portion to indicate the location, position, depth, orientation, or other characteristic of the surgical instrument. Thus, the fluorescent portions 22 of the surgical instrument 14 can be easily identified or detected within the enhanced image data provided by the camera system 10 .

[0017] As described below with specific reference to FIGS. 5A and 5B, the fluorescent agent may be incorporated into the various fluorescent portions 22 of the surgical instrument 14 in a pattern, shape, and / or alphanumeric characters to identify the surgical instrument 14 or to indicate the size, orientation, or proportion of the instrument 14 depicted in the image data. The presence of the fluorescent agent in the surgical instrument 14 may also allow the surgeon to quickly check and verify that no portion of the surgical instrument 14 is left at the surgical site 26. In some cases, the display controller 20 may be configured to process image data associated with the fluorescent emission 32 (e.g., corresponding pixels in the field of view 44) to identify or classify one or more surgical instruments 14 within the surgical site 26. For example, the display controller 20 may be configured to process a characteristic shape of the surgical instrument 14 or one or more symbols depicted in the image data captured by the first image sensor 42a (e.g., in the NIR range) to identify a type or category of the instrument 14 based on a computer vision template. Such identification is further described with reference to FIG. 12.

[0018] In various embodiments, the fluorescent agent in the surgical instrument 14 may be excited using a light source that emits excitation light within the excitation wavelength range of the particular fluorescent agent. For example, if ICG is used as the fluorescent agent, ICG fluorescence may be excited using light in the wavelength range of about 600 nm to about 900 nm, and in some cases, about 780 nm. In such a case, the light source 36 may be a light emitting diode or a laser emitting diode having a center frequency within the excitation range of ICG, or a center within that excitation range. The image sensor 42 may be, for example, a complementary metal-oxide-semiconductor (CMOS) sensor or a charge-coupled device (CCD) sensor. The camera 60 may also include optical elements, such as, for example, lenses, filters, mirrors, and prisms, and may target and independently detect the wavelengths of light associated with the visible light source 40 and the fluorescent emission 32.

[0019] In some embodiments, the camera 60 is implemented as an endoscopic camera and can include an image sensor 42, light sources 36, 40, and a light filter 52. Thus, the camera 60 can include both a first light source 36 as an excitation light source for exciting the fluorescent agent, and a second light source 40 in the form of a white light source for illuminating the surgical site 26 in the visible wavelength range. The camera 60 can further include a corresponding image sensor 42a or detector for detecting the fluorescent emission 32, and an image sensor 42b or detector for detecting and recording image data in the visible light range. In some cases, the camera 60 can have additional light sources for exciting multiple fluorescent agents or for detecting other non-visible attributes of the surgical field. An example of a camera system that can be used to detect fluorescent agents in a surgical instrument is the Arthrex Synergy ID™ camera system, which has a camera head and a camera control unit. The Arthrex Synergy ID™ camera system has a light source for exciting the fluorescence from ICG and can detect visible light and near infra-red (NIR) light, such as the light emitted by ICG.

[0020] Referring again to FIG. 1, exemplary enhanced image data 70 is displayed on the display device 24. In that example, the working or distal end of a shaver 14a is shown displaying a first fluorescent portion 22a including a directional orientation marker 72. A similar example of a surgical instrument 14 in the form of a shaver 14a is shown with improved detail in FIG. 4A. As shown, the orientation marker 72 can be overlaid on the visible image data to provide a clear indication of the relative orientation of the shaver 14a within the surgical site 26. While the orientation marker 72 may seem inconsequential when the surgical instrument 14 is clearly visible in the image data, the overlay 62 aligned with the fluorescent emission 32 displayed in the enhanced image data can provide a clear indication of the orientation and / or location of the surgical instrument 14 even when the cavity of the surgical site is obscured or obscured by particles, blood, tissue debris, etc.

[0021] 1 further illustrates an example of a surgical instrument 14 in the form of an anchor 14b. In various cases, the anchors or various surgical implants may become overgrown with tissue, calcium, or other substances that may block them from the visible light emission 38 and visibility from the corresponding second image sensor 42b. As shown, a colored overlay 62 is generated by the display controller 20 on a portion of the image data associated with the second fluorescent portion 22b. The overlay or superimposed color may highlight a portion of the anchor 14b such that the location of the hexalobes or drive head 74 is visible in the enhanced image data. If the drive head 74 is hidden behind biological tissue, the excitation emission 34 and its generated fluorescent emission 32 may penetrate the tissue such that the display controller 20 may detect the fluorescent portion 22 and illustrate the location of the head 74 in the enhanced image data.

[0022] 3 and 4, an application of the camera system 10 is described with reference to a typical shoulder repair operation. As depicted in the figures, the camera 60 is implemented as an endoscope incorporating a second light source 40 configured to output a visible light emission 38 within the field of view 44 of the image sensors 42a, 42b. As shown in FIG. 3, the first light source 36 associated with the excitation emission 34 may be incorporated into a dedicated illumination device 80. The illumination device 80 may include an elongated shaft 82 extending between a proximal end 82a and a distal end 82b. The excitation emission 34 may be output from the first light source 36 via the distal end 82b of the elongated shaft 82. The control circuitry and power supply may be enclosed within a housing 84 connected to the proximal end 82a. In this configuration, the excitation emission 34 may originate from a different origin than the field of view 44. The dedicated illumination device 80 may project the excitation emission 34 into various portions or regions of the surgical site 26 without the need to steer the camera 60. Thus, an embodiment of the camera system 10 incorporating a dedicated lighting device 80 that is separate from the camera 60 can provide independent illumination of various areas within the surgical site 26 without steering the camera 60 or regardless of the position of the camera 60.

[0023] Although described with reference to the excitation emission 34 being output from the dedicated illumination device 80, either or both of the light sources 36, 40 can be implemented within the dedicated illumination device 80 to output light in various ranges of wavelengths. In some embodiments, the illumination device 80 or the camera 60 can be configured to emit a light beam having a diameter small enough to target an item in the surgical field for further action by the surgeon. In one embodiment, the beam diameter can be less than about 5 mm. In some cases, the beam diameter can be less than about 2 mm or less than about 1 mm. In general, the illumination device 80 or the camera 60 can be configured to emit a light beam of sufficient brightness and density to be detected within the surgical field. For example, in some cases, the high sensitivity sensor 42 can be measured to measure a light beam of less than 10 nW / cm. 2The light sources 36, 40 may be positioned proximally relative to the distal end of the light emitting device 80 or camera 60. Additionally, the light sources 36, 40 may be positioned remotely from the distal end and the light emitting device 80 or camera 60, from a light source in communication with the distal end, such as by optical fiber. The light emitted by the light emitting device 80 and / or camera 60 may have a variable shape that may be adjusted, such as by using optical elements to allow a user to better illuminate a desired target.

[0024] In some embodiments, one or both of the light sources 36, 40 may be incorporated into a surgical instrument 14 other than the endoscopic camera system 10, such as a probe, shaver 14a, ablation device, or other instrument. In some examples, an LED may be located at the distal end of the device or instrument. In some examples, a probe or other device may be formed at least in part with a light pipe that can receive light from an LED, laser, or other light source external to the body and deliver radiation to the distal end of the instrument. The light-emitting device 80 may be powered by a separate power supply coupled to the light-emitting device. Additionally, the light-emitting device 80 may be battery powered. A battery-powered light-emitting device may be configured for single use or may be configured with a rechargeable battery for multiple uses. The light-emitting device 80 may be packaged in a sterile container for single use. Additionally, the light-emitting device 80 may be configured for sterilization and repeated use. The light-emitting device 80 may be a rigid device or a flexible device. The light-emitting device may be an articulable device.

[0025] Additionally, the lighting device 80 or light sources 36, 40 may be positioned outside the surgical field or site 26 and light may be directed through biological tissue for detection by a camera 60 positioned in the surgical field. Additionally, the lighting device may direct light from the surgical field through tissue for detection by a device positioned outside the surgical field. In some cases, the lighting device 80 may be positioned outside the body and direct light through tissue for detection by a camera 60 positioned inside the body. Additionally, the lighting device 80 may be positioned inside the body and direct light through tissue for detection by a camera (e.g., camera 60) positioned outside the body. Additionally, the lighting device 80 may be positioned at a first portion of the surgical site 26 and direct light through tissue for detection at a second portion of the surgical site 26.

[0026] As best shown in FIG. 3, shoulder cavity 86 is revealed through cutout 88. However, in a typical arthroscopic procedure, shoulder cavity 86 is sealed off, such that the internal anatomy of patient 28 is not visible, as depicted in FIG. 3. To accurately visualize the shoulder surgery corresponding to FIG. 3, the distal ends of camera 60 and dedicated light source 80 would protrude through external tissue into shoulder cavity 86, similar to the examples shown in FIGS. 7 and 10A, as will be described later. Thus, cutout 88 in FIG. 3 can provide a simplified representation of an arthroscopic procedure to reveal internal anatomy, as well as represent an open procedure in which camera 60 and dedicated lighting device 80 can be positioned externally to provide illumination to shoulder cavity 86.

[0027] 3 and 4, a plurality of sutures 92a, 92b and anchors 94a, 94b are shown implemented in association with a tendon 96 and a humerus 98 of the shoulder of the patient 28. As shown, the sutures 92 may include a first suture 92a and a second suture 92b. The first suture 92a is associated with a first anchor 94a that connects the first suture to the humerus 98. The second suture 92b is associated with the humerus 98 via a second anchor 94b. Although clearly depicted in FIG. 3, the view of the surgical site 26 may be obscured by blood and particulate matter within the shoulder cavity 86. Thus, the view and relative orientation of the camera 60 with respect to the surgical site 26 may not be readily apparent from the image data depicted on the display device 24.

[0028] In addition to obstructions to the field of view 44 within the shoulder cavity 44, in some cases, an anchor (represented in FIG. 4 as second anchor 94b) may be masked or hidden beneath tissue or abnormal growth. In such cases, the second anchor 94b may be almost completely hidden from view and may be difficult to detect within the image data captured by the camera 60. To improve the visibility of the second anchor 94b, a fluorescent agent may be incorporated into a portion of the second anchor 94b, designated as a first fluorescent portion 100a (see FIG. 4) incorporated into the drive head 74 or hexalobule. In addition to the first fluorescent portion incorporated into the second anchor 94b, each of the first suture 92a and second suture 92b may also include a corresponding second fluorescent portion 100b and third fluorescent portion 100c. Each of the fluorescent portions 100a, 100b, and 100c may, in this example, be illuminated by an excitation emission 34 output from a first light source 36 of the dedicated lighting device 80. In response to receiving the excitation emission 34, each of the fluorescent portions 100a, 100b, 100c may become excited to output a corresponding fluorescent emission 32.

[0029] In some cases, the fluorescent emissions 32 output from the fluorescent units 100a, 100b, 100c may vary in wavelength due to different compositions or combinations of fluorescent agents incorporated therein. In other cases, the concentration of a common fluorescent agent (e.g., ICG dye) may be incorporated at different levels in each of the fluorescent units 100a, 100b, 100c. Thus, in response to receiving the excitation emission 34, each of the fluorescent emissions 32 output from the fluorescent units 100a, 100b, 100c may vary in wavelength or intensity based on the composition of the fluorescent agent or the concentration of the fluorescent agent incorporated therein. Based on the variation in intensity or wavelength associated with the fluorescent emissions 32, the display controller 20 may be operable to distinguish between the different fluorescent units 100a, 100b, 100c and overlay each of the fluorescent units 100a, 100b, 100c with a different characteristic color 102. Thus, the camera system 10 can be configured to distinguish between the multiple fluorescent portions 100a, 100b, 100c and assign different respective characteristic colors 102 or patterns such that the enhanced image data displayed on the display device 24 clearly distinguishes the location of each of the surgical instruments 14 (e.g., 92a, 92b, and 94b).

[0030] For clarity, the sutures 92a, 92b, and second anchors 94b depicted in FIG. 4 may appear in the image data as dull and nearly indistinguishable from their surroundings when viewed alone via the visible light emission 38. However, based on the overlays 62 applied by the display controller 20 onto the corresponding fluorescent portions 100a, 100b, 100c, the enhanced image data may clearly distinguish each of the surgical instruments 14 based on the corresponding characteristic colors 102a, 102b, 102c, or pseudocolors, overlaid onto the image data associated with the visible light emission 38. As shown, the characteristic colors 102 may include a first color 102a, a second color 102b, and a third color 102c. The first color 102a may be incorporated onto the first fluorescent portion 100a coating the drive head 74, or the hexalobes of the second anchor 94b. The second color 102b and the third color 102c may be incorporated into the construction material forming the first suture 92a and the second suture 92b, respectively. Each of the characteristic colors 102 may be visually distinguishable based on a pre-defined display configuration stored within the display controller 20.

[0031] In some cases, the characteristic color 102 or pattern associated with the enhanced image data may be customized or modified to suit the preferences of a particular user. For example, some users may prefer a wide range of colors to help distinguish between various surgical instruments 14, while other users may prefer subtle color differences that do not distract the user's view from other aspects within the surgical site 26. In some cases, the display controller 20 may adjust the color template or color configuration of the characteristic color 102 or pattern based on the color of the local environment manifested in the image data captured by the second image sensor 42 associated with the visible light emission 38. For example, if the image data illuminated by the visible light emission 38 is displayed primarily in warm colors (e.g., red, yellow, orange), the display controller 20 may assign a cool color template (e.g., blue, purple, green) to distinguish the fluorescent portions 100a, 100b, 100c from the rest of the image data in the field of view 44. Similarly, if the image data is dark, a bright or contrasting color or pattern may be automatically applied to make the image data stand out. Thus, the camera system 10 can provide various formats and color templates associated with the enhanced image data to aid in visualization of the surgical site 26 .

[0032] 5A and 5B, a typical surgical instrument 14 is shown including a fluorescent portion 22 configured to assist a user in recognizing the orientation or position of the surgical instrument 14 when depicted in the enhanced image data generated by the camera system 10. As shown in FIG. 5A, a working end of a shaver 14a is shown demonstrating a plurality of longitudinal markings 110 formed by the fluorescent portion 22. The longitudinal markings may extend along a longitudinal axis 112 of the shaver 14a and may be evenly spaced radially around the elongated body 114. A shaver head 116 is shown in phantom on the opposite side of the plane depicted in FIG. 5A. In this configuration, the longitudinal markings 110 including the fluorescent portion 22 may be illuminated to output a fluorescent emission 32 in response to an excitation emission 34, such that the enhanced image data may reveal the orientation of the surgical instrument 14 or shaver 14a relative to the actuator direction (e.g., the direction of the shaver head 116).

[0033] 5B, the surgical instrument 14 is illustrated as an exemplary needle or probe 14c including a plurality of transverse markings 120 corresponding to the fluorescent portions 22. As shown, the transverse markings 120 are implemented as a plurality of graduated segments that indicate graduations associated with the position of the surgical instrument 14 or probe 14c. Similar to the longitudinal markings 110, the transverse markings 120 may incorporate a fluorescent agent within the fluorescent portion 22 and output a fluorescent emission 32 in response to receiving an excitation emission 34. In addition to the transverse markings 120, the probe 14c may include one or more letters 122 or symbols, which may also incorporate a fluorescent dye or fluorescent agent, such that the letters 122 may be overlaid in the image data to highlight the associated symbol within the image data. The longitudinal markings 110 and lateral markings 120 can be implemented in various combinations to assist an operator of an associated surgical instrument 14 in identifying the orientation, position, and / or relative measurements of the surgical instrument 14 when presented in enhanced image data on the display device 24.

[0034] In some cases, the longitudinal markings 110, the transverse markings 120, or various additional fluorescent portions 22 incorporated on the surgical instrument 14 may be disposed within grooves 124 or indentations formed on the outer surface of the surgical instrument 14. By including the fluorescent portions 22 in the grooves or indentations associated with the orientation or position markings 110, 120, the generated fluorescent emissions 32 output from the grooves 124 or indentations may be captured within the field of view 44 of the camera system 10 through an orientation opening associated with an inner surface of each of the grooves 124 that faces or faces the corresponding image sensor 42a, 42b of the camera 60. In this configuration, the dimensional or orientation markings 110, 120 incorporated on the surgical instrument 14 may be hidden from the field of view 44 of the camera 60 until a portion of the fluorescent emissions 32 are output from the corresponding fluorescent portions 22 disposed within the grooves 124. The result of the fluorescent portions 22 disposed within the grooves 124 can be improved precision achieved when the inner surface of each of the grooves 124 is visible through the corresponding orientation opening, as well as a view exposing only the fluorescent emission 32. In this manner, the dimensional and orientation features (e.g., 110, 120) of the surgical instrument 14 can provide improved precision in determining the relative position or orientation of the surgical instrument 14.

[0035] 6, an exemplary shaver 14a is shown within the field of view 44 of the camera 60 revealing enhanced image data including an overlay 62 of a distinctive color 102 on the longitudinal markings 110 formed by the grooves 124 and the fluorescent portion 22. As shown, the longitudinal markings 110 can assist an operator in identifying the orientation of the shaver head 116 revealed by the arrow 126. For example, as a result of viewing two of the three longitudinal markings 110 on the display device 24, a user of the shaver 14a can visually identify from the longitudinal markings 110 revealed by the overlay 62 that the shaver head 116 is oriented toward the opposite side of the longitudinal markings 110. As shown, the longitudinal markings 110 are positioned on the left-facing side of the shaver 14a, such that the operator can recognize that the shaver head 116 is oriented toward the right side depicted on the display device 24. Such an indication of the orientation of the surgical instrument 14 may be particularly beneficial if the shaver head 116 is hidden behind tissue 128 or debris in the field of view 44. Thus, the longitudinal markings 110 may assist the user in determining the relative orientation of the surgical instrument 14.

[0036] Referring now to FIG. 7, an additional exemplary illustration of an arthroscopic procedure on a shoulder 130 of a patient 28 is shown. FIG. 8 illustrates enhanced image data associated with a field of view 44 captured by a camera 60 positioned as depicted in FIG. 7. As illustrated in FIGS. 7 and 8, a probe 14c is illustrated penetrating biological tissue 132 within a shoulder cavity 134. As previously described, an excitation emission 34 may be output from a first light source 36 incorporated into a dedicated illumination device 80. The excitation emission 34 may be delivered into the cavity 134, penetrate the biological tissue 132 (e.g., cartilage, muscle, tendon, bone, etc.) and impinge on the fluorescent portion 22 formed by the lateral marking 120. In response to receiving the excitation emission 34, a fluorescent agent incorporated into the fluorescent portion 22 of the lateral marking 120 may output a fluorescent emission 32. Light energy emitted from the fluorescent portion 22 may also be transmitted through the biological tissue 132 into the cavity 134 such that the near-infrared image sensor 42 a can capture the fluorescent emission 32 within the field of view 44 .

[0037] In response to detecting the fluorescent emission 32 in the image data captured by the first image sensor 42a, the display controller 20 of the camera system 10 can overlay pixels in the image data associated with the fluorescent emission 32 with an overlay 62 (e.g., a characteristic color 102 or pattern) to generate enhanced image data. Thus, by detecting the fluorescent emission 32, the camera system 10 can provide detection and tracking of the position of one or more surgical instruments 14 through the biological tissue 132. Once detected, the display controller 20 can further overlay, mark, or enhance a corresponding portion of the image data to reveal the surgical instruments 14 that would otherwise be completely hidden from a conventional camera system.

[0038] 9, a typical surgical cavity 140 is shown demonstrating a distal tip of a probe or needle 142 beginning to penetrate and protrude through biological tissue 144. As depicted in the enhanced image data displayed on the display device 24 of the camera system 10, the distal tip 146 of the needle 142 is overlaid with a distinctive pattern or color 102. As with other examples, the distinctive pattern or color 102 overlaid on the distal tip 146 of the needle 142 can be detected by the display controller 24 in response to a corresponding presence of fluorescent emission 32 in the image data captured by the combined image sensors 42a, 42b. In the example provided, the distal tip 146 of the needle 142 can be blindly introduced into the surgical cavity 140. It can therefore be very difficult for a surgeon or physician to accurately determine the location of the obscure instruments 148 and graspers 150 to effectively guide and interact with the distal tip 146. However, due to the incorporation of the fluorescent portion 22 on the distal tip 146, the fluorescent emission 32 can penetrate the biological tissue 144 and be detected by the display controller 20 before the distal tip 146 begins to protrude through the biological tissue 144. For example, in response to identifying the fluorescent emission 32 within the field of view 44, the display controller 20 can enhance the corresponding portion of the image data associated with the fluorescent emission 32 with the overlay 62. In this manner, the surgeon can identify the location of the biological tissue 144 through which the distal tip 146 of the needle 142 will protrude before the distal tip 146 breaks the surface of the biological tissue 144. In this manner, the enhanced image data provided by the camera system 10 can improve the accuracy associated with the operation by indicating the location of the surgical instrument that is otherwise invisible in the visible light range captured by the second imager or visible light image sensor 42b.

[0039] In some examples, the excitation light source or first light source 36 can output excitation emission 34 at an intensity sufficient to penetrate biological tissue, as described herein. For example, the first light source 36 can output approximately 1 mW / cm 2 ~1W / cm2 The excitation emission 34 may be output at an intensity ranging from 100 nm to 900 nm. In some cases, the light intensity may be higher or lower depending on the particular light emitter technology implemented and its application. Depending on the application and the duration for which the excitation emission 34 may be activated, the intensity of the excitation emission 34 may be limited or pulsed to control the generation of excess heat and limit damage to the biological tissue. As previously mentioned, the excitation emission 34 may include wavelengths of radiation ranging from approximately 650 nm to 900 nm in the near infrared range. For reference, the visible light emission 38 associated with the second light source 40 may be output at wavelengths corresponding to visible colors of light associated with the acuity of the human eye, ranging from 400 nm to approximately 650 nm. The penetration of the excitation emission 34 and / or the fluorescent emission 32 through the biological tissue may extend from a depth of approximately 1 mm to a biological tissue depth or thickness of more than 10 mm. Experimental results have demonstrated that the intensity of the excitation emission 34 and the fluorescent emission 32 as well as the emission in the near infrared range is lost at a rate of approximately 3% to 10% / mm of the penetrated biological tissue. Thus, the first image sensor 42a can detect the fluorescence emission 32 or excitation emission 34 after the corresponding light energy has penetrated several millimeters of biological tissue. Thus, the camera system 10 can identify the relative locations or orientations of the various surgical instruments 14 to reveal locations within the enhanced image data in various cases where the surgical instruments 14 may be hidden behind layers of biological tissue having various thicknesses.

[0040] 10A and 10B, yet another exemplary application of the surgical camera system 10 is shown illustrating an arthroscopic shoulder repair of a patient 28. As illustrated in FIG. 9, a front cannula 152 provides access to a surgical cavity 154 to manipulate a plurality of sutures 156a, 156b. In operation, a surgeon may access the surgical cavity 154 via a skid 158. To reach the sutures 156 within the surgical cavity 154, a grasper 160 may be implemented to selectively engage one of the sutures 156. As illustrated in FIG. 10B, the field of view 44 of the camera 60 illustrates an arthroscopic view of a first suture 156a, a second suture 156b, and a lasso 162 that may be further implemented to manipulate and loop the sutures 156. Surgeons and physicians, even with extensive procedural knowledge and associated visible colors incorporated into the sutures 156, may still have difficulty distinguishing a first suture 156a from a second suture 156b. Differentiating the sutures 156 may be especially difficult when the fluid within the surgical cavity 154 is obstructed by debris or blood that may further obscure any defining features of the sutures 156.

[0041] As previously described with reference to FIGS. 3 and 4, the first suture 156a can include a first concentration of a fluorescent agent, and the second suture 156b can include a second concentration of a fluorescent agent. Thus, in response to receiving the excitation emission 34, each of the sutures 156a, 156b can output different intensities of fluorescent emission 32. These intensities of fluorescent emission 32 can be identified and differentiated by the display controller 20 based on the image data in the near infrared range captured by the first image sensor 42a. In response to the different intensities of the fluorescent emission 32, the display controller 20 can overlay each of the sutures 156a, 156b with different characteristic patterns 164a, 164b, as evidenced by FIG. 10B. In this manner, the display controller 20 can identify the various intensities of fluorescent emission 32 to differentiate between multiple surgical instruments 14 identified within the field of view 44 of the camera system 10. The overlay 62 of sutures 156a, 156b, shown as a distinctive pattern, can likewise be implemented as a distinctive color or marker (e.g., a notification window, overlaid graphics, etc.) to help identify and distinguish between the surgical instruments 14 depicted within the image data of the camera system 10.

[0042] 11, an exemplary flow chart illustrating a method for detecting an object using the camera system 10 as described herein is shown. The method 170 may begin in response to the operation of the camera system 10 or the initiation of an object detection routine 172. As described in various examples, the camera 60 may be controlled by the camera controller 46 to capture image or sensor data via one or more of the image sensors 42a, 42b (174). Once the image data is captured by the image sensors 42a, 42b, the display controller 20 may detect (176) one or more portions of the image data or pixels within the field of view 44 that include wavelengths of light corresponding to the fluorescent emission 32 from the fluorescent portion 22. Based on the image data processed by the display controller 20, the method 170 may continue at step 178 by determining whether one or more surgical instruments 14 have been detected in response to the presence of the fluorescent emission 32. If the instrument 14 is not detected in step 178, the method 170 may return to step 174 and continue to capture image or sensor data and process the image data to identify the fluorescent emission 32 in steps 174 and 176.

[0043] If an object associated with the fluorescent emission 32 is detected in the image data at step 178, the method 170 may continue by marking, overlaying, or annotating the image data to highlight the area in the field of view 44 where the fluorescent emission 32 was detected (180). The marked or annotated image data generated at step 180 may correspond to enhanced image data including one or more overlays 62 in the form of distinctive colors, patterns, or other indicative features that may assist a viewer in recognizing the location, orientation, dimensions, proportions, or other information associated with the surgical instrument 14 where the fluorescent emission 32 was emitted and detected by the camera system 10. Examples of surgical instruments may include biters, graspers, retrievers, picks, punches, hooks, probes, dissectors, retractors, or scissors. In some cases, the surgical instruments 14 may respond to items configured to trigger an alert or notification of the camera system 10 to indicate detection of their presence. For example, tools, implants, sponges, or various other partial components of surgical instruments within the surgical site 26 may be detected by the camera system 10 in response to the presence of the fluorescent emission 32. In response to such detection, the method 170 may output (182) an instruction (e.g., an alert, command, notification, etc.) indicating the presence of the fluorescent portion 22 and alerting the surgeon or medical professional to the presence of the corresponding surgical instrument 14. In some cases, the programming of the camera system 10 may define a particular surgical instrument 14 that may be associated with the fluorescent emission 32. In such a case, the notification output at step 182 may indicate a particular type or category of surgical instrument 14 identified in the image data by the camera system 10. Following step 182, the detection routine may continue until the detection routine is deactivated by the operator, as indicated at step 184.

[0044] 12, a flow chart is shown illustrating a method 190 for displaying enhanced image data according to the present disclosure. The method 190 may be initiated in response to the initiation of an enhanced image data display routine by the camera system 10 (192). Once initiated, the method 190 may continue to step 194 to capture image or sensor data using the image sensors 42a and 42b. Once captured, the display controller 20 may scan the image data to detect a portion of the image data using a wavelength corresponding to the fluorescent emission 32 as detected by the first image sensor 42a (196). In some cases, the method 190 may identify a plurality of fluorescent emissions 32 depicted within the image data at a plurality of intensity levels corresponding to a plurality of fluorescent portions 22 that may include various concentrations of fluorescent agents (198). As previously discussed, each of the fluorescent portions 22 of the surgical instrument 14 detected within the field of view 44 may include a unique concentration of fluorescent agent, such that the generated fluorescent emissions 32 may be output and detected by the first image sensor 42a at different intensity levels. Based on the different intensity levels, the display controller 20 can assign the overlay 62 as different characteristic colors within the image data and generate enhanced image data for display on the display device 24 (200).

[0045] In some cases, the display controller 20 may distinguish different intensities of the fluorescent emissions 32 over time, such that a distinctive color or pattern associated with the enhanced image data overlay 62 may be maintained even if the corresponding surgical instrument 14 is not simultaneously presented in the image data. For example, the display controller 20 may be pre-configured to associate lower intensity fluorescent emissions 32 with a first color, medium intensity fluorescent emissions 32 with a second color, and third intensity fluorescent emissions 32 with a third color. The relative intensities may correspond to a percentage or relative level of brightness associated with each of the fluorescent emissions 32. For example, if three levels of brightness are detected, the maximum intensity may be associated with the third color. The intermediate intensity may be associated with the second color, and the minimum or lowest intensity may be associated with the first color. Once the enhanced image data is generated, it may be further selectively displayed on the display device 24 by controlling an interface of the display controller (202). Following step 202, the display routine may continue until deactivated (204).

[0046] 13, a block diagram of the camera system 10 is shown. As described throughout this disclosure, the system 10 can include a camera 60 in communication with the display controller 20. The camera 60 can include a plurality of light sources 36, 40, at least one image sensor 42 (e.g., 42a, 42b), a camera controller 46, and a user interface 210. In various embodiments, the camera 60 can correspond to an endoscope having an elongated scope including a narrow distal end suitable for various non-invasive surgical techniques. For example, the distal end can include a diameter of less than 2 mm. As indicated, the camera 60 can communicate with the display controller 20 via a communication interface. Although shown as being connected via a conductive connection, the communication interface can correspond to a wireless communication interface operating via one or more wireless communication protocols (e.g., WiFi, 802.11b / g / n, etc.).

[0047] The light sources 36, 40 may correspond to various light emitters configured to generate light in the visible and / or near infrared range. In various implementations, the light sources 36, 40 may include light emitting diodes (LEDs), laser diodes, or other illumination technologies. As previously mentioned, the first light source 36 may generally correspond to a laser emitter configured to output emission in the near infrared range including wavelengths of approximately 650 nm to 900 nm. In some examples, the first light source 36 may output excitation emission 34 ranging from 650 nm to 680 nm with a center frequency of approximately 670 nm. In some cases, the first light source 36 may output excitation emission 34 in a wavelength range of approximately 740 nm to 780 nm. More generally, the wavelengths associated with the first light source 36 and the excitation emission 34 may be selected to effectively activate the fluorescent agent of the fluorescent portion 22. The second light source 40 may correspond to a white light source within the visible spectrum, including wavelengths ranging from approximately 380 nm to 700 nm, or approximately 400 nm to 650 nm.

[0048] The image sensors 42a, 42b can accommodate a variety of sensors and configurations, including, for example, charge-coupled device (CCD) sensors, complementary metal oxide semiconductor (CMOS) sensors, or similar sensor technologies. As previously described, the system 10, and in particular the display controller 20, can process or compare image data captured by each of the image sensors 42 to identify the fluorescent emissions 32 and apply an overlay 62 in the form of one or more colors (e.g., characteristic colors 102), patterns, markers, graphics, messages, and / or annotations indicative of the presence and / or location of the fluorescent emissions 32 within the image data. In operation, the optical filters 52a, 52b (e.g., bandpass filters) can filter and effectively separate the combined wavelengths of the fluorescent emissions 32 and the visible light emissions 38 in the field of view 44. The filtered light received by the first image sensor 42a can thus provide a map identifying the location of the fluorescent emissions 32 within the image data and the corresponding location of the fluorescent portions 22 of the surgical instrument 14.

[0049] The camera controller 46 can correspond to control circuitry configured to control the operation of the image sensors 42a, 42b and the light sources 36, 40 to provide for the simultaneous generation or capture of image data in the visible light spectrum associated with the fluorescent emission 32, as well as the near infrared spectrum or wavelengths. Additionally, the camera controller 46 can be in communication with a user interface 210, which can include one or more input devices, indicators, displays, etc. The user interface can provide control of the camera 60, including the operation of one or more routines, as described herein. The camera controller 46 can be implemented by various forms of controllers, microcontrollers, application-specific integrated controllers (ASICs), and / or various control circuits, or combinations thereof.

[0050] The display controller 20 may include a processor 212 and a memory 214. The processor 212 may include one or more digital processing devices, including, for example, a central processing unit (CPU) having one or more processing cores, a graphics processing unit (GPU), a digital signal processor (DSP), a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), etc. In some configurations, multiple processing devices are combined in a System on a Chip (SoC) configuration, while in other configurations, the processing devices may correspond to separate components. In operation, the processor 212 executes program instructions stored in the memory 214 to perform the operations described herein.

[0051] The memory 214 may include one or more data storage devices, including, for example, magnetic or solid state drives that store digital data, and random access memory (RAM) devices. The memory 214 may include one or more stored program instructions, object detection templates, image processing algorithms, and the like. As shown, the memory 214 may include a detection module 216 and an annotation module 218. The detection module 216 includes instructions for processing the image data to identify the fluorescent emission 32 from the first image sensor 42a and to detect a location within the field of view 44 from which the fluorescent portion 22 of the surgical instrument 14 emitted the fluorescent emission 32. In some cases, the detection module 216 may include instructions for detecting or identifying a type or classification associated with the surgical instrument 14 in the image data captured by the camera 60. For example, the processor 212 may access instructions in the detection module 216 to perform various processing tasks on the image data, including pre-processing, filtering, masking, cropping, and various enhancement techniques to improve detection capabilities and efficiency. Additionally, the detection module 216 may provide instructions for handling various feature detection tasks including template matching, character recognition, feature identification or matching, etc. In some examples, the detection module 216 may also include various training models for object detection and / or labeling of the surgical instrument 14 or associated objects. In some implementations, detection of a surgical instrument, either by identity, presence, or classification, may initiate instructions to output an alert or notification on the display device 24, the control console 16, an external device or server 220, or various connected devices associated with the surgical camera system 10.

[0052] The annotation module 218 may include instructions indicating various marking or overlay options for generating the enhanced image data, as well as corresponding display filters for superimposing or applying the overlay 62 to the image data. As previously discussed, the enhanced image data may also include one or more graphics, annotations, labels, markers, and / or identifiers indicating the location, presence, identity, or other information related to the classification or identification of the surgical instrument 14. The annotation module 218 may further provide instructions for generating graphics, labels, overlays, or other related graphical information that may be applied to the image data captured by the second image sensor 42b (e.g., a visible light sensor) to generate enhanced image data for display on the display device 24.

[0053] The display controller 20 may further include one of more formatting circuits 222, which may process image data received from the camera 60, communicate with the processor 212, and output enhanced image data to the display device 24. The formatting circuit 222 may include one or more signal processing circuits, an analog-to-digital converter, a digital-to-analog converter, and the like. The display controller may include a user interface 224, which may be in the form of an integrated interface (e.g., touch screen, input buttons, electronic display, and the like) or may be implemented by one or more connected input devices (e.g., tablet) or peripheral devices (e.g., keyboard, mouse, and the like). As shown in the figure, the controller 20 also communicates with an external device or server 220, which may correspond to a network, a local or cloud-based server, a device hub, a central controller, or a variety of devices that may communicate with the display controller 20, and more generally, the camera system 10, via one or more wired (e.g., Ethernet) or wireless communication (e.g., WiFi, 802.11b / g / n, and the like) protocols. For example, the display controller 20 can receive updates to the various modules and routines, as well as communicate sample image data from the camera 60 to a remote server to improve operation, diagnostics, and updates to the system 10. The user interface 224, the external server 220, and / or the surgical control console 16 can communicate with the controller 20 through one or more I / O circuits 226. This I / O circuitry can support a variety of communication protocols, including, but not limited to, Ethernet / IP, TCP / IP, Universal Serial Bus, Profibus, Profinet, Modbus, serial communications, and the like.

[0054] In various embodiments, the present disclosure provides a surgical camera system configured to capture image data indicative of a surgical instrument including a fluorescent agent. The surgical camera system includes a camera including at least one sensor configured to capture image data including a first wavelength range and a second wavelength range. An excitation light source emits excitation emissions at an excitation wavelength. A controller is in communication with the at least one sensor of the camera. The controller is configured to process the image data from the at least one sensor and detect at least one fluorescent portion of the image data in response to fluorescent emissions generated by the fluorescent agent in the second wavelength range. The controller is further configured to generate enhanced image data revealing the at least one fluorescent portion of the surgical instrument in the image data.

[0055] In various embodiments, the systems and methods described herein may include only one or more of the following features or steps, or a combination thereof: - the first wavelength range includes wavelengths in the visible light range from 400 nm to 650 nm, the second wavelength range includes wavelengths ranging from 650 nm to 900 nm in the near infrared range; - the fluorescent emission is emitted from the fluorescent agent at an output wavelength different from the excitation wavelength; the visible light source emits light within a first wavelength range; The excitation light source, the visible light source, and the camera are incorporated into an endoscope; - the endoscope has a diameter of less than about 2 mm; at least one sensor of the camera includes a first sensor configured to capture first data within a first wavelength range and a second sensor configured to capture second data within a second wavelength range; - generating enhanced image data by selectively applying an overlay defined by second data from a second sensor onto the first data from the first sensor; the controller is further configured to determine a plurality of intensity levels of fluorescent emissions output from the at least one fluorescent moiety generated by the fluorescent agent within a second wavelength range; the controller is further configured to assign a distinctive color or pattern to each of the plurality of intensity levels; -Enhancing the image data includes overlaying a distinctive color or pattern on the fluorescent portion that manifests each of a plurality of intensity levels as a distinctive color or pattern in the enhanced image data.

[0056] In various embodiments, the present disclosure provides a method for displaying a surgical instrument, the method can include illuminating a fluorescent portion of the surgical instrument with light including a first wavelength range corresponding to visible light and a second wavelength range including an excitation emission. The method can further include capturing first image data including the first wavelength range and capturing second image data including the second wavelength range that manifests a fluorescent emission output from the fluorescent portion in response to the excitation emission. The method can further include generating enhanced image data manifesting the first image data with at least one overlay or graphic manifesting the fluorescent portion defined by the second image data overlaid on the first image data and communicating the enhanced image data for display on a display device.

[0057] In various embodiments, the systems and methods described herein may include only one or more of the following features or steps, or a combination thereof: - positioning a surgical instrument within a surgical field; - targeting a surgical instrument with excitation emission; - detecting fluorescence emission in the image data; and - outputting an indication of the surgical instrument detected within the image data in response to detecting the fluorescent emission; - displaying the detected fluorescence emission in a predefined false color as an overlay on a display; The fluorescent light emitted from the fluorescent portion is output at a wavelength different from the excitation wavelength; - discriminating the intensity of the fluorescent emission output from the fluorescent portion generated by the fluorescent agent at a plurality of intensity levels; - assigning a distinctive color or pattern to each of a plurality of intensity levels; - enhancing the image data includes overlaying a distinctive color or pattern on the fluorescent portions that represent each of a plurality of intensity levels within the enhanced image data; - detecting fluorescent emission emitted from the fluorescent agent through the biological tissue; and / or The excitation emission is transmitted through the biological tissue.

[0058] In some embodiments, the present disclosure provides a surgical camera system configured to capture image data indicative of a surgical instrument including a fluorescent agent. The surgical camera system comprises a camera including at least one sensor configured to capture image data including a first wavelength range and a second wavelength range. An excitation light source emits an excitation emission at an excitation wavelength. A controller is in communication with the camera sensor. The controller is configured to process image data from the at least one image sensor including the first wavelength range and the second wavelength range, and identify a plurality of intensity levels of at least one fluorescent emission output from at least one fluorescent portion produced by the fluorescent agent within the second wavelength range. The controller is further configured to assign a distinctive color or pattern to each of the plurality of intensity levels, and generate enhanced image data manifesting the plurality of intensity levels of the fluorescent emission having the distinctive color or pattern. In some embodiments, enhancing the image data includes overlaying a distinctive color or pattern on the fluorescent portion manifesting each of the plurality of intensity levels in the enhanced image data.

[0059] In some embodiments, the surgical instrument can include a body forming an exterior surface including a proximal end and a distal end. The fluorescent portion can include a fluorescent agent disposed on the exterior surface. The fluorescent portion can include at least one marking extending on the exterior surface, the fluorescent portion configured to emit fluorescent emissions in the near infrared range in response to an excitation emission.

[0060] In various embodiments, the systems and methods described herein may include only one or more of the following features or steps, or a combination thereof: at least one marking of the fluorescent portion indicates at least one of the group consisting of an identity of the surgical instrument, an orientation of the surgical instrument, and a dimension of the surgical instrument; - at least one marking includes a plurality of graduated segments defining a scale associated with a position or orientation of the surgical instrument; - the at least one marking includes a plurality of transverse graduated markings extending between the proximal end and the distal end; - the at least one marking comprises at least one longitudinal marking along the longitudinal axis between the proximal end and the distal end; the at least one marking comprises one or more indicator symbols formed on the outer surface by a fluorescent portion, the indicator symbols comprising at least one of a pattern, a shape, and an alphanumeric character; - the indicator symbol identifies a unit of measure or scale of the at least one marking; - the at least one marking is disposed within a groove or indentation formed in the exterior surface; an orientation opening of the fluorescent portion is exposed within the groove or indentation in response to an orientation of the surgical instrument; the orientation opening is illuminated by the excitation emission based on an orientation of the surgical instrument relative to the light source from which the excitation emission is output; the orientation is distinguishable based on the spread of fluorescent emission projected through the aperture; The light source is integrated into the endoscope; the fluorescent agent is an indocyanine green dye having an excitation wavelength of about 600 nm to about 900 nm and an emission wavelength of about 830 nm; the surgical instrument is selected from the group consisting of sutures, pins, screws, plates, surgical tools, and implants; and / or The surgical instrument is selected from the group consisting of a biter, grasper, retriever, pick, punch, hook, probe, dissector, retractor, or scissors.

[0061] In some embodiments, the surgical detection system may be configured to identify at least one surgical instrument within the field of operation. The system may include a camera including at least one sensor configured to capture image data including a first wavelength range and a second wavelength range. The excitation light source emits an excitation emission at an excitation wavelength. A controller is in communication with the at least one sensor of the camera, the controller configured to process image data from the at least one sensor and to identify fluorescent emissions in the image data output from at least one fluorescent portion of the surgical instrument. The controller is further configured to detect the presence of the surgical instrument in response to the presence of the fluorescent emissions.

[0062] In various embodiments, the systems and methods described herein may include only one or more of the following features or steps, or a combination thereof: - the fluorescent emission includes wavelengths of light in the near infrared range, approximately 650 nm to 900 nm; the controller is further configured to detect a number of pixels in the image data within the near infrared range corresponding to a location of the surgical instrument; the controller is further configured to identify the surgical instrument in response to at least one of a pattern, a shape, and an alphanumeric character of the plurality of pixels; the controller is further configured to output an indication identifying the presence of a surgical instrument; the instructions are output on a display device as notifications indicating the location of the surgical instrument within the image data; the controller is further configured to: access a database including at least one computer vision template characterizing an appearance of a potential surgical instrument associated with the surgical procedure; and, in response to a plurality of pixels within the near infrared range corresponding to the computer vision template, identify the potential surgical instrument as the at least one surgical instrument; the controller is further configured to output a notification on a display device identifying a type or category of the at least one surgical instrument in response to an identification associated with the computer vision template; the surgical instrument is selected from the group consisting of a sponge, a suture, a pin, a screw, a plate, a surgical tool, and an implant; the surgical instrument is selected from the group consisting of a biter, grasper, retriever, pick, punch, hook, probe, dissector, retractor, needle, or scissors; the at least one surgical instrument comprises a plurality of surgical instruments, and the at least one fluorescent emission comprises a plurality of fluorescent emissions output from the plurality of surgical instruments, and the controller is further configured to distinguish between the plurality of surgical instruments in response to at least one of the intensities or patterns of the fluorescent emissions output from the plurality of surgical instruments; and / or The plurality of surgical instruments includes a plurality of sutures, and the controller is configured to distinguish between the plurality of sutures in response to a characteristic pattern of fluorescence of the surgical instruments.

[0063] In some embodiments, the surgical camera system may be configured to capture image data indicative of a surgical instrument including a fluorescent agent. The surgical camera system may include an endoscopic camera including at least one sensor configured to capture image data within a field of view including a first wavelength range and a second wavelength range. An excitation light source emits excitation emission at an excitation wavelength. A controller is in communication with the camera sensor. The controller is configured to process image data from the at least one sensor within the field of view depicting the cavity, and detect in the image data fluorescent emission output from at least one fluorescent portion of the surgical instrument. The fluorescent emission is transmitted through biological tissue forming at least a portion of the cavity. In response to the fluorescent emission, the controller generates enhanced image data revealing at least one fluorescent portion of the surgical instrument overlaid on the biological tissue depicted in the image data.

[0064] In various embodiments, the systems and methods described herein may include only one or more of the following features or steps, or a combination thereof: the excitation light source includes an elongated shaft forming a needle-like projection configured to output an excitation emission into the cavity; - an excitation light source configured to output excitation emission from a distal penetrating end of a needle forming an elongate shaft; the excitation light source originates from a first origin that is distinct from a second origin of view; The excitation light source is separate from the endoscopic camera, and each of the excitation light source and the endoscopic camera independently accesses the cavity. the controller is further configured to detect, in the image data, fluorescent emission transmitted through the biological tissue into the cavity; the controller is further configured to output instructions identifying, in the image data, a presence of fluorescent emission emitted from at least one fluorescent portion of the surgical instrument; - the instructions are output as enhanced image data including an overlay on the image data that indicates a location in the image data of the surgical instrument implanted within the biological tissue; the controller is further configured to output the enhanced image data to a display screen that indicates the location of the surgical instrument superimposed on the biological tissue as an overlay depicted in the image data; -The excitation light source is approximately 0.1 mW / cm 2 ~1W / cm 2 , 0.5mW / cm 2 ~500mW / cm 2 , 0.01mW / cm 2 ~200mW / cm 2 and the like, which can vary significantly depending on the application and emitter technology implemented; The excitation light is emitted at an excitation wavelength of about 600 nm to about 900 nm, and the fluorescence light is output at an emission wavelength of about 830 nm.

[0065] A surgical camera system and method that completely and effectively overcomes the shortcomings associated with the prior art has been disclosed in the above description and drawings. However, it will be apparent that variations and modifications of the disclosed embodiments may be made without departing from the principles described herein. The representations of the embodiments herein are provided merely as examples and are not limiting, with the true scope and spirit being set forth in the following claims.

[0066] As used herein, approximation words such as, but not limited to, "approximately," "substantially," or "about" refer to the following conditions, which, when so modified, are not necessarily absolute or complete, but are understood to be deemed approximately sufficient by one of ordinary skill in the art to warrant specifying the condition as being present. The extent to which the description may vary depends on how large the change may be set and still allow one of ordinary skill in the art to recognize the modified feature as having the required properties or potential of the unmodified feature. In general, but subject to the foregoing description, numerical values ​​herein modified by approximation words such as "approximately" may vary from the stated value by up to ±0.5%, ±1%, ±2%, ±3%, ±4%, ±5%, ±10%, ±12%, or ±15%.

[0067] It will be understood that any described process, or steps within a described process, can be combined with other disclosed processes or steps to form structures within the scope of the present devices. The exemplary structures and processes disclosed herein are for illustrative purposes and should not be construed as limiting.

[0068] It should also be understood that variations and modifications can be made in the structures and methods described above without departing from the concepts of the present device, and further that such concepts are intended to be covered by the following claims unless those claims are specifically and explicitly recited in those language.

[0069] The above description is to be considered as a description of the illustrated embodiments only. Modifications of the device will occur to those skilled in the art and those who make or use the device. Therefore, it should be understood that the embodiments shown in the drawings and described above are for illustrative purposes only and are not intended to limit the scope of the device, which is defined by the following claims, as interpreted under the principles of patent law.

Claims

1. 1. A surgical camera system configured to capture image data indicative of a surgical instrument including a fluorescent agent, the surgical camera system comprising: a camera including at least one sensor configured to capture image data including a first wavelength range and a second wavelength range; an excitation light source that emits excitation emission at an excitation wavelength; a controller in communication with the at least one sensor of the camera, the controller comprising: processing the image data from the at least one sensor; and detecting at least one fluorescent portion of the image data in response to fluorescent emissions produced by the fluorescent agent within the second wavelength range; and generating enhanced image data that reveals the at least one fluorescent portion of the surgical instrument in the image data.

2. The surgical camera system of claim 1 , wherein the first wavelength range includes wavelengths between 400 nm and 650 nm in the visible light range.

3. The surgical camera system of claim 2 , wherein the second wavelength range includes wavelengths ranging from 650 nm to 900 nm in the near infrared range.

4. The surgical camera system of claim 1 , wherein the fluorescent emission is emitted from the fluorescent agent at an output wavelength different from the excitation wavelength.

5. The surgical camera system of claim 1 , further comprising a visible light source that emits light within the first wavelength range.

6. The surgical camera system of claim 5 , wherein the excitation light source, the visible light source, and the camera are incorporated into an endoscope.

7. The surgical camera system of claim 6 , wherein the endoscope has a diameter of less than about 2 mm.

8. 2. The surgical camera system of claim 1, wherein the at least one sensor of the camera includes a plurality of sensors including a first sensor configured to capture first data within the first wavelength range and a second sensor configured to capture second data within the second wavelength range.

9. The controller:

9. The surgical camera system of claim 8, further configured to generate the enhanced image data by selectively applying an overlay defined by the second data from the second sensor onto the first data from the first sensor.

10. The controller:

2. The surgical camera system of claim 1, further configured to determine a plurality of intensity levels of the fluorescent emission output from the at least one fluorescent portion produced by the fluorescent agent within the second wavelength range.

11. The controller: The surgical camera system of claim 10 , further configured to assign a unique color or pattern to each of the plurality of intensity levels.

12. 12. The surgical camera system of claim 11, wherein the enhancing of the image data includes overlaying a distinctive color or pattern on the fluorescent portion that manifests each of the plurality of intensity levels as the distinctive color or pattern within the enhanced image data.

13. A surgical instrument comprising: a body defining an exterior surface including a proximal end and a distal end; a fluorescent portion including a fluorescent agent disposed on the exterior surface, the fluorescent portion including at least one marking extending over the exterior surface, the fluorescent portion configured to emit fluorescent emissions in a near infrared range in response to an excitation emission; A surgical instrument, wherein the at least one marking of the fluorescent portion indicates at least one of the group consisting of an identity of the surgical instrument, an orientation of the surgical instrument, and a size of the surgical instrument.

14. The surgical instrument of claim 13 , wherein the at least one marking comprises a plurality of graduated segments designating a scale associated with a position or orientation of the surgical instrument.

15. The surgical instrument of claim 13, wherein the at least one marking includes a plurality of transverse graduated markings extending between the proximal end and the distal end.

16. The surgical instrument of claim 13, wherein the at least one marking includes at least one longitudinal marking along a longitudinal axis between the proximal end and the distal end.

17. 14. The surgical instrument of claim 13, wherein the at least one marking comprises one or more indicator symbols formed on the exterior surface by the fluorescent portion, the indicator symbols comprising at least one of a pattern, a shape, and an alphanumeric character.

18. The surgical instrument of claim 17 , wherein the indicator symbol identifies a unit of measure or scale of the at least one marking.

19. The surgical instrument of claim 13 , wherein the at least one marking is disposed within a groove or indentation formed in the exterior surface.

20. 20. The surgical instrument of claim 19, wherein the orientation opening of the fluorescent portion is exposed within the groove or indentation in response to an orientation of the surgical instrument.

21. 21. The surgical instrument of claim 20, wherein the orientation opening is illuminated by the excitation emission based on the orientation of the surgical instrument relative to a light source from which the excitation emission is output.

22. The surgical instrument of claim 21 , wherein the orientation is distinguishable based on a spread of the fluorescent emission projected through the opening.

23. The surgical instrument of claim 22 , wherein the light source is integrated within an endoscope.

24. 14. The surgical instrument of claim 13, wherein the fluorescent agent is an indocyanine green dye comprising an excitation wavelength of about 600 nm to about 900 nm and an emission wavelength of about 830 nm.

25. 14. The surgical instrument of claim 13, wherein the surgical instrument is selected from the group consisting of a suture, a pin, a screw, a plate, a surgical tool, and an implant.

26. 14. The surgical instrument of claim 13, wherein the surgical instrument is selected from the group consisting of a biter, a grasper, a retriever, a pick, a punch, a hook, a probe, a dissector, a retractor, or a scissors.

27. 1. A surgical detection system configured to identify at least one surgical instrument within a field of operation, the system comprising: a camera including at least one sensor configured to capture image data including a first wavelength range and a second wavelength range; an excitation light source that emits excitation emission at an excitation wavelength; a controller in communication with the at least one sensor of the camera, the controller comprising: processing image data from the at least one sensor; identifying said fluorescent emission in said image data output from at least one fluorescent portion of a surgical instrument; and a controller configured to: detect the presence of the surgical instrument in response to the presence of the fluorescent emission.

28. The surgical detection system of claim 27, wherein the fluorescent emissions include wavelengths of light in the near infrared range of approximately 650 nm to 900 nm.

29. The controller:

30. The surgical detection system of claim 28, further configured to detect a number of pixels in the image data within the near infrared range that correspond to a location of the surgical instrument.

30. The controller:

30. The surgical detection system of claim 29, further configured to identify the surgical instrument in response to at least one of a pattern, a shape, and an alphanumeric character of the plurality of pixels.

31. The controller:

30. The surgical detection system of claim 29, further configured to output an indication identifying the presence of the surgical instrument.

32. The surgical detection system of claim 31 , wherein the indication is output on a display device as a notice indicating the location of the surgical instrument within the image data.

33. The controller: accessing a database including at least one computer vision template characterizing an appearance of a potential surgical instrument associated with a surgical procedure; 30. The surgical detection system of claim 29, further configured to: identify the potential surgical instrument as the at least one surgical instrument in response to the plurality of pixels within the near infrared range that correspond to the computer vision template.

34. 34. The surgical detection system of claim 33, wherein the controller is further configured to output a notification to a display device identifying a type or category of the at least one surgical instrument in response to the identification associated with the computer vision template.

35. 28. The surgical sensing system of claim 27, wherein the surgical instrument is selected from the group consisting of a sponge, a suture, a pin, a screw, a plate, a surgical tool, and an implant.

36. 28. The surgical detection system of claim 27, wherein the surgical instrument is selected from the group consisting of a biter, a grasper, a retriever, a pick, a punch, a hook, a probe, a dissector, a retractor, a needle, or scissors.

37. The at least one surgical instrument comprises a plurality of surgical instruments, the at least one fluorescent emission comprises a plurality of fluorescent emissions output from the plurality of surgical instruments, and the controller:

28. The surgical detection system of claim 27, further configured to distinguish between said plurality of surgical instruments in response to at least one of an intensity or pattern of the fluorescent emissions output from said plurality of surgical instruments.

38. 38. The surgical detection system of claim 37, wherein the plurality of surgical instruments comprises a plurality of sutures, and the controller is configured to distinguish between the plurality of sutures in response to a characteristic pattern of fluorescence of the surgical instrument.

Images