Field of view substrate detection system

Abstract

A surgical system can include an endoscope configured to direct light from a distal end of the endoscope toward a target site to illuminate a field of view around a target; a distally-extending optical fiber extending from the distal end of the endoscope configured to deliver a therapeutic energy towards the target and configured to receive, as return light, at least some of the light that is reflected from the target site in response to the illumination; and processing circuitry configured to determine, based at least in part on the return light, a condition associated with the field of view at the target site.

Description

FIELD OF VIEW SUBSTRATE DETECTION SYSTEMPRIORITY CLAIM

[0001] This application claims the benefit of priority to U.S. Provisional Patent Application Serial No. 63 / 677,746, filed luly 31, 2024, the contents of which are incorporated herein by reference.FIELD OF THE DISCLOSURE

[0002] This document relates generally to surgical systems, such as endoscopic systems.BACKGROUND OF THE DISCLOSURE

[0003] A practitioner, such as a physician, clinician, operator, or user, can use an endoscope to provide visual access to an internal location of a patient. In some examples, the endoscope can use a laser system to perform laser therapy, such as ablation, coagulation, vaporization, fragmentation, lithotripsy, and others. In laser therapy, the practitioner can use the endoscope to deliver surgical laser energy to various target treatment areas, such as soft or hard tissue. In lithotripsy, the practitioner can use the endoscope to deliver surgical laser energy to break down calculi structures in the patient’s kidney, gallbladder, ureter, or other stone-forming regions, or to ablate large calculi into smaller fragments.

[0004] Sometimes, when using a laser tissue ablation system, the field of view around the target can become bloody, dusty, cloudy or some other visually obscuring condition. This can delay the procedure being performed.SUMMARY

[0005] In an example, surgical system can include an endoscope configured to direct light from a distal end of the endoscope toward a target site to illuminate a field of view around a target; a distally-extending optical fiberextending from the distal end of the endoscope configured to deliver a therapeutic energy towards the target and configured to receive, as return light, at least some of the light that is reflected from the target site in response to the illumination; and processing circuitry configured to determine, based at least in part on the return light, a condition associated with the field of view at the target site.

[0006] In an example, a surgical system can include a control system including a machine learning module, wherein the machine learning module is configured to assess a database including a plurality of different spectra and each of the plurality of different spectra corresponds to a different condition associated with a field of view, wherein the condition comprises at least one of visually clear, becoming visually clear, visually obscured or becoming visually obscured; an input interface for receiving a specific spectrum of light reflected from a target site in the field of view; an output interface for communicate with a medical device; and if the control system determines that the condition associated with the field of view is visually obscured or becoming visually obscured, the control system being configured to generate an instruction to the medical device, thereby causing the medical device to perform an action to improve the condition associated with the field of view.

[0007] In an example, a method for operating a surgical system can include directing a therapeutic energy through an optical fiber extending from a distal end of an endoscope towards a target; directing a light from a distal end of the endoscope toward the target to illuminate a field of view proximate the target; receiving, with the optical fiber extending from the distal end of the endoscope, as return light, at least some of the light that is reflected from the field of view; and determining, with processing circuitry, from an analysis the return light, a condition associated with the field of view at the target.BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Various embodiments are illustrated by way of example in the figures of the accompanying drawings. Such embodiments are demonstrativeand not intended to be exhaustive or exclusive embodiments of the present subject matter.

[0009] FIG. 1 shows a schematic drawing of an example of a surgical system, in accordance with one embodiment.

[0010] FIG. 2 shows a flow chart of an example of a method for operating a surgical system, in accordance with one embodiment.

[0011] FIG. 3 shows a schematic diagram of an example of a computer- based clinical decision support system (CDSS) that is configured to determine, from a spectral analysis, whether a view of a field of view around a target is obscured, in accordance with one embodiment.DETAILED DESCRIPTION

[0012] In a laser therapy treatment, a practitioner can position a distal end of an endoscope close to a target, such as a kidney stone. The endoscope can include an optical fiber that can deliver therapeutic energy such as a laser light to the target, such as via a distal end of the optical fiber. During the treatment, the endoscope can provide illumination to the target, can form an image of the illuminated target, such as by using a camera at a distal end of the endoscope, and can display the image to the practitioner, such as on a display or a headset.

[0013] FIG. 1 shows a schematic drawing of an example of a surgical system 100, in accordance with one embodiment.

[0014] In this embodiment, the surgical system 100 can include an endoscope 102. The endoscope 102 can be manipulated by the practitioner, who can position the endoscope 102 as needed to view and ablate one or more target sites 104, such as kidney stones, in one or more internal locations of the patient. A field of view 105 surrounds a target site 104. The field of view 105 is the field of view seen by the practitioner through the endoscope 102 during a procedure. In some examples the field of view 105 is viewable on a display 122.

[0015] In some examples, the endoscope 102 can be rigid. In one or more examples, the endoscope 102 can be elongated along an elongation axis. The endoscope 102 can include one or more channels, passages, or apertures thatextend through the endoscope 102 along the elongated axis. For example, the endoscope 102 can include a working channel. In some examples, the practitioner can perform irrigation and / or suction through the working channel. In some examples, the practitioner can pass instruments, such as an optical fiber, brushes, biopsy needles or forceps, through the working channel. In some examples, the practitioner can perform minimally invasive surgery through the working channel, such as to remove unwanted tissue or foreign objects from the body of the patient. As another example, the endoscope 102 can include a separate irrigation channel different from the working channel, which can supply irrigant to a target site 104, such as to flush away pieces of the target, using an irrigation system 107 including an irrigation pump via an irrigation channel 109.

[0016] The surgical system 100 can include an illumination subsystem that can direct illumination from a distal end 106 of the endoscope 102 toward the target site 104. The illumination subsystem can include a light source 108, such as an arc lamp. In some examples, the arc lamp can be a xenon arc lamp. In other examples, the light source 108 can include an LED or other form of light.

[0017] The surgical system 100 can include an optical channel, guide or fiber 116 that can receive light from the light source 108 and direct the light 112 to a distal end of the optical channel / guide / fiber 116.

[0018] In some configurations, the optical channel / guide / fiber 116 can have a distal end that can couple, via a coupler, to a proximal end of an endoscopic illumination channel / guide / fiber 118 that extends along a length of the endoscope 102. In these configurations, the optical channel / guide / fiber 116 can remain with the light source housing 108 for use in multiple surgical procedures. In these configurations, the endoscopic illumination channel / guide / fiber 118 may remain with the endoscope 102, and may optionally be cleaned with the endoscope 102 and reused for subsequent surgical procedures. In these configurations, the endoscopic illumination channel / guide / fiber 118 can direct the light 112 along a length of the endoscope 102 to a distal end of the endoscopic illumination channel / guide / fiber 118, located at the distal end 106 of the endoscope 102. In these configurations, theillumination source light 112 can emerge from the distal end of the endoscopic illumination channel / guide / fiber 118 to illuminate the target site 104.

[0019] In other configurations, the optical channel / guide / fiber 116 can couple, via a coupler at or near the light source 108, to the light source 108, and can extend along the endoscope 102 to the distal end 106 of the endoscope 102. In these configurations, the light 112 can emerge from the distal end of the light optical channel / guide / fiber 116 to illuminate the target site 104.

[0020] In each of these configurations, the endoscope 102 can receive the light 112 from the distal end of the optical channel / guide / fiber 116 and can direct the light 112 from a distal end 106 of the endoscope 102 toward the target site 104. Any or all of these channel / guide / fiber configurations can be used with any or all of the analysis techniques described in detail below.

[0021] The surgical system 100 can include a camera 120, such as a video camera, disposed on the distal end 106 of the endoscope 102. In some examples, the camera 120 can include a lens, a sensor element located at a focal plane of the lens, and electronics that can convert an electrical signal produced by the sensor element into a digital signal. The camera elements can be located in a relatively small, sealed package at the distal end 106 of the endoscope 102. The camera 120 can capture, or generate, a real-time video image of the illuminated target site 104.

[0022] The surgical system 100 can include the display 122, such as a video display, which can display the video image of the illuminated target site 104. For example, the display 122 can be mounted on or in a rack of equipment, away from the endoscope 102, and separate from the light source 108. The display 122 can provide, or display, a real-time video image of the target site 104, illuminated with the light 112, to the practitioner.

[0023] The surgical system 100 can further include a laser source 124 that can generate diagnostic or therapeutic energy such as a laser light 126, such as in pulsed or continuous-wave diagnostic / therapeutic laser light. The laser 124 can be located away from the endoscope 102, such that the endoscope 102 can be positionable by the practitioner, while the laser 124 can be disposed in a laserhousing that can remain in a fixed position, spaced apart from the endoscope 102, during a procedure.

[0024] In some examples, the laser 124 can produce light configured for performing lithotripsy. In some examples, the laser 124 can include a thulium fiber laser, which can produce light having one or more wavelengths between about 1920 nm and about 1960 nm. In some examples, the laser 124 can include a thulium: YAG (yttrium aluminum garnet) laser, which can produce light at a wavelength of 2010 nm. In some examples, the laser 124 can include a holmiunrYAG laser, which can produce light at a wavelength of 2120 nm. In some examples, the laser 124 can include an erbium: YAG laser, which can produce light at a wavelength of 2940 nm. In some examples, the therapeutic laser light 126 produced by the laser 124 can include a first wavelength, such as a wavelength between about 1908 nm and about 2940 nm, or between about 1920 nm and 1960 nm, between about 1900 nm and about 1940 nm, greater than about 1900 nm, greater than about 1800 nm, or others. For these (and other) therapeutic laser light sources, the therapeutic laser light 126 can have a wavelength or wavelengths in a portion of the electromagnetic spectrum at which water (a major component of tissue) or calculus has a relatively high absorption. During a procedure, the tissue / cal cuius can absorb the therapeutic laser light 126, can heat locally to a relatively high temperature, and can break apart due to local thermal strains within the tissue / calculus.

[0025] The surgical system 100 can include a distally-extending optical fiber 130 extending from the distal end 106 of the endoscope 102 via, for example, the working channel of the endoscope 102. In some examples, the distally-extending optical fiber 130 can be a multi-mode optical fiber. In some examples, the distally-extending optical fiber 130 can have a distal end that extends from a distal end 106 of the endoscope 102. In some examples, the distally-extending optical fiber 130 can direct the laser light 126 to and from the target site 104. The endoscope 102 can direct the diagnostic / therapeutic laser light 126 toward the target site 104 through the distally-extending optical fiber 130. The distally-extending optical fiber 130 can then receive, as return light 132, at least some of the light 112 that is reflected from the area of the field ofview 105 and the target site 104. The distally-extending optical fiber 130 can also receive, as return diagnostic / therapeutic laser light 134, at least some of the diagnostic / therapeutic laser light 126 that is reflected from the field of view 105 and the target site 104. The endoscope 102 can further include a mechanism to extend and retract the optical fiber 130 from the distal end of the endoscope 102.

[0026] A laser optical fiber 136 can couple to the distally-extending optical fiber 130 to deliver the laser light 126 to the target 104 and to direct the return diagnostic / therapeutic laser light 134 and / or the return light 132 (in response to the light 112 from the light source 108) from the target 104 and redirect such light to an optical sensor 140, such as a spectrometer. The return diagnostic / therapeutic laser light 134 can be spectrally separated from the return light 132. In other words, the diagnostic / therapeutic laser light 126 and the return diagnostic / therapeutic laser light 134 can have a first wavelength, and the light 132 can have a spectrum that does not include the first wavelength. To filter the return diagnostic / therapeutic laser light 134 out of the detection path, the surgical system 100 can further include an optical filter (not shown). In one embodiment, a beam splitter 138 can be implemented to direct the return illumination source light 132 toward an optical sensor 140.

[0027] As noted above, sometimes when using a laser ablation system described herein, the field of view 105 around the target can become visually obscured, such as becoming bloody, dusty, cloudy or have some other visually obscuring condition. This can delay the procedure being performed. As used herein, the terms “becoming” and “will become” refer to a situation where a change in the return illumination source light 132 acquired by the optical sensor 140 and / or a change in the video images acquired by the camera associated with the target site 104 is detected by any suitable software and / or hardware and / or perceived by a human eye. For example, when a subsequently acquired image frame shows that the target site contains more blood than the previously acquired image frame, it indicates that the field of view 105 is becoming bloody.

[0028] Additionally or alternatively, the present surgical system 100 can include processing circuitry 144 configured to determine, based at least in parton the return light 132, a condition associated with the field of view 105 at the target site as further described below.

[0029] For example, the present surgical system 100 can include the optical sensor 140 that can measure and analyze an optical spectrum of the return illumination source light 132. In some examples, the optical sensor 140 can be disposed in or on a laser housing 128. For example, in some configurations, the optical sensor 140 can include a spectrometer that can analyze light that returns proximally through distally-extending optical fiber 130, including the return light 132. In one example, the spectrometer and the processor circuitry 144 (described below) can use the spectral profile of the field of the field of view 105 and / or the target site 104 to determine a material composition of the field of view 105, such as by matching the measured spectral profile of the field of view 105 to one or more of a specified plurality of predetermined spectral profiles that correspond to known material compositions. The spectrometer can generate a sensor output signal 142 that includes data that represents light intensity (or amplitude, or other suitable photometric quantity) as a function of wavelength. The processor circuitry 144 can receive and interpret the sensor output signal 142.

[0030] As noted, the surgical system 100 can include the processor circuitry 144 that can analyze the optical spectrum of the return light 132. In some examples, the processor circuitry 144 may be referred to as a controller. In some examples, the processor circuitry 144 may be implemented purely in software. In some examples, the processor circuitry 144 may be implemented purely in hardware. In some examples, the processor circuitry 144 may be implemented as a combination of software and hardware. In some examples, the processor circuitry 144 may be implemented on a single processor. In some examples, the processor circuitry 144 may be implemented on multiple processors. In some examples, the multiple processors may be housed in a common housing. In some examples, at least two of the multiple processors may be spaced apart in different housings. In some examples, the processor circuitry 144 can be disposed in or on the laser housing 128.

[0031] In one example, the processor circuitry 144 can determine, from the analysis of the optical spectrum of the return light 132, that the field of view is visually obscured or is becoming visually obscured. For example, the optical spectrum of the return light 132 detected by the sensor 140 at a subsequent detection time may indicate that the target site contains more blood than that detected by the sensor 140 based on the optical spectrum of the return light 132 detected in a previous detection time. In one embodiment, the processing circuitry 144 can then control the pump of the irrigation system 107, the suction system, and / or other aspects of the endoscope 102, such as the position of the optical fiber 130 and the laser parameters to mitigate the problem before it is noticeable by the practitioner. In another embodiment, the processing circuitry send instructions to a pump controller (not shown) to control the irrigation system 107 and / or the suction system as described above. Likewise, if the processor circuitry determines that the field of view is visually clear or becoming visually clear (e.g., based on the spectral analysis of the return light 132 and / or the video images), the circuitry can control the irrigation system and / or suction system or send instructions to the pump controller of the irrigation system and / or suction system to stop the mitigating actions. In another example, the processor circuitry can alert the user that a problem reading the field of view may arise by providing at least one of an acoustic, visual, or haptic signal. In a further example, the processor circuitry can advise the user on steps to take to mitigate or prevent the problem by providing instructions on the display 122.

[0032] Accordingly, the system includes the optical sensor 140 which is configured to measure an optical spectrum of the return light 134, and the processing circuitry 144 which is configured to analyze the optical spectrum of the return light 134, and determine, from the analysis of the optical spectrum of the return light, that a field of view 105 of the target site 104 is visually obscured or will become visually obscured, and the system can deliver instructions to automatically perform an action to mitigate of prevent the field of view 105 from becoming visually obscured, or the system can alert a user to take precautionary steps. This mitigating action will improve the condition associated with the field of view by providing a view that is easier to see through the endoscope than thevisually obscured view. To improve the condition associated with the field of view means that the condition of the field of view changes from more visually obscured state to a less visually obscured state.

[0033] As will be further detailed below, in one example, the processing circuitry 144 can compare the optical spectrum of the return light 134 to a plurality of optical spectra in a database, each corresponding to a condition associated with the field of view of the target site, and further wherein the determination of the condition of the field of view is based at least in part on a best match between the optical spectrum of the return light and the plurality of optical spectra such that the system can determine, by a machine learning model, for example, that the view will become obscured if action is not taken.

[0034] In one example, the processing circuitry 144 can be configured to analyze the optical spectrum of the return light 134 continually over time during a procedure. The processing circuitry 144 can then then compare spectra from different times during the procedure and, optionally using the machine learning model, can determine, over time, that the field of view 105 is becoming visually obscured in some way. This means the field of view 105 is changing, over time, from a less visually obscured state to a more visually obscured state, as objectively determined by the processing circuitry 144. The processing circuitry 144 can further determine, over time, when the field of view 105 is becoming visually clear, and can then decrease the rate of irrigation, or other mitigating actions, for example. Again, becoming visually clear means the field of view 105 is changing, over time, from a more visually obscured state to a less visually obscured state, as objectively determined by the processing circuitry 144.

[0035] For example, in one embodiment, the processing circuitry 44 can be configured to analyze the optical spectrum of the return light 134 by forming, from a comparison of the optical spectrum of the return light 134 to a plurality of different optical spectra in a database, each corresponding to a condition associated with the field of view of the target site, a determination that the return light 134 defines a spectrum indicating that the field of view 105 around the target is dusty or is becoming dusty.

[0036] If the processing circuitry 44 determines that the field of view 105 is dusty or is becoming dusty, the processing circuitry 44 can deliver instructions to the pump of the irrigation and / or suction systems associated with the endoscope to deliver increased irrigation (and in some embodiments, increased suction as well) through the endoscope 102 proximate the target 104. For example, if the irrigation is at zero, the irrigation could be turned on. If the irrigation is already on, the rate of irrigation can be increased (and in some embodiments, the rate of suction can be increased in order to maintain a desired pressure at the target site 104).

[0037] In another step, the processing circuitry 44 can instruct the mechanism in the endoscope to automatically advance or retract the optical fiber 130 closer to or away from the target 104 based on the condition of the target site, so the laser will be more effective and result in less dustiness of the view. In further examples, the processing circuitry 44 can also change various laser parameters such as pulse width, pulse frequency, peak power, and / or pulse power which will change the dustiness level of the field of view.

[0038] In another example, the processing circuitry 44 can be configured to analyze the optical spectrum of the return light 134 by forming, from a comparison of the optical spectrum of the return light 134 to a plurality of different optical spectra in a database, a determination that the return light defines a spectrum indicating that a field of view 105 around the target 104 is bloody or is becoming bloody.

[0039] Again, if the processing circuitry 44 determines that the field of view is bloody or is becoming bloody, the processing circuitry 44 can deliver instructions to the irrigation system and / or suction system associated with the endoscope to deliver increased irrigation (and in some embodiments, increased suction as well) through the endoscope proximate the target. Likewise, the processing circuitry can also change the laser distances or operating parameters, as discussed above.

[0040] In another example, the processing circuitry 44 can be configured to analyze the optical spectrum of the return light 134 by forming, from a comparison of the optical spectrum of the return light 134 to a plurality ofdifferent optical spectra in a database, a determination that the return light defines a spectrum indicating that a field of view 105 around the target includes bubbles or is becoming bubbly.

[0041] If the processing circuitry 44 determines that the field of view includes bubbles or is becoming bubbly the processing circuitry 44 can deliver instructions to the endoscope to advance the optical fiber 130 closer to the target 104.

[0042] As will be detailed below, the processing circuity 44 can optionally include a machine learning module where the processing circuitry 44 can include a database including a plurality of different spectra and each of the plurality of different spectra can be characterized or defined as corresponding to a different situation or condition in the field of view at the target site. For example, the different situation can include the field of view being characterized as being either clear, bloody, becoming bloody, dusty, becoming dusty, bubbly, or becoming bubbly. Thus, each of the plurality of spectra in the database can be labelled as having the characteristics of a specific field of view. Thus, the machine learning module of the processing circuitry can implement a machine learning algorithm and be configured to predict, from analysis of a specific field of view spectrum compared to the plurality of different spectra in the database, that the field of view proximate the target will become visually obscured if a preventative action is not taken.

[0043] In an example, the determination of the condition associated with the field of view at the target site is based at least in part on a best match between the optical spectrum of the return light and the plurality of the different optical spectra. Best match means that between two or more different possible characterizations of a specific spectrum, the processing circuitry determines the better of the possible choices. For example, the processing circuitry 44 can include different comparison ranges for the different plurality of spectra so that if there is an overlap in the spectra, compared to the return spectrum being compared, the processing circuitry 44 can be configured to choose the condition closest to the present optical spectrum of return light. In some examples, the processing circuitry can use a statical probability of different spectra occurringto determine a best match. In one example, the processing circuitry can compare specific spectra, over time, to determine what the best match is based on whether the specific spectra are changing in some fashion. Thus the comparison results can be weighted in some manner so the processing circuitry 44 will choose a best match based on the weighted results of the comparison.

[0044] In some examples, the processing circuitry 144 can alert a user to perform the action to mitigate or prevent the target site from becoming or remaining visually obscured. For example, the display 122 can be configured to display a visual alert such as a flashing signal or an audio alert such as a beeping signal or the system can alert the user using some other acoustic, visual, or haptic signal if the processor determines that the target site will become visually obscured if not mitigated. Thus, a user can take action before the site is obscured, but it is not readily noticeable.

[0045] Accordingly, by employing a spectrometer, a trained machine learning model or Al model (detailed below), and an additional computer, the present system can detect and predict the occurrence of problematic field of view conditions during a procedure. The information of the condition can then be utilized to adjust irrigation levels and / or suction levels, and / or control the laser system, thereby improving the overall surgical outcome. For example, the algorithm may predict, based on the trend of the spectral analysis and / or imaging analysis obtained in the most recent detection times (e.g., in the last 10 seconds) using, for example, an extrapolation approach, that the field of view will become obscure in the next 10 seconds to a degree that the physician can no longer clearly see the field of view. The processing circuity 44 may, based thereon, automatically increase the irrigation levels and / or suction levels, and / or control the laser system so as to provide a clear field of view to the physician. Likewise, in some embodiments, the algorithm may predict, based on the trend of the spectral analysis and / or imaging analysis obtained in the most recent detection times (e.g., in the last 10 seconds) that the field of view will become clear in the next 10 seconds to a degree that the physician can clearly see the field of view. The processing circuity 44 may then, based thereon, automatically decrease theirrigation levels and / or suction levels, and / or control the laser system to terminate the mitigation action.

[0046] FIG. 2 shows a flow chart of an example of a method (200) for operating a surgical system, in accordance with one embodiment.

[0047] Referring also to FIG. 1, the method (200) includes the user directing therapeutic energy such as a laser through an optical fiber extending from a distal end of an endoscope towards a target, and directing an illumination light from a distal end of an endoscope toward a target site to illuminate a field of view proximate the target (210). The method further includes receiving, with the optical fiber extending from the distal end of the endoscope, as return light, at least some of the light that is reflected from the field of view proximate the target (220), and measuring, with an optical sensor, an optical spectrum of the return light (230). The method further includes analyzing, with processing circuitry, the optical spectrum of the return light (240). The method then includes determining, with the processing circuitry, from the analysis of the optical spectrum of the return light, whether a field of view of the target site is visually obscured or will become visually obscured (250), and performing an action to mitigate or prevent the target site from becoming or remaining visually obscured.

[0048] As noted above, some examples of determining whether the field of view is visually obscured or will become visually obscured can include at least one of determining that the field of view is bloody, becoming bloody, dusty, becoming dusty, bubbly, or becoming bubbly. The system can determine whether the field of view is becoming visually obscured or becoming visually clear by comparing the return spectra over time to determine that the condition of the field of view is changing over time, i.e. becoming visually obscure or becoming visually clear.

[0049] Performing the action to mitigate or prevent the target site from becoming or remaining visually obscured can include the processing circuitry automatically delivering instructions to the endoscope to perform the action to mitigate or prevent the target site from becoming or remaining visually obscured. For example, the action to mitigate or prevent can include at least oneof: advancing or retracting the optical fiber, changing the operating parameters of the laser, such as pulse width and pulse power, and / or increasing a level of irrigation proximate the field of view.

[0050] In other examples, the system can signal or alert the user to perform the action(s) to mitigate or prevent the target site from becoming or remaining visually obscured.

[0051] In one example, the system can include a machine learning module which is configured to predict a time, based on the return light spectra changing over time, when the field of view at the target site will become visually obscured and to then automatically cause the collection of return signals to be stopped at the predicted time.

[0052] In another example, the system can be configured to predict a time, based on the return light spectra changing over time, when the field of view at the target site will become visually clear and then cause the return signals to be collected or analyzed at the predicted time.

[0053] FIG. 3 shows a schematic diagram of an example of a computer- based clinical decision support system (CDSS) 300 that is configured to diagnose and suggest action based on a spectral analysis of a field of view during an endoscopic procedure. Specifically, the CDSS 300 can be configured to determine, from a spectral analysis, whether a view of a field of view around a target is obscured or is in danger of becoming obscured. In various embodiments, the CDSS 300 includes an input interface 302 through which the optical property which is specific to a field of view is provided as input features to an artificial intelligence (Al) model 304, including a processor which performs an inference in which the spectrum is applied to the Al model to generate a characterization of the field of view, and a user interface (UI) 308 through which information can be communicated to a user, e.g., a practitioner.

[0054] In some embodiments, the input interface 302 may be a direct data link between the CDSS 300 and one or more medical devices, such as surgical system 100 or endoscope 102, which generate at least some of the input features. For example, the input interface 302 may transmit the spectrum directly to the CDSS 300 during a therapeutic and / or diagnostic medicalprocedure. Additionally, or alternatively, the input interface 302 may be a classical user interface that facilitates interaction between a user and the CDSS 300. For example, the input interface 302 may facilitate a user interface through which the user may manually enter information. Additionally, or alternatively, the input interface 302 may provide the CDSS 300 with access to an electronic patient record from which one or more input features may be extracted. In any of these cases, the input interface 302 is configured to collect one or more input features in association with a specific patient on or before a time at which the CDSS 300 is used to assess the spectrum and whether the field of view is in danger of becoming obscured.

[0055] For example, the database can include a plurality of spectra and their associated visual characterizations (i.e., clear, cloudy, bloody, etc.) along with associated metadata of each spectrum. For example, various spectra can be collected in the database along with associated metadata of each scan, such as the optical fiber being used, the laser settings being used, the distance of the optical fiber from the target, and other data associated with each optical scan defining a given spectrum. These spectra and metadata can be categorized and input into the CDSS 400. The Al model, through machine learning, can then develop the ability to characterize any given spectrum based on the previous plurality of spectra in the database.

[0056] In use, the input will also include the specific spectrum of the present field of view received from the endoscope 102. Also, over time, the specific spectra received from the endoscope can be compared to each other over time to allow the system to further determine if the condition is improving or getting worse based on comparisons of earlier and later spectra. Moreover, the input can also include metadata associated with the scan including the optical fiber being used, the laser parameters, and so on.

[0057] Based on one or more of the above input features, the processor, such as processor circuitry 144, performs an inference operation using the Al model 304 to generate a characterization of the field of view and suggest an action to prevent any obscuring of the view. For example, input interface 302 may deliver the spectrum of the field of view and the associated metadata intothe input layer of the Al model which propagates this input feature through the Al model to the output layer. The Al model can provide a computer system the ability to perform tasks, without explicitly being programmed, by making inferences based on patterns found in the analysis of data. Al model explores the study and construction of algorithms (e.g., machine-learning algorithms) that may learn from existing data and make predictions about new data. Such algorithms operate by building an Al model from example training data in order to make data-driven predictions or decisions expressed as outputs or assessments.

[0058] There are two common modes for machine learning (ML): supervised ML and unsupervised ML. Supervised ML uses prior knowledge (e.g., examples that correlate inputs to outputs or outcomes) to learn the relationships between the inputs and the outputs. The goal of supervised ML is to learn a function that, given some training data, best approximates the relationship between the training inputs and outputs so that the ML model can implement the same relationships when given inputs to generate the corresponding outputs. Unsupervised ML is the training of an ML algorithm using information that is neither classified nor labeled and allowing the algorithm to act on that information without guidance. Unsupervised ML is useful in exploratory analysis because it can automatically identify structure in data.

[0059] Common tasks for supervised ML are classification problems and regression problems. Classification problems, also referred to as categorization problems, aim at classifying items into one of several category values (for example, is this object an apple or an orange?). Regression algorithms aim at quantifying some items (for example, by providing a score to the value of some input). Some examples of commonly used supervised-ML algorithms are Logistic Regression (LR), Naive-Bayes, Random Forest (RF), neural networks (NN), deep neural networks (DNN), matrix factorization, and Support Vector Machines (SVM). The present system is interested in classifying a field of view based on a received spectrum, for example.

[0060] Some common tasks for unsupervised ML include clustering, representation learning, and density estimation. Some examples of commonly used unsupervised-ML algorithms are K-means clustering, principal component analysis, and autoencoders.

[0061] Another type of ML is federated learning (also known as collaborative learning) that trains an algorithm across multiple decentralized devices holding local data, without exchanging the data. This approach stands in contrast to traditional centralized machine-learning techniques where all the local datasets are uploaded to one server, as well as to more classical decentralized approaches which often assume that local data samples are identically distributed. Federated learning enables multiple actors to build a common, robust machine learning model without sharing data, thus allowing to address critical issues such as data privacy, data security, data access rights and access to heterogeneous data.

[0062] In some examples, the Al model may be trained continuously or periodically prior to performance of the inference operation by the processor such as processor circuitry 144. Then, during the inference operation, the patient specific input features provided to the Al model may be propagated from an input layer, through one or more hidden layers, and ultimately to an output layer that corresponds to the spectral profile.

[0063] In some examples, the Al model can include a database, which can include data corresponding to a patient. The database can provide a patient record to the CDSS 300. In some examples, the Al model can receive a measured spectrum.

[0064] During and / or subsequent to the inference operation, the spectral profile may be communicated to the user via the user interface (UI), such as output interface 308, and / or automatically cause the processor or an alarm connected to the processor to perform a desired action. In some examples, the CDSS 300 can optionally be used to determine the action taken in response to a given spectrum.

[0065] Accordingly, the present disclosure can include a control system including a machine learning module, wherein the machine learning module iscoupled to a database including a plurality of different spectra and each of the plurality of different spectra are characterized as being either visually clear, bloody, becoming bloody, dusty, becoming dusty, bubbly, or becoming bubbly. The machine learning can include associated metadata, such as the fiber being used, the laser settings being used, and other data associated with each scan. The system can further include an input interface where a specific spectrum from a light reflected from a specific field of view is input to be analyzed by the machine learning module, an output interface where the specific spectrum is characterized, and if the control system determines that the specific spectrum is bloody, becoming bloody, dusty, becoming dusty, bubbly, or becoming bubbly, the control system being configured to perform an action to mitigate a problem causing the specific field of view to being or becoming visually obscured. As noted above, in one example, the input can further include the metadata of the specific spectrum being analyzed, such as the optical fiber being used and the laser settings. Moreover, the specific spectrum can be compared to earlier spectra to allow the system to determine changes, over time, of the condition of the field of vison.

[0066] In various examples, the control system action can include advancing or retracting an optical fiber, changing the laser operating parameters, and increasing or decreasing irrigation proximate the field of view.

[0067] In summary, when using a laser tissue ablation system the field of view around the target can become bloody, dusty, cloudy or some other obscuring condition. This can delay the procedure being performed.

[0068] In the present system, by employing a spectrometer, a trained machine learning model or Al model, and an additional computer, the present system can detect and predict the occurrence of problematic field of view conditions during a procedure. The information of the condition can then be utilized to adjust irrigation levels and / or control the laser system, thereby improving the overall surgical outcome.

[0069] Thus, the present system can determine the substrate surrounding the optical fiber's position by analyzing the reflected light captured during the process. This reflected light is directed back up through the laser fiber and into aspectrometer. A computer then interprets the spectrometer data and employs a machine leaming / artificial intelligence model to predict the substrate in the field of view in which the fiber operates. Leveraging this information, the system can optimize the surgery by adjusting the irrigation system and laser parameters for enhanced precision and effectiveness.

[0070] In the foregoing detailed description, the method and apparatus of the present disclosure have been described with reference to specific embodiments thereof. It will, however, be evident that various modifications and changes may be made thereto without departing from the broader spirit and scope of the present disclosure. The present specification and figures are accordingly to be regarded as illustrative rather than restrictive.

[0071] To further illustrate the device and related method disclosed herein, a non-limiting list of examples is provided below. Each of the following non limiting examples can stand on its own or can be combined in any permutation or combination with any one or more of the other examples.

[0072] In Example 1, a surgical system can include an endoscope configured to direct light from a distal end of the endoscope toward a target site to illuminate a field of view around a target; a distally-extending optical fiber extending from the distal end of the endoscope configured to deliver a therapeutic energy towards the target and configured to receive, as return light, at least some of the light that is reflected from the target site in response to the illumination; and processing circuitry configured to determine, based at least in part on the return light, a condition associated with the field of view at the target site.

[0073] In Example 2, the surgical system of Example 1 can optionally be configured to further comprise an optical sensor configured to analyze an optical spectrum of the return light, wherein the condition of the field of view at the target site comprises whether the field of view of the target site is visually obscured or will become visually obscured based on the optical spectrum.

[0074] In Example 3, the surgical system of any one of Examples 1-2 can optionally be configured such that the processing circuitry is furtherconfigured to determine whether the visually obscured field of view results from dust, blood, and / or bubbles generated by the therapeutic energy.

[0075] In Example 4, the surgical system of any one of Examples 1-3 can optionally be configured such that the processing circuitry is further configured to analyze the optical spectrum of the return light comparing the optical spectrum of the return light against a plurality of optical spectra stored in a database, each spectrum corresponding to a different condition associated with the field of view at the target site, and further wherein the determination on the condition associated with the field of view at the target site is based at least in part on a best match between the optical spectrum of the return light and the plurality of the optical spectra.

[0076] In Example 5, the surgical system of any one of Examples 1-4 can optionally be configured to further comprise an irrigation system having a pump, wherein upon the processing circuitry determining that the field of view is visually obscured or is becoming visually obscured, the processing circuitry is further configured to deliver instructions to the pump to deliver increased irrigation through the endoscope to the target site.

[0077] In Example 6, the surgical system of any one of Examples 1-5 can optionally be configured to further comprise an irrigation system having a pump, wherein upon the processing circuitry determining that the field of view is visually clear or is becoming visually clear, the processing circuitry is further configured to deliver instructions to the pump to decrease irrigation through the endoscope to the target site.

[0078] In Example 7, the surgical system of any one of Examples 1-6 can optionally be configured to further comprise a mechanism for advancing or retracting the optical fiber, wherein the processing circuitry is further configured to further deliver instructions to the mechanism to advance or retract the optical fiber from the target and / or change a parameter of the therapeutic energy based at least in part on the condition associated with the field of view at the target site.

[0079] In Example 8, the surgical system of any one of Examples 1-7 can optionally be configured such that the processing circuitry is further configured to (i) instruct the mechanism to advance the optical fiber to the targetsite and / or (ii) increase a power of the therapeutic energy to a desired power when the condition associated with the field of view at the target site is visually clear or is becoming visually clear.

[0080] In Example 9, the surgical system of any one of Examples 1-8 can optionally be configured such that the processing circuitry is further configured to (i) instruct the mechanism to retract the optical fiber from the target site and / or (ii) decrease a power of the therapeutic energy when the field of view is visually obscured or is becoming visually obscured.

[0081] In Example 10, the surgical system of any one of Examples 1-9 can optionally be configured such that the processing circuitry includes a machine learning module for implementing a machine learning algorithm to determine, based at least in part on the return light, the condition associated with the field of view in the target site.

[0082] In Example 11, the surgical system of any one of Examples 1-10 can optionally be configured such that the condition includes at least one of a clear, becoming clear, bloody, becoming bloody, dusty, becoming dusty, bubbly, or becoming bubbly field of view.

[0083] In Example 12, the surgical system of any one of Examples 1-11 can optionally be configured such that the machine learning module of the processing circuitry is configured to predict that the condition associated with the field of view at the target site will become visually obscured if a preventative action is not taken based on the return light received at a plurality of times.

[0084] In Example 13, the surgical system of any one of Examples 1-12 can optionally be configured such that the machine learning module is configured to predict a time when the field of view at the target site will become visually obscured and to then automatically cause the collection of return signals to be stopped at the predicted time.

[0085] In Example 14, the machine learning module is configured to predict a time when the field of view at the target site will become visually clear and then cause the return signals to be collected or analyzed at the predicted time.

[0086] In Example 15, the surgical system of any of claims 1-14 can optionally be configured such that upon determining that the condition associated with the field of view at the target site will become visually obscured if a preventative action is not taken, the processing circuitry alerts a user by providing at least one of an acoustic, visual or haptic signal so as to allow the user to perform an action to mitigate or prevent the target site from becoming or remaining visually obscured.

[0087] In Example 16, a surgical system can include a control system including a machine learning module, wherein the machine learning module is configured to assess a database including a plurality of different spectra and each of the plurality of different spectra corresponds to a different condition associated with a field of view, wherein the condition comprises at least one of visually clear, becoming visually clear, visually obscured or becoming visually obscured; an input interface for receiving a specific spectrum of light reflected from a target site in the field of view; an output interface for communicate with a medical device; and if the control system determines that the condition associated with the field of view is visually obscured or becoming visually obscured, the control system being configured to generate an instruction to the medical device, thereby causing the medical device to perform an action to improve the condition associated with the field of view.

[0088] In Example 17, the surgical system of Example 16 can optionally be configured such that the control system action can include delivering instructions to an endoscope to advance or retract an optical fiber from a target site proximate the field of view and / or change a parameter of a therapeutic energy based at least in part on the condition associated with the field of view at the target site.

[0089] In Example 18, the surgical system of any one of Examples 16-17 can optionally be configured such that the control system action can include delivering instructions to an irrigation system of an endoscope to increase or decrease a level of irrigation proximate the field of view.

[0090] In Example 19, a method for operating a surgical system can include directing a therapeutic energy through an optical fiber extending from adistal end of an endoscope towards a target; directing a light from a distal end of the endoscope toward the target to illuminate a field of view proximate the target; receiving, with the optical fiber extending from the distal end of the endoscope, as return light, at least some of the light that is reflected from the field of view; and determining, with processing circuitry, from an analysis the return light, a condition associated with the field of view at the target.

[0091] In Example 20, the method of Example 19 can optionally be configured such that determining the condition includes determining whether the field of view is visually obscured or will become visually obscured.

[0092] In Example 21, the method of any of Examples 19-20 can optionally be configured to further include the processing circuitry automatically delivering instructions to the endoscope to perform one or more actions action to mitigate or prevent the target site from becoming or remaining visually obscured.

[0093] In Example 22, the method of any of Examples 19-21 can optionally be configured such that the action to mitigate or prevent can include at least one of advancing or retracting an optical fiber, changing operating parameters of the therapeutic energy, and increasing a level of irrigation proximate the field of view.

[0094] Each of these non-limiting examples can stand on its own, or can be combined in various permutations or combinations with one or more of the other examples.

[0095] The above detailed description includes references to the accompanying drawings, which form a part of the detailed description. The drawings show, by way of illustration, specific embodiments in which the invention can be practiced. These embodiments are also referred to herein as “examples.” Such examples can include elements in addition to those shown or described. However, the present inventor also contemplates examples in which only those elements shown or described are provided. Moreover, the present inventor also contemplates examples using any combination or permutation of those elements shown or described (or one or more aspects thereof), either withrespect to a particular example (or one or more aspects thereof), or with respect to other examples (or one or more aspects thereof) shown or described herein.

[0096] In the event of inconsistent usages between this document and any documents so incorporated by reference, the usage in this document controls.

[0097] In this document, the terms “a” or “an” are used, as is common in patent documents, to include one or more than one, independent of any other instances or usages of “at least one” or “one or more.” In this document, the term “or” is used to refer to a nonexclusive or, such that “A or B” includes “A but not B,” “B but not A,” and “A and B,” unless otherwise indicated. In this document, the terms “including” and “in which” are used as the plain-English equivalents of the respective terms “comprising” and “wherein.” Also, in the following claims, the terms “including” and “comprising” are open-ended, that is, a system, device, article, composition, formulation, or process that includes elements in addition to those listed after such a term in a claim are still deemed to fall within the scope of that claim. Moreover, in the following claims, the terms “first,” “second,” and “third,” etc. are used merely as labels, and are not intended to impose numerical requirements on their objects.

[0098] Method examples described herein can be machine or computer- implemented at least in part. Some examples can include a computer-readable medium or machine-readable medium encoded with instructions operable to configure an electronic device to perform methods as described in the above examples. An implementation of such methods can include code, such as microcode, assembly language code, a higher-level language code, or the like. Such code can include computer readable instructions for performing various methods. The code may form portions of computer program products. Further, in an example, the code can be tangibly stored on one or more volatile, non- transitory, or non-volatile tangible computer-readable media, such as during execution or at other times. Examples of these tangible computer-readable media can include, but are not limited to, hard disks, removable magnetic disks, removable optical disks (e.g., compact disks and digital video disks), magneticcassettes, memory cards or sticks, random access memories (RAMs), read only memories (ROMs), and the like.

[0099] The above description is intended to be illustrative, and not restrictive. For example, the above-described examples (or one or more aspects thereof) may be used in combination with each other. Other embodiments can be used, such as by one of ordinary skill in the art upon reviewing the above description. The Abstract is provided to allow the reader to quickly ascertain the nature of the technical disclosure. It is submitted with the understanding that it will not be used to interpret or limit the scope or meaning of the claims. Also, in the above Detailed Description, various features may be grouped together to streamline the disclosure. This should not be interpreted as intending that an unclaimed disclosed feature is essential to any claim. Rather, inventive subject matter may lie in less than all features of a particular disclosed embodiment. Thus, the following claims are hereby incorporated into the Detailed Description as examples or embodiments, with each claim standing on its own as a separate embodiment, and it is contemplated that such embodiments can be combined with each other in various combinations or permutations. The scope of the invention should be determined with reference to the appended claims, along with the full scope of equivalents to which such claims are entitled.

Claims

WHAT IS CLAIMED IS:

1. A surgical system comprising: an endoscope configured to direct light from a distal end of the endoscope toward a target site to illuminate a field of view around a target; a distally-extending optical fiber extending from the distal end of the endoscope configured to deliver a therapeutic energy towards the target and configured to receive, as return light, at least some of the light that is reflected from the target site in response to the illumination; and processing circuitry configured to determine, based at least in part on the return light, a condition associated with the field of view at the target site.

2. The surgical system of claim 1, further comprising an optical sensor configured to analyze an optical spectrum of the return light, wherein the condition of the field of view at the target site comprises whether the field of view of the target site is visually obscured or will become visually obscured based on the optical spectrum.

3. The surgical system of claim 2, wherein the processing circuitry is further configured to determine whether the visually obscured field of view results from dust, blood, and / or bubbles generated by the therapeutic energy.

4. The surgical system of claim 2, wherein the processing circuitry is further configured to analyze the optical spectrum of the return light comparing the optical spectrum of the return light against a plurality of optical spectra stored in a database, each spectrum corresponding to a different condition associated with the field of view at the target site, and further wherein the determination on the condition associated with the field of view at the target site is based at least in part on a best match between the optical spectrum of the return light and the plurality of the optical spectra.

5. The surgical system of claim 4, further comprising an irrigation system having a pump, wherein upon the processing circuitry determining that the fieldof view is visually obscured or is becoming visually obscured, the processing circuitry is further configured to deliver instructions to the pump to deliver increased irrigation through the endoscope to the target site.

6. The surgical system of claim 4, further comprising an irrigation system having a pump, wherein upon the processing circuitry determining that the field of view is visually clear or is becoming visually clear, the processing circuitry is further configured to deliver instructions to the pump to decrease irrigation through the endoscope to the target site.

7. The surgical system of claim 4, further comprising a mechanism for advancing or retracting the optical fiber, wherein the processing circuitry is further configured to further deliver instructions to the mechanism to advance or retract the optical fiber from the target and / or change a parameter of the therapeutic energy based at least in part on the condition associated with the field of view at the target site.

8. The surgical system of claim 7, wherein the processing circuitry is further configured to (i) instruct the mechanism to advance the optical fiber to the target site and / or (ii) increase a power of the therapeutic energy to a desired power when the condition associated with the field of view at the target site is visually clear or is becoming visually clear.

9. The surgical system of claim 7, wherein the processing circuitry is further configured to (i) instruct the mechanism to retract the optical fiber from the target site and / or (ii) decrease a power of the therapeutic energy when the field of view is visually obscured or is becoming visually obscured.

10. The surgical system of claim 1, wherein the processing circuitry includes a machine learning module for implementing a machine learning algorithm to determine, based at least in part on the return light, the condition associated with the field of view in the target site.

11. The surgical system of claim 10, wherein the condition includes at least one of a clear, becoming clear, bloody, becoming bloody, dusty, becoming dusty, bubbly, or becoming bubbly field of view.

12. The surgical system of claim 10, wherein the machine learning module of the processing circuitry is configured to predict that the condition associated with the field of view at the target site will become visually obscured if a preventative action is not taken based on the return light received at a plurality of times.

13. The surgical system of claim 12, wherein the machine learning module is configured to predict a time when the field of view at the target site will become visually obscured and to then automatically cause the collection of return light to be stopped at the predicted time.

14. The surgical system of claim 12, wherein the machine learning module is configured to predict a time when the field of view at the target site will become visually clear and then cause the return light to be collected or analyzed at the predicted time.

15. The surgical system of claim 12, wherein upon determining that the condition associated with the field of view at the target site will become visually obscured if a preventative action is not taken, the processing circuitry alerts a user by providing at least one of an acoustic, visual or haptic signal so as to allow the user to perform an action to mitigate or prevent the target site from becoming or remaining visually obscured.

16. A surgical system comprising: a control system including a machine learning module, wherein the machine learning module is configured to assess a database including a plurality of different spectra and each of the plurality of different spectra corresponds to a different condition associated with a field of view, wherein the conditioncomprises at least one of visually clear, becoming visually clear, visually obscured or becoming visually obscured; an input interface for receiving a specific spectrum of light reflected from a target site in the field of view; an output interface for communicate with a medical device; and if the control system determines that the condition associated with the field of view is visually obscured or becoming visually obscured, the control system being configured to generate an instruction to the medical device, thereby causing the medical device to perform an action to improve the condition associated with the field of view.

17. The surgical system of claim 16, wherein the control system action can include delivering instructions to an endoscope to advance or retract an optical fiber from a target site proximate the field of view and / or change a parameter of a therapeutic energy based at least in part on the condition associated with the field of view at the target site.

18. The surgical system of claim 16, wherein the control system action can include delivering instructions to an irrigation system of an endoscope to increase or decrease a level of irrigation proximate the field of view.

19. A method for operating a surgical system, the method comprising: directing a therapeutic energy through an optical fiber extending from a distal end of an endoscope towards a target; directing a light from a distal end of the endoscope toward the target to illuminate a field of view proximate the target; receiving, with the optical fiber extending from the distal end of the endoscope, as return light, at least some of the light that is reflected from the field of view; and determining, with processing circuitry, from an analysis the return light, a condition associated with the field of view at the target.

20. The method of claim 19, wherein determining the condition includes determining whether the field of view is visually obscured or will become visually obscured.

21. The method of claim 20, further including the processing circuitry automatically delivering instructions to the endoscope to perform one or more actions action to mitigate or prevent the target site from becoming or remaining visually obscured.

22. The method of claim 21, wherein the action to mitigate or prevent can include at least one of advancing or retracting an optical fiber, changing operating parameters of the therapeutic energy, and increasing a level of irrigation proximate the field of view.

Images