Combination source algorithm model

Abstract

A system for identifying and classifying a target in an image obtained during a medical procedure may include a medical device and processing circuitry connected to the medical device. The processing circuit can be configured to receive an image of a surgical site and analyze the image of the surgical site to locate a target in the image. The processing circuitry can also cause light to be emitted from a light source associated with the medical device toward a target. At least a portion of the emitted light scattered by or reflected from the target can be collected at an optical splitter, and the collected light can be analyzed using a spectrometer coupled to the optical splitter to identify or classify the target. The processing circuitry can provide an output to the user based on at least one of the identification or classification of the target.

Description

[0001] Priority requirements

[0002] This application claims priority to U.S. Provisional Patent Application No. 63 / 515896, filed July 27, 2023, and U.S. Provisional Patent Application No. 63 / 652256, filed May 28, 2024, the contents of which are incorporated herein by reference. Technical Field

[0003] This disclosure relates to the characteristics of identifying a target during medical procedures. Background Technology

[0004] During medical procedures such as lithotripsy, users, such as doctors or robots, may interact with one or more targets of various types, sizes, or other characteristics (e.g., material composition). These targets may include stones (e.g., kidney stones or gallstones), tumors, tissue masses, etc., within the patient's body. The target may be located in a medium (e.g., a liquid medium such as water or saline), and the doctor may use medical devices such as endoscopes (e.g., surgical lasers) to perform surgery on the target.

[0005] The characteristics of a target may vary depending on the type of surgery and may change during the procedure. One or more characteristics of the target can influence how the medical procedure is initiated (e.g., what type of medical instruments are used, what type of laser is used, etc.), and conditions during the procedure (including changes in the target) can influence how the physician treats the target during the procedure. Summary of the Invention

[0006] When performing diagnostic or therapeutic procedures, it is necessary to know one or more characteristics of an anatomical target (e.g., a tumor or stone) within the patient's body. For example, a user attempting to visualize a cancerous tumor needs to determine the tumor's size (e.g., height and width) to determine if the tumor can be surgically removed, needs to establish baseline measurements for the tumor, and needs to monitor changes in the tumor's size after cancer treatment has begun, etc. Similarly, a user performing endoscopic procedures (e.g., stone or tissue ablation or removal) may want to determine the size of the stone or tissue mass to be removed in order to decide whether the target is small enough to be removed endoscopically (e.g., through a ureter or through a catheter sheath with a specific inner diameter), or whether the target should be further reduced in size before removal. During laser procedures such as laser lithotripsy, a physician may want to know the position of the endoscope within the patient's body. For example, the physician may want to know the distance between the tip of the endoscope and the target. The distance between the tip of the endoscope and the target helps determine, for example, the amount or intensity of ablation energy (e.g., laser energy, radiation, ultrasound energy, etc.) delivered to the target.

[0007] Furthermore, as the target changes during the procedure, it may be necessary to reposition the endoscope, or adjust (or stop) the intensity of the ablation energy altogether. For example, the composition of the target may change during the procedure (e.g., the material composition of a kidney stone may change as the stone is ablated), or the tissue contacted by the laser energy may change from the target tissue to a non-target tissue. Similarly, the physician may need to know when the ablation energy can contact non-target tissue or objects.

[0008] A model can be used to calculate, predict, or suggest how to perform or during surgery. For example, the model can be used to predict how much ablation energy (e.g., ablation energy from a laser or acoustic transducer) is needed or appropriate to ablate or reduce kidney stones. The model can be based at least in part on endoscopic video or images collected from an imaging device (e.g., a camera), or on spectral measurements of the target or spectral analysis of signals from the surgical scene. For example, the endoscopic video can be used to locate one or more targets in the surgical scene. Response signals from the target (e.g., light reflected from, scattered by, or otherwise emitted from the target (e.g., illumination in response to the target)) can then be collected by optical sensors or detectors and analyzed by a device such as a spectrometer to identify or classify the target. The accuracy or precision of the model can be improved or increased by using data or measurements of one or more parameters (e.g., temperature or pressure) collected by one or more sensors that may be included on or connected to a medical device (e.g., an endoscope).

[0009] A system for identifying and classifying a target in an image acquired during a medical procedure may include a medical device and processing circuitry. The processing circuitry may be configured, or can be configured, to receive an image of a surgical site and analyze the image of the surgical site to locate a target within the image. The processing circuitry may also cause an electromagnetic signal (e.g., light) to be emitted from a light source associated with (e.g., included on, connected to, or coupled to) the medical device toward the target. The electromagnetic signal may be an optical or visible wavelength signal (e.g., in the human visible light range between 100 nanometers (nm) and 1 millimeter (mm)) or a non-optical or invisible wavelength signal. The invisible signal may include ultraviolet light, infrared light, X-rays, microwaves, gamma rays, etc. At least a portion of the emitted light reflected back from or scattered by the target may be collected at an optical detector or other similar light collector and passed to an optical sensor (e.g., a spectrometer). The spectrometer may analyze the collected light to identify or classify the target. Based on at least one of the identifiers or classifications of the target, the processing circuit can provide output to a user (e.g., a doctor).

[0010] Therefore, during surgery, the system can use endoscopic images (e.g., images from a camera or other imaging sensor attached to the endoscope) combined with spectral analysis of the surgical scene to identify or classify targets by type. The endoscopic images can be used for large-scale observation of the target (e.g., stones) and provide a coarse estimate of the target's characteristics, while the spectral data can be used for small-scale observation of the target and provide more accurate target features.

[0011] The medical device may include an endoscope, and an image of the surgical site may be captured by an endoscopic camera. The processing circuitry may compare one or more colors in the image of the surgical site with one or more colors determined during spectral analysis of the surgical site. The processing circuitry may calibrate or otherwise adjust one or more colors in the image of the surgical site based on the spectral analysis of the surgical site to create a calibrated endoscopic image of the surgical site. The calibrated endoscopic image of the surgical site may be output to a graphical user interface, such as a computer screen or monitor, tablet computer, etc. The calibrated endoscopic image may be labeled to include, for example, information about the target, information about the surgical site, identification of non-target anatomical structures, information about changes in the target, or information about changes in the surgical site.

[0012] In one example, the processing circuitry may receive data from one or more additional sources, such as historical data relating to a patient undergoing the medical procedure, historical data relating to previous medical procedures performed by members of the demographic category to which the patient belongs, or real-time data obtained from sensors coupled to or connected to the medical device. This historical data may be retrieved from a database connected to the processing circuitry. The historical data relating to the patient may include information about previous medical procedures, pre-existing conditions of the patient, or physical characteristics of the patient. Sensors coupled to the medical device may include at least one of a temperature sensor, a pressure sensor, or a flow sensor.

[0013] The output may include warnings or recommendations. Warnings may include at least one of visual, auditory, or tactile warnings (e.g., when ablation energy is being applied to a non-target anatomical structure). Recommendations may include recommendations regarding subsequent medical procedures, such as the type of medical endoscope used in the subsequent medical procedure, the type and intensity of the laser used in the subsequent medical procedure, the type and amount of anesthesia used in the subsequent medical procedure, or the risks associated with the subsequent medical procedure.

[0014] The image of the surgical site can be analyzed using a first algorithm, and the identification or classification of the target can be performed using a second algorithm. The outputs from the first algorithm and the second algorithm can be used as inputs to a recommendation algorithm.

[0015] One potential advantage of using endoscopic images, along with spectral measurements (optical or non-spectral) and / or sensor data, is that targets can be identified more accurately and precisely, and adjustments can be made in real time during the procedure based on changes in the condition of the target or the surgical site. For example, when the target is a stone, knowing whether the stone is formed of soft or hard tissue will affect how the stone is ablated. This increased accuracy, and any adjustments made during the procedure, can lead to faster and more effective procedures, greater patient comfort post-operatively, reduced need for follow-up procedures, and provide useful information to the physician if follow-up procedures are required. Other examples of target characteristics that can determine how the current procedure should be performed or what follow-up procedures may be needed later may include: the ability to identify whether the target is the prostate capsule or an adenoma, or whether the target is cancerous (or potentially or possibly cancerous) or benign. Attached Figure Description

[0016] In the accompanying drawings (which are not necessarily drawn to scale), the same reference numerals may describe similar parts in different views. Similar reference numerals with different letter suffixes may indicate different instances of similar parts. These drawings illustrate, by way of example and not limitation, the various embodiments discussed in this document.

[0017] Figure 1 An example of a lithotripsy system including an optical detector is shown.

[0018] Figure 2 A flowchart illustrating the use of multiple algorithms to generate output based on multiple data inputs is provided.

[0019] Figure 3 An example is given of a method for identifying and classifying a target in an image during a medical procedure.

[0020] Figure 4 It is a block diagram of a machine example capable of performing any one or more of the techniques (e.g., methods) discussed in this article.

[0021] Figure 5 A schematic diagram of an exemplary computer-based clinical decision support system (CDSS) is shown. Detailed Implementation

[0022] A system that uses a combined source model to determine one or more target features may include an imaging device. This imaging device may be coupled to the distal end of a medical endoscope, such as an endoscope. The system may also include a light sensor or optical sensor, such as a spectrometer. The spectrometer may be used to detect and analyze signals from a target located at or near the distal end of the medical endoscope. The detected signals may include light reflected from or otherwise scattered or emitted from the target, such as electromagnetic signals (optical or non-optical) emitted in response to the target (e.g., illuminating the target with visible light, emitting infrared signals towards the target, etc.). The system may also include processing circuitry and a memory. The memory may include instructions that, when executed by the processing circuitry, enable the processing circuitry to identify or classify the target in a medical procedure. For example, in lithotripsy, the system may use images from a camera or other imaging sensor combined with spectral analysis of the surgical scene to identify or classify the target by type. In such an example, endoscopic images may be used for large-scale observation of the target (e.g., stones) and provide a coarse estimate of the target features, while spectral data may be used for small-scale observation of the target and provide more accurate target features. For example, the system can classify targets as tissue, stones, muscle tissue, endothelial tissue, or as the prostate capsule or prostate adenoma. Furthermore, the system can classify targets as normal or abnormal (e.g., cancerous). In one example, spectral analysis of endoscopic video and light signals from the surgical scene (including targets within the surgical scene) can be synchronized, allowing spectral analysis to be performed simultaneously (or substantially simultaneously) with the acquisition of endoscopic images.

[0023] Target identification and classification can be performed using one or more algorithms. Algorithms may include artificial intelligence (AI) or machine learning (ML) or other algorithms (e.g., non-AI or non-ML deterministic algorithms) or processes. Additionally or alternatively, hardware-based feedback loops or feedback control may be used as part of the target identification and / or classification or for recommendation. In one example, a single algorithm may be used to identify a target or classify that target (or determine the characteristics of the target). Alternatively, multiple algorithms may be used independently of each other. Their outputs may be fed to another model (e.g., a recommendation algorithm) to aggregate the outputs and make predictions or recommendations based on analysis by other algorithms. For example, imaging data from imaging sensors (e.g., endoscopic video or still frames, acoustic imaging, ultrasound imaging, etc.) may be fed to a first algorithm to detect or identify the target. Spectra captured from the light signal from the target (e.g., endoscopic light reflected back along a surgical fiber, fluorescence emitted from the target in response to illumination, etc.) may be fed to the same algorithm or a different second algorithm to help classify the target (e.g., determine whether the target is a stone or a tumor) or determine its characteristics (e.g., the material composition or size of the target). When the second algorithm classifies the target or determines its features, the results of the first and second algorithms can be passed to the third algorithm. The outputs of the first and second algorithms are then combined to make predictions, recommendations, or reminders to the user. For example, the third algorithm can make predictions about treatment locations, target predictions, and provide suggestions for future surgeries.

[0024] In one example, the model can apply weighting factors, allowing for different weighting, scoring, ranking, and prioritization of the outputs of the first and second algorithms. For instance, when the data passed from the imaging sensor to the first algorithm is a blurry endoscopic image, the output from the second algorithm (whose spectrum is captured from the light signal from the target and passed to this algorithm) can be weighted or ranked higher than the output of the first algorithm.

[0025] Predictions can be used to provide users with insights into the treatment site or to suggest optimal treatment configurations. These configurations may include recommendations for settings (or adjustments) of the endoscope or treatment devices used by the endoscope (e.g., lasers or other ablation components). For example, the system may recommend or suggest appropriate laser settings, or adjust the current laser settings (e.g., increasing, decreasing, or terminating laser radiation) for the ablation target. Additionally or alternatively, the system may recommend or suggest increasing or decreasing irrigation or aspiration at the surgical site. Therefore, rather than implementing the system to make sequential decisions from different algorithmic models, both endoscopic image data and spectral measurements can be passed to one or more algorithms, thus delivering a deterministic outcome (prediction or recommendation) at one or more steps.

[0026] In the example, the data input into the algorithm model may include one or more other measurements obtained from one or more additional sensors. These additional sensors may be located at or near the treatment site, or connected to the medical device, and may include temperature sensors, pressure sensors, flow sensors, etc. Therefore, examples of these other measurements may include temperature or pressure at the anatomical site, flushing flow rate, flushing temperature, aspiration flow rate, temperature of the aspirated fluid, etc.

[0027] Additionally or alternatively, other data or factors (e.g., anatomical or demographic information about the patient and / or medical procedure) may be passed to one or more algorithms and used to create predictions or recommendations. This additional data may include information about the patient. Patient information may include information about the medical procedure to be performed on the patient, such as the anatomical treatment site. Patient information may also include information such as the patient's age, sex, or ethnicity, or pre-existing medical conditions. Patient information may also include information about previous surgeries performed on the target anatomical structure, preoperative imaging data (e.g., from a CT scan diagnosing the patient with stones) or information derived therefrom (e.g., information about the number, size, location, etc. of stones), tissue biopsy results, previous stone composition results, etc. Other data may additionally include information about the medical devices used during the procedure (e.g., surgical configuration data), such as the use of catheter sheaths, the size of the laser fiber, the size of the endoscope working channel or insertion tube, the type of endoscope light source, or the age of the light source.

[0028] Some or all of these factors can influence or alter the algorithm model's recommendations regarding laser settings, irrigation fluid flow and temperature, fiber size, etc. The predictions, suggestions, or recommendations made by the algorithm can be adjusted as the type, composition, or condition of the target at the treatment site is identified or changes during the procedure, and can determine when the algorithm model recommends pausing laser emission or adjusting the position or other settings of the endoscope. In one example, the system can identify a blood vessel as a non-target and, in response, change one or more settings, such as stopping laser emission until the laser fiber moves away from the vessel. In another example, in situations with low visibility at the surgical area (e.g., caused by smoke, high concentrations of dust particles, etc.), the system can cause or recommend stopping laser emission and performing irrigation or aspiration to help clean the surgical area. Potential advantages of the systems described herein include more efficient treatment procedures, shorter treatment durations, and more effective therapeutic results. In this disclosure, ablation energy includes laser or acoustic ablation or any similar form of energy used to ablate a target or tissue, and terms such as laser, ablation, and acoustic energy can be used interchangeably.

[0029] Figure 1 An example of a lithotripsy system including an optical detector is illustrated. System 100 may include a surgical laser 102 and a graphical user interface 104 (GUI). The GUI 104 may include a touchscreen or other input mechanism (e.g., a button, switch, or other similar actuation member on an endoscope handle) configured to operate or control the surgical laser 102. The surgical laser 102 may include one or more laser sources configured to emit laser radiation. Figure 1 As shown in the dashed box in the example, the light source may include an ablation laser 106 and / or an illumination source 108. Illumination source 108 may include a probe laser, a light-emitting diode (LED), a xenon-based light source, or any similar visible light source. The ablation laser 106 may emit infrared radiation, while the illumination source 108 may emit an illumination beam, either a targeting beam or visible light, to indicate where the end of the endoscope is aimed (and therefore where the ablation energy from the ablation laser 106 is located). Alternatively, illumination source 108 may be used to illuminate a target 126. Target 126 may be a piece of tissue, fragment, or object, such as a kidney stone, tumor, prostate capsule, etc., to be ablated. The emitted light 128 from the ablation laser 106 or illumination source 108 may be emitted via an optical fiber 116 (e.g., connected to a surgical optical fiber 118 via an optical connector 120). In the example, the structure of surgical optical fiber 118 may be the same as or different from that of optical fiber 116. Surgical optical fiber 118 may be located wholly or partially outside the surgical laser 102. Therefore, the emitted light 128 can be emitted from the illumination source 108 through the optical fiber 116, optical connector 120, and surgical optical fiber 118 to the distal end of the surgical optical fiber 118. The distal end of the surgical optical fiber 118 can be inserted into a endoscope 124, such as an endoscope, ureteroscope, or laryngoscope. In this example, at least a portion of the emitted light 128 emitted from the distal end of the surgical optical fiber 118 and endoscope 124 can be reflected, scattered, etc. (reflected light 130) by the target 126 through a medium located between the end of the endoscope 124 and the target 126. The endoscope 124 may include an imaging device 122, such as a camera for capturing video or images of the surgical scene (including the target 126). The medium can be any medium, such as air, water, saline, carbon dioxide, etc.

[0030] The surgical laser 102 may also include or be coupled to an optical component (e.g., a beam splitter 110) configured to collect at least a portion of the aperture of the reflected light 130 passing through the surgical fiber 118. In an example, the beam splitter 110 may be replaced by a dedicated fiber configured to collect at least a portion of the reflected light 130. The portion of the reflected light 130 collected by the beam splitter 110 or the dedicated fiber may be sent to a processor 112 connected to or coupled to the surgical laser 102. An optical detector 132 (e.g., a spectrometer or other similar optical detector or sensor) may be located between the beam splitter 110 and the processor 112, allowing for spectral analysis of the reflected light 130 to determine one or more properties of the target 126.

[0031] Processor 112 and / or optical detector 132 can analyze a portion of the reflected light 130 collected by spectrometer 110 (or receive analysis by a spectrometer connected to or coupled to processor 112 and / or optical detector 132) to analyze the reflected light 130. Surgical laser 102 may optionally or additionally include controller 114 circuitry communicatively coupled to processor 112. Processor 112 or controller 114 can determine one or more characteristics of the reflected light 130 signal specific to illumination source 108, such as the type of light source, wavelength, optics, pulse width, optical configuration, or age of illumination source 108. Although Figure 1 Examples include a single illumination source; however, it should be understood that system 100 may include multiple illumination sources of different types (e.g., LED sources, xenon-based sources, broadband illumination sources, etc.). Furthermore, while illumination source 108 is shown as part or a component of surgical laser 102, the illumination source may be an external component of laser system 100 (e.g., external to surgical laser 102), such as by attachment to endoscope 124. It should also be noted that system 100 may include one or more other components, such as a fluid pump, suction unit, smoke evaluator, gas blower, or any other components that may be required or used during laser surgery.

[0032] Figure 2 A flowchart illustrating the use of multiple algorithms to generate output based on multiple data inputs is shown. For example... Figure 2As shown, data from multiple data sources can be used as input to multiple algorithms. These algorithms can be trained learning algorithms, untrained deterministic algorithms, or combinations thereof. For example, data from a first data source 200 can be input to a first algorithm 206. The first data source 200 may include data from an imaging sensor (e.g., a camera or video source capable of capturing video or still images). The data in the first data source 200 may include still frames or videos of surgical sites captured from cameras, medical devices, or medical endoscopes (e.g., endoscopes) included therein, or connected to them. The first data source 200 may also include red, green, and blue (RGB) intensities from an endoscopic camera. The first algorithm 206 can analyze the images or videos from the first data source 200 (e.g., image segmentation, etc.) to locate targets (or potential targets) in the images.

[0033] Data from the second data source 202 can be input into the second algorithm 208. The second data source 202 may include signals (e.g., light) collected from a target. For example, a light source (e.g., a xenon light source or any visible or invisible light source) included on or connected to the medical device may be emitted at a surgical scene (e.g., toward a target identified in an image from the second data source 202). Light reflected, scattered, etc., by the target may be collected by a beam splitter included in, connected to, or otherwise associated with the medical device.

[0034] Optical sensors, such as spectrometers, can analyze the spectrum of light collected from a target to identify or classify the target. In one example, spectral analysis can be used to confirm the identification of a potential target in an image from a first data source 200. Additionally or alternatively, spectral analysis can be used to determine the characteristics or composition of a target identified in an image from the first data source 200. For example, when a kidney stone is identified in an endoscopic image, light can be used to illuminate the stone. Based at least in part on the spectral analysis of the light reflected from the stone, the composition of the stone (e.g., whether it is formed from uric acid or calcium oxalate), the size of the stone, etc., can be determined by a second algorithm 208. In one example, the morphology of a target acquired in an endoscopic image can aid or complement spectral analysis. For example, the overall morphology or structure of a target, such as a stone, can better predict certain types of stones compared to spectral analysis alone. For example, uric acid stones typically exhibit smooth surface features, while calcitonite and calcium-based stones have fine, sharp protrusions (e.g., prominent crystalline structures). Therefore, this system can combine spectral analysis to use the morphology of the stone (e.g., surface features of the stone, such as whether the stone is smooth or textured, the size of the stone, etc.) to identify the type of stone, the composition of the stone, etc.

[0035] Data from third data source 204 can be input into third algorithm 210. Data from third data source 204 may include data from one or more sensors (e.g., temperature sensors, pressure sensors, flow sensors, etc.) included in or connected to a medical device. Additionally or alternatively, data from third data source 204 may include patient-specific data, such as data from a patient record. Data from a patient record may include data such as the patient's age, sex, weight, or other relevant physical characteristics. Data from a patient record may also include information about the patient's medical history, such as previous medical procedures, pre-existing conditions, allergies, blood type, blood antibodies, response to anesthesia, etc. Data from a patient record may also include clinical record reports, which may include information such as noticing narrowing, irregular anatomical tissue calcification, etc., that may require treatment or examination during or after a medical procedure. In one example, historical data may include data about people in the same or similar, or substantially the same or similar, demographic categories as the patient (e.g., people of similar age, sex, etc.) or people with similar medical conditions to the patient.

[0036] The outputs from the first algorithm 206, the second algorithm 208, and the third algorithm 210 can be used as input to the fourth algorithm 212. The fourth algorithm 212 can be a recommendation algorithm to analyze the outputs of the first algorithm 206, the second algorithm 208, and the third algorithm 210 to generate output 214. For example, based on endoscopic video or images, the first algorithm 206 can generate an endoscopic image of the surgical site with the identified target, and the second algorithm 208 can provide an analysis of the target based on spectral analysis (as described above). Based on the outputs from the first algorithm 206 and the second algorithm 208, the fourth algorithm 212 can compare the colors on the endoscopic image with the colors determined during spectral analysis and calibrate or adjust the colors on the endoscopic image to create a calibrated endoscopic image of the surgical site. This calibrated endoscopic image can represent a “true-color” image of the surgical site.

[0037] In one example, output 214 may include an annotated version of a calibrated endoscopic image of the surgical site. The annotated endoscopic image can identify targets in the endoscopic image and provide detailed information about the target, such as size or composition. In another example, the calibrated endoscopic image can indicate non-targets that should not be exposed to laser or other ablation energy, such as non-target tissue (e.g., muscle). The annotation may include information about changes in the target or surgical site. Changes in the target or surgical site can be determined by comparing a first image obtained earlier in the same procedure (e.g., a change in the size or composition of a kidney stone as it is ablated) with a later image. Alternatively, the change can be determined based on a first image obtained in an earlier, separate procedure and an image obtained in the current procedure. For example, the system can identify new targets that were not present in previous procedures during the current procedure, indicating an increase or decrease in mass size, etc. The annotated image can be displayed on a graphical user interface such as a monitor in the operating room.

[0038] Output 214 may include warnings or alarms. For example, during a laser ablation procedure, the fourth algorithm 212 may cause an alarm output when one of the algorithms determines that the composition of the target has changed, the laser is contacting non-target tissue, or any other situation that may affect whether laser ablation should continue (or continue at the current intensity) or whether the endoscope tip should be moved or repositioned. The alarm may be a visual alarm such as a flashing or changing color of the endoscope light, an audible alarm such as a beeping sound, or a tactile alarm such as a vibration of the endoscope handle. The alarm may also be a message displayed on a graphical user interface. In one example, output 214 may cause a controller (e.g., controller 114) to adjust or stop the ablation energy, or it may cause an actuator connected to the endoscope to reposition the endoscope tip based on the determination made by the fourth algorithm 212.

[0039] Figure 3 An example is provided for identifying and classifying a target in an image during a medical procedure. Method 300 may include multiple operations or steps. The operations described herein are merely examples, and the method may omit one or more of the listed operations, may repeat operations, may include other operations, or may perform operations simultaneously, substantially simultaneously, or in a different order as needed.

[0040] At 302, method 300 may include: receiving an image of the surgical site. This image may be captured from an imaging device, a sensor, or a device connected to the medical device (e.g., a camera located at or near the endoscope). At 304, method 300 may include: analyzing the image of the surgical site to locate one or more targets in the image of the surgical site. This analysis may be performed using a trained learning algorithm (e.g., artificial intelligence or machine learning algorithm) to identify potential targets at the surgical site, such as kidney stones. The analysis may include computer vision techniques such as image segmentation analysis or any similar techniques capable of object detection.

[0041] At 306, method 300 may include: emitting a signal (e.g., visible or invisible light, laser radiation, etc.) toward a target identified in the image received at 302. For example, an endoscope may include or be coupled to a visible light source, such as a xenon-based light source or any similar light source capable of illuminating the surgical scene.

[0042] At 308, method 300 may include: collecting at least a portion of a signal reflected or scattered by a target in response to a signal emitted at 306 (hereinafter referred to as a "response signal"), and step 310 may include: analyzing the collected response signal. Analysis of the response signal may include comparing one or more colors on an image of the surgical site with one or more colors determined during spectral analysis of the surgical site. In one example, when the scene is illuminated with illumination light, an optical sensor or optical detector (e.g., a spectrometer) may be used to analyze the spectrum or response signal of the illumination light. The light analyzed by the spectrometer may include light reflected or scattered by tissue, a target, or other objects in the surgical scene. The analysis may also include calibrating or adjusting one or more colors on an image of the surgical site based on or using spectral analysis of the surgical site to create a calibrated endoscopic image of the surgical site (e.g., a true-color image).

[0043] At 312, method 300 may include providing output. In one example, the output may include a warning, such as a visual, auditory, or tactile warning. The warning may be in response to a portion of the endoscope (e.g., the tip) being in an unwanted position or laser energy being directed toward non-target tissue. Additionally or alternatively, the output may include a prediction or recommendation. The recommendation may include recommendations regarding subsequent medical procedures. Such recommendations may include the type of medical endoscope used in the subsequent medical procedure, the type and intensity of ablation energy (e.g., laser radiation) used in the subsequent medical procedure, the type or amount of anesthesia used in the subsequent medical procedure, etc. The recommendation may also include information about the patient's risks during the subsequent medical procedure, such as the risk of decreased blood pressure, blood type antibody allergy, etc. The prediction may include whether the target is likely to become cancerous, etc.

[0044] Method 300 can be a computer-implemented method that uses or employs different algorithms to perform the operations of the method. For example, a first algorithm can be used to analyze an image of a surgical site, and a second algorithm can be used to identify or classify a target. In such an example, the output from the first algorithm (analyzed image of the surgical site) and the output from the second algorithm (identification or classification of the target) can be used as input to a recommendation algorithm. The recommendation algorithm can then produce an output at 312. These algorithms can include artificial intelligence (AI) or machine learning (ML) or other algorithms (e.g., non-AI or non-ML deterministic algorithms) or processes. Additionally or alternatively, portions of method 300 can be executed using hardware-based feedback loops or feedback control. Hardware-based feedback loops or feedback control, AI or ML algorithms, and non-AI or non-ML algorithms can be used individually or in combination as needed.

[0045] Figure 4 This is a block diagram illustrating examples of machines 400 that can be used to assist in performing one or more of the techniques (e.g., methods) discussed herein. Machine 400 may operate as a standalone device or may be connected (e.g., networked) to other machines. For example, machine 400 may be included in or connected to a medical device (e.g., endoscope, surgical laser, surgical fiber optics, etc.), or may be included on or connected to one or more sensors, or may include components of a system. Additionally or alternatively, machine 400 may operate one or more of the algorithms described above or as referred to below. Figure 5 The discussion pertains to a computer-based clinical decision support system (CDSS). In a networked deployment, machine 400 can operate as a server machine, a client machine, or both in a server-client network environment. In one example, machine 400 can act as a peer-to-peer (P2P) (or other distributed) network environment. Machine 400 can be a personal computer (PC), tablet computer, set-top box (STB), personal digital assistant (PDA), mobile phone, network device, network router, switch, or bridge, or any machine capable of executing instructions (sequentially or otherwise) specifying the actions to be taken by that machine. Furthermore, while only a single machine is illustrated, the term "machine" can include any collection of machines that individually or jointly execute a set (or more) of instructions to perform any one or more methods discussed herein, such as cloud computing, Software as a Service (SaaS), or other computer cluster configurations.

[0046] As described herein, examples may include logic or multiple components or mechanisms, or may be operated by logic or multiple components or mechanisms. A circuit set is a collection of circuits implemented in a tangible entity including hardware (e.g., simple circuits, gates, logic, etc.). Circuit set membership can change flexibly over time and with variations in the underlying hardware. A circuit set includes components that can perform a specified operation individually or in combination during operation. In one example, the hardware of a circuit set may be invariably designed to perform a specific operation (e.g., hardwired). In one example, the hardware of a circuit set may include physically connected components (e.g., execution units, transistors, simple circuits, etc.) that include physically modified computer-readable media (e.g., magnetic, electrical, movable placement of invariant aggregated particles, etc.) to encode instructions for a specific operation. When the physical components are connected, the underlying electrical characteristics of the hardware composition change, for example, from an insulator to a conductor, and vice versa. Instructions enable embedded hardware (e.g., execution units or loading mechanisms) to create members of the circuit set in the hardware via variable connections to perform specific operations of a portion during operation. Thus, when the device is operating, computer-readable media are communicatively coupled to other components of the circuit set components. In one example, any physical component can be used in multiple members of multiple circuit sets. For instance, under operation, an execution unit can be used in a first circuit of a first circuit set at one point in time, and reused at different times by a second circuit of the first circuit set or a third circuit of the second circuit set.

[0047] Machine 400 (e.g., a computer system) may include a hardware processor 402 (e.g., a central processing unit (CPU), graphics processing unit (GPU), hardware processor core, field-programmable gate array (FPGA), or any combination thereof), main memory 404, and static memory 406, some or all of which may communicate with each other via interconnect (e.g., bus) 430. Machine 400 may also include a display unit 410, an alphanumeric input device 412 (e.g., a keyboard), and a user interface (UI) navigation device 414 (e.g., a mouse). In one example, display unit 410, input device 412, and UI navigation device 414 may be a touch screen display. Machine 400 may additionally include a storage device 408 (e.g., a drive unit), a signal generation device 418 (e.g., a speaker), a network interface device 420, and one or more sensors 416 (e.g., a global positioning system (GPS) sensor, a compass, an accelerometer, or other sensors). Machine 400 may include an output controller 428, which may be connected serially (e.g., Universal Serial Bus (USB)), in parallel, or otherwise wired or wireless (e.g., infrared (IR), near field communication (NFC), etc.) to communicate with or control one or more peripheral devices (e.g., printers, card readers, etc.).

[0048] Storage device 408 may include machine-readable medium 422 (e.g., non-transitory medium) on which one or more sets of data structures or instructions 424 (e.g., software) embodying or used by one or more of any of the techniques or functions described herein are stored. During execution of instructions 424 by machine 400, instructions 424 may also reside wholly or at least partially in main memory 404, static memory 406, or hardware processor 402. In one example, one or any combination of hardware processor 402, main memory 404, static memory 406, or storage device 408 may constitute the machine-readable medium.

[0049] Although machine-readable medium 422 is exemplified as a single medium, the term "machine-readable medium" can include a single medium or multiple media (e.g., a centralized or distributed database and / or associated caches and servers) configured to store one or more instructions 424. The term "machine-readable medium" can include any non-transitory medium capable of storing, encoding, or carrying instructions executable by machine 400 and causing machine 400 to perform any or more of the techniques disclosed herein, or capable of storing, encoding, or carrying data structures used by or associated with those instructions. Examples of non-limiting machine-readable media can include solid-state memory as well as optical and magnetic media. In one example, a mass machine-readable medium includes a machine-readable medium having multiple particles with invariant (e.g., rest) mass. Therefore, a mass machine-readable medium is not a transient propagating signal. Specific examples of high-capacity machine-readable media may include: non-volatile memory, such as semiconductor storage devices (e.g., electrically programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM)) and flash memory devices; magnetic disks, such as internal hard disks and removable hard disks; magneto-optical disks; and CD-ROMs and DVD-ROMs.

[0050] Figure 5A schematic diagram of an exemplary computer-based clinical decision support system (CDSS) 500 is illustrated, which is configured to locate a target in an image and identify or classify the target based on spectral analysis of the image. The CDSS 500 can also be configured to output alerts, recommendations, or predictions. The recommendations may include recommended changes to settings (e.g., changed laser settings, changed endoscope position, etc.) or recommendations for subsequent medical procedures. Recommendations for subsequent medical procedures or changes to settings may be based on information about: the target being treated, the tissue block or portion being treated, patient information, circumstances encountered during the medical procedure, or a combination thereof. In various implementations, CDSS 500 may include: an input interface 502 through which one or more inputs, such as images of surgical sites specific to a patient, are provided as input features to an artificial intelligence (AI) model 504; a processor, such as processor 402, capable of performing inference operations, wherein information about the target, tissue block or portion, patient, medical procedure, or any other appropriate or desired input is applied to the AI ​​model to generate warnings, recommendations, or predictions; and a user interface (UI) through which warnings, recommendations, or predictions can be communicated to a user, such as a clinician.

[0051] In some implementations, input interface 502 may be a direct data link between CDSS 500 and one or more medical devices or sensors that generate at least some of the input features. For example, during treatment and / or diagnostic medical procedures, input interface 502 may directly transmit information about a target (e.g., the composition or density of the target) to CDSS 500. As described above, spectral analysis of the signal from the target can be used to determine information about the target. Additionally or alternatively, input interface 502 may be a classic user interface that facilitates interaction between a user and CDSS 500. For example, input interface 502 may facilitate a user interface through which a user can manually input information about the patient or about the medical procedure. Additionally or alternatively, input interface 502 may provide CDSS 500 with access to an electronic patient record from which one or more input features can be extracted. This patient record may be stored in a database 506 connected to CDSS 500. In any of these cases, input interface 502 may be configured to collect one or more of the following input features associated with a particular patient when or before evaluation using CDSS 500: An image 510 of the surgical site, which can be obtained from an imaging sensor (e.g., an endoscopic camera), and may include information about the target location, the target size, or non-target tissue; 512 Information from spectral analysis of the surgical site. This information may include target features and can be used in conjunction with target images captured by the imaging sensor of the endoscope; Environmental conditions measured by one or more sensors at or near the endoscope; The location of the laser fiber or endoscope; Information about medical procedures; Instructions from non-target organizations; Is the target or organization normal or abnormal? Information about the patient, such as medical history; and / or Information about the patient's condition during medical procedures.

[0052] Based on one or more of the aforementioned input features, processor 402 can use an AI model to perform inference operations to generate output. This output may include alarms or warnings. It may include changes to one or more laser settings to be implemented or recommended changes to one or more laser settings suggested to the user. It may include recommendations for subsequent medical procedures. The AI ​​model can provide a computer system with the ability to perform tasks by inferring based on patterns discovered in data analysis without explicit programming. The AI ​​model can explore the research and construction of algorithms (e.g., machine learning algorithms) that can learn from existing data and make predictions on new data. Such algorithms operate by building AI models from example training data to make data-driven predictions or decisions, manifested as outputs or evaluations.

[0053] Two typical paradigms for machine learning (ML) include supervised ML and unsupervised ML. Supervised ML uses prior knowledge (e.g., examples that relate inputs to outputs or outcomes) to learn the relationship between inputs and outputs. The goal of supervised ML is to learn a function that, given some training data, best approximates the relationship between the training inputs and outputs, so that the ML model can achieve the same relationship to generate the corresponding output given the inputs. Unsupervised ML is the training of an ML algorithm using information that is neither classified nor labeled, allowing the algorithm to act on this information without guidance. Unsupervised ML is useful in exploratory analytics because it can automatically identify structures in the data.

[0054] Some tasks used for supervised machine learning can include classification and regression problems. Classification problems (also known as categorization problems) aim to classify an item into one of several category values ​​(e.g., is this object an apple or an orange). Regression algorithms can be designed to quantify some items (e.g., by assigning scores to some input values). Some examples of supervised ML algorithms include logistic regression (LR), Naive Bayes, random forest (RF), neural networks (NN), deep neural networks (DNN), matrix factorization, and support vector machines (SVM).

[0055] Some tasks in unsupervised machine learning (ML) can include clustering, representation learning, and density estimation. Examples of unsupervised ML algorithms include K-means clustering, principal component analysis, and autoencoders.

[0056] Another type of machine learning is federated learning (also known as collaborative learning), which trains algorithms on multiple distributed devices holding local data without exchanging data. This approach contrasts sharply with centralized machine learning techniques (where all local datasets are uploaded to a single server) and more classic decentralized methods (which typically assume that local data samples are distributed similarly). Federated learning enables multiple participants to build a general, robust machine learning model without sharing data, thus allowing for the resolution of key issues such as data privacy, data security, data access permissions, and access to heterogeneous data.

[0057] In some examples, the AI ​​model can be trained continuously or periodically before the inference operation is performed by the processor 402. Then, during the inference operation, patient-specific input features provided to the AI ​​model can propagate from the input layer through one or more hidden layers and ultimately to the output layer corresponding to one or more changes in laser settings, one or more predictions or recommendations, target identification and / or classification, etc. For example, based on target features, CDSS500 can identify a target as a kidney stone formed of a specific material (e.g., uric acid). Based on the stone's identification and classification, CDSS can recommend settings such as the laser ablation energy source. Throughout the procedure, the characteristics of the stone may change (e.g., changes in size, changes in material composition, etc.). Based on the characteristics of the stone and the determined changes in target features (e.g., changes in stone composition), CDSS500 can recommend changes in laser settings such as laser intensity and / or frequency, changes in irrigation and / or aspiration settings, or make any other recommendations related to the procedure that can be propagated to the output layer. In another example, CDSS can make predictions such as the time required to complete the procedure or predictions that the temperature or pressure at the treatment site may increase over a certain period of time. Additionally or alternatively, the system can make recommendations for subsequent medical procedures. Such recommendations can be based at least in part on the conditions or events during the current medical procedure. For example, if a patient has an adverse reaction to an anesthetic used during surgery, the system can recommend using a different type of anesthetic in subsequent procedures.

[0058] During or after the reasoning operation, the recommendation may be communicated to the user via a user interface (UI) and / or the processor 402 may automatically adjust the settings of the medical device (e.g., laser settings, flushing flow rate, aspiration rate, etc.).

[0059] Additional notes and examples: Example 1 is a system for identifying and classifying a target in an image acquired during a medical procedure. The system includes a medical device and processing circuitry connected to the medical device. The processing circuitry is configured to: receive an image of a surgical site; analyze the image of the surgical site to locate a target in the image of the surgical site; emit light from a light source associated with the medical device toward the target; collect at least a portion of the emitted light reflected back from or scattered by the target at a beam splitter associated with the medical device; analyze the collected light using an optical sensor coupled to the beam splitter to perform at least one of the following: identifying or classifying the target; and provide output to a user based on at least one of the identification or classification of the target.

[0060] In Example 2, the subject matter of Example 1 may optionally include the following: the medical device is an endoscope, and an image of the surgical site is captured by an endoscopic camera; and the processing circuitry is configured to: compare one or more colors in the image of the surgical site with one or more colors determined during spectral analysis of the surgical site; and calibrate or otherwise adjust the one or more colors in the image of the surgical site based on the spectral analysis of the surgical site to create a calibrated endoscopic image of the surgical site.

[0061] In Example 3, the subject of Example 2 may optionally include the following subject, wherein the processing circuitry is used to: output calibrated endoscopic images of the surgical site on a graphical user interface.

[0062] In Example 4, the subject matter of any one or more of Examples 2 to 3 may optionally include the subject matter of annotating a calibrated endoscopic image of the surgical site.

[0063] In Example 5, the subject of Example 4 may optionally include the following subject, wherein annotating the calibrated endoscopic image of the surgical site includes: annotating the calibrated endoscopic image of the surgical site with at least one of the following: information about the target, information about the surgical site, identification of non-target anatomical structures, information about changes in the target, or information about changes in the surgical site.

[0064] In Example 6, the topic of any one or more of Examples 1 to 5 may optionally include the topic of using a first algorithm to analyze the image of the surgical site and using a second algorithm to identify or classify the target, wherein the output from the first algorithm and the output from the second algorithm are used as input to the recommendation algorithm.

[0065] In Example 7, the topic of Example 6 may optionally include the following topic, wherein the processing circuitry is configured to: receive data from one or more additional sources; and input the data from the one or more additional sources into the recommendation algorithm, wherein the output is based at least in part on the analysis of the data from the one or more additional sources.

[0066] In Example 8, the subject of Example 7 may optionally include the subject of the processing circuitry being configured to induce regulation of the medical device based at least in part on one or more of the data from the one or more additional sources or the identification or classification of the target.

[0067] In Example 9, the subject matter of any one or more of Examples 7 to 8 may optionally include the subject matter of the data from the one or more additional sources including historical data relating to a patient undergoing the medical procedure, historical data relating to previous medical procedures performed by members of the demographic category to which the patient belongs, or real-time data obtained from sensors coupled to or connected to the medical device.

[0068] In Example 10, the subject matter of Example 9 may optionally include the following: the historical data relating to the patient includes information about previous medical procedures, pre-existing diseases of the patient, or physical characteristics of the patient.

[0069] In Example 11, the subject matter of any one or more of Examples 1 to 10 may optionally include the following subject matter, wherein the output includes recommendations, wherein the recommendations include recommendations regarding subsequent medical procedures, and wherein the recommendations regarding subsequent medical procedures include one or more of the following: the type of medical endoscope used during the subsequent medical procedure; the type and intensity of ablation energy used during the subsequent medical procedure; the type and amount of anesthesia used during the subsequent medical procedure; or the risks during the subsequent medical procedure.

[0070] In Example 12, the subject matter of any one or more of Examples 1 to 11 may optionally include the subject matter of: determining one or more morphological features of the target; and using the determined one or more morphological features, in conjunction with the analysis of the collected light by the optical sensor, to perform at least one of the following: identifying the target or classifying the target.

[0071] Example 13 is a computer-implemented method for identifying and classifying a target in an image acquired during a medical procedure. The method includes: receiving an image of a surgical site from an optical sensor connected to a medical device used during the procedure; analyzing the image using a first algorithm; emitting a signal from a light source connected to the medical device toward the target; collecting at least a portion of a response signal reflected back from or scattered by the target in response to the emitted signal at a beam splitter connected to the medical device; analyzing the collected response signal using an optical sensor coupled to the beam splitter to perform at least one of the following: identifying or classifying the target; and providing output to a user based at least in part on the identification or classification of the target.

[0072] In Example 14, the subject matter of Example 13 may optionally include the following: comparing one or more colors on an image of the surgical site with one or more colors determined during spectral analysis of the surgical site; and calibrating or otherwise adjusting the one or more colors on an image of the surgical site based on the spectral analysis of the surgical site to create a calibrated endoscopic image of the surgical site.

[0073] In Example 15, the subject of Example 14 may optionally include the subject of annotating the calibrated endoscopic image of the surgical site, wherein annotating the calibrated endoscopic image of the surgical site includes: annotating the calibrated endoscopic image of the surgical site with at least one of information about the target, information about the surgical site, identification of non-target anatomical structures, information about changes in the target, or information about changes in the surgical site; and outputting the calibrated endoscopic image of the surgical site on a graphical user interface.

[0074] In Example 16, the subject matter of any one or more of Examples 13 to 15 may optionally include the subject matter of using a first algorithm to analyze an image of the surgical site and using a second algorithm to identify or classify the target, wherein the outputs from the first algorithm and the second algorithm are used as inputs to a recommendation algorithm, and wherein the method further includes: receiving data from one or more additional sources, wherein the data from the one or more additional sources includes historical data relating to a patient undergoing the medical procedure, historical data relating to previous medical procedures performed by members of the demographic category to which the patient belongs, or real-time data obtained from sensors coupled to or connected to the medical device; inputting the data from the one or more additional sources into the recommendation algorithm, and wherein the output is based at least in part on the analysis of the data from the one or more additional sources; and causing the medical device to adjust at least in part based on one or more of the data from the one or more additional sources or the identification or classification of the target.

[0075] Example 17 is a non-transitory computer-readable medium having instructions stored thereon that, when executed by a processor of a computing device connected to a medical device, cause the processor to: receive an image of a surgical site; analyze the image of the surgical site to locate a target in the image of the surgical site; emit light from a light source associated with the medical device toward the target; collect at least a portion of the emitted light reflected back from or scattered by the target at a beam splitter associated with the medical device; analyze the collected light using an optical sensor coupled to the beam splitter to perform at least one of: identifying or classifying the target; and providing output to a user based on at least one of the identification or classification of the target.

[0076] In Example 18, the subject of Example 17 may optionally include the following: wherein the instructions cause the processor to perform the following operations: compare one or more colors on an image of the surgical site with one or more colors determined during spectral analysis of the surgical site; calibrate or otherwise adjust the one or more colors on the image of the surgical site based on the spectral analysis of the surgical site to create a calibrated endoscopic image of the surgical site; annotate the calibrated endoscopic image of the surgical site; output the calibrated endoscopic image of the surgical site on a graphical user interface; receive data from one or more additional sources; and input the data from the one or more additional sources into a recommendation algorithm, wherein the output is at least partially based on the analysis of the data from the one or more additional sources.

[0077] In Example 19, the topic of Example 18 may optionally include the following: a first algorithm is used to analyze the image of the surgical site, and a second algorithm is used to identify or classify the target, wherein the outputs from the first algorithm and the second algorithm are used as inputs to the recommendation algorithm.

[0078] In Example 20, the subject of Example 19 may optionally include the following: the output includes recommendations regarding subsequent medical procedures, and the recommendations regarding subsequent medical procedures include one or more of the following: the type of medical endoscope used during the subsequent medical procedure; the type and intensity of ablation energy used during the subsequent medical procedure; the type and amount of anesthesia used during the subsequent medical procedure; or the risks during the subsequent medical procedure.

[0079] The above detailed description includes reference to the accompanying drawings, which form part of the detailed description. The drawings illustrate specific embodiments that can be practiced by way of illustration. These embodiments are also referred to herein as "examples". These examples may include elements other than those shown or described. However, the inventors also contemplate examples in which only the shown or described elements are provided. Furthermore, the inventors contemplate examples of any combination or substitution of those elements (or one or more aspects thereof) shown or described, whether with respect to a particular example (or one or more aspects thereof) or to other examples (or one or more aspects thereof) shown or described herein.

[0080] All publications, patents, and patent documents cited in this document are incorporated herein by reference in their entirety, as if they were cited individually. In the event of any inconsistency between the usage herein and that of the cited references, the usage in the cited references shall be considered supplementary to the usage herein; in the case of irreconcilable inconsistencies, the usage herein shall prevail.

[0081] In this document, the terms “a” or “an” are used as commonly in patent literature to include one or more and are independent of any other instances or uses of “at least one” or “one or more”. In this document, unless otherwise stated, the term “or” is used to refer to non-exclusivity, such that “A or B” includes “a but not B”, “B but not A”, and “A and B”. In this document, unless otherwise stated, the term “and / or” is used to refer to non-exclusivity, such that “A and / or B” includes “A but not B”, “B but not A”, and “A and B”. In the appended claims, the terms “comprising” and “wherein” are used as concise English equivalents of the corresponding terms “comprising” and “wherein”. Furthermore, in the following claims, the terms “including” and “comprising” are open-ended, meaning that a system, apparatus, article, or process that includes other elements besides those listed in the claims following such terms is still considered to fall within the scope of the claim. Furthermore, in the following claims, the terms “first,” “second,” and “third,” etc., are used merely as labels and are not intended to impose numerical requirements on their objects.

[0082] The above description is illustrative and not restrictive. For example, the above examples (or one or more aspects thereof) may be used in combination with each other. Other embodiments may be used, for example, as may be used by one of ordinary skill in the art upon reading the above description. This abstract is intended to allow the reader to quickly determine the nature of the technical disclosure and to understand at the time of submission that it will not be used to interpret or limit the scope or meaning of the claims. Furthermore, in the above detailed description, various features may be combined to simplify this disclosure. This should not be construed as meaning that any unclaimed disclosed feature is essential to any claim. Rather, the subject matter of the invention may be less than all the features of a particular disclosed embodiment. Therefore, the following claims are hereby incorporated into the detailed specification, wherein each claim stands independently as a separate embodiment. The scope of the embodiment should be determined by reference to the appended claims and the full scope of their equivalents.

Claims

1. A system for identifying and classifying targets in images acquired during medical surgery, the system comprising: Medical devices; as well as A processing circuit, connected to the medical device, is used for: Receive images of the surgical site; Analyze the image of the surgical site to locate the target in the image of the surgical site; Light is emitted from a light source associated with the medical device toward the target; At a beam splitter associated with the medical device, at least a portion of the emitted light reflected back from or scattered by the target is collected. The collected light is analyzed using an optical sensor coupled to the beam splitter to perform at least one of the following: identifying the target or classifying the target; and Output is provided to the user based on at least one of the identifier or the classification of the target.

2. The system according to claim 1, wherein, The medical device is an endoscope, and the image of the surgical site is captured by an endoscopic camera, wherein the processing circuitry is used for: The image of the surgical site is compared with one or more colors determined during spectral analysis of the surgical site; and The colors of one or more on the image of the surgical site are calibrated or otherwise adjusted based on spectral analysis of the surgical site to create a calibrated endoscopic image of the surgical site.

3. The system according to claim 2, wherein, The processing circuit is used for: The calibrated endoscopic image of the surgical site is output on the graphical user interface.

4. The system according to claim 2, wherein, The processing circuit is used for: The calibrated endoscopic images of the surgical site are labeled.

5. The system according to claim 4, wherein, Annotating the calibrated endoscopic image of the surgical site includes: annotating the calibrated endoscopic image of the surgical site with at least one of the following: information about the target, information about the surgical site, identification of non-target anatomical structures, information about changes in the target, or information about changes in the surgical site.

6. The system according to claim 1, wherein, A first algorithm is used to analyze the image of the surgical site, and a second algorithm is used to identify or classify the target, wherein the outputs from the first algorithm and the second algorithm are used as inputs to a recommendation algorithm.

7. The system according to claim 6, wherein, The processing circuit is used for: Receive data from one or more additional sources; and The data from the one or more additional sources is input into the recommendation algorithm, and the output is based at least in part on the analysis of the data from the one or more additional sources.

8. The system according to claim 7, wherein, The processing circuit is used for: The adjustment of the medical device is caused at least in part based on one or more of the data from the one or more additional sources or the identifier or classification of the target.

9. The system according to claim 7, wherein, The data from the one or more additional sources includes historical data relating to a patient undergoing the medical procedure, historical data relating to previous medical procedures performed by members of the demographic category to which the patient belongs, or real-time data obtained from sensors coupled to or connected to the medical device.

10. The system according to claim 9, wherein, The historical data related to the patient includes information about previous medical procedures, pre-existing conditions of the patient, or the patient's physical characteristics.

11. The system according to claim 1, wherein, The output includes recommendations, wherein the recommendations include recommendations regarding subsequent medical procedures, and wherein the recommendations regarding subsequent medical procedures include one or more of the following: the type of medical endoscope used during the subsequent medical procedure; the type and intensity of ablation energy used during the subsequent medical procedure; the type and amount of anesthesia used during the subsequent medical procedure; or the risks during the subsequent medical procedure.

12. The system according to claim 1, wherein, The processing circuit is also used for: Determine one or more morphological features of the target; and Using one or more determined morphological features, combined with the analysis of the collected light by the optical sensor, at least one of the following is performed: identifying the target or classifying the target.

13. A computer-implemented method for identifying targets in images acquired during medical surgery and classifying the targets, the computer-implemented method comprising: Images of the surgical site are received from an optical sensor connected to a medical device used during the medical procedure. The image is analyzed using the first algorithm; A signal is emitted from a light source connected to the medical device toward the target; At the beam splitter connected to the medical device, at least a portion of the response signal reflected back from or scattered by the target in response to the emitted signal is collected; The collected response signal is analyzed using an optical sensor coupled to the beam splitter to perform at least one of the following: identifying the target or classifying the target; and Output is provided to the user based at least in part on the identifier or classification of the target.

14. The method according to claim 13, wherein the method comprises: Compare one or more colors on the image of the surgical site with one or more colors determined during spectral analysis of the surgical site; as well as Based on spectral analysis of the surgical site, one or more colors on the image of the surgical site are calibrated or otherwise adjusted to create a calibrated endoscopic image of the surgical site.

15. The method of claim 14, wherein the method comprises: The calibration endoscopic image of the surgical site is annotated, wherein the annotation of the calibration endoscopic image of the surgical site includes annotating the calibration endoscopic image of the surgical site with at least one of the following: information about the target, information about the surgical site, identification of non-target anatomical structures, information about changes in the target, or information about changes in the surgical site; and The calibrated endoscopic image of the surgical site is output on the graphical user interface.

16. The method according to claim 13, wherein, The method uses a first algorithm to analyze the image of the surgical site and a second algorithm to identify or classify the target, wherein the outputs from the first algorithm and the second algorithm are used as inputs to a recommendation algorithm, and wherein the method further includes: Receive data from one or more additional sources, wherein the data from the one or more additional sources includes historical data related to a patient undergoing the medical procedure, historical data related to previous medical procedures performed by members of the demographic category to which the patient belongs, or real-time data obtained from sensors coupled to or connected to the medical device; The data from the one or more additional sources is input into the recommendation algorithm, and the output is based at least in part on the analysis of the data from the one or more additional sources; and The adjustment of the medical device is caused at least in part based on one or more of the data from the one or more additional sources or the identifier or classification of the target.

17. A non-transitory computer-readable medium storing instructions that, when executed by a processor of a computing device connected to a medical device, cause the processor to perform the following operations: Receive images of the surgical site; Analyze the image of the surgical site to locate the target in the image of the surgical site; Light is emitted from a light source associated with the medical device toward the target; At a beam splitter associated with the medical device, at least a portion of the emitted light reflected back from or scattered by the target is collected. The collected light is analyzed using an optical sensor coupled to the beam splitter to perform at least one of the following: identifying the target or classifying the target; and Output is provided to the user based on at least one of the identifiers or classifications of the target.

18. The non-transitory computer-readable medium according to claim 17, wherein, The instruction causes the processor to perform the following operations: Compare one or more colors on the image of the surgical site with one or more colors determined during spectral analysis of the surgical site; Based on spectral analysis of the surgical site, the one or more colors on the image of the surgical site are calibrated or otherwise adjusted to create a calibrated endoscopic image of the surgical site; The calibrated endoscopic images of the surgical site are labeled; Output the calibrated endoscopic image of the surgical site on the graphical user interface; Receive data from one or more additional sources; and Data from the one or more additional sources is input into the recommendation algorithm, and the output is based at least in part on the analysis of the data from the one or more additional sources.

19. The non-transitory computer-readable medium according to claim 18, wherein, A first algorithm is used to analyze the image of the surgical site, and a second algorithm is used to identify or classify the target, wherein the outputs from the first algorithm and the second algorithm are used as inputs to a recommendation algorithm.

20. The non-transitory computer-readable medium according to claim 19, wherein, The output includes recommendations regarding subsequent medical procedures, wherein the recommendations regarding subsequent medical procedures include one or more of the following: the type of medical endoscope used during the subsequent medical procedure; the type and intensity of ablation energy used during the subsequent medical procedure; the type and amount of anesthesia used during the subsequent medical procedure; or the risks during the subsequent medical procedure.

Images