ALGORITHM MODELS FOR UNITED SOURCES

Abstract

A system for identifying and classifying a target in an image acquired during a medical procedure may comprise a medical device and a processing circuit associated with the medical device. The processing circuit may be configured to receive an image of a surgical site and analyze the image to locate a target within the image. The processing circuit may further cause light to be emitted from a light source associated with the medical device toward the target. At least a portion of the emitted light, which is scattered or reflected by the target, may be collected at an optical splitter, and the collected light may be analyzed using a spectrometer coupled to the optical splitter to identify or classify the target.The processing circuit can provide a user with an output based on at least one of the identification or classification of the target.

Description

PRIORITY CLAIM

[0001] This application claims priority over the preliminary US patent application with serial number 63 / 515,896, filed on July 27, 2023, and the preliminary US patent application with serial number 63 / 652,256, filed on May 28, 2024, the contents of which are hereby incorporated by reference. TECHNICAL FIELD

[0002] The present disclosure relates to the determination of a property of a target during a medical procedure. BACKGROUND

[0003] During a medical procedure, such as lithotripsy, a user, such as a physician or a robot, can interact with one or more targets of varying types, sizes, or other properties, such as material composition. The target might be a stone (for example, a kidney stone or gallstone), a tumor, a piece of tissue, or the like within a patient's body. The target object might be in a medium (e.g., a liquid such as water or saline), and the physician might use a medical device (such as an endoscope) or a surgical laser to perform surgical procedures on the target objects.

[0004] The characteristics of the target can vary depending on the type of procedure and may change during the procedure. One or more characteristics of the target may affect how the medical procedure is initiated (e.g., what type of medical instruments are used, what types of lasers are used, etc.), and the conditions during the procedure, including changes in the target, may affect how a physician proceeds during the procedure. SUMMARY

[0005] When performing a diagnostic or therapeutic procedure, it is desirable to know one or more properties of anatomical targets within a patient, such as a tumor or a stone. For example, a user who wants to visualize a cancerous tumor needs to determine the tumor's dimensions (e.g., height and width) to determine whether the tumor can be surgically removed, to establish a baseline for the tumor, to monitor changes in tumor size after the start of cancer treatment, or similar purposes. Similarly, a user performing an endoscopic procedure such as ablation or removal of stones or tissue might want to determine the size of the stone or tissue to be removed, for example, to decide whether the target object is small enough to be removed with the endoscope.through the ureter or through an access sheath with a specific inner diameter, or whether the target object should be further reduced in size before removal. During laser procedures such as laser lithotripsy, the physician may need to know the position of the endoscope within the patient's body. For example, the physician may need to know the distance between the tip of the endoscope and the target object. This distance is helpful in determining, for example, the amount or intensity of ablation energy (e.g., laser energy, radiation, ultrasound energy, or the like) to be delivered to the target object.

[0006] Furthermore, it may be necessary to reposition the endoscope and adjust (or stop entirely) the intensity of the ablation energy if the targets change during a procedure. For example, the target composition may change during the procedure (e.g., the material composition of a kidney stone may change during ablation), or the tissue touched by the laser energy may change from target tissue to non-target tissue. Similarly, a physician may need to be vigilant about when the ablation energy touches non-target tissue or objects.

[0007] A model can be used to calculate, predict, or suggest how a surgical procedure should be performed or how to proceed during the procedure. For example, the model can be used to predict how much ablation energy, such as from a laser or acoustic transducer, is required or appropriate to ablate or reduce the size of a kidney stone. The model can base the prediction, at least in part, on endoscopic videos or images acquired by an imaging device such as a camera, optical spectrum measurements of the target, or spectral analyses of signals from the surgical field. For example, the endoscopic video can be used to locate one or more targets in a surgical scene.A response signal from a target, such as light reflected, scattered, or otherwise emitted by the target (e.g., in response to illumination), can then be detected by an optical sensor or light detector and analyzed by a device such as a spectrometer to identify or classify the target. The accuracy or precision of the model can be improved or enhanced by using data or measurements of one or more parameters, such as temperature or pressure, at the surgical site, acquired by one or more sensors attached to or connected to a medical device, such as an endoscope.

[0008] A system for identifying and classifying a target in an image acquired during a medical procedure may comprise a medical device and a processing circuit. The processing circuit may be configured or configurable to receive an image of a surgical site and analyze the image of the surgical site to locate a target within the image. The processing circuit may further cause an electromagnetic signal, such as light, to be emitted from a light source connected to the medical device (e.g., contained within, attached to, coupled to, or the like) in the direction of the target. The electromagnetic signal may be a signal with an optical or visible wavelength (e.g., wavelength 0.80 ...The signal can be in the visible range for humans, between 100 nanometers (nm) and 1 millimeter (mm), or it can have a non-optical or non-visible wavelength. The non-visible signal can include ultraviolet light, infrared light, X-rays, microwaves, gamma rays, or the like. At least some of the emitted light, reflected or scattered by the target, can be collected by an optical detector or other similar light collector and directed to an optical sensor, such as a spectrometer. The spectrometer can analyze the collected light to at least identify or classify the target. Based on at least the identification or classification of the target, the processing circuitry can provide an output to a user, such as a physician.

[0009] During a procedure, the system can use an endoscopic image (e.g., an image from a camera or other imaging sensor located on or connected to the endoscope) in conjunction with a spectral analysis of the surgical scene to identify a target or to classify or categorize the target by type. The endoscopic image(s) can be used to view the target (e.g., a stone) on a large scale and provide a rough estimate of its properties, while the spectroscopic data can be used to view the target on a small scale and provide more precise information about its properties.

[0010] The medical device may include an endoscope, and the image of the surgical site may be captured with an endoscope camera. The processing circuitry may compare one or more colors in the image of the surgical site with one or more colors determined during spectral analysis of the surgical site. Based on this spectral analysis, the processing circuitry may calibrate or otherwise adjust one or more colors in the image of the surgical site to produce a calibrated endoscopic image of the surgical site. The calibrated endoscopic image of the surgical site may be displayed on a graphical user interface such as a computer screen or monitor, a tablet, or similar device.The calibrated endoscopic image can be annotated to include, for example, information about the target, information about the surgical site, identification of non-target anatomy, information about a change in the target, or information about a change in the surgical site.

[0011] In one example, the processing circuit can receive data from one or more additional sources, such as historical data about a patient undergoing the medical procedure, historical data about a previous medical procedure undergone by members of a demographic group to which the patient belongs, or real-time data obtained from a sensor coupled or connected to the medical device. The historical data can be retrieved from a database connected to the processing circuit. The patient's historical data can include information about a previous medical intervention, a pre-existing condition of the patient, or a physical characteristic of the patient. The sensor coupled to the medical device can include at least a temperature sensor, a pressure sensor, or a flow sensor.The output may include a warning or a recommendation. The warning may include at least a visual, audible, or haptic warning, for example, if the ablation energy comes into contact with a structure not part of the target anatomy. The recommendation may include a recommendation regarding a subsequent medical procedure, such as a type of medical endoscope to use during the subsequent procedure, a type and intensity of laser to use during the subsequent procedure, a type and amount of anesthesia to use during the subsequent procedure, or a risk during the subsequent procedure.

[0012] The image of the surgical site can be analyzed using a first algorithm, and the target can be identified or classified using a second algorithm. The output of the first algorithm and the output of the second algorithm can be used as inputs for a recommendation algorithm.

[0013] A potential advantage of using endoscopic images in conjunction with spectral measurements (optical or non-optical) and / or sensor data is that targets can be identified more accurately and precisely, and adjustments can be made in real time based on changes in the target or conditions at the surgical site during the procedure. For example, if the target is a stone, knowing whether the stone is composed of soft or hard tissue can influence how the stone is ablated. This increased accuracy, along with any adjustments made during the procedure, can lead to faster and more efficient procedures, greater patient comfort after the procedure, a reduction in the need for follow-up interventions, and helpful information for the clinician when a follow-up intervention is required.Other examples of target features that can determine how the current intervention should proceed or what follow-up interventions may be necessary later include the ability to determine whether a target is a prostatic capsule or an adenoma, or whether the target is cancerous (or potentially or probably cancerous) or benign. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In the drawings, which are not necessarily to scale, the same numbers may denote similar components in different views. The same numbers with different letter suffixes may represent different instances of similar components. The drawings illustrate, generally by way of example but without limitation, various embodiments discussed in this description. Fig. Figure 1 shows an example of a lithotripsy system with an optical detector. Fig. Figure 2 illustrates a flowchart for the use of multiple algorithms to generate an output based on multiple data inputs. Fig. Figure 3 illustrates a procedure for identifying and classifying a target in an image during a medical procedure. Fig. Figure 4 is a block diagram of an example of a machine on which one or more of the techniques (e.g., methodologies) described herein can be performed. Fig. Figure 5 shows a schematic diagram of an exemplary computer-assisted clinical decision support system (CDSS). DETAILED DESCRIPTION

[0015] A system that uses a combined source model to determine one or more target characteristics may include an imaging device. The imaging device may be coupled to the distal tip of a medical viewing device, such as an endoscope. The system may further include a light sensor or an optical sensor, such as a spectrometer. The spectrometer may be used to detect and analyze a signal from a target at or near the distal tip of the medical viewing device. The detected signal may include light reflected from the target or otherwise scattered or emitted, for example, in response to an electromagnetic signal (optical or non-optical) emitted toward the target (e.g., illumination of the target with visible light, an infrared signal emitted toward the target, or the like). The system may also include processing circuitry and memory.The memory can contain instructions which, when executed by the processing circuitry, can cause the processing circuitry to identify or classify the target of a medical procedure. For example, during a lithotripsy procedure, the system can use an image from a camera or other imaging sensor in conjunction with a spectral analysis of the surgical scene to identify a target or to classify or categorize the target by type. In such an example, the endoscopic images can be used to view the target (e.g., a stone) on a large scale and provide a rough estimate of the target's properties, while the spectroscopic data can be used to view the target on a small scale and provide more accurate target properties.For example, the system can classify the target as tissue, stone, muscle tissue, endothelial tissue, or as a prostatic capsule or prostatic adenoma. Furthermore, the system can classify the target object as normal or abnormal (e.g., cancerous). In one example, the endoscopic video and the spectral analysis of light signals from the surgical field, including target objects within the surgical field, can be synchronized so that the spectral analysis can be performed concurrently (or substantially concurrently) with the acquisition of the endoscopic video.

[0016] The identification and classification of the target can be performed by one or more algorithms. An algorithm may involve artificial intelligence (AI) or machine learning (ML), or another algorithm (e.g., a deterministic algorithm without AI or ML) or process. Additionally or alternatively, part of the target identification and / or classification, or a recommendation, may be performed using a hardware-based feedback loop or feedback control. In one example, a single algorithm may be used to identify and classify the target (or to determine a property of the target). Alternatively, multiple algorithms may be used independently. Their outputs may be fed to another model (e.g., a algorithm).a recommendation algorithm) to aggregate the outputs and generate a prediction or recommendation based on the analysis by the other algorithms. For example, image data from the imaging sensor (e.g., endoscopic videos or still images, acoustic imaging, ultrasound imaging, or the like) can be sent to a first algorithm for target detection or identification. Spectra captured from a signal of light from the target object (e.g., endoscopic light reflected back along a surgical fiber, fluorescence emission from the target object in response to illumination of the target object, or the like) can be sent to the same algorithm or to a second, different algorithm to assist in classifying the target (e.g., determining whether the target is a stone or a tumor) or in determining a property, such asto assist in determining the material composition or size of the target. If a second algorithm classifies the target or determines a property of the target, the results of the first and second algorithms can be passed to a third algorithm to aggregate the outputs of the first and second algorithms, for example, to make a prediction or recommendation, alert the user, or the like. For example, the third algorithm can make a prediction about the treatment location, a prediction about the target, a recommendation for a future intervention, or the like.

[0017] In one example, the model can apply weighting factors so that the results of the first and second algorithms can be weighted, evaluated, ranked, prioritized, or similarly treated differently. For instance, if the data from the imaging sensor, which is passed to the first algorithm, is a blurry endoscopic image, the result of the second algorithm (to which the spectra captured from the light signal of the target are passed) can be weighted or ranked more highly than the result of the first algorithm.

[0018] The prediction(s) can be used to provide the user with insights into the treatment site or to suggest an optimal treatment configuration. The treatment configuration may include a recommendation for a setting (or an adjustment of a setting) of the endoscope or of a treatment device used by the endoscope, such as a laser or other ablation element. For example, the system may recommend or suggest a suitable laser setting or an adjustment to the current laser setting (e.g., whether to increase, decrease, or stop the laser beam) for ablating the target. Additionally or alternatively, the system may recommend or suggest to the user to increase or decrease the irrigation or suction of the surgical site.Instead of implementing the system to make sequential decisions from different algorithm models, both endoscopic image data and spectral measurements can be transmitted to one or more algorithms to make decisions (predictions or recommendations) in one or more steps.

[0019] In one example, the data input into the algorithm models can include one or more inputs from one or more additional measurements acquired by one or more supplementary sensors. These supplementary sensors can be located at or near the treatment site or coupled to the medical device and may include a temperature sensor, a pressure sensor, a flow sensor, or the like. Examples of these supplementary measurements could therefore include the temperature or pressure at the anatomical site, the irrigation flow rate, the irrigation temperature, the suction flow rate, the temperature of the suction fluid, or the like.

[0020] Additionally or alternatively, other data or factors, such as anatomical or demographic information about the patient and / or the medical procedure, can be fed into one or more algorithms and used to generate the prediction or recommendation. This other data may include patient information. Patient information may include details about the medical procedure to be performed on the patient, such as the anatomical treatment site. Patient information may also include details such as the patient's age, sex, ethnicity, or pre-existing medical conditions. Furthermore, patient information may include details of previous procedures on the target anatomy, preoperative imaging data (e.g., X-rays, X-rays, etc.).This includes information from a CT scan in which stones were diagnosed in the patient, or information derived from such a scan (e.g., information about the number, size, or location of the stones), results of tissue biopsies, previous findings regarding the composition of the stones, or similar information. The other data may additionally include information about the medical instruments to be used during the procedure (e.g., data on the procedure configuration), such as the use of an access sheath, the size of the laser fiber, the size of the working channel or the introducer tube of the endoscope, the type of endoscopic light source, or the age of the light source.

[0021] Some or all of these factors can influence or modify the algorithm models' suggestions for laser settings, fluid flushing rate and temperature, fiber size, or similar parameters. The algorithms' predictions, suggestions, or recommendations can be adjusted as the target type, composition, or treatment site conditions are identified or change during the procedure, and can help determine when the algorithm models suggest interrupting laser emission or adjusting the position or other settings of the endoscope. For example, the system might identify a blood vessel as a non-target and then modify one or more settings, such as stopping laser emission until the laser fiber is clear of the blood vessel. In another example, the system might adjust settings based on poor visibility in the surgical field (e.g.,(caused by smoke, a high concentration of dust particles, or similar factors) may cause or recommend that the laser stop firing and that a flushing or suction system be activated to clear the field. Potential advantages of the system described herein include more efficient treatment procedures, shorter treatment times, and more effective treatment outcomes. In this disclosure, this ablation energy includes laser or acoustic ablation, or any similar form of energy used to ablate a target or tissue, and the terms laser, ablation, and acoustic energy may be used interchangeably.

[0022] Fig. Figure 1 shows an example of a lithotripsy system with an optical detector. The system 100 can include a surgical laser 102 and a graphical user interface 104 (GUI). The graphical user interface 104 can include a touchscreen or other input mechanism (e.g., a button, switch, or other similar actuating element on the handle of the endoscope) configured to operate or control the surgical laser 102. The surgical laser 102 can include one or more laser sources configured to emit laser radiation. As shown in the dashed box in the example of Figure 1, the system can include a surgical laser 102 and a graphical user interface 104. Fig. As shown in Figure 1, the light sources can include an ablation laser 106 and / or an illumination source 108. The illumination source 108 can include a probe laser, a light-emitting diode (LED), a xenon-based light source, or a similar visible light source. The ablation laser 106 can emit infrared radiation, while the illumination source 108 can emit a targeting beam or an illumination beam of visible light to indicate the direction of the endoscope tip (and thus the ablation energy of the ablation laser 106). Additionally or alternatively, the illumination source 108 can be used to illuminate a target 126. The target 126 can be a piece of tissue, residue, or an object, such as a kidney stone to be ablated, a tumor, a prostatic capsule, or the like.The light 128 emitted by the ablation laser 106 or the illumination source 108 can be emitted through an optical fiber 116, which can be connected to a surgical fiber 118 via an optical connector 120. In one example, the structure of the surgical fiber 118 can be the same as or different from that of the optical fiber 116. The surgical fiber 118 can be located wholly or partially outside the surgical laser 102. The emitted light 128 can thus be emitted from the illumination source 108 via the optical fiber 116, the optical connector 120, and the surgical fiber 118 to a distal end of the surgical fiber 118. The distal end of the surgical fiber 118 can be inserted into an endoscope 124, such as an endoscope, a ureteroscope, a laryngoscope, or the like.In one example, at least some of the light 128 emitted from the distal end of the surgical fiber 118 and the endoscope 124 can be reflected, scattered, or otherwise affected by the target 126 through a medium between the tip of the endoscope 124 and the target 126 (reflected light 130). The endoscope 124 may include an imaging device 122, such as a camera, to record videos or images of the surgical scene, including the target object 126. The medium can be any medium, such as air, water, saline solution, carbon dioxide, or the like.

[0023] The surgical laser 102 may further comprise or be coupled to an optical component, such as an optical splitter 110, which is configured to collect at least a portion of the reflected light 130 passing through the opening of the surgical fiber 118. In one example, the optical splitter 110 may be replaced by a special fiber configured to collect at least a portion of the reflected light 130. The portion of the reflected light 130 collected by the optical splitter 110 or the special fiber may be sent to a processor 112 that is connected to or coupled to the surgical laser 102. An optical detector 132 (e.g.,a spectrometer or other similar light detector or optical sensor) can be arranged between the optical splitter 110 and the processor 112 so that a spectral analysis of the reflected light 130 can be performed to determine one or more properties of the target 126.

[0024] The processor 112 and / or the optical detector 132 can analyze the portion of the reflected light 130 collected by the optical splitter 110 (or received for analysis by a spectrometer connected or coupled to the processor 112 and / or the optical detector 132) to analyze the reflected light 130. The surgical laser 102 may optionally or additionally include a control circuit 114 that is communicatively coupled to the processor 112. The processor 112 or the control unit 114 can determine one or more properties of the reflected light 130 signal that are specific to the light source 108, such as the type of light source, wavelength, optical component, pulse width, optical configuration, or age of the light source 108. While the example in Fig. While the system 100 is described as comprising a single light source, it is understood that the system 100 may include multiple light sources of different types (e.g., an LED source, a xenon-based source, a broadband light source, or the like). Although the light source 108 is shown as part or a component of the surgical laser 102, the light source may also be an external component of the laser system 100 (e.g., outside the surgical laser 102), for example, by being attached to the endoscope 124. Furthermore, it should be noted that the system 100 may include one or more additional components, such as a liquid pump, a suction device, a smoke analyzer, a gas insufflator, or any other component that may be required or used during a laser procedure.

[0025] Fig. Figure 2 shows a flowchart for using multiple algorithms to generate an output based on multiple data inputs. As shown in Fig. As shown in Figure 2, data from multiple data sources can be used as inputs for multiple algorithms. These algorithms can be trained learning algorithms, untrained deterministic algorithms, or a combination of both. For example, data from a first data source 200 can be input into a first algorithm 206. The first data source 200 can include data from an imaging sensor, such as a camera, or a video source capable of recording video or still images. The data in the first data source 200 can include still images or videos of a surgical site captured by a camera contained in or attached to a medical device or medical viewing device, such as an endoscope, or the like. The first data source 200 can further include red-green-blue (RGB) intensities from the endoscopic camera.The first algorithm 206 can perform an analysis of the image or video from the first data source 200, such as image segmentation or the like, to locate a target (or a potential target) in the image.

[0026] Data from a second data source 202 can be input into a second algorithm 208. The second data source 202 can include a signal (e.g., light) detected by a target. For example, a light source attached to or coupled with the medical device (e.g., a xenon light source or any visible or invisible light source) can be emitted at the surgical site, for example, in the direction of the target identified in the image from the second data source 202. The light reflected, scattered, or otherwise emitted by the target can be detected by an optical splitter contained in, connected to, or otherwise associated with the medical device.

[0027] An optical sensor, such as a spectrometer, can analyze the spectra of the light collected from the target to identify or classify the target. For example, spectral analysis can be used to confirm the identification of a potential target in the images from the first data source 200. Additionally or alternatively, spectral analysis can be used to determine a property or composition of a target identified in the images from the first data source 200. For example, if a kidney stone is identified in an endoscopic image, light can be used to illuminate the kidney stone. Based at least partially on the spectral analysis of the light reflected by the stone, the composition of the stone (e.g., whether the stone is composed of uric acid or calcium oxalate), the size of the stone, or similar properties can be determined by the second algorithm 208.In one example, the morphology of the target object captured in the endoscopic image can be helpful for spectral analysis. For instance, the gross morphology or structure of a target such as a stone may be a better predictor of certain types of stones than spectral analysis alone. For example, uric acid stones typically have smooth surface features, while brushite and calcium-based stones have thin, sharp projections (e.g., protruding crystal-like structures). Thus, the system can use the stone's morphology (e.g., surface features such as whether the stone is smooth or textured, the size of the stone, etc.) in conjunction with spectral analysis to identify the type of stone, its composition, and so on.

[0028] Data from a third data source 204 can be input into a third algorithm 210. The data from the third data source 204 can include data from one or more sensors contained in or connected to the medical device, such as a temperature sensor, a pressure sensor, a flow sensor, or the like. Additionally or alternatively, the data from the third data source 204 can include patient-specific data, such as data from a patient record. The data from the patient record can include data such as the patient's age, sex, weight, or other relevant physical characteristics. The data from the patient record can also include information about the patient's medical history, such as previous medical procedures, pre-existing conditions, allergies, blood type, blood antibodies, response to anesthesia, or the like.The data from the patient record may also include clinical reports containing information such as notes on constrictions, irregular anatomical tissue calcifications, or the like, which may require treatment or review during the medical procedure or in a follow-up procedure. For example, the historical data may include information on individuals belonging to the same or a similar, or substantially the same or similar, demographic class as the patient (e.g., individuals of similar age, sex, or the like) or who have similar medical conditions to the patient.

[0029] The results of the first algorithm 206, the second algorithm 208, and the third algorithm 210 can be used as inputs for a fourth algorithm 212. The fourth algorithm 212 can be a recommendation algorithm that analyzes the results of the first algorithm 206, the second algorithm 208, and the third algorithm 210 to generate a result 214. For example, the first algorithm 206 can generate an endoscopic image of the surgical area with identified targets based on the endoscopic video or endoscopic images, and the second algorithm 208 can provide an analysis of the targets based on spectral analysis (as described above).Based on the outputs of the first algorithm 206 and the second algorithm 208, the fourth algorithm 212 can compare the colors on the endoscopic image with the colors determined during spectral analysis and calibrate or adjust the colors on the endoscopic image to produce a calibrated endoscopic image of the surgical site. The calibrated endoscopic image can represent a "true-color image" of the surgical site.

[0030] In one example, output 214 might include an annotated version of the calibrated endoscopic image of the surgical site. The annotated endoscopic image can identify targets within the endoscopic image and provide details about those targets, such as size or composition. In another example, the calibrated endoscopic image might show non-targets, such as non-target tissue (e.g., muscle) that should not come into contact with laser or other ablation energy. The annotation might include information about a change in the target or a change in the surgical site. The change in the target or surgical site could be determined by comparing an initial image acquired earlier in the same procedure—for example, a change in the size or composition of a kidney stone during its ablation—with a subsequent image.Alternatively, the change can be determined using an initial image taken during a previous, separate procedure and an image taken during the current procedure. For example, the system can identify new targets during the current procedure that were not present during a previous procedure, indicate that a mass has increased or decreased in size, or similar observations. The annotated image can be displayed on a graphical user interface, such as a monitor in the treatment room.

[0031] Output 214 can include a warning or an alarm. For example, if during a laser ablation procedure one of the algorithms detects that the target composition has changed, that the laser is coming into contact with non-target tissue, or that any other condition exists that could affect whether the laser ablation should continue (or continue at the current intensity), or whether the endoscope tip should be moved or repositioned, the fourth algorithm, 212, can trigger the output of an alarm. The alarm can be visual, such as the endoscopic light flashing or changing color; audible, such as a beep; or haptic, such as the endoscope handle vibrating. The warning can also be a message displayed on the graphical user interface.In one example, output 214 can cause a control unit, such as control unit 114, to adjust or stop the ablation energy, or cause an actuator connected to the endoscope to reposition the tip of the endoscope based on a determination by the fourth algorithm 212.

[0032] Fig. Figure 3 illustrates a procedure for identifying and classifying a target in an image during a medical procedure. Procedure 300 may include or comprise a series of operations or steps. The operations described here are only examples, and the procedure may omit one or more of the listed operations, repeat operations, include other operations, or perform the operations concurrently, substantially concurrently, or in a different order, as needed or desired.

[0033] In 302, the procedure 300 may include receiving an image of a surgical site. The image may be acquired by an imaging device, sensor, or device connected to a medical device, such as a camera located at or near the tip of an endoscope. In 304, the procedure 300 may include analyzing an image of the surgical site to locate a target (or targets) within the image. The analysis may be performed by a trained learning algorithm (such as an artificial intelligence or machine learning algorithm) to identify potential targets, such as kidney stones, within the surgical site. The analysis may involve a computer vision technique, such as image segmentation analysis or a similar technique capable of object detection.

[0034] In 306, the procedure 300 may involve emitting a signal (e.g., visible or invisible light, laser radiation, or the like) in the direction of a target identified in the image received in 302. For example, the endoscope may include or be coupled to a source of visible light, such as a xenon-based light source or a similar light source capable of illuminating the surgical area.

[0035] In 308, the procedure 300 may include collecting at least part of a signal reflected or scattered by the target object in response to the signal emitted in 306 (hereinafter referred to as the “response signal”), and 310 may include analyzing the collected response signal. The analysis of the response signal may include comparing one or more colors in the image of the surgical site with one or more colors determined during the spectral analysis of the surgical site. In an example, if the scene is illuminated with an illumination light, an optical sensor or detector, such as a spectrometer, may be used to analyze the spectra of the illumination light or the response signal. The light analyzed by the spectrometer may include light reflected or scattered by tissue, targets, or other objects in the surgical scene.The analysis may further include calibrating or adjusting one or more colors on the image of the surgical site based on or using spectral analysis of the surgical site to produce a calibrated endoscopic image (e.g., a true-color image) of the surgical site.

[0036] In procedure 300, 312 may include providing an output. In one example, the output may include a warning, such as a visual, audible, or haptic alert. The warning may be in response to part of the endoscope (e.g., the tip) being in an undesired position or laser energy being directed at non-target tissue, or the like. Additionally or alternatively, the output may include a prediction or a recommendation. The recommendation may include a suggestion regarding a subsequent medical intervention. Such a recommendation may include a type of medical endoscope to be used during the subsequent medical intervention, a type and intensity of ablation energy (e.g.,The recommendation may include information about the laser radiation to be used during the subsequent medical procedure, the type or amount of anesthesia to be used during the subsequent medical procedure, or similar information. It may also include information about a risk to the patient during a subsequent medical procedure, such as the risk of a drop in blood pressure, blood group antibody allergies, or similar issues. The prediction may include whether a target is likely to be cancerous, or similar information.

[0037] Procedure 300 can be a computer-implemented procedure that uses or employs different algorithms to perform the operations of the procedure. For example, the image of the operational area can be analyzed using a first algorithm, and the identification or classification of the target can be performed using a second algorithm. In such an example, an output of the first algorithm (the analyzed image of the operational area) and an output of the second algorithm (the identification or classification of the target) can be used as inputs for a recommendation algorithm. The recommendation algorithm can, in turn, produce the output at 312. These algorithms can include artificial intelligence (AI) or machine learning (ML), or another algorithm (e.g., a non-AI or non-ML deterministic algorithm) or process.Additionally or alternatively, parts of procedure 300 can be performed using a hardware-based feedback loop or feedback control. The hardware-based feedback loop or feedback control, the AI ​​or ML algorithms, and the non-AI or non-ML algorithms can be used separately or in combination, as required or desired.

[0038] Fig. Figure 4 is a block diagram of an example of a Machine 400 that can be used to perform one or more of the techniques (e.g., methodologies) described herein. The Machine 400 can be operated as a standalone device or connected (e.g., networked) to other machines. For example, the Machine 400 can be integrated into or connected to a medical device (e.g., an endoscope, a surgical laser, a surgical fiber, or the like), or it can be integrated into or connected to one or more sensors, which may include components of the system. Additionally or alternatively, the Machine 400 can perform one or more of the algorithms described above or the one described below with respect to Fig. 5. Operate the computer-aided clinical decision support system (CDSS) described. In a networked deployment, Machine 400 can operate as a server machine, a client machine, or both in a server-client network environment. For example, Machine 400 can function as a peer machine in a peer-to-peer (P2P) network environment (or other distributed network environment). Machine 400 can be a personal computer (PC), a tablet PC, a set-top box (STB), a personal digital assistant (PDA), a mobile phone, a web device, a network router, a switch or bridge, or any other machine capable of executing instructions (sequentially or otherwise) that specify the actions to be performed by that machine.Although only a single computer is depicted, the term "computer" can encompass any collection of computers that, individually or collectively, execute a set (or multiple sets) of instructions to perform one or more of the methods described herein, such as cloud computing, Software as a Service (SaaS), or other computer cluster configurations.

[0039] Examples, as described here, can include or operate logic or a set of components or mechanisms. Circuit sets are a collection of circuits implemented in tangible units that include hardware (e.g., simple circuits, gates, logic, etc.). Membership in a circuit set can be flexible over time and due to the underlying hardware variability. Circuit sets include members that, alone or in combination, can perform specific operations when in operation. In one example, the hardware of the circuit set may be immutably designed to perform a specific operation (e.g., hardwired). In another example, the hardware of the circuit set may include variably connected physical components (e.g., execution units, transistors, simple circuits, etc.), including a computer-readable medium that is physically modified (e.g.,(magnetic, electrical, movable arrangement of unchanging mass particles, etc.) to encode commands for the specific operation. When the physical components are connected, the underlying electrical properties of a hardware component are changed, for example, from an insulator to a conductor or vice versa. The commands allow the embedded hardware (e.g., the execution units or a loading mechanism) to create elements of the circuit set in the hardware via the variable connections in order to execute parts of the specific operation. Accordingly, the computer-readable medium is communicatively coupled with the other components of the circuit set element during the operation of the device. In one example, each of the physical components can be used in more than one element of more than one circuit set.For example, execution units can be used in operation at one time in a first circuit of a first circuit set and reused at another time by a second circuit in the first circuit set or by a third circuit in a second circuit set.

[0040] The machine 400 (e.g., a computer system) may include a hardware processor 402 (e.g., a central processing unit (CPU), a graphics processing unit (GPU), a hardware processor core, a field-programmable gate array (FPGA), or any combination thereof), main memory 404, and static memory 406, some or all of which may communicate with each other via a connection (e.g., a bus) 430. The machine 400 may further include a display unit 410, an alphanumeric input device 412 (e.g., a keyboard), and a user interface navigation device 414 (e.g., a mouse). In an example, the display unit 410, the input device 412, and the user interface navigation device 414 may be a touchscreen display. The machine 400 can additionally include a storage device 408 (e.g. a drive unit), a signal generation device 418 (e.g.The machine 400 may include a loudspeaker, a network interface device 420, and one or more sensors 416, such as a GPS (Global Positioning System) sensor, a compass, an accelerometer, or another sensor. The machine 400 may include an output control unit 428, such as a serial (e.g., Universal Serial Bus (USB)), parallel, or other wired or wireless (e.g., infrared (IR), near field communication (NFC), etc.) connection for communicating with or controlling one or more peripheral devices (e.g., a printer, a card reader, etc.).

[0041] The storage device 408 can comprise a machine-readable medium 422 (e.g., a non-transitory medium) on which one or more sets of data structures or instructions 424 (e.g., software) are stored, embodying or utilizing one or more of the techniques or functions described herein. The instructions 424 may also be stored, in whole or in part, in main memory 404, static memory 406, or the hardware processor 402 during their execution by the machine 400. In an example, any one or any combination of the hardware processor 402, main memory 404, static memory 406, or storage device 408 can constitute a machine-readable medium.

[0042] While the machine-readable medium 422 is presented as a single medium, the term "machine-readable medium" can encompass a single medium or multiple media (e.g., a centralized or distributed database and / or associated caches and servers) configured to store the one or more instructions 424. The term "machine-readable medium" can include any non-transitory medium capable of storing, encoding, or transmitting instructions for execution by the machine 400, and causing the machine 400 to execute one or more of the techniques of this disclosure, or capable of storing, encoding, or transmitting data structures used by or associated with such instructions. Examples of non-restrictive machine-readable media include solid-state storage as well as optical and magnetic media.In one example, a massed machine-readable medium comprises a machine-readable medium with a plurality of particles having an unchanging (e.g., rest) mass. Accordingly, massed machine-readable media are not transient propagation signals. Specific examples of massed machine-readable media include: non-volatile memories, such as semiconductor memory devices (e.g., electrically programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM)) and flash memory devices; magnetic disks, such as internal hard disks and removable media; magneto-optical disks; and CD-ROM and DVD-ROM disks.

[0043] Fig.Figure 5 shows a schematic diagram of an exemplary computer-assisted clinical decision support system (CDSS) 500, configured to localize targets in an image and to identify or classify targets based on spectral analysis of the image. The CDSS 500 can further be configured to issue a warning, a recommendation, or a prediction. The recommendation may include suggested changes to settings (e.g., changed laser settings, changed position of the endoscope, or the like) or a recommendation for a subsequent medical intervention. The recommendation for the subsequent medical intervention or the changed settings may be based on information about a target, tissue sample, or tissue segment to be treated, patient information, conditions that arose during the medical intervention, or a combination thereof.In various embodiments, the CDSS 500 can include an input interface 502 through which one or more inputs, such as an image of the surgical site, specific to a patient, are supplied as input features to an artificial intelligence (AI) model 504, wherein a processor, such as the processor 402, can perform an inference operation in which the information about the target, a piece or part of the tissue, the patient, the medical procedure, or other suitable or desired inputs is applied to the AI ​​model to generate the warning, recommendation, or prediction, and a user interface (UI) through which the warning, recommendation, or prediction can be communicated to a user, e.g., a clinician.

[0044] In some embodiments, the input interface 502 can be a direct data connection between the CDSS 500 and one or more medical devices or sensors that generate at least some of the input features. For example, during a therapeutic and / or diagnostic medical procedure, the input interface 502 can directly transmit information about the target, such as the target's composition or density, to the CDSS 500. This target information can be determined, as described above, by spectral analysis of a signal from the target. Additionally or alternatively, the input interface 502 can be a conventional user interface that facilitates interaction between a user and the CDSS 500.For example, the input interface 502 can provide a user interface through which the user can manually enter information about the patient or information about the medical procedure. Additionally or alternatively, the input interface 502 can grant the CDSS 500 access to an electronic patient record from which one or more input features can be extracted. The patient record can be stored in a database 506 connected to the CDSS 500. In each of these cases, the input interface 502 can be configured to capture one or more of the following input features associated with a particular patient at or prior to a time when the CDSS 500 is used for assessment:

[0045] An image of the surgical site 510, which may contain information about the target location, the size of the target or non-target tissue, can be captured by an imaging sensor, such as an endoscopic camera.

[0046] Information from the spectral analysis of the surgical area 512. The information may include target features and may be used in conjunction with the image of the target acquired by the endoscope's imaging sensor.

[0047] An environmental condition measured by one or more sensors on or near the endoscope; A location for the laser fiber or the endoscope; Information about the medical procedure; An indication of non-target tissue; Whether a target or tissue is normal or abnormal; Information about the patient, such as their medical history; and / or Information about the patient's condition during the medical procedure.

[0048] Based on one or more of the input features mentioned above, the 402 processor can perform an inference operation using the AI ​​model to generate an output. The output can include a warning or an alarm. The output can include a change to one or more laser settings to be implemented, or a recommended change to one or more laser settings to be suggested to the user. The output can include a recommendation for a subsequent medical intervention. The AI ​​model can give a system the ability to perform tasks without being explicitly programmed by drawing inferences based on patterns found during data analysis. The AI ​​model can facilitate the study and construction of algorithms (e.g.,Researchers explore machine learning algorithms that can learn from existing data and make predictions based on new data. Such algorithms work by building an AI model from sample training data to make data-driven predictions or decisions, expressed as outputs or ratings.

[0049] Two examples of machine learning (ML) modes are supervised ML and unsupervised ML. Supervised ML uses prior knowledge (e.g., examples that correlate inputs with outputs or results) to learn the relationships between inputs and outputs. The goal of supervised ML is to learn a function that, given training data, best approximates the relationship between the training inputs and outputs, so that the machine learning model can implement the same relationships when it receives inputs to generate the corresponding outputs. Unsupervised ML is the training of an ML algorithm using information that is neither classified nor labeled, allowing the algorithm to react to this information without guidance. Unsupervised ML is useful in exploratory analysis because it can automatically identify structures in data.

[0050] Certain tasks for supervised machine learning (ML) can include classification problems and regression problems. Classification problems, also known as categorization problems, aim to assign items to one of several categories (e.g., is this object an apple or an orange?). Regression algorithms aim to quantify certain items (e.g., by assigning a score to the value of an input). Examples of supervised ML algorithms include logistic regression (LR), Naive Bayes, Random Forest (RF), neural networks (NN), deep neural networks (DNN), matrix factorization, and support vector machines (SVM).

[0051] Tasks for unsupervised machine learning can include clustering, representation learning, and density estimation. Examples of unsupervised machine learning algorithms include K-means clustering, principal component analysis, and autoencoders.

[0052] Another type of machine learning is federated learning (also known as collaborative learning), in which an algorithm is trained across multiple decentralized devices that store local data without exchanging that data. This approach contrasts with centralized machine learning techniques, where all local datasets are uploaded to a server, and with more traditional decentralized approaches, which often assume that local data samples are identically distributed. Federated learning allows multiple actors to build a shared, robust machine learning model without exchanging data, thereby addressing critical issues such as data privacy, data security, data access rights, and access to heterogeneous data.

[0053] In some examples, the AI ​​model can be continuously or periodically trained by the 402 processor before the inference operation is performed. During the inference process, the patient-specific input features provided to the AI ​​model can then be passed from an input layer through one or more hidden layers to an output layer, corresponding to changes in one or more laser settings, one or more predictions or recommendations, target identification and / or classification, or similar actions. For example, based on the target features, the CDSS 500 can identify the target as a kidney stone composed of a specific material such as uric acid. Based on the stone identification and classification, the CDSS can recommend a setting for an ablation energy source, such as a laser.Throughout the entire procedure, the stone's properties can change (e.g., a change in size, a change in material composition, etc.). Based on the stone's properties and a detected change in the target properties (e.g., a change in the stone's composition), the CDSS 500 can recommend a change to the laser settings, such as laser intensity and / or frequency, which can be transmitted to the output layer, a change to the irrigation settings and / or suction, or another recommendation relevant to the procedure. In another example, the CDSS can make a prediction, such as the time required to complete the procedure or that the temperature or pressure at the treatment site may increase within a certain period. Additionally or alternatively, the system can recommend a subsequent medical intervention.Such a recommendation can be based, at least in part, on a condition or event occurring during the current medical procedure. For example, if a patient has an adverse reaction to a type of anesthetic used during the procedure, the system may recommend a different type of anesthetic for a subsequent procedure.

[0054] During or after the inference operation, the recommendation can be communicated to the user via a user interface (UI) and / or cause processor 402 to automatically adjust a setting (e.g., a laser setting, a flushing rate, a suction rate, or the like) of the medical device. ADDITIONAL REMARKS AND EXAMPLES:

[0055] Example 1 is a system for identifying and classifying a target in an image acquired during a medical procedure, the system comprising: a medical device; and a processing circuit associated with the medical device, the processing circuit being used to: receive an image of a surgical site; analyze the image of the surgical site to locate a target in the image of the surgical site; cause light to be emitted from a light source associated with the medical device in the direction of the target; collect at least some of the emitted light, which is reflected or scattered by the target, at an optical splitter associated with the medical device; analyze the collected light using an optical sensor coupled to the optical splitter to identify or classify the target;and provide output to a user based on at least one of the identification or classification of the target;

[0056] In Example 2, the subject of Example 1 optionally includes an subject in which the medical device is an endoscope and the image of the surgical site is captured by an endoscopic camera, and in which the processing circuit performs the following tasks: comparing one or more colors on the image of the surgical site with one or more colors determined during the spectral analysis of the surgical site; and calibrating or otherwise adjusting the one or more colors on the image of the surgical site based on the spectral analysis of the surgical site to produce a calibrated endoscopic image of the surgical site.

[0057] In Example 3, the subject of Example 2 optionally includes a subject in which the processing circuit has the following tasks: outputting the calibrated endoscopic image of the surgical area to a graphical user interface.

[0058] In Example 4, the subject of one or more of Examples 2-3 optionally includes an item in which the processing circuit has the following task: to annotate the calibrated endoscopic image of the surgical area.

[0059] In Example 5, the subject of Example 4 optionally includes an subject in which annotating the calibrated endoscopic image of the surgical site includes annotating the calibrated endoscopic image of the surgical site with at least one of the following information: information about the target, information about the surgical site, identification of non-target anatomy, information about a change in the target, or information about a change in the surgical site.

[0060] In Example 6, the subject of one or more of Examples 1 to 5 optionally includes an subject in which the image of the operational area is analyzed using a first algorithm and the identification or classification of the target is performed using a second algorithm, with an output of the first algorithm and an output of the second algorithm being used as inputs for a recommendation algorithm.

[0061] In Example 7, the subject of Example 6 optionally includes an subject in which the processing circuit has the following tasks: receiving data from one or more additional sources; and inputting the data from the one or more additional sources into the recommendation algorithm, the output being based at least partially on an analysis of the data from the one or more additional sources.

[0062] In Example 8, the subject of Example 7 optionally includes an item in which the processing circuit serves to effect an adaptation of the medical device, at least in part, on the basis of one or more data from one or more additional sources or the identification or classification of the target.

[0063] In Example 9, the subject of one or more of Examples 7-8 optionally includes an subject in which the data from one or more additional sources include historical data on a patient undergoing the medical procedure, historical data on a previous medical procedure undergone by a member of a demographic class to which the patient belongs, or real-time data captured by a sensor coupled or connected to the medical device.

[0064] In Example 10, the subject of Example 9 optionally includes an item where the historical data relating to the patient contains information about a previous medical intervention, a pre-existing condition of the patient, or a physical characteristic of the patient.

[0065] In Example 11, the subject of one or more of Examples 1 to 10 optionally includes an item in which the output contains a recommendation, wherein the recommendation includes a recommendation regarding a subsequent medical intervention, and wherein the recommendation regarding a subsequent medical intervention includes one or more of the following elements: a type of medical endoscope to be used during the subsequent medical intervention, a type and intensity of ablation energy to be used during the subsequent medical intervention, a type and amount of anesthesia to be used during the subsequent medical intervention, or a risk during the subsequent medical intervention.

[0066] In Example 12, the subject of one or more of Examples 1 to 11 optionally includes an subject in which the processing circuit additionally serves to determine one or more morphological properties of the target and to use the determined one or more morphological properties in conjunction with the analysis of the light detected by the optical sensor to at least identify or classify the target.

[0067] Example 13 is a computer-implemented method for identifying and classifying a target in an image acquired during a medical procedure, comprising: receiving an image of a surgical site from an optical sensor connected to a medical device used during the medical procedure; analyzing the image using a first algorithm; emitting a signal toward a target from a light source connected to the medical device; collecting at least part of a response signal reflected or scattered by the target in response to the emitted signal at an optical splitter connected to the medical device; analyzing the collected response signal using an optical sensor coupled to the optical splitter to identify or classify the target;and providing output to a user that is based, at least in part, on the identification or classification of the target.

[0068] In Example 14, the subject of Example 13 optionally includes an item that compares one or more colors on the image of the surgical site with one or more colors determined during the spectral analysis of the surgical site; and calibrating or otherwise adjusting the one or more colors on the image of the surgical site based on the spectral analysis of the surgical site to produce a calibrated endoscopic image of the surgical site.

[0069] In Example 15, the subject of Example 14 optionally includes an subject that annotates the calibrated endoscopic image of the surgical site, wherein annotating the calibrated endoscopic image of the surgical site includes annotating the calibrated endoscopic image of the surgical site with at least one of the following information: information about the target, information about the surgical site, identification of non-target anatomy, information about a change in the target or information about a change in the surgical site; and outputting the calibrated endoscopic image of the surgical site on a graphical user interface.

[0070] In Example 16, the subject of one or more of Examples 13-15 optionally includes an subject in which the image of the surgical site is analyzed using a first algorithm and the identification or classification of the target is performed using a second algorithm, wherein an output of the first algorithm and an output of the second algorithm are used as inputs for a recommendation algorithm, and wherein the procedure further includes: receiving data from one or more additional sources, wherein the data from the one or more additional sources include historical data on a patient undergoing the medical procedure, historical data on a previous medical procedure undergone by a member of a demographic class to which the patient belongs, or real-time data obtained from a sensor coupled or connected to the medical device;Inputting the data from the one or more additional sources into the recommendation algorithm, the output being based at least partially on an analysis of the data from the one or more additional sources; and initiating an adjustment of the medical device, based at least partially on one or more of the data from the one or more additional sources or on the identification or classification of the target.

[0071] Example 17 is a non-volatile, computer-readable medium containing instructions which, when executed by a processor of a computer connected to a medical device, cause the processor to: receive an image of a surgical site; analyze the image of the surgical site to locate a target in the image of the surgical site; cause light to be emitted from a light source connected to the medical device in the direction of the target; collect at least some of the emitted light reflected or scattered by the target at an optical splitter connected to the medical device; analyze the collected light using an optical sensor connected to the optical splitter to identify or classify the target;and provide output to a user based on at least one of the identification or classification of the target;

[0072] In Example 18, the subject of Example 17 optionally includes an item in which the instructions cause the processor to: compare one or more colors on the surgical site image with one or more colors determined during the surgical site spectral analysis; calibrate or otherwise adjust the one or more colors on the surgical site image based on the surgical site spectral analysis to produce a calibrated endoscopic image of the surgical site; annotate the calibrated endoscopic image of the surgical site; output the calibrated endoscopic image of the surgical site to a graphical user interface; receive data from one or more additional sources;and input the data from one or more additional sources into a recommendation algorithm, the output being based at least partially on an analysis of the data from one or more additional sources.

[0073] In Example 19, the subject of Example 18 optionally includes a subject in which the image of the operation site is analyzed using a first algorithm and the identification or classification of the target is performed using a second algorithm, with an output of the first algorithm and an output of the second algorithm being used as inputs for a recommendation algorithm.

[0074] In Example 20, the subject of Example 19 optionally includes an item where the output contains a recommendation regarding a subsequent medical intervention, and where the recommendation regarding a subsequent medical intervention includes one or more of the following elements: a type of medical endoscope to be used during the subsequent medical intervention, a type and intensity of ablation energy to be used during the subsequent medical intervention, a type and amount of anesthesia to be used during the subsequent medical intervention, or a risk during the subsequent medical intervention.

[0075] The foregoing detailed description contains references to the accompanying drawings, which form part of the detailed description. The drawings illustrate specific embodiments that can be implemented. These embodiments are also referred to herein as "examples." Such examples may include additional elements beyond those shown or described. However, the inventors also consider examples in which only the elements shown or described are provided. Furthermore, the inventors also consider examples that use any combination or permutation of the elements shown or described (or one or more aspects thereof), either in relation to a particular example (or one or more aspects thereof) or in relation to other examples shown or described herein (or one or more aspects thereof).

[0076] All publications, patents, and patent documents mentioned in this document are hereby incorporated by reference in their entirety, as if they had been individually referenced. In the event of any discrepancies between this document and the referenced documents, the references in the incorporated documents shall be considered supplementary to this document; in the case of irreconcilable discrepancies, the reference in this document shall prevail.

[0077] In this document, the terms "a" or "an" are used, as is customary in patent documents, to include one or more instances, irrespective of other instances or uses of "at least one" or "one or more". In this document, the term "or" is used to indicate a non-exclusive OR connection, such that "A or B" includes "A but not B", "B but not A", and "A and B", unless otherwise indicated. In this document, the term "and / or" is used to indicate a non-exclusive OR connection, such that "A and / or B" includes "A but not B", "B but not A", and "A and B", unless otherwise indicated. In the appended claims, the terms "including" and "in the" are used as simple linguistic equivalents of the respective terms "comprising" and "in the". Furthermore, in the following claims, the terms "including" and "comprising" are open, i.e.,A system, device, object, or method that includes further elements in addition to those listed in a claim after such a term remains within the scope of that claim. Furthermore, in the following claims, the terms "first," "second," and "third," etc., are used merely as designations and are not intended to impose any numerical requirements on their objects.

[0078] The above description is for illustrative purposes only and is not limiting. For example, the examples described above (or one or more aspects thereof) may be used in combination with one another. Other embodiments may also be used, such as those that might occur to a person skilled in the art after reviewing the above description. The summary is intended to enable the reader to quickly grasp the nature of the technical disclosure and is presented with the understanding that it is not intended to be used for interpreting or limiting the scope or meaning of the claims. Furthermore, various features may be summarized in the above detailed description to streamline the disclosure. This should not be interpreted as meaning that an unclaimed disclosed feature is essential to a claim. Rather, the inventive subject matter may consist of fewer than all features of a particular disclosed embodiment.The following claims are hereby incorporated into the detailed description, each claim constituting a separate embodiment. The scope of the embodiments should be determined with reference to the appended claims and the full scope of the equivalents to which these claims are entitled. QUOTES INCLUDED IN THE DESCRIPTION

[0000] This list of documents cited by the applicant was automatically generated and is included solely for the reader's convenience. The list is not part of the German patent or utility model application. The DPMA accepts no liability for any errors or omissions. Cited patent literature

[0000] US 63 / 515,896

[0001] US 63 / 652.256

[0001]

Claims

[1] A system for identifying and classifying a target in an image taken during a medical procedure, the system comprising: a medical device; and a processing circuit connected to the medical device, wherein the processing circuit serves to: to receive an image of an operating site; to analyze the image of the surgical site in order to locate a target within the image of the surgical site; to cause light to be emitted from a light source connected to the medical device in the direction of the target; Collecting at least a portion of the emitted light that is reflected or scattered by the target object at an optical splitter connected to the medical device; Analyzing the collected light using an optical sensor connected to the optical splitter to at least identify or classify the target object; and Providing an output to a user based on at least one of the identification or classification of the target object. [2] The system according to claim 1, wherein the medical device is an endoscope and the image of the surgical site is captured by an endoscopic camera, and wherein the processing circuit serves to: to compare one or more colors on the image of the surgical site with one or more colors determined during the spectral analysis of the surgical site; and to calibrate or otherwise adjust one or more colors on the image of the surgical site based on the spectral analysis of the surgical site in order to produce a calibrated endoscopic image of the surgical site. [3] The system according to claim 2, wherein the processing circuit serves to: to output the calibrated endoscopic image of the surgical area to a graphical user interface. [4] The system according to claim 2, wherein the processing circuit serves to: to annotate the calibrated endoscopic image of the surgical site. [5] The system according to claim 4, wherein the annotation of the calibrated endoscopic image of the surgical site comprises annotating the calibrated endoscopic image of the surgical site with at least one of the following information: information about the target, information about the surgical site, identification of non-target anatomy, information about a change in the target or information about a change in the surgical site. [6] The system according to claim 1, wherein the image of the operating site is analyzed using a first algorithm and the identification or classification of the target is carried out using a second algorithm, wherein an output of the first algorithm and an output of the second algorithm are used as inputs for a recommendation algorithm. [7] The system according to claim 6, wherein the processing circuit serves to: to receive data from one or more additional sources; and to input the data from one or more additional sources into the recommendation algorithm, and wherein the output is based at least partially on an analysis of the data from one or more additional sources. [8] The system according to claim 7, wherein the processing circuit serves to: to effect an adaptation of the medical device, at least in part, based on one or more data from one or more additional sources or on the identification or classification of the target. [9] The system according to claim 7, wherein the data from one or more additional sources comprise historical data on a patient undergoing the medical procedure, historical data on a previous medical procedure to which a member of a demographic class to which the patient belongs has undergone, or real-time data obtained from a sensor coupled or connected to the medical device. [10] The system according to claim 9, wherein the historical data relating to the patient includes information about a previous medical intervention, a pre-existing condition of the patient or a physical characteristic of the patient. [11] The system according to claim 1, wherein the output comprises a recommendation, wherein the recommendation comprises a recommendation regarding a subsequent medical intervention, and wherein the recommendation regarding a subsequent medical intervention comprises one or more of the following elements: a type of medical endoscope to be used during the subsequent medical intervention, a type and intensity of ablation energy to be used during the subsequent medical intervention, a type and amount of anesthesia to be used during the subsequent medical intervention, or a risk during the subsequent medical intervention. [12] The system according to claim 1, wherein the processing circuit further serves to: to determine one or more morphological characteristics of the target; and to use one or more specific morphological features in conjunction with the analysis of the light collected by the optical sensor to identify or classify the target. [13] A computer-aided method for identifying and classifying a target in an image taken during a medical procedure, comprising: Receiving an image of a surgical site from an optical sensor connected to a medical device used during the medical procedure; Analyzing the image using a first algorithm; Emitting a signal towards a target from a light source connected to the medical device; Collecting at least part of a response signal that is reflected or scattered by the target object in response to the emitted signal, on an optical splitter connected to the medical device; Analyzing the collected response signal using an optical sensor connected to the optical splitter to identify or classify the target object; and Providing output to a user that is based, at least in part, on the identification or classification of the target object. [14] The method of claim 13, comprising: Comparing one or more colors on the image of the surgical site with one or more colors determined during spectral analysis of the surgical site; and Calibrating or otherwise adjusting one or more colors on the image of the surgical site based on the spectral analysis of the surgical site to produce a calibrated endoscopic image of the surgical site. [15] The method of claim 14, comprising: Annotating the calibrated endoscopic image of the surgical site, wherein the annotation of the calibrated endoscopic image of the surgical site includes annotating the calibrated endoscopic image of the surgical site with at least one of the following information: information about the target, information about the surgical site, identification of non-target anatomy, information about a change in the target, or information about a change in the surgical site; and Output of the calibrated endoscopic image of the surgical site to a graphical user interface. [16] Method according to claim 13, wherein the image of the surgical site is analyzed using a first algorithm and the identification or classification of the target is carried out using a second algorithm, wherein an output of the first algorithm and an output of the second algorithm are used as inputs for a recommendation algorithm and wherein the method further comprises: Receiving data from one or more additional sources, wherein the data from the one or more additional sources include historical data on a patient undergoing the medical procedure, historical data on a previous medical procedure undergone by a member of a demographic class to which the patient belongs, or real-time data obtained from a sensor coupled or connected to the medical device; Inputting the data from one or more additional sources into the recommendation algorithm, the output being based at least partially on an analysis of the data from one or more additional sources; and Initiating an adjustment of the medical device based at least in part on one or more of the data from the one or more additional sources or the identification or classification of the target. [17] A non-volatile, computer-readable medium containing instructions which, when executed by a processor of a computer connected to a medical device, cause the processor to: to receive an image of an operating site; to analyze the image of the surgical site in order to locate a target within the image of the surgical site; to cause light to be emitted from a light source connected to the medical device in the direction of the target; to collect at least a portion of the emitted light that is reflected or scattered back from the target on an optical splitter connected to the medical device; to analyze the collected light using an optical sensor connected to the optical splitter in order to at least identify or classify the target; and to provide output to a user based on at least one of the identification or classification of the target. [18] The non-volatile, computer-readable medium according to claim 17, wherein the instructions cause the processor to: to compare one or more colors on the image of the surgical site with one or more colors determined during the spectral analysis of the surgical site; to calibrate or otherwise adjust one or more colors on the image of the surgical site based on the spectral analysis of the surgical site in order to create a calibrated endoscopic image of the surgical site; Annotate the calibrated endoscopic image of the surgical site; output the calibrated endoscopic image of the surgical site on a graphical user interface; to receive data from one or more additional sources; and to input the data from the one or more additional sources into a recommendation algorithm, the output being based at least partially on an analysis of the data from the one or more additional sources. [19] The non-volatile computer-readable medium according to claim 18, wherein the image of the operation site is analyzed using a first algorithm and the identification or classification of the target is carried out using a second algorithm, wherein an output of the first algorithm and an output of the second algorithm are used as inputs for a recommendation algorithm. [20] The non-volatile, computer-readable medium according to claim 19, wherein the output includes a recommendation regarding a subsequent medical intervention and wherein the recommendation regarding a subsequent medical intervention includes one or more of the following elements: a type of medical endoscope to be used during the subsequent medical intervention, a type and intensity of ablation energy to be used during the subsequent medical intervention, a type and amount of anesthetic to be used during the subsequent medical intervention, or a risk during the subsequent medical intervention.

Images