System and method for determining target characteristics during laser treatment

Abstract

To provide a system and method for determining target characteristics during laser treatment. [Solution] The method includes the step of (i) obtaining a relationship between (i) the number of pixels related to a light beam reflected from a target or an object located in close proximity to the target on an endoscopic image acquired from a video sensor coupled to the endoscope, and (ii) the distance from the tip of the endoscope to the target. The method further includes the step of (ii) measuring the number of pixels related to the light beam reflected from the target or the object located in close proximity to the target during the procedure, and (iii) determining at least one of the size of the target or the distance from the tip of the endoscope to the target, at least in part based on the relationship acquired in step (i) and the number of pixels measured in step (ii).

Description

Technical Field

[0001] Cross - Reference to Related Applications This application claims the benefit of priority of U.S. Provisional Patent Application No. 63 / 262,290, filed on October 8, 2021, and U.S. Provisional Patent Application No. 63 / 365,886, filed on June 6, 2022, the contents of which are incorporated herein in their entirety.

[0002] This disclosure relates to determining the characteristics of a target during a laser treatment.

Background Art

[0003] During a surgical laser treatment such as laser lithotripsy, a physician may need to interact with targets of various sizes, such as tumors, stones, tissue fragments, etc. inside a patient's body. The target can be placed in a medium (e.g., a liquid medium such as water or saline), and the physician can use a scope such as an endoscope for a surgical procedure on the target. The scope can include a laser fiber tip, an endoscope image sensor, etc., attached or connected to the end of the scope.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Patent Document 2

Patent Document 3

Summary of the Invention

Problems to be Solved by the Invention

[0005] Endoscopes and other similar surgical scopes or catheters can be used in a variety of surgical procedures. For example, in ENT procedures such as tonsillectomy, sinus surgery, or other similar procedures; in procedures in the digestive system; or in procedures within the abdomen. Many endoscopic procedures, such as lithotomy or other similar procedures (e.g., tumor removal, tissue ablation, or kidney stone ablation / removal), can utilize laser light in combination with imaging sensors, such as cameras attached to the end of the scope.

[0006] When a physician performs either a diagnostic or therapeutic procedure, it is important to know the dimensions of a target, such as a tumor or stone. For example, a physician attempting to visualize a cancerous tumor may need to determine the tumor's dimensions (e.g., height and width) to determine whether the tumor can be surgically removed, and to determine baseline measurements of the tumor to monitor changes in its size after cancer treatment has begun. Similarly, a physician performing an endoscopic procedure such as ablation or removal of a stone or tissue may need to determine the size of the stone or tissue fragment to be removed to determine whether the target is small enough to be removed using a scope, such as removal through a ureter or access sheath with a specific inner diameter, or whether the target must be further reduced before removal. Furthermore, during laser procedures such as laser lithotripsy, a physician may need to know the distance between the tip of the scope and the target. The distance between the tip of the scope and the target is important, for example, to determine the amount / intensity of laser light / radiation delivered to the target and / or to monitor the condition of the medium. [Means for solving the problem]

[0007] Calibration processes performed before or at the very beginning of a medical procedure may be used to obtain a relationship between (i) pixel measurements (e.g., total number of pixels) of beam dimensions (e.g., laser beam diameter footprint, laser beam cross-sectional area, laser beam radius, or arc length of the laser beam footprint) and / or (ii) object dimensions on an endoscopic image (e.g., diameter, cross-sectional area, perimeter, radius or arc length of the laser fiber tip) and the distance from the tip of the endoscope to the target. In one example, the relationship may be between 1) a video image of light reflection from the target, or 2) a measurement of the number of pixels in a video image of an object such as a laser fiber that is very close to the target (e.g., within 1 μm, 10 μm, 100 μm, or 1 mm) or in direct contact with the target, and 1) the size of that light image, or the size of a fiber that is very close to the surface of the target (e.g., within 1 μm, 10 μm, 100 μm, or 1 mm) or in direct contact with the target, and / or 2) the distance from the tip of the endoscope to the target. Next, a calibration curve can be obtained that allows the distance from the tip of the endoscope to any target or the size of the target to be determined based on the determined relationship. As used herein, the terms “very close” or “close” refer to a situation in which an object (e.g., a laser fiber) is almost in contact with the target or is at most X mm away from the target, where X is based on the desired measurement accuracy. For example, if the desired measurement accuracy is ±1%, the object (e.g., a laser fiber) may be within 1% of the distance from the tip of the endoscope to the target. In one example, assuming that the distance between the tip of the endoscope and the target is 10 mm and the desired measurement accuracy is at least 1%, when performing the calibration process, the object (e.g., the fiber tip) may be 0.1 mm or less from the target. In general, closer distances may provide higher measurement accuracy.

[0008] In one implementation, the calibration process utilizes the fact that when the laser fiber tip is close to the target surface in the medium, the diameter of the laser aiming beam footprint is equal to the diameter of the glass of the laser fiber, which is a known diameter, and that when the endoscope and / or fiber tip is moved relative to the target, the refractive index of the medium adjacent to the target or anatomical feature corresponds to the change in the number of pixels of the aiming beam footprint diameter on the video image. As a result, a relationship can be determined between the pixel measurement of the laser beam diameter (or other parameters such as the cross-sectional area of ​​the laser beam, the laser beam radius, or the arc length of the laser beam footprint) on the endoscopic image and the distance from the endoscope tip to the target. Based on this relationship, bringing the fiber tip into contact with any target can provide the video system with a scale for measuring its size and distance from the video sensor (or the endoscope tip). In one example, bringing the fiber tip into contact with the target may not be necessary for calibration. In such an example, the system can complete the calibration process as long as the fiber tip is visible within the field of view of the video sensor attached to the endoscope.

[0009] The calibration process may involve actions or steps such as detecting the laser aiming beam circle / footprint or fiber tip at a first position / distance close to the target; measuring the diameter, cross-sectional area, or other relevant parameters (e.g., perimeter, radius, or arc length) of the laser aiming beam footprint or fiber tip at a pixel on the video image at the first position / distance; measuring the first distance or a first quantity of the laser fiber extending from the scope / video sensor; moving at least a portion of the endoscope to a second position / distance relative to the first position / distance with respect to the target; measuring the second diameter, cross-sectional area, or other relevant parameters (e.g., perimeter, radius, or arc length) of the laser aiming beam footprint or fiber tip at a pixel on the video image at the second position; and measuring the second distance or a second quantity of the laser fiber extending from the scope / video sensor at the second position / distance. Subsequently, a relationship between the pixel measurements and the distance from the tip of the endoscope to the target can be established. Before and / or during laser treatment, the distance between the target and the tip of the endoscope can be determined at least in part on the established relationship and pixel measurements of the laser beam diameter (or other parameters such as the cross-sectional area of ​​the laser beam, the radius of the laser beam, or the arc length of the laser beam footprint) acquired during treatment by bringing the tip of the laser fiber into contact with the target (or bringing the tip of the laser fiber close to the target without contacting the target).

[0010] Optionally, the method may include the step of determining the refractive index of a medium (e.g., water or saline) in close proximity to the target or anatomical feature. The refractive index may be determined based on the ratio of the initial number of pixels in the targeting beam footprint diameter (or the cross-sectional area, radius, or arc length of the laser beam footprint) to the change in the diameter of the laser targeting beam footprint (or the cross-sectional area, radius, or arc length of the laser beam footprint) between a first position and a second position. Or, to put it another way, it is the change in the diameter of the targeting beam footprint (or the cross-sectional area, radius, or arc length of the laser beam footprint) as the endoscope is moved relative to the target and / or fiber tip.

[0011] In addition, the method may further include the steps of measuring the number of pixels corresponding to beam dimensions such as the target area covered by the user beam dimension footprint, the cross-sectional area of ​​the laser beam, the laser beam radius, or the arc length of the laser beam footprint on the endoscopic image, and determining, based thereon, at least one dimension (e.g., height or width) or other characteristics of the target. For example, the pixel size may be calculated by dividing the diameter of the laser aiming beam by its corresponding total number of pixels on the endoscopic image, and the height / width of the target may be estimated based on the calculated pixel size and the total number of pixels corresponding to the height / width of the target on the endoscopic image.

[0012] As described above, the relationship between pixel measurements on the endoscopic image and the distance from the tip of the endoscope to the target can be determined using object dimensions such as the diameter, cross-sectional area, perimeter, radius, or arc length of the laser fiber tip. In this implementation, the calibration process may include the steps of measuring the dimensions of the fiber tip (e.g., the diameter, cross-sectional area, perimeter, radius, or arc length of the laser fiber tip) (first number of pixels) in pixels on the video image at a first position close to the target, and measuring a first amount of the laser fiber extending from the scope at the first position (e.g., using markings on the insulating jacket of the laser fiber that can be seen by the endoscope camera). The calibration process may then include the steps of moving at least a portion of the scope to a second position relative to the target, measuring the dimensions of the fiber tip (second number of pixels) in pixels on the video image at the second position, and measuring a second amount of the laser fiber extending from the scope at the second position.

[0013] Additionally or alternatively, calibration curves can be theoretically calculated based on known parameters such as the dimensions of the aiming beam, the number of vertical pixels in the endoscope sensor (e.g., camera), and the vertical field of view in the medium. For example, for each distance or position, the height corresponding to half an angle of the field of view in the medium can first be determined based on the field of view in the medium. From there, the number of pixels per millimeter on the endoscope image and the number of pixels corresponding to a laser fiber of a particular size can be calculated. These steps can be performed iteratively to obtain the number of pixels corresponding to a laser of a particular size at different distances or positions, thereby establishing a calibration curve. It should be noted that the parameters used in the calibration process may differ depending on whether the curve is determined empirically or theoretically. For example, if a calibration curve is empirically obtained that has a relationship between (i) pixel measurements of the laser beam dimensions and / or object dimensions on a video image and (ii) the distance from the tip of the endoscope to the target, then parameters may not be necessary.

[0014] In drawings that are not necessarily drawn to scale, like numbers may represent like components in different figures. Like numbers with different letter prefixes may represent different examples of like components. The drawings generally, by way of example and not limitation, illustrate the various embodiments discussed in this document.

Brief Description of the Drawings

[0015] [Figure 1A] A diagram showing an example of a fiber tip positioned close to a target and a corresponding endoscopic image. [Figure 1B] A diagram showing an example of a fiber tip positioned close to a target and a corresponding endoscopic image. [Figure 1C] A diagram showing an example of a calibration curve empirically obtained by the process described in FIGS. 1A and 1B. [Figure 2] A diagram showing an example of a theoretically obtained calibration curve. [Figure 3] A block diagram showing an example of a machine in which one or more embodiments may be implemented. [Figure 4] An exemplary flowchart of a method for empirically obtaining a calibration curve. [Figure 5] An exemplary flowchart of a method for theoretically obtaining a calibration curve. [Figure 6] A schematic diagram of an exemplary computer-based clinical decision support system (CDSS). [Figure 7] A diagram showing an exemplary method of using a calibration curve to determine target characteristics.

Modes for Carrying Out the Invention

[0016] Disclosed herein are systems and methods for determining target characteristics during laser treatment. The system may include a scope, such as an endoscope, a video sensor (e.g., a camera or other endoscopic imaging sensor) used at the end of the scope (e.g., the distal end), and a laser fiber inserted into or attached to the scope. In one example, the laser fiber may be a surgical fiber connected to a probe laser, an ablation laser, or any similar laser or combination of lasers used during the laser treatment. For example, an ablation laser may emit infrared radiation, and a probe laser may emit visible light to indicate the location of the tip of the scope (and therefore where the ablation energy from the ablation laser is directed) or may be used to illuminate a target. Exemplary laser fibers and systems are provided, for example, in Patent Documents 1, 2, and 3, the entire contents of which are incorporated herein by reference. The target may be a tissue fragment, debris, or an object such as a tumor or kidney stone. The fiber tip may have known dimensions, such as a diameter (e.g., 200 micrometers (μm)), which can be used to calibrate the number of pixels of the video sensor. In addition, if the laser fiber extends beyond the end of the scope to perform the laser procedure, the distance of the laser fiber extending from the video sensor (and thereby the end of the scope) can be obtained, for example, by using markings on the insulating jacket of the laser fiber that can be seen by the video sensor. Since the laser fiber is positioned close to (or, in some embodiments, in contact with) the target during the procedure, the distance of the laser fiber extending from the end of the scope is approximately the same as the distance from the tip of the endoscope to the target.A calibration curve can be obtained based on the estimated distance from the tip of the endoscope to the target and parameters such as the number of pixels related to the beam dimensions (e.g., laser beam diameter footprint, laser beam cross-sectional area, laser beam radius, or arc length of the laser beam footprint; hereafter, "beam dimensions" or "laser beam dimensions") or the object dimensions on the image acquired by the endoscope (e.g., diameter, cross-sectional area, perimeter, radius, or arc length of the laser fiber tip; hereafter, "object dimensions" or "laser fiber dimensions"). The calibration curve may be specific to the particular type of medium in which the target is placed (e.g., air, water, etc.). Therefore, once calibrated, the system may not need to be calibrated again. This calibration curve can be determined empirically by processes such as those discussed with respect to Figures 1A and 1B below, or it can be determined theoretically as discussed with respect to Figure 2 below. It should be noted that the parameters used in the calibration process may differ depending on whether the curve is determined empirically or theoretically. For example, it may not be necessary when empirically obtaining a calibration curve that has a relationship between (i) pixel measurements of the laser beam dimensions and / or object dimensions on the video image and (ii) the distance from the tip of the endoscope to the target.

[0017] Figures 1A and 1B show examples of laser fiber tips positioned close to a target and corresponding endoscopic images of the laser aiming beam footprint. In the example shown in Figure 1A, an endoscope 100 is shown with a laser fiber 102 extending 10 mm from the tip / end of the scope 100 (e.g., from a video sensor used at the distal end of the endoscope 100). The distance from the tip of the endoscope to the tip of the laser fiber 102 can be measured using one or more markings on the laser fiber 102 (e.g., marks on the insulating jacket of the fiberscope that can be seen by the endoscopic video sensor, such as multiple scales 108). When the tip of the laser fiber 102 is positioned on or near the surface of the target 104, a pixel measurement (e.g., X1 pixel) corresponding to the dimensions of the laser fiber 102 (e.g., a 200 μm fiber) can be observed on the corresponding endoscopic image 106A (based on light reflection from the target). X1 pixels can correspond to the dimensions of the laser beam footprint emitted from the laser fiber 102 and / or the dimensions of the glass at the tip of the laser fiber 102 (e.g., the diameter of the glass tip). The dimensions of the laser beam footprint can be equal to the dimensions of the glass at the tip of the laser fiber 102 if the tip of the laser fiber 102 is close to or in contact with the surface of the target 104.

[0018] While keeping the tip of the laser fiber 102 on or near the surface of the target 104, as the endoscope 100 is brought closer to the target 104 (e.g., by shortening the length of the tip of the laser fiber 102 extending from the tip of the endoscope 100 or from the video sensor used at the tip of the endoscope 100 to, for example, 5 mm), the same fiber with a diameter of 200 μm will correspond to more pixels (e.g., X2 pixels) on the endoscope image 106B. That is, as the tip of the endoscope 100 approaches the tip of the laser fiber 102 and thus the surface of the target 104, the corresponding endoscope image appears larger, as shown in endoscope images 106A and 106B. Further, as shown in FIGS. 1A and 1B, depending on how much the laser fiber 102 extends, a different number of multiple graduations 108 can be seen, which can provide a distance indication from the tip of the endoscope 100 to the tip of the laser fiber 102 at each position relative to the target 104.

[0019] Figure 1C shows an example of a calibration curve empirically obtained by the process described in Figures 1A and 1B. As a result of the process described above for Figures 1A and 1B, the relationship between the pixel measurements of the dimensions of the laser beam footprint on the endoscopic images 106A and 106B and the distance from the tip of the endoscope 100 to the target can be determined. Once the pixel measurement values ​​and distance values ​​are plotted as shown in Figure 1C, a calibration curve 108 can be obtained. Based on the calibration curve 108, subsequently bringing the tip of the laser fiber 102 into contact with any target (or placing the target within the field of view of the video sensor) can provide a scale for the video system to measure its distance from the video sensor (or the tip of the endoscope 100) using the pixel measurements of the dimensions of the laser beam footprint on the endoscopic image. In addition, the pixel size can be calculated based on the dimensions of the laser aiming beam footprint and the corresponding number of pixels on the endoscopic images 106A and 106B, and the dimensions of target 104 can be estimated based on this calculation and the number of pixels corresponding to target 104 on the endoscopic images 106A and 106B. Figures 1A to 1C illustrate the use of a laser fiber to establish a calibration curve 108, but it should be noted that any suitable object having known dimensions and located very close to the surface of the target (e.g., 1 μm, 10 μm, 100 μm, or 1 mm) can be used to establish a calibration curve in a medium based on the method described above, and is therefore within the scope of the present invention.

[0020] In an alternative example, calibration can occur as long as the tip of the laser fiber 102 (or any suitable object) is within the field of view of the video sensor used at the tip of the endoscope 100, regardless of whether the tip of the laser fiber 102 (or object) is in contact with the target 104. This can be determined because the medium is known, and therefore the cone of the field of view within the medium is known as well as the dimensions of the laser fiber 102 (or object). Thus, calibration can be performed based on these known parameters and the design of the endoscope 100 and the laser fiber 102. For example, if the laser fiber 102 appears just at the edge of the field of view, it can be determined that the laser fiber 102 extends at a distance of 3-4 mm from the tip of the endoscope 100. This is a calibration based on the design of the endoscope 100 and known dimensions such as the diameter of the laser fiber 102 (e.g., 200 microns, 365 microns, etc.). Similarly, if the tip of the laser fiber 102 is close to the center of the field of view, it can be determined that the laser fiber 102 extends at a distance of 8-10 mm from the tip of the endoscope 100. The exact distance at which the tip of the laser fiber 102 extends from the tip of the endoscope 100 can be determined from the markings on the laser fiber 102, as discussed above.

[0021] In various embodiments, calibration curves for different types of media (e.g., water, saline, or air) can be established using the methods described above. The endoscope 100 operates in one of these media (e.g., water, saline, or air) during laser treatment. The type of media may be entered by the user before and / or during laser treatment in a control system, user interface, etc., coupled to / connected to the endoscope 100. Additionally or alternatively, the type of media may be determined based on divergence. For example, water and air have different refractive indices, such that objects appear larger in water than in air. Thus, the type of media may be determined by comparing how much larger an object appears in the media with its actual size, and a calibration curve corresponding to that media may be used.

[0022] The medium may vary depending on the medical procedure being performed. For example, for gastrointestinal (GI) procedures and gynecological procedures, the medium is air. Therefore, in one embodiment, the system can automatically select / select the appropriate medium and its corresponding calibration curve based on the procedure selected by the user or entered into the system. For example, a surgeon may identify a GI procedure, and the system can automatically select air and a calibration curve established for air. Conversely, if the procedure is to treat gallstones, the system can select saline as the medium and obtain the corresponding calibration curve. The size of any targets found during the procedure (e.g., lesions, stones, etc.) can then be estimated using the calibration curve corresponding to the selected medium.

[0023] During medical procedures, the medium can change. For example, during a GI procedure, once the endoscope enters the bile duct, the surgeon may apply saline irrigation. In such a situation, the system can provide the surgeon with both options (e.g., air or saline) to choose from. In another example, the system may automatically select a new medium type depending on which medium, air or saline, is more prevalent. For example, if saline is applied and there is more saline than air in the bile duct, the system can determine that a new type of medium is present. The system can then recalibrate the number of pixels in the video sensor, at least partially based on the change in medium, using the configuration method described above. Based on the recalibration, the system then adjusts the determined size of the target (e.g., target height or width) and / or the distance from the tip of the scope to the target. Alternatively, upon detecting a new type of medium (for example, receiving a signal indicating that a pump for applying saline solution to a patient's body has been activated) and / or receiving input from a surgeon indicating the presence of a new type of medium, the system may automatically acquire a calibration curve corresponding to the new type of medium established before or during the procedure, and based on the acquired calibration curve, estimate the size of the target and / or the distance from the tip of the scope to the target.

[0024] In some situations, a surgeon may irrigate the anterior end of the scope or the target to clean it, but the medium does not actually change. Therefore, before adjusting the medium type and changing the calibration curve, the surgeon or the system can check to determine whether the medium has actually changed. For example, if a laser is used, short, low-energy pulses may be emitted. If vapor bubbles are present, the medium is determined to be water or saline. If vapor bubbles are absent, the medium is air. In one example, such a medium check can be performed at any time during the procedure, and if there is an error in the selection of the medium, the medium type can be automatically corrected by the system and / or manually corrected by the surgeon. From there, the system can use the correct calibration curve (corresponding to the medium present in the patient's body) and correct the estimated size and / or distance to the target using the correct calibration curve.

[0025] In one example, the system can make treatment recommendations based on the size of the target (e.g., lesion or polyp) and present these recommendations to the user for approval or selection. For example, depending on the size of a polyp estimated using the method described above, one treatment option might be to "snare" the polyp. This type of procedure may be feasible if the polyp is below a certain size, but a different approach may be taken if the polyp exceeds a certain size. In another example, in gynecological procedures, the size of a uterine fibroid can determine what type of treatment should be applied or used.

[0026] Similarly, in laser procedures, laser settings may vary based on the type and / or size of the target (e.g., a gallstone). For example, if the gallstone is large, the surgeon may decide to break the gallstone into smaller fragments and then switch settings (e.g., laser intensity) to reduce the smaller fragments to dust. Once the size of the target is estimated using the methods described above, the system may recommend first dividing the target into smaller fragments. The surgeon can then choose to accept or reject this recommendation. Alternatively, the system may automatically perform the proposed procedure based on the estimated target size. Regardless of whether the surgeon accepts the system's recommendation or the system automatically performs the recommended procedure, the system may automatically, or upon receiving approval, adjust the laser intensity or other parameters (e.g., pulse width, duty cycle, frequency, etc.) to divide the target and reduce the divided fragments of the target.

[0027] Such a process has the advantage of significantly reducing the time required to perform medical procedures. For example, breaking up a large stone into fragments and then reducing the smaller fragments to dust can reduce the overall procedure time by 60-70 percent compared to the time required to reduce the large stone itself to dust without fragmenting it. Therefore, enabling a system to automatically estimate the target size and perform the corresponding procedure based on the target size can allow medical procedures to proceed and be completed more quickly and efficiently.

[0028] Figure 2 shows an example of a theoretically obtained calibration curve 200. In the example shown in Figure 2, various parameters may be known, such as the dimensions of the laser beam footprint, including the 200 μm laser beam footprint / diameter of the targeting beam, the number of vertical pixels related to the endoscope sensor, and the vertical field of view in the medium (based on the medium used and the refractive index of the laser fiber). For example, the diameter of the targeting beam emitted from the 200 μm laser fiber may be known to be 0.4 mm at the target. Furthermore, the endoscope video sensor may be known to have an array of a specific size, such as a 200 × 200 array, and the vertical field of view of the video sensor in water may be known to be 70 degrees. Using this information, the distance from the tip of the endoscope to the target, and the total number of pixels of the laser beam diameter on the endoscope image can be plotted to obtain the calibration curve 200.

[0029] More specifically, based on known parameters, the height corresponding to the half-angle of the field of view in the medium can be calculated for each distance. For example, at each distance, the corresponding height of the field of view can be calculated by multiplying the distance by the tangent of the half-angle of the vertical field of view (35 degrees). The number of pixels per millimeter can be calculated, for example, using 100 pixels divided by the height of the field of view (i.e., the number of pixels in half of a 200x200 array). Multiplying this number by the diameter of the aiming beam (e.g., 0.4 mm) can provide the number of pixels per 200 μm fiber at a given distance.

[0030] Once the distance and the number of pixels are plotted, a calibration curve 200 similar to the calibration curve 108 in Figure 1C is obtained and can be used to determine the distance from the tip of the scope to the target at any given distance. The advantage of this calibration method is that, since the relationship between the number of pixels of the laser beam diameter on the endoscopic image and the distance from the tip of the endoscope to the target correlates with the refractive index of the medium near the target, once the system is calibrated, it does not need to be recalibrated (since treatment is always performed endoscopically in a medium such as water).

[0031] Figure 3 is a block diagram of an exemplary machine 300 on which any one or more of the techniques (e.g., methodologies) discussed herein may be performed. In alternative embodiments, machine 300 may operate as a standalone device or be connected to other machines (e.g., network connection). In a network deployment, machine 300 may operate as a server machine, a client machine, or both in a server-client network environment. In one example, machine 300 may function as a peer machine in a peer-to-peer (P2P) (or other distributed) network environment. Machine 300 may be a personal computer (PC), a tablet PC, a set-top box (STB), a personal digital assistant (PDA), a mobile phone, a web appliance, a network router, a switch or bridge, or any machine capable of executing instructions (sequential or otherwise) specifying actions to be performed by that machine. Furthermore, although only a single machine is shown, the term “machine” shall be construed to include any set of machines that individually or jointly execute a set of instructions (or sets of instructions) to perform any one or more of the methodologies discussed herein, such as cloud computing, software as a service (SaaS), and other computer cluster configurations.

[0032] The examples described herein include or can operate through logic or several components or mechanisms. A circuit set is a collection of circuits implemented in a tangible entity that includes hardware (e.g., simple circuits, gates, logic, etc.). Membership in a circuit set can be flexible with respect to variability over time and the underlying hardware. A circuit set includes members that can perform specified operations individually or in combination when in operation. In one example, the hardware of a circuit set may be designed immutably (e.g., hardwired) to perform a particular operation. In one example, the hardware of a circuit set may include variably connected physical components (e.g., execution units, transistors, simple circuits, etc.) including a computer-readable medium that has been physically modified (e.g., magnetic, electrical, or movable arrangement of particles of invariant mass) to encode instructions for a particular operation. When connecting to physical components, the underlying electrical properties of the hardware components are changed, for example, from an insulator to a conductor, or vice versa. Instructions allow embedded hardware (e.g., an execution unit or loading mechanism) to create members of the circuit set in the hardware via variable connections to perform a portion of a particular operation when in operation. Therefore, the computer-readable medium is communicatively coupled to other components of the circuit set members when the device is operating. In one example, any of the physical components may be used in two or more members of two or more circuit sets. For example, during operation, the execution unit may be used in a first circuit of a first circuit set at one point in time, and then reused at a different point in time by a second circuit in the first circuit set or by a third circuit in the second circuit set.

[0033] The machine (e.g., computer system) 300 may include a hardware processor 302 (e.g., a central processing unit (CPU), a graphics processing unit (GPU), a hardware processor core, a field-programmable gate array (FPGA), or any combination thereof), main memory 304, and static memory 306, some or all of which may communicate with each other via an interlink (e.g., a bus) 330. The machine 300 may further include a display unit 310, an alphanumeric input device 312 (e.g., a keyboard), and a user interface (UI) navigation device 314 (e.g., a mouse). In one example, the display unit 310, the input device 312, and the UI navigation device 314 may be touchscreen displays. Machine 300 may further include a storage device (e.g., a drive unit) 308, a signal generating device 318 (e.g., a speaker), a network interface device 320, and one or more sensors 316 such as a Global Positioning System (GPS) sensor, a compass, an accelerometer, or other sensors. Machine 300 may also include an output controller 328 such as a serial (e.g., Universal Serial Bus (USB)) connection, a parallel connection, or other wired or wireless (e.g., infrared (IR), near-field communication (NFC)) connection for communicating with or controlling one or more peripheral devices (e.g., a printer, a card reader, etc.).

[0034] The storage device 308 may include a machine-readable medium 322 in which one or more sets of data structures or instructions 324 (e.g., software) that embody any one or more of the techniques or functions described herein, or that are used by them, are stored. The instructions 324 may reside, all or at least partially, in the main memory 304, static memory 306, or hardware processor 302 during their execution by machine 300. In one example, one or any combination of the hardware processor 302, main memory 304, static memory 306, or storage device 316 may constitute the machine-readable medium.

[0035] Although the machine-readable medium 322 is shown as a single medium, the term “machine-readable medium” may include one or more mediums configured to store one or more instructions 324 (e.g., a centralized or distributed database, and / or associated caches and servers).

[0036] The term “machine-readable medium” may include any medium capable of storing, encoding, or carrying instructions for execution by machine 300, causing machine 300 to execute any one or more of the techniques of the Disclosure, or any medium capable of storing, encoding, or carrying data structures used by or associated with such instructions. Non-limiting machine-readable mediums may include solid-state memory, as well as optical and magnetic media. In one example, a massed machine-readable medium comprises a machine-readable medium having a plurality of particles having immutable (e.g., stationary) mass. Thus, a massed machine-readable medium is not a transient propagating signal. Specific examples of massed machine-readable mediums may include non-volatile memory such as semiconductor memory devices (e.g., electrically programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), and flash memory devices), magnetic disks such as internal hard disks and removable disks, magneto-optical disks, and CD-ROM and DVD-ROM disks.

[0037] Instruction 324 may further be transmitted or received over a communication network 326 using a transmission medium via a network interface device 320 that utilizes any one of several transport protocols (e.g., Frame Relay, Internet Protocol (IP), Transmission Control Protocol (TCP), User Datagram Protocol (UDP), Hypertext Transmission Protocol (HTTP), etc.). Illustrative communication networks may include, among others, local area networks (LANs), wide area networks (WANs), packet data networks (e.g., the Internet), mobile phone networks (e.g., cellular networks), plain old telephone (POTS) networks, and wireless data networks (e.g., the Institute of Electrical and Electronics Engineers (IEEE) 802.11 family of standards, IEEE 802.16 family of standards, IEEE 802.15.4 family of standards, and peer-to-peer (P2P) networks, known as Wi-Fi®). In one example, the network interface device 320 may include one or more physical jacks (e.g., Ethernet jacks, coaxial jacks, or telephone jacks) or one or more antennas for connecting to the communication network 326. In one example, the network interface device 320 may include multiple antennas for wireless communication using at least one of the following techniques: single-input multiple-output (SIMO), multiple-input multiple-output (MIMO), or multiple-input single-output (MISO). The term "transmission medium" shall be interpreted as including any intangible medium that can store, encode, or carry instructions for execution by machine 300, and which includes digital or analog communication signals, or other intangible mediums that facilitate communication of such software.

[0038] The calibration curves discussed in Figure 1C and / or Figure 2 can be obtained empirically or theoretically. The calibration curve can represent the relationship between (i) the number of pixels representing the laser beam diameter (or other dimensions described above) on the image, or the number of pixels representing the dimensions of an object (such as the diameter, cross-sectional area, perimeter, radius, or arc length of an object) on the image (e.g., an endoscopic image), and (ii) the distance from the tip of the scope to the target. Endoscopic images can be obtained from a video sensor, such as a camera coupled to the endoscope (e.g., located at the distal end of the endoscope).

[0039] Figure 4 shows an exemplary flowchart of a method 400 for empirically obtaining a calibration curve according to the present invention. Operation 402 includes measuring a first number of pixels from the endoscope sensor at a first position. In one example, the first number of pixels may be the number of pixels of the dimensions (or other dimensions described above) of the laser beam aiming footprint, or the number of pixels of an object (e.g., a portion of a surgical fiber, such as the tip of a surgical fiber) located very close to (or, in some embodiments, in contact with) the target on the endoscope image (e.g., an image taken from a video sensor such as a camera associated with the endoscope). The first position may be a first distance from the target. The distance from the target may be measured in operation 404. The measurement may be based on the amount of laser fiber extending from the tip of the scope, for example, using one or more markings on the insulating jacket of the fiberscope that may be visible by the endoscope video sensor. Operation 406 may include moving at least a portion of the endoscope or fiber tip to a second position relative to the first position. In one example, the distal end of the endoscope (e.g., the end of the endoscope from which the fiber tip protrudes and / or the video sensor is located) may be moved away from or closer to the target. Additionally or alternatively, the portion of the fiber tip extending from the tip of the endoscope may be extended or made longer, or retracted or made shorter, so that the tip of the endoscope is closer to or further away from the target at a second position.

[0040] Operation 408 may include measuring a second number of pixels from the endoscope sensor at a second position, and operation 410 may include measuring a second distance from the target (e.g., the distance from the target at the second position). As discussed above with respect to Figures 1A and 1B, at each of the first and second positions there will be a corresponding number of pixels on the endoscope image (X1 pixels at the first position and X2 pixels at the second position). In general, the closer the tip of the scope and / or fiber is to the target, the more pixels there will be on the corresponding endoscope image. For example, at 10 millimeters, the number of pixels may be 14.3 pixels per millimeter, and at 5 millimeters, the number of pixels may be 28.6 pixels per millimeter. Operation 412 may include plotting a first pixel count, a first distance, a second pixel count, and a second distance to obtain a calibration curve representing the relationship between the number of pixels of the dimensions of the laser beam aiming footprint (or other dimensions described above), or the number of pixels of an object located very close to (or, in some embodiments, in contact with) the target on the endoscopic image, and the distance from the tip of the endoscope to the target. From this determined relationship, the distance from the target and / or the target dimensions can be determined when the tip of the scope is at any distance from the target. Figures 1A, 1B, and 4 show measurements at two locations, but it should be noted that the calibration curve can be established based on measurements at three or more locations. Generally, the more locations used, the more accurate the calibration curve will be.

[0041] In one example, the dimensions of the laser beam footprint and / or aiming beam may be known or measured at a specific distance from the target. Similarly, the number of pixels in a video sensor array may be a known quantity, as may the field of view of the video sensor in a given medium. For example, as discussed above, a 200x200 array video sensor can have a 70-degree field of view in water. These known values / quantities can be used to theoretically generate calibration curves, as discussed with respect to Figure 2.

[0042] Figure 5 shows an exemplary flowchart of method 500 for theoretically obtaining a calibration curve. Operation 502 may include an operation to determine the aiming beam dimensions of the laser fiber. For example, a 200 μm laser fiber may have a aiming beam with a diameter of 0.4 mm, which may be a known quantity or can be measured directly by pointing the laser at a target. Operation 504 may include an operation to determine the endoscope array size. This may also be a known quantity based on the type of video sensor used in the endoscope (e.g., camera). For example, the video sensor may be known to have a 200 × 200 array. Operation 506 may include an operation to determine the refractive index of a medium such as air or water. The type of medium may be determined based on the medical procedure being performed. For example, for gastrointestinal (GI) procedures and gynecological procedures, the medium is air. Thus, the system may automatically select / select the appropriate medium based on the procedure selected by the user or entered into the system. For example, a surgeon may identify a GI procedure, and the system may automatically select air. Conversely, if the procedure is to treat gallstones, the system can select saline solution as the medium. Based on the medium, the vertical field of view within the medium can be estimated.

[0043] Based on these (or other similar) known parameters, operation 508 may include an operation to determine the height corresponding to a half-angle field of view in the medium at a given distance. Based on this, the number of pixels per millimeter in the half-angle field of view can be calculated. For example, a 200x200 array may be known to have a vertical field of view of 70 degrees (and thus a half-angle of 35 degrees) in water. Therefore, at any distance, the height of the corresponding field of view can be calculated by multiplying the distance by the tangent of the half-angle of the vertical field of view in the medium (35 degrees). The number of pixels per millimeter can then be calculated using 100 pixels (the number of pixels in half of a 200x200 array), which is obtained by dividing the field of view height by twice.

[0044] Operation 512 may include determining the number of pixels corresponding to a fiber having a known diameter of the aiming beam, and plotting the number of pixels versus distance to obtain a calibration curve. In one example, the number of pixels per millimeter may be multiplied by the diameter of the aiming beam (e.g., 0.4 mm) to calculate the number of pixels per 200 μm at a particular distance. The number of pixels per fiber may be plotted at a particular distance to generate a calibration curve, such as that discussed with respect to Figure 2. Again, based on the calibration curve, the distance from the target and / or the dimensions of the target may be determined regardless of where the scope or fiber is positioned.

[0045] Essentially, as long as the fiber (or other suitable object) is visible within the video sensor's field of view, a calibration curve can be obtained, since the cone of the array's field of view is known for a particular medium. For example, if the fiber is at the edge of the field of view, the distance from the endoscope tip to the fiber is known (based on the scope's design), and the diameter of the fiber (or other suitable object) (or other dimensions discussed above) is known. Calibration can therefore be performed based on these known parameters and parameters specific to the medium in which the scope is positioned.

[0046] Figure 6 shows an exemplary computer-based clinical decision support system (CDSS) 600 configured to determine target-related information, such as the distance from the tip of the scope to the target and the size of the target, based on the number of pixels in an image captured by a video sensor coupled to or attached to the scope, or any other similar information about the target. In various embodiments, the CDSS 600 includes an input interface 602 to which information specific to the patient procedure, such as the size of the surgical fiber, information about a video or imaging sensor, e.g., the size of the camera array, and / or information about the scope, is provided as input features to an artificial intelligence (AI) model 604; a processor that performs inference operations on which parameters are applied to the AI ​​model to generate a determination of the distance from the target and / or the size of the target; and an output interface 608 to which the determined distance from the target or the size of the target is communicated to a user, e.g., a clinician.

[0047] In some embodiments, the input interface 602 may be a direct data link between the CDSS 600 and one or more medical devices that generate at least some of the input features. For example, the input interface 602 may directly transmit the dimensions or size of a surgical fiber to the CDSS 600 during a therapeutic and / or diagnostic medical procedure. In one example, information about the surgical fiber and / or scope used during the procedure may be stored in the database 606. Additionally or alternatively, the input interface 602 may be a classic user interface that facilitates interaction between the user and the CDSS 600. For example, the input interface 604 may facilitate a user interface in which the user can manually input information about the surgical fiber and / or scope. Additionally or alternatively, the input interface 602 may provide the CDSS 600 with access to an electronic patient record, or the scope and / or surgical fiber used during the procedure, as well as any of the dimensions discussed above from which one or more input features can be extracted. In any of these cases, the input interface 602 is configured to collect the following input features in relation to one or more of the following: a specific patient, a type of medical procedure, a type of scope, a type of video sensor connected to the scope, or a type of surgical fiber to be used, at or before the time when the CDSS 600 is used to evaluate information regarding the size of the surgical fiber, a video or imaging sensor, a scope, or a medium in which the procedure is performed:

[0048] Examples of input features may include the dimensions of the surgical fiber used during the procedure.

[0049] Examples of input features may include parameters such as the type of video or imaging sensor coupled to the scope, and the size of the sensor's camera or video array.

[0050] Examples of input features can include the type of scope used during the procedure.

[0051] Examples of input features can include the amount of surgical fiber extending from the tip of the scope.

[0052] Examples of input features may include the type of medium in which the procedure is performed and / or scope is set.

[0053] An example of an input feature could be a video image from the video sensor 610.

[0054] An example of an input feature could include the number of pixels in the video sensor's field of view, which is 612.

[0055] Based on one or more of the above input features, the processor uses the AI ​​model 604 to perform inference operations to generate a determined distance from the tip of the scope to the target and / or the size of the target. For example, the input interface 602 may deliver one or more of the above input features to the input layer of the AI ​​model 604, and the input layer of the AI ​​model 604 propagates these input features through the AI ​​model 604 to the output layer. The AI ​​model 604 can provide a computer system with the ability to perform tasks without being explicitly programmed by performing inference based on patterns found in the analysis of the data. The AI ​​model 604 explores the study and construction of algorithms (e.g., machine learning algorithms) that can learn from existing data and make predictions about new data. Such algorithms work by building an AI model from exemplary training data to make data-driven predictions or decisions, which are expressed as outputs or evaluations.

[0056] Machine learning (ML) has two common modes: supervised ML and unsupervised ML. Supervised ML uses prior knowledge (e.g., associating inputs with outputs or results) to learn the relationship between inputs and outputs. The goal of supervised ML is to learn a function that best approximates the relationship between training inputs and outputs, so that given some training data, the ML model can implement the same relationship when given inputs to produce the corresponding outputs. Unsupervised ML trains ML using unclassified and unlabeled information, allowing the algorithm to act on that information without guidance. Unsupervised ML is useful in exploratory analysis because it can automatically identify structures within the data.

[0057] Common tasks in supervised machine learning are classification and regression problems. Classification problems, also called categorization problems, aim to classify items into one of several categorical values ​​(e.g., Is this object an apple or an orange?). Regression algorithms aim to quantify some items (e.g., by providing a score for some input values). Some commonly used examples of supervised machine learning include logistic regression (LR), Naive Bayes, random forest (RF), neural networks (NN), deep neural networks (DNN), matrix factorization, and support vector machines (SVM).

[0058] Some common tasks in unsupervised machine learning include clustering, representation learning, and density estimation. Some examples of commonly used unsupervised machine learning algorithms are K-means clustering, principal component analysis, and autoencoders.

[0059] Another type of machine learning is federative learning (also known as collaborative learning), which trains algorithms across multiple distributed devices that hold local data without exchanging data. This approach contrasts with traditional centralized machine learning techniques where all local datasets are uploaded to a single server, as well as more classical distributed techniques that often assume local data samples are identically distributed. Federative learning allows multiple actors to build a common, robust machine learning model without sharing data, and thus enables addressing critical issues such as data privacy, data security, data access rights, and access to heterogeneous data.

[0060] In some cases, the AI ​​model 604 may be trained continuously or periodically before the processor performs inference calculations. Then, during the inference calculations, patient-specific input features provided to the AI ​​model 604 may be propagated from the input layer, through one or more hidden layers, to the output layer, which ultimately corresponds to information about the target. For example, when evaluating an image, the system may determine the number of pixels on the endoscopic image, how far the surgical fiber extends from the tip of the endoscope, the dimensions of the laser aiming beam footprint on the endoscopic image, the height of the video sensor's field of view, and a calibration curve. The system can then use the calibration curve to determine the distance from the tip of the scope and / or surgical fiber to the target, and characteristics of the target, such as the height or width of the target.

[0061] During and / or after the inference operation, information about the target can be communicated to the user via the output interface 608 (e.g., the user interface (UI)) and / or cause the surgical laser connected to the processor to automatically perform the desired action. For example, based on the size of the target, the system may cause the surgical laser to emit energy to ablate the target, adjust the amount of ablation energy, or move a portion of the scope.

[0062] Figure 7 shows an exemplary method for determining target characteristics according to this specification. Operation 701 may include measuring the number of pixels related to beam dimensions (e.g., laser beam diameter footprint, laser beam cross-sectional area, laser beam radius, or arc length of the laser beam footprint) and / or (ii) object dimensions on the endoscopic image in the laser procedure (e.g., diameter, cross-sectional area, perimeter, radius, or arc length of the laser fiber tip). Operation 702 may include obtaining a calibration curve for the surgical fiber. The calibration curve may be obtained theoretically or empirically by methods such as those described in Figures 1A–1C, Figure 2, or Figures 4–5, or by any other suitable process. Operation 704 may include using the measured number of pixels on the endoscopic image obtained in operation 701 and the calibration curve obtained in operation 702 to determine the distance from the target. Additionally or alternatively, operation 706 may include determining the size of the target, such as target characteristics, such as target height or width. For example, the pixel size can be calculated by dividing the diameter of the laser aiming beam by the corresponding total number of pixels on the endoscopic image, and the height / width of the target can be estimated based on the calculated pixel size and the total number of pixels corresponding to the height / width of the target on the endoscopic image. In one example, once the calibration process is complete, the aiming beam can be projected from the fiber tip onto the surface of the target, and this projection conforms to the calibrated parameters. Thus, the user can know how many pixels are visible based on the amount of the fiber tip that can be seen in the endoscopic image, and therefore how far the fiber tip and / or the tip of the scope are located from the target. Similarly, the user can know the number of pixels corresponding to the dimensions of the aiming beam on the surface of the target and can determine the height and / or width of the target in pixels, which can be used to determine the estimated size of the target.

[0063] For example, once a calibration curve is obtained in operation 702, whether it was obtained theoretically or empirically, the calibration curve may be used in operation 704 to determine the distance from the target, or in operation 706 to determine the target characteristics. Or, to put it another way, operations 704 and 706 may be performed independently of each other or in any order (for example, the size of the target may be determined first, then the distance from the target may be determined), or operations 704 and 706 may be performed simultaneously or substantially simultaneously with each other.

[0064] Additional notes and examples Example 1 is a method for acquiring target characteristics, comprising the steps of (i) acquiring a relationship between (i) the number of pixels associated with a light beam reflected from a target or an object located in close proximity to a target on an endoscopic image acquired from a video sensor coupled to the endoscope, and (ii) acquiring a relationship between the distance from the tip of the endoscope to the target, (ii) measuring the number of pixels associated with a light beam reflected from a target or an object located in close proximity to a target during the procedure, and (iii) determining at least one of the size of the target or the distance from the tip of the endoscope to the target, at least in part based on the relationship acquired in step (i) and the number of pixels measured in step (ii).

[0065] In Example 2, the subject of Example 1 optionally includes the steps of obtaining a relationship between (i) the number of pixels related to a light beam reflected from a target or an object located close to a target on an endoscopic image and (ii) the distance from the tip of the endoscope to the target, the steps of: measuring a first number of pixels related to the dimensions of the light beam or the dimensions of the object when the target is at a first distance from the tip of the endoscope; measuring a first distance from the tip of the endoscope to the target; measuring a second number of pixels related to the dimensions of the light beam or the dimensions of the object when the target is at a second distance from the tip of the endoscope; measuring a second distance from the tip of the endoscope to the target; and establishing a relationship based at least in part on the measured first number of pixels, second number of pixels, first distance, and second distance.

[0066] In Example 3, the subject of Example 2 optionally includes that the dimensions of the light beam include at least one of the following: the diameter footprint of the laser beam, the cross-sectional area of ​​the laser beam, the radius of the laser beam, or the arc length of the laser beam footprint.

[0067] In Example 4, any one or more subjects from Examples 2-3 optionally include the fact that the dimensions of the object include at least one of the following: the diameter, cross-sectional area, perimeter, radius, or arc length of the laser fiber tip.

[0068] In Example 5, any one or more subjects from Examples 1-4 may optionally include a step of recommending a treatment procedure based at least partially on the determined size of the target and / or the determined distance from the tip of the endoscope to the target.

[0069] In Example 6, any one or more subjects from Examples 1-5 optionally include a step of determining the type of medium in which the target is located, and the relationship obtained is at least partially based on a calibration curve corresponding to the type of medium.

[0070] In Example 7, the subject of Example 6 optionally includes the fact that the type of medium is determined at least in part on one of the following: the type of medical procedure, a comparison of the observed size of the target with the actual size of the target, or user input.

[0071] In Example 8, any one or more subjects from Examples 6-7 optionally include the type of medium being at least one of water, air, or saline solution.

[0072] In Example 9, any one or more subjects from Examples 1 to 8 optionally include the steps of determining whether a new type of medium exists on which the target is located, and if so, (i) obtaining an updated relationship corresponding to the new type of medium, and (ii) determining the size of the target and / or the distance from the tip of the endoscope to the target, at least in part, based on the updated relationship.

[0073] In Example 10, any one or more subjects from Examples 1 to 9 optionally include the step of obtaining a relationship between (i) the number of pixels associated with a light beam reflected from a target or an object located in close proximity to a target on an endoscopic image and (ii) the distance from the tip of the endoscope to the target, the step of obtaining (i) the dimensions of the light beam or the dimensions of the object, (ii) the number of pixels corresponding to the video sensor, and (iii) the reflectance of the medium on which the target is located, the step of calculating the number of pixels associated with the dimensions of the light beam or the dimensions of the object for each distance from the tip of the endoscope to the target, and the step of establishing a relationship between the distance from the tip of the endoscope to the target and the corresponding number of pixels associated with the dimensions of the light beam or the dimensions of the object, at least in part.

[0074] In Example 11, any one or more subjects from Examples 1 to 10 optionally include the steps of: calculating the pixel size associated with the endoscopic image; determining the number of pixels corresponding to a target on the endoscopic image; and determining the size of the target, at least in part, based on the calculated pixel size and the determined number of pixels corresponding to the target.

[0075] Example 12 is a system comprising at least one processor and memory coupled to at least one processor, wherein the memory is configured to store instructions that, when executed by at least one processor, cause at least one processor to perform an operation, the operation comprising: (i) obtaining a relationship between (i) the number of pixels related to a light beam reflected from a target or an object located in close proximity to a target on an endoscope image acquired from a video sensor coupled to the endoscope and (ii) the distance from the tip of the endoscope to the target; (ii) measuring the number of pixels related to a light beam reflected from a target or an object located in close proximity to a target during the procedure; and (iii) determining at least one of the size of the target or the distance from the tip of the endoscope to the target, at least in part based on the relationship obtained in step (i) and the number of pixels measured in step (ii).

[0076] In Example 13, the subject of Example 12 optionally includes the steps of: (i) obtaining a relationship between the number of pixels related to a light beam reflected from a target or an object located in close proximity to a target on an endoscopic image and (ii) the distance from the tip of the endoscope to the target, the steps of: measuring a first number of pixels related to the dimensions of the light beam or the dimensions of the object when the target is at a first distance from the tip of the endoscope; measuring a first distance from the tip of the endoscope to the target; measuring a second number of pixels related to the dimensions of the light beam or the dimensions of the object when the target is at a second distance from the tip of the endoscope; measuring a second distance from the tip of the endoscope to the target; and establishing a relationship based at least in part on the measured first number of pixels, second number of pixels, first distance, and second distance.

[0077] In Example 14, the subject of Example 13 optionally includes that the dimensions of the light beam include at least one of the following: the diameter footprint of the laser beam, the cross-sectional area of ​​the laser beam, the radius of the laser beam, or the arc length of the laser beam footprint.

[0078] In Example 15, any one or more subjects from Examples 13-14 optionally include the fact that the dimensions of the object include at least one of the diameter, cross-sectional area, perimeter, radius, or arc length of the laser fiber tip.

[0079] In Example 16, any one or more subjects from Examples 12–15 optionally include a further step in which the action recommends a therapeutic procedure based at least partially on the determined size of the target and / or the determined distance from the tip of the endoscope to the target.

[0080] In Example 17, any one or more subjects from Examples 12-16 optionally include a step in which the operation determines the type of medium in which the target is located, and the relationship obtained is at least partially based on a calibration curve corresponding to the type of medium.

[0081] In Example 18, the subject of Example 17 optionally includes the fact that the type of medium is determined at least in part on one of the following: the type of medical procedure, a comparison of the observed size of the target with the actual size of the target, or user input.

[0082] In Example 19, any one or more subjects from Examples 12-18 optionally include the operation further comprising the steps of determining whether a new type of medium on which the target is located exists, and if so, (i) obtaining an updated relationship corresponding to the new type of medium, and (ii) determining the size of the target and / or the distance from the tip of the endoscope to the target, at least in part on the updated relationship.

[0083] In Example 20, any one or more subjects from Examples 12 to 19 optionally include the step of obtaining a relationship between (i) the number of pixels associated with a light beam reflected from a target or an object located in close proximity to a target on an endoscopic image and (ii) the distance from the tip of the endoscope to the target, which includes the steps of obtaining (i) the dimensions of the light beam or the dimensions of the object, (ii) the number of pixels corresponding to the video sensor, and (iii) the reflectance of the medium on which the target is located.

[0084] In Example 21, any one or more subjects from Examples 12 to 20 optionally further include the steps of: calculating the pixel size related to the endoscopic image; determining the number of pixels corresponding to a target on the endoscopic image; and determining the size of the target, at least in part, based on the calculated pixel size and the determined number of pixels corresponding to the target.

[0085] The above detailed description includes references to accompanying drawings that form part of the detailed description. The drawings illustrate, as examples, specific embodiments that may be implemented. These embodiments are also referred to herein as “examples.” Such examples may include elements in addition to those illustrated or described. However, the inventors also intend examples in which only those illustrated or described elements are provided. Furthermore, the inventors also intend examples using any combination or permutation of those elements (or one or more embodiments thereof) illustrated or described in relation to a particular example (or one or more embodiments thereof) or in relation to other examples (or one or more embodiments thereof) illustrated or described herein.

[0086] All publications, patents, and patent documents referenced herein are incorporated herein by reference in their entirety, as if they were incorporated individually by reference. If there is any inconsistency in usage between this document and those documents incorporated by reference, the usage in the incorporated reference shall be deemed to supersede the usage in this document, and in the event of any conflicting inconsistency, the usage in this document shall prevail.

[0087] In this document, the terms “a” or “an” are used to include one or more, independently of any other instances or uses of “at least one” or “one or more,” as is common in patent documents. In this document, the term “or” is used to refer to non-exclusive or, unless otherwise indicated, such as “A or B” including “A but not B,” “B but not A,” and “A and B.” In the attached claims, the terms “including” and “in which” are used as plain English equivalents of the terms “equipped with” and “wherein.” Also in the attached claims, the terms “including” and “equipped with” are unrestrictive; that is, any system, device, article, or process that includes elements in addition to those listed after such terms in a claim is still considered to be within the scope of that claim. Furthermore, in the attached claims, terms such as “first,” “second,” and “third” are used simply as labels and are not intended to impose numerical requirements on their subjects. [Explanation of symbols]

[0088] 100 scopes, endoscopes 102 Laser Fiber 104 Target 106A Endoscopic image 108 divisions, calibration curve 200 Calibration Curve 300 Machines 302 Hardware Processors 304 Main Memory 306 Static Memory 308 Storage Devices 310 Display Unit 312 Alphanumeric input devices 314 User Interface (UI) Navigation Devices, UI Navigation Devices 316 Sensors 318 Signal Generating Devices 320 Network Interface Devices, Network Interfaces 322 Machine-readable media 324 Command 328 Output Controller 330 Interlink 600 Clinical Decision Support Systems (CDSS), CDSS 602 Input Interface 604 Artificial Intelligence (AI) Model, AI Model 606 Databases 608 Output Interface 610 Video Sensor 612 pixels

Claims

1. A system for determining the characteristics of a target using endoscopic images obtained from a sensor connected to an endoscope, wherein the system is Processor, and Memory where instructions are stored Equipped with, When an instruction is executed by the aforementioned processor, the processor will: (i) Determine the position of the laser fiber within the field of view of the endoscopic image, and the distal end of the laser fiber is close to the target, (ii) Obtain a calibration curve showing the relationship between the position of the laser fiber in the field of view and the distance of the target to the tip of the endoscope, wherein the calibration curve is at least partially established based on the cone of the field of view in the medium and at least one of the design of the endoscope or the dimensions of the laser fiber, A system for determining at least one characteristic value of the target based at least partially on the calibration curve obtained in (iii) and the position of the laser fiber within the field of view of the endoscopic image determined in (i).

2. The system according to claim 1, wherein the at least one characteristic value of the target includes at least one of the size of the target or the distance between the target and the tip of the endoscope.

3. The system according to claim 1, wherein the calibration curve is established at least in part based on the distance the laser fiber extends from the tip of the endoscope, and the position of the laser fiber in the field of view corresponds to the distance the laser fiber extends from the tip of the endoscope.

4. The system according to claim 1, wherein the medium comprises at least one of physiological saline, water, or air.

5. The system according to claim 1, wherein the dimensions of the laser fiber include the diameter of the laser fiber.

6. The system according to claim 1, wherein the step of determining the position of the laser fiber within the field of view includes the step of determining whether the laser fiber is near the edge of the field of view or near the center of the field of view.

7. The system according to claim 1, wherein the step of determining the position of the laser fiber within the field of view includes the step of detecting one or more markings on the insulating jacket of the laser fiber that are visible in the endoscopic image.

8. The system according to claim 1, wherein the calibration curve can be obtained without the distal end of the laser fiber contacting the target.

9. The system according to claim 1, wherein the step of obtaining the calibration curve includes the step of correlating the position of the laser fiber in the field of view with the design of the endoscope and the dimensions of the laser fiber.

10. The aforementioned instruction further to the processor To determine whether a change has occurred in the aforementioned medium, The system according to claim 1, wherein, in response to the aforementioned change, an updated calibration curve corresponding to the new medium is obtained, and the updated calibration curve is established based at least partially on the cone of the field of view in the new medium.

Images