Ureteroscope imaging system and method

Abstract

The present disclosure teaches determining kidney stone size based on calibration data generated from an endoscopic imager. During a lithotripsy procedure, an image is received by a deployed endoscopic probe. Based on the received imaging data, the generated calibration data, and known characteristics of the endoscope probe, one or more kidney stones in the image are identified and sized. And displaying the determined size value together with the kidney stone image.

Description

[0001] Cross Reference to Related Applications

[0002] This application claims the benefit of U.S. Provisional Patent Application No. 63 / 446,988, filed February 20, 2023, the disclosure of which is incorporated herein by reference. TECHNICAL FIELD

[0003] The present disclosure relates generally to endoscopy imaging surgical procedures. In particular, but not exclusively, the present disclosure relates to the processing of kidney stone images during endoscopy lithotripsy procedures. BACKGROUND

[0004] One surgical procedure to address kidney stones, also known as nephrolithiasis, is ureteroscopy. A probe with a camera or other sensor is inserted into the patient’s urethra to find and destroy the stones. An ideal procedure is one in which the medical professional quickly identifies and smoothly eliminates each kidney stone during the procedure.

[0005] Properly addressing a kidney stone requires an accurate estimate of its size, shape, and composition in order to use the correct tool to identify and address it. Because an endoscopic probe is used, rather than the surgeon’s own eyes, it can not always be clear from the returned images the exact location or size of each stone. Therefore, there is a need for an imaging system for ureteroscopy that provides the operator with automatically generated information along with the received images. SUMMARY

[0006] This Summary is provided to introduce some concepts in a simplified form that are further described below in the DETAILED DESCRIPTION. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to determine the scope of the claimed subject matter.

[0007] The present disclosure provides a ureteroscopy imaging solution that addresses the shortcomings of conventional solutions. For example, a system according to the present disclosure can provide timely information during a lithotripsy procedure by identifying and providing the size of a kidney stone visible on an endoscopic image.

[0008] In general, the present disclosure provides the calculation and display of kidney stone size during an endoscopic surgical procedure. An endoscopic image of a known object is taken to provide calibration data. Accurate size data is overlaid on the displayed image provided by the deployed endoscopic probe.

[0009] In some examples, the present disclosure provides an endoscopic imaging method, the method comprising: receiving calibration data, the calibration data being generated from imaging data acquired from an endoscope probe; receiving imaging data comprising at least one kidney stone image from the endoscope probe while the endoscope probe is deployed; applying the calibration data to the imaging data to determine a size of the kidney stone; and displaying the determined size.

[0010] In some embodiments, displaying the determined size comprises displaying the imaging data with the determined size visually associated with the at least one kidney stone image. In some embodiments, displaying the determined size comprises displaying the kidney stone image labeled with a predefined size. The method can further comprise displaying a shape overlaid on the kidney stone image, wherein the determined size is visually associated with the overlaid shape. In some embodiments, the calibration data is generated by the endoscope probe from at least three images taken of one or more predefined objects of known dimensions. In some embodiments, the calibration data comprises a plurality of camera parameters and a plurality of distortion parameters. The method can further comprise determining a focal length of the imaging data based on the calibration data. In some embodiments, the imaging data further comprises at least one surgical device component image having one or more known dimensions, and determining the size of the kidney stone comprises determining a distance from a camera of the deployed endoscope probe to the kidney stone based on the image and the one or more known dimensions of the surgical device component. In some embodiments, the surgical device component is a laser fiber. In some embodiments, the steps of applying the calibration data to the imaging data to determine the size of the kidney stone and displaying the determined size occur while the endoscope probe is deployed.

[0011] The method can further comprise: receiving additional imaging data comprising at least one kidney stone image while the endoscope probe is still deployed; determining an updated size of the kidney stone based on the additional imaging data; and displaying the updated size. In some embodiments, displaying the determined size further comprises comparing the determined size to at least one threshold; selecting one or more display parameters based on the comparison of the determined size to the at least one threshold; and displaying the determined size with the selected one or more display parameters. In some embodiments, the selected one or more display parameters comprise a color selected from a plurality of colors respectively associated with ranges of size values.

[0012] In some embodiments, the present disclosure can be implemented as a computer-readable storage medium containing instructions that, when executed by a processor of a computing device, can cause the processor to implement any of the methods described herein. For some embodiments, the present disclosure can be implemented as a computer comprising a processor; and a memory containing instructions that, when executed by the processor, cause the computing system to implement any of the methods described herein.

[0013] In some examples, the disclosure can be implemented as a computing system comprising: a processor; a display; and a memory comprising instructions that, when executed by the processor, cause the computing system to: receive calibration data generated from imaging data acquired from an endoscope probe; receive, from the endoscope probe while the endoscope probe is deployed, imaging data comprising at least one image of a kidney stone; apply the calibration data to the imaging data to determine a size of the kidney stone; and display the determined size on the display.

[0014] In some implementations, the computing system comprises instructions that, when executed by the processor, cause the computing system to display the imaging data with the determined size visually associated with the at least one image of the kidney stone. In some implementations, wherein the surgical device component is a laser fiber. In some implementations, the imaging data further comprises at least one image of the surgical device component having one or more known dimensions, and determining the size of the kidney stone comprises determining a distance from a camera of the deployed endoscope probe to the kidney stone based on the image and the one or more known dimensions of the surgical device component. In some implementations, the computing system comprises instructions that, when executed by the processor, cause the computing system to, while the endoscope probe is still in the deployed state, receive additional imaging data comprising at least one image of the kidney stone; determine an updated size of the kidney stone based on the additional imaging data; and display the updated size.

[0015] In some examples, the disclosure comprises a computer-readable storage medium containing instructions that, when executed by a processor of a computing device, cause the processor to receive calibration data generated from imaging data acquired from an endoscope probe; receive, from the endoscope probe while the endoscope probe is deployed, imaging data comprising at least one image of a kidney stone; apply the calibration data to the imaging data to determine a size of the kidney stone; and display the determined size.

[0016] In some embodiments, the computer-readable storage medium comprises instructions that, when executed by the processor, cause the computing system to display the imaging data with the determined size visually associated with the at least one image of the kidney stone. In some embodiments, the computer-readable storage medium comprises instructions that, when executed by the processor, cause the computing system to, while the endoscope probe is still in the deployed state, receive additional imaging data comprising at least one image of the kidney stone; determine an updated size of the kidney stone based on the additional imaging data; and display the updated size. BRIEF DESCRIPTION OF DRAWINGS

[0017] For ease of identification of the discussion of any element or action, the most significant digit in the reference number refers to the figure number where the element first appears.

[0018] Figure 1 An endoscopic imaging system is shown in accordance with one or more embodiments of the present disclosure.

[0019] Figure 2 A logic flow is shown in accordance with one or more embodiments of the present disclosure.

[0020] Figure 3 Calibration imaging is shown in accordance with one or more embodiments of the present disclosure.

[0021] Figure 4 A logic flow is shown in accordance with one or more embodiments of the present disclosure.

[0022] Figure 5A An endoscopic image is shown in accordance with one or more embodiments of the present disclosure.

[0023] Figure 5B is a visualization graph that processes data from Figure 5A endoscopic image acquisition of a kidney stone size in accordance with one or more embodiments of the present disclosure.

[0024] Figure 5C is a display of an endoscopic image including a kidney stone size in accordance with one or more embodiments of the present disclosure. DETAILED DESCRIPTION

[0025] The foregoing has broadly outlined some features and technical advantages of the present disclosure in order that the detailed description of the present disclosure that follows can be better understood. The disclosed embodiments can be readily substituted for a variety of architectures and techniques to be used in the implementation of the same. Those of ordinary skill in the art will appreciate the various adaptations and variations of the embodiments discussed herein that are not specifically described. Those of ordinary skill in the art will also appreciate that the inherent nature of the drawings is to provide a convenient illustration of the described embodiments. The drawings are not necessarily to scale, nor are the various components of the drawings necessarily drawn to scale. However, it will be apparent to those of ordinary skill in the art that the drawings provided can be used to not only depict the various embodiments, but also to explain how the various embodiments can be implemented.

[0026] Figure 1 An endoscopic imaging system 100 is shown in accordance with non-limiting examples of the present disclosure. Generally, the endoscopic imaging system 100 is a system for real-time processing and display of images captured by an endoscope deployed for a lithotripsy procedure. Although the focus of the present disclosure is on lithotripsy procedures using a ureteroscope, the present disclosure is also applicable to other endoscope forms such as, for example, a colonoscope, a bronchoscope, etc.

[0027] The endoscopic imaging system 100 includes a computing device 102. Optionally, the endoscopic imaging system 100 includes an imager 104 and a display device 106. In one example, the computing device 102 can receive an image or a set of images representing a patient’s urethra. For example, the computing device 102 can receive an endoscopic image 118 from the imager 104. In some embodiments, the imager 104 can be a camera or other sensor that is deployed with an endoscope during a lithotripsy procedure.

[0028] Although the present disclosure describes illustrative embodiments using visible spectrum camera images, the imager 104 can be any endoscopic imaging device, such as, for example, a fluoroscopy imaging device, an ultrasound imaging device, an infrared or ultraviolet imaging device, a computed tomography (CT) imaging device, a magnetic resonance (MR) imaging device, a positron emission tomography (PET) imaging device, or a single photon emission computed tomography (SPECT) imaging device.

[0029] The imager 104 can generate information elements or data, including indications of a kidney stone. The computing device 102 is communicatively coupled to the imager 104 and can receive data, including the endoscopic image 118, from the imager 104. Generally, the endoscopic image 118 can include indications of shape data and / or appearance data of the urethra. The shape data can include landmarks, surfaces, and boundaries of a three-dimensional surface of the urethra. In some examples, the endoscopic image 118 can be constructed from two-dimensional (2D) or three-dimensional (3D) images.

[0030] Generally, the display device 106 can be a digital display that is arranged to receive rendered image data and display the data in a graphical user interface. The computing device 102 can be any of a variety of computing devices. In some embodiments, the computing device 102 can be incorporated in and / or implemented by a console of the display device 106. For some embodiments, the computing device 102 can be a workstation or server that is communicatively coupled to the imager 104 and / or the display device 106. For other embodiments, the computing device 102 can be provided by a cloud-based computing device, such as by a compute-as-a-service system that can be accessed over a network (e.g., the Internet, an intranet, a wide area network, etc.). The computing device 102 can include a processor 108, a memory 110, input and / or output (I / O) devices 112, and a network interface 114.

[0031] The processor 108 can include circuitry or processor logic such as, for example, any of a variety of commercial processors. In some examples, the processor 108 can include multiple processors, multi-threaded processors, multi-core processors (whether the cores are co-resident on the same or separate dies), and / or some other kind of multi-processor architecture by which multiple physically independent processors are linked in some fashion. Additionally, in some examples, the processor 108 can include graphics processing portions and can include dedicated memory, multi-threading, and / or some other parallel processing capability. In some examples, the processor 108 can be an application specific integrated circuit (ASIC) or a field programmable integrated circuit (FPGA).

[0032] The memory 110 can include logic, a portion of which includes an array of integrated circuits, forming a non-volatile memory that permanently stores data or a combination of non-volatile memory and volatile memory. It should be appreciated that the memory 110 can be based on any of a variety of technologies. In particular, the array of integrated circuits included in the memory 110 can be arranged to form one or more types of memory such as, for example, dynamic random access memory (DRAM), NAND memory, NOR memory, etc.

[0033] The I / O devices 112 can be any of a variety of devices that receive input and / or provide output. For example, the I / O devices 112 can include a keyboard, a mouse, a joystick, a foot pedal, a display (e.g., touch, non-touch, etc.) that is different from the display device 106, a haptic feedback device, an LED, etc. One or more features of an endoscope can also provide input to the imaging system 100.

[0034] The network interface 114 can include logic and / or features to support a communication interface. For example, the network interface 114 can include one or more interfaces to operate in accordance with various communication protocols or standards to communicate over direct or network communication links. Direct communication can be via using a communication protocol or standard described in one or more industry standards, including successors and variants thereof. For example, the network interface 114 can facilitate communication over a bus, such as, for example, a Peripheral Component Interconnect Express (PCIe), a Non-Volatile Memory Express (NVMe), a Universal Serial Bus (USB), a System Management Bus (SMBus), a SAS (e.g., Serial Attached, Small Computer System Interface (SCSI)) interface, a Serial AT Attachment (SATA) interface, etc. Additionally, the network interface 114 can include logic and / or features to enable communication over various wired or wireless network standards (e.g., 802.11 communication standards). For example, the network interface 114 can be arranged to support a wired communication protocol or standard such as Ethernet. As another example, the network interface 114 can be arranged to support a wireless communication protocol or standard such as, for example, Wi-Fi, Bluetooth, ZigBee, LTE, 5G, etc.

[0035] The memory 110 can include instructions 116 and endoscope images 118. During operation, the processor 108 can execute the instructions 116 to cause the computing device 102 to receive the endoscope images 118 from the imager 104 via the input and / or output (I / O) device 112. The processor 108 can also execute the instructions 116 to identify urological structures and matter that need to be removed. Further, the processor 108 can also execute the instructions 116 to generate images to be displayed on the display device 106.

[0036] The memory 110 can also include endoscope data 120 that provides information about the endoscope system, including visible endoscope components; a calibration module 122 that directs the automated system and / or a system user to capture images needed to generate calibration data; an object modeling module 124 that is capable of determining the size of an object from received endoscope images 118; a display processing module 126 that includes tools needed to augment endoscope images with computational values; and user configuration data 128 to reference when customizable options are available to the user.

[0037] The above will be described in greater detail below, for example, in connection with Figure 2 The endoscope imaging system 100 can be equipped with only the computing device 102 by some examples. That is, the endoscope imaging system 100 can include the computing device 102, and a user of the endoscope imaging system 100 can provide the imager 104 and the display device 106 that are compatible with the computing device 102.

[0038] It should be noted that the endoscopic imaging system 100 includes custom components that are specifically configured, programmed, and / or arranged to perform the logical flows and methods detailed herein. For example, the processor 108 may be pre-configured to perform object recognition and graphics processing operations as described elsewhere.

[0039] Figure 2 A logic flow 200 according to some embodiments of this disclosure is illustrated. The logic flow 200 can be implemented to generate calibration data for endoscopic imaging and processing based on automated detection of kidney stones. Although reference is made to endoscopic imaging system 100 and... Figure 1 The logical flow 200 is described, but the example is not limited to this, and the logical flow 200 can be derived from having... Figure 1 The system is implemented using components not shown in the diagram.

[0040] Logic flow 200 can begin at box 202, “Locating the Camera and Target.” The endoscopic camera or other sensor used in endoscopic lithotripsy is placed in front of one or more objects with known visual characteristics, such as those described below. Figure 3 The chessboard pattern described and discussed.

[0041] At box 204, the system captures an image of the located object. In box 206, after each set of images, the position of the camera and / or one or more objects is adjusted as needed for image processing.

[0042] At box 208, the system can also adjust parameters of the visual environment of the image. For example, lighting conditions may vary. The camera, the target object, or both can be immersed in a medium such as water to better account for the refractive index of the environment in which the endoscope will be deployed.

[0043] At decision box 210, the system evaluates whether the captured image is sharp enough and varied enough to generate calibration data. If the image is insufficient, additional images can be captured as described above. If the image is suitable for calibration, calibration data is generated for use during subsequent operations.

[0044] Those skilled in the art should recognize that the frequency and detail of calibration can vary depending on the operator's needs and the capabilities of the equipment. It is conceivable that, in some implementations, the logic described above may represent a "factory calibration" performed during the initial assembly and testing of the endoscope or endoscope camera. Calibration data may be stored in the endoscope device's local memory or in another accessible storage location, such as a part of the endoscopic imaging system.

[0045] For some devices and / or some applications, it can be necessary to provide calibration data frequently. For some applications, it can be important to provide calibration data within a short time before the imager is used. The system can provide guidance to the user for certain calibration data that is necessary for certain functions of the display.

[0046] The precision and format of the resulting calibration data can vary depending on the nature of the device and the needs of the system. As one example, the calibration data can include values representing distortion along one or more spatial axes. Intensity modulation, transparency, tilt, and color correction can all be included in the calibration data to make the camera image more closely resemble the imaged object. Calibration can also determine related information of the imager such as field of view and focal length.

[0047] In one specific example, the camera calibration data can include a parameter matrix A:

[0048]

[0049] where a and b are scaling factors for the two image axes, g is a skew parameter for the two image axes, and (u0, v0) are the coordinates of the principal point used in the calibration. In addition to these values, the system can generate k1 and k2 values, which represent radial distortion coefficients.

[0050] Figure 3 An imager 104, such as an endoscopic camera, is shown placed in front of an object target 302 for calibration. The object target 302 is a planar image with a known size, shape, and geometry. Each image shows the same object target 302 placed at different orientations relative to the imager 104. The distance, position, and angle of the object target 302 are known. Comparing the known shape and size of the object target 302 to the image generated by the imager 104 allows the system to determine calibration parameters of the imager 104.

[0051] Figure 4 A logical flow 400 according to some embodiments of the disclosure is shown. The logical flow 400 can be implemented to process and display imaging data during an endoscopic surgical procedure. Although reference is made to the endoscopic imaging system 100 and Figure 1 The logical flow 400 is described, but the example is not limited thereto, and the logical flow 400 can be implemented by a system having Figure 1 components not shown in the system 100.

[0052] At block 402, the system 100 receives calibration data, which can have been stored in the system memory 110 when the calibration data was generated, or can be accessed from another system, if the calibration data was generated locally.

[0053] At block 404, the system 100 receives data about one or more endoscope components that will be visible on the camera for image processing. In some implementations, the visible component can be the laser fiber. The data can be provided based on the model and configuration of the endoscope. In some embodiments, the data about the laser fiber or another endoscope component can be manually entered by a user based on the calibration images described with respect to the logic flow 200 and Figure 2 and Figure 3 the size and shape of the laser fiber can be generated. In some embodiments, the user can be prompted to manually enter data about the laser fiber or another endoscope component.

[0054] As described above, with respect to calibration, the focal length can be one value that is included in the received calibration data. Alternatively, at block 406, the system 100 calculates the focal length from the received data and / or information about the endoscope imager.

[0055] At block 408, the endoscope is deployed as part of a lithotripsy procedure. The technique of the lithotripsy procedure varies depending on the needs of the patient and the available technology. Any portion of the urinary tract can be the destination of the endoscope, and the procedure can target kidney stones at different locations during a single procedure.

[0056] The remaining portion of the logic flow 400, represented by blocks 410-420, occurs while the endoscope is deployed and the lithotripsy procedure is being performed. These steps 410-420 represent that images are captured, processed, and displayed to the medical professional performing the procedure. The displayed images are “live,” meaning that the delay between receiving and displaying the images is short enough that the professional can use the images to control the lithotripsy tool in real time based on what is displayed.

[0057] At block 410, the system receives one or more endoscope images from the imager. Based on the expected processing delay as well as the rate at which the imager captures images, the system can process more than one image at a time; each individual image is referred to as a “frame,” and the speed at which they are captured is referred to as the “frame rate.” For example, in a case where the system can take up to 0.1 seconds to process a group of frames and the imager has a frame rate of 50 frames per second, the system can process 5 or more frames as a group to have sufficient throughput to process the available data. The system can also process fewer than all of the received frames; in some implementations, it can select a portion of the received frames, such as every second or third frame, to process.

[0058] Figure 5A An endoscope image 500 is shown, representing a single “frame” captured by the imager during deployment of an endoscope for a lithotripsy procedure. Although in this example the image 500 is shown as a single frame, the system can process more than one frame at a time, as described above. Figure 5AThe image is displayed in grayscale, but the system receives and processes a color image (e.g., which can be represented in standard RGB or CMYK encoding, or any other image format that can be read and displayed). Two kidney stones and a laser fiber are visible in the image.

[0059] At block 412, the system identifies kidney stones in the received endoscope image. Stones can be identified by shape, size, color, movement relative to the background, or any combination of these. The physician's movement of the endoscope can also affect the identification of kidney stones. For example, an object that is centered in the imager's field of view for several seconds can be identified as a kidney stone, while a similar object that is quickly moved out of the field of view can not be identified as a kidney stone. In some embodiments, the system can limit identification to only those stones that occupy a certain portion of the field of view, to avoid expending resources on fragments that are smaller than a threshold size or too far from the endoscope probe.

[0060] Various image recognition processes can be used to ensure the persistence of an object, so that once an image is positively identified as a kidney stone, subsequent received frames identify the same portion of the image as a kidney stone. When an image portion with the same color, location, and / or shape as an identified stone is found in a later frame, it is preferentially identified as a kidney stone. In other embodiments that do not use such persistence, the system can require the use of the same initial identification process in each subsequent frame set, which can more quickly end false-positive false labeling.

[0061] Figure 5B is a representation of the visual processing performed by the system 100 to isolate the measurement system elements in the image. Image portions 502a and 502b are each identified as a kidney stone. The system identifies the size of the image portions as 454 pixels and 98 pixels, respectively.

[0062] In the presented example, the size measurement represents the maximum length between any two points in the identified portion of the image. Other methods of generating size are known, such as taking the height (the difference between the maximum and minimum y-coordinates of the pixels in the portion) or the width (the difference in x-coordinates) of the portion.

[0063] While a single length value is typically used when considering the size of a kidney stone, the system is not limited to this value. Size values can be evaluated in two or three dimensions, representing the cross-sectional area or volume of the object, respectively. For example, the total number of pixels in an identified image portion can be used as an estimate of the cross-sectional area of the stone. Known size values taken as the diameter of a sphere or the model of a cross-section can also be used to extrapolate the three-dimensional volume of the image portion in pixels.

[0064] At block 414, the system identifies one or more endoscope components in the received image. As with identifying kidney stones, shape, size, color, and relative motion can all be used to identify a component, which can be a laser fiber, grasper, or any other tool used in lithotripsy surgery. In many implementations, one component will be used that is expected to remain in the field of view of the imager throughout the procedure. The system can again assume image persistence to prioritize identifying that component as remaining in the same area of the image in subsequent frames.

[0065] In Figure 5B In this example, image portion 504 is identified as a laser fiber. The portion of the laser fiber visible in the image is 294 pixels long, tapering from a width of 48 pixels at the edge of the image to 22 pixels at the tip.

[0066] At block 416, the system determines a depth value for the kidney stones based on the known size of the endoscope component and the image dimensions. In the example of image portion 504 identified as a laser fiber, the proportions of the width measurement provide an angle of the fiber relative to the imager, and the length in pixels can be used to determine an approximate depth to which the fiber extends. This depth can then be used as an approximate depth of the kidney stones in the image.

[0067] At block 418, based on the received and calculated image data, a size of each identified kidney stone is determined. In some implementations, this size can be calculated by the following formula:

[0068] s = (p / r)*(d / f)

[0069] where s is the size of the stone in mm, p is the size of the stone image in pixels, r is the resolution of the image in pixels / mm, d is the depth value for the stone in mm, and f is the focal length of the imager in mm.

[0070] At block 420, the calculated size value for each kidney stone is added to the image display. This can be done by superimposing the number on a portion of the kidney stone image itself or placing it nearby. This portion of the image can be highlighted and can include a tooltip or other indicia. In some implementations, the color of the size value on the display can vary according to the calculated size: for example, green for stones smaller than 3 mm, yellow for stones between 3 and 6 mm, and red for stones larger than 6 mm. These thresholds and other aspects of the size display can be customizable in settings available to users of the system.

[0071] Figure 5CAn example is shown of a display image 510 with added labels 512a and 512b showing the size of the kidney stones. The approximate polygonal outline of each image is displayed with a label that indicates the size of the object in millimeters. For larger kidney stones, the label 512a is black, while for smaller kidney stones, the label 512b is white. The color of the label can represent some aspect of the object, such as its size relative to a threshold, but can also be chosen automatically or manually based on other factors of the display, such as contrast for clarity.

[0072] The system can include a timer before the display value is changed. For example, even if the system processes a new set of frames 10 times per second, once a value has been determined and output for display, it can take 1 second for that display value to be changed. This is to avoid the display fluctuating too quickly, thereby reducing its value to the user.

[0073] The terms used herein should have their ordinary meanings in the relevant art, or meanings as indicated in context, but if an explicit definition is provided, that definition should be used.

[0074] References herein to “one embodiment” or “an embodiment” do not necessarily refer to the same embodiment, although they can. Unless context dictates otherwise, throughout the description and the claims, the word “comprise” and variations of the word, such as “comprising” and “comprises,” will be understood to imply the inclusion of a stated integer or group of integers but not the exclusion of any other integer or group of integers. Similarly, the words “include,” “including,” and “includes” will be understood to be in an open-ended way and to mean “including, but not limited to.” The word “or” as used in a phrase such as “A or B” will be understood to encompass both A and B individually, as well as any combination of A and B, unless the context clearly indicates otherwise. Any term not explicitly defined herein shall have its ordinary meaning as understood by one of ordinary skill in the art.

Claims

1. An endoscopic imaging method, the method comprising: Receive calibration data, which is generated based on imaging data acquired from the endoscope probe; When the endoscope probe is deployed, imaging data including at least one image of a kidney stone is received from the endoscope probe; The calibration data is applied to the imaging data to determine the size of the kidney stone; as well as Display the determined size.

2. The method according to claim 1, wherein, Displaying the determined size includes: displaying imaging data having a determined size that is visually associated with the at least one kidney stone image.

3. The method according to claim 1 or 2, wherein, The determined size display includes: displaying images of kidney stones labeled with predefined sizes.

4. The method according to claim 2 or 3, further comprising: The shape overlaid on the image of the kidney stone is shown, wherein the determined size is visually associated with the overlaid shape.

5. The method according to any one of claims 1 to 4, wherein, The calibration data is generated by the endoscope probe based on at least three images taken of one or more predefined objects of known size.

6. The method according to any one of claims 1 to 5, wherein, The calibration data includes multiple camera parameters and multiple distortion parameters.

7. The method according to any one of claims 1 to 6, further comprising: The focal length of the imaging data is determined based on the calibration data.

8. The method according to any one of claims 1 to 7, in, The imaging data also includes at least one image of a surgical device component having one or more known sizes, and Determining the size of the kidney stone includes: determining the distance from the camera of the deployed endoscopic probe to the kidney stone based on the image and one or more known dimensions of the surgical device components.

9. The method according to claim 8, wherein, The surgical equipment component is a laser fiber.

10. The method according to any one of claims 1 to 9, wherein, The steps of applying the calibration data to the imaging data to determine the size of the kidney stone and displaying the determined size occur simultaneously with the deployment of the endoscopic probe.

11. The method according to any one of claims 1 to 10, further comprising: While the endoscope probe is still deployed, additional imaging data including at least one image of a kidney stone is received; Based on the additional imaging data, the updated size of the kidney stone is determined; as well as The updated size is displayed.

12. The method according to any one of claims 1 to 11, wherein, The determined size also includes: The determined size is compared with at least one threshold. One or more display parameters are selected based on a comparison of the determined size with the at least one threshold; and The determined size is displayed using one or more selected display parameters.

13. The method according to claim 12, wherein, The selected display parameters include colors chosen from a variety of colors that are associated with a range of size values.

14. A computer-readable storage medium including instructions that, when executed by a processor of a computing device, cause the processor to perform the method according to claim 1.

15. A computing system, the computing system comprising: processor; and The memory includes instructions that, when executed by a processor, cause the computing system to perform the method according to claim 1.

Images