Augmented reality ureteroscope system

Abstract

Disclosed herein is a ureteroscope system comprising a ureteroscope configured for insertion into a ureter of a patient's body. The system includes an image processing module with a console having one or more processors and a memory having logic stored therein. The logic can perform various operations such as providing a ureteroscope video image with small debris removed to provide unobstructed visibility of larger debris. The logic can perform various operations such as object tracking, object enhancement, sizing, etc. An overlay provides a representation of a marker on top of the ureteroscope video image. A method of performing a lithotripsy procedure is also disclosed in conjunction with the system.

Description

[Technical field]

[0001] The present invention relates to an augmented reality ureteroscope system. [Background technology]

[0002] Kidney stones can be treated in a variety of ways. Small kidney stones may pass through the ureter without treatment. Larger kidney stones or kidney stones that block the ureter may need to be removed by a medical procedure. Laser lithotripsy is a procedure to remove stones (e.g., kidney stones) from a patient's ureter. Laser lithotripsy involves inserting a laser optical fiber through the ureter to the stone. A laser is then activated to break up the stone into small pieces that can be naturally passed by the patient or removed by a retrieval instrument. A typical procedure involves inserting a ureteroscope through the urethra, bladder, ureter, and optionally into the kidney, such that the distal tip of the scope is positioned in close proximity to the stone. A laser optical fiber is inserted through the working channel of the ureteroscope to the stone. A laser is then activated to break up the stone into pieces small enough to be retrieved by a retrieval device, such as a basket device, or naturally passed through the ureter by the patient.

[0003] During laser ablation of kidney stones, small fragments (e.g., fast moving dust particles) may break off from the stone and become suspended in the urinary fluid. The small fragments may be so numerous that they affect the visibility of the object through the urinary fluid. In some cases, the small fragments may significantly block the view of the larger fragments, creating difficulties in further performing the stone fragmentation process. Such difficulties may hinder the identification and tracking of larger stone fragments that require further ablation. In some cases, the surgeon may need to pause the ablation process to regain visibility before retargeting the larger stone fragments. Summary of the Invention [Problem to be solved by the invention]

[0004] Thus, disclosed herein are ureteroscopic systems and methods for enhancing ureteroscopic visibility of objects, such as kidney stone fragments, tracking the fragments, and assessing fragment size during a laser lithotripsy procedure. [Means for solving the problem]

[0005] Briefly summarized, disclosed herein is a ureteroscope system comprising a ureteroscope configured for insertion into a ureter of a patient's body. The ureteroscope comprises an elongated flexible shaft having a camera disposed at a distal end of the ureteroscope, and an image processing module operatively coupled to the ureteroscope. The module comprises a console having one or more processors and a non-transitory computer readable medium. Logic stored on the medium, when executed by the one or more processors, is configured to perform various operations summarized below.

[0006] The operations include receiving imaging data and defining a first image and a second image from the imaging data. The first image includes a plurality of objects, the plurality of objects including a first subset of the plurality of objects that obstructs visibility of one or more objects of a second subset of the plurality of objects in the first image. In the second image, the one or more occluded objects in the first image are visually unobstructed. The operations include rendering the first image or the second image on a display device of the system.

[0007] The imaging data may include video imaging data, and the first image and the second image may include video images. The object may include a kidney stone fragment. The operations may further include removing a first subset of the objects from the first image to define a second image, in some embodiments, the second subset includes objects that are permanent in the first image and the first subset includes objects that are temporary in the first image.

[0008] The operations may further include (i) tracking a position of one or more objects in the first image or the second image, and (ii) defining a tracking image overlay, the tracking overlay including tracking markers associated with the tracked objects. The operations may further include rendering the tracking overlay over the first image or the second image on a display device.

[0009] The operations may further include (i) identifying edges of one or more objects in the first image or the second image, (ii) enhancing the edges in an image overlay, and (iii) rendering the edge enhancement overlay on the display device over the first image or the second image.

[0010] The operations may further include (i) defining a size of one or more objects in the first image or the second image by calculating an area enclosed by a perimeter, and (ii) defining a sizing image overlay including size markers associated with the sized objects, each size marker providing a visual indication of the size of a respective object. The operations may further include rendering the sizing image overlay over the first image or the second image on a display device.

[0011] The operations may further include comparing each calculated area to an area limit stored in the non-transitory computer readable medium and altering the size indicator if the respective calculated area exceeds the area limit.

[0012] The operations may further include defining a maximum length of one or more objects in the first image or the second image, each maximum length defined by a maximum distance between two points on a perimeter of a respective object. The operations may further include defining a length image overlay, the length overlay including a line indicator visually representing each maximum length, and the operations may further include drawing the length overlay over the first image or the second image on a display device.

[0013] Also summarized herein is a method for performing a lithotripsy procedure on a patient. The method includes inserting a ureteroscope into the patient's ureter, receiving imaging data from the ureteroscope, defining a first image of kidney stone fragments from the imaging data, and rendering a second image on a display device, the second image including one or more permanent fragments of the first image and having one or more temporary fragments of the first image removed. The method further includes ablating the fragments in the second image and removing the ablated portion of the fragments from the second image.

[0014] The method may further include the steps of defining a perimeter of the debris and overlaying edge markers associated with the debris on the second image, the edge markers visually highlighting the perimeter of the debris in the second image.

[0015] The method may further include overlaying a size indicator associated with the debris on the second image, the size indicator visually indicating a size of the debris in the second image. The method may further include overlaying a position marker associated with the debris on the second image, the position marker tracts a displacement of the debris in the second image.

[0016] The method may further include overlaying a length marker associated with the fragment on the second image, the length marker representing a maximum length of the fragment in the second image. The method may further include rendering a third image on the display device, the third image being a high contrast black and white image of the second image.

[0017] The method may further include providing an image processing module operatively coupled to the ureteroscope, the image processing module comprising a console having one or more processors and a non-transitory computer readable medium, the logic stored on the medium being configured, when executed by the one or more processors, to perform operations including one or more of the steps summarized above.

[0018] These and other features of the concepts provided herein will become more apparent to those of ordinary skill in the art in view of the following description and accompanying drawings, which describe in more detail certain embodiments of such concepts.

[0019] A more particular description of the present disclosure will be made by reference to specific embodiments thereof which are illustrated in the accompanying drawings, it being understood that these drawings depict only typical embodiments of the invention and are therefore not to be considered as limiting the scope of the invention, exemplary embodiments of the invention will be described and explained with additional specificity and detail through the use of the accompanying drawings, in which: [Brief description of the drawings]

[0020] [Figure 1A] 1 illustrates a ureteroscope system, according to some embodiments. [Figure 1B] 1B is a block diagram of a console of the system of FIG. 1A according to some embodiments. [Figure 2A] 1 illustrates an exemplary raw ureteroscope image, according to some embodiments. [Figure 2B] 2B illustrates an exemplary enhanced image of the raw ureteroscope image of FIG. 2A, according to some embodiments. [Figure 2C] 1 illustrates an example flow diagram of an imaging process for a ureteroscope system, according to some embodiments. [Figure 3A] 1A-1C show exemplary enhanced ureteroscope images including tracking overlays, in accordance with some embodiments. [Figure 3B] FIG. 3B illustrates a ureteroscope image of FIG. 3A at a later time point relative to FIG. 3A, in accordance with some embodiments. [Figure 3C] 1 illustrates an example flow diagram of a tracking process for a ureteroscope system, according to some embodiments. [Figure 4A] FIG. 13 illustrates an exemplary enhanced ureteroscope image including an overlay showing an enhanced periphery of an object in the image, in accordance with some embodiments. [Figure 4B] FIG. 13 illustrates an exemplary enhanced ureteroscope image including an overlay showing the maximum length line of an object in the image, according to some embodiments. [Figure 4C] 1A-1C show exemplary enhanced ureteroscope images including an overlay showing size markers of objects in the image, in accordance with some embodiments. [Figure 4D] FIG. 13 illustrates an exemplary process flow diagram of an edge enhancement process, a maximum line length determination process, and a size determination process of a ureteroscope system, in accordance with some embodiments. [Diagram 5] 2C shows an exemplary high-contrast black-and-white image of the improved image of FIG. 2B in accordance with some embodiments. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0021] Before some specific embodiments are disclosed in more detail, it should be understood that the specific embodiments disclosed herein do not limit the scope of the concepts provided herein. It should also be understood that the specific embodiments disclosed herein can have features that can be easily separated from the specific embodiment and, optionally, combined with or substituted for features of any of the other numerous embodiments disclosed herein.

[0022] With regard to the terms used herein, it should also be understood that the terms are intended to describe certain particular embodiments, and that the terms do not limit the scope of the concepts provided herein. Ordinal numbers (e.g., first, second, third, etc.) are generally used to distinguish or identify different features or steps within a group of features or steps, and do not impose any order or numerical limitations. For example, the "first", "second", and "third" features or steps do not necessarily have to appear in that order, and a particular embodiment including such features or steps is not necessarily limited to three features or steps. Designations such as "left", "right", "upper", "lower", "front", "rear", "upper", "lower", etc. are used for convenience and do not imply, for example, a particular fixed position, orientation, or direction. Instead, such designations are used, for example, to reflect a relative position, orientation, or direction. The singular forms "one", "one", and "the" include plural references unless the context clearly dictates otherwise. Additionally, the words "including," "having," and "having" are intended to have the same meaning as the word "comprising" when used in this application, including the claims.

[0023] Finally, in the following description, the terms "or" and "and / or" as used herein should be interpreted as an inclusive or, meaning any one or any combination. By way of example, "A, B or C" or "A, B and / or C" means "any of the following: A; B; C; A and B; A and C; B and C; A, B and C." Exceptions to this definition will occur only if combinations of elements, components, features, steps or acts are in any way inherently mutually exclusive.

[0024] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art. Any method disclosed herein includes one or more steps or acts for implementing the described method. Method steps and / or acts may be interchanged with one another. In other words, unless a specific order of steps or acts is required for proper operation of an embodiment, the order and / or use of specific steps and / or acts may be changed. Furthermore, only subroutines or portions of methods described herein may be independent methods within the scope of the present disclosure. Stated otherwise, some methods may include only some of the steps described in a more detailed method.

[0025] In certain contexts, the term "logic" refers to hardware, firmware, and / or software configured to perform one or more functions. As with hardware, the term logic may include circuitry having data processing or storage functionality. Examples of such circuitry may include, but are not limited to, a microprocessor, one or more processor cores, programmable gate arrays, microcontrollers, application specific integrated circuits, wireless receiver, transmitter and / or transceiver circuitry, semiconductor memory, or combinational logic.

[0026] FIG. 1A illustrates an embodiment of a ureteroscope system 100 shown in a treatment environment. An operator 30 (e.g., a physician) is shown performing an invasive treatment on a patient 50. The ureteroscope system 100 includes a ureteroscope 110 coupled to an imaging module 120. The ureteroscope 110 includes an elongated flexible shaft 115 configured for insertion into a ureter of the patient 50. The shaft 170 includes a camera (not shown) located at a distal end of the shaft 170 and a working channel 116 extending along the shaft 170. During operation, images acquired by the camera are rendered on a display device 105 coupled to the imaging module 120. The display device 105 may also include a graphical user interface (GUI) 106. The operator 30 is shown using a urological laser device 70 configured to perform a laser lithotripsy procedure. The laser device 70 includes a fiber optic laser 71 inserted into the working channel 116.

[0027] During treatment, the flexible shaft 115 of the ureteroscope 110 is inserted into the ureter of the patient 50 to a treatment location. The imaging module 120 renders on the display device 105 images acquired by a camera located at the distal end of the shaft 115. The images show tissue and other objects (e.g., kidney stones) at the treatment location. The surgeon 30 performs the treatment by operating the laser device 70 while viewing the images acquired by the ureteroscope 110 and rendered on the display device 105.

[0028] In some embodiments, the ureteroscope 110 may include a remote interface 118. The remote interface 118 may be communicatively coupled to the imaging module 120 and may facilitate operation of the imaging system 100 via a handle 117.

[0029] A treatment procedure may typically include positioning the working distal end of the laser shaft 71 at a desired location verified by acquired images. By manipulating the handle 117 of the ureteroscope 110, the operator 30 may position the distal end of the ureteroscope 110 at a desired location and aim the distal end of the fiber optic laser 71 at the object to be ablated, such as a kidney stone or its fragments. During kidney stone ablation, the operator 30 may reposition the ureteroscope 110 several times to view different kidney stone fragments and redirect the fiber optic laser 71 to the different fragments.

[0030] During ablation of a kidney stone, fragments of various sizes are fragmented and detached from the stone. Some fragments may be small enough to exit the patient through the ureter without further fragmentation. Other fragments may require further ablation or removal from the patient by a retrieval device. Thus, visibility of the fragments helps to facilitate further ablation or retrieval. In some cases, the amount of detached fragments may be so large that it may interfere with the visibility of other objects through the ureteroscope 110.

[0031] 1B is a block diagram of various components of a console 125 of imaging module 120. Console 125 includes one or more processors 130 and a memory 140 (e.g., a non-transitory computer-readable medium) that stores logic modules configured to perform operations described below when executed by processor 130. Console 125 is communicatively coupled to display device 105 and ureteroscope 110.

[0032] The logic modules include imaging logic 141 and image processing logic 142. The imaging logic 141 is configured to capture raw imaging data from the ureteroscope 110 and provide processed imaging data to the display device 105. The imaging logic 141 may communicate with the remote interface 118 to render an image on the display device as defined by the surgeon 30.

[0033] The image processing logic 142 includes image enhancement logic 143, object tracking logic 144, and object size determination logic 145. The image enhancement logic 143 is described below in conjunction with Figures 2A-2C. The object tracking logic 144 is described below in conjunction with Figures 3A-3C, and the object size determination logic 145 is described below in conjunction with Figures 4A-4D.

[0034] The console 125 may also include other hardware or electrical components to facilitate operation of the imaging module 120 and the ureteroscope 110, such as a power supply, I / O ports, a power management module, signal conditioners, processing electronics, a GUI signal processing unit, computing electronics, a graphics processing unit, a field programmable gate array, etc.

[0035] FIG. 2A illustrates an exemplary raw ureteroscope image 201 that may be acquired during ablation of a kidney stone. During ablation of a kidney stone, fragments of various sizes may be fragmented and detached from the stone. Some fragments may be small enough (e.g., dust) to exit the patient through the ureter without further ablation. Other fragments may require further ablation or removal from the patient by a retrieval device. In some cases, the amount of detached smaller fragments, including dust particles, may be so great that it may obstruct the visibility of other objects by the ureteroscope 110. The image 201 illustrates how the small fragments 205 may block the view of the larger fragments 211-214. Thus, increasing the visibility of the larger fragments 211-214 may help facilitate further ablation or retrieval of the larger fragments 211-214.

[0036] 2B illustrates the enhanced ureteroscope image 202 after image processing by the image enhancement logic 143. As shown, the image enhancement logic 143 has effectively removed small debris 205 from the raw image 201, resulting in clearer visualization of the larger debris 211-214 in the enhanced image 202. The image enhancement logic 143 may include one or more image analysis and processing algorithms (e.g., image subtraction, image comparison, or median filter techniques, for example) to distinguish objects that are transient between video image frames from objects that are persistent between video image frames. For example, the smaller debris 205 may be floating around in the ureter and thus may be temporarily displaced by fluid flow between video image frames. The larger debris 211-214 may not be floating around as much as the smaller debris 205 and thus may be persistently positioned across multiple video image frames. Thus, the temporary smaller debris 205 may obstruct the camera's view of portions of the larger debris 211-214 in some frames, while providing an unobstructed view of said portions in other frames. Upon distinguishing the temporary debris 205 from the permanent debris 211-214, the image augmentation logic 143 may effectively remove the temporary debris 205 from the raw image 201, making the permanent debris 211-214 more clearly visible, as shown in the augmented image 202. In summary, the image augmentation logic 143 may generate the augmented image 202 by removing the temporary debris 205 from the raw image 201.

[0037] 2C is a flow chart 250 illustrating exemplary process steps for defining and displaying an augmented image 202 on a display device 105, according to some embodiments. The process includes obtaining imaging data from a ureteroscope (step 251) and generating a raw ureteroscope image 201 from the imaging data (step 252). The process further includes identifying a transient object (e.g., debris 205) among multiple frames of the raw ureteroscope image 201 (step 253). Once the transient object is identified, the augmented image 202 is generated by removing or subtracting the transient object from the raw ureteroscope image 201 (step 254). The raw ureteroscope image 201 or the augmented image 202 are then selectively rendered on the display device 105 (step 255).

[0038] 3A-3B show a tracking overlay 301 displayed on top of the augmented image 202. FIG. 3A shows the augmented image 202 including the fragments 211-214 at a first time point. FIG. 3B shows the augmented image 202 including the fragments 211-214 at a later time point. As shown, the fragment 211 has been displaced from a first position in FIG. 3A to a second position in FIG. 3B. During stone fragmentation, stone fragments may be displaced due to fluid flow or the laser ablation process. In some cases, it may be difficult for the surgeon to visually track the location of a stone fragment such as the fragment 211. In some cases, searching for a fragment to relocate may cause delays in the stone fragmentation process. Thus, it may be advantageous for the imaging system 100 to include object tracking logic 144 configured to track the location of the stone fragments defined within the augmented image 202. Such tracking may increase the surgeon's situational awareness and reduce cognitive load.

[0039] In some embodiments, the object tracking logic 144 may assign identifying markers 321-324 to the stone fragments 211-214, respectively. By assigning identifying markers, the surgeon may more easily re-identify and track the location of the identified stone fragments 211-214. In some embodiments, the object tracking logic 144 may automatically assign identifying markers to defined fragments in the video image. In other embodiments, the object tracking logic 144 may be configured to facilitate manual selection (e.g., by mouse pointer) of the stone fragments to be identified and tracked. In some embodiments, other tracking markers may also be shown in the tracking overlay 301, such as, for example, arrows 321A indicating the displacement path. In the illustrated embodiment, the tracking overlay 301 may be displayed over the augmented image 202 or the raw image 201.

[0040] 3C is a flow chart 350 illustrating exemplary process steps for tracking objects in a raw ureteroscope image 201 or an augmented image 202, according to some embodiments. The process includes identifying persistent objects (e.g., debris 211-214) among multiple frames of the raw ureteroscope image 201 (step 351). Once persistent objects (e.g., objects 211-214) are identified, one or more of the persistent objects are identified as objects to be tracked (step 352). Next, an overlay is generated (step 353) including indicators (e.g., identifying indicators 321-324) indicating the locations of the tracked objects. The overlay 301 is then rendered on the display device 105 over the raw ureteroscope image 201 or the augmented image 202 (step 354).

[0041] 4A-4C show the augmented image 202 including further exemplary overlays that may be related to the sizing process performed by the object size determination logic 144 in some embodiments. FIG. 4A shows an edge enhancement overlay 401 displayed over the augmented image 202. As shown in FIG. 4A by way of example, the object size determination logic 144 defines and enhances (or otherwise makes more visible) edges 421-424 that extend around each of the fragments 211-214, respectively. By enhancing the edges of the stone, the surgeon may more easily visualize the size and shape of the fragment and / or identify the portion of the stone to which to direct the laser for ablation. In some embodiments, the object size determination logic 144 may include image processing algorithms for identifying edges (e.g., algorithms known as Canny, Sobel, and Gaussian). In some embodiments, the object size determination logic 144 may automatically define the fragments in the image for edge enhancement. For example, the object size determination logic 144 may perform a size assessment of persistent fragments in the image and enhance the edges of fragments that exceed a specified size stored in memory 140. In some embodiments, the object size determination logic 144 may be configured to facilitate manual selection (e.g., with a mouse pointer) of stone fragments for inclusion of edge enhancement. In the illustrated embodiment, an edge enhancement overlay 401 may be displayed over the augmented image 202 or the raw image 201.

[0042] FIG. 4B shows a fragment length overlay 402 displayed on top of the augmented image 202 along with the edge enhancement overlay 401 of FIG. 4A. In some embodiments, the fragment length overlay 402 may be displayed alone on the augmented image 202. The object size determination logic 144 may define size indication lengths for the defined fragments, such as lengths 431-434 for fragments 411-414, respectively, as shown. Each of the lengths 431-434 may be a distinct characteristic of a respective stone size (e.g., imaged area). In some embodiments, the size indication length may be a maximum distance between two time points along the periphery. In some embodiments, the lengths 431-434 may be shown in the overlay 402 as line markers that may help the surgeon determine whether further ablation or fragmentation is warranted. In some embodiments, the object size determination logic 144 may define measurements for the size indication lengths 431-434 and display length measurement markers in the overlay 402. In some cases, the size indication length may facilitate a decision by the surgeon that further ablation of the fragment is necessary or that the patient can naturally expel the fragment through the ureter. In a manner similar to edge enhancement, the object size determination logic 144 may automatically define fragments for length indication according to prescribed criteria and / or may facilitate manual selection of fragments for length indication.

[0043] FIG. 4C shows a fragment size overlay 403 displayed on top of the augmented image 202. The object size determination logic 144 may define the fragment size or image area by calculating the area enclosed by the perimeter. The defined fragment size indicators, such as size indicator indicators 441-444 of the fragments 211-214, may each provide a representation of the fragment's size that allows the surgeon to see the size of the fragments 211-214, e.g., relative to each other. In the illustrated embodiment, the indicators 441-444 have square shapes. In other embodiments, the indicators 441-444 may be any other polygon, chevron, fragment outline, or any other type of mark suitable for visually representing the fragment's size. In a similar manner to the length indication, the object size determination logic 144 may automatically identify fragments for size indication according to defined criteria and / or facilitate manual selection of fragments for size indication.

[0044] In some embodiments, the object size determination logic 144 may include size criteria related to the ureter, such as, for example, the flow area of ​​the ureter. The object size determination logic 144 may further determine whether the fragment size exceeds a defined size criterion, and if so, the object size determination logic 144 may include a secondary indicum, such as indicium 441A, change the shape of the size indicium, change the color of the size indicium, or provide a visual indication in any suitable manner of differentiation. In summary, the object size determination logic 144 may automatically indicate to the surgeon which fragments require further fragmentation and which fragments may be naturally passed from the patient through the ureter.

[0045] FIG. 4D is a flow chart 450 illustrating exemplary process steps for assessing / estimating the size of an object in a raw ureteroscope image 201 or an augmented image 202, according to some embodiments. The process includes identifying persistent objects (e.g., debris 211-214) among multiple frames of the raw ureteroscope image 201 (step 451). Once persistent objects are identified, one or more persistent objects are identified as objects for border enhancement (step 452). Borders (e.g., edges 421-424) are identified (step 453). An overlay is then generated that includes a highlight of the identified borders (step 454). The overlay 401 is then rendered on the display device 105 over the raw ureteroscope image 201 or the augmented image 202 (step 455). With border enhancement, the surgeon can more easily visually assess the size of the object.

[0046] With further reference to FIG. 4D, the process steps for assessing / estimating object size may further include identifying one or more persistent objects for which it may be advantageous to determine a representative length (step 462). The process further includes, for each of the identified objects, identifying a perimeter and determining a line that defines a maximum length between two time points on the perimeter (step 463). An overlay 402 is then generated that includes a line that extends across the identified objects (step 464). This overlay is then rendered on the display device 105 over the raw ureteroscope image 201 or the augmented image 202 (step 465).

[0047] Referring again to FIG. 4D, the process steps of assessing / estimating the size of an object may further include identifying one or more persistent objects for size assessment / estimation (step 472). The process further includes identifying a perimeter and calculating an area enclosed by the perimeter for each of the identified objects (step 473). An overlay 403 is then generated including an indicator representing the calculated area (step 474). The overlay is then rendered on the display device 105 over the raw ureteroscope image 201 or the augmented image 202 (step 475). In some embodiments, the process steps may include comparing the calculated area to an area size limit stored in memory (step 476). The process may further include modifying the size indicator if the calculated area exceeds the size limit (step 477).

[0048] 5 shows a high contrast ureteroscope image 501 after further image processing of the enhanced image 202 by the image enhancement logic 143. The high contrast image 501 is a high contrast black / white image showing the debris 211-214. The high black / white contrast allows the surgeon to more easily visualize and focus on important objects in the image, such as the debris 211-214. Any or all of the overlays described above may be placed on top of the high contrast image 501.

[0049] The image processing logic 142 may facilitate the rendering of any combination of images 201, 202, 501 and overlays 301, 401, 402, 403. The image processing logic 142 may also facilitate the surgeon, via the GUI 151 or remote interface 118, to freely switch between any of these combinations.

[0050] The embodiments of the present invention may be embodied in other specific forms without departing from the spirit of the disclosure. The described embodiments are to be considered in all respects as merely illustrative and not restrictive. The scope of the embodiments is therefore indicated by the appended claims, rather than by the foregoing description. All changes that come within the meaning and range of equivalency of the claims are to be embraced within their scope.

Claims

1. 1. A ureteroscope system comprising:

1. A ureteroscope configured for insertion into a ureter of a patient's body, the ureteroscope comprising: an elongated flexible shaft; a camera disposed at a distal end of the shaft; and an image processing module operatively coupled to the ureteroscope, the image processing module comprising a console including one or more processors and a non-transitory computer readable medium, the non-transitory computer readable medium having stored thereon logic configured, when executed by the one or more processors, to perform the following operations: receiving imaging data including a first image, the first image including a plurality of objects, the plurality of objects including a first subset of the plurality of objects that obstructs visibility of one or more objects of a second subset of the plurality of objects; generating a second image including a second subset of the plurality of objects in a visually unobstructed state; causing a rendering of the first image or the second image on a display device; and A ureteroscope system comprising:

2. The system of claim 1 , wherein the operations further include causing a rendering of the second image on the display device.

3. the imaging data includes video imaging data; The system of claim 1 or 2, wherein the first image and the second image comprise video images.

4. The system of claim 1 or 2, wherein the object comprises a kidney stone fragment.

5. The system of claim 1 or 2, wherein the operations further comprise removing the first subset from the first image to define the second image.

6. the second subset includes objects that are persistent in the first image; The first subset includes objects that are transient in the first image. The system according to claim 1 or 2, comprising:

7. The system of claim 1 or 2, wherein the operations further comprise tracking a position of one or more objects in the first image or the second image.

8. The system of claim 7 , wherein the operations further comprise defining a tracking image overlay, the tracking image overlay including a tracking marker associated with the tracked object.

9. The system of claim 8 , wherein the operations further include causing a rendering of the tracking image overlay on the display device over the first image or the second image.

10. The system of claim 1 or 2, wherein the operations further comprise identifying a perimeter of one or more objects in the first image or the second image.

11. The operations further include defining an edge-enhanced image overlay; The system of claim 10 , wherein the edges are enhanced in the edge-enhanced image overlay.

12. The system of claim 11 , wherein the operations further include causing a rendering of the edge-enhanced image overlay on the display device over the first image or the second image.

13. 11. The system of claim 10, wherein the operations further include defining a size of one or more objects in the first image or the second image, the size being defined by calculating an area enclosed by a perimeter.

14. The operations further include defining a sizing image overlay, the sizing image overlay including a size indicator associated with the sized object; The system of claim 13 , wherein each size indicia provides a visual indication of a respective size of the object.

15. The system of claim 14 , wherein the operations further include causing a rendering of the sized image overlay on the display device over the first image or the second image.

16. The operation includes: comparing each calculated area to an area limit stored in the non-transitory computer readable medium; and modifying said size indicator if each said calculated area exceeds said area limit.

17. 11. The system of claim 10, wherein the operations further include defining a maximum length of one or more objects in the first image or the second image, each maximum length being defined by a maximum distance between two points on a perimeter of a respective one of the objects.

18. 20. The system of claim 17, wherein the operations further include defining a length image overlay, the length image overlay including a line marker visually representing each maximum length.

19. 20. The system of claim 18, wherein the operations further include causing a rendering of the length image overlay on the display device over the first image or the second image.

Images