Portable Ergonomic Endoscope with Disposable Intubation

Abstract

The present invention relates to an endoscope system comprising a disposable portion and a reusable portion. The two portions can be docked and detached. The disposable portion comprises an elongated cannula having a bendable portion near its front end, providing a "steerable" front end tip. The imaging system comprises at least two independent cameras and two independent light sources. The cameras and light sources are configured to image a target object simultaneously. By employing different illumination, different filters and controlling the spectral response, different features of the target object can be captured. In some embodiments, the system processor can coordinate the cameras and light sources and synthesize the resulting images to display an enhanced composite image of the object to the user.

Description

[0001] Related applications

[0002] This application is a continuation-in-part of the following patents: U.S. patent application No. 17 / 362,043 filed on June 29, 2021; International patent application No. PCT / US19 / 36060 filed on June 7, 2019; U.S. patent application No. 16 / 363,209 filed on March 25, 2019, with issued patent No. 10,278,563; and International patent application No. PCT / US17 / 53171 filed on September 25, 2017.

[0003] This application incorporates by reference the entire contents of the above-identified patent applications and claims the benefits claimed by each of the above-identified patent applications and any applications directly or indirectly incorporated by reference therein, including the filing dates of the U.S. provisional applications, U.S. non-provisional applications, and international applications.

[0004] This patent application claims the benefit of the following provisional applications and incorporates these applications by reference:

[0005] U.S. Provisional Application No. 63 / 218,362, filed July 4, 2021;

[0006] U.S. Provisional Application No. 63 / 213,499, filed June 22, 2021;

[0007] U.S. Provisional Application No. 63 / 210,034, filed June 13, 2021;

[0008] U.S. Provisional Application No. 63 / 197,639, filed June 7, 2021;

[0009] U.S. Provisional Application No. 63 / 197,611, filed June 7, 2021;

[0010] U.S. Provisional Application No. 63 / 183,151, filed May 3, 2021;

[0011] U.S. Provisional Application No. 63 / 153,252, filed February 24, 2021;

[0012] U.S. Provisional Application No. 63 / 149,338, filed February 14, 2021;

[0013] U.S. Provisional Application No. 63 / 138,751, filed January 18, 2021;

[0014] U.S. Provisional Application No. 63 / 129,703, filed December 23, 2020;

[0015] U.S. Provisional Application No. 63 / 124,803, filed December 13, 2020;

[0016] U.S. Provisional Application No. 63 / 121,924, filed December 6, 2020;

[0017] U.S. Provisional Application No. 63 / 121,246, filed December 4, 2020;

[0018] U.S. Provisional Application No. 63 / 107,344, filed October 29, 2020;

[0019] U.S. Provisional Application No. 63 / 087,935, filed October 6, 2020;

[0020] U.S. Provisional Application No. 63 / 083,932, filed September 27, 2020;

[0021] U.S. Provisional Application No. 63 / 077,675, filed September 13, 2020; and

[0022] U.S. Provisional Application No. 63 / 077,635, filed September 13, 2020.

[0023] This patent application also incorporates by reference the following international, non-provisional, and provisional applications:

[0024] International patent application No. PCT / US17 / 53171, filed on September 25, 2017;

[0025] U.S. Patent No. 8,702,594, issued on April 22, 2014;

[0026] U.S. patent application No. 16 / 363,209, filed March 25, 2019;

[0027] International patent application No. PCT / US19 / 36060, filed on June 7, 2019;

[0028] U.S. patent application No. 16 / 972,989, filed on December 7, 2020;

[0029] U.S. Provisional Application No. 62 / 816,366, filed March 11, 2019;

[0030] U.S. Provisional Application No. 62 / 671,445, filed May 15, 2018;

[0031] U.S. Provisional Application No. 62 / 654,295, filed April 6, 2018;

[0032] U.S. Provisional Application No. 62 / 647,817, filed March 25, 2018;

[0033] U.S. Provisional Application No. 62 / 558,818, filed September 14, 2017;

[0034] U.S. Provisional Application No. 62 / 550,581, filed August 26, 2017;

[0035] U.S. Provisional Application No. 62 / 550,560, filed August 25, 2017;

[0036] U.S. Provisional Application No. 62 / 550,188, filed August 25, 2017;

[0037] U.S. Provisional Application No. 62 / 502,670, filed May 6, 2017;

[0038] U.S. Provisional Application No. 62 / 485,641, filed April 14, 2017;

[0039] U.S. Provisional Application No. 62 / 485,454, filed April 14, 2017;

[0040] U.S. Provisional Application No. 62 / 429,368, filed December 2, 2016;

[0041] U.S. Provisional Application No. 62 / 428,018, filed November 30, 2016;

[0042] U.S. Provisional Application No. 62 / 424,381, filed November 18, 2016;

[0043] U.S. Provisional Application No. 62 / 423,213, filed November 17, 2016;

[0044] U.S. Provisional Application No. 62 / 405,915, filed October 8, 2016;

[0045] U.S. Provisional Application No. 62 / 399,712, filed September 26, 2016;

[0046] U.S. Provisional Application No. 62 / 399,436, filed September 25, 2016;

[0047] U.S. Provisional Application No. 62 / 399,429, filed September 25, 2016;

[0048] U.S. Provisional Application No. 62 / 287,901, filed January 28, 2016;

[0049] U.S. Provisional Application No. 62 / 279,784, filed January 17, 2016;

[0050] U.S. Provisional Application No. 62 / 275,241, filed January 6, 2016;

[0051] U.S. Provisional Application No. 62 / 275,222, filed January 5, 2016;

[0052] U.S. Provisional Application No. 62 / 259,991, filed November 25, 2015;

[0053] U.S. Provisional Application No. 62 / 254,718, filed November 13, 2015;

[0054] U.S. Provisional Application No. 62 / 139,754, filed March 29, 2015;

[0055] U.S. Provisional Application No. 62 / 120,316, filed February 24, 2015; and

[0056] U.S. Provisional Application No. 62 / 119,521, filed February 23, 2015.

[0057] All of the above non-provisional, provisional and international patent applications are collectively referred to herein as the “commonly assigned consolidated application.” Technical Field

[0058] The present invention generally relates to endoscopes. More particularly, some embodiments relate to a portable endoscope device including a reusable handle portion and a disposable or single-use cannula portion. Background Art

[0059] Conventional rigid and flexible endoscopes have relatively expensive lenses or fiber optic systems and are used repeatedly. Therefore, they must be rigorously sterilized and disinfected after each use. Disposable endoscopes are an emerging category of endoscopic instruments. In some cases, the manufacturing cost of an endoscope can become cheap enough to be used on a single patient. Disposable or single-use endoscopes reduce the risk of cross-contamination and nosocomial infections.

[0060] The subject matter described or claimed in this patent specification is not limited to specific embodiments that solve any particular disadvantages or operate only in environments such as those described above. Instead, the above background is provided only to illustrate the feasibility of some embodiments described herein in exemplary technical fields. Summary of the Invention

[0061] In some embodiments, a first camera system of a multi-camera, multi-spectral endoscope includes: a disposable cannula insertable into a patient; a first camera and a first light source and a second camera and a second light source, all disposed at a front end of the cannula; wherein: the first light source is configured to emit light primarily within a first wavelength range, and the second light source is configured to emit light primarily within a second wavelength range different from the first wavelength range; the fields of view of the first camera and the second camera and the illumination fields of the first light source and the second light source at least partially overlap, so that the two cameras observe the same target of the patient at substantially the same time, and the same target is illuminated by the two light sources at substantially the same time; the first camera includes a first two-dimensional (2D) image sensor and a first color filter, and the second camera includes a second two-dimensional (2D) sensor and a second color filter that allows a wavelength different from that of the first color filter to pass; a processing system receives images captured by the first camera and the second camera and processes the images into a composite image, superimposing an image of a selected portion of the target captured by the first camera on an image of the target captured by the second camera, the attributes of the images being different from those of the rest of the target, thereby highlighting the selected portion of the target; a display receives the composite image from the processing system and displays at least some of the received composite images.

[0062] The endoscope may further include one or more of the following features: (a) a reusable portion is optionally fixed to the cannula and carries the display, wherein the display includes a second camera system having a field of view including the front end of the cannula, wherein the display is configured to optionally display images from the second camera system and the composite image, so that the user can view the image of the front end of the cannula when the cannula is inserted into the patient and view the composite image after the insertion; (b) the spatial resolution of the first camera is lower than that of the second camera but the sensitivity is higher; (c) the first light source emits light for fluorescent imaging, the second light source emits white light, the first camera and the first filter are configured to mainly image fluorescence from targets in the patient's body, and the second camera and the second The filter is configured to primarily image reflected white light from the target; (d) the first light source optionally emits light different from fluorescence imaging or blue light, the second light source emits white light, the first camera and the first filter are configured to selectively image primarily fluorescence or reflected blue light from the target in the patient, and the second camera and the second filter are configured to primarily image reflected white light from the target; (e) the first and second cameras and the first and second light sources optionally operate in the following situations: (i) a blue mode in which the first light source is turned on but the second light source is turned off, the first camera captures a fluorescence image in which the blue background is filtered out, and the second camera captures a fluorescence image plus the primary blue background; (ii) a white mode in which the first light source is turned on but the second light source is turned off, the first camera captures a fluorescence image in which the blue background is filtered out, and the second camera captures a fluorescence image plus the primary blue background; (ii) a white mode in which the first light source is turned on but the second light source is turned off, the first camera captures a fluorescence image in which the blue background is filtered out, and the second camera captures a fluorescence image plus the primary blue background; (f) the processing system is configured to spatially correlate or correspond the captured images, i.e., the blue mode, and generate a first corrected and enhanced image by combining features of the two; (g) the processing system is configured to spatially correlate or correspond the captured images, i.e., the white mode, and generate a second corrected and enhanced image by combining features of the two; (h) the processing system is configured to combine the first corrected and enhanced image with the second corrected and enhanced image to generate the composite image; (i) the cannula includes two channels, each of which the channels being configured to function as fluid passages for the flow of fluid into or out of a patient or as working channels for surgical tools, whereby one of the channels can be used to remove fluid or debris from a patient during a procedure performed with a surgical tool passing through the other channel; (j) a fluid hub at a rearward end of the cannula, wherein the cannula is configured to rotate relative to a forward portion of the fluid hub about a longitudinal cannula axis; and (k) a fluid hub secured to the rearward end of the cannula and a reusable portion optionally secured to the fluid hub, the reusable portion including a thumb lever operably coupled to the forward end of the cannula and configured to flex the forward end of the cannula relative to the remainder of the cannula upon manual manipulation of the thumb lever;(l) a fluid hub at the rear end of the cannula, the reusable portion being selectively secured to the fluid hub via relative linear motion and quarter-turn relative rotational motion; (m) the reusable portion including a thumb lever and a drive gear driven thereby, the fluid hub including a driven gear meshing with the drive gear and operatively connected to the front end of the cannula to bend the front end in a selected direction in response to manual operation of the thumb lever;

[0063] In some embodiments, the endoscope includes: a disposable cannula that can be inserted into a patient's body; a first camera system located at the front end of the cannula; a reusable part that is located at the rear end of the cannula and is optionally connected to the cannula; a display carried by the reusable part; a second camera system carried by the display, the second camera system having a field of view including the front end of the camera; wherein the display is configured to display images captured by the second camera system and to display the area around the front end of the cannula when the cannula is inserted into the patient's body, and to display images captured by the first camera system after the cannula is inserted into the patient's body.

[0064] The aforementioned endoscope further includes one or more of the following features: (a) the second camera system includes two cameras spaced apart from each other in a direction transverse to the longitudinal axis of the cannula and provides depth-of-field images of the front end of the cannula and its surroundings; (b) the first camera system includes a first camera that captures images within a first wavelength range and a second camera that captures images within a different wavelength range; and (c) further includes a processing system configured to combine various aspects of the images captured by the first and second cameras into a composite image to enhance medically observable anatomical features.

[0065] In some embodiments, an endoscopic method includes: providing a disposable cannula that can be inserted into the body of a patient; optionally connecting the cannula to a reusable portion carrying a display screen; using a first camera located at the front end of the cannula and a second camera also located at the front end of the cannula to simultaneously capture images of the patient's organs, the first camera capturing images within a first wavelength range and the second camera capturing images within a different second wavelength range; processing the images into a composite image, superimposing images of a selected portion of the target captured by the first camera on images of the target captured by the second camera, the attributes of these images being different from the rest of the target, thereby highlighting the selected portion of the target; and displaying at least some of the received composite images.

[0066] The method may further include capturing an image of the front end of the cannula using a second camera system carried by the display when the cannula is inserted into the patient's body, and optionally displaying an image of the front end of the camera and its surroundings on the display.

[0067] As used herein, the grammatical conjunctions "and," "or," and "and / or" are intended to indicate that one or more alternatives are possible or exist for the circumstances, objects, or subjects they connect. In this manner, as used herein, the term "or" in all instances means an inclusive or, not an exclusive or.

[0068] As used herein, the term "surgery" or "procedure" refers to any physical intervention with patient tissue and does not necessarily involve cutting patient tissue or closing a pre-existing wound. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] To further illustrate the above and other advantages and features of the subject matter protected by this patent specification, specific embodiments are described with reference to the accompanying drawings. These drawings should be understood as depicting only exemplary embodiments and should not be considered as limiting the scope of protection of this patent specification or the appended claims. The subject matter of the present invention is described and explained with specificity and detail using the following drawings, in which:

[0070] Figure 1A 、 Figure 1B Figure 1C are side, top, and rear views of a portable ergonomic endoscope with a disposable cannula in some embodiments of the present invention;

[0071] Figure 2A and Figure 2B is a perspective view of a portable ergonomic endoscope with a disposable cannula in some embodiments of the present invention;

[0072] Figure 3A and Figure 3B is a perspective view illustrating the engagement and separation of a reusable portion and a disposable portion of a portable ergonomic endoscope in some embodiments;

[0073] Figure 4A and Figure 4B are perspective and schematic views of a front end tip including multiple cameras and illumination modules for use with a portable ergonomic endoscope in accordance with some embodiments of the present invention;

[0074] Figure 5 is a schematic diagram of a dual-camera dual-light source system for multispectral imaging and surgical applications in some embodiments;

[0075] Figure 6 is a conceptual diagram illustrating design aspects of a dual-camera, dual-light source system for multispectral imaging and surgical applications in some embodiments;

[0076] Figure 7 is a diagram illustrating possible color filter array configurations for a dual-camera, dual-light source system for multispectral imaging and surgical applications in some embodiments;

[0077] Figure 8 This is a graph showing the quantum efficiency of Nyxel and traditional pixels versus wavelength;

[0078] Figure 9 is a schematic diagram illustrating further aspects of combining multispectral image data from a dual-camera, dual-light source system in some embodiments.

[0079] Figure 10 is a perspective view illustrating a combined, spatially registered image displayed to a user on an endoscopy system in some embodiments; and

[0080] Figure 11 is a perspective view of an endoscope system having one or more front-facing cameras in some embodiments. DETAILED DESCRIPTION

[0081] A detailed description of preferred embodiments is provided below. Although several embodiments are described, it should be understood that the new subject matter described in this patent specification is not limited to any one embodiment or combination of embodiments described herein, but includes many alternatives, modifications, and equivalents. In addition, although many specific details are set forth in the following description in order to provide a thorough understanding, some embodiments can still be implemented without some or even all of these details. Moreover, for the sake of clarity, certain technical materials known in the prior art are not described in detail to avoid unnecessarily diluting the new subject matter described herein. It should be clear that the various features of one or several specific embodiments described herein can be used in combination with the features or other features of other described embodiments. In addition, the same reference numerals and indications in the various figures represent the same elements.

[0082] Some embodiments describe a portable ergonomic endoscope system that includes an imaging system having at least two independent cameras and two independent light sources. The cameras and light sources are configured to simultaneously image a target object (e.g., tissue). By using different lighting, different filters, and controlling the spectral response, different features of the target object can be captured. In some embodiments, a system processor can coordinate the cameras, light sources, and combine the resulting images to display an enhanced composite image of the target object to the user. In some embodiments, the system can be configured to perform NBI (narrow band imaging). In some embodiments, the system can also be configured to perform fluorescence imaging.

[0083] As used herein, a color filter array (CFA) refers to a filter placed on a pixel to allow a certain bandwidth of light to pass through. Conventional consumer cameras, such as those in mobile phones, use RGB CFAs. Specialized CFAs can be designed for other specialized applications.

[0084] As used herein, narrowband imaging (NBI) refers to a color imaging technique used in endoscopic diagnostic medical tests in which specific blue and green wavelengths of light are used to enhance detail in certain aspects of mucosal surfaces. In some embodiments, a special filter can be electronically activated by a switch in the endoscope, forcing the use of ambient light with wavelengths preferably of 415 nanometers (blue) and 540 nanometers (green). Because hemoglobin's peak light absorption occurs at these wavelengths, blood vessels appear very dark, improving their visibility and enabling better identification of other surface structures.

[0085] As used herein, fluorescence imaging (FI) refers to imaging of fluorescence, sometimes using fluorescent dyes, to label, highlight, or enhance certain biological mechanisms and / or structures. Fluorescence itself is a form of luminescence produced by the emission of light of a certain wavelength by a substance after absorbing electromagnetic radiation. For example, in blue light endoscopy, a fluorescent dye (Hexvix) is injected into the bladder. The tissue is then illuminated with blue light (approximately 405 nm), and Hexvix emits fluorescence at a wavelength of approximately 610 nm. Note that in FI, the camera sees fluorescence emitted from within the object, while in NBI, the camera sees the reflection of light of various bandwidths from the object.

[0086] In some embodiments, a novel dual-camera and dual-light source (DCDL) system is described for multispectral or multicolor imaging. Embodiments of surgical applications with simultaneous white light, fluorescence, and infrared imaging are disclosed.

[0087] The method is applicable to general multispectral, multiband imaging. Some embodiments include an endoscope system comprising two independent camera / LED systems integrated into the same cannula or endoscope. A white light camera, referred to as camera W, is coupled with a white light LED, referred to as light source W. A fluorescence camera, referred to as camera F, is coupled with a blue light LED, referred to as camera C. In this configuration, when either or both light source C and light source W are turned off, camera F functions as an infrared camera.

[0088] In some embodiments, camera W is optimized for white-light endoscopy, using strong, high-quality white LEDs to illuminate the object, resulting in high image resolution. Camera F is optimized for sensitivity, as fluorescent light sources are typically weak. To maximize the sensitivity and signal-to-noise ratio of the CMOS sensor pixels for high-quality imaging, the following measures are implemented.

[0089] In some embodiments, a special color filter array (CFA) is used on the pixel array (e.g. Figure 7(as shown), such that the CMOS sensor array is sensitive to the red or infrared spectrum (near 600 nm or higher). In some embodiments, to further improve sensitivity, it is preferred to use relatively large pixels (e.g., 2.2 μm x 2.2 μm) for the CMOS sensor of camera F. In this case, camera F preferably has a lower spatial resolution than the pixels of camera W (e.g., 1.75 μm x 1.75 μm or 1.0 μm x 1.0 μm), but much higher sensitivity.

[0090] Figure 1A 、 Figure 1B and Figure 1C The figures are side, top, and rear views of a portable, ergonomic endoscope with a disposable cannula, in some embodiments. System 100 is designed for simple and quick use, minimal patient discomfort, and high placement accuracy. System 100 consists of a disposable, or single-use, portion 102 and a reusable portion 104. These two portions 102 and 104 can be mated and separated via a connector, as described in further detail below. The cannula 120 includes an imaging and illumination module at its front end 110. A cable (not shown) located within the cannula provides control signals and power to the camera and LED illumination module on the front end 110 and transmits video image data from the imaging module to a handle 140 and display 150 for user viewing. In the illustrated embodiment, the handle 140 includes two control buttons 142 and 144, which can be configured for power on / off and image capture, respectively. In some embodiments, the handle 140 is shaped like a pistol grip, as shown, and includes a rechargeable battery 141, accessible through a battery door 148. In some embodiments, the battery 141 is an 18650 lithium-ion battery. Also included within the handle 140 is an electronics module 143 mounted on a printed circuit board (PCB) 145. The electronics module 143 and the PCB 145 are configured to perform various processes, such as video processing and capture, Wi-Fi transmission of data to external devices, lighting control, user interface processing, and diagnostics. The electronics module 143 is also configured to include at least one non-volatile memory module for storing videos and images captured from the imaging module. In some embodiments, the display 150 can be both tilted and swiveled to provide the user with an optimal viewing angle. The swivel joint 152 is configured to provide rotation of the display 150, such as Figure 1C As shown by the dashed arrow in FIG, the hinge joint 154 is configured to provide a rotation of the display 150, as shown in FIG. Figure 1B. In some embodiments, the hinge joint is configured to allow the display at the front end to be tilted approximately 90 degrees, or close to 90 degrees. Such tilting is useful, for example, when giving the operator an unobstructed or less obstructed view. The handle 140 also includes a thumb stick 146 that can be moved upward or downward, as indicated by the dashed arrow. Moving the thumb stick 146 up and down causes the front end 110 to bend upward and downward, as indicated by dashed lines 180 and 182, respectively. Further details regarding the operation of the thumb stick 146 to control the steering of the front end 110 and the cannula 120 are provided in U.S. patent application No. 17 / 362,043, filed on June 29, 2021, which is incorporated herein by reference and referred to herein as the "'043 application."

[0091] The rear end of the cannula 120 is connected to a fluid hub 172, which in this embodiment includes two fluid ports 132 and 134. At the rear end of the fluid hub is a collar 168. In some embodiments, the collar 168 is configured to be rotatable to allow for a "plug and twist lock" type of mating of the disposable portion 102 and the reusable portion 104, as will be described in further detail below. In some embodiments, at least a portion of the fluid hub 172, along with the cannula 120 and the front end 110, can be manually rotated relative to the handle 140 along the primary longitudinal axis of the cannula 120, as indicated by the solid arrow 124. Thus, the rotatable portion of the fluid hub 172 causes the cannula 120 and the front end 110 to rotate, as indicated by the solid arrow 122. In some embodiments, the combination of rotating the cannula 120 and the front end 110 and moving the thumb lever 146 allows the user to "steer" the direction of the front end 110 as desired. In some embodiments, the preferred working length of cannula 120 is approximately 12 inches, although shorter or longer lengths may be used depending on the medical application, and the preferred outer diameter is 5.5 to 6.5 inches, although larger or smaller diameters may be used depending on the medical application and advances in camera and lighting technology.

[0092] Figure 2A and Figure 2B is a perspective view of a portable ergonomic endoscope with a disposable cannula in some embodiments. Figure 2A A syringe 230 is shown for supplying fluid, such as saline, through a fluid chamber (not shown) within the cannula 120 via tubing 232, a connector 234, and a fluid port 134. In some embodiments, the cannula 120 is semi-rigid. The cannula 120 has sufficient rigidity so that it does not collapse under the longitudinal thrusts and pulls expected during the medical procedure in which it is to be performed. On the other hand, the cannula 120 is sufficiently flexible so that it can bend when navigating tortuous anatomical structures.

[0093] Figure 3A and Figure 3BThe figure is a perspective view illustrating the connection and disconnection of the reusable and disposable parts of the portable ergonomic endoscope in some embodiments. The disposable part 102 and the reusable part 104 are connectable and disconnectable via mechanical and electrical connectors. The electrical connection is made via the USB-C plug 302 ( Figure 3A ) and the USB-C socket 304 ( Figure 3B ). The mechanical connection includes both a structural connection that fixedly connects the disposable portion 102 and the reusable portion 104, and a steering connection by which the steering input from the steering structure of the reusable portion 104 can be transferred to the steering component of the disposable portion 102. In this embodiment, the structural connection includes a male circular portion 312 on the disposable portion 102, which is shaped to mate with a female socket 314 on the reusable portion 104. The structural connection also includes a twist-lock mechanism in which the male portion 322 can be inserted into the female opening 324 and then locked by twisting the male portion 322 approximately a quarter turn (90 degrees). The twisting action can be manually implemented by a textured or knurled ring 168. In this way, the connection can be configured as a "plug-in" connection. The steering connection is achieved by engaging the drive gear 334 on the reusable portion 104 with the driven gear 332 on the disposable portion 102.

[0094] Figure 4A and Figure 4B are stereoscopic and schematic diagrams of a front end including multiple cameras and lighting modules for a portable ergonomic endoscope in some embodiments. Figure 4A In the figure, front end 110 is shown connected to the front end of cannula 120. In some embodiments, front end 110 includes a housing 410 that is molded separately from the front end of cannula 120 and bonded together. Housing 410 houses two camera modules: camera F module 420 and camera W module 430. Each camera F 420 and camera W 430 module includes a lens and a sensor. The sensor of each camera F 420 and camera W 430 includes a color sensor, a color filter array, and electronics and circuitry, as described in further detail below. Flanking camera F module 420 are two blue LEDs 422 and 424 configured to emit laser light suitable for fluorescence endoscopy. In some embodiments, blue LEDs 422 and 424 are configured to emit light at approximately 410 nanometers (violet-blue). Flanking camera W module 430 are two white light LEDs 430 and 434 configured to emit white light suitable for visible white light endoscopy. Figure 4AAlso shown is a port 412, which is configured to provide fluid (into or out of the patient) and / or provide an opening through which a tool or other device may pass (e.g., a needle). Figure 4A A total of four LEDs (two white and two blue) are shown, but generally, other numbers of LEDs may be provided depending on factors such as the desired quality of illumination, the size of the endoscope, and the characteristics of the LEDs, such as size and brightness. In some embodiments, three or fewer LEDs may be provided, and in some embodiments, ten or more LEDs may be provided. Furthermore, the number of white and blue LEDs need not be equal, as this also depends on various factors. Groups of three, four, or more LEDs may be used. Other light sources may also be used, such as optical fibers that transmit light generated elsewhere.

[0095] exist Figure 4B In the embodiment shown, there are two separate devices / fluid channels 414 and 416. In this case, both have an inner diameter of 2.2 mm. In some embodiments, channel 414 can be connected to fluid port 134 ( Figure 1A ), while channel 416 is connected to fluid port 132 ( Figure 1A In some embodiments, to improve sensitivity to fluorescence, the CMOS sensor of camera F 420 is configured with larger pixels than that of camera W 430. For example, the pixels of camera F may be 2.2um x 2.2um, arranged in a 400x400 matrix size, while the pixels of camera W are 1.0um x 1.0um or 1.75um x 1.75um, arranged in a higher spatial resolution matrix size. Because white light LEDs tend to be relatively intense, the camera W 430 module may include a CMOS sensor with smaller pixels, such as 1.75um x 1.75um or 1um x 1um, thereby achieving higher spatial resolution with a matrix size of up to 720x720.

[0096] In some embodiments, camera F 420 is used to perform blue light (fluorescence) endoscopy with a portion of the CFA. One embodiment is shown in FIG. Figure 7 In some embodiments, an infrared camera is used as camera F.

[0097] Figure 5The figure is a schematic diagram of a dual-camera, dual-light source system for multispectral imaging and surgical applications in some embodiments. As shown, front end 110 includes camera and illumination modules, namely, camera F, light source C, camera W, and light source W. Camera F 420 is configured to capture images of a specific color or bandwidth, such as a narrowband fluorescence centered around 610 nanometers. The optical filters in camera F 420 are designed to block incident light of other wavelengths, for example, by using a specially designed CFA array. Camera F can be used for either NBI or FI, depending on the specific application. Light source C (422 and 424) used in camera F 420 can be a laser for fluorescence imaging or simple blue or green light for NBI. LEDs or specialized light sources can also be used. In some embodiments, camera W 430 is a conventional white light camera, such as a cell phone camera. A typical RGB CFA can be used, and an infrared filter can also be used. Typically, an infrared filter that blocks 50% of wavelengths above 650 nanometers is used. Light source W (432 and 434) of light source W can be LED lights with various hues that approximate daylight. Cannula 120 includes cables 450 and 452. Image F refers to the image captured by camera F, which can be fluorescence or, in the case of NBI, a reflection of green or blue light. Image W refers to the image captured by camera W, which can be fluorescence or, in the case of NBI, a reflection of green or blue light.

[0098] Because the endoscope has two cameras that can operate simultaneously and with different lighting combinations, such as light source C and light source W (or other light bands), the system takes advantage of having two "eyes" looking at the same target, but seeing different aspects of the target at the same time, thereby extracting more information from the object and target. For example, when the blue light is on, camera F sees most of the fluorescence emission, while camera W sees the object's reflection of light source C (which may be very strong) and a little fluorescence at the same time. Because the two cameras are synchronized and relatively spatially aligned, different types of comprehensive information are delivered to the user, improving the clinical experience compared to seeing only one of the two types of information about the object or target.

[0099] In some embodiments, Nyxel technology, developed by OmniVision, can be used. Nyxel pixels, used in the F 420 camera, offer significantly improved pixel sensitivity, particularly in the red and near-infrared bandwidths. This is particularly useful for detecting fluorescence around 610 nanometers.

[0100] Front-end processing and main system processing are performed in the electronics module 143. In some embodiments, images are synthesized and displayed on the display 150.

[0101] Figure 6This is a conceptual diagram illustrating design aspects of a dual-camera, dual-light source system for multispectral imaging and surgical applications in some embodiments. Generally, it is desirable to capture multicolor or multispectral images of a target object (e.g., human tissue). Typically, a visible light image of the object plus images captured using other color bands are used to better describe the target tissue and shape. Two cameras (camera F and camera W) are associated with two light sources (light source C and light source W). Camera F is an optical camera sensitive to certain color bands, such as red and infrared. The output of camera F is image F. Light source C is a light source (C band) other than white light. In dual-spectral imaging (DBI), light source C can be green or blue. In fluorescence imaging, it can also be a light source that excites an object to emit fluorescent colors. Camera W is an optical camera sensitive to certain color bands (B), such as white light. The output of camera W is image W. Light source W is a light source that emits a specific color band B, such as white light.

[0102] Figure 7 This figure shows possible color filter array configurations for a dual-camera, dual-light source system used for multispectral imaging and surgical applications in some implementations. In some embodiments, Camera F uses Nyxel pixels (from Omnivision) and a "red-only" filter array, namely the CamF RRRR filter. This configuration allows red and / or infrared wavelengths to pass while filtering out background blue and green light.

[0103] Compared to Nyxel CFA or Old CFA, Camera F can achieve four times the red resolution because one out of four pixels in the Nyxel or Old CFA array is used to capture red. Figure 7 Every pixel in the array of camera F is used to capture the color red.

[0104] Figure 8 The relationship between quantum efficiency and wavelength for Nyxel and conventional pixels is shown in this figure. The quantum efficiency is shown for the Nyxel pixel, a new sensor developed by OminiVision. Curve 810 represents a Nyxel blue pixel. Curve 812 represents a conventional blue pixel. Curve 820 represents a Nyxel green pixel. Curve 822 represents a conventional green pixel. Curve 830 represents a Nyxel red pixel. Curve 832 represents a conventional red pixel. Curves 830 and 832, in particular, show that Nyxel red pixels are significantly more sensitive to the red or infrared wavelength range than conventional red pixels.

[0105] Figure 9Further aspects of multi-band image data in conjunction with a dual camera, dual light source system are shown in some embodiments. With the availability of a global shutter capable camera F, camera W can capture image frames with different combinations of light source C and light source W being "on" or "off". In "Surgical Example 1", light source C (blue light) is "on", but light source W is "off", and the images captured are Figure F of camera F and Figure WB of camera W. Figure F and Figure WB are spatially aligned or related. This is possible due to a short time lag between the images taken by the different cameras (or complete synchronization when both cameras are taking pictures at the same time). Figure WB provides a background image under the illumination of light source C, which can be used to correct the background of Figure F. When only light source C is on, the Figure F data is composited with Figure WB to produce "eImgB".

[0106] In the case of blue-light endoscopy, the signal-to-noise ratio in Figure F is low (due to the weak fluorescence signal), so a CMOS sensor with high signal-to-noise ratio pixels is used. On the other hand, Figure W has a higher signal-to-noise ratio (due to the strong white light), so a CMOS sensor with smaller pixels can be used to improve spatial resolution.

[0107] In "Surgical Example 2", camera F is used to capture Figure 1R when light source C is "off". Figure W captures a standard white light image with light source W "on". In this case, Figure 1R provides a "heat map" of the target; it is useful when using energy devices such as lasers or radiofrequency for tissue modification. Figure 1R can alert the user to hot spots or cold spots. The data in Figure 1R and Figure W can be spatially aligned or associated, which is also due to the short time difference (or no time difference) between the images taken by different cameras. Figure 1R and Figure W can also be combined or superimposed to provide the precise location of hot spots and cold spots. That is, hot spots and cold spots can be viewed against the background of an ordinary standard white light image, providing the viewer with a positioning context for the hot spots and cold spots.

[0108] In "Surgical Example 3," Image W is combined with eImgB. By combining Examples 1 and 2, high-quality eImgB data is spatially registered with the white-light image Image W. The observer can obtain either the high-resolution Image W, the fluorescent eImgB, or a composite of both. In some embodiments, surgeons can utilize the existing images to better visualize their target. Fluorescent eImgB, white-light Image W, and infrared Image IR seamlessly switch between different visualization modes.

[0109] In the fourth "Example 4" (not in Figure 9 As clinical cases accumulate, artificial intelligence algorithms (or machine learning) can be designed for automatic diagnosis.

[0110] Figure 10is a stereogram in which, in some embodiments, a combined, spatially registered image is displayed to a user on an endoscope system. In the displayed view, a normal white light image (image W) 1020 is displayed on a large portion of the display screen 150. The embodiment shown is Figure 9 FIG3 shows "Example 3" in which the eImgB image is combined with a standard white light image (FIG. W) and spatially registered. In this case, regions 1010 and 1012 are derived from the eImgB data and clearly show the cancerous tumor. The user can easily view the cancerous regions 1010 and 1012 in spatial registration with the normal color image of the surrounding tissue. This blending or combination provides a greatly enhanced view of the target tissue. In some embodiments, the user can switch between the two images by pressing a toggle button, such as button 142, button 144 (in the Figure 1B and 2B ), or by using the soft button 1040 on the touch display 150 to easily switch between different modes (such as embodiment 1, 2 or 3).

[0111] Figure 11 is a stereoscopic view of an endoscope system having one or more forward-facing cameras in some embodiments. The embodiment shown has two forward (distal) cameras 1140 and 1142. The forward-facing cameras allow the user to see exactly where the tip is without having to move the screen away. During a surgical procedure, particularly immediately following or during initial insertion of the tip 110, the user's line of sight can be primarily focused on the display screen 150. With the forward-facing cameras 1140 and 1142, the precise location of the tip and its surroundings can be seen on the display screen 150. Image enhancement, such as artificially providing depth of field, may be beneficial in certain procedures. Two cameras or other means (such as lidar imaging) can be used to simulate depth of field centered on the tip to improve usability.

[0112] Although the foregoing has been described in detail for the sake of clarity, it will be apparent that certain changes and modifications may be made without departing from the principles of the invention. It should be noted that there are many alternative ways of implementing the processes and apparatus described herein. Therefore, the present embodiments should be considered illustrative rather than restrictive, and the body of work described herein is not to be limited to the details given herein, which may be modified within the scope and range of equivalents of the appended claims.

Claims

1. A multi-camera, multi-spectral endoscope comprising: a disposable cannula (120) for insertion into a patient; The first camera (420) and the first light source (422, 424) as well as the second camera (430) and the second light source (432, 434) are all disposed at the front end (110) of the cannula; wherein: The first light source is configured to emit light primarily within a first wavelength range, and the second light source is configured to emit light primarily within a second wavelength range different from the first wavelength range; the fields of view of the first camera and the second camera and the fields of illumination of the first light source and the second light source at least partially overlap, so that the two cameras observe the same target on the patient at substantially the same time, and the same target is illuminated by the two light sources at substantially the same time; The first camera includes a first two-dimensional (2D) image sensor and a first color filter, and the second camera includes a second 2D sensor and a second color filter that is different from the first color filter in terms of wavelength allowed to pass; a processing system receiving images captured by the first camera and the second camera and processing the images into a composite image that superimposes an image of a selected portion of an object captured by the first camera on an image of the object captured by the second camera, the images having attributes different from those of the remaining portion of the object, thereby emphasizing the selected portion of the object; and The display screen receives the composite image from the processing system and displays at least a portion of the received composite image, The invention comprises a reusable portion (104) which is fixed to the cannula and carries a display, wherein the display includes a second set of camera systems (1140, 1142) whose field of view includes the front end of the cannula, and wherein the display is configured to display images from the second set of camera systems and the composite image, whereby a user can view images of the front end of the cannula while the cannula is being inserted into the patient and view the composite image after insertion.

2. The endoscope according to claim 1, wherein: The first camera has lower spatial resolution than the second camera, but higher sensitivity.

3. The endoscope according to claim 1, wherein: The first light source emits light for fluorescence imaging, the second light source emits white light, and the first camera and the first filter are restricted to primarily imaging fluorescence from a target in the patient, and the second camera and the second filter are configured to primarily image reflected white light from the target.

4. The endoscope according to claim 1, wherein: The first light source emits a light source or blue light for exciting fluorescence, the second light source emits white light, the first camera and the first filter are configured to selectively image mainly the fluorescence or reflected blue light from the target in the patient's body, and the second camera and the second filter are configured to mainly image the reflected white light from the target.

5. The endoscope according to claim 1, wherein: the first and second cameras and the first and second light sources: a. Blue mode: The first light source is turned on, but the second light source is turned off. The blue background is filtered out in the fluorescent image captured by the first camera, while the fluorescent image captured by the second camera is added with the main blue background. and b. White mode: The second light source is turned on, but the first light source is turned off. The first camera captures red or infrared images, and the second camera mainly captures standard white light images.

6. The endoscope according to claim 5, characterized in that: The processing system is configured to spatially correlate or register the captured image with the blue pattern and generate a first corrected and enhanced image by combining features of both.

7. The endoscope according to claim 6, characterized in that: The processing system is configured to spatially correlate or register the captured image with the white pattern and generate a second corrected and enhanced image by combining features of both.

8. The endoscope according to claim 7, characterized in that: The processing system is configured to combine the first corrected and enhanced image with the second corrected and enhanced image to produce a composite image.

9. The endoscope according to claim 1, wherein: The cannula includes two channels (414, 416), each channel configured to serve as a fluid passage for fluid flow into or out of a patient or a working channel for a surgical tool, whereby one of the channels can be used to clear fluid or debris from the patient during surgery using a surgical tool passing through the other channel.

10. The endoscope of claim 1, comprising a fluid hub (172) located at a rearward end of the cannula, wherein the cannula is configured to rotate with the forward end portion of the fluid hub relative to the rearward end portion of the fluid hub about a longitudinal cannula axis.

11. The endoscope according to claim 1 comprises a fluid hub (172) fixed to the rear end of the cannula and a reusable portion (104) fixed to the fluid hub, wherein the reusable portion comprises a thumb lever (146) operatively connected to the front end of the cannula and configured to bend the front end of the cannula relative to the rest of the cannula by manual operation of the thumb lever.

12. The endoscope of claim 1, comprising a fluid hub (172) at the rear end of the cannula and a reusable portion (104) secured to the fluid hub by a quarter-turn relative rotational motion followed by a relative linear motion.

13. The endoscope according to claim 12, characterized in that: The reusable portion includes a thumb lever, a drive gear driven thereby, and the fluid hub includes a driven gear meshing with the drive gear and operatively connected to the front end of the cannula to bend the front end to a selected direction according to manual operation of the thumb lever.

14. An endoscope comprising: a disposable cannula (120) for insertion into a patient; A first camera system (420, 430) is located at the front end of the cannula; a reusable portion positioned at the rear end of the cannula and connected to the cannula; a display (150) carried on the reusable portion; a second camera system (1140, 1142) carried by the display, the second camera system having a field of view including the leading end of the cannula; The display is configured to display images captured by the second camera system and to display the area around the front end of the cannula when the cannula is inserted into the patient's body, and to display images captured by the first camera system after the cannula is inserted into the patient's body.

15. The endoscope according to claim 14, characterized in that: The second camera system includes two cameras spaced apart from each other in a direction transverse to the longitudinal axis of the cannula and provides depth-of-field images of the leading end of the cannula and its surroundings.

16. The endoscope according to claim 14, wherein: The first camera system includes a first camera that captures images within a first wavelength range and a second camera that captures images within a different wavelength range.

17. The endoscope of claim 16, comprising a processing system configured to combine aspects of the images captured by the first and second cameras into a composite image to enhance anatomical features of medical interest.

Images