Capsule endoscope with dynamically adjustable color illumination spectrum

Abstract

In a capsule endoscope device, system and method for dynamically adjusting a color illumination spectrum, LEDs of different color groups can emit light corresponding to different wavelength ranges recorded by an image sensor. A driving circuit can deliver driving currents during non-overlapping time pulses to independently activate each different color group of LEDs. A single color group of LEDs can be independently activated at any time, and different color groups of LEDs can be sequentially activated in successive time pulses to simulate white light or a multicolor illumination spectrum over multiple time pulses. The active modes of the color groups of LEDs can be dynamically adjusted in real time to achieve a flexible and customizable illumination spectrum. The illumination spectrum is ideally used for imaging a variety of different environments, for example, within the gastrointestinal tract (GI).

Description

[Technical field]

[0001] [Cross-references to related literature] This application claims the benefit of U.S. Patent Application No. 17 / 124,277, filed December 16, 2020, the entire contents of which are incorporated herein by reference.

[0002] SUMMARY OF THE DISCLOSURE Embodiments of the present invention relate to ingestible or intracorporeal capsule endoscopic imaging devices and systems, for use, for example, in imaging of the gastrointestinal (GI) tract. [Background technology]

[0003] Conventional capsule endoscopes use light-emitting diodes (LEDs) as illumination sources. Conventional LEDs have a fixed and non-adjustable wavelength or illumination spectrum. A typical illumination spectrum for white LEDs combines red, green, and blue LEDs in equal proportions (1:1:1 ratio). These LEDs illuminate the GI locus with a constant spectrum that does not change.

[0004] However, the GI tract is a diverse network of organs, passages, and cavities. Different anatomical regions have different optimal absorption spectra and therefore different corresponding optimal illumination spectra. For example, illumination with wavelengths around 415 nm (blue light) and 540 nm (green light) has maximum absorption peaks in areas of dense superficial vascular networks (e.g., near the bronchial tree and colorectal regions). Conversely, white light illumination spectra are optimal for imaging subepithelial lesions (SELs), which are commonly found in the stomach. Conventional LEDs use the same fixed illumination profile for all areas that the capsule endoscope passes through the GI channel. While a fixed illumination distribution may be optimal for some areas, the same illumination distribution is generally secondary for other areas.

[0005] Thus, there is a long felt need in the art for an illumination source that provides optimal illumination spectra in a variety of different regions of the gastrointestinal tract. Summary of the Invention [Problem to be solved by the invention]

[0006] The embodiments of the present invention address the above long-felt need in the art by providing a type of adaptive illumination source that provides different illumination spectra that are independently determined to be optimal for different corresponding regions of the GI channel. By independently activating different color groups of LEDs in ordered time pulses, adaptive illumination can be achieved to illuminate only a single color group at any given time (and no other color groups are present at the same or overlapping times). The color groups are then cumulatively combined to form white light and / or polychromatic (non-white) spectra in multiple time pulses to illuminate an image. The adaptive illumination source provides the flexibility to customize the illumination profile to be the optimal choice for various environmental imaging.

[0007] In an embodiment of the present invention, a capsule endoscope device, system and method for dynamically adjusting a color illumination spectrum are provided. A capsule-like body may have at least a transparent optical end and encapsulates electronic components therein. An image sensor for recording images inside the body is housed inside the sac behind the at least one transparent optical end. A plurality of different color LED groups can emit light to be recorded by the image sensor. The plurality of different color LED groups correspond to a plurality of different corresponding wavelength sub-ranges. A driving circuit sends a driving current to independently activate each different color LED group during different (e.g., non-overlapping completely or partially) time pulses. This allows a single color LED to be independently activated at any time, and the plurality of different color groups of LEDs are sequentially activated in successive time pulses to simulate a white light or polychromatic illumination spectrum in multiple time pulses. [Means for solving the problem]

[0008] In one embodiment of the present invention, a capsule endoscope device, system, and method for delivering a dynamically adjustable color illumination spectrum is provided. A plurality of different color LED groups can be sequentially activated during fully or partially non-overlapping time pulses, where the plurality of different LED groups illuminate the biological tissue with light in a plurality of different corresponding wavelength sub-ranges. A plurality of different color images can be sequentially stored and recorded in an image sensor, where the plurality of images depict the scattering of light emitted from the corresponding plurality of color groups of LEDs during the fully or partially non-overlapping time pulses. Two or more images from the plurality of different color images can be combined to form a white light or non-white multi-color image.

[0009] In one embodiment of the present invention, a capsule endoscope device, system, and method for emitting a dynamically adjustable color illumination spectrum is provided. A plurality of different color groups of light emitting diodes (LEDs) can be sequentially activated during fully or partially non-overlapping time pulses, where the LEDs of the plurality of different color groups illuminate the biological tissue with light in a plurality of different corresponding wavelength sub-regions. A white light or non-white multicolor image can be recorded on an image sensor. Each stored white light or non-white multicolor image is depicted by scattering of light emitted from two or more accumulations of the LEDs of the plurality of color groups during two or more combinations during the associated fully or partially non-overlapping time pulses. [Brief description of the drawings]

[0010] What is particularly pointed out and distinctly claimed as the subject matter of this invention is the invention, both as to organization and method of operation, together with its objects, features and advantages, which may be best understood by reference to the following detailed description taken in conjunction with the accompanying drawings, wherein:

[0011] [Figure 1A]1 is an exploded schematic diagram of an intracorporeal capsule endoscope having dynamically adjustable color illumination spectrum and its components according to some embodiments of the present invention.

[0012] [Figure 1B] 1B is a schematic diagram of an external magnetic control system for moving the intracorporeal capsule endoscope of FIG. 1A within the GI channel according to some embodiments of the present invention.

[0013] [Figure 2A] 1 is a schematic diagram of multiple color group LEDs and their respective illumination spectra for producing white light illumination according to some embodiments of the present invention.

[0014] [Figure 2B] 1 is a schematic diagram of LEDs in a single color group divided into multiple color groups by filters according to some embodiments of the present invention.

[0015] [Figure 3A] 11 is a graph of intensity of multiple wavelength sub-ranges corresponding to multiple different corresponding color groups of LEDs according to some embodiments of the present invention.

[0016] [Figure 3B] 3B is a diagram of an illumination spectrum of intensity for three exemplary wavelength ranges corresponding to three different corresponding color groups of the LEDs of FIG. 3A according to some embodiments of the present invention.

[0017] [Figure 3C] 1 illustrates illumination spectra of wavelength range intensity of sunlight (e.g., AM1.5), an exemplary LED white light, and an exemplary fluorescent light according to some embodiments of the present invention.

[0018] [Figure 4A] 1B is a close-up view of the LED array of FIG. 1A according to some embodiments of the present invention.

[0019] [Figure 4B] 11 is a graph of multiple time pulse modes during which LEDs of different color groups are independently activated according to some embodiments of the present invention. [Figure 4C] 11 is a graph of multiple time pulse modes during which LEDs of different color groups are independently activated according to some embodiments of the present invention. [Figure 4D] 11 is a graph of multiple time pulse modes during which LEDs of different color groups are independently activated according to some embodiments of the present invention.

[0020] [Figure 5A] 1 is a schematic diagram of an image sensor according to some embodiments of the present invention, the image sensor having an exemplary color pattern produced by a color filter array (eg, a Bayer filter).

[0021] [Figure 5B] 1 is a graph of quantum efficiency (QE) over a range of illumination wavelengths for an image sensor according to some embodiments of the present invention.

[0022] [Figure 6A] FIG. 1 is a schematic diagram of a system for illuminating biological tissue to generate white light or polychromatic (non-white light) images according to some embodiments of the present invention.

[0023] [Figure 6B] 6B is a graph of intensities of three exemplary wavelength sub-ranges corresponding to three different corresponding color groups of LEDs, according to some embodiments of the present invention, which are combined to produce the white light image of FIG. 6A.

[0024] [Figure 6C]6B is a graph of an illumination spectrum of intensity for two exemplary wavelength ranges for two different color groups corresponding to LEDs combined to produce the multi-color (non-white light) image of FIG. 6A according to some embodiments of the present invention.

[0025] [Figure 6D] Schematic diagram of biological tissue penetrated at different depths by incident light of different wavelength ranges corresponding to LEDs of different color groups (ii) in accordance with some embodiments of the present invention; (i) corresponding polychromatic (non-white) light absorption spectra and wavelength ranges corresponding to LEDs of different corresponding color groups;

[0026] [Figure 7A] 1A-1C are (i) a top view of an exemplary circuit board for mounting LEDs, (ii) an exemplary circuit diagram of LEDs mounted on the exemplary circuit board, and (iii) a side view of the exemplary circuit board, and (iv) a side view of the exemplary circuit board, in accordance with some embodiments of the present invention.

[0027] [Figure 7B] 1A-1C are schematic diagrams of structures and packages capable of housing an LED substrate according to an embodiment of the present invention.

[0028] [Figure 7C] FIG. 1 is a schematic diagram of a driver circuit including multiple dedicated control circuits, each of which independently activates a different corresponding color group of LEDs, according to some embodiments of the present invention.

[0029] [Figure 8] 1 is a table listing different color groups of LEDs and their corresponding peak wavelengths, half widths, and beam angles according to some embodiments of the present invention.

[0030] [Figure 9]1 is a flowchart of a method for operating an intracorporeal capsule endoscope to emit a dynamically adjustable color illumination spectrum according to some embodiments of the present invention.

[0031] It will be appreciated that for simplicity and clarity of illustration, elements shown in the figures have not necessarily been drawn to scale. For example, the size of some elements may be shown exaggerated relative to other elements for clarity. Further, where considered appropriate, reference numerals may be repeated among the figures to indicate corresponding or analogous elements. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0032] Color is a property of electromagnetic waves and refers to wavelengths within the visible or invisible (e.g., IR) range of light, which may be divided into subranges or groups called "color groups", such as the red, blue, and green color groups (within the visible light range) and the IR color group (non-visible light range).

[0033] A capsule endoscope according to an embodiment of the present invention (e.g., as shown in FIG. 1A ) has an illumination source with a dynamically adjustable illumination spectrum. The illumination spectrum is adjusted by independently and sequentially activating each LED (e.g., red, green or blue LEDs, and / or IR LEDs) in the same color group in multiple consecutive time pulses (e.g., red time pulses, green time pulses, and / or blue time pulses, as shown in the time pulse patterns of FIGS. 4B-4D ). At any one time, only the LEDs (e.g., red, green, blue, or IR LEDs) of a single color group are activated, and no other color groups are activated at the same or overlapping times. However, during multiple activation periods, these color groups are combined to cumulatively form a color group combination. Such combinations may be referred to as white light or polychromatic (non-white) light. The activation period or “pulse width” of each color group of LEDs is the amount of time those LEDs are activated and emit light. An image sensor (e.g., as shown in FIG. 5A) can record light during activation of each individual color group alone to generate an archival collection of multiple independent color images (e.g., red, blue, and green images) and / or mixed color images (e.g., white light images). A drive circuit (e.g., as shown in FIG. 7C) can dynamically adjust the combination or sequence of activation color groups of LEDs, the temporal pulse width (e.g., to adjust activation period) and / or the drive current (e.g., to adjust illumination intensity and / or relative emission power) for each activation color group of LEDs to achieve different color combinations of multiple images. These combinations of LED color groups can be mixed in adjustable proportions to optimize the illumination spectrum of the ambient environment imaged in the GI channel. Due to the dynamic and flexible nature of this illumination spectrum, embodiments of the present invention can create optimal illumination profiles for a variety of different anatomical regions to improve the accuracy of images in the GI channel.

[0034] In some embodiments, the dynamic illumination spectrum of the LEDs may be adjusted based on a real-time feedback loop. The feedback loop may be based on visual information from a captured image (e.g., image by image) or based on real-time location information of the exact or approximate location or anatomical region where the capsule is located. The feedback loop inputs captured environmental information (e.g., visual or location information), calculates or searches for new optimal illumination spectrum parameters associated with that environmental information, and outputs new parameters for resetting the LED activation time pulse or drive current. For example, an image that is too red can trigger a drive circuit to shorten the time pulse of the red LEDs. In another example, the position feedback information can indicate that the capsule has entered the stomach or is within a certain distance from the lumen or wall, causing a corresponding illumination adjustment. These embodiments dynamically adjust the illumination pattern in real-time to provide immediate (or near-immediate) improved imaging of different anatomical regions of the GI channel.

[0035] Adaptation can occur during real-time recording by adjusting the LEDs (e.g., adjusting the activation mode, duration, and / or drive current of the LEDs) and / or during post-processing by adjusting the archived images (e.g., combining different ratios of color images). For example, adjusting the LEDs can involve activating red LEDs with a pulse length N times that of the blue LEDs (e.g., adjusting the LED illumination time). An equivalent effect can be performed with the archived images by combining N times more red images than blue images (e.g., adjusting the weights of the combination of the various color images).

[0036] A combination of different LED color groups can simulate a white light spectrum to achieve high color quality in-vivo imaging. A white light illumination spectrum simulating sunlight under 1.5 atmospheres is called "AM1.5" (as shown in FIG. 3C). An embodiment of the present invention can use a combination of multiple colors from multiple color groups to simulate the illumination spectrum of AM1.5 (or other solar energy or synthetic spectrum, such as "white LED" or "fluorescent lamp" in FIG. 3C).

[0037] Each LED can generate one or more illumination spectra within a single color group. For example, as shown in FIG. 3A, the red spectrum may include color groups R1, R2, and / or R3. These color groups are represented by Gaussian intensity distributions with maxima at 632 nm, 611 nm, and 591 nm, respectively. The blue spectrum may include color groups B1, B2, and / or B3. These color groups are represented by Gaussian intensity distributions with maxima at 400 nm, 425 nm, and 460 nm, respectively. The green spectrum may include color groups G1 and / or G2. These color groups are represented by Gaussian intensity distributions with maxima at 515 nm and 577 nm, respectively. For example, to simulate AM1.5 white light, some embodiments of the present invention can activate color groups (e.g., R1, G1, B1; R1, R2, G1, B1; or R1, R2, R3, G1, G2, B1, B2, B3).

[0038] Some embodiments of the present invention may omit or reduce the activation of color groups (e.g., activating or biasing only a subset of color groups) to simulate polychromatic (non-white) light illumination. To optimize illumination efficiency, the intensity of color groups that are poorly absorbed for some anatomical regions may be omitted and / or reduced. An example in which omitting color groups may prove useful is narrow band imaging (NBI). NBI is a type of endoscopic diagnostic program that omits or reduces red light and uses disproportionately green and blue LEDs to illuminate areas of shallow vessel density. NBI takes advantage of the fact that hemoglobin has poor absorption of red light and has favorable absorption peaks for green and blue light (e.g., as shown in FIG. 6D). In various embodiments of the present invention, these polychromatic light illumination spectrums may be produced by activating any combination of multiple color groups (e.g., higher R1 illumination relative to G1 and B1; R1, R2, R3, G1, B1; G1, G2, B1; R1, G1, B1, B2, B3; R1, R2, R1, G2, B1, B2; G1, G2, B1, B2; B1, B2, B3; G1, G2; R1, R2, R3).

[0039] Embodiments of the invention can illuminate in non-visible wavelength ranges, including infrared (IR), ultraviolet (UV), radio, and / or other non-visible wavelength ranges and combinations of visible and non-visible wavelengths. For IR spectra, or spectra including combinations of IR and other color groups, longer wavelengths typically penetrate deeper into biological tissue than shorter wavelengths. This difference between the penetration depths associated with various wavelengths of light provides a depth profile of the imaging tissue. This depth profile is used to simulate a three-dimensional image of the biological tissue. In embodiments of the invention, the IR illumination spectrum may be generated and optimized with any combination of multiple IR color groups (e.g., Near-IR1, Mid-IR1, Far-IR1, Near-IR, Near-IR2, Near-IR3), any combination of multiple IR color groups, and / or any other color groups from the same and / or different corresponding color groups (e.g., Near-IR1, Near-IR2, Radio, R1, R2; Near-IR1, G1, Mid-IR1, B3, Far-IR1, Far-IR2).

[0040] An embodiment of the present invention can match an output illumination spectrum emitted by one or more color group LEDs with an optimal input imaging spectrum of one or more corresponding color filters of an image sensor to achieve a maximum or higher than threshold quantum efficiency (QE). The image sensor (e.g., 110 shown in FIG. 1A and / or FIG. 5A) can be a photoelectric sensor such as a complementary metal oxide semiconductor (CMOS) image sensor that includes a photogrid of pixels (e.g., millions). To generate a color image, a color filter array such as a Bayer filter can be implemented in front of the image sensor. The color filter array includes a plurality of color filters (e.g., one color filter for each pixel). Each of the plurality of color filters allows light of a particular wavelength range within an allowed spectrum (e.g., red, green, or blue) to transmit to each individual pixel (e.g., above a threshold QE) and blocks or reduces light of the remaining wavelengths outside of the spectrum from reaching the individual pixel (e.g., passing light with a value lower than the threshold QE as shown in the QE diagram of FIG. 5B). The image sensor can also be paired with a micro-charge amplifier (e.g., one micro-charge amplifier per pixel). When a photon strikes a pixel, the charge amplifier converts the corresponding photoelectric current into an output voltage. Matching the illumination spectrum of the LEDs with the sensor input imaging spectrum reduces wasted photons outside the imaging spectrum that the sensor tolerates, which would otherwise go undetected (or be detected with a corresponding QE below a threshold). Compared to the same intensity of light illumination used outside these optimal wavelength ranges, these optimal wavelength illuminations increase the contribution of the LED light exposed to the sensor pixel to the photoelectric current, called the "quantum efficiency" (QE). An embodiment of the present invention can activate any combination of LEDs in a color group to achieve an illumination spectrum of LEDs that matches or falls within the sensor input imaging spectrum, thereby increasing the sensor exposure and improving the quantum efficiency of the system.

[0041] Referring to FIG. 1A, FIG. 1A is an exploded schematic diagram of an intracorporeal capsule endoscope 100 and its components for illuminating the gastrointestinal tract with a dynamically adjustable color illumination spectrum according to an embodiment of the present invention. The intracorporeal capsule endoscope 100 may be housed inside a capsule-like body 104. The capsule-like body 104 may have two concave shells or hemispheres at opposite ends of its longitudinal axis. The capsule-like body 100 may have a single-sided or double-sided imaging system such that the capsule-like body 104 may have at least one transparent optical end or housing 108 at one and / or both ends of its longitudinal axis, respectively. The capsule-like body 104 may encapsulate electronic components therein. Each transparent shell 108 may house one or more lenses 112 and an LED array 102. Image sensor(s) 110 configured to record intracorporeal images may be housed inside the capsule-like body 104 behind the at least one transparent optical shell 108. The LED array 102 may include a number of LEDS 130 mounted on a circuit board. Exemplary circuit diagrams of the LED circuit board and the LEDS 130 are shown in FIG. 7A(i) and FIG. 7A(ii), respectively. The lens 112 may include one or more lenses for focusing light emitted by the LEDS 130 onto the image sensor 110. The image sensor 110 may include one or more color filter arrays (e.g., as shown in FIG. 5A). According to an embodiment of the invention, the image sensor 110 and / or the image processing board 128 may include one or more processing circuit boards for performing operations (e.g., processing, storing, and / or transmitting image data). According to an embodiment of the invention, the image sensor 110 and / or the image processing board 128 may include one or more memories for storing data (e.g., a particular color, a multi-color, or a white light image).

[0042] The LED array 102 may include multiple different color groups of LEDS 130. The multiple different color groups are configured to emit light to be recorded and / or stored by an external device processor operating in the image sensor 110, the image processing board 128, and / or an external memory. The multiple different color groups of LEDS 130 may correspond to multiple corresponding wavelength sub-ranges such that the LEDS 130 in each color group emit light that falls in the illumination spectrum within the corresponding wavelength sub-range of that color group. The LED array 102 may include or be operably connected to a driving circuit. The driving circuit is configured to adjust the combination, sequence, and / or duration of the time pulses and / or the power and driving current of the LEDS 130. Furthermore, the driving circuit may be coupled to an illumination processor (e.g., the image sensor 110 and / or the image processing board 128) to control the operation of said illumination processor. The drive circuitry is arranged to deliver drive currents during different (e.g., non-overlapping) time pulses to independently activate each of the different color groups of LEDS 130 such that a single color group of LEDS is independently activated at any time. The different color groups of LEDS 130 can be sequentially activated in successive time pulses to cumulatively simulate a white light or polychromatic illumination spectrum in the multiple time pulses. The image sensor 110 is configured to sequentially record a set of images of the multiple different colors. These sets of images depict the scattering of light emitted from the corresponding multiple different color groups of LEDS 130 during the non-overlapping time pulses. In some embodiments, the illumination processor can activate a color group of LEDS that approximately matches an input optimal imaging spectrum of the color filter of the image sensor 110 to increase or maximize the quantum efficiency of the capsule.

[0043] In various embodiments, the capsule endoscope 100 can pass through the GI channel magnetically (e.g., by an external magnetic control system as shown in FIG. 1B) or by peristaltic propulsion (non-magnetic). In an embodiment using magnetic propulsion, the capsule-like body 104 can also house one or more permanent magnets 124 having permanent magnetic dipoles (e.g., north-south). The permanent magnets 124 allow the capsule endoscope 100 to be magnetically guided when exposed to a magnetic field generated by an external magnetic control system, as shown in FIG. 1B. The capsule-like body 104 can also house a wireless communication system 122. The wireless communication system 122 includes a wireless (e.g., radio frequency (RF)) processing board and an antenna for wirelessly transmitting information to and receiving information from a remote device or controller. The capsule-like body 104 can also house one or more batteries or power sources 132 to power components of the endoscope 100.

[0044] The wireless communication system 122 can transmit intrabody information, such as intrabody image data captured by the detection device 110, location information (e.g., anatomical location, geographic location), magnetic field information for interacting with and being controlled by the external magnet control system, and / or other sensory feedback such as intrabody conditions (e.g., temperature, pressure, pH, etc.). The wireless communication system 122 can receive command or control information, such as lighting activation mode or instruction and / or parameters, from a remote device or external controller. The wireless communication system 122 can transmit image and / or image quality or location information to and / or receive image and / or image quality or location information from a remote device or external controller. This allows for determining and realizing optimized parameters of the driving circuit of the LED array 102 through a feedback circuit. This feedback loop can trigger real-time adjustment of LED activation based on environmental conditions. The feedback loop can be performed locally in the capsule 100 (e.g., in the image processing board 128) and / or externally in a remote device or controller. In case the feedback loop is implemented locally in the image processing board 128, the image and / or image quality information may be received directly by the image processing board 128, where the optimal parameters of the driving circuit of the LED array 102 are determined locally.

[0045] 1B, which shows a schematic diagram of an external magnetic control system 126 for moving an intracorporeal capsule endoscope 100 in a GI channel according to an embodiment of the present invention. The external magnetic control system 126 can generate a magnetic field for guiding the capsule endoscope 100 through one or more permanent magnets 124 included in the capsule endoscope 100. The external magnetic control system 126 includes a fixing means adapted to position one or more external permanent magnets 134 horizontally and vertically using a vertically and horizontally adjustable mechanism and an adjustable base. The external magnetic control system 126 has degrees of freedom to move along two axes such that the capsule endoscope 100 moves in three dimensions. Mechanical and operational details of the external magnetic control system 126 are described, for example, in U.S. Patent Application Publication No. 2015 / 0380140, the entire contents of which are incorporated herein by reference.

[0046] 2A, which illustrates a schematic diagram of multiple color groups of LEDs (130R, 130B, 130G) and their respective illumination spectrums for generating white light illumination, according to some embodiments of the present invention. In some examples, any combination of color groups of LEDs and their corresponding temporal pulse widths and / or drive currents may be configured to generate the aforementioned white light illumination spectrum. In some examples, any combination of color groups of LEDs and their corresponding temporal pulse widths and / or drive currents may be configured to generate a modified and / or optimized version of the aforementioned white light illumination spectrum.

[0047] 2B, which illustrates a schematic of LEDs in a single color group (e.g., blue) separated into multiple color groups (e.g., blue and yellow) by filters (e.g., phosphor filters) according to some embodiments of the present invention. The combination of light from the multiple separated color groups forms white light illumination. In some implementations, filters may be applied to the LED array 102 to convert a single color input spectrum and used to generate white light illumination.

[0048] 3A, which graphically illustrates the intensity of multiple wavelength ranges corresponding to multiple different color groups of LEDs, according to some embodiments of the present invention. In some examples, any combination of these color groups of LEDs and their corresponding temporal pulse widths and / or drive currents may be configured to generate and optimize white light illumination or polychromatic (non-white) light illumination.

[0049] Referring to Figure 3B, Figure 3B graphically displays illumination spectra corresponding to the intensities of three exemplary wavelength ranges of three different corresponding color groups from the LEDs of Figure 3A, in accordance with some embodiments of the present invention. In some embodiments, any combination of these color groups of LEDs and their corresponding temporal pulse widths and / or drive currents may be configured to generate and optimize white light illumination or polychromatic (non-white) light illumination.

[0050] 3C, according to some embodiments of the present invention, Fig. 3C is a graphical representation of the illumination spectrum of wavelength band intensity of sunlight 302 (e.g., AM1.5), an exemplary LED white light 304, and an exemplary fluorescent lamp 306. In some aspects, any combination of color groups of LEDs and their corresponding temporal pulse widths and / or drive currents may be configured to be used to generate and optimize white light illumination such that its illumination spectrum simulates the illumination spectrum of sunlight 302 (including AM1.5) or fluorescent light.

[0051] FIG. 4A is a schematic diagram illustrating a close-up view of an LED array 102 according to some embodiments of the present invention. In some aspects, the front side of the LED array 102 is implemented with a group of LEDs 130. Each group of LEDs 130 may include one or more different color LED groups (e.g., 130R, 130G, and / or 130B). In some aspects, the different color groups of LEDs 130 may be arranged in one or more clusters, where each cluster includes LEDs from the different color groups (e.g., 130R, 130B, and 130G). In some embodiments, the back side of the LED array 102 may be implemented with a plurality of dedicated drive circuits. According to some embodiments of the present invention, each drive circuit may include a plurality of dedicated control circuits (e.g., as shown in FIG. 7C). Such a plurality of dedicated control circuits are configured to send drive currents to independently activate the different corresponding color LED groups by generating current pulses of various time pulse widths and / or various currents.

[0052] 4B-4D graphically illustrate time pulses of several modes during which different color groups of LEDs are independently activated according to some embodiments of the present invention, where Te (or a multiple thereof) represents the exposure time of an image captured by an imaging sensor, and TB, TG, TR, and TIR represent illumination time pulses during which the blue, green, red, and infrared color groups of LEDs are activated, respectively. In some embodiments, any combination of color groups of LEDs and their corresponding time pulse widths and / or drive currents may be configured to generate and optimize white light illumination or polychromatic (non-white) light illumination. In FIG. 4B-4D, the intensity (height of the illumination block) of the various color groups is constant, but both the time pulse (width) and intensity (height) of each color group of LEDs may be independently activated differently.

[0053] FIG. 5A illustrates, in accordance with some embodiments of the present invention, an image sensor 110 having an exemplary color pattern created by a color filter array, such as a Bayer filter. The color filter array includes multiple color filters (e.g., one for each pixel) that allow each pixel to absorb light in a single wavelength range associated with a single color group. For example, the Bayer filter allows wavelength sub-ranges associated with the three different corresponding color groups (e.g., red, green, blue) shown in FIG. 3B. Laterally adjacent pixels (e.g., horizontally and / or vertically, but not diagonally) absorb light in different wavelength sub-ranges associated with different corresponding color groups. According to some embodiments of the present invention, the image sensor 110 may be configured such that incident light is split by filters to multiple color panels 110R(1), 110G(1), and 110B(1). Each color panel includes a different single color group. Light passing through each color group panel 110R(1), 110G(1), or 110B(1) forms a single color group image 110R(2), 110G(2), and 110B(2), respectively. In some embodiments of the invention, the output illumination spectrum emitted by each color group of LEDs can match or substantially overlap the optimal input spectrum (e.g., having high quantum efficiency) of the corresponding color group of the color filter array of image sensor 110.

[0054] FIG. 5B graphically illustrates the quantum efficiency (QE) of an image sensor (e.g., 110 in FIG. 5A) over a range of illumination wavelengths, according to some embodiments of the present invention. Curves 502, 504, 506 illustrate the quantum efficiency of the image sensor exposed to color filter arrays of three color groups, red, green, and blue, respectively. Curve 508 illustrates the quantum efficiency of the image sensor exposed to light without an overlying color filter array. The absence of a color filter array causes the image sensor to capture a monochrome or black and white image. An embodiment of the present invention can match the output illumination spectrum emitted by the LEDs of each color group to the optimal input imaging spectrum of the corresponding color filter array of the image sensor associated with a maximum QE or a quantum efficiency (QE) above a threshold. In some embodiments of the present invention, the time pulse and current intensity of the LEDs can be adjusted or tuned to variables to optimize the QE (e.g., to achieve a maximum QE or a QE above a threshold).

[0055] FIG. 6A illustrates a schematic of a system for illuminating biological tissue to generate white light and / or polychromatic (non-white light) images according to some embodiments of the present invention. The biological tissue may be sequentially illuminated by LEDs (e.g., 130R, 130G, 130B) of different corresponding color groups and may scatter the corresponding color groups of the LED illumination. The color groups of the LED illumination may then be absorbed by an image sensor (e.g., 110 in FIG. 1A) to form images 606-618 specific to the corresponding color groups. The color group specific images 606-618 may be combined to generate a white light image 602 and / or a polychromatic (non-white) image 604. According to embodiments of the present invention, the polychromatic (non-white) image 604 may be generated in a more effective manner than Flexible Spectral Imaging Color Enhancement (FICE). FICE is an endoscopic imaging technique that illuminates biological tissue with white light. The white light is scattered by the biological tissue and recorded by an image sensor (e.g., a color CCD) to form a white light image. An adaptive filtering engine (i.e., a FICE engine) separates a white light image matching specific wavelengths into a multi-color (non-white) image, which is a complex and time-consuming process. An embodiment of the present invention can generate a multi-color (non-white) image 604 that is similar or equivalent to a FICE image by using color-specific illumination rather than white light illumination. This avoids the need to separate color-specific images using an adaptive filtering engine, and allows an embodiment of the present invention to generate a multi-color (non-white) image 604 faster and more efficiently than a FICE engine, eliminating the need for the time-consuming adaptive filtering color separation used in the FICE engine.

[0056] Please refer to Figure 6B, which graphically illustrates three exemplary wavelength ranges corresponding to the color groups of three different LEDs each combined to generate the white light image of Figure 6A, in accordance with an embodiment of the present invention. In some embodiments, any combination of color groups of LEDs and their corresponding temporal pulse widths and / or drive currents may be configured to generate, optimize and / or modify a white light illumination spectrum.

[0057] Please refer to FIG. 6C, which graphically displays two exemplary wavelength range intensity illumination spectra for two different corresponding color groups corresponding to LEDs, in accordance with an embodiment of the present invention, which are combined to generate the polychromatic (non-white) light image in FIG. 6A. In some embodiments, any combination of color groups of LEDs and their corresponding temporal pulse widths and / or drive currents may be configured to generate, optimize and / or modify a polychromatic (non-white) illumination spectrum.

[0058] Please refer to FIG. 6D. FIG. 6D (ii) shows a schematic diagram of biological tissue through which incident light in different subranges of wavelengths of different color groups corresponding to LEDs penetrates at different depths, and (i) shows a schematic diagram of corresponding polychromatic (non-white) light absorption spectrum and corresponding wavelength subranges of different color groups corresponding to LEDs according to some embodiments of the present invention. In the example of FIG. 6D, a polychromatic (non-white) illumination spectrum of blue and green color groups induces a corresponding polychromatic (non-white) light absorption spectrum of blue and green color groups through an image sensor. Curve 602 in FIG. 6D(i) shows that hemoglobin has absorption bands mainly at about 415 nm (corresponding to blue LED light) and 540 nm (corresponding to green LED light). An embodiment of the present invention can take advantage of the color preference of hemoglobin by independently activating a preferred color group of LEDs and emitting a polychromatic (non-white) illumination spectrum including blue and green LED light. The illumination spectrum can be customized to the intensity and wavelength of the hemoglobin curve (e.g., 602) to better image blood-rich anatomical regions, such as the dense network of superficial vessels near the bronchial tree and colorectal region.

[0059] FIG. 6D(ii) shows that green LED light penetrates biological tissue to a greater depth than blue LED light. Typically, LED light with a relatively long wavelength (shorter frequency) penetrates tissue to a greater depth, and LED light with a relatively short wavelength (higher frequency) penetrates tissue to a lesser depth. For example, IR LED light with a relatively long wavelength may be used to penetrate biological tissue to a greater depth than visible light. The difference between the different penetration depths can be used to generate a topological diagram or depth profile of the biological tissue being imaged. Such a topological diagram or depth profile of the biological tissue being imaged can be used to simulate a three-dimensional model of the biological tissue and construct a three-dimensional image of the anatomical structure.

[0060] Please refer to FIG. 7A. FIG. 7A shows, in accordance with some embodiments of the present invention, (i) a top view of an exemplary circuit board for connecting LEDs, (ii) an exemplary circuit diagram of LEDs 130 connected to the exemplary circuit board, and (iii), (iv) a side view of the exemplary circuit board. As shown in FIG. 7A(ii), the cathode pins of LEDs 130R, 130G, 130B can be connected to a common pin (connection "4" in FIG. 7A(ii)) to which a battery and / or power source 132 provides a negative signal. Furthermore, according to some embodiments of the present invention, the anode pins of LEDs 130R, 130G, 130B (connections 3, 2, 1, respectively in FIG. 7A(ii)) can be provided with a positive signal by the battery and / or power source 132. In some embodiments, as shown in FIG. 7A(ii), the LED cluster 130 may be mounted on the front side of the circuit board shown in FIG. 7A(i).

[0061] Please refer to Fig. 7B, which shows a schematic diagram of a structure and package capable of housing an LED circuit board according to an embodiment of the present invention.

[0062] Please refer to FIG. 7C. FIG. 7C shows a schematic diagram of a driver circuit 700 including multiple dedicated control circuits (e.g., 700R, 700G, 700B) according to some embodiments of the present invention. Each dedicated control circuit independently activates a different color group of LEDs (e.g., 130R, 130G, 130B, respectively) by generating current pulses of various time pulse widths and / or various currents. The driver circuit 700 may be configured to adjust the combination, sequence, and / or duration of the pulses and / or drive currents of the LEDs (e.g., 130R, 130G, 130B) of each color group. The driver circuit 700 may be mounted on the back side of a substrate holding the LED array (e.g., 102 in FIG. 1) and / or may be coupled to a lighting processor (e.g., mounted on the image processing board 128 in FIG. 1) whose operation is controlled.

[0063] Please refer to Figure 8, which is a table of several different color groups of LEDs and their corresponding peak wavelengths, half widths, and beam angles listed according to some embodiments of the present invention. These values ​​are used as examples only, and any other values ​​or combinations of these or other color groups and their corresponding peak wavelengths, half widths, and beam angles can be used.

[0064] Please refer to FIG. 9. FIG. 9 is a flow chart of a method of operating an intracorporeal capsule endoscope to emit a dynamically adjustable color illumination spectrum according to some embodiments of the present invention. The operations of FIG. 9 may be performed using one or more of the capsule endoscope devices or components disclosed in FIGS. 1-8. For example, one or more of the operations of FIG. 9 are performed by one or more processors of an image sensor (e.g., 110 in FIG. 1 and / or FIG. 5A), an image processing board (e.g., 128 in FIG. 1A), and / or an external processor operating an external memory using one or more memories.

[0065] In operation 900, one or more processors (e.g., image sensor 110 of FIGS. 1A and 5A, image processing board 128 of FIG. 1A, and / or an external processor) can activate the LEDs of a first color group (e.g., 130R, 130B, or 130G of FIGS. 2A, 2B, 4A, and 6A) during a first time pulse T1. The one or more processors can control one or more control circuits (e.g., 700R, 700G, 700B of FIG. 7C). For example, each different control circuit is dedicated to independently activating (e.g., by a drive current) a different one of 1-N different color LED groups. The LEDs of the first color group can illuminate biological tissue with light in a unique wavelength range that is different (e.g., does not overlap partially or completely) with the wavelength ranges of the other color groups.

[0066] In operation 902, one or more processors can activate LEDs of a second color group (e.g., 130R, 130B, or 130G of FIGS. 2A, 2B, 4A, and 6A) during a second time pulse T2 that is different from the other time pulses (e.g., T1, T3, . . . ). The LEDs of the second color group can illuminate the biological tissue with light within a wavelength sub-range that is unique to the other color groups. After the second color group is activated, the process or processor can sequentially activate each ith color group of LEDs during an ith time pulse Ti, where i=3, N-1, when i=N, and continue with operation 904.

[0067] In operation 904, the one or more processors can activate an Nth LEDs color group (e.g., 130R, 130B, or 130G in FIGS. 2A, 2B, 4A, and 6A) during an Nth time pulse T.sub.N. The one or more processors can sequentially activate LEDs in 1-N different color groups during non-overlapping time pulses T.sub.1-T.sub.N. The LEDs in 1-N different color groups can illuminate the biological tissue with light within a plurality of different wavelength sub-ranges such that the LEDs in each color group emit light within a unique illumination light spectrum sub-range for that color group. This unique illumination spectrum sub-range is distinct from the spectrum sub-ranges of all other color groups. After one complete cycle of activating the entirety of the 1-N LED color groups in the T.sub.1-T.sub.N time pulses, the process or processor returns to operation 900 and repeats the illumination pattern or adjusts the illumination pattern, for example, as follows:

[0068] In some embodiments, the one or more processors can dynamically adjust the activation of the LED color groups, such as by dynamically adjusting the activation duration T1 of the T1-TN and / or the intensity of the drive circuit current used to activate one or more of the LEDs of the 1-N different color groups, with fully or partially non-overlapping time pulses, to adjust the proportion of the images of the 1-N different color groups that are activated and combined. By adjusting the illumination proportions of the various color group combinations, the total cumulative color illumination spectrum (e.g., of the white light or non-white multi-color image combined in operation 910) is dynamically adjusted. In some embodiments, the one or more processors can dynamically adjust the activation of the LED color groups in response to a feedback loop (e.g., the duration Ti of the T1-TN is adjusted with fully or partially non-overlapping time pulses and / or the intensity of the drive current). The feedback loop can be based on images collected by the capsule endoscope in real time, position information, etc. For example, a feedback loop indicating that the capsule is located in the stomach or capturing an image having a particular absorption spectrum can cause one or more processors to adjust activation of LED color groups to be optimized for stomach illumination or optimal light for the particular absorption spectrum.

[0069] In some embodiments, the one or more processors can illuminate the biological tissue at different penetration depths with light emitted from LEDs of different corresponding 1-N color groups (e.g., as shown in FIG. 6D ). The one or more processors can then generate a depth profile of the biological tissue based on the differences in these penetration depths. The one or more processors can then simulate a three-dimensional image of the biological tissue based on the depth distribution.

[0070] In operation 906, one or more processors (e.g., in the image sensor 110 of FIG. 1A and FIG. 5A, in the image processing board 128 of FIG. 1A and / or in an external processor) can sequentially record 1-N different color images depicting the scattering of light emitted from corresponding 1-N color LEDs groups during non-overlapping or partially overlapping T1-TN time pulses. In some embodiments, the one or more processors can select an activation mode of the LED color groups such that the illumination spectrum of the activated color group combination (e.g., combined to form a white light or non-white multicolor image) matches the absorption spectrum of the image sensor. In some embodiments, the one or more processors can match the illumination spectrum emitted by the LEDs of one or more color groups with the optimal absorption spectrum of the image sensor, e.g., associated with a maximum or higher than threshold quantum efficiency (QE).

[0071] In some embodiments, the one or more processors can record color group specific images such that each image renders an individual color group and is recorded during activation of a single color group of LEDs rather than recording the LEDs of other color groups. Additionally or alternatively, the one or more processors can record mixed color group images such that each image renders multiple color groups and is recorded in multiple time pulses when multiple color LED groups are activated. In some embodiments, if the image sensor records mixed color group images, the process or processor can skip or reduce instances of operation 910 for synthesizing color specific images.

[0072] In operation 908, one or more processors (e.g., in the image sensor 110 of FIGS. 1A and 5A, the image processing board 128 of FIG. 1A, and / or an external processor) may store 1 to N different color groups of the image recorded in operation 906.

[0073] In operation 910, one or more processors (e.g., image sensor 110 of FIGS. 1A and 5A, image processing board 128 of FIG. 1A, and / or an external processor) can combine color-group-specific images from two or more different color groups to form a white-light or non-white multi-color image. In some embodiments, such one or more processors can combine all and / or a subset of 1-N different color images to form a white-light and / or multi-color (non-white) light image.

[0074] Other or additional operations may be performed in the same order or in a different order.

[0075] In some embodiments, color can refer not only to electromagnetic (EM) waves in the visible spectrum, but also to EM waves in the non-visible spectrum, such as EM waves in the infrared (IR) or ultraviolet (UV) color groups.

[0076] Embodiments of the invention include non-volatile computer or processor readable media or articles such as computer or processor non-volatile storage media (memory, disk drives, or USB flash memory) encoding or storing instructions, such as computer executable instructions, that are performed when executed by a processor or controller (e.g., image sensor 110 of FIGS. 1A and 5A, image processing board 128 of FIG. 1A, and / or an external processor).

[0077] In the above description, the embodiments are examples or implementations of the present invention. The various occurrences of "an embodiment," "an embodiment," or "some embodiments" do not necessarily refer to the same embodiment. Although various features of the present invention may be described in the context of a single embodiment, the features of the embodiments may also be provided alone or in any suitable combination. In contrast, for clarity, the present invention may be described in the context of a single embodiment herein, but the present invention may also be practiced in a single embodiment. References to "some embodiments," "an embodiment," "one embodiment," or "other embodiments" referred to herein mean that a particular feature, structure, or characteristic described in connection with an embodiment is included in at least some embodiments of the present invention, but not necessarily in all embodiments. Furthermore, it should be recognized that the above-mentioned aspects of the present invention may coexist in combination or in other manners in an embodiment of the present invention.

[0078] The descriptions, instances, methods, and materials set forth in the claims and the specification are not to be construed as limiting, but are merely illustrative. Although certain features of the invention have been described and illustrated herein, many modifications, substitutions, changes, and equivalents may occur to those skilled in the art. It is therefore to be understood that the appended claims are intended to cover all modifications and changes that fall within the true spirit of the invention.

[0079] Although the present invention has been described with reference to a limited number of embodiments, these embodiments should not be interpreted as limitations on the scope of the present invention, but as examples of some preferred embodiments. Other possible changes, modifications, and applications are also within the scope of the present invention. Different embodiments are disclosed herein. Features of some embodiments may be combined with features of other embodiments. Thus, some embodiments may be combinations of features of multiple embodiments. Thus, the scope of the present invention should not be limited by what has been described above, but should be limited by the appended claims and their legal equivalents.

Claims

1. A capsule endoscope capable of dynamically adjusting a color illumination spectrum, comprising: an encapsulated body having at least one transparent optical end and having electronic components encapsulated therein; an image sensor housed within the sac behind the at least one transparent optical end and positioned to record images of the interior of the living body; a plurality of light emitting diodes (LEDs) of different color groups corresponding to a plurality of different wavelength sub-ranges, configured to emit light to be recorded by the image sensor; a drive circuit configured to deliver drive currents during fully or partially non-overlapping time pulses to independently activate each of the LEDs of the plurality of different color groups, independently activating LEDs of a single color group of the LEDs of the plurality of different color groups at any one time, and sequentially activating the LEDs of the plurality of different color groups in successive said time pulses to simulate a white light or polychromatic lighting spectrum in a plurality of said time pulses; A capsule endoscope capable of dynamically adjusting a color illumination spectrum, wherein the LEDs of the plurality of different color groups each emit light that is absorbed by biological tissue inside the living body at a different penetration depth, whereby differences between the penetration depths generate a depth profile of the illuminated biological tissue, and the capsule endoscope includes one or more processors for simulating a three-dimensional image of the biological tissue based on the depth profile.

2. 2. The capsule endoscope according to claim 1, wherein the driving circuit dynamically adjusts durations of the fully or partially non-overlapping time pulses or intensities of the driving currents for activating one or more of the LEDs of the plurality of different color groups.

3. 3. The capsule endoscope according to claim 2, wherein the drive circuit dynamically adjusts the duration of the fully or partially non-overlapping time pulses or the intensity of the drive current by a feedback loop based on image or position information collected in real time by the capsule endoscope.

4. 2. The capsule endoscope according to claim 1, wherein the drive circuit is connected to a plurality of dedicated control circuits, each of the plurality of dedicated control circuits independently activating LEDs of a single color group among the LEDs of the plurality of different color groups.

5. 2. The capsule endoscope according to claim 1, wherein the image sensor is configured to record an image generated by activated LEDs of a single color group among the LEDs of the plurality of different color groups when the LEDs of the single color group are activated and the LEDs of other color groups are not activated.

6. 2. The capsule endoscope according to claim 1, further comprising one or more processors that combine two or more images from an image group including images produced by each of the LEDs of the plurality of different color groups to form a white light or non-white multi-color image.

7. 2. The capsule endoscope according to claim 1, wherein the image sensor is configured to record a white light or non-white multi-color image, the white light or non-white multi-color image depicting scattering of light emitted from two or more of the LEDs of the plurality of different color groups during two or more corresponding time pulses.

8. 2. The capsule endoscope according to claim 1, wherein the capsule endoscope includes one or more processors, the one or more processors configured to activate one or more of the LEDs of the plurality of different color groups, the LEDs of the plurality of different color groups emitting light in a wavelength range matching an absorption spectrum of the image sensor.

9. The capsule endoscope according to claim 1 , wherein the LEDs of the plurality of different color groups are arranged in one or more clusters, each of the clusters being composed of two or more LEDs from the LEDs of the plurality of different color groups.

10. A method of operating a capsule endoscope to emit a dynamically adjustable color illumination spectrum, comprising: sequentially activating light emitting diodes (LEDs) of a plurality of different color groups during fully or partially non-overlapping time pulses to cause the LEDs of the plurality of different color groups to emit light in a plurality of different corresponding wavelength sub-ranges into biological tissue within the living organism; sequentially recording, by an image sensor, images of the different color groups depicting scattering of light emitted from the LEDs of the different color groups corresponding to the non-overlapping time pulses; storing a set of images including images produced by each of the LEDs of the plurality of different color groups; combining two or more of the images to form a white light or non-white multi-color image; illuminating the biological tissue with light emitted from the LEDs of the plurality of different color groups at different penetration depths; generating a depth profile of the biological tissue based on differences between penetration depths of light emitted from the LEDs of the plurality of different color groups; and simulating a three-dimensional image of the biological tissue based on the depth profile.

11. The method of claim 10, further comprising: adjusting a ratio of combining images generated by each of the plurality of LEDs of different color groups; and dynamically adjusting durations of the fully or partially non-overlapping time pulses or intensity of drive currents for activating one or more of the plurality of LEDs of different color groups to adjust an illumination spectrum of the white light or non-white multi-color image.

12. A method for operating a capsule endoscope as described in claim 11, comprising dynamically adjusting the duration of the non-overlapping time pulses or the intensity of the drive current based on image or position information acquired in real time by the capsule endoscope in response to a feedback circuit.

13. A method for operating a capsule endoscope as described in claim 10, comprising independently activating each of the plurality of different color groups of LEDs via dedicated control circuits for each of the different color groups of LEDs.

14. A method for operating a capsule endoscope as described in claim 10, comprising, when LEDs of a single color group among the plurality of different color groups are activated and LEDs of other color groups are not activated, recording an image generated by the activated LEDs of the single color group.

15. A method for operating a capsule endoscope as described in claim 10, wherein when the LEDs of the plurality of different color groups are activated, the white light or non-white multicolor image is synthesized from a combination of two or more of the images produced by each of the LEDs of the plurality of different color groups recorded during the plurality of fully or partially non-overlapping time pulses.

16. A method for operating a capsule endoscope as described in claim 10, comprising matching an illumination spectrum of the white light or non-white multi-color image with an absorption spectrum of the image sensor.

17. A method of operating a capsule endoscope for emitting a dynamically adjustable color illumination spectrum, comprising: sequentially activating light emitting diodes (LEDs) of a plurality of different color groups during fully or partially non-overlapping time pulses to cause the LEDs of the plurality of different color groups to emit light in a plurality of different corresponding wavelength sub-ranges into biological tissue within the living organism; recording on an image sensor a white light or non-white multi-color image depicting scattering of light cumulatively emitted from two or more of the LEDs of the plurality of different color groups during two or more associated combinations of the non-overlapping time pulses; and storing said white light or non-white multi-color image; generating a depth profile of the biological tissue based on differences between penetration depths of light emitted from the LEDs of the plurality of different color groups; and simulating a three-dimensional image of the biological tissue based on the depth profile.

18. A method for operating a capsule endoscope as described in claim 17, comprising adjusting an illumination spectrum of the white light or non-white multi-color image by dynamically adjusting a duration of the non-overlapping time pulses or an intensity of a drive current for activating one or more of the LEDs of the plurality of different color groups.

Images