Portable hyperspectral system

Abstract

To provide a hyperspectral imaging system capable of providing an inexpensive and efficient method.SOLUTION: A capsule hyperspectral system comprising an imaging capsule having an illumination system having a plurality of light emitters configured to emit a plurality of different illumination lights from the imaging capsule, and a hyperspectral imaging system having at least one imaging sensor, wherein the illumination system and the hyperspectral imaging system are configured to illuminate the subject with a sequence of different illumination lights and to cooperatively image the subject between each of the different illumination lights in the sequence, and a hyperspectral processing system having at least one processor, wherein the hyperspectral processing system is operably coupled with the hyperspectral imaging system; Receive images of the object therefrom and generate a multispectral reflectance datacube of the object from the received images of the object.SELECTED DRAWING: Figure 1A

Description

[Technical Field]

[0001] The present disclosure relates to a portable hyperspectral system. The present disclosure also relates to a capsule hyperspectral system. The present disclosure also relates to a capsule hyperspectral system and a hyperspectral imaging system having a tethered imaging capsule. [Background technology]

[0002] Traditionally, the inspection, characterization, and classification of objects, samples, or features have been performed in a variety of fields. Often, this operation can be performed by imaging the subject using a color camera with red, green, and blue (RGB) channels in the spectral range visible to the human eye. Several applications may require distinguishing between features whose RGB values ​​are too similar. Hyperspectral imaging systems extend imaging beyond the RGB limit to several spectral channels. This added dimension expands the ability to inspect, characterize, and classify objects.

[0003] Typical hyperspectral imaging systems typically require expensive imaging devices (e.g., imaging devices often costing over $40,000 as of 2020) with a large footprint (e.g., starting from one cubic inch) combined with long analysis (e.g., several minutes per image) and significant computing power. Such systems can have high power, long processing times, and large space requirements. On the other hand, low-power devices that are portable and enable real-time operation could expand the range of applications to numerous applications.

[0004] Some exemplary use cases for hyperspectral imaging systems include: Hyperspectral imaging systems can be used in performing agricultural inspections, which may assess the health, maturity, or quality of products in the field; Landscape mapping and survey devices using hyperspectral imaging systems may be equipped on unmanned aerial vehicles for real-time hyperspectral assessment; Hyperspectral imaging systems configured for environmental monitoring may be mounted on static assemblies for continuous low-energy analysis; To study public and health safety, hyperspectral imaging systems may be assembled on automated or manual screening devices; Forensics may use hyperspectral imaging systems to detect the authenticity of items such as banknotes; Robotics may use hyperspectral imaging systems to increase the accuracy of automated operations that require vision; Autonomous vehicles may use hyperspectral imaging systems to improve the detection of roads, streets, and objects that closely match in color; Head-up displays may use hyperspectral imaging to improve high-speed, high-sensitivity vision in low-light conditions; Medical diagnostics may use hyperspectral imaging systems to detect early stages of disease and improve patient outcomes.

[0005] The use of hyperspectral imaging systems for cancer detection has the potential to have a significant impact on healthcare. For example, East Asia, where the incidence rate of esophageal cancer (EC) is 17.9 in men and 6.8 in women (per 100,000 people), may be the largest market in which hyperspectral imaging systems could be utilized for cancer detection. Esophageal screening has already been tested in China, showing a 47% reduction in mortality risk between screened and unscreened patients. A less invasive, faster, and cheaper screening device could increase the likelihood of early detection, resulting in better patient outcomes and lower costs for providers.

[0006] As an example of esophageal disease, EC often remains asymptomatic until late in the disease course, highlighting the importance of efficient screening procedures. Endoscopy is often not recommended by physicians unless the individual is high-risk or symptomatic. The most common screening procedure used today is upper gastrointestinal (GI) endoscopy. This procedure involves inserting a thick, flexible tube containing a camera and illuminating it down the patient's throat. An FDA-approved endoscope costs approximately $40,000 (as of 2015), with an additional $25,000 for the image processor unit. The endoscope then allows physicians to examine the lining of the esophagus and assess whether further testing is necessary. Due to the large and invasive nature of the endoscope, deep sedation is required during the entire screening process, making this procedure inconvenient for screening. This is in part because deep sedation carries the risk of respiratory failure and requires close examinations before and after the procedure and observation by trained personnel. Therefore, this procedure is expensive in terms of equipment costs and the necessary sedation.

[0007] Further, exemplary low resolution hardware is disclosed in PCT application WO / 2015 / 157727, the contents of which are incorporated herein in their entirety.

[0008] In view of the above, it would be advantageous to have a hyperspectral imaging system of the present disclosure that may provide an inexpensive and efficient method for diagnosing esophageal and / or liver diseases as well as other applications. Summary of the Invention

[0009] In some embodiments, the present disclosure relates to an endoscopic system. In some aspects, the endoscopic system relates to a capsule hyperspectral system. In some aspects, the capsule hyperspectral system can include a tethered imaging capsule and a hyperspectral imaging system.

[0010] In some embodiments, a capsule hyperspectral system can include an imaging capsule having an illumination system with multiple light emitters configured to emit multiple different illumination lights from the imaging capsule; a hyperspectral imaging system with at least one imaging sensor, where the illumination system and the hyperspectral imaging system are configured to illuminate an object with a sequence of different illumination lights and cooperatively image the object during each of the different illumination lights in the sequence; and a hyperspectral processing system having at least one processor, where the hyperspectral processing system is operatively coupled to the hyperspectral imaging system and configured to receive images of the object therefrom and generate a multispectral reflectance data cube of the object from the received images of the object. In some aspects, a tether has a capsule end coupled to the imaging capsule and a system end coupled to the hyperspectral processing system. The tether can be communicatively connected to the hyperspectral imaging system and the hyperspectral processing system to pass data therebetween. In some aspects, the illumination system includes at least three LEDs having at least three different color bands, such as at least one LED being a white light LED and / or at least two LEDs being colored LEDs having different color bands. This can include a uniformly spaced array of multiple LEDs, such as at least six LEDs, including at least two white light LEDs and at least four colored LEDs with at least two different color bands.

[0011] In some embodiments, the emission wavelength of each LED is selected to visually distinguish and differentiate between white and / or pinkish surfaces on healthy tissue and red surfaces on non-healthy tissue.

[0012] In some embodiments, the at least one imaging sensor and the plurality of light emitters are disposed on a plate and are oriented in the same direction.

[0013] In some embodiments, the hyperspectral imaging system includes a lens system that is a fixed lens system, a detachable lens system, a replaceable lens system, or an interchangeable lens system. In some aspects, the lens system has at least one lens with a field of view (FOV) ranging from at least about 90 degrees to less than about 360 degrees, or from about 120 degrees to about 180 degrees. In some aspects, the hyperspectral imaging system includes an optical lens, an optical filter, a dispersive optic, or a combination thereof. In some aspects, the hyperspectral imaging system includes a first optical lens, a second optical lens, and a dichroic mirror / beam splitter. In some aspects, the hyperspectral imaging system includes an optical lens, a dispersive optic, and the at least one imaging sensor is a photodetector array.

[0014] In some embodiments, the at least one imaging sensor is positioned eccentrically relative to the central axis of the imaging capsule. In some aspects, the at least one imaging sensor is positioned about 10 degrees to about 35 degrees away from the central axis. In some aspects, the hyperspectral imaging system further comprises an optical filtering system positioned between the optical inlet of the capsule and the at least one imaging sensor. In some aspects, the optical filtering system includes a noise reduction filter, such as a median filter.

[0015] In some embodiments, the imaging capsule includes a capsule cover, the capsule cover having a texture on an exterior surface. In some aspects, the texture includes at least one dimple, the at least one dimple configured to facilitate swallowing of the tethered imaging capsule by a patient. In some aspects, the texture includes at least one channel, the at least one channel configured to facilitate swallowing of the tethered imaging capsule by a patient.

[0016] In some embodiments, the display is operatively coupled to the hyperspectral processing system, and the illumination system is calibrated to display objects imaged by the at least one imaging sensor on the display.

[0017] In some embodiments, the capsule includes a control system (e.g., a control system in an illumination system or a hyperspectral imaging system) configured to control the sequence of different illumination lights and imaging of the at least one imaging sensor.

[0018] In some embodiments, the hyperspectral processing system includes a control system, a memory, and a display, wherein the control system is configured to generate a multispectral reflectance datacube, store the multispectral reflectance datacube in the memory, and cause the display to display the multispectral reflectance datacube or an image representation thereof.

[0019] In some embodiments, the at least one photodetector is configured to detect target electromagnetic radiation absorbed, transmitted, refracted, reflected, and / or emitted by at least one physical point on the target, the target radiation comprising at least two target waves, each having an intensity and a unique wavelength, detect the intensity and wavelength of each target wave, and transmit the detected target electromagnetic radiation and the detected intensity and wavelength of each target wave to a hyperspectral processing system. In some aspects, the hyperspectral processing system is configured to use the detected target electromagnetic radiation to form a target image of the target, the target image comprising at least two pixels, each pixel corresponding to a physical point on the target, use the intensity and wavelength of each detected target wave to form at least one intensity spectrum for each pixel, and generate a multispectral reflectance data cube from the at least one intensity spectrum for each pixel. In some aspects, the hyperspectral processing system is configured to: convert the intensity spectrum of each pixel formed using a Fourier transform into a complex-valued function based on the intensity spectrum of each pixel, each complex-valued function having at least one real component and at least one imaginary component; apply a denoising filter at least once to both the real and imaginary components of each complex-valued function to generate denoised real and imaginary values ​​for each pixel; form a phasor point on a phasor plane for each pixel by plotting the denoised real values ​​against the denoised imaginary values ​​of each pixel; map the phasor point back to a corresponding pixel on the target image based on the geometric location of the phasor point on the phasor plane; assign an arbitrary color to the corresponding pixel based on the geometric location of the phasor point on the phasor plane; and generate an unmixed color image of the target based on the assigned arbitrary color. In some aspects, the hyperspectral processing system is configured to display the unmixed color image of the target on a display of the hyperspectral processing system. In some embodiments, the hyperspectral processing system uses only the first harmonic or only the second harmonic of the Fourier transform to generate an unmixed color image of the object.In some embodiments, the hyperspectral processing system generates an unmixed color image of the object using only the first and second harmonics of the Fourier transform. In some embodiments, the object radiation includes a fluorescent wavelength or at least one of at least four wavelengths. In some embodiments, the hyperspectral processing system is configured to form an unmixed color image of the object with a signal-to-noise ratio for at least one spectrum ranging from 1.2 to 50. In some embodiments, the hyperspectral processing system forms an unmixed color image of the object with a signal-to-noise ratio for at least one spectrum ranging from 2 to 50. In some embodiments, the hyperspectral processing system is configured to use a reference material to assign an arbitrary color to each pixel.

[0020] In some embodiments, the hyperspectral processing system is configured to assign an arbitrary color to each pixel using a reference material, and an unmixed color image of the reference material is generated prior to generating the unmixed color image of the object. In some aspects, the hyperspectral processing system is configured to assign an arbitrary color to each pixel using a reference material, and an unmixed color image of the reference material is generated prior to generating the unmixed color image of the object, and the reference material includes a physical structure, a chemical molecule, a biological molecule, a physical and / or biological change caused by a disease, or any combination thereof.

[0021] In some embodiments, the illumination system and the hyperspectral imaging system are cooperatively configured to illuminate the reference object with a first illumination light, image the reference object during the first illumination light, illuminate the reference object with a second illumination light different from the first illumination light, image the reference object during the second illumination light, illuminate the reference object with a third illumination light different from the first illumination light and the second illumination light, and image the reference object during the third illumination light. In some aspects, the third illumination light is white light illumination. In some aspects, the reference object includes a color reference image. In some aspects, the first illumination light, the second illumination light, and the third illumination light each include illumination by at least two LEDs.

[0022] In some embodiments, the hyperspectral processing system is configured to obtain a spectrum for each pixel of the image and generate a transformation matrix from the spectrum for each pixel.

[0023] In some embodiments, the illumination system and hyperspectral imaging system are cooperatively configured to illuminate the object with a first illumination light, image the object during the first illumination light, illuminate the object with a second illumination light, image the object during the second illumination light, illuminate the object with a third illumination light, and image a reference object during the third illumination light. In some aspects, the hyperspectral processing system is configured to generate a multispectral reflectance datacube from the transformation matrix and images of the object acquired during the first illumination light, the second illumination light, and the third illumination light. In some aspects, the multispectral reflectance datacube is obtained from a pseudo-inverse method using the images of the object.

[0024] In some embodiments, a computer method can include illuminating an object with an illumination system of the imaging capsule; receiving detected object electromagnetic radiation absorbed, transmitted, refracted, reflected, and / or emitted by at least one physical point on the object from at least one imaging sensor of the imaging capsule, the object radiation including at least two object waves, each object wave having an intensity and a unique wavelength; and transmitting the detected object electromagnetic radiation and the detected intensity and wavelength of each object wave from the imaging capsule to a hyperspectral processing system.

[0025] In some embodiments, a computer method can include a hyperspectral processing system that performs the following: forming a target image of the target using the detected target electromagnetic radiation, the target image including at least two pixels, each pixel corresponding to a physical point on the target; forming at least one intensity spectrum for each pixel using the intensity and wavelength of each detected target wave; and generating a multispectral reflectance data cube from the at least one intensity spectrum for each pixel.

[0026] In some embodiments, a computer method includes a hyperspectral processing system that performs the following: based on the intensity spectrum of each pixel, converting the formed intensity spectrum of each pixel using a Fourier transform into a complex-valued function, each complex-valued function having at least one real component and at least one imaginary component; applying a denoising filter at least once to both the real and imaginary components of each complex-valued function to generate denoised real and imaginary values ​​for each pixel; forming one phasor point on a phasor plane for each pixel by plotting the denoised real values ​​against the denoised imaginary values ​​for each pixel; mapping the phasor point back to a corresponding pixel on an image of the object based on a geometric location of the phasor point on the phasor plane; assigning an arbitrary color to the corresponding pixel based on the geometric location of the phasor point on the phasor plane; and generating an unmixed color image of the object based on the assigned arbitrary color. In some aspects, the hyperspectral processing system is configured to display the unmixed color image of the object on a display of the hyperspectral processing system.

[0027] In some embodiments, the computer method can include illuminating a reference object with a first illumination light emitted from the imaging capsule, acquiring an image of the reference object using the imaging capsule during the first illumination light, illuminating the reference object with a second illumination light emitted from the imaging capsule, acquiring an image of the reference object using the imaging capsule during the second illumination light, illuminating the reference object with a third illumination light emitted from the imaging capsule, and acquiring an image of the reference object using the imaging capsule during the third illumination light. In some aspects, the computer method can include acquiring a spectrum for each pixel of the image and generating a transformation matrix from the spectrum of each pixel. In some aspects, the computer method can include illuminating the object with the first illumination light from the imaging capsule, acquiring an image of the object using the imaging capsule during the first illumination light, illuminating the object with the second illumination light from the imaging capsule, acquiring an image of the object using the imaging capsule during the second illumination light, illuminating the object with the third illumination light from the imaging capsule, and acquiring an image of the reference object using the imaging capsule during the third illumination light. In some embodiments, the hyperspectral processing system is configured to generate a multispectral reflectance data cube from a transformation matrix and images of the object acquired during the first illumination light, the second illumination light, and the third illumination light. In some embodiments, the multispectral reflectance data cube is obtained from a pseudo-inverse method using the images of the object.

[0028] The foregoing summary is illustrative only and is not intended to be in any way limiting. In addition to the exemplary aspects, embodiments, and features described above, further aspects, embodiments, and features will become apparent by reference to the drawings and the following detailed description.

[0029] The above and following information, as well as other features of the present disclosure, will become more fully apparent from the following description and appended claims, taken in conjunction with the accompanying drawings, in which: The present disclosure will be described with additional specificity and detail through the use of the accompanying drawings, with the understanding that these drawings illustrate only some embodiments in accordance with the present disclosure and therefore should not be considered limiting of its scope. [Brief explanation of the drawings]

[0030] [Figure 1A] 1 includes a schematic diagram of a capsule hyperspectral system including an imaging capsule and a hyperspectral processing system. [Figure 1B] 1 includes a cross-sectional schematic view of one embodiment of an imaging capsule. [Figure 1C] 1 includes a cross-sectional schematic view of one embodiment of an imaging capsule. [Figure 1D] Includes a diagram of the imaging capsule tethered to the drone. [Figure 1E] Included is a diagram of an imaging capsule configured as a ground vehicle. [Figure 1F] Includes a diagram of the imaging capsule tethered to a small crane. [Figure 2A] 1 includes a schematic diagram of the front end plate of the capsule with the imaging sensor and array of LEDs. [Figure 2B] 1 includes a schematic diagram of the front end plate of the capsule with two imaging sensors and an array of LEDs. [Figure 2C] 1 includes a schematic diagram of a tethered end plate of a capsule having an imaging sensor and an array of LEDs. [Figure 2D] 1 includes a schematic diagram of a tethered end plate of a capsule with two imaging sensors and an array of LEDs. [Figure 3A] 1 includes a schematic diagram of a side plate of the capsule with an imaging sensor and an array of LEDs. [Figure 3B] 1 includes a schematic diagram of the capsule's side plate with two imaging sensors and an array of LEDs. [Figure 4A] 10 includes a tether end view of one embodiment of a capsule having a textured cover with indentations. [Figure 4B] 1 includes a side view of one embodiment of a capsule having indentations in a textured cover. [Figure 4C] 1 includes a tether end view of one embodiment of a capsule having a channel in a textured cover. [Figure 4D] 1 includes a side view of one embodiment of a capsule having a channel in a textured cover. [Figure 5] 1 includes a flowchart of a protocol for using the imaging capsule and hyperspectral processing system to convert an image into a hyperspectral unmixed color image. [Figure 6] 1 includes a schematic diagram of the workflow for generating a multispectral reflectance datacube. [Figure 7A] It includes an image (e.g., a representation of a multispectral reflectance datacube) showing the esophagus under normal white light illumination. [Figure 7B] 1 includes an image showing the esophagus in a false-color hyperspectral phasor image. [Figure 7C] Included is a graph showing the corresponding GS histogram (e.g., phasor plot) of the esophagus. [Figure 8A] Include an image (e.g., a representation of a multispectral reflectance datacube) showing the intestine under normal white light illumination. [Figure 8B] 1 includes an image showing the intestine in a false-color hyperspectral phasor image. [Figure 8C] Include a graph showing the corresponding GS histogram (e.g., phasor plot) of the intestine. [Figure 9] Included is a schematic diagram of a computing device that can be used with the systems and methods of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0031] The elements and components of the drawings may be arranged in accordance with at least one of the embodiments described herein, and the arrangement may be altered by one skilled in the art in accordance with the disclosure provided herein.

[0032] In the following detailed description, reference is made to the accompanying drawings, which form a part hereof. In the drawings, like symbols generally identify like elements unless context dictates otherwise. The illustrative embodiments described in the detailed description, drawings, and claims are not meant to be limiting. Other embodiments may be utilized, and other changes may be made, without departing from the spirit or scope of the subject matter presented herein. It will be readily understood that the aspects of the present disclosure, as generally described and illustrated in the figures herein, can be arranged, substituted, combined, separated, and designed in a variety of different configurations, all of which are expressly contemplated herein.

[0033] The present invention relates to a compact, low-cost tethered endoscope developed for colored-light, white-light, and hyperspectral-based screening of tissues, such as throat tissue. The tethered endoscope can be swallowed so that it can be used to visualize and diagnose esophageal diseases. This tethered imaging capsule can be designed for single use or a limited number of uses and may be intended for use by medical assistants, nurses, or physicians in primary care settings before referral to a specialist (e.g., a gastroenterologist for esophageal endoscopy). The technical advantages of this design provide improved overall effectiveness of the esophageal disease screening process. However, the imaging capsule may be untethered or coupled to a machine, such as a drone, ground vehicle, or crane. The size of the capsule is small enough to be swallowed, thereby allowing the size of the machine to be equally small to fit into small spaces.

[0034] An exemplary capsule hyperspectral system may include a tethered imaging capsule, a tether, an optical illumination system (e.g., colored light and white light), a hyperspectral imaging system, and a hyperspectral processing system. An exemplary optical illumination system may comprise an LED illumination system. In some embodiments, the capsule may include at least three light sources ("illumination sources") for illuminating the object, two of which may be colored and the third white, and the illumination sources generate electromagnetic radiation ("illumination source radiation") including at least one wave ("illumination wave") or band for each of the three light sources. The capsule hyperspectral system may further include an imaging sensor (e.g., a camera) and a display.

[0035] The imaging capsule of the present invention is a significant improvement over first-generation devices with low resolution (400 x 400 pixels). It has been found that low-resolution cameras lack the resolution and precise location capabilities to image suspicious areas of esophageal disease, such as esophageal squamous cell carcinoma. A condition known as Barrett's esophagus may not be clearly visible at this low resolution. The present imaging capsule therefore provides an improvement by enabling high resolution from hyperspectral imaging. Here, the improved imaging capsule can provide high resolution and precise location capabilities to image suspicious areas of esophageal disease, such as esophageal squamous cell carcinoma. The condition known as Barrett's esophagus can be clearly seen with the high-resolution imaging capsule.

[0036] The high-definition imaging capsule of the present invention with hyperspectral processing can provide an integrated custom hardware illumination system that can utilize at least three LEDs for visualization and imaging, preferably with two-color illumination followed by white light illumination, in three sequential steps. At least one of the LEDs is white. The number of LEDs in the capsule can range from three to six or more LEDs, which can enable software-based hyperspectral decomposition of high-resolution images (e.g., up to 1280 x 1080 pixels at a 60 Hz frame rate) using a non-specialized CMOS-based imaging sensor. A frame rate of 60 Hz or greater is used to minimize motion artifacts during screen capture. Imaging can have effective filtering of wavelength bands of approximately 10 nm, the visible wavelength range of approximately 410 nm to approximately 750 nm, or the device's effective spectral range. For example, six LEDs can enable the identification of 32 spectral bands in the visible light spectrum.

[0037] The imaging sensor can be configured as a high-definition camera with resolution comparable to existing top-of-the-line, FDA-approved gastrointestinal (GI) endoscopes (e.g., Olympus endoscopes with 1080p resolution). High frame rates (60+ fps) can significantly reduce motion artifacts in screen-captured images for detailed analysis by automated machine vision software programs or medical professionals.

[0038] A tether (e.g., a wire, guidewire, cord, data cord, etc.) can be attached to the capsule. The tether can allow a moderately experienced user to manually and precisely rotate the camera position within the esophagus to improve imaging of suspected diseased areas. This tether can not only provide power and a data transfer link to the capsule, but can also mark its surface with visible markings at regular intervals so that the user can accurately measure the location of suspected diseased areas within the esophagus for later follow-up examinations.

[0039] The surface of the capsule casing may have texture (e.g., surface dimples, and grooves or channels not parallel to the longitudinal axis of the capsule) that may remove fluid from the front of the device and facilitate swallowing and recovery after the screening test.

[0040] The capsule hyperspectral system may be intended for use in screening tissue within the gastrointestinal tract, for example, after swallowing a tethered capsule. The tissue may be any gastrointestinal tissue, such as esophageal tissue, which may be useful in identifying esophageal disease. The tethered capsule may be used in primary care facilities with limited access to secondary or tertiary GI specialists.

[0041] The capsule hyperspectral system can be used to visualize esophageal diseases and conditions that can be diagnosed using this system. Some examples include, but are not limited to, Barrett's esophagus, esophageal cancer (EC), gastric cancer, gastroesophageal reflux disease (GERD), peptic ulcer disease, dysphagia, and esophageal varices. Because esophageal varices are commonly associated with liver disease, the capsule hyperspectral system can be used to diagnose liver disease.

[0042] In some embodiments, the capsule hyperspectral system can be configured as a low-cost, easy-to-use HD-TIC system intended for annual or periodic medical screening of the esophagus for dysplasia due to esophageal cancer (adenocarcinoma and squamous cell carcinoma) and related symptoms of liver disease (e.g., esophageal varices, other signs of portal hypertension).

[0043] However, the capsule hyperspectral system may be used in other environments, such as crevasses, wells, small tunnels or conduits, air flow paths, ventilation systems, nature, or any other location or use. The target can be any object of interest for illumination and imaging.

[0044] The description of the capsule hyperspectral system provided herein can be applied to any environment for illumination and imaging as described.

[0045] FIG. 1A illustrates one embodiment of a capsule hyperspectral system 100. The capsule hyperspectral system 100 is shown to include a tethered imaging capsule 102 attached at its end to a tether 104. The capsule 102 includes an optical illumination system 106 having at least three light emitters 107 configured to emit various colors (e.g., red, blue, green, yellow, orange, purple, etc.) as well as white light. The capsule hyperspectral system 100 may include the imaging capsule 102 with a hyperspectral imaging system 108 having at least one imaging sensor 109. The tether 104 is operably coupled to a hyperspectral processing system 110 having at least one processor 111, which may be at least a portion of one or more computers, as shown in FIG. 9.

[0046] Capsule 102 may be tethered to hyperspectral processing system 100, or may be separate or untethered, in which case capsule 102 may include a memory card that can be plugged into hyperspectral processing system 100, or capsule 102 may be plugged directly into hyperspectral processing system 100.

[0047] The illumination system 106 may include an LED illumination system including three or more LEDs as light emitters 107. The LEDs may be calibrated for the camera (e.g., imaging sensor 109) of the imaging capsule 102. The LEDs may also be adjusted to match the display 112 so that an image of the illuminated tissue (e.g., esophagus) can be freely displayed on any display system having a display 112. In some embodiments, the imaging sensor 109 is centered on the axis of the tethered imaging capsule 102 as shown (e.g., FIG. 2A). In some embodiments, the imaging sensor 109 may have an eccentric position relative to the axis 114 of the tethered imaging capsule (e.g., FIG. 2B). For example, the imaging sensor 109 may be eccentrically positioned at angle 116 relative to the axis of the tethered imaging capsule, with the camera positioned 35 degrees, + / - 1%, 2%, 5%, 10%, or 20% from the axis. That is, the light directed from the imaging sensor 109 may be at angle 116 from the axis 114.

[0048] In some embodiments, the capsule hyperspectral system 100 may further include a uniformly arranged array of multiple LEDs. For example, a design may include at least or up to six LEDs (e.g., three pairs) for uniform illumination, such as arranged around an imaging sensor. The LED emission wavelengths may be selected to easily distinguish the white / pinkish surface of a healthy esophagus from the red surface of a non-healthy esophagus. However, it should be recognized that at least three LEDs can perform the operations described herein in three different lighting conditions. The three different lighting conditions can use two lights for each condition, thereby using six lights. The light pairs for each lighting condition can be improved, along with the light coverage, to improve imaging. While three pairs of LEDs are a good example, there can be six different light colors. Alternatively, there can be at least two different pairs of colored lights and a pair of white lights, which can include a same-color pair (both green) or a different-color pair (e.g., red and blue). The number and different colors of LEDs are not limited herein.

[0049] In some embodiments, capsule 102 can provide color images to any type of display system. The illuminators can illuminate with any combination of light colors during imaging, which can be varied and displayed.

[0050] 1A also shows an irrigation system 160 that can include a pump that provides irrigation fluid (e.g., water) to the site being imaged to cleanse the site. Irrigation can remove debris or body material to improve imaging. Irrigation system 160 can include an irrigation conduit 162 having an opening at or near capsule 102 to release fluid around capsule 102. Conduit 162 can be around tether 104 or otherwise associated with the tether.

[0051] 1B-1C illustrate a hyperspectral imaging system 108, which may include an optical system 118. The optical system 118 may include at least one optical component. The at least one optical component may include at least one photodetector, such as an imaging sensor 109, and optionally a lens system 120 (e.g., one or more lens systems) optically coupled to the imaging sensor 109. The photodetector may be any photodetector, which may be a photodiode or other imaging sensor. The detector may be a camera imaging device (e.g., a photomultiplier tube, a photomultiplier tube array, a digital camera, a hyperspectral camera, an electron-multiplying charge-coupled device, a sci-CMOS, or a combination thereof). The photodetector may be configured to detect electromagnetic radiation of interest (“target radiation”) absorbed, transmitted, refracted, reflected, and / or emitted by at least one physical point on the target, the target radiation including at least two target waves (“target waves”), each wave having an intensity and a different wavelength, detect the intensity and wavelength of each target wave, and transmit the detected target radiation and the detected intensity and wavelength of each target wave to hyperspectral processing system 110. Hyperspectral processing system 110 is described in more detail below.

[0052] The at least one optical component may include an optical lens 122, an optical filter 124, a dispersion optics 130, or a combination thereof. The at least one optical component may further include a first optical lens 126, a second optical lens 128, and an optical filter 124, which may be configured as a dichroic mirror / beam splitter. The at least one optical component may further include the optical lens 122, the dispersion optics 130, and the at least one imaging sensor 109 is a photodetector array 109a.

[0053] In some embodiments, the at least one optical component may include an optical filtering system having at least one optical filter 124 disposed between the object to be imaged and the at least one imaging sensor 109. The object radiation emitted from the object may include electromagnetic radiation emitted by the object. In some aspects, the electromagnetic radiation emitted by the object includes fluorescence. In some aspects, the noise reduction filter of the optical filter 124 may include a median filter.

[0054] As shown in FIG. 1C, the capsule hyperspectral system 100 may further include a lens system. The lens system of one or more lenses may include a lens 122 having a field of view (FOV) ranging from about 120 degrees to greater than 180 degrees, or from about 120 degrees to about 190 degrees, or from about 120 degrees to about 180 degrees. The lens system may be configured to image the posterior surface of the gastroesophageal sphincter. In some embodiments, the lens system may be configured as a replaceable or interchangeable lens system. The lens system may also be disposed at the distal end of the capsule hyperspectral system, such as a capsule. The lens system may include two or more lenses.

[0055] The capsule 102 may include an illumination system and a detection system as described in WO 2018 / 089383, which is incorporated herein by specific reference in its entirety, such as in Figures 14 to 21.

[0056] FIG. 1D shows an imaging capsule 102a configured for use from a drone 140 (e.g., an unmanned aerial vehicle). The drone 140 includes a tether system 104 having mechanical components 144 (e.g., pulleys, spindles, winches, etc.) for raising or lowering the capsule 102a on the tether 104a. The tether 104a can be lengthened or shortened as needed or desired for use in imaging. For example, the drone 140 may not be able to descend, but the mechanical components 144 can lower the tether 104a to lower the capsule 102a. The capsule 102a and / or drone 140 can include a controller (e.g., a computer) described herein that can operate the drone 140 and capsule 102a for imaging purposes. The drone 140 and / or capsule 102a can include a transceiver that can transmit data to the hyperspectral processing system 110, such as via wireless data transmission. The tether 104a may provide a data line for data communication between the drone 140 and the capsule 102a, or each may include a transceiver for wireless data communication. The drone 140 may also be controlled using a remote controller 146, which may wirelessly control the operation of the drone 140. The remote controller 146 may communicate with the drone 140 directly or through a control module, which may be part of the computer of the hyperspectral processing system 110.

[0057] FIG. 1E shows an imaging capsule 102b configured to be used from a ground vehicle 148 (e.g., an unmanned ground vehicle, a remotely controlled device). The capsule 102b can be attached to the vehicle 148 in any manner and serve as the vehicle's body. The vehicle 148 can be the size of a regular RC car or can be small to take advantage of the capsule 102b's small size, which can be swallowed. The small ground vehicle 148 can be used to access small locations inaccessible to humans or larger facilities. The capsule 102b and / or the vehicle 148 can include a controller (e.g., a computer) described herein that can operate the vehicle 148 and capsule 102b for imaging purposes. The vehicle 148 and / or capsule 102b can include a transceiver that can transmit data to the hyperspectral processing system 110, such as via wireless data transmission. The vehicle 148 can also be controlled using a remote controller 146, which can wirelessly control the operation of the vehicle 148. The remote controller 146 may communicate directly with the vehicle 148 or may communicate through a control module that may be part of the computer of the hyperspectral processing system 110. Alternatively, the ground vehicle may be configured like a tank, dog, insect, or spider, with wheels, treads, legs, and other moving members to propel the vehicle.

[0058] FIG. 1F shows an imaging capsule 102a configured for use with a small crane 150 (e.g., a winch). The crane 150 includes a tether system 152 having a mechanical component 154 (e.g., a winch) for raising or lowering the capsule 102c on a tether 104b. The tether 104b can be lengthened or shortened as needed or desired for use in imaging. For example, the crane 150 may not be able to lower itself when deployed to a use position (e.g., attached to a well), but the mechanical component 154 can lower the tether 104b to lower the capsule 102c. The capsule 102c and / or the crane 150 can include a controller (e.g., a computer) as described herein that can operate the crane 150 and the capsule 102c for imaging purposes. The crane 150 and / or the capsule 102c can include a transceiver that can transmit data to the hyperspectral processing system 110, such as via wireless data transmission. Tether 104b may provide a data line for data communication between crane 150 and capsule 102c, or each may include a transceiver for wireless data communication. Also, remote controller 146 may be used to control crane 150, which may wirelessly control the operation of crane 150. Remote controller 146 may communicate with crane 150 directly or through a control module, which may be part of the computer of hyperspectral processing system 110.

[0059] FIG. 2A shows a front view of an imaging capsule 102 with an imaging sensor 109 and six LED light emitters (e.g., light emitters 107) arranged around it. In this example, the front view is looking down the axis 114 of the capsule 102. The light emitters 107 are arranged on a pattern plate 130, and the light emitters 107 can be any of the colors and combinations described herein. The light emitters 107 can include white light LEDs and specific combinations of narrowband colored LEDs. During manufacturing, the arrangement of the imaging sensor 109 and light emitters 107 on the plate 130 can be modified to accommodate specific variations of the high-definition imaging capsule (e.g., variations for white light imaging and / or variations for hyperspectral imaging). The pinout alignment of this plate 130 matches the wiring to the tether and / or capsule. The light emitters 107 can illuminate simultaneously, in pairs, or sequentially (e.g., in a pair order), as selected by the user in software settings and electronic switching. For white light illumination, all of the white LEDs are illuminated at once. For hyperspectral imaging, colored LEDs and optionally white LEDs are illuminated in a sequence synchronized with the frame rate of the imaging sensor 109. In one example, this arrangement enables the generation of a hyperspectral data cube that, after post-processing using hyperspectral decomposition, can be used to identify dysplastic regions within the esophagus by color differences (e.g., shifts or differences in response wavelengths of illuminated regions of the esophagus, indicating possible precancerous or cancerous lesions). However, it should be recognized that any imaging of tissue or any other object or environment can be performed. The schematic diagram of the imager and LEDs in FIG. 2A shows a single imaging sensor 109. The schematic diagram of the imager and LEDs in FIG. 2B shows a pair of imaging sensors 109 that are off-center from the central axis.

[0060] Each light emitter 107 may include a coherent electromagnetic radiation source. The coherent electromagnetic radiation source may include a laser, a diode, a two-photon excitation source, a three-photon excitation source, or a combination thereof. The light emitter radiation may include an illumination wave having a wavelength in the range of 300 nm to 1,300 nm. The illumination source radiation may include an illumination wave having a wavelength in the range of 300 nm to 700 nm. The illumination source radiation may include an illumination wave having a wavelength in the range of 690 nm to 1,300 nm.

[0061] In one example, LED1 could be blue, LED2 orange, LED3 green, LED4 red, and LED5 and LED6 both white. During imaging, you illuminate with two colored lights, which can be any combination of these two lights. Then, for the next imaging, you illuminate with two different lights. And for the third imaging, you use two white LEDs. This helps analyze each part of the light spectrum with different lights. This helps build the color of the object. In another example, only three LEDs are used. There could be two colored LEDs and one white LED.

[0062] 2B shows a front view of one embodiment of a tethered imaging capsule 102 having multiple imaging sensors 109 surrounded by light emitters 107 in an array on a plate 130. The imaging sensors 109 are off-center from a central axis. The imaging sensors 109 may be oriented parallel, or their surfaces may be at an angle such that they both point to a common point on the central axis.

[0063] FIG. 2C shows an end view of the tethered imaging capsule 102 with the imaging sensor 109 surrounded by the light emitter 107, which is an LED light emitter. The light emitter 107 is arranged on a pattern plate 130 and can include a combination of white light LEDs or narrowband colored LEDs. During manufacturing, the configuration or pattern / arrangement of the imaging sensor 109 and light emitter 107 on the plate 130 can be modified to accommodate specific variations of the high-definition imaging capsule (e.g., variations for white light imaging or hyperspectral imaging). The pinout alignment of this LED plate 130 matches the wiring from the capsule 102 to the tether 104. The light emitters 107 can be illuminated one at a time, in pairs, or sequentially, as selected by the user in software settings and electronic switching. For white light illumination, the white LEDs are illuminated all at once. For hyperspectral imaging, the LEDs are illuminated in a sequence synchronized with the frame rate of the imaging sensor 109. This allows for the generation of hyperspectral data cubes that, after post-processing with hyperspectral decomposition methods, can characterize objects, such as identifying dysplastic regions within the esophagus via color differences (e.g., shifts or differences in response wavelengths of illuminated regions of the esophagus that indicate possible precancerous or cancerous lesions). The schematic of the imager and LEDs in Figure 2C shows a single imaging sensor 109 offset from the tether 104.

[0064] FIG. 2D shows a plate 130 having multiple imaging sensors 107 offset from tether 104 and surrounded by imaging sensors 109 .

[0065] FIG. 3A shows a side view of a tethered imaging capsule 102 with an imaging sensor 109 surrounded by an LED light emitter 107. The light emitter 107 is placed on a pattern plate 130, which can be any of the colors or color combinations described herein, such as a specific combination of white light LEDs and narrowband colored LEDs. During manufacturing, this plate 130 can be modified to accommodate specific variations of the high-definition imaging capsule (e.g., white light imaging or hyperspectral imaging). The pinout alignment of this LED plate 130 matches the wiring from the capsule to the tether. The light emitters 107 can be illuminated one at a time, in pairs, or sequentially, as selected by the user through software configuration and electronic switching. For white light illumination, all white LEDs are illuminated at once. For hyperspectral imaging, the LEDs are illuminated in a sequence synchronized with the frame rate of the imaging camera. The imager and LED schematic in FIG. 3A shows a single imaging sensor 109, while FIG. 3B shows multiple imaging sensors 109.

[0066] In some embodiments, tether 104 can be any type of suitable tether for attaching capsule device 102 to the rest of the system. Tether 104 can have any cross-sectional shape in some embodiments. While FIG. 2C shows a square cross-sectional profile and FIG. 2D shows a circular cross-sectional profile, other cross-sectional shapes may be used. Tether 104 can have a cross-sectional shape that helps provide a reference for the capsule's location within the body. For example, each surface of the shape can have an identifier that can be observed and tracked to determine the orientation of capsule 102 and its camera. Rotating tether 104 until the next body surface is up can provide a rotation angle based on the number of sides, such as 90 degrees for a square. In some aspects, the tether can be a non-circular tether 104 that can be polygonal, such as a triangle, rectangle, square, pentagon, or other polygon, and is shown with a square cross-sectional profile. Non-circular tether 104 can be configured to create an angular reference for the user to know where the capsule is located during use; for example, each surface is labeled to track and observe the surface facing up. That angular reference can then be used to allow the user to precisely rotate and position the camera in the same position for follow-up studies. The tether 104 may include markings 104a (e.g., ruler lines, inches, centimeters, etc.) on its surface configured to determine the position of the tethered imaging capsule 102 when the tethered imaging capsule is deployed. The non-circular tether 104 allows for precise manual rotation and positioning of the capsule relative to the side wall of the esophagus.

[0067] In some embodiments, the tether 104 is tethered to the capsule 102. The coupling of the tether 104 to the capsule 102 can include a mechanical portion to form a mechanical coupling, and can include an optical and / or electronic coupling for transmitting data from the capsule to the system and vice versa. The capsule hyperspectral system 100 can include a mechanical coupling that forms a semi-rigid connection 132 between the tether 104 and the tethered imaging capsule 102 to withstand manual manipulation for positioning the capsule. The semi-rigid connection can be via epoxy or other bonding materials. Alternatively, a silicone connection can be used to provide the semi-rigid connection. The semi-rigidity provides flexibility to prevent the tether from breaking from the capsule 102.

[0068] In some embodiments, the imaging capsule may include a capsule cover (e.g., silicone). In some aspects, the capsule cover has a texture on its surface. In some examples, the texture includes dimples and / or channels or other features. The texture may be configured to allow a patient to easily swallow the tethered imaging capsule. That is, the texture may assist the patient in swallowing the capsule. While the cover may be applied to capsules adapted to be swallowed by a patient, other types of capsules for environmental or object imaging may also have covers.

[0069] FIG. 4A shows a bottom view of the tethered end of capsule 102, and FIG. 4B shows a side view of the tethered capsule 102, where capsule 102 has a cover 160 with a textured surface 162. The textured surface 162 of capsule cover 160 can be used to promote ease of swallowing and direct liquid expulsion away from the imager end of capsule 102. By strategically placing the textured surface structure, this effectively creates hydrophilic regions on capsule cover 160 that promote the accumulation of liquid droplets. This acts to effectively attract water away from the direction of the lens(es) in the imager portion of capsule 102. This figure shows an exemplary texture, namely, an array of small dimples 164 and large dimples 166.

[0070] FIG. 4C shows a bottom view of the tethered end of capsule 102, and FIG. 4D shows a side view of the tethered capsule 102, where capsule 102 has a cover 160 with a textured surface 162. The textured surface 162 of capsule cover 160 is provided to facilitate swallowing and to direct liquid expulsion away from the imager end of capsule 102. By strategically locating the textured surface 162 structure, it effectively creates hydrophilic regions on capsule cover 160 that promote droplet accumulation. This acts to effectively attract water away from the direction of one or more lenses of the imager. This figure shows an exemplary texture, namely, an array of long channels 166 and short, raised channels 164.

[0071] The capsule system may have integrated hardware and software components. The hardware may include, for example, a small, high-definition camera (e.g., an imaging sensor) with custom lighting (e.g., light emitters such as LEDs). This lighting may enable the use of hyperspectral post-processing techniques to assist non-GI medical professionals in the early detection of signs of esophageal disease.

[0072] The capsule system may have the following designed features and advantages: The capsule imaging system can provide high-definition video with image resolution comparable to the latest generation Olympus and Pentax endoscopes costing over 100 times the cost. The tether may be strong and flexible, whether circular or noncircular (e.g., polygonal, such as a flat rectangle). This configuration allows the tether to (i) facilitate patient swallowing, (ii) facilitate capsule retrieval after use (e.g., examination by a single nurse, no assistant required), and (iii) allow medical professionals to manually rotate and position the capsule at a precise location within the upper GI tract (e.g., the esophagus and upper stomach) in a controlled, mechanical, and analog manner. The capsule imaging system may have a variety of lens options (e.g., 120-degree FOV to +170-degree FOV or approximately 140-degree FOV with different magnifications) for optimal screening of throat tissue, such as for imaging various types of cancer, as well as difficult-to-access objects or environments. Different lens systems can be interchangeable so that different needs can use different lens systems. The capsule illumination system can be configured with appropriate light emitters that emit broadband white light for general illumination, or with custom LED configurations for illumination suitable for hyperspectral analysis. This can make the video images compatible with existing hyperspectral analysis software (see, e.g., the incorporated references).

[0073] In some embodiments, the swallowable version of the capsule does not require or use an additional cornstarch-based taste enhancer to make the capsule palatable. The swallowable capsule can include a textured cover on the outer capsule casing to make it easier to swallow. The capsule can be enlarged in diameter with the addition of dimples or channels, and optionally, texture can be added to the cover to make the capsule easier to swallow without the taste enhancer.

[0074] The hyperspectral processing system can be operably coupled to the imaging capsule via a tether or wireless data communication. Accordingly, the tether can include a data transmission line, such as an optical data transmission line and / or an electrical data transmission line. However, in one aspect, the hyperspectral imaging system can include a battery for operating components within the capsule and a transceiver for transmitting and receiving data to and from the hyperspectral processing system. The capsule can also have memory for storing images or videos acquired during use, which can then be downloaded to the hyperspectral processing system.

[0075] The hyperspectral processing system can be or include a computer with specialized software capable of performing the imaging described herein. Thus, Figure 9 illustrates an example of hardware components of a hyperspectral processing system. The memory device of the hyperspectral processing system can include computer-executable code that causes the computer to perform the methods described herein for imaging tissue using a capsule hyperspectral system.

[0076] The capsule system hardware may be optimized to produce high-quality images that can be analyzed by a medical assistant or medical professional. The imaging hardware may also be optimized to be compatible with hyperspectral imaging and associated automated machine learning algorithms. Accordingly, this hardware may be used in the hyperspectral systems and methods disclosed in the PCT application entitled "A Hyperspectral Imaging System" (International Publication No. WO 2018 / 089383), the contents of which are incorporated herein in their entirety. Briefly, hyperspectral decomposition systems and methods that may be used with the hyperspectral endoscopic system of the present disclosure are outlined herein.

[0077] Another example of a hyperspectral imaging system is shown schematically in FIG. 5. In this example, the hyperspectral imaging system further includes at least one detector 109 or detector array 109a. The imaging system may use the detector or detector array to form an image of an object 401 (forming an object image). The image may include at least two waves and at least two pixels. The system may form an image of the object using the intensity of each wave “intensity spectrum” 402 (spectrum formation). The system may transform the intensity spectrum of each pixel by using a Fourier transform 403, thereby forming a complex-valued function based on the detected intensity spectrum of each pixel. Each complex-valued function may have at least one real component 404 and at least one imaginary component 405. The system may apply a denoising filter 406 to both the real and imaginary components of each complex-valued function at least once. This allows the system to obtain denoised real and denoised imaginary values ​​for each pixel. The system may plot the denoised real values ​​against the denoised imaginary values ​​for each pixel, thereby allowing the system to form a point on the phasor plane 407 (plot on the phasor plane). The system may form at least one additional point on the phasor plane by using at least one or more pixels of the image. The system may select at least one point on the phasor plane based on its geometric location on the phasor plane. The system may map 408 the selected point on the phasor plane back to a corresponding pixel on the image of the object and assign a color to the corresponding pixel, the color being assigned based on the geometric location of the point on the phasor plane. As a result, the system may thereby generate an unmixed color image of the object 409.

[0078] The imaging system includes causing a photodetector to detect object radiation, transmitting the intensity and wavelength of each detected object wave to the imaging system, acquiring the detected object radiation including at least two object waves, forming an image of the object using the detected object radiation (the "object image"), the object image including at least two pixels, each pixel corresponding to a physical point on the object, forming at least one spectrum for each pixel (the "intensity spectrum") using the intensity and wavelength of each detected object wave, and using a Fourier transform to convert the formed intensity spectrum for each pixel into a complex-valued function based on the intensity spectrum for each pixel, each complex-valued function having at least one real component and at least one imaginary component. applying a denoising filter at least once to both the real and imaginary components of each complex-valued function to generate denoised real and denoised imaginary values ​​for each pixel, plotting the denoised real values ​​against the denoised imaginary values ​​for each pixel to form a point on a phasor plane (a "phasor point") for each pixel, mapping the phasor points back to corresponding pixels on the object image based on the geometric locations of the phasor points on the phasor plane, assigning arbitrary colors to the corresponding pixels based on the geometric locations of the phasor points on the phasor plane, and generating an unmixed color image of the object based on the assigned arbitrary colors. The imaging system may also be configured to display the unmixed color image of the object on a display of the imaging system.

[0079] The imaging system may be configured to generate an unmixed color image of the object using at least one harmonic of the Fourier transform. The imaging system may be configured to use at least a first harmonic of the Fourier transform to generate the unmixed color image of the object. The imaging system may be configured to use at least a second harmonic of the Fourier transform to generate the unmixed color image of the object. The imaging system may be configured to use at least the first harmonic and the second harmonic of the Fourier transform to generate the unmixed color image of the object.

[0080] Methods of operation can include those described herein. The imaging capsule can be coupled to a hyperspectral processing system via a tether or wirelessly. In some embodiments, the imaging capsule can then be swallowed by a patient for imaging of the patient's throat. The hyperspectral processing system can activate multiple light emitters in an illumination system to illuminate the esophagus. The hyperspectral processing system can cause the hyperspectral imaging system to cause at least one sensor to image the esophagus and transmit image data to the hyperspectral processing system. The hyperspectral processing system can then process the image data as described herein and in the incorporated references to generate an image of the tissue. The image data can be used to generate a multispectral reflectance data cube from a series of images.

[0081] In some embodiments, the imaging capsule can then be lowered into a crevasse, well, or other small-opening environment to image in areas inaccessible to humans. The hyperspectral processing system can activate multiple light emitters in the illumination system to illuminate the environment or its objects. The hyperspectral processing system can cause the hyperspectral imaging system to cause at least one sensor to image the environment and transmit image data to the hyperspectral processing system. The hyperspectral processing system can then process the image data as described herein and in the incorporated references to generate an image of the environment and the objects therein. The image data can be used to generate a multispectral reflectance datacube from the series of images.

[0082] The unmixed color image of the object may be formed with at least one spectral signal-to-noise ratio in the range of 1.2 to 50. The unmixed color image of the object may be formed with at least one spectral signal-to-noise ratio in the range of 2 to 50.

[0083] The object may be any object, and the environment may be any environment, whether in a living or inanimate environment. The object may be any object having a particular spectrum of colors. For example, the object may be tissue, a fluorescent genetic label, an inorganic object, or a combination thereof. In the environment, the object may be a plant or leaf to check the plant's health or the crop's readiness for cultivation.

[0084] A hyperspectral imaging system may be calibrated by using a reference material to assign an arbitrary color to each pixel. The reference material may be any known reference material. For example, the standard may be any reference material for which an unmixed color image of the reference material is determined prior to generating an unmixed color image of the subject. For example, the reference material may be a physical structure, a chemical molecule, a biological molecule, a physical structure change, and / or a biological activity (e.g., a physiological change) resulting from a disease.

[0085] Figure 6 illustrates a two-stage imaging protocol. Stage 1 involves imaging at least two color standards 502a, 502b, which may be the same or different (e.g., different colors of the standards). Each color standard 502a, 502b is illuminated with a sequence of colors shown as blue 504, green 506, and red 508, although other colors can be used, such as replacing red 508 with white light illumination. Both color standards 502a, 502b are imaged with the same colors in the same color sequence. The illumination for each color can be a single color or a single LED, although better illumination can be achieved with a pair of LEDs, which can be the same color or a color pair (e.g., red and blue). During illumination, three sequential images are acquired to match the three sequential illuminations: illuminating two LEDs, then capturing an image, then illuminating two LEDs, then capturing an image, then illuminating two LEDs, then capturing an image, etc.

[0086] Using at least one pair of LEDs per illumination can be beneficial due to the spectral characteristics of the LEDs. Using at least one pair of LEDs per image capture can extend the sampling range. For example, a blue LED will only sample the blue region and not so much the yellow or other colored regions. If the illumination is a blue LED with a yellow LED, the information comes from the blue and yellow LEDs, which is better. A second illumination and imaging step uses a red and green LED for imaging. A third step uses a pair of white LEDs. The data now contains the color spectrum of the object for different illuminations. For each location object (e.g., pixel), the system knows what the spectrum is. For each pixel in the image, the data contains three sets of encoding information from the three sets of illumination and imaging. Using the three different illumination and imaging sets of encoded data, the system can determine that a particular pixel corresponds to a point of color on the object, and that a point of color on the object has a particular spectrum as indicated by the pixel spectrum graph 510, such as a graph of each unique color object on a color reference. Regardless of the number of sets of images captured, three matrices exist.

[0087] The data is provided to obtain a transformation matrix 512. The protocol finds the transformation matrix and then maximizes it by multiplying it with whatever data was collected. This is the one that most closely matches the spectrum or color of the object. There are many different colors of the object (color reference), which provides many different spectra 510. The process repeats for all of the different colors until a matrix is ​​found that works well for the majority of the acquired spectra 510. Essentially, the matrix is ​​a correction matrix from the three different illuminations from the LEDs. Once the protocol finds that matrix, it remains fixed unless the equipment needs to be changed. This effectively calibrates the system using the equipment used to have the transformation matrix. The transformation matrix allows the imaged object to be reconstructed.

[0088] In one example, once the system has a series of images, it knows the spectrum. The pixels of the color reference 502a, 502b correspond to this spectrograph 510, and then the protocol finds the transformation matrix. The system visually determines the transformation matrix for each of the three acquired images, and the protocol multiplies the image pixel wavelength data by the transformation matrix to obtain a hyperspectral cube. The hyperspectral cube contains X and Y dimensions, with the third dimension being wavelength. Thus, for every pixel, the protocol obtains a visually usable spectrum from the transformation.

[0089] Stage 1 can reconstruct a hyperspectral cube from a digital image using a pseudo-inverse method. In Stage 1, a CMOS camera is used to capture images of the ColorChecker® standard (US X-Rite Passport Model #MSCCP (502a, 502b)). The transformation matrix T is constructed by a generalized pseudo-inverse method based on singular value decomposition (SVD), where:

[0090] T=RxPINV(D)

[0091] T=RD+(least squares solution of RD-T)

[0092] T = RD + = R(DTD) - 1DT.

[0093] If matrix R contains the spectral reflectance coefficients of the calibration samples, then PINV(D) is the pseudo-inverse function, and matrix D is the corresponding camera signal of the calibration samples.

[0094] The predicted spectral reflectance coefficients R can then be calculated using matrix multiplication for both calibration (stage 1) and validation (stage 2) subjects (described below).

[0095] R=TxD

[0096] This approach in stage 2 may have the advantage that the spectral sensitivity of the camera does not need to be known in advance.

[0097] The transformation matrix is ​​the part that uses the detected intensities to form at least one spectrum for each pixel.

[0098] Stage 2 shows that a target object, shown as a hand, is imaged with low-quality imaging 514a (or signal averaging) and / or high-quality imaging 514b in a lighting sequence of three illuminations and images. The protocol can average the signals to ultimately increase the signal-to-noise ratio of the data. Again, the protocol can include acquiring one image using two LEDs illuminating the object, then acquiring a second image using two LEDs (e.g., a different combination of LEDs), and then acquiring a third image using a third illumination pattern of LEDs, such as white LEDs. The protocol then multiplies these three images as a matrix that is multiplied with the previously acquired transformation matrix to generate a multispectral reflectance data cube. This operation is repeated for all images desired to be converted to a hyperspectral data cube.

[0099] It should be appreciated that some of the subject matter of FIG. 5 maps onto stage 2 of FIG.

[0100] The Fourier transform is performed after spectral formation, as shown in Figure 5. The hyperspectral decomposition systems and methods disclosed in the following publications may be used: F. Cutrale, V. Trivedi, L.A. Trinh, C.L. Chiu, J.M. Choi, M.S. Artiga, and S.E. Fraser, "Hyperspectral phasor analysis enables multiplexed 5D in vivo imaging," Nature Methods 14, 149-152 (2017); and W. Shi, E.S. Koo, M. Kitano, H.J. Chiang, L.A. Trinh, G. Turcatel, B. Steventon, C. Arnesano, D. Warburton, S.E. Fraser, and F. Cutrale, "Pre-processing visualization of hyperspectral fluorescent data with spectrally encoded enhanced representations," Nature Communications 11, 726 (2020). The contents of these publications are incorporated herein in their entireties. Hyperspectral data may be rapidly analyzed via a GS plot of the Fourier coefficients of the normalized spectrum by using the following equation:

[0101] z(n)=G(n)+iS(n)

[0102] TIFF2026009883000002.tif19170

[0103] TIFF2026009883000003.tif19170

[0104] where λs and λf are the start and end wavelengths of the band of interest, respectively, and I is the intensity ω=2π / τs, where τs is the number of spectral channels (32 in this example), and n is the harmonic (for consistency, it is usually chosen to be either n=1 or 2).

[0105] Figure 6 provides a schematic diagram of the two-stage "pseudo-inverse" method used to reconstruct a multispectral reflectance datacube from a sequence of camera images. In Stage 1, color standards are imaged under a series of different lighting conditions to obtain their spectral reflectance coefficients, which are then used to solve the transformation matrix T; in Stage 2, the transformation matrix T is used to recover spectral information from a target object (e.g., a human hand) under the same lighting sequence. The multispectral reflectance datacube is then generated as described above.

[0106] Here, the present invention uses the reflectance of light from a target object by taking into account a transformation matrix. The sum of reflected light is a different type of signal that can be used in a hyperspectral system to generate a multispectral reflectance data cube. For example, a protocol obtains a multispectral reflectance data cube by forming at least one spectrum for each pixel using the detected intensity and wavelength of each target wave ("intensity spectrum") to generate the multispectral reflectance data cube. Thus, in FIG. 5, the spectrum formation step 402 provides a multispectral reflectance data cube.

[0107] 5 then operates from the multispectral reflectance datacube, such as by performing a Fourier transform 403. This processing allows for real-time data extraction.

[0108] Figures 7A-7C show an example involving the esophagus. Figure 7A shows the esophagus under normal white light illumination (e.g., a representation of a multispectral reflectance datacube). Figure 7B shows the esophagus in a false-color hyperspectral phasor image. Figure 7C shows the corresponding GS histogram (e.g., a phasor plot). Once the multispectral reflectance datacube from Stage 2 is obtained, a protocol can create real and imaginary components. This protocol uses a Fourier transform to convert the intensity spectrum of each pixel (e.g., the multispectral reflectance datacube) into a complex-valued function based on the intensity spectrum of each pixel, with each complex-valued function having at least one real component 404 and at least one imaginary component 405 (see, e.g., Figure 5). These are essentially real and imaginary images, which are then put into a histogram. Next, a number or encoding of the multispectral reflectance datacube is obtained. The process performs encoding of the spectral signals using harmonic and Fourier transforms. In this example, the protocol uses the second harmonic, resulting in two values ​​at one particular harmonic, one real and one imaginary. The protocol then creates a histogram, as shown in Figure 7C. The protocol applies a denoising filter at least once to both the real and imaginary components of each complex-valued function to generate a denoised real value and a denoised imaginary value for each pixel.

[0109] The protocol then creates a point on the phasor plane ("phasor point") for each pixel by plotting the denoised real value against the denoised imaginary value for each pixel, and maps the phasor point back to the corresponding pixel on the target image based on the geometric location of the phasor point on the phasor plane. The protocol then assigns an arbitrary color to the corresponding pixel based on the geometric location of the phasor point on the phasor plane. An unmixed color image of the target is generated based on the assigned arbitrary color, as shown in Figure 7B. Once the unmixed color image is acquired, the protocol displays the unmixed color image of the target on the display of the imaging system.

[0110] 8A-8C show an example involving a small intestine mimic. FIG. 8A shows the small intestine under normal white light illumination (e.g., a multispectral reflectance datacube representation). FIG. 8B shows the representation in a false-color hyperspectral phasor image. FIG. 8C shows the corresponding GS histogram (e.g., a phasor plot). FIG. 8B is obtained by processing as described in connection with FIG. 7B.

[0111] As shown in Figures 7A-7C and 8A-8C, the esophageal and small intestinal tissue mimics appear visually similar, and any discoloration may be attributed to shadows or uneven lighting by an untrained medical assistant. However, when the images were hyperspectrally processed and graphed on a GS chart, the spectral distributions were clearly different. Therefore, this difference can be programmed into an automated wavelength-recognition software algorithm (e.g., looking for spectrally well-defined changes in "color") to rapidly identify areas of the esophagus where dysplasia is occurring. This would therefore facilitate rapid screening and early detection of Barrett's esophagus. The significance of detecting Barrett's esophagus in this case is that it is an early indicator of the risk of esophageal adenocarcinoma.

[0112] With reference to Figure 5, it should be appreciated that steps may be performed in a different order. For example, denoising filter 406 may be applied between 401 and 402 or between 402 and 403. Thus, Figure 5 may be modified accordingly.

[0113] In some embodiments, the hyperspectral processing system is configured to: convert the intensity spectrum of each pixel formed using a Fourier transform into a complex-valued function based on the intensity spectrum of each pixel, each complex-valued function having at least one real component and at least one imaginary component; form a phasor point on a phasor plane for each pixel by plotting the real values ​​against the imaginary values ​​of each pixel; map the phasor point back to a corresponding pixel on the target image based on the geometric location of the phasor point on the phasor plane; assign an arbitrary color to the corresponding pixel based on the geometric location of the phasor point on the phasor plane; and generate an unmixed color image of the target based on the assigned arbitrary color.

[0114] In some embodiments, the hyperspectral processing system further includes at least one of applying a denoising filter to both the real and imaginary components of each complex-valued function at least once to generate denoised real and denoised imaginary values ​​for each pixel, where the denoised real and denoised imaginary values ​​are used to form a phasor point on the phasor plane for each pixel; applying the denoising filter to the target image before forming the intensity spectrum; and applying the denoising filter before the formed intensity spectrum for each pixel is transformed.

[0115] In some embodiments, the hyperspectral processing system forms a target image of the target using the detected target electromagnetic radiation, the target image comprising at least two pixels, each pixel corresponding to a physical point on the target;

[0116] The method includes using a Fourier transform to generate complex-valued functions, each complex-valued function having at least one real component and at least one imaginary component, forming one phasor point on a phasor plane for each pixel by plotting the real values ​​against the imaginary values ​​of each pixel, mapping the phasor points back to corresponding pixels on the target image based on the geometric positions of the phasor points on the phasor plane, assigning arbitrary colors to the corresponding pixels based on the geometric positions of the phasor points on the phasor plane, and generating an unmixed color image of the target based on the assigned arbitrary colors.

[0117] In some embodiments, the method may include obtaining the multispectral reflectance datacube using a machine learning protocol.

[0118] For example, the system may be used as follows: A patient may sit in a chair during an annual physical examination. A medical assistant or nurse may spray a topical pain reliever on the back of the patient's throat to suppress the gag reflex and minimize discomfort. A tethered capsule may be administered with a coil tether attached to the capsule, allowing the patient to swallow the capsule by sipping water. To minimize air bubble formation in the esophagus, the water may be mixed with a common, digestible surfactant. Gravity can unwind the tether, allowing the capsule to reach the gastroesophageal (GE) sphincter within 3 to 5 seconds. The medical assistant manually begins to retract the capsule from the GE sphincter at the top of the stomach and can view an external display screen (e.g., LCD) that may display a real-time image of the throat tissue from the capsule. If the medical assistant notices an abnormal configuration of the esophageal lining, the medical assistant may annotate it on the video in relation to the distance markings on the tether. Depending on the configuration of the interchangeable lenses, low-magnification lenses with a wider field of view (FOV) can clearly show esophageal changes associated with gastroesophageal reflux disease (GERD) and Barrett's esophagus. Also, when using specialized lenses with larger FOVs or higher magnifications, a medical assistant can manually rotate and position the capsule near the esophageal wall to examine suspicious areas of early EC. Noncircular tether shapes can be used for rotation. For example, a square tether cross-sectional profile can be rotated 90 degrees to each side of the tether. Depending on the number of lenses and imaging sensors, multiple views (front, side, and / or back) that clearly show the tissue can be acquired in parallel to visualize esophageal changes associated with gastroesophageal reflux disease (GERD) and Barrett's esophagus. This entire screening process can take less than five minutes per patient. After withdrawing the capsule through the tether, the recorded HD video images can be reviewed by an expert physician or by automated machine vision software that utilizes hyperspectral analysis methods for detailed analysis of each video frame.

[0119] In another example, a drone can fly over a natural environment and lower a capsule for imaging and hyperspectral processing, as described herein. Ground vehicles can reach areas where people cannot fit through small paths and then perform imaging and hyperspectral processing. This can be useful for exploring natural sinkholes as well as tombs or other man-made structures. A small crane can be attached to a well and lower a capsule via a tether, which can then image the well's walls, bottom, or other objects or contents.

[0120] Those skilled in the art will understand that, for the processes and methods disclosed herein, the functions performed in the processes and methods may be implemented in a different order. Furthermore, the outlined steps and operations are provided only as examples, and some of the steps and operations may be optional, combined into fewer steps and operations, or expanded into additional steps and operations without detracting from the essence of the disclosed embodiments.

[0121] In one embodiment, the method may include aspects performed on a computing system. Accordingly, the computing system may include a memory device having computer-executable instructions for performing the method. The computer-executable instructions may be part of a computer program product including one or more algorithms for performing any of the methods recited in any one of the claims.

[0122] In one embodiment, any of the operations, processes, or methods described herein may be performed or adapted to be performed in response to the execution of computer-readable instructions stored on a computer-readable medium and executable by one or more processors. The computer-readable instructions may be executed by processors in a wide range of computing systems, from desktop computing systems, portable computing systems, tablet computing systems, handheld computing systems, as well as network elements, and / or any other computing device. The computer-readable medium may be non-transitory. The computer-readable medium is a physical medium on which computer-readable instructions are stored such that they are physically readable from the physical medium by a computer / processor.

[0123] There are a variety of vehicles (e.g., hardware, software, and / or firmware) that may implement the processes and / or systems and / or other technologies described herein, and the preferred vehicle may vary depending on the context in which the processes and / or systems and / or other technologies are deployed. For example, if an implementer determines that speed and accuracy are paramount, the implementer may select a primarily hardware and / or firmware vehicle; if flexibility is paramount, the implementer may select a primarily software implementation. Or, still alternatively, the implementer may select some combination of hardware, software, and / or firmware.

[0124] The various operations described herein can be implemented, individually and / or collectively, by a wide range of hardware, software, firmware, or virtually any combination thereof. In one embodiment, some portions of the subject matter described herein may be implemented via an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a digital signal processor (DSP), or other integrated format. However, some aspects of the embodiments disclosed herein may be equivalently implemented in an integrated circuit, in whole or in part, as one or more computer programs running on one or more computers (e.g., as one or more programs running on one or more computer systems), as one or more programs running on one or more processors (e.g., as one or more programs running on one or more microprocessors), as firmware, or virtually any combination thereof; designing circuitry and / or writing code for software and / or firmware is possible in light of this disclosure. Furthermore, the mechanisms of the subject matter described herein can be distributed as a program product in various forms, and exemplary embodiments of the subject matter described herein apply regardless of the particular type of signal-bearing medium used to actually effect the distribution. Examples of physical signal-bearing media include, but are not limited to, recordable-type media such as floppy disks, hard disk drives (HDDs), compact disks (CDs), digital versatile disks (DVDs), digital tape, computer memory, or any other physical medium that is a non-transitory, transmission medium. Examples of physical media having computer-readable instructions omit transitory or transmission-type media such as digital and / or analog communications media (e.g., fiber optic cables, wave guides, wired communications links, wireless communications links, etc.).

[0125] It is common to describe devices and / or processes in the manner described herein and then use engineering practices to integrate such described devices and / or processes into a data processing system. That is, at least a portion of the devices and / or processes described herein can be integrated into a data processing system with a reasonable amount of experimentation. A typical data processing system generally includes one or more of the following: a system unit housing; a video display device; memory, such as volatile and non-volatile memory; a processor, such as a microprocessor and a digital signal processor; computational entities, such as an operating system, drivers, a graphical user interface, and application programs; one or more interaction devices, such as a touchpad or screen; and / or a control system, including feedback loops and control motors (e.g., feedback for sensing position and / or velocity, controlling motors to move and / or adjust components and / or quantities). A typical data processing system may be implemented utilizing any suitable commercially available components, such as those commonly found in data computing / communications and / or network computing / communications systems.

[0126] The subject matter described herein may depict different components contained within or connected to different other components. Such depicted architectures are merely exemplary; in fact, many other architectures that achieve the same functionality may be implemented. In a conceptual sense, any arrangement of components to achieve the same functionality is effectively “associated” such that the desired functionality is achieved. Thus, any two components herein that combine to achieve a particular function may be considered to be “associated” with each other such that the desired functionality is achieved, regardless of the architecture or intermediate components. Similarly, any two components so associated may also be considered to be “operably connected” or “operably coupled” to each other to achieve the desired functionality, and any two components that can be so associated may also be considered to be “operably coupled” to each other to achieve the desired functionality. Specific examples of operably coupleable components include, but are not limited to, physically matable and / or physically interacting components and / or wirelessly interacting and / or wirelessly interacting components and / or logically interacting and / or logically interacting components.

[0127] 9 illustrates an exemplary computing device 600 (e.g., a computer) that may be configured in some embodiments to perform the methods (or portions thereof) described herein. In a very basic configuration 602, the computing device 600 generally includes one or more processors 604 and a system memory 606. A memory bus 608 may be used for communication between the processors 604 and the system memory 606.

[0128] Depending on the desired configuration, the processor 604 may be of any type, including but not limited to a microprocessor (μP), a microcontroller (μC), a digital signal processor (DSP), or any combination thereof. The processor 604 may include one or more levels of cache, such as a level 1 cache 610 and a level 2 cache 612, a processor core 614, and registers 616. An exemplary processor core 614 may include an arithmetic logic unit (ALU), a floating point unit (FPU), a digital signal processing core (DSP Core), or any combination thereof. An exemplary memory controller 618 may also be used with the processor 604, or in some implementations, the memory controller 618 may be an internal part of the processor 604.

[0129] Depending on the desired configuration, the system memory 606 may be of any type, including, but not limited to, volatile memory (such as RAM), non-volatile memory (e.g., ROM, flash memory, etc.), or any combination thereof. The system memory 606 may include an operating system 620, one or more applications 622, and program data 624. The applications 622 may include a determination application 626 configured to perform the operations described herein, including those described with respect to the methods described herein. The determination application 626 may obtain data such as pressure, flow rate, and / or temperature and then determine changes to the system to alter the pressure, flow rate, and / or temperature.

[0130] Computing device 600 may have additional features or functionality and additional interfaces to facilitate communication between basic configuration 602 and any necessary devices and interfaces. For example, a bus / interface controller 630 may be used to facilitate communication between basic configuration 602 and one or more data storage devices 632 via a storage interface bus 634. Data storage device 632 may be a removable storage device 636, a non-removable storage device 638, or a combination thereof. Examples of removable and non-removable storage devices include, for example, magnetic disk devices such as floppy disk drives and hard disk drives (HDDs), optical disk drives such as compact disk (CD) drives or digital versatile disk (DVD) drives, solid-state drives (SSDs), and tape drives. Exemplary computer storage media may include volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information, such as computer-readable instructions, data structures, program modules, or other data.

[0131] System memory 606, removable storage device 636, and non-removable storage device 638 are examples of computer storage media including, but not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVDs) or other optical storage devices, magnetic cassettes, magnetic tape, magnetic disk storage devices or other magnetic storage devices, or any other medium that can be used to store desired information and that can be accessed by computing device 600. Any such computer storage media may be part of computing device 600.

[0132] The computing device 600 may also include an interface bus 640 to facilitate communication from various interface devices (e.g., output device(s) 642, peripheral interface 644, and communication device(s) 646) to the basic configuration 602 via a bus / interface controller 630. Exemplary output device(s) 642 include a graphics processing unit 648 and an audio processing unit 650, which may be configured to communicate with various external devices, such as a display or speakers, via one or more A / V ports 652. Exemplary peripheral interface 644 includes a serial interface controller 654 or a parallel interface controller 656, which may be configured to communicate with external devices, such as input devices (e.g., keyboard, mouse, pen, voice input device, touch input device, etc.) or other peripherals (e.g., printer, scanner, etc.) via one or more I / O ports 658. Exemplary communication device 646 includes a network controller 660, which may be configured to facilitate communication with one or more other computing devices 662 over a network communication link via one or more communication ports 664.

[0133] A network communication link may be an example of a communication medium. Communication media may typically be embodied by computer-readable instructions, data structures, program modules, or other data in a modulated data signal, such as a carrier wave or other transmission mechanism, and may include any information delivery media. A "modulated data signal" may be a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. By way of example, and not limitation, communication media may include wired media, such as a wired network or direct-wired connection, and wireless media, such as acoustic, radio frequency (RF), microwave, infrared (IR), and other wireless media. As used herein, the term computer-readable media may include both storage media and communication media.

[0134] Computing device 600 may be implemented as part of a small form factor portable (or mobile) electronic device such as a mobile phone, a personal data assistant (PDA), a personal media player device, a wireless web watch device, a personal headset device, an application-specific device, or a hybrid device including any of the above functionality. Computing device 600 may also be implemented as a personal computer, including both laptop and non-laptop computer configurations. Computing device 600 may also be any type of network computing device. Computing device 600 may also be an automated system as described herein.

[0135] The embodiments described herein may involve the use of special purpose or general purpose computers containing various computer hardware or software modules.

[0136] Embodiments within the scope of the present invention also include computer-readable media for carrying or storing computer-executable instructions or data structures. Such computer-readable media may be any available media that can be accessed by a general-purpose or special-purpose computer. By way of example, and not limitation, such computer-readable media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage devices, magnetic disk storage devices or other magnetic storage devices, or any other medium that can be used to carry or store desired program code means in the form of computer-executable instructions or data structures and that can be accessed by a general-purpose or special-purpose computer. When information is transferred or provided over a network or another communications connection (either hardwired, wireless, or a combination of hardwired or wireless) to a computer, the computer properly views the connection as a computer-readable medium. Thus, such a connection is properly termed a computer-readable medium. Combinations of the above should also be included within the scope of computer-readable media.

[0137] Computer-executable instructions comprise, for example, instructions and data that cause a general-purpose computer, special-purpose computer, or special-purpose processing device to perform a particular function or group of functions. Although the subject matter has been described in language specific to structural features and / or methodological acts, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. The specific features and acts described above are disclosed as example forms that satisfy the claims that follow.

[0138] In some embodiments, a computer program product can include a non-transitory tangible memory device having computer-executable instructions that, when executed by a processor, cause the method to include providing a dataset having object data for objects and condition data for conditions; processing the object data of the dataset with an object encoder to obtain potential object data and potential object condition data; processing the condition data of the dataset to obtain potential condition data and potential condition subject data with a condition encoder; processing the potential object data and potential object condition data to obtain generated object data with an object decoder; processing the potential condition data and the potential condition subject data to obtain generated condition data with a condition decoder; comparing the potential object condition data with the potential condition data to determine differences; processing the potential object data and the potential condition data and one of the potential object condition data or the potential condition object data with an identifier to obtain an identifier value; selecting selected objects from the generated object data based on the generated object data, the generated condition data, and differences between the potential object condition data and the potential condition object data; and providing the selected objects with a recommendation for verification of the physical form of the objects in a report. The non-transitory tangible memory device may also have other executable instructions for any of the methods or method steps described herein. The instructions may also be instructions for performing non-computational tasks, such as synthesis of a molecule and / or experimental protocols for validating a molecule. Other executable instructions may also be provided.

[0139] The present disclosure is not limited to the specific embodiments described herein, but are intended as illustrations of various aspects. As will be apparent to those skilled in the art, many modifications and variations can be made without departing from the spirit and scope thereof. Functionally equivalent methods and apparatuses within the scope of the present disclosure, in addition to those enumerated herein, will be apparent to those skilled in the art from the foregoing description. Such modifications and variations are intended to be included within the scope of the appended claims. The present disclosure is to be limited only by the terms of the appended claims, along with the full scope of equivalents to which such claims are entitled. It is to be understood that the present disclosure is not limited to particular methods, reagents, compounds, compositions, or biological systems, which can, of course, vary. It is also to be understood that the terminology used herein is for the purpose of describing particular embodiments only, and is not intended to be limiting.

[0140] With respect to the use of virtually any plural and / or singular term herein, those skilled in the art can convert from plural to singular and / or from singular to plural as appropriate to the context and / or application. Various singular / plural permutations may be expressly set forth herein for clarity.

[0141] In general, it will be understood by those skilled in the art that the terms used in this specification, and particularly in the appended claims (e.g., the body of the appended claims), are generally intended as "open" terms (e.g., the term "including" should be interpreted as "including but not limited to," the term "having" should be interpreted as "having at least," the term "includes" should be interpreted as "includes but not limited to," etc.). Where a specific number of introduced claim recitations are intended, such intention will be explicitly set forth in the claim; in the absence of such recitation, it will be further understood by those skilled in the art that no such intention exists. For example, to aid in understanding, the following appended claims may include the use of the introductory phrases "at least one" and "one or more" to introduce claim recitations. However, the use of such phrases should not be interpreted as meaning that the introduction of a claim recitation with the indefinite article "a" or "a" limits any particular claim that includes such an introduced claim recitation to embodiments that include only one such recitation, even if the same claim includes the introductory phrase "one or more" or "at least one" and an indefinite article such as "a" or "a" (e.g., "a" and / or "an" should be interpreted as meaning "at least one" or "one or more"). The same applies to the use of definite articles used to introduce claim recitations. Moreover, even when a specific number of introduced claim recitations is explicitly recited, those skilled in the art will recognize that such recitation should be interpreted as meaning at least the recited number (e.g., the bare recitation of "two recitations," without any other modifiers, means at least two recitations, or two or more recitations).Furthermore, when a convention similar to "at least one of A, B, and C, etc." is used, generally such a configuration is intended in the sense that one of ordinary skill in the art would understand the convention (e.g., "a system having at least one of A, B, and C" includes, but is not limited to, systems having A only, B only, C only, A and B together, A and C together, B and C together, and / or A, B, and C together, etc.). When a convention similar to "at least one of A, B, or C, etc." is used, generally such a configuration is intended in the sense that one of ordinary skill in the art would understand the convention (e.g., "a system having at least one of A, B, or C" includes, but is not limited to, systems having A only, B only, C only, A and B together, A and C together, B and C together, and / or A, B, and C together, etc.). It will be further understood by those skilled in the art that virtually any disjunctive word and / or phrase presenting two or more alternative terms, whether in the specification, claims, or drawings, should be understood to contemplate the possibility of including one of the terms, either of the terms, or both terms. For example, the phrase "A or B" is understood to include the possibilities of "A" or "B" or "A and B."

[0142] Furthermore, where features or aspects of the present disclosure are described in terms of a Markush group, those skilled in the art will recognize that the present disclosure is also thereby described in terms of any individual member or subgroup of members of the Markush group.

[0143] As will be understood by those skilled in the art, for any and all purposes, e.g., with respect to providing a written description, all ranges disclosed herein also encompass any and all possible subranges and combinations of subranges. Any recited range can be readily recognized as fully descriptive and permitting division of that same range into at least equal halves, thirds, quarters, fifths, tenths, etc. As a non-limiting example, each range discussed herein can be readily divided into a lower third, middle third, upper third, etc. Also, as will be understood by those skilled in the art, all terms such as "up to," "at least," etc., are inclusive of the recited numbers and refer to ranges that can be subsequently divided into subranges as discussed above. Finally, as will be understood by those skilled in the art, ranges include individual members. Thus, for example, a group having 1 to 3 cells refers to a group having 1, 2, or 3 cells. Similarly, a group having 1 to 5 cells refers to a group having 1, 2, 3, 4, or 5 cells, etc.

[0144] From the foregoing, it will be understood that various embodiments of the present disclosure have been described herein for purposes of illustration, and that various modifications can be made without departing from the scope and spirit of the present disclosure. Accordingly, the various embodiments disclosed herein are not intended to be limiting, with the true scope and spirit being indicated by the following claims.

[0145] All references cited herein are incorporated herein by specific reference in their entirety. Publication:PCT / US2015 / 025468, F.Cutrale, V.Trivedi, LATrinh, CLChiu, JMChoi, MSArtiga, SEFraser, etc. Hyperspectral phasor analysis enables multiplexed 5D in vivo imaging (Nature Methods). 14,149-152(2017)) W. Shi, ESKoo, M. Kitano, HJChiang, LATrinh, G. Turcatel, B.S. Steventon, C. Arnesano, D. Warburton, SEFraser, F. Cutrale, and Pre-processing Visualization of Hyperspectral Fluorescent Data with Spectrally Encoded Enhanced Representations」(Nature Communications 11,726(2020))

Claims

1. 1. A capsule hyperspectral system, comprising: An imaging capsule, an illumination system having a plurality of light emitters configured to emit a plurality of different illumination lights from the imaging capsule; a hyperspectral imaging system having at least one imaging sensor, the illumination system and the hyperspectral imaging system cooperatively configured to illuminate an object with a sequence of different illumination lights and image the object during each of the different illumination lights in the sequence; 1. A capsule hyperspectral system comprising an imaging capsule comprising: a hyperspectral processing system having at least one processor, the hyperspectral processing system operatively coupled to the hyperspectral imaging system and configured to receive images of the object from the hyperspectral imaging system and generate a multispectral reflectance data cube of the object from the received images of the object.

2. The capsule hyperspectral system of claim 1 , further comprising a tether having a capsule end coupled to the imaging capsule and a system end coupled to the hyperspectral processing system.

3. The capsule hyperspectral system of claim 2 , wherein the tether is communicatively connected to the hyperspectral imaging system and the hyperspectral processing system to pass data therebetween.

4. 3. The capsule hyperspectral system of claim 2, further comprising a semi-rigid connection between the tether and the imaging capsule, the semi-rigid connection configured to withstand manual manipulation for positioning the capsule without detaching the tether from the imaging capsule.

5. The capsule hyperspectral system of claim 2 , wherein the tether has a non-circular cross-sectional profile.

6. The capsule hyperspectral system of claim 2 , wherein the tether includes an outer surface having markings thereon, the markings configured to indicate a distance from the imaging capsule when deployed.

7. The capsule hyperspectral system of claim 1 , wherein the illumination system comprises at least three LEDs having at least three different color bands.

8. 8. The capsule hyperspectral system of claim 7, wherein at least one LED is a white light LED.

9. The capsule hyperspectral system of claim 8 , wherein at least two of the LEDs are colored LEDs having different color bands.

10. The capsule hyperspectral system of claim 7 , wherein the illumination system comprises a uniformly spaced array of LEDs.

11. 11. The capsule hyperspectral system of claim 10, wherein the illumination system comprises at least six LEDs, the at least six LEDs including at least two white light LEDs and at least four colored LEDs having at least two different color bands.

12. 8. The capsule hyperspectral system of claim 7, wherein the emission wavelength of each LED is selected to visually identify and distinguish between white and / or pinkish surfaces on healthy tissue and red surfaces on non-healthy tissue.

13. The capsule hyperspectral system of claim 1 , wherein the at least one imaging sensor and the plurality of light emitters are disposed on a plate and are oriented in the same direction.

14. The capsule hyperspectral system of claim 1 , wherein the hyperspectral imaging system further comprises a lens system, the lens system being a fixed lens system, a detachable lens system, a replaceable lens system, or an interchangeable lens system.

15. 15. The capsule hyperspectral system of claim 14, wherein the lens system has at least one lens with a field of view (FOV) ranging from at least about 90 degrees to less than about 360 degrees.

16. 15. The capsule hyperspectral system of claim 14, wherein the lens system has at least one lens with a field of view (FOV) ranging from about 120 degrees to about 180 degrees.

17. The capsule hyperspectral system of claim 14 , wherein the hyperspectral imaging system comprises an optical lens, an optical filter, a dispersive optic, or a combination thereof.

18. The capsule hyperspectral system of claim 14 , wherein the hyperspectral imaging system comprises a first optical lens, a second optical lens, and a dichroic mirror / beam splitter.

19. The capsule hyperspectral system of claim 14 , wherein the hyperspectral imaging system comprises an optical lens, a dispersive optics system, and the at least one imaging sensor is a photodetector array.

20. The capsule hyperspectral system of claim 1 , wherein the at least one imaging sensor is positioned eccentrically relative to a central axis of the imaging capsule.

21. 21. The capsule hyperspectral system of claim 20, wherein the at least one imaging sensor is positioned between about 10 degrees and about 35 degrees from the central axis.

22. The capsule hyperspectral system of claim 1 , wherein the hyperspectral imaging system further comprises an optical filtering system disposed between the optical entrance of the capsule and the at least one imaging sensor.

23. 23. The capsule hyperspectral system of claim 22, wherein the optical filter system includes a noise reduction filter.

24. 24. The capsule hyperspectral system of claim 23, wherein the noise reduction filter comprises a median filter.

25. The capsule hyperspectral system of claim 2 , wherein the imaging capsule comprises a capsule cover, the capsule cover having a textured outer surface.

26. 26. The capsule hyperspectral system of claim 25, wherein the texture includes at least one dimple, the at least one dimple configured to allow a patient to easily swallow the tethered imaging capsule.

27. 26. The capsule hyperspectral system of claim 25, wherein the texture includes at least one channel, the at least one channel configured to allow a patient to easily swallow the tethered imaging capsule.

28. 10. The capsule hyperspectral system of claim 1, further comprising a display operatively coupled to the hyperspectral processing system, wherein the illumination system is calibrated to display the object imaged by the at least one imaging sensor on the display.

29. The capsule hyperspectral system of claim 1 , wherein the capsule includes a control system configured to control the sequence of different illumination lights and imaging of the at least one imaging sensor.

30. 2. The capsule hyperspectral system of claim 1, wherein the hyperspectral processing system includes a control system, a memory, and a display, the control system configured to generate the multispectral reflectance datacube, store the multispectral reflectance datacube in the memory, and display the multispectral reflectance datacube or an image representation thereof on the display.

31. the at least one photodetector detecting target electromagnetic radiation absorbed, transmitted, refracted, reflected, and / or emitted by at least one physical point on the target, the target radiation comprising at least two target waves, each target wave having an intensity and a unique wavelength; Detect the intensity and wavelength of each target wave, 10. The capsule hyperspectral system of claim 1, configured to transmit the detected target electromagnetic radiation and the detected intensity and wavelength of each target wave to a hyperspectral processing system.

32. the hyperspectral processing system comprises: forming a target image of the target using the detected target electromagnetic radiation, the target image comprising at least two pixels, each pixel corresponding to a physical point on the target; forming at least one intensity spectrum for each pixel using the intensity and wavelength of each detected target wave; 32. The capsule hyperspectral system of claim 31, configured to generate a multispectral reflectance data cube from at least one intensity spectrum for each pixel.

33. the hyperspectral processing system comprises: using a Fourier transform to transform the formed intensity spectrum of each pixel into a complex-valued function based on the intensity spectrum of each pixel, each complex-valued function having at least one real component and at least one imaginary component; applying a denoising filter at least once to both the real and imaginary components of each complex-valued function to produce a denoised real value and a denoised imaginary value for each pixel; forming one phasor point on the phasor plane for each pixel by plotting the denoised real value against the denoised imaginary value for each pixel; mapping the phasor points back to corresponding pixels on the target image based on the geometric locations of the phasor points on the phasor plane; assigning a random color to a corresponding pixel based on the geometric location of the phasor point on the phasor plane; 33. The capsule hyperspectral system of claim 32, configured to generate an unmixed color image of the object based on the assigned arbitrary colors.

34. the hyperspectral processing system comprises:

34. The capsule hyperspectral system of claim 33, configured to display the unmixed color image of the object on a display of the hyperspectral processing system.

35. 34. The capsule hyperspectral system of claim 33, wherein the hyperspectral processing system uses only the first harmonic or only the second harmonic of the Fourier transform to generate the unmixed color image of the object.

36. 34. The capsule hyperspectral system of claim 33, wherein the hyperspectral processing system uses only the first and second harmonics of the Fourier transform to generate the unmixed color image of the object.

37. the target radiation is The fluorescence wavelength, 34. The capsule hyperspectral system of claim 33, comprising at least one of: at least four wavelengths;

38. 34. The capsule hyperspectral system of claim 33, wherein the hyperspectral processing system is configured to form the unmixed color image of the object with a signal-to-noise ratio for the at least one spectrum in the range of 1.2 to 50.

39. 34. The capsule hyperspectral system of claim 33, wherein the hyperspectral processing system is configured to form the unmixed color image of the object with a signal-to-noise ratio for the at least one spectrum in the range of 2 to 50.

40. 34. The capsule hyperspectral system of claim 33, wherein the hyperspectral processing system is configured to use a reference material to assign an arbitrary color to each pixel.

41. 34. The capsule hyperspectral system of claim 33, wherein the hyperspectral processing system is configured to assign an arbitrary color to each pixel using a reference material, and the unmixed color image of the reference material is generated prior to generating the unmixed color image of the object.

42. 34. The capsule hyperspectral system of claim 33, wherein the hyperspectral processing system uses a reference material to assign an arbitrary color to each pixel, the unmixed color image of the reference material is generated prior to generating the unmixed color image of the subject, and the reference material includes a physical structure, a chemical molecule, a biological molecule, a physical change and / or a biological change caused by a disease, or any combination thereof.

43. the illumination system and the hyperspectral imaging system, illuminating a reference object with a first illumination light; imaging the reference object during a first illumination light; illuminating the reference object with a second illumination light different from the first illumination light; imaging the reference object during the second illumination light; illuminating the reference object with a third illumination light different from the first illumination light and the second illumination light; The capsule hyperspectral system of claim 1 , cooperatively configured to image the reference object during the third illumination light.

44. 44. The capsule hyperspectral system of claim 43, wherein the third illumination light is white light illumination.

45. 44. The capsule hyperspectral system of claim 43, wherein the reference object comprises a color reference image.

46. 44. The capsule hyperspectral system of claim 43, wherein the first illumination light, the second illumination light, and the third illumination light each include illumination by at least two LEDs.

47. the hyperspectral processing system comprises: Obtain the spectrum of each pixel in the image, 44. The capsule hyperspectral system of claim 43, configured to generate a transformation matrix from the spectrum of each pixel.

48. the illumination system and the hyperspectral imaging system, illuminating the object with the first illumination light; imaging the object during the first illumination light; illuminating the object with the second illumination light; imaging the object during the second illumination light; illuminating the object with the third illumination light; 48. The capsule hyperspectral system of claim 47, cooperatively configured to image the reference object during the third illumination light.

49. 49. The capsule hyperspectral system of claim 48, wherein the hyperspectral processing system is configured to generate the multispectral reflectance data cube from the transformation matrix and images of the object acquired during the first illumination light, the second illumination light, and the third illumination light.

50. 50. The capsule hyperspectral system of claim 49, wherein the multispectral reflectance data cube is obtained from a pseudo-inverse method using the image of the object.

51. The capsule hyperspectral system of claim 1 , further comprising an irrigation system having an irrigation source and an irrigation conduit, the irrigation conduit having an opening in the capsule.

52. The capsule hyperspectral system of claim 1 , wherein the tether includes an irrigation conduit coupled to an irrigation system, the irrigation conduit having an opening in the capsule.

53. the hyperspectral processing system comprises: using a Fourier transform to transform the formed intensity spectrum of each pixel into a complex-valued function based on the intensity spectrum of each pixel, each complex-valued function having at least one real component and at least one imaginary component; forming one phasor point on the phasor plane for each pixel by plotting the real values ​​against the imaginary values ​​of each pixel; mapping the phasor points back to corresponding pixels on the target image based on their geometric locations on the phasor plane; assigning a color to the corresponding pixel based on the geometric location of the phasor point on the phasor plane; 33. The capsule hyperspectral system of claim 32, configured to generate an unmixed color image of the object based on the assigned arbitrary colors.

54. the hyperspectral processing system comprises: applying a denoising filter at least once to both the real and imaginary components of each complex-valued function to generate denoised real and denoised imaginary values ​​for each pixel, the denoised real and denoised imaginary values ​​being used to form the one phasor point on the phasor plane for each pixel; applying a denoising filter to the target image before forming the intensity spectrum; and applying a noise reduction filter before the intensity spectrum of each pixel formed is transformed.

55. the hyperspectral processing system comprises: using the detected object electromagnetic radiation to form an object image of the object, the object image comprising at least two pixels, each pixel corresponding to a physical point on the object; using a Fourier transform to generate complex-valued functions, each complex-valued function having at least one real component and at least one imaginary component; forming one phasor point on the phasor plane for each pixel by plotting the real values ​​against the imaginary values ​​for each pixel; mapping the phasor points back to corresponding pixels on the target image based on the geometric locations of the phasor points on the phasor plane; assigning a color to the corresponding pixel based on the geometric location of the phasor point on the phasor plane; 32. The capsule hyperspectral system of claim 31, configured to generate an unmixed color image of the object based on the assigned arbitrary colors.

56. 32. The capsule hyperspectral system of claim 31, further comprising obtaining the multispectral reflectance datacube using a machine learning protocol.

57. illuminating the object with an illumination system of the imaging capsule; receiving detected object electromagnetic radiation from at least one imaging sensor of the imaging capsule that has been absorbed, transmitted, refracted, reflected, and / or emitted by at least one physical point on the object, the object radiation comprising at least two object waves, each object wave having an intensity and a unique wavelength; transmitting the detected target electromagnetic radiation and the intensity and wavelength of each target wave detected from the imaging capsule to a hyperspectral processing system.

58. the hyperspectral processing system comprises: forming a target image of the target using the detected target electromagnetic radiation, the target image comprising at least two pixels, each pixel corresponding to a physical point on the target; generating at least one intensity spectrum for each pixel using the intensity and wavelength of each detected target wave; 58. The computer method of claim 57, further comprising: generating the multi-spectral reflectance data cube from the at least one intensity spectrum for each pixel.

59. the hyperspectral processing system comprises: using a Fourier transform to transform the formed intensity spectrum of each pixel into a complex-valued function based on the intensity spectrum of each pixel, each complex-valued function having at least one real component and at least one imaginary component; applying a denoising filter at least once to both the real and imaginary components of each complex-valued function to produce a denoised real value and a denoised imaginary value for each pixel; forming one phasor point on the phasor plane for each pixel by plotting the denoised real values ​​against the denoised imaginary values ​​for each pixel; mapping the phasor points back to corresponding pixels on the target image based on the geometric locations of the phasor points on the phasor plane; assigning an arbitrary color to the corresponding pixel based on the geometric location of the phasor point on the phasor plane; 59. The computer method of claim 58, further comprising generating an unmixed color image of the object based on the assigned arbitrary colors and performing.

60. 60. The computer method of claim 59, further comprising the hyperspectral processing system displaying the unmixed color image of the object on a display of the hyperspectral processing system.

61. 60. The computer method of claim 59, wherein the hyperspectral processing system uses only the first harmonic or only the second harmonic of the Fourier transform to generate the unmixed color image of the object.

62. 60. The computer method of claim 59, wherein the hyperspectral processing system uses only the first and second harmonics of the Fourier transform to generate the unmixed color image of the object.

63. the target radiation is The fluorescence wavelength, and at least four wavelengths.

64. 60. The computer method of claim 59, wherein the hyperspectral imaging system is configured to form the unmixed color image of the object with a signal-to-noise ratio in the at least one spectrum in the range of 1.2 to 50.

65. 60. The computer method of claim 59, wherein the hyperspectral imaging system forms the unmixed color image of the object with a signal-to-noise ratio in the at least one spectrum in the range of 2 to 50.

66. 60. The computer method of claim 59, wherein the hyperspectral processing system is configured to use a reference material to assign an arbitrary color to each pixel.

67. 60. The computer method of claim 59, wherein the hyperspectral processing system is configured to assign an arbitrary color to each pixel using a reference material, and the unmixed color image of the reference material is generated prior to generating the unmixed color image of the object.

68. 60. The computer method of claim 59, wherein the hyperspectral processing system assigns an arbitrary color to each pixel using a reference material, the unmixed color image of the reference material being generated prior to generating the unmixed color image of the object, and the reference material comprising a physical structure, a chemical molecule, a biological molecule, a physical and / or biological change caused by a disease, or any combination thereof.

69. the hyperspectral processing system comprises: using a Fourier transform to transform the formed intensity spectrum of each pixel into a complex-valued function based on the intensity spectrum of each pixel, each complex-valued function having at least one real component and at least one imaginary component; forming one phasor point on the phasor plane for each pixel by plotting the real values ​​against the imaginary values ​​for each pixel; mapping the phasor points back to corresponding pixels on the target image based on the geometric locations of the phasor points on the phasor plane; assigning a color to the corresponding pixel based on the geometric location of the phasor point on the phasor plane; 59. The computer method of claim 58, further comprising generating an unmixed color image of the object based on the assigned arbitrary colors.

70. the hyperspectral processing system comprises: applying a denoising filter at least once to both the real and imaginary components of each complex-valued function to generate denoised real and denoised imaginary values ​​for each pixel, the denoised real and denoised imaginary values ​​being used to form the one phasor point on the phasor plane for each pixel; applying a denoising filter to the target image before forming the intensity spectrum; and applying a denoising filter before the intensity spectrum of each formed pixel is transformed.

71. the hyperspectral processing system comprises: using the detected object electromagnetic radiation to form an object image of the object, the object image comprising at least two pixels, each pixel corresponding to a physical point on the object; using a Fourier transform to generate complex-valued functions, each complex-valued function having at least one real component and at least one imaginary component; forming one phasor point on the phasor plane for each pixel by plotting the real values ​​against the imaginary values ​​for each pixel; mapping the phasor points back to corresponding pixels on a target image based on the geometric locations of the phasor points on the phasor plane; assigning a color to the corresponding pixel based on the geometric location of the phasor point on the phasor plane; 59. The computer method of claim 58, further comprising generating an unmixed color image of the object based on the assigned arbitrary colors.

72. 59. The capsule hyperspectral system of claim 58, further comprising obtaining the multispectral reflectance datacube using a machine learning protocol.

73. illuminating a reference object with a first illumination light emitted from the imaging capsule; acquiring an image of the reference object during the first illumination; illuminating the reference object with a second illumination light emitted from the imaging capsule; acquiring an image of the reference object during the second illumination; illuminating the reference object with a third illumination light emitted from the imaging capsule; acquiring an image of the reference object during the third illumination light.

74. 74. The computer method of claim 73, wherein the third illumination is white light illumination.

75. 74. The computer method of claim 73, wherein the reference object comprises a color reference image.

76. 74. The computer method of claim 73, wherein the first illumination light, the second illumination light, and the third illumination light each comprise illumination by at least two LEDs.

77. obtaining a spectrum for each pixel of the image; 74. The computer method of claim 73, further comprising generating a transformation matrix from the spectrum of each pixel.

78. illuminating the object with the first illumination light; capturing an image of the object during the first illumination light; illuminating the object with the second illumination light; capturing an image of the object during the second illumination light; illuminating the object with the third illumination light; 78. The computer method of claim 77, further comprising: acquiring an image of the reference object during the third illumination light.

79. 80. The computer method of claim 78, wherein a hyperspectral processing system generates a multispectral reflectance datacube from images of the object acquired during the first illumination light, the second illumination light, and the third illumination light.

80. 80. The capsule hyperspectral system of claim 79, wherein the multispectral reflectance data cube is obtained from a pseudo-inverse method using the image of the object.

81. the hyperspectral processing system comprises: using a Fourier transform to transform the formed intensity spectrum of each pixel into a complex-valued function based on the intensity spectrum of each pixel, each complex-valued function having at least one real component and at least one imaginary component; forming one phasor point on the phasor plane for each pixel by plotting the real values ​​against the imaginary values ​​for each pixel; mapping the phasor points back to corresponding pixels on the target image based on the geometric locations of the phasor points on the phasor plane; assigning a color to the corresponding pixel based on the geometric location of the phasor point on the phasor plane; 80. The computer method of claim 79, comprising generating an unmixed color image of the object based on the assigned arbitrary colors.

82. the hyperspectral processing system comprises: applying a denoising filter at least once to both the real and imaginary components of each complex-valued function to generate denoised real and denoised imaginary values ​​for each pixel, the denoised real and denoised imaginary values ​​being used to form the one phasor point on the phasor plane for each pixel; applying a denoising filter to the target image before forming the intensity spectrum; and applying a denoising filter before the intensity spectrum of each formed pixel is transformed.

83. the hyperspectral processing system comprises: using the detected object electromagnetic radiation to form an object image of the object, the object image comprising at least two pixels, each pixel corresponding to a physical point on the object; using a Fourier transform to generate complex-valued functions, each complex-valued function having at least one real component and at least one imaginary component; forming one phasor point on the phasor plane for each pixel by plotting the real values ​​against the imaginary values ​​for each pixel; mapping the phasor points back to corresponding pixels on the target image based on the geometric locations of the phasor points on the phasor plane; assigning a color to the corresponding pixel based on the geometric location of the phasor point on the phasor plane; 80. The computer method of claim 79, comprising generating an unmixed color image of the object based on the assigned arbitrary colors.

84. 80. The capsule hyperspectral system of claim 79, further comprising obtaining said multispectral reflectance datacube using a machine learning protocol.

Images