Systems and methods for using SWIR detection in a vacuum cleaner

Abstract

A system and method for utilizing SWIR sensing in an autonomous vacuum cleaner includes a first illumination source for emitting radiation at a first wavelength within the SWIR towards a field of view (FOV), and a receiver that operates to acquire SWIR image data based on radiation reflected from elements disposed within the FOV and, based on the SWIR image data, determines the presence of an aqueous liquid within the FOV.

Description

Technical Field

[0001] The present disclosure generally relates to cleaning machines. More specifically, the present disclosure relates to automated cleaning machines having short range infra-red (SWIR) sensing capabilities.

[0002] Cross - reference to related applications This application claims priority to U.S. Provisional Patent Application No. 63 / 345,594, filed on May 25, 2022, the entire disclosure of which is incorporated herein by reference.

Background Art

[0003] Automated or semi - automated cleaning machines, such as robotic devices, are commonly used in domestic, public, and industrial environments. Such automated or semi - automated devices may include vacuum cleaners, lawn mowers, robotic mops, floor sweepers, window cleaners, and other devices.

[0004] In recent years, there have been rapid advancements in the field of autonomous or robotic vacuum cleaners, particularly in the areas of vacuum cleaners and floor sweepers, whose main purpose is to autonomously navigate a user's home while cleaning the floor and / or carpet. In order to navigate an area autonomously or with minimal external control, an autonomous device generally can utilize mapping, localization, object recognition, and path planning methods. Some of the methods utilized may include a vision system that can capture still or moving images of the surrounding area. However, such systems may suffer from low-light conditions and poor quality of the captured images, and may have significant problems in identifying certain obstacles or conditions such as splashes of liquid on the floor. There is a need for improved methods and systems to overcome the drawbacks of conventional autonomous vacuum cleaners that use vision systems operating in the visible spectrum.

Summary of the Invention

[0005] Some embodiments may include an autonomous cleaning system and a method for utilizing SWIR sensing in autonomous cleaning. The system includes a first illumination source for emitting radiation at a first wavelength within a short-wave infrared range (SWIR) towards a field of view (FOV), and a receiver for obtaining SWIR image data based on radiation reflected from elements disposed within the FOV and operating within the SWIR, and determining the presence of aqueous liquid within the FOV based on the SWIR image data.

[0006] Some embodiments include a second illumination source for emitting radiation at a second wavelength within the SWIR towards a target within the FOV, and determining spectral characteristics of the target based on absorption of the target at the first wavelength and absorption of the target at the second wavelength. Operating as

[0007] According to some embodiments of the present disclosure, the second light source is for emitting radiation towards the target by emitting radiation at the second wavelength in the SWIR towards a second FOV included within the first FOV.

[0008] According to some embodiments of the present disclosure, the second light source can include a low-power light source.

[0009] According to some embodiments of the present disclosure, the receiver can include one or more germanium (Ge) photodiodes (PDs).

[0010] According to some embodiments of the present disclosure, the first light source can include a laser source.

[0011] According to some embodiments of the present disclosure, the first wavelength can be between 1300 nm and 1400 nm.

[0012] According to some embodiments of the present disclosure, the second light source can include a light-emitting diode (LED).

[0013] According to some embodiments of the present disclosure, the second wavelength is between 800 nm and 1000 nm.

[0014] Some embodiments may further include a third light source for emitting radiation at a third wavelength in the SWIR towards the target within the FOV, and the processor determines spectral characteristics of the target based on absorption of the target at the first wavelength, absorption of the target at the second wavelength, and absorption of the target at the third wavelength. operates as

Brief Description of the Drawings

[0015] The subject matter regarded as the present disclosure is pointed out in detail and claimed explicitly in the concluding part of this specification. However, the present disclosure can be best understood by referring to the following detailed description together with the accompanying drawings, regarding both the arrangement of operations and the methods, along with its purpose, features, and advantages.

Figure 1

Figure 2A

Figure 2B

Figure 2C

Figure 3

Figure 4

Figure 5

[0016] It will be understood that, for the sake of simplicity and clarity, the elements shown in the figures are not necessarily drawn to scale. For example, the dimensions of some elements may be exaggerated relative to other elements for clarity. Further, reference numerals may be repeated among the figures to indicate corresponding or similar elements where appropriate.

Best Mode for Carrying Out the Invention

[0017] Those skilled in the art will recognize that the present disclosure may be embodied in other specific forms without departing from its spirit or essential characteristics. Accordingly, the foregoing embodiments are to be considered in all respects illustrative rather than limiting of the disclosure described herein. The scope of the present disclosure is, therefore, indicated by the appended claims rather than by the foregoing description, and all changes that come within the meaning and range of equivalents of the claims are, therefore, intended to be embraced therein.

[0018] In the following detailed description, numerous specific details are set forth in order to provide a thorough understanding of the present disclosure. However, it will be understood by those skilled in the art that the present disclosure may be practiced without these specific details. In other instances, well-known methods, procedures, and components have not been described in detail so as not to obscure the present disclosure. Some features or elements described with respect to one embodiment may be combined with features or elements described with respect to other embodiments. For the sake of clarity, discussion of the same or similar features or elements may not be repeated.

[0019] The embodiments described herein are not limited in this regard. For example, descriptions using terms such as "processing", "computing", "calculating", "determining", "establishing", "analyzing", "checking" refer to operations and / or conversions of data represented as physical (e.g., electronic) quantities in a computer's registers and / or memories into other data similarly represented as physical quantities in the computer's registers and / or memories, or to operations and / or processes related to a non-transitory storage medium that can store instructions for causing a processor to execute operations and / or processes, when executed by a computer, a computing platform, a computing system, or other electronic computing device, or a processor.

[0020] The embodiments described herein are not limited in this regard. However, the terms "plurality" and "a plurality" as used herein may include, for example, "multiple" or "two or more". The term "plurality" may be used throughout this specification to describe two or more components, devices, elements, units, parameters, and the like. As used herein, the term "set" may include one or more items.

[0021] Unless explicitly stated otherwise, embodiments of the methods described herein are not restricted to a particular order or sequence. Additionally, some of the embodiments of the methods described, or some of their elements, may occur or be executed simultaneously, at the same time, or concurrently.

[0022] The embodiments described herein can overcome the drawbacks of a general vision system implemented in an autonomous vacuum cleaner. An autonomous or robotic vacuum cleaner that uses a camera or sensor to detect visible light can utilize light wavelengths from 400 to 700 nanometers (nm). Such cameras or sensors have some drawbacks as they may not be able to detect certain obstacles on the floor or area being cleaned. For example, liquids such as water, cleaning fluid, urine, or any other liquid may not be detected by a visible camera due to the transparency of the liquid. An autonomous vacuum cleaner utilizing a SWIR imaging system can easily detect and identify liquids, such as liquids containing water. Detection of water-based liquids can be based on specific properties of water. For example, water absorbs wavelengths in the area of 1450 nm and thus returns dark areas in the captured image. Another advantage of a vacuum cleaner utilizing a SWIR imaging system can include specific material detection. Material detection can be useful for identifying the material of the area being cleaned, such as a wooden floor, parquet floor, marble floor, ceramic tile floor, etc. As described in the embodiments of the present disclosure, utilizing a SWIR imaging system in a vacuum cleaner can improve the current system and overcome significant drawbacks of vacuum cleaners known in the art.

[0023] FIG. 1 shows a schematic diagram of the components of an architecture according to one exemplary and non-limiting autonomous vacuum cleaning system in accordance with an embodiment of the present disclosure. The autonomous vacuum cleaning system 100 may include a short-wave infrared (SWIR) imaging system 110, a control system 120, and a processor 130. As used herein, the autonomous vacuum cleaning system 100, also referred to herein as "system 100," can be any autonomous or semi-autonomous vacuum cleaner. For example, it can be a mobile robot, a robotic vacuum cleaner, an autonomous floor cleaner, a floor-running sweeper, or any other autonomous or semi-autonomous cleaning machine, sweeper, scrubber, vacuum cleaner, or any other cleaning machine.

[0024] The SWIR imaging system 110 may include one or more illumination sources 112, one or more SWIR receivers or sensors 111, and an optics 113. The control system 120 may include an imaging system control unit 121, an analysis unit 122, and a navigation unit 123. The processor 130 may be connected to the SWIR imaging system 110 and the control system 120. In some embodiments of the present disclosure, the processor 130 may be implemented as an internal processor in each of the control system 120 and / or the imaging system 110, or as an external processor, for example, external to the system 100. The SWIR imaging system 110 may be operative to detect light reflected from the FOV and provide one or more images of the FOV or a portion thereof based on the light reaching the imaging system 110.

[0025] The illumination source 112 may include one or more illumination sources for illuminating or emitting radiation in the SWIR band towards an object, area, or scene, all of which are referred to herein as the target 140. The term "target" refers to any object, area, subject, element, surface, content, and / or anything included or disposed within the FOV of the imaging sensor, such as solid, liquid, flexible, and hard objects. The emitted radiation illumination from the system 100 is indicated by 141, and the illumination reflected from the target 140 towards the system 100 is indicated by 142. A portion of the emitted radiation 141 may also be reflected, deflected, or absorbed in other directions by the target 140 (not shown).

[0026] The illumination source 112 can include any light source or illumination source that can provide a certain level of illumination to the area surrounding the system 100. Any suitable type of illumination source 112 can be used, such as light-emitting diodes (LEDs), laser diodes, quasi-continuous wave (QCW) lasers, vertical-cavity surface-emitting lasers (VCSELs), Q-switched (QS) lasers, and so on. Any other illumination source, or any combination of illumination sources, can be included in the illumination source 112. The illumination source 112 can emit light of any bandwidth that can be detected by the receiver 111 for generating an image. For example, the light 141 emitted by the illumination source 112 can be within the SWIR area of the electromagnetic spectrum.

[0027] The illumination source 112 can include a first illumination source for emitting radiation towards the FOV at a first wavelength within the SWIR, for example, within a range of wavelengths from 1300 to 1500 nm. For example, by emitting radiation or transmitting illumination at a specific wavelength, such as 1450 nm or near 1450 nm, it may be possible to detect, observe, identify, or monitor the presence of a water-based liquid within the FOV as water-absorbed radiation at the 1450 nm wavelength. The SWIR image data can be acquired by the receiver 111 based on the radiation reflected from the target 140. Based on the SWIR image data, the presence of a water-based liquid within the FOV can be determined by identifying a dark area within the image. For example, a dark or black area within the SWIR image can indicate the presence of water, a water-based liquid, and any material containing water.

[0028] According to embodiments of the present disclosure, the illumination source 112 may be operated near 1400 nm in the SWIR spectrum, which is also known as the "solar blind region", where the SWIR becomes resilient to ambient noise from the sun. Illumination near 1400 nm, for example, between 1300 nm and 1500 nm, can result in a higher signal-to-noise ratio (SNR) and an overall improvement in image quality under common low visibility scenarios such as glare, rain, smoke, and fog. This can enable the SWIR camera 111 to generate high-resolution images and an increased detection range.

[0029] The illumination source 112 may further include a second illumination source for emitting radiation at a second wavelength within the SWIR towards a target within the FOV or towards a second FOV included within the first FOV. The second wavelength is pre-determined to be different from the first wavelength (illuminated by the first illumination source) and, according to embodiments of the present disclosure, enables the classification or detection of the material of a surface, plane, object, obstacle, or any other target, area, object, or surface within the FOV. For example, the first illumination source may be a VCSEL emitting illumination at 1450 nm, while the second illumination source may be an LED emitting illumination at 940 nm. Any additional illumination source may be required on top of the first illumination source for accurate material classification based on spectral analysis.

[0030] The receiver 111 may include one or more SWIR sensors or detectors for receiving and collecting radiation reflected from the target 140 and / or from the illuminated FOV. The receiver 111 may operate in the SWIR to acquire SWIR image data based on radiation reflected from elements, regions, or areas disposed within the illuminated FOV. The receiver 111 may generate or produce an electrical signal representative of the imagery of the illuminated FOV in response to the detected electromagnetic radiation. The receiver 111 may be capable of capturing an image of the surrounding, adjacent, or nearby area with respect to the position of the autonomous cleaning system 100. For example, the receiver 111 may be a forward-facing camera for capturing an image in the forward travel direction of the autonomous cleaning system 100, or may be implemented as a forward-facing camera. In some embodiments, the processing of the output of the receiver 111 may be performed by the processor 130 and, additionally or alternatively, by an external processor (not shown).

[0031] The receiver 111 and / or the illumination source 112 may utilize an optical system 113. The optical system 113 may include one or more optical elements such as mirrors, lenses, diffusers, or any other elements for improving, enhancing, and / or supporting the receiver 111 and / or the illumination source 112. For example, the optical system 113 may be arranged to collect, concentrate, optionally filter, and focus the reflected radiation 142 onto the focal plane of the receiver 111.

[0032] According to some embodiments of the present disclosure, receiver 111 may include a plurality of photodetection devices, such as a photodetector array or “PDA” that may include a number of photosites. Each photosite includes one or more photodiodes for detecting impinging light and a capacitance for storing the charge supplied by the photodiodes. Hereinafter, “photosite” is often replaced by the acronym “PS”. The term “photosite” relates to a single sensor element of an array of sensors. Each PS may include one or more photodiodes. A PS may also include some circuitry or additional components in addition to the photodiodes.

[0033] According to some embodiments of the present disclosure, receiver 111 may include a plurality of germanium (Ge) photodetectors (PDs) that operate to detect reflected SWIR radiation. In some embodiments, receiver 111 may include a SWIR focal plane array (FPA), while the plurality of Ge PDs may be or form part of the SWIR FPA. Receiver 111 may generate an electrical signal for each of the plurality of Ge PDs that represents the amount of incident SWIR light within its detectable spectral range. The amount of light detected at receiver 111 may include the amount of SWIR radiation reflected from target 140. And it may also include additional SWIR light arriving, for example, from sunlight or an external light source.

[0034] The term "Ge PD" relates to any PD in which light induced excitation of electrons (detectable later as a photocurrent) occurs at or within Ge, within a Ge alloy (e.g., SiGe), or at an interface between Ge (or a Ge alloy) and another material (e.g., silicon, SiGe). Specifically, the term "Ge PD" relates to both pure Ge PDs and Ge-silicon PDs. When a Ge PD containing both Ge and silicon is used, different concentrations of germanium may be used. For example, the relative proportion of Ge in a Ge PD can range from 5% to 99% (regardless of whether alloyed with silicon or adjacent to it). For example, the relative proportion of Ge in a Ge PD may be between 15% and 40%. Note that materials other than silicon, such as aluminum, nickel, silicides, or any other suitable material, may also be part of the Ge PD. In some implementations of the present disclosure, the Ge PD may be a pure Ge PD (containing more than 99.0% Ge).

[0035] In some embodiments of the present disclosure, the receiver 111 can be implemented as a PD array fabricated on a single chip. The PDs, such as Ge PDs, can be configured in any suitable arrangement, such as a rectangular matrix (rows and columns of Ge PDs), honeycomb tiling, and even an irregular configuration. Preferably, the number of Ge PDs in the receiver 111 can enable the generation of high-resolution images. For example, the number of PDs can be on the order of 1 megapixel, 10 megapixels, or more.

[0036] The exemplary embodiments disclosed herein relate to a system and method for high SNR active SWIR imaging using a receiver with a Ge-based PD. Compared to InGaAs technology, the main advantage of Ge receiver technology is its compatibility with CMOS processes, enabling the manufacture of the receiver as part of a CMOS production line. For example, a Ge PD can be integrated into a CMOS process by growing a Ge epilayer on a silicon (Si) substrate, such as in Si photonics. The Ge PD is thus also more cost-effective than an equivalent InGaAs photoreceiver.

[0037] The control system 120 may control the operation of the imaging system 110 by the imaging system control unit 121. The imaging system control unit 121 may control the operations of the receiver 111, the illumination source 112, and the optical system 113. The control unit 121 may trigger the activation of the illumination source 112 according to a predefined method and adapted to the operation of the receiver 111. For example, in some embodiments of the present disclosure, the control unit 121 may trigger the illumination source such that it operates only during the time period when the receiver 111 is open for detecting reflected light. In some embodiments of the present disclosure, the control unit 121 may trigger the illumination source 112 to operate constantly in pulses or any other method during a specific time period.

[0038] According to some embodiments of the present disclosure, the control unit 121 may be configured to control the activation of the receiver 111 for a relatively short integration time, thus limiting the impact of the accumulation of dark current noise on the quality of the generated signal. For example, the control unit 121 may operate to control the activation of the receiver 111 during an integration time such that the accumulated dark current noise does not exceed the readout noise that is independent of the integration time.

[0039] The control system 120 can control and / or perform the analysis of the imaging data by the analysis unit 122. The analysis unit 122 can be connected to the imaging system 110. Specifically, it can receive information from the receiver 111. The analysis unit 122 can receive an electrical signal generated in response to the electromagnetic radiation detected by the receiver 111. The received signal represents an image of the illuminated scene within the FOV. In some embodiments of the present disclosure, the signal detected by the receiver 111 can be processed by the analysis unit 122 and / or transferred to the processor 130 via the analysis unit 122 for processing into a SWIR image of the target 140 or the illuminated FOV. The analysis unit 122 can further process the electrical signal generated in response to the electromagnetic radiation to detect or observe the target 140, perform spectral analysis of the target 140, determine the spectral characteristics of the target 140, and / or determine the presence of one or more materials included in the target 140.

[0040] According to some embodiments of the present disclosure, the spectral analysis can be performed based on the absorption of the target 140 at a first wavelength and the absorption of the target at a second wavelength. The target 140 can be illuminated at two different SWIR wavelengths. For example, the first illumination source can emit radiation at a first wavelength, and the second illumination source can emit radiation at a second wavelength different from the first wavelength. Based on the absorption of the target 140 at the first wavelength and the absorption of the target 140 at the second wavelength, spectral analysis of the target 140 is performed, and the spectral characteristics of the target 140 can be determined or are determined.

[0041] According to some embodiments of the present disclosure, the illumination 112 can emit illumination at a specific wavelength that is selected to enable the detection, observation, or determination of the presence of a specific material, for example, for the detection of water or any liquid containing water. For example, since water has a prominent absorption peak near a wavelength of 1450 nm, the illumination source 112 can emit radiation at a wavelength of 1400 nm or 1450 nm.

[0042] The control system 120 can further control the navigation of the cleaning system 100 by the navigation unit 123. According to some embodiments of the present disclosure, based on the generated SWIR image of the FOV, the analysis unit 122 can detect or observe the target 140. For example, any obstacle or element detected within the FOV is identified and can be used to control the navigation of the cleaning system 100. The imaging system 110 can include additional types of sensors for providing information about its surrounding environment to the cleaning system 100 for navigation purposes. For example, the imaging system 110 can include one or more position sensing devices, one or more physical contact sensors, infrared (IR) sensors, one or more cameras or sensors that detect visible light, or any other type of sensor. The navigation unit 123 can receive a signal from any of the sensors of the imaging system 110 and adjust the position and trajectory of the cleaning system 100 based on the received signal.

[0043] All information and data collected by the imaging system 110 can be supplied to the control system 120 and specifically to the analysis unit 122 and the navigation unit 123. The analysis unit 122 receives the images captured by the imaging system 110 and can analyze the images to find significant features or landmark features within the FOV, for example, within the area surrounding the cleaning system 100. Significant features or landmark features can include high-contrast features detectable within the image, such as spilled water, table legs, shoes, cables, carpets, or any other object. The detected landmark features are used by the navigation unit 123 to triangulate and determine the position of the cleaning system 100 within the local environment and avoid obstacles without contact. For example, by identifying a puddle of a spilled liquid, such as water, soapy water, urine, or any other aqueous liquid, the cleaning system 100 can avoid driving into the liquid and pass around it, etc.

[0044] Figures 2A, 2B, and 2C are schematic diagrams of one exemplary autonomous cleaning system according to embodiments of the present disclosure. Figure 2A is a plan view of the cleaning system 100, and Figures 2B and 2C are side views of the cleaning system 100. The cleaning system 100 is the cleaning system 100 of Figure 1 and may include, incorporate, or utilize the same. The cleaning system 100 may include the main body 200, the cleaning head 213, and other components that enable the movement of the system 100, such as a motor, a transaction unit, a power supply unit, a battery, one or more wheels, and / or any other elements (not shown) necessary for movement and cleaning. The shape and design of the main body 200 shown in Figures 2A-2C, as well as any other components, serve as an exemplary design, and it should be understood by those skilled in the art that the cleaning system 100 may have any shape, design, structure, or appearance.

[0045] According to embodiments of the present disclosure, the main body 200 may include a camera 210 for capturing an image of the area surrounding the cleaning system 100. The camera 210 may include the imaging system 110 of Figure 1. The camera 210 may be any type of camera that may include the SWIR device or receiver 111 of Figure 1. The control system 120 of Figure 1 may be embodied in the software and electronics included within the cleaning system 100. It is capable of processing the images captured by the camera 210 and enables the cleaning system 100 to understand, interpret, and autonomously navigate the local environment.

[0046] The exemplary diagrams of FIGS. 2A and 2B show a FOV 220 having a 45-degree horizontal FOV 205 and a 5-degree vertical FOV 223. The first light source included in the cleaning system 100 can illuminate an area that can be disposed at a minimum distance 221, e.g., 25 centimeters (cm) from the cleaning system 100, and a maximum distance 222, e.g., 50 cm from the cleaning system 100. The first light source can emit illumination at a wavelength between 1300 nm and 1500 nm, e.g., 1450 nm, to identify or detect aqueous liquids on the surface illuminated by the first light source, as described in embodiments of the present disclosure.

[0047] According to embodiments of the present disclosure, at least one additional light source can be used to illuminate a second FOV 230 at a second wavelength different from the first wavelength (used by the first light source) to enable high-precision material classification. By illuminating the second FOV 230 within the FOV 220 with the second light source at the second wavelength, for example, the material classification related to the target or area 215 included in the second FOV 230 can be improved. Since each material has a unique spectral response, for example, by illuminating the detected material related to the target 215 at two or more wavelengths, it is possible to identify, determine, or detect the spectral response of the detected material.

[0048] According to the embodiments described herein, for each additional wavelength, it may be possible to use a first main illuminator, e.g., a VCSEL, to illuminate the main FOV 220 and add one or more additional light sources having a limited FOV 230. Since the additional light sources are limited within the main FOV 220 and may require low energy to illuminate the partial FOV 230, this makes it possible to reduce the total power required for the cleaning system 100. Each of the one or more additional light sources can include, for example, an IR LED that illuminates at a specific, predefined selected wavelength.

[0049] According to embodiments of the present disclosure, the material of target 215, e.g., the floor, can be determined based on at least two images of a particular area, target, or space that includes the detected material. Since every material has a unique spectral response, e.g., a unique spectral reflectance, such unique spectral responses can be used to determine the material of an illuminated area. Embodiments described herein can determine the material of a detected target based on at least two images taken at two different wavelengths within the SWIR. For example, a first image can be acquired based on illumination at a first wavelength related to a main, larger FOV 220, and a second image can be acquired based on illumination at a second wavelength related to a partial FOV 230 included in FOV 220. Other examples can include illuminating the same FOV with two or more wavelengths from two or more illumination sources.

[0050] Embodiments described herein can include illuminating a detected material, e.g., FOV 230, at a first wavelength and a second wavelength, and acquiring or identifying two values related to the reflectance and / or absorption of the detected material (a first value related to the first wavelength and a second value related to the second wavelength). The two values of reflectance and / or absorptance can match the spectral response of a particular material that can be identified or determined as the detected material.

[0051] According to embodiments of the present disclosure, the two values related to reflectance and / or absorption can be the minimum number of values based on which a spectral response can be determined. However, embodiments described herein can allow for any number of additional illumination sources emitting radiation at any number of additional wavelengths. For example, a third illumination source can be used to direct radiation at a third wavelength within the SWIR towards a target within the FOV, and the spectral characteristics of the target can be determined based on the absorption of the target at the first wavelength, the absorption of the target at the second wavelength, and the absorption of the target at the third wavelength.

[0052] Refer to FIG. 2C, which is a side view of cleaning system 100 according to an embodiment of the present disclosure. System 100 can provide an indication of any obstacles that are near the body 200 of cleaning system 100 or that are present in an area disposed proximate to body 110. Thereby, cleaning system 100 can avoid obstacles, such as obstacle 228, without contacting them. Based on identifying an obstacle in an image captured by imaging system 110, control system 120 of FIG. 1 can generate a control signal and can instruct cleaning system 100 to stop and / or adjust its position and trajectory. This can prevent cleaning system 100 from causing any damage to itself or to the obstacles.

[0053] The exemplary diagram of FIG. 2C shows FOV 220 consisting of a 17-degree vertical FOV 223. The first light source can illuminate an area that can be disposed at a minimum distance 221, such as 25 cm from cleaning system 100, and a maximum distance 224, such as 200 cm from cleaning system 100. For example, an obstacle 228 disposed within a distance 224 of about 150 cm from cleaning system 100 and having a reflectivity difference of at least 20% from the background can be detected in an image captured by imaging system 110 of cleaning system 100.

[0054] For example, in some embodiments of the present disclosure, the 17-degree vertical FOV 223 can be selected to enable capturing of the floor or other items at a height 227 of 10 cm within a distance 222 of 50 cm from cleaning system 100. When a single light source can be implemented, for example, in imaging system 110 of FIG. 1, the light source can have a Gaussian profile so as to have a sharp image at a height 227 of 10 cm within a distance 222 of 50 cm from cleaning system 100. Adding one or more light sources can extend the detection range and total power, respectively.

[0055] The image captured by the imaging system 110 can be analyzed to find significant features or landmark features within the FOV, such as an obstacle 228 that may have high contrast features that can be detected in the SWIR image. Detecting the obstacle 228 can be used, for example, by the navigation unit 123 of FIG. 1, to determine the position of the cleaning system 100 within the local environment and to avoid contact with the obstacle 228, such as by passing around it.

[0056] Refer to FIG. 3. FIG. 3 is a flowchart relating to a method for using a SWIR vision system in an autonomous cleaning system, according to some embodiments of the present disclosure. One embodiment of a method for using a SWIR vision system in an autonomous cleaning system can be performed, for example, by the exemplary system shown in FIG. 1.

[0057] In operation 310, the first light source can emit radiation at a first wavelength within the SWIR towards the first FOV. By way of example, the first light source, such as the light source 112 of FIG. 1, can emit radiation at wavelengths in the range between 1300 nm and 1500 nm. In some embodiments of the present disclosure, by emitting radiation, or transmitting illumination, at a specific wavelength, such as 1450 nm, or a wavelength close to 1450 nm, it may be possible to detect, observe, identify, or monitor the presence of an aqueous liquid within the first FOV.

[0058] In operation 320, a receiver operating in the SWIR can receive radiation reflected from an element disposed within the FOV, such as from an area, object, item, or any other component disposed within the FOV. The receiver, such as the receiver 112 of FIG. 1, can include one or more SWIR sensors or detectors for receiving and collecting radiation reflected from an element disposed within the FOV. The receiver can generate an electrical signal representative of an image of the illuminated FOV in response to the detected electromagnetic radiation.

[0059] In operation 330, SWIR image data can be acquired based on radiation reflected from elements disposed within the FOV. For example, elements disposed within the FOV can refer to any object, area, target, surface, content, and / or anything included in or disposed within the FOV of the imaging sensor, such as solids, liquids, flexible, and hard objects.

[0060] In operation 340, the presence of an aqueous liquid within the FOV can be determined based on the SWIR image data. For example, as water absorption radiation at a wavelength of 1450 nm, the SWIR image data is acquired based on radiation reflected from the illuminated FOV, while the presence of an aqueous liquid within the FOV can be determined by identifying dark areas within the image.

[0061] In operation 350, a second light source can emit radiation at a second wavelength in the SWIR towards a target within the FOV and / or towards a second FOV included within the first FOV. In accordance with embodiments of the present disclosure, the second wavelength is selected to be different from the first wavelength so as to enable detection and spectral classification of materials related to surfaces, planes, objects, obstacles, or any other target within the FOV.

[0062] In operation 360, the spectral characteristics of the target can be determined based on the absorption of the target at the first wavelength and the absorption of the target at the second wavelength. Based on the absorption of the target at the first wavelength and the absorption of the target at the second wavelength, spectral analysis of the target is performed, and the determination of the spectral characteristics of the target is made or can be made.

[0063] Unless otherwise stated explicitly, embodiments of the methods described herein are not restricted to a particular order or sequence. Further, all operations described herein are intended as examples only, and other or different operations may be used. Additionally, embodiments of the methods described, or some of their elements, may occur or be executed at the same time.

[0064] In some embodiments, the receiver 111 is activated multiple times to create "time slices", each covering a specific distance range. The image processor 130 may combine these slices to create a single image with a greater visual depth, enabling the determination or measurement of the distance of objects within the FOV.

[0065] According to some embodiments of the present disclosure, different techniques may be used to determine depth based on the output of one or more photosites, such as the photosites included in the receiver 111 of FIG. 1. The embodiments described herein may be used to determine the distance of objects within the FOV of a SWIR electro-optical system, such as the imaging system 110 of FIG. 1, as well as other electro-optical systems sensitive to other parts of the electromagnetic spectrum.

[0066] According to some embodiments of the present disclosure, a method for determining depth may be used by the control system 120 of the cleaning system 100 to provide information about its surrounding environment, for example, for navigation purposes. For example, the navigation unit 123 of FIG. 1 may adjust the position and trajectory of the cleaning system 100 based on depth measurements indicating the distance of one or more obstacles, landmarks, or any other elements located in the vicinity of the cleaning system 100.

[0067] Embodiments of the present invention can generate a depth image of a scene based on detections of a Short-Wavelength Infrared (SWIR) electro-optical imaging system (SEI system). The SEI system described herein can be implemented in any of the systems of the cleaning system 100, such as the imaging system 110, the control system 120, or the processor 130 of FIG. 1, according to examples of the technical subject matter disclosed herein. The SEI system can be any of the systems described above or any other suitable SWIR electro-optical system (e.g., a sensor, a camera, a lidar, etc.). The methods described herein can be executed by one or more processors of the SEI system, such as the processor 130 of FIG. 1, and / or one or more processors external to the SEI system, or a combination of both.

[0068] Multiple detection signals of the SEI system can be acquired. Each detection signal can indicate the amount of light captured by at least one Focal Plane Array (FPA) detector of the SEI system from a particular direction within the Field of View (FOV) of the SEI system over each respective detection time frame (i.e., the detection time frame measured from the trigger of illumination by a related light source, such as a laser, at which each respective detection signal is captured). The at least one FPA can include a plurality of individual photosites. Each photosite can include a Ge element that converts incident photons into detected charge. Even if it does not contain Ge, rather, contains other elements, any type of photosite characterized by a high dark current can be implemented in the SEI system.

[0069] For each of the multiple directions within the FOV, different detection signals (among the multiple detection signals described above) indicate the level of reflected SWIR illumination from different distance ranges along that direction.

[0070] Refer to FIG. 4. FIG. 4 illustrates the timings of three different detection signals arriving from the same direction within the FOV according to an embodiment of the present disclosure. Diagram 571 in FIG. 4 shows the timings of three different detection signals arriving from the same direction within the FOV. The y-axis (ordinate) in the figure indicates the level of the response of the detection system to the reflected photons arriving from the relevant direction. The reflected illumination is generated from one or more light sources (e.g., lasers, LEDs) that are optionally controlled by the same processor that controls the FPA, and is reflected from a portion of the FOV (e.g., corresponding to the spatial volume detectable by a single photosite). Note that the different detection signals can be associated with similar but not completely overlapping portions of the FOV (e.g., when the sensor, the scene, or the intermediate optical system between the two is moving in time, the detection signals from the same photosite can be reflected from somewhat different angles within the FOV in different detection time windows associated with different detection signals).

[0071] Note that diagram 571 does not show the detection level of each signal, but rather shows the response of the detection signal to the photons reflected from a perfect reflector at different times from the start of the light emission. Diagram 572 shows three objects arranged at different distances from the SEI system, for example, from the cleaning system 100. Note that in many instances, only one object is detected at each time in each direction, which is the object closest to the SEI system. However, in some scenarios, two or more objects can be detected (e.g., when the foreground object is partially transparent or does not block the light from the entire photosite).

[0072] Diagram 573 shows the levels of three returning signals in the direction of one of the objects - in this example, a dog in the near field, a shoe in the intermediate range, and a table in the far field (the selection of objects is arbitrary, and only the light reflected from a portion of each object is typically detected by a single photosite). The light returning from an object at distance D1 is represented by the shape of a dog for three different detection signals (corresponding to different detection timing windows and different ranges from the SEI system). Similarly, the levels of the detection signals corresponding to the light reflected from objects at distances D2 and D3 are represented by the symbol of a shoe and the symbol of a table, respectively.

[0073] As shown in Diagram 574, the reflection from an object placed at a given distance can be converted into a tuple (or any other representation of data as a direction - related data structure (DADS) in any suitable form) that shows the relative levels of the detection signals in different time windows. In the example shown, each number in the tuple represents the detected signal level in one detection window. The indication of the detection levels in the tuple may be corrected for the distance from the sensor (since the reflected light from the same object decreases with distance), but this is not necessarily the case. In the example shown, three partially overlapping time windows are used, but any number of time windows may be used. The number of time windows may be the same for different areas of the FOV, but it does not have to be.

[0074] The embodiments described herein may include processing a plurality of detection signals to determine a three-dimensional (3D) detection map that includes a plurality of 3D positions within a field of view (FOV) in which an object is detected. Processing may include compensating for a dark current (DC) level accumulated during collection of the plurality of detection signals generated from Ge elements. And compensating may include applying different degrees of dark current compensation to detection signals detected by different photosites of at least one focal plane array. Referring to examples of the accompanying drawings, different detection signals may be acquired at different times by different readout structures of any of the applicable photosites described above. Alternatively, the detection signals may be acquired by groups of interconnected photosites, as described in more detail below. Other implementations may also be used.

[0075] In addition to, or instead of, compensating for the accumulated dark current, processing may include compensating for a high integration noise level and / or readout noise level during readout of the plurality of detection signals. Compensating may include applying different degrees of noise level compensation to detection signals detected by different photosites of at least one focal plane array.

[0076] Compensation for dark current collection, readout noise, and / or integration noise may be performed in any suitable manner. Such as by using any combination of one or more of the following: software, hardware, and firmware. In particular, compensation for dark current collection may be implemented using any combination of one or more of the systems, methods, and computer program products described above, and any portions thereof. Some non-limiting examples of systems, methods, and computer program products may be used to compensate for dark current and to apply a degree of dark current compensation to detection signals detected by different photosites.

[0077] In some embodiments, compensating may be performed during acquisition of the plurality of detection signals (e.g., at the hardware level of the sensor). And processing may be performed on the detection signals that have already been compensated for dark current accumulation.

[0078] In some embodiments of the present disclosure, compensating may optionally include subtracting a first dark current compensation offset from a first detection signal detected by a first photosite corresponding to a first detection range, and subtracting a second dark current compensation offset different from the first dark current compensation offset from a second detection signal detected by a first photosite corresponding to a second detection range that is further away from the SEI system than the first detection range.

[0079] Embodiments described herein may include coordination of active illumination (e.g., by at least one light source of the SEI system) and acquisition of detection signals. Optionally, embodiments described herein may further include: (a) triggering emission of a first illumination (e.g., laser, LED) in coordination with the start of exposure of a first gate image in which a plurality of first detection signals are detected in different directions among a plurality of directions; (b) triggering emission of a second illumination (e.g., laser, LED) in coordination with the start of exposure of a second gate image in which a plurality of second detection signals are detected in different directions; and (c) triggering emission of a third illumination (e.g., laser, LED) in coordination with the start of exposure of a third gate image in which a plurality of third detection signals are detected in different directions.

[0080] In such cases, the processing may optionally include the following. That is, based on at least one detection signal from each of the first image, the second image, and the third image, determining the presence of a first object at a first 3D position within a first direction among different directions, and based on at least one detection signal from each of the first image, the second image, and the third image, determining the presence of a second object at a second 3D position within a second direction among different directions. Here, the distance of the first object from the SEI system is at least twice the distance of the second object from the SEI system.

[0081] Optionally, applying different degrees of DC compensation to detection signals detected by different photosites of at least one FPA may include using the detected dark current levels of different reference photosites that are shielded from the light reaching from the FOV.

[0082] Optionally, compensating may include applying different degrees of DC compensation to detection signals simultaneously detected by different photosites of at least one FPA.

[0083] Regarding integration noise and readout noise, note that compensation for such noise may be correlated to the number of illumination pulses used to illuminate a portion of the FOV during the acquisition of each detection signal by at least one processor executing the embodiment. Different numbers of illumination pulses may result in significant non-linearity of the detected signals. It is optionally corrected as part of the processing prior to determining the distance / 3D position of different objects within the FOV.

[0084] Referring to the use of DADS to determine the distance / 3D position of different objects within the FOV, by way of example, different conversion functions of DADS (e.g., tuples) with respect to distance can be used for different directions within the FOV to compensate for, among other things, non-uniformity of detection channels across the FOV (e.g., sensor and / or detected object), non-uniformity of illumination (e.g., using multiple light sources, light source non-uniformity, or non-uniformity of the optical system), etc.

[0085] According to some embodiments of the present disclosure, different detection signals from the same direction within the FOV can correspond to different detection windows. It can be for objects at the same distance or different distances. For example, the detection window can correspond to a range of distances that is about 50 cm (e.g., between 80 cm from the SEI system and 130 cm from the SEI system). In different examples, some or all of the detection windows used to determine the distance / 3D position of objects within the FOV can be in distance ranges between 0.01 m - 0.1 m, between 0.1 m - 10 m, between 5 m - 25 m, between 20 m - 50 m, between 50 m - 100 m, between 100 m - 250 m, etc. The distance ranges associated with different detection signals can overlap. For example, a first detection window can detect returning light from an object whose distance from the SEI system is between 0 cm - 50 cm, a second window can correspond to an object between 25 cm - 75 cm, and a third window can correspond to an object between 50 cm - 150 cm.

[0086] Embodiments disclosed herein may include a system for generating a depth image of a scene based on detection of a short-wavelength infrared (SWIR) electro-optical imaging system (SEI system). The system includes at least one processor configured to acquire a plurality of detection signals of the SEI system, each detection signal indicating an amount of light captured by at least one FPA detector of the SEI system over respective detection time frames from a particular direction within the FOV of the SEI system, where the at least one FPA includes a plurality of individual photosites, each photosite including a Ge element that converts incident photons into detected charge, and where for each of a plurality of directions within the FOV, different detection signals indicate reflected SWIR illumination levels from different distance ranges along the direction. And the system is configured to process the plurality of detection signals to determine a three-dimensional (3D) detection map including a plurality of 3D positions within the FOV where an object is detected, where processing includes correcting a dark current (DC) level accumulated during collection of the plurality of detection signals resulting from the Ge element, and where correcting includes applying different degrees of DC compensation to detection signals detected by different photosites of the at least one FPA.

[0087] In some embodiments of the present disclosure, correcting may include subtracting a first DC compensation offset from a first detection signal detected by a first DE corresponding to a first detection range, and subtracting a second DC compensation offset different from the first DC compensation offset from a second detection signal detected by the first DE corresponding to a second detection range that is further from the SEI system than the first detection range.

[0088] In some embodiments of the present disclosure, at least one processor may be further configured as follows. That is, (a) trigger the emission of first illumination in coordination with the start of exposure of a first gate image, where a plurality of first detection signals are detected for different directions among a plurality of directions; (b) trigger the emission of second illumination in coordination with the start of exposure of a second gate image, where a plurality of second detection signals are detected for different directions; and (c) trigger the emission of third illumination in coordination with the start of exposure of a third gate image, where a plurality of third detection signals are detected for different directions. In such cases, at least one processor may be further configured to determine the following as part of the determination of the 3D detection map. That is, (a) the presence of a first object at a first 3D position within a first direction among different directions based on at least one detection signal from each of the first image, the second image, and the third image; and (b) the presence of a second object at a second 3D position within a second direction among different directions based on at least one detection signal from each of the first image, the second image, and the third image. Here, the distance of the first object from the SEI system is at least twice the distance of the second object from the SEI system. The gate image (or its equivalent) can be achieved, for example, by utilizing different readout structures of the photosites of the PDA in any of the methods described above.

[0089] In some embodiments of the present disclosure, applying different degrees of DC compensation to detection signals detected by different photosites of at least one FPA includes using the detected dark current levels of different reference photosites that are shielded from the light reaching from the FOV. Optionally, compensating may include applying different degrees of DC compensation to detection signals simultaneously detected by different photosites of at least one FPA. Optionally, one or more (and perhaps all) of the at least one processor may be part of the SEI system.

[0090] Referring now to FIG. 5, FIG. 5 is a block diagram depicting a computing device that may be included within an embodiment of a system for utilizing SWIR sensing in a vacuum cleaner, according to some embodiments of the present disclosure.

[0091] The computing device 1 may include, for example, a central processing unit (CPU) processor, a chip, or any suitable computing device or controller 2 that may be a computing device, an operating system 3, a memory 4, executable code 5, a storage system 6, an input device 7, and an output device 8. The controller 2 (or one or more controllers or processors across multiple units or devices) may be configured to execute the methods described herein and / or to execute or operate as various modules, units, etc. Two or more computing devices 1 may be included in a system according to an embodiment of the present disclosure, and one or more computing devices 1 may act as components of a system according to an embodiment of the present disclosure. For example, the computing device 1 may be included in the imaging system 1, the control system 120, and / or the processor 130 of the vacuuming system 100 of FIG. 1.

[0092] The operating system 3 may be any code segment designed and / or configured to perform tasks including, for example, adjusting, scheduling, arbitrating, supervising, controlling, or otherwise managing the operation of the computing device 1, and / or may include or be such (e.g., similar to the executable code 5 described herein). For example, it may be to schedule the execution of a software program or task, or to enable communication between a software program or other module or unit. The operating system 3 may be a commercial operating system. Note that the operating system 3 may be any component, and for example, in some embodiments, the system may include computing devices that do not require or have an operating system 3.

[0093] The memory 4 may be, for example, a random access memory (RAM), read only memory (ROM), dynamic RAM (DRAM), synchronous DRAM (SD-RAM), double data rate (DDR) memory chip, flash memory, volatile memory, non-volatile memory, cache memory, buffer, short-term memory unit, long-term memory unit, or other suitable memory unit or storage unit, and / or may include them. The memory 4 may be a plurality of, perhaps different, memory units, and / or may include it. The memory 4 may be a non-transitory readable medium of a computer or processor, or a non-transitory storage medium of a computer, such as RAM.

[0094] The executable code 5 may be any executable code, such as an application, program, process, task, or script. The executable code 5 may be executed by the controller 2, perhaps under the control of the operating system 3. For example, the executable code 5 may be an application that enables the automatic acquisition of SWIR images. For clarity, a single item of executable code 5 is shown in FIG. 4, but systems according to some embodiments may include multiple executable code segments similar to the executable code 5 that may be loaded into the memory 4 and cause the controller 2 to execute the methods described herein.

[0095] The storage system 6 may be, for example, a flash memory known in the art, a memory existing within or embedded in a microcontroller or chip known in the art, a hard disk drive, a write-once (CD-R) drive, a Blu-ray Disc (BD), a Universal Serial Bus (USB) device, or other suitable removable and / or fixed storage unit, or may include them. The content may be stored in the storage system 6 and loaded from the storage system 6 into the memory 4, where it may be processed by the controller 2. In some embodiments, some of the components shown in FIG. 4 may be omitted. For example, the memory 4 may be a non-volatile memory having the storage capacity of the storage system 6. Thus, although shown as a separate component, the storage system 6 may be embedded in or included in the memory 4.

[0096] Input device 7 may be, for example, any suitable input device, component, or system such as a removable keyboard or keypad, mouse, etc., or may include them. Output device 8 may include one or more (possibly removable) displays or monitors, speakers, and / or any other suitable output device. As indicated by block 7 and block 8, any applicable input / output (I / O) device may be connected to computing device 1. For example, a wired or wireless network interface card (NIC), a Universal Serial Bus (USB) device, or an external hard drive may be included in input device 7 and / or output device 8. As will be recognized, any suitable number of input devices 7 and output devices 8 may be operably connected to computing device 1 as indicated by blocks 7 and 8.

[0097] According to some embodiments, the system may include components such as, but not limited to, multiple central processing units (CPUs), or any other suitable general-purpose or specific processor, controller (e.g., a controller similar to controller 2), microprocessor, microcontroller, field programmable gate array (FPGA), programmable logic device (PLD), or application specific integrated circuit (ASIC). According to some embodiments, the system may include multiple input units, multiple output units, multiple memory units, and multiple storage units. The system may additionally include other suitable hardware components and / or software components. Where applicable, the modules or units described herein may be similar to device 1 described herein or may include components thereof.

[0098] Unless otherwise indicated, the functions described above in this specification can be performed by executable code and instructions stored on a computer-readable medium and executed on one or more processor-based systems. Further, those skilled in the art will understand that the present disclosure can be implemented in other computer system configurations, including multiprocessor systems, microprocessor-based electronic devices, minicomputers, mainframe computers, and the like.

[0099] The terms "for example," "e.g.," and "optionally" as used herein are intended to introduce non-limiting examples. While a given reference is made to a given exemplary system component or algorithm, other components and algorithms can be used as well, and / or the exemplary components can be combined with fewer components, and / or divided into additional components.

[0100] While certain features of the present disclosure have been illustrated and described herein, many modifications, substitutions, changes, and equivalents will occur to those skilled in the art. Accordingly, it is to be understood that the appended claims are intended to cover all such modifications and changes as fall within the true spirit of the present disclosure.

[0101] Various embodiments have been presented. Each of these embodiments, of course, can include features from the other embodiments presented. And embodiments not specifically described can include various features described herein.

Claims

1. An autonomous cleaning system, a first illumination source for emitting radiation at a first wavelength within a short-wave infrared range (SWIR) towards a field of view (FOV), a receiver operating within the SWIR and configured to acquire SWIR image data based on radiation reflected from an element disposed within the FOV, a processor for determining the presence of an aqueous liquid within the FOV based on the SWIR image data, comprising an autonomous cleaning system.

2. The autonomous cleaning system further comprises a second illumination source for emitting radiation at a second wavelength within the SWIR towards a target within the FOV, wherein the processor is operative to determine spectral characteristics of the target based on absorption of the target at the first wavelength and absorption of the target at the second wavelength, The autonomous cleaning system according to claim 1.

3. The second illumination source is for emitting radiation towards the target by emitting radiation at the second wavelength within the SWIR towards a second FOV included within a first FOV, The autonomous cleaning system according to claim 2.

4. The second illumination source includes a low-power illumination source, The autonomous cleaning system according to claim 2.

5. The receiver comprises one or more germanium (Ge) photodiodes (PDs), The autonomous cleaning system according to claim 1.

6. The first illumination source includes a laser source, The autonomous cleaning system according to claim 1.

7. The first wavelength is between 1300 nm and 1400 nm, The autonomous cleaning system according to claim 1.

8. The second light source includes a light emitting diode (LED). The autonomous cleaning system according to claim 1.

9. The second wavelength is between 800 nm and 1000 nm. The autonomous cleaning system according to claim 1.

10. The autonomous cleaning system further includes a third light source for emitting radiation at a third wavelength within the SWIR towards the target within the FOV, The processor is operative to determine spectral characteristics of the target based on absorption of the target at the first wavelength, absorption of the target at the second wavelength, and absorption of the target at the third wavelength. The autonomous cleaning system according to any one of claims 1 to 9.

11. A method for using a short-wave infrared (SWIR) vision system in an autonomous cleaning system, the method comprising: emitting radiation at a first wavelength within the SWIR towards a field of view (FOV) by a first light source; receiving radiation reflected from an element disposed within the FOV by a receiver operating within the SWIR; obtaining SWIR image data based on the radiation reflected from the element disposed within the FOV; determining the presence of an aqueous liquid within the FOV based on the SWIR image data; and a method.

12. The method further includes emitting radiation at a second wavelength within the SWIR towards a target within the FOV by a second light source; Determining spectral characteristics of the target based on absorption of the target at the first wavelength and absorption of the target at the second wavelength; The method according to claim 11, comprising: **Claim 13** The step of emitting radiation at a second wavelength within the SWIR towards a target within the FOV includes: Emitting radiation at a second wavelength within the SWIR towards a target of a second FOV included within a first FOV. The method according to claim 12, comprising: The method according to claim 12. **Claim 14** The second illumination source includes a low-power illumination source. The method according to claim 12. **Claim 15** The receiver comprises one or more germanium (Ge) photodiodes (PDs). The method according to claim 12. **Claim 16** The first illumination source includes a laser source. The method according to claim 11. **Claim 17** The first wavelength is between 1300 nm and 1400 nm. The method according to claim 11. **Claim 18** The second illumination source comprises a light emitting diode (LED). The method according to claim 12. **Claim 19** The second wavelength is between 800 nm and 1000 nm. The method according to claim 12. **Claim 20** The method further comprises: Emitting radiation at a third wavelength within the SWIR towards the target within the FOV by a third illumination source; Determining spectral characteristics of the target based on absorption of the target at the first wavelength, absorption of the target at the second wavelength, and absorption of the target at the third wavelength; The method according to any one of claims 11 to 19, comprising .

Images