Laser triangulation camera based on SWIR illumination source

Abstract

A triangulation system including an illumination unit including a SWIR illumination source for emitting a light beam; optics capable of shaping the light beam into a line; an imaging unit including at least one SWIR detector configured to detect the light beam when reflected by an object in the field of view; wherein deformation of the line in relation to an initial path of the line to the imaging unit translates into at least a depth of the object in the field of view; and wherein the depth allows for calculation by an application associated with at least one processor of at least a height and profile of the object based on known parameters of the system, including at least an angle of the line, and a distance between the imaging unit and the illumination source.

Description

LASER TRIANGULATION CAMERA BASED ON SWIR ILLUMINATION SOURCECROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims priority to U.S. provisional application No. 63 / 710,887 filed October 23, 2024, incorporated herein by reference in its entirety.FIELD OF THE DISCLOSURE

[0002] The present disclosure relates to triangulation, and, more particularly, to a laser triangulation camera such as may be based on shortwave infrared (SWIR) range (l.Omicrons to 1.7microns) illumination, and a SWIR detector for operation in environments optically challenged by sunlight, low-light, and / or mixed lighting conditions.BACKGROUND

[0002] There exist different depth camera technologies for calculating depth of objects in a scene, such as time-of-flight (direct, gated, and indirect), structured light, active stereo, laser triangulation, etc. Such technologies are used to support applications for inspection, 3D modelling, object identification, object counting, etc.

[0003] However, a key limitation of current state-of-the-art laser triangulation solutions is the collection of data in environments optically challenged by background illumination. In indoor or low-light environments, laser triangulation solutions using visible or near infrared (NIR) illumination sources typically work fine; however, in visual environments contaminated by bright sunlight or mixed lighting (such as both bright sunlight and low-light) conditions in the imaging field of view, the depth readings can be easily corrupted through pixel saturation, blooming, low SNR (Signal to Noise Ratio), or by background illumination obfuscating the projection of the laser line. Thus, if a laser triangulation camera illumination is based on visible or NIR light, then the illumination is a critical failure point for operation in environments with significant background illumination (such as lights in a factory or warehouse, vehicle lights required for night-time operation, additional vehicles operating in the same field / mine / factory, or bright sunlight in an outdoor setting).

[0004] Traditional laser triangulation cameras typically operate with lasers (edge emitting lasers (EEL), vertical cavity edge emitting lasers (VCSEL), light emitting diodes (LED)) in the visible range or NIR range. The laser(s) projects a line into the FOV of the image sensor. If an object is present, the position of the reflected laser line on the image sensor changes. This change in position can be translated into a depth measurement. However, if the laser triangulation camera operates in sunlight, then the image sensor is challenged to differentiate between the photons creating the projected laser line and the photons from the background illumination. Consequently, an errant depth measurement could result. An obvious solution may be to increase the active illumination of the laser triangulation camera to increase the SNR of the active illumination, but this can lead to pixel saturation on the image sensor and / or violation of laser eye safety thresholds meant to protect the human eye from injury or damage.

[0005] There thus exists a need to avoid or at least reduce the problems of current state-of-the-art laser triangulation solutions, and to provide a depth camera that can operate in environments optically challenged by large amounts of background light, bright sunlight, and / or mixed lighting conditions. This need may be satisfied by an insensitivity to background illumination along the measurement path.SUMMARY

[0006] Some embodiments of the invention disclosed herein are set forth below, and any combination of these embodiments (or portions thereof) may be made to define another embodiment.

[0007] In one aspect, a triangulation system comprises: an illumination unit including a first SWIR illumination source for emitting a light beam; optics capable of shaping the light beam into a line, wherein projection of the line is fixed to a specific measurement position within a field of view; an imaging unit including at least one SWIR detector configured to detect the light beam when reflected by an object in the field of view; wherein deformation of the line in relation to an initial path of the line to the imaging unit translates into at least a depth of the object in the field of view; and wherein the depth allows for calculation by an application associated with at least one processor of at least a height and profile of the object based on known parameters of the system, including at least an angle of the line, and a distance between the imaging unit and the illumination source.

[0008] In some embodiments, the system further comprises a bandpass fdter in an optical path of the imaging unit configured to limit detected photons by the imaging unit to a SWIR wavelength at or near a wavelength of the illumination unit.

[0009] In some embodiments, the illumination unit includes second a SWIR illumination source.

[0010] In some embodiments, the second SWIR illumination source is configured for flood illumination.

[0011] In some embodiments, the second SWIR illumination source comprises an light emitting diode (LED).

[0012] In some embodiments, the first and second SWIR illumination sources are configured to act independently.

[0013] In some embodiments, a first operating mode of the first and second SWIR illumination sources is configured to provide line illumination only.

[0014] In some embodiments, a second operating mode of the first and second SWIR illumination sources is configured to provide flood illumination only.

[0015] In some embodiments, a third operating mode of the first and second SWIR illumination sources is configured to provide combined line illumination and flood illumination.

[0016] In some embodiments, the SWIR detector comprises a germanium-silicon (GsSi) detector.

[0017] In some embodiments, the SWIR detector comprises an Indium gallium arsenide (InGaAs) detector.

[0018] In some embodiments, the SWIR detector comprises a quantum dot SWIR detector.

[0019] In some embodiments, the illumination unit comprises a vertical cavity edge emitting laser (VCSEL).

[0020] In some embodiments, the illumination unit comprises an edge emitting laser (EEL).

[0021] In some embodiments, the illumination unit emits light near a wavelength of 1130nm.

[0022] In some embodiments, the illumination unit emits light near a wavelength of 1370nm.

[0023] In some embodiments, the illumination unit emits light near a wavelength of 1550nm.BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The following drawings are illustrative of particular examples of the present disclosure and therefore do not limit the scope of the present disclosure. The drawings are not to scale and are intended for use in conjunction with the explanations in the following detailed description. In the drawings, like reference numerals may represent like parts and assemblies throughout the several views.

[0025] FIG. 1 illustrates aspects of the disclosure.

[0026] FIG. 2 illustrates aspects of the disclosure

[0027] FIG. 3 illustrates aspects of the disclosure.

[0028] FIG. 4 illustrates aspects of the disclosure.

[0029] FIG. 5 illustrates aspects of the disclosure.

[0030] FIG. 6 illustrates aspects of the disclosure.DETAILED DESCRIPTION

[0031] The following discussion omits or only briefly describes conventional features of similar systems and methods that are apparent to those skilled in the art. Those of ordinary skill may thus recognize that other elements may be desirable and / or necessary to implement the devices, systems, and methods described herein. It is noted that various examples are described in detail with reference to the drawings. Reference to these various examples does not limit the scope of the claims attached hereto. Additionally, any examples set forth in this specification are intended to be non-limiting and merely set forth some of the many possible implementations for the appended claims. Further, particular features described herein can be used in combination with other described features in each of the various possible combinations and permutations. As such, it is understood that the detailed description is exemplary and explanatory only and is not restrictive of the broad inventive concepts upon which the examples disclosed herein are based.

[0032] Unless otherwise specifically defined herein, all terms are to be given their broadest reasonable interpretation. This includes meanings implied from the specification as well as meanings understood by those skilled in the art and / or as defined in dictionaries, treatises, etc.

[0033] It is noted that, as used in the specification and the appended claims, the singular forms “a,” “an” and “the” include plural referents unless otherwise specified. The terms “includes”and / or “including,” when used in this specification, specify the presence of stated features, elements, and / or components, but do not preclude the presence or addition of one or more other features, steps, operations, elements, components, and / or groups thereof.

[0034] Relative terms such as “horizontal,” “vertical,” “up,” “down,” “top,” and “bottom” as well as derivatives thereof (e.g., “horizontally,” “downwardly,” “upwardly,” etc.) should be construed to refer to the orientation as then-described or as shown in the drawing figure under discussion. These relative terms are for convenience of description and normally are not intended to require a particular orientation in actuality. Terms including “inwardly” versus “outwardly,” “longitudinal” versus “lateral” and the like are to be interpreted relative to one another or relative to an axis of elongation, or an axis or center of rotation, as appropriate. Terms concerning attachments, coupling and the like, such as “connected” and “interconnected,” refer to a relationship wherein structures are secured or attached to one another either directly or indirectly through intervening structures, as well as both movable or rigid attachments or relationships, unless expressly described otherwise. The phrases “configured to”, “operatively” or “operably connected” indicates such an attachment, coupling or connection that allows the pertinent structures to operate as intended by virtue of that relationship.

[0035] Reference throughout the specification to “exemplary”, “one example”, “an example” or “some examples” means that a particular feature, structure, or characteristic is described in connection with at least one example of the subject matter disclosed. Thus, the appearance of the phrases “in one example”, “in an example”, “by way of example”, “in some examples”, and other like-phrases in various places throughout the specification do not necessarily refer to a single or the same example. Further, the particular features, structures or characteristics of “one example”, “an example”, “some examples”, or other like-phrases may be combined in any suitable manner with each other to form additional examples of such combinations. It is intended that examples of the disclosed subject matter cover modifications and variations thereof. Terms such as “first,” “second,” “third,” etc., merely identify one of a number of portions, components, steps, operations, functions, and / or points of reference as disclosed herein, and likewise do not necessarily limit embodiments of the present disclosure to any particular configuration or orientation.

[0036] The terms “proximal,” “distal,” “anterior,” “posterior,” “medial,” “lateral,” “superior,” and “inferior” are defined by their standard usage indicating a directional term of reference. Forexample, “proximal” refers to a position that is situated nearer to a body or point of attachment or interest, while “distal” refers to a position that is situated away from the body or point of attachment or interest. In another example, “anterior” refers to the front of a body or structure, while “posterior” refers to the rear of a body or structure, in relation to a relative viewpoint. In another example, “medial” refers to the direction towards the midline of a body or structure, and “lateral” refers to the direction away from the midline of a body or structure. In some examples, “lateral” or “laterally” may refer to any sideways direction. In another example, “superior” refers to the top of a body or structure, while “inferior” refers to the bottom of a body or structure. It should be understood, however, that the directional term of reference may be interpreted within the context of a specific body or structure, such that a directional term referring to a location in the context of the reference body or structure may remain consistent as the orientation of the body or structure changes.

[0037] Moreover, throughout this disclosure, various aspects may be presented in a range format. It should be understood that a description in range format is merely for convenience and brevity and should not be construed as an inflexible limitation on the scope of the disclosure. Accordingly, the description of a range should be considered to have specifically disclosed all the possible subranges as well as individual numerical values within that range. For example, description of a range such as from 1 to 6 should be considered to have specifically disclosed subranges such as from 1 to 3, from 1 to 4, from 1 to 5, from 2 to 4, from 2 to 6, from 3 to 6, etc., as well as individual numbers within that range, for example, 1, 2, 2.7, 3, 4, 5, 5.3, 6, and any whole and partial increments there between. This applies regardless of the breadth of the range. In relation to a range and as used herein, the term “about” in reference to a measurable value, such as an amount, a temporal duration, and the like, is meant to encompass variations of plus or minus 20%, plus or minus 10%, plus or minus 5%, plus or minus 1%, and plus or minus 0.1% of the specified value, as such variations are appropriate.

[0038] Processor-implemented modules and systems are disclosed herein that may provide access to and transformation of a plurality of types of non-transitory digital content, including but not limited to data streams, sensor output, and mathematical models, and the algorithms applied herein may track, deliver, manipulate, transform, transceive and report the accessed content. Described embodiments of these modules, apps, systems and methods are intended to be exemplary and not limiting.

[0039] Those of skill in the art would further appreciate that the various illustrative logical blocks, modules, circuits, and algorithmic steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps are described generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.

[0040] An exemplary computing processing system for use in association with the embodiments, by way of non-limiting example, is capable of executing software, such as an operating system (OS), applications / apps, user interfaces, and / or one or more other computing algorithms, such as the algorithms, decisions, models, modules, programs and subprograms discussed herein. The operation of the exemplary processing system is controlled primarily by non-transitory computer readable instructions / code, such as instructions stored in a computer readable storage medium, such as hard disk drive (HDD), optical disk, solid state drive, Random Access Memory (RAM), a flash memory, or the like. Such instructions may be executed within the central processing unit (CPU) to cause the system to perform the disclosed operations.

[0041] In many known computer servers, workstations, mobile devices, personal computers, and the like, the CPU is implemented in an integrated circuit called a processor. It is appreciated that, although the exemplary processing system may comprise a single CPU, such description is merely illustrative, as the processing system may comprise a plurality of CPUs. As such, the disclosed system may also exploit the resources of remote CPUs through a communications network or some other data communications means. In operation, the CPU fetches, decodes, and executes the instructions from the computer readable storage medium. Information, such as the computer instructions and other computer readable data, is transferred between components of the computing system via the system's main data-transfer path.

[0042] In addition, the processing system may contain a peripheral communications controller and bus, which is responsible for communicating instructions from CPU to, and / or receiving datafrom, peripherals as discussed herein throughout. An example of a peripheral bus is the Peripheral Component Interconnect bus that is well known in the pertinent art.

[0043] An operator display, a user interface, and a graphical user interface (GUI), as discussed throughout, may be used to display visual output and / or presentation data generated by or at the request of processing system, such as responsive to operation of the aforementioned computing programs / applications / algorithms. Such visual output may include text, graphics, animated graphics, and / or video, for example. Moreover, these aspects may receive user input as discussed throughout, such as via peripherals connected to the peripheral bus.

[0044] Further, the processing system may contain a network adapter which may be used to couple to one or more communication networks, which may include or provide access to the Internet, an intranet, an extranet, or the like. Communications network may provide the processing system with means of communicating and transferring software, data and other information electronically. Network adaptor may communicate to and from the network using any available wired or wireless technologies. Such technologies may include, by way of nonlimiting example, wired Ethernet or fiber optic connections, cellular WAN infrastructures such as 3G, 4G / LTE, or 5G networks, Wi-Fi, Bluetooth®, Bluetooth® Low Energy (BLE), or Zigbee® links, infrared connections, or the like.

[0045] The disclosure remedies the issues with the known art by providing a laser triangulation camera defined by the use of a projected line from a SWIR illumination source. Further embodiments further refine this and related solutions.

[0046] Referring now to Figs. 1-2, a laser triangulation camera system 100 may comprise an illumination source unit 101 including a SWIR illumination source for emitting a light beam, with corresponding optics 102 to shape the beam into a line. The projection of the line may be fixed to a specific measurement position within the field of view 103.

[0047] An imaging unit 104 including at least one SWIR detector (also referred to as an image sensor or pixelated image detector) for detecting the SWIR light beam reflected by an object 105 is also part of the disclosure. The SWIR detector may include, but is not limited to, germaniumsilicon (GsSi), Indium gallium arsenide (InGaAs), and quantum dot SWIR detectors. A bandpass filter 106 may be inserted into the optical path between the SWIR camera lens and SWIR image sensor, limiting the photons to the SWIR wavelength(s) at or near the wavelength(s) of the illuminate source 101.

[0048] The deformation, or baseline shift, of the projected SWTR laser line in relation to the known line projection from the fixed geometry of the SWIR laser line projection to the SWTR detector, without an object, translates into the depth of the object. Corresponding algorithm(s) 107 are used to determine the height and profile of the object 105 based on known parameters of the camera 104, angle of illumination 101, angle of camera 104, baseline between camera 104 and illumination source 101, height of camera above platform, etc.

[0049] Basic triangulation in the embodiments is more clearly illustrated in FIG. 1. Shown is an object 105 in the field of view 103 on a base plate. The object 105 is illuminated by an illumination unit 101, such as SWIR laser illumination. Aspects of the illumination unit 101 may be controlled by controllable optics 102, such as may be controlled by the algorithms discussed throughout, resident in the illumination unit and / or at the image sensor. By way of example, the algorithms may determine depth based on camera angle, laser angle, baseline distance between camera and illumination source, camera height, laser height, and / or other geometric requirements to determine depth for laser triangulation, by way of example.

[0050] Thereby, the illumination may be provided according to algorithms 107, and sensed by an image sensor 104 (and / or by the illumination unit 101 itself) according to the algorithms 107. By way of example, the image sensor 104 may be a camera with SWIR sensing. Also by way of non-limiting example, the illumination unit 101 may include a VCSEL or EEL at or near one of the SWIR wavelength ranges of 1130nm (i.e. within 1100-1160nm, 1110-1150 nm, or 1120-1140nm), 1370nm (i.e. within 1340-1400nm, 1350-1390nm, or 1360-1380nm), or 1550nm (i.e. within 1520-1580nm, 1530-1570nm, or 1540-1560nm). FIG. 2 illustrates the referenced shift 108 of projected SWIR laser line resulting from baseline displacement due to the object 105 in the field of view 103, which may then be sensed.

[0051] FIG. 3 shows the display from the image sensor on a computer screen. This display provides an example of the sensing of the SWIR laser line shift with due to the contours of the object in the field of view.

[0052] There are numerous benefits resultant from using a SWIR illumination source for laser triangulation. Such benefits include: tolerance to ambient light from background sources such as lighting in a factory or warehouse, lights on a vehicle platform used during night-time operation, lights from other vehicles operating in the same field / mine / factory setting, or bright sunlight in an outdoor setting, for example; laser eye safety thresholds are much higher at SWIRwavelengths than at visible wavelengths and NIR wavelengths, and this results in the ability to use increased power levels of illumination increasing the SnR; in dusty or optically contaminated environments, increasing the laser power and / or increasing the width of the projected line increases the integrity of the projected line; and the application of the disclosed algorithm(s) to ‘fit’ the line based on partial line information, neighboring pixels, prior frame information, etc., allows for creation of the ‘best fit’ of the line. Thus, the issues with the known art are solved by using a SWIR illumination source and SWIR detector to reduce the impact of background illumination.

[0053] In some embodiments, the illumination unit 101 may include a second illumination source 101a, such as may emit SWIR flood illumination independent of and separable from the first line projection. In particular, the flood illumination may be used to capture additional data for edge detection, object detection, object identification, etc. Multiple illumination sources can thus be simultaneously combined with advantage for faster and more accurate depth sensing such as is shown in FIG. 4.

[0054] FIG. 5 illustrates that the illumination sources (101, 101a)(line illumination and flood illumination) may act independently, such as with modes for line illumination only, flood illumination only, and combined line illumination and flood illumination. That is, in multiple illumination source embodiments, the illumination sources (101, 101a) may be projected in different sequences: simultaneous projection of SWIR flood illumination source and additional wavelength(s) band illumination source(s); or alternating frames with SWIR flood illumination for one frame followed by alternative band illumination source for the subsequent frame, by way of example.

[0055] Additionally and alternatively, the embodiments may include an image / object sensor such as a multispectral camera. The multispectral camera may be at least one SWIR wavelength band supported, with a corresponding SWIR illumination source (such as in an operating band within the l.Omicrons to 1.7microns range). Additional supported wavelength bands for sensing may include a combination of visible wavelengths, NIR wavelengths, and thermal wavelengths and corresponding illumination source(s).

[0056] Additionally and alternatively, a hyperspectral camera with at least one SWIR wavelength band may be supported, such as with a corresponding SWIR illumination source (band within the 1.Omicrons to 1 ,7microns range). Additional supported wavelength bands mayinclude a combination of visible wavelengths, NTR wavelengths, and thermal wavelengths and corresponding illumination source(s). Again, the illumination sources may be projected in different sequences, such as: simultaneous projection of SWIR flood illumination source and additional wavelength(s) band illumination source(s); or alternating frames with SWIR flood illumination for one frame followed by alternative band illumination source for the subsequent frame

[0057] Additionally and alternatively, a SWIR laser triangulation camera combining a single image sensor may be used, such as in combination with at least two illumination sources of which one is a projected line for the laser triangulation and the second may be a flood illumination covering the FOV for a 2D image. The illumination sources again may be projected in different sequences: simultaneous projection of line illumination and flood illumination; alternating frames with line illumination for one frame and then flood illumination for the following frame; line illumination for all frames with alternating frames with flood illumination; or flood illumination for all frames with alternating frames for line illumination.

[0058] Yet further, an event-based vision sensor may be employed. For example, such eventbased vision sensor may include pixels receptive to SWIR photons. A SWIR illumination source may then be provided, projecting a line.

[0059] The aforementioned algorithms may increase the efficacy or integrity of the projected line during optically contaminated environments. The use of neighboring pixel data, best-fit algorithms, multiple frames, and / or combinations of prior mentioned algorithms may be used to determine line continuity in such cases. An optical path, i.e., a laser line, contaminated by organic matter, i.e., chipped wood, is illustrated in the example of FIG. 6.

[0060] Additionally and alternatively, a SWIR laser triangulation image sensor (traditional SWIR, multi spectral, hyperspectral, event-based vision sensor) may be corresponded with an illumination source(s) and an artificial intelligence (as part of the algorithms) targeted for a specific application. By way of non-limiting example, such specific applications may include: lumber grading and defect detection; fruit, vegetable, grain, etc. volumetric sizing and grading; seafood identification, sizing, grading; track, rail inspection and defect identification; road inspection and defect identification; box, package sizing in logistics operations; or object detection and identification in logistics, retail, manufacturing, etc.

[0061] Example 1 : A triangulation system, apparatus, method, and algorithms are disclosed. Included are: an illumination unit including a SWIR illumination source for emitting a light beam; optics capable of shaping the light beam into a line, wherein projection of the line is fixed to a specific measurement position within a field of view; an imaging unit including at least one SWIR detector configured to detect the light beam when reflected by an object in the field of view; a bandpass filter in an optical path of the imaging unit configured to limit detected photons by the imaging unit to a SWIR wavelength at or near a wavelength of the illumination unit; wherein deformation of the line in relation to an initial path of the line to the imaging unit translates into at least a depth of the object in the field of view; and wherein the depth allows for calculation by an application associated with at least one processor of at least a height and profile of the object based on known parameters of the system, including at least an angle of the line, and a distance between the imaging unit and the illumination source.

Claims

CLAIMSWhat is claimed is:

1. A triangulation system, comprising:an illumination unit including a first SWIR illumination source for emitting a light beam; optics capable of shaping the light beam into a line, wherein projection of the line is fixed to a specific measurement position within a field of view;an imaging unit including at least one SWIR detector configured to detect the light beam when reflected by an object in the field of view;wherein deformation of the line in relation to an initial path of the line to the imaging unit translates into at least a depth of the object in the field of view; andwherein the depth allows for calculation by an application associated with at least one processor of at least a height and profile of the object based on known parameters of the system, including at least an angle of the line, and a distance between the imaging unit and the illumination source.

2. The system of claim 1, further comprising a bandpass filter in an optical path of the imaging unit configured to limit detected photons by the imaging unit to a SWIR wavelength at or near a wavelength of the illumination unit.

3. The system of claim 1, wherein the illumination unit includes second a SWIR illumination source.

4. The system of claim 3, wherein the second SWIR illumination source is configured for flood illumination.

5. The system of claim 3, wherein the second SWIR illumination source comprises an light emitting diode (LED).

6. The system of claim 3, wherein the first and second SWIR illumination sources are configured to act independently.

7. The system of claim 6, wherein a first operating mode of the first and second SWIR illumination sources is configured to provide line illumination only.

8. The system of claim 6, wherein a second operating mode of the first and second SWIR illumination sources is configured to provide flood illumination only.

9. The system of claim 6, wherein a third operating mode of the first and second SWIR illumination sources is configured to provide combined line illumination and flood illumination.

10. The system of claim 1, wherein the SWIR detector comprises a germanium-silicon (GsSi) detector.

11. The system of claim 1, wherein the SWIR detector comprises an Indium gallium arsenide (InGaAs) detector.

12. The system of claim 1, wherein the SWIR detector comprises a quantum dot SWIR detector.

13. The system of claim 1, wherein the illumination unit comprises a vertical cavity edge emitting laser (VCSEL).

14. The system of claim 1, wherein the illumination unit comprises an edge emitting laser (EEL).

15. The system of claim I, wherein the illumination unit emits light near a wavelength of 1130nm.

16. The system of claim 1, wherein the illumination unit emits light near a wavelength of 1370nm.

17. The system of claim 1, wherein the illumination unit emits light near a wavelength of 1550nm.

Images