Structured lighting system

Abstract

According to one aspect of the present disclosure, a system is disclosed. The system may include a light assembly, an imager, and / or a controller. The light assembly may be configured to emit an array of lighting elements onto a scene. The imager may be configured to capture one or more images of the illumination element projected onto the scene. The controller may be configured to selectively activate the light assembly between a first state and a second state. In the first and second states, respectively, first and second sets of the lighting elements are selectively activated to emit first and second patterns of lighting elements.

Description

[0001] Cross-reference to related applications

[0002] This application claims priority under 35 U.S.C. §119(e) to U.S. Provisional Application No. 63 / 376,486, filed on September 21, 2022, entitled “Structured Lighting System,” the disclosure of which is incorporated herein by reference in its entirety. Technical Field

[0003] The present invention relates generally to imaging systems and, more particularly, to imaging systems with structured illumination. Summary of the invention

[0004] In accordance with the present disclosure, problems associated with structured lighting systems have been greatly reduced or eliminated.

[0005] According to one aspect of the present disclosure, a system is disclosed. The system may include a light assembly, an imager, and / or a controller. The light assembly may be configured to emit a linear array of lighting elements onto a scene. The imager may be configured to capture one or more images of the lighting elements projected onto the scene. The controller may be configured to selectively activate the light assembly between a first state and a second state. In the first state, a first group of lighting elements may be selectively activated to emit a first pattern of lighting elements. In the second state, a second group of lighting elements may be selectively activated to emit a second pattern of lighting elements.

[0006] In some examples, the first pattern of lighting elements can be a substantially linear pattern and the second pattern of lighting elements is a non-linear pattern. In some embodiments, the first group of lighting elements can be all or substantially all lighting elements. In some embodiments, the second group of lighting elements can be a subset of the first group of lighting elements. In other embodiments, the first group of lighting elements is a subset of the first group of lighting elements.

[0007] In other instances, the first and second patterns of lighting elements may be substantially linear patterns. In some embodiments, the second group of lighting elements may be a subset of the first group of lighting elements. In some embodiments, the first and second groups of lighting elements may correspond to substantially different areas of a scene. In some such embodiments, the first and second groups of lighting elements may substantially overlap a common area of ​​the scene. In other such embodiments, the first and second groups of lighting elements may correspond to mutually exclusive areas of the scene.

[0008] In other instances, the first and second patterns of lighting elements may be substantially non-linear patterns. In some embodiments, the second group of lighting elements may be a subset of the first group of lighting elements. In some embodiments, the first and second groups of lighting elements may correspond to substantially different areas of a scene. In some such embodiments, the first and second groups of lighting elements may overlap with a common area of ​​the scene. In other such embodiments, the first and second groups of lighting elements may correspond to mutually exclusive areas of the scene.

[0009] In some examples, the scene can correspond to a vehicle interior. In addition, the first and second groups of lighting elements can correspond to substantially different areas of the scene, such that the first group of lighting elements substantially corresponds to a driver area of ​​the scene and the second group of lighting elements substantially corresponds to a passenger area of ​​the scene.

[0010] In some embodiments, the controller may be further configured to selectively activate the light assembly between a third state in addition to the first and second states. In the third state, the third group of lighting elements may be selectively activated to emit a third pattern of lighting elements. In some embodiments, the second and third groups are subsets of the first group of lighting elements. In some such embodiments, the second and third groups of lighting elements may correspond to different areas of the scene. In some embodiments, the first pattern of lighting elements is a linear pattern.

[0011] After studying the following specification, claims and drawings, those skilled in the art will understand and appreciate these and other aspects, objects and features of the present disclosure. It will also be understood that the features of each embodiment disclosed herein can be used in combination with the features in other embodiments, or as a substitute for the features. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In the attached picture:

[0013] Figure 1 is a schematic representation of an embodiment of the system;

[0014] Figure 2 is a schematic representation of various aspects of the systems housed in the vehicle;

[0015] Figure 3A is a schematic representation of a light assembly in a first state, the light assembly activating a first group of light emitters to form a first group of lighting elements in a linear pattern;

[0016] Figure 3B is a schematic representation of a light assembly in a second state, the light assembly activating a second group of light emitters to form a second group of lighting elements in a quasi-random pattern;

[0017] Figure 3Cis a schematic representation of a light assembly in a third state, the light assembly activating a third group of light emitters to form a third group of lighting elements in a quasi-random pattern;

[0018] Figure 3D is a schematic representation of a light assembly in a fourth state, the light assembly activating a fourth group of light emitters to form a fourth group of lighting elements in a linear pattern;

[0019] Figure 3E is a schematic representation of a light assembly in a fifth state, the light assembly activating a fifth group of light emitters to form a fifth group of lighting elements in a linear pattern;

[0020] Figure 4A is a schematic representation of a first set of lighting elements forming a linear pattern that illuminates a static planar scene;

[0021] Figure 4B is a schematic representation of a second set of lighting elements forming a quasi-random pattern onto a static planar scene;

[0022] Figure 4C is a schematic representation of a third set of lighting elements forming a quasi-random pattern onto a static planar scene;

[0023] Figure 4D is a schematic representation of a fourth set of lighting elements forming a linear pattern onto a static planar scene; and

[0024] Figure 4E is a schematic representation of a fifth set of lighting elements forming a linear pattern onto a static planar scene. DETAILED DESCRIPTION

[0025] For the purposes of the description herein, the specific devices and processes shown in the drawings and described in the present disclosure are merely exemplary embodiments of the inventive concepts defined in the appended claims. Therefore, unless the claims explicitly state otherwise, the specific features associated with the embodiments disclosed herein are not intended to be limiting.

[0026] This disclosure is aimed at Figure 1 The system 100 shown, and Figure 1-4E The components thereof are shown. The system 100 may include a light assembly 110, an imager 120, and / or a controller 130. In some embodiments, the system 100 may be associated with a vehicle 10. Specifically, the system 100 may be generally disposed in a cabin 11 of the vehicle 10. Accordingly, the system 100 may be a driver and / or cabin monitoring system.

[0027] The light assembly 110 can be configured to emit light 20 having a first wavelength range. In some embodiments, the light assembly 110 can be part of a rearview assembly of the vehicle 10. Examples of the light assembly 110 are shown in FIG. Figure 3A -E. Light can be emitted by one or more light emitters 111. In addition, the light assembly 110 may have an optical element 112 for receiving and / or transmitting light from the one or more light emitters 111. The emitted light can illuminate at least a portion of the scene 30. The illumination can be at a low power level that is safe for the eyes. In some embodiments, the scene 30 can be a vehicle interior. Accordingly, the scene 30 can correspond to, for example, a passenger compartment, a driver's seat, a front passenger seat, a front row, and / or a second row of the vehicle 10. In some embodiments, the scene 30 can generally correspond to the field of view of the imager 120. The first wavelength range can include or be generally limited to light in the infrared and / or near infrared region of the electromagnetic spectrum. In some embodiments, the first wavelength range can be generally centered at 810, 850, or 940 nm.

[0028] In addition, the light 20 emitted by the light assembly 110 may be structured light. Accordingly, the emitted light 20 may illuminate the scene with the pattern of the lighting elements 113. Figure 4A -E is shown on a static planar scene 30 to show various lighting element patterns. The lighting element 113 can be, for example, a point, a spot, a line, a shape and / or a combination thereof. In this way, the lighting element 113 can be projected onto the scene 30 and onto objects therein. In some embodiments, the pattern of the lighting element 113 can be a substantially linear array. In other embodiments, the pattern of the lighting element 113 can be non-linear. In this way, the pattern of the lighting element 113 can be quasi-random. In addition, the pattern of the lighting element can be divided into two or more subgroups of the lighting element 113. Each subgroup can be different from each other. In some embodiments, the subgroups can mutually exclude each other. In other embodiments, the subgroups may not mutually exclude each other. In some embodiments, the emitted light 20 of the lighting element 113 can be substantially in phase and / or coherent.

[0029] In some embodiments, one or more of the light emitters 111 may be lasers. Light emitters 111 may be, for example, one or more vertical cavity surface emitting lasers (VCSELs). Accordingly, light assembly 110 may include an array of VCSELs. In some such embodiments, the VCSELs may be single-mode VCSELs.

[0030] In addition, the optical assembly 110 can have one or more optical elements 112. The optical element 112 can be diffuse, diffractive and / or refractive. In some embodiments, the optical element 112 can be configured to split one or more light beams into a structured light pattern. Therefore, the optical element 112 can be a diffraction grating. In some embodiments, one or more of the light emitters 111 can be associated with and / or disposed in the optical element 112 substrate. The optical element 112 substrate can be, for example, gallium arsenide (GaAs). Accordingly, the VCSEL array can be disposed in a GsAs substrate.

[0031] In addition, if Figure 3A -E, individually or in groups, the light emitters 111 can be selectively activated and various lighting element 113 patterns can be achieved, such as the corresponding Figure 4A -E. Therefore, the light emitters 111 may be addressable. In this way, the light assembly 110 may be configured to selectively activate one or more groups of light emitters 111. Therefore, the corresponding lighting elements 113 emitted by the corresponding light emitters 111 may be selectively emitted together and / or individually. Accordingly, the light assembly 100 may be configured to operate between a first state, a second state, a third state, a fourth state, and / or a fifth state by activating corresponding groups in a set of groups of lighting elements 111. In some embodiments, a scene 30 may be defined as an area that the lighting elements may substantially cover when all lighting elements 113 are activated.

[0032] In the first state, as shown in FIG3a, the light assembly 110 can be configured to emit light 20 forming a first group of lighting elements 113. In addition, the first group can be all or substantially all lighting elements 113, as shown in FIG4a. In some embodiments, the first group of lighting elements 113 can be arranged in a substantially linear pattern. In other embodiments, the first group of lighting elements 113 can be arranged in a substantially nonlinear pattern. In this way, the second group of lighting elements 113 can be arranged in a quasi-random pattern. In some embodiments, the scene 30 can be defined as an area substantially covered by the lighting elements 113 when the light assembly 110 is in the first state.

[0033] In the second state, as shown in FIG3b, the light assembly 110 can be configured to emit light 20 forming a second group of lighting elements 113, as shown in FIG4b. The second group can be substantially less than all lighting elements 113. Accordingly, in some embodiments, the second group of lighting elements 113 can be smaller than the first group of lighting elements 113 and / or included in the first group of lighting elements. In this way, the second group of lighting elements 113 can be a subset of the lighting elements 113 of the first group of lighting elements 113. In addition, the second group of lighting elements 113 can be arranged in a substantially nonlinear pattern. In this way, the second group of lighting elements 113 can be arranged in a quasi-random pattern. In some embodiments, the second group of lighting elements 113 can substantially span the eternal scene 30. In other embodiments, the second group of lighting elements 113 can be substantially confined to a first portion of the scene 30. The first portion of the scene 30 can be substantially smaller than the entire scene 30.

[0034] In a third state, as shown in FIG. 3c , the light assembly 110 may be configured to emit light forming a third group of lighting elements 113, as shown in FIG. 4c . The third group of lighting elements 113 may be substantially less than all lighting elements 113. Accordingly, in some embodiments, the third group of lighting elements 113 may be smaller than the first and / or second group of lighting elements 113 and / or be included in the first and / or second group of lighting elements. Thus, the third group of lighting elements 113 may be a subset of the lighting elements 113 of the first and / or second group of lighting elements 113. Furthermore, the third group of lighting elements 113 may be arranged in a substantially nonlinear pattern. Thus, the third group of lighting elements 113 may be arranged in a quasi-random pattern.

[0035] In addition, the third group of lighting elements 113 can be substantially different from the second group of lighting elements 113. In some embodiments, the third group of lighting elements 113 can be substantially equal in size to the second group of lighting elements 113. In addition, in some embodiments, the third group of lighting elements 113 can be substantially confined to a second portion of the scene 30. The second portion of the scene 30 can be substantially smaller than the permanent scene 30. In addition, the second portion of the scene 30 can be substantially different from the first portion of the scene 30. In addition, the first and second portions of the scene 30 can substantially overlap or substantially exclude each other. Accordingly, the second and third groups of lighting elements 113 can have some common lighting elements 113 or exclude each other.

[0036] In some embodiments, the second and third groups of lighting elements 113 together can be substantially equivalent to the first group of lighting elements 113. As such, the second and third groups of lighting elements 113 together can include all or substantially all lighting elements 113. In other embodiments, the second and third groups of lighting elements 113 together can be equivalent to substantially less than all lighting elements 113.

[0037] In a fourth state, as shown in FIG. 3d , the light assembly 110 may be configured to emit light 20 forming a fourth group of lighting elements 113, as shown in FIG. 4d . The fourth group of lighting elements 113 may also be substantially less than all lighting elements 113. Accordingly, in some embodiments, the fourth group of lighting elements 113 may be smaller than the first group of lighting elements 113 and / or included in the first group of lighting elements. In this way, the fourth group of lighting elements 113 may be a subgroup of lighting elements 113 of the first group of lighting elements 113. In addition, the fourth group of lighting elements 113 may be arranged in a substantially linear pattern. In some embodiments, the fourth group of lighting elements 113 may include a selection of adjacent lighting elements 113 from the first group of lighting elements 113. In other embodiments, the fourth group of lighting elements 113 may include a selection of non-adjacent lighting elements 113 from the first group of lighting elements 113. For example, the fourth group of lighting elements 113 may be equivalent to every other or every other, etc., of the area of ​​the lighting elements 113 of the first group. In some embodiments, the fourth group of lighting elements 113 may be substantially confined to a third portion of the scene 30. The third portion of scene 30 may be substantially smaller than the entire scene 30. In some embodiments, the third portion may be substantially equal to the first or second portion of scene 30. In other embodiments, the third portion may be substantially different from the first and second portions of scene 30.

[0038] In the fifth state, as shown in FIG. 3e, the light assembly 110 may be configured to emit light 20 forming a fifth group of lighting elements 113, as shown in FIG. 4e. The fifth group of lighting elements 113 may also be substantially less than all lighting elements 113. Accordingly, in some embodiments, the fifth group of lighting elements 113 may be smaller than the first group of lighting elements 113 and / or included in the first group of lighting elements. In this way, the fifth group of lighting elements may be a subgroup of the lighting elements 113 of the first group of lighting elements 113. In addition, the fifth group of lighting elements 113 may be arranged in a substantially linear pattern. In some embodiments, the fourth group of lighting elements 113 may include a selection of adjacent lighting elements 113 from the first group of lighting elements 113. In other embodiments, the fourth group of lighting elements 113 may include a selection of non-adjacent lighting elements 113 from the first group of lighting elements 113. For example, the lighting elements 113 of the fourth group may be equivalent to every other or every other, etc., of the area of ​​the lighting elements 113 of the first group.

[0039] In addition, the fifth group of lighting elements 113 can be substantially different from the fourth group of lighting elements 113. In some embodiments, the fifth group of lighting elements 113 can be substantially equal in size to the fourth group of lighting elements 113. In addition, in some embodiments, the fifth group of lighting elements 113 can be substantially confined to a fourth portion of the scene 30. The fourth portion can be substantially smaller than the permanent scene 30. In some embodiments, the fourth portion of the scene 30 can be substantially equal to the first or second portion of the scene 30. In other embodiments, the fourth portion of the scene 30 can be substantially different from the first and second portions of the scene. In addition, the fourth portion of the scene 30 can be different from the third portion of the scene 30. In addition, the third and fourth portions of the scene 30 can substantially overlap or substantially exclude each other. Accordingly, the fourth and fifth groups of lighting elements 113 can have some common lighting elements 113 or exclude each other.

[0040] In some embodiments, the fourth and fifth groups of lighting elements 113 together may include all or substantially the same as the first group of lighting elements 113. Thus, the fourth and fifth groups of lighting elements 113 together may include all or substantially all lighting elements 113. In other embodiments, the fourth and fifth groups of lighting elements 113 together may be equivalent to substantially less than all lighting elements 113.

[0041] Imager 120 can be any device configured to capture light in a second wavelength range and generate one or more corresponding images. For example, imager 120 can be a camera. Accordingly, imager 120 can be a pixel sensor of a semiconductor charge coupled device (CCD) or complementary metal oxide semiconductor (CMOS) technology. The second wavelength range can include the entirety or part of the first wavelength range. Therefore, the second wavelength range can include or be substantially limited to light in the infrared and / or near infrared region of the electromagnetic spectrum. In some embodiments, the second wavelength range can be substantially centered at 810, 850, or 940 nm. In some embodiments, imager 120 can have a field of view of a scene. In addition, imager 120 can be configured to generate an image based at least in part on illumination reflected from a scene and / or an object within the scene from light assembly 110. Accordingly, the one or more images can capture one or more of the lighting elements 113 irradiated onto the scene 30. In some embodiments, imager 120 can be associated with an interior rearview assembly of vehicle 10.

[0042] The controller 130 may be communicatively connected to the imager 120 and / or the light assembly 110. Accordingly, the controller 130 may be configured to switch the light assembly 110 between the first state, the second state, the third state, the fourth state, and / or the fifth state. In addition, the controller 130 may be configured to receive the one or more images from the imager 120. In addition, the controller 130 may include a memory 131 and a processor 132.

[0043] The memory 131 may be a non-transitory computer readable medium (CRM). Therefore, the memory 131 may be a tangible device that may be configured to store one or more instructions, such as one or more algorithms, to provide configuration and operation of the controller 130. Examples of the memory 131 include a conventional hard disk, a solid-state memory, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), an electronically erasable programmable read-only memory (EEPROM), an optical or magnetic disk, and a dynamic random access memory (DRAM).

[0044] The processor 132 may be communicatively connected to the memory 131. In addition, the processor 132 may be any device or electronic circuit configured and / or operable to process or execute one or more sets of electronic instructions, such as algorithms. These instructions may be stored in the memory 131. Examples of the processor 132 may include a central processing unit (CPU), a microprocessor, and / or an application specific integrated circuit (ASIC).

[0045] In some embodiments, controller 130 may be configured to create a depth map of scene 30. Controller 130 may be configured to compare the one or more images with a calibration image showing lighting element 113 illuminating scene 30. The calibration image may provide controller 130 with an understanding of the pattern of structured light. Depth data may be extracted from the one or more images based on triangulation of one or more reflective lighting elements relative to other reflective lighting elements. For example, controller 130 generates a depth map according to the teachings of U.S. Pat. No. 11,310,466, which is incorporated herein by reference.

[0046] In some embodiments, the controller 130 may be configured to detect vibrations of a surface illuminated by the lighting elements 113 based at least in part on the one or more lighting elements 113 captured in the image. Due to constructive and / or destructive interference, vibration or movement of the surface may cause the associated lighting elements 113 to change their reflected intensity. In this way, the reflecting lighting elements 113 may form a changing speckle pattern. Therefore, surface vibrations can be derived from changes or shifts in the light intensity of one or more of the lighting elements 113 in a series of images. For example, the controller 130 may be configured to track the intensity of each of the one or more lighting elements 113 over time and analyze changes thereto. In an embodiment where the system 100 is fixedly attached to the vehicle, the system 100 vibrates with it and thus will only measure motion relative to itself. For example, the controller 130 may detect vibrations according to the teachings of U.S. Pat. No. 11,310,466, which is incorporated herein by reference.

[0047] Embodiments of the present disclosure may have various advantages. For example, embodiments in which the light assembly 110 is configured to emit the lighting elements 113 in a linear pattern in one state and in a nonlinear pattern in another state may specifically have the advantage of utilizing the functionality achieved by each of these lighting patterns in the case of a single light assembly without having to select pattern type or otherwise. For example, a nonlinear pattern may provide depth mapping by allowing triangulation or positioning of a surface illuminated by the lighting element 113 based on the unique relationship of the lighting element 113 to other lighting elements 113. Additionally, a linear pattern may provide an increased number of lighting elements 113, thereby giving greater sensitivity and / or accuracy to vibration sensing, as each point may provide its own measurement. Furthermore, a linear pattern may be used for 2D imaging due to its more uniform illumination relative to a nonlinear patterned light assembly. Furthermore, in embodiments where light assembly 110 is configured to illuminate in two states corresponding to different regions of a scene, system 100 may have the advantage of being operable to perform depth mapping or vibration sensing across an entire scene, but also being configured to focus on a region of interest and reduce or eliminate exposure of other regions of the scene to illumination. This may be particularly beneficial in instances where a person or animal occupies part of the scene.

[0048] In this document, relational terms such as "first," "second," and the like are used merely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions.

[0049] As used herein, the term "and / or" when used in a list of two or more items means that any one of the listed items may be taken alone, or any combination of two or more of the listed items may be taken. For example, if a composition is described as containing components A, B, and / or C, the composition may contain: only A; only B; only C; a combination of A and B; a combination of A and C; a combination of B and C; or a combination of A, B, and C.

[0050] The terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements does not include only those elements, but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element preceded by "comprises..." does not preclude the presence of additional identical elements in the process, method, article, or apparatus that includes the element, without further constraints.

[0051] The term "substantially" and variations thereof should be understood by one of ordinary skill in the art to describe equal or approximately equal values ​​or features of the description. For example, a "substantially planar" surface is intended to mean a planar or approximately planar surface. In addition, "substantially" is intended to mean that two values ​​are equal or approximately equal. If there is a use of the term that is unclear to one of ordinary skill in the art, given the context in which the term is used, "substantially" may mean values ​​that are within about 10% of each other, such as within about 5% of each other, or within about 2% of each other.

[0052] As used herein, "communicatively connected" may mean connected either directly or indirectly through one or more electrical components.

[0053] It should be understood that while several embodiments have been described in this disclosure, those skilled in the art may understand numerous variations, changes, transformations and modifications, and that this disclosure is intended to encompass such variations, changes, transformations and modifications within the scope of the appended claims unless the language thereof expressly states otherwise.

Claims

1. A system comprising: a light assembly configured to emit the array of lighting elements onto a scene; an imager configured to capture one or more images of the lighting element projected onto the scene; as well as A controller configured to selectively activate the light assembly between the following two states: a first state in which a first group of said lighting elements are selectively activated to emit a first pattern of lighting elements, and and a second state in which a second group of said lighting elements are selectively activated to emit a second pattern of said lighting elements. 2 . The system of claim 1 , wherein the first pattern of lighting elements is a substantially linear pattern and the second pattern of lighting elements is a non-linear pattern.

3. The system of claim 2, wherein the first group of lighting elements is all or substantially all of the lighting elements.

4. The system of claim 2, wherein the second second group of lighting elements is a subset of the first group of lighting elements.

5. The system of claim 2, wherein the first group of lighting elements is a subset of the first group of lighting elements.

6. The system of claim 1, wherein the first and second patterns of lighting elements are substantially linear patterns.

7. The system of claim 6, wherein the second group of lighting elements is a subset of the first group of lighting elements.

8. The system of claim 6, wherein the first and second groups of lighting elements correspond to substantially different areas of the scene.

9. The system of claim 8, wherein the first and second groups of lighting elements substantially overlap a common area of ​​the scene.

10. The system of claim 8, wherein the first and second groups of lighting elements correspond to mutually exclusive areas of the scene.

11. The system of claim 1, wherein the first and second patterns of lighting elements are substantially non-linear patterns.

12. The system of claim 11, wherein the second group of lighting elements is a subset of the first group of lighting elements.

13. The system of claim 11, wherein the first and second groups of lighting elements correspond to substantially different areas of the scene.

14. The system of claim 13, wherein the first and second groups of lighting elements generally extend over a common area of ​​the scene.

15. The system of claim 13, wherein the first and second groups of lighting elements correspond to substantially mutually exclusive areas of the scene.

16. The system of claim 1, wherein: The scene corresponds to the interior of a vehicle; the first and second groups of lighting elements correspond to substantially different areas of the scene; The first set of lighting elements generally corresponds to a driver area of ​​the scene; and The second group of lighting elements generally corresponds to a passenger area of ​​the scene.

17. The system of claim 1, wherein: The controller is further configured to selectively activate the light assembly to a third state; In the third state, a third group of lighting elements are selectively activated to emit a third pattern of lighting elements; and The second and third groups are subsets of the first group of lighting elements.

18. The system of claim 17, wherein the second and third groups of lighting elements correspond to different areas of the scene.

19. The system of claim 17, wherein the first pattern of lighting elements is a linear pattern.

20. The system of claim 19, wherein the second pattern of lighting elements is a non-linear pattern.

Images