Device and method for producing and imaging with structured light

Abstract

Disclosed herein is a structured light device and associated systems and methods. The device comprises a light source, a digital micromirror device (DMD), and a control device coupled to the DMD. The light source is configured to emit light onto the DMD. A micromirror array of the DMD reflects the light incident on the DMD out of the structured light device, when at least one mirror of the DMD is in a neutral position. The DMD is configured to dynamically vary the configuration of the DMD mirrors.

Description

DEVICE AND METHOD FOR PRODUCING AND IMAGING WITH STRUCTURED LIGHTTechnical Field

[0001] The following relates generally to a device and associated method for the production and application of structured light, and specifically, a device, and associated method, comprising a light source and digital micromirror device for producing structured light to minimize specular reflections.Introduction

[0002] Visible or infra-red light may be used in imaging applications, such as flash LIDAR or structured light sensors, to illuminate objects for imaging by visible light cameras. In some applications, specular reflections off illuminated objects may result in distortion, saturation or glare of the imaging camera, resulting in ineffective imaging.

[0003] Image processing techniques may be applied to reduce the negative impact of such specular reflections, however, it is advantageous to avoid such specular reflections altogether, as the data captured by the imaging camera will inherently be limited by such reflections.

[0004] Accordingly, there is need for a device and method which may avoid the effect of these specular reflections.Summary

[0005] Described herein is a structured light device, according to an embodiment. The device includes a light source for emitting light onto a DMD, a digital micromirror device (DMD) for producing patterns of light, and a control device for controlling the mirror configuration of the DMD, coupled to the DMD. The light source is configured to emit light onto the DMD. A micromirror array of the DMD reflects the light incident on the DMD out of the structured light device, when at least one mirror of the DMD is in a neutral position. The DMD is configured to dynamically vary a mirror configuration to reduce specular reflections.

[0006] According to some embodiments, the light source comprises a laser.

[0007] According to some embodiments, the DMD mirror configuration is varied according to feedback received by a detection device, and transmitted to the structured light device.

[0008] According to some embodiments, the DMD mirror configuration is dynamically varied to maximize resolution of identifiable features of a target object.

[0009] According to some embodiments, the DMD mirror configuration corresponds to a sequence of X image projections to maximize the detection of orthogonal features.

[0010] According to some embodiments, the DMD mirror configuration is varied by iterating through a preset pattern of mirror configurations.

[0011] According to some embodiments, the DMD mirror configuration is varied by moving mirrors corresponding to specular reflection locations detected by the detection device into a deflected position.

[0012] According to some embodiments, the DMD mirror configuration is varied according to a binary iterative search algorithm.

[0013] According to some embodiments, the structured light device is coupled to the detection device.

[0014] Described herein is a system for structured light imaging. The system includes a structured light device, and a detection device for receiving reflected light from the structured light device. The structured light device is configured to direct structured light onto a target object. The detection device is configured to receive the structured light deflected off the target object. The structured light device is configured to dynamically vary a mirror configuration to reduce specular reflections according to the structured light received by the detection device.

[0015] Described herein is a method of imaging using structured light. The method includes directing light towards a target object with a structured light device, receiving light reflected off the target object with a detection device, and generating a structured light pattern for the structured light device based on the reflection of the light received by the detection device.

[0016] Other aspects and features will become apparent, to those ordinarily skilled in the art, upon review of the following description of some exemplary embodiments.Brief Description of the Drawings

[0017] The drawings included herewith are for illustrating various examples of articles, methods, and apparatuses of the present specification. In the drawings:

[0018] Figure 1 is a block diagram of a structured light device, according to an embodiment;

[0019] Figure 2 is a block diagram of the digital micromirror device of the device of Figure 1 , and an associated structured light output, according to an embodiment;

[0020] Figure 3 is a block diagram of the device of Figures 1-2, applied to an imaging system, according to an embodiment;

[0021] Figure 4 is a block diagram of a structured light device, according to another embodiment;

[0022] Figure 5 is an example output of data which may be generated by the system of Figure 3, according to an embodiment; and

[0023] Figure 6 is a flow chart depicting a method of using the device of Figures 1 - 3 according to an embodiment;

[0024] Figure 7 is a flow chart depicting a method 600 of using the device of Figures 1-3 according to another embodiment.Detailed Description

[0025] Various apparatuses or processes will be described below to provide an example of each claimed embodiment. No embodiment described below limits any claimed embodiment and any claimed embodiment may cover processes or apparatuses that differ from those described below. The claimed embodiments are not limited to apparatuses or processes having all of the features of any one apparatus or process described below or to features common to multiple or all of the apparatuses described below.

[0026] Further, although process steps, method steps, algorithms or the like may be described (in the disclosure and I or in the claims) in a sequential order, such processes, methods and algorithms may be configured to work in alternate orders. In other words, any sequence or order of steps that may be described does not necessarily indicate a requirement that the steps be performed in that order. The steps of processes described herein may be performed in any order that is practical. Further, some steps may be performed simultaneously.

[0027] When a single device or article is described herein, it will be readily apparent that more than one device I article (whether or not they cooperate) may be used in place of a single device I article. Similarly, where more than one device or article is described herein (whether or not they cooperate), it will be readily apparent that a single device I article may be used in place of the more than one device or article.

[0028] Described generally herein is a device and method for producing structured light output, for the purpose of avoiding specular reflections stemming from the projection of such structured light. Such avoidance of reflection may be advantageous in imaging applications, such as flash LIDAR, structured light imaging or actively illuminated imaging. The device described herein comprises a light source, a digital micromirror device (DMD), and a control device coupled to the digital micromirror device. The light source is directed towards the DMD, which reflects and directs the light originating from the light source through an aperture. In some examples, the device may comprise optics between the light source and DMD and / or DMD and aperture. In some examples, the device may be sealed.

[0029] Referring first to Figure 1 , pictured therein is a system block diagram of a structured light device 100, according to an embodiment. The device 100 comprises a light source 102, DMD 104, aperture 106, housing 108, and control device 110.

[0030] Light source 102 comprises any device or apparatus which may produce and output electromagnetic radiation primarily in the visible spectrum. Light source 102 may comprise a laser, LED, halogen light source, fluorescent light source, or any other suitable source. In some embodiments, a laser, LED, or other source producing a specificor narrow bandwidth of visible light, near IR or other wavelength range output may be preferable.

[0031] DMD 104 comprises a microelectromechanical system device having an array of micromirrors, each preferable individually controllable. On the input or reception of a control signal, each micromirror may be adjusted in position. Each micromirror may be adjusted from a neutral position to a deflected position. In a neutral position, the micromirrors are aligned in a flat plane. Mirrors in a neutral position may direct light along the path of light 114, while mirrors in the deflected position may direct light along the path of light 116. Light 114 and 116 are present for illustration purposes. In other examples, light deflected off of mirrors in neutral and deflected positions may follow differing paths than those illustrated herein.

[0032] DMD 104 is positioned such that light emitted by light source 102 will be incident on the mirror array of DMD 104 and reflect this incident light (e.g. light 114) away, towards aperture 106, when mirrors are in the neutral position.

[0033] Aperture 106 comprises an opening or area which structured light may be directed outwards from device 100. Aperture 106 is positioned on or within device 100 such that light source 102 directs light towards DMD 104, and light is reflected off of DMD 104 and through aperture 106 when mirrors of DMD 104 are in a neutral position. When mirrors of DMD 104 are in a deflected position, light incident on DMD 104 from light source 102 will not pass through aperture 106, and will instead be directed elsewhere (e.g. along the path of light 116).

[0034] In some examples of device 100, light reflected by deflected mirrors of DMD 104 will be captured by a deflected light target (e.g. deflected light target 112). Such a deflected light target may be configured to receive deflected light, and absorb the light, minimizing or eliminating internal reflections of the deflected light.

[0035] Housing 108 comprises a mechanical enclosure, holding the components of device 100 described herein, and any other necessary components. Housing 108 may be configured to shield the components of device 100 from dust, debris, vibration, shock, water and / or radiation damage, or any other environmental damage. In some examples,housing 108 may be constructed from polymer, aluminum, steel, titanium, composite, ceramic or any other suitable material.

[0036] Control device 110 comprises a microcontroller, processor, or other similar device coupled to DMD 104, such that each individual mirror of the DMD 104 may be addressed and controlled. For example, control device 110 may provide a signal to DMD 104, such that certain individual mirrors of DMD 104 are placed in neutral positions, and other individual mirrors are place in deflected positions. In some examples, DMD 104 may be further configured to place individual mirrors in intermediate positions, resulting in partially attenuated light corresponding to the position of mirrors in intermediate positions. In some examples, intermediate positions may comprise high frequency oscillations of mirrors between neutral and deflected positions.

[0037] Control device 110 may be further configured to receive an external signal (e.g. from detection device 124 to which it may be coupled), directing a certain structured light pattern of device 100. In some examples, control device 110 may not be present within device 100, or may be external to device 100.

[0038] In some embodiments, device 100 may comprise additional components. For example, optics (such as mirrors, lenses, filters such as laser line filters and bandpass filters, and diffraction gratings) may be present between light source 102 and DMD 104 and / or DMD 104 and aperture 106 and / or at aperture 106.

[0039] Referring now to Figure 2, pictured therein is a depiction of DMD 104 micromirror array, comprising micro mirrors 122A-F. The DMD of Figure 2 comprises a 6- mirror array, each mirror numbered 112A-F. Mirrors 112A, B, C, D, and F are each in the neutral position in Figure 2, while mirror 112E is in the extreme deflected position.

[0040] When light is reflected off DMD 104 when in the configuration of Figure 2, a light pattern such as light pattern 120 of Figure 2 may be projected through aperture 106. Mirrors in neutral position direct light outwards through aperture 106, while the mirror 112E in the deflected position direct light towards deflected light target 112. The resulting structured light pattern 120 is proportional to the mirror deflection pattern of the DMD 104. By adjusting the mirror deflection pattern, various structured light patterns may beproduced. DMDs with greater resolution (i.e. more micro mirrors) may generate a greater variety of structured light patterns.

[0041] When all mirrors of the DMD 104 are in the neutral position, the light produced by device 100 may be referred to as full range light. In some configurations, full range light may refer to the initial structured light pattern produced by DMD 104, before a control signal is provided to DMD 104 to adjust the structured light pattern according to feedback received from another device.

[0042] The DMD mirror configuration may be dynamically varied to maximize resolution of identifiable features of a target object.

[0043] The DMD mirror configuration may correspond to a sequence of X image projections to maximize the detection of orthogonal features. The light may have a preset pattern. The preset pattern may include a sequence of X images having two diagonal lines orthogonal to each other and at + / - 45 degrees to the axis separating the detection device 124 and the structed light device 100. This has the advantage of increasing probability of detection of features on two orthogonal axis, and consequent improvement in pose estimation of the target object 118.

[0044] The configuration of the DMD 104 may be varied based on feedback from the control device 110 to maximize accuracy of pose estimation of the target object 118, and / or minimize the number of images to determine the target pose.

[0045] The DMD may be configured to dynamically vary a mirror configuration according to a strategy. This strategy could be to reduce specular reflections, to maximize the resolution of measurement of key features of a target, and / or to minimize the time to determine the pose of the target.

[0046] Referring now to Figure 3, pictured therein is a block diagram depicting a system 200 for imaging with structured light, for the reduction of specular reflection. The system comprises structured light device 100, target object 118 and detection device 124. Device 100 comprises the device of Figures 1-2.

[0047] Detection device 124 may comprise a visible light camera, such as a CMOS or CCD based camera, a photodiode array or other imaging device. When device 100 isconfigured to output a narrow wavelength band, detection device 124 may be fitted with a corresponding band pass filter, blocking out undesirable wavelengths.

[0048] In operation of system 200, device 100 directs light towards target object 118. This directed light is reflected off of object 118, and directed back towards detection device 124. Detection device 124 measures and records this reflected light. In some examples, specular reflections are present in the data recorded by detection device 124. This data captured by device 124 may be processed to determine the presence of specular reflections. If specular reflections are present, a feedback or control signal may be passed to device 100 (e.g. to control device 110), to adjust the mirror configuration of the DMD 104 of the device 100.

[0049] Once this feedback signal is received, the device 100 outputs and directs a structured light pattern towards target object 118, which is again reflected and received by detection device 124. The signal received by detection device 124 is again processed to identify the presence of specular reflections within the captured data. If specular reflections are detected again, a feedback or control signal may again be passed to device 100 (e.g. to control device 110), to adjust the mirror configuration of the DMD 104 of the device 100, to a new untested structured light pattern.

[0050] This process may be repeated until no specular reflections are detected, or until a structured light pattern that minimizes specular reflections is determined.

[0051] In some examples, the initial illumination of target object 118 may comprise full range light. In other examples, the initial illumination may comprise a specific structured light pattern.

[0052] In some examples, the signal received by device 124 may be analyzed to determine a specific structured light pattern that will minimize specular reflections, according to the patterns of specular reflections detected by device 124. For example, if specular reflections are detected in the upper left quadrant of the image recorded by device 124, corresponding mirrors of DMD 104 may be actuated to the deflected position, to reduce or eliminate specular reflections. Structured light produced by this new DMD mirror pattern may be directed towards target object 118, and imaged by device 124 to confirm the presence or absence of specular reflections.

[0053] In some examples, this analysis may comprise the application of a trained machine learning model, such as a neural network.

[0054] In some examples, this analysis may comprise an iterative binary search algorithm.

[0055] In some examples, structured light patterns may be varied at high frequency, until a pattern that minimizes specular reflections (according to feedback provided by device 124) is determined. In such examples, every mirror may be individually actuated to identify mirrors contributing to specular reflections. These identified mirrors may then be placed in the deflected position, to minimize specular reflections.

[0056] Referring now to Figure 4, pictured therein is a block diagram of a structured light device 300, according to an embodiment. Components of device 300 may be analogous to components of device 100, with reference characters incremented by 200.

[0057] Device 300 comprises laser 302, DMD 304, aperture 306, housing 308, and control device 310.

[0058] Laser 302 comprises a near I laser.

[0059] DMD 304 comprises a 1024x768 mirror array DMD.

[0060] Housing 308 may include an aluminum (or cast aluminum) housing. Housing 308 is insulated against heat and radiation. Housing 308 is further configured to radiate heat produced by device 300 away from device 300. In some examples, housing 308 or other components may be coupled to laser 302 and / or DMD 304 to act as a heat sink for these components. Components of device 300 are mounted to housing 308 in a manner that reduces vibrational loads on each component.

[0061] Control device 310 is coupled to DMD 304, and configured to transmit control signals to DMD 304. Control device 310 comprises a microcontroller, which is configured to receive an external control signal, convert this signal to the appropriate format for the control of DMD 304 and transmit this signal to DMD 304. In other examples, the control device may comprise a FPGA or ASIC.

[0062] Aperture 306 comprises a light transparent lens.

[0063] Device 300 may be configured to be used with a photodiode array detection device, fitted with a bandpass filter centered on the wavelength of the transmitted light source.

[0064] In operation of device 300, laser 302 directs light towards DMD 304. DMD 304 directs light 314 through aperture 306, towards target object 318. Target object 318 may comprise a spacecraft. Light reflected off of object 318 may be captured by device 324, which may provide feedback to control device 310.

[0065] Referring now to Figure 5, pictured therein is an example output 400, such as may be captured by system 200. Data captured by detection device 124 may be assembled into a two-dimensional image, three-dimensional point cloud, three- dimensional mesh or three- dimensional solid model of the target object. By applying the system 200 of Figure 3, objects may be imaged in three dimensions, while minimizing or eliminating specular reflections which may impact the quality of target object imaging.

[0066] Referring now to Figure 6, pictured therein is a flow chart describing a method 500 of using the structured light device. Method 500 comprises 502, 504, and 506.

[0067] At 502, full range light is directed towards a target object with a structured light device.

[0068] At 504, full range light reflected off the target object is received by a detection device.

[0069] At 506, a structured light pattern is generated for the structured light device based on the reflection of the full range light received by the detection device.

[0070] Referring now to Figure 7, pictured therein is a flow chart describing a method 600 of using the structured light device. Method 600 comprises 602, 604, and 606, as well as the steps of method 500.

[0071] At 602, structured light is directed towards a target object with a structured light device.

[0072] At 604, structured light reflected off the target object is received with a detection device.

[0073] At 606, an updated structured light pattern for the structured light device is generated based on the reflection of the structured light received by the detection device, to reduce specular reflections.

[0074] After the completion of 606, method 600 may proceed back to 602, and repeat as necessary, until specular reflections are minimized sufficiently.

[0075] While the above description provides examples of one or more apparatus, methods, or systems, it will be appreciated that other apparatus, methods, or systems may be within the scope of the claims as interpreted by one of skill in the art.

Claims

Claims:1 . A structured light device comprising: a light source for emitting light onto a digital micromirror device (DMD); the DMD for producing structured light; and a control device for controlling the mirror configuration of the DMD, coupled to the DMD; wherein a micromirror array of the DMD reflects the light incident on the DMD out of the structured light device, when at least one mirror of the DMD is in a neutral position; and wherein the DMD is configured to dynamically vary a mirror configuration to reduce specular reflections.

2. The device of claim 1 , wherein the light source comprises a laser.

3. The device of claim 1 , wherein the DMD mirror configuration is varied according to feedback received by a detection device, and transmitted to the structured light device.

4. The device of claim 3, wherein the DMD mirror configuration is varied by iterating through a preset pattern of mirror configurations.

5. The device of claim 3, wherein the DMD mirror configuration is varied by moving mirrors corresponding to specular reflection locations detected by the detection device into a deflected position.

6. The device of claim 3, wherein the DMD mirror configuration is dynamically varied to maximize resolution of identifiable features of a target object.

7. The device of claim 3, wherein the DMD mirror configuration corresponds to a sequence of X image projections to maximize the detection of orthogonal features.

8. The device of claim 3, wherein the DMD mirror configuration is varied according to a binary iterative search algorithm.

9. The device of any one of claims 2 to 6, wherein the structured light device is coupled to the detection device.

10. A system for structured light imaging, the system comprising: the structured light device of claim 1 ; and a detection device for receiving reflected light from the structured light device; wherein the structured light device is configured to direct structured light onto a target object; wherein the detection device is configured to receive the structured light deflected off the target object; and wherein the structured light device is configured to dynamically vary a mirror configuration to reduce specular reflections according to the structured light received by the detection device.

11. The system of claim 10, wherein the DMD mirror configuration is dynamically varied to maximize resolution of identifiable features of a target object.

12. The system of claim 10, wherein the DMD mirror configuration corresponds to a sequence of X image projections to maximize the detection of orthogonal features.

13. The system of claim 10, wherein the structured light device comprises a laser.

14. The system of claim 10, wherein the structured light device comprises a digital micromirror device (“DMD”), wherein the DMD mirror configuration is varied according to feedback received by a detection device, and transmitted to the structured light device.

15. The system of claim 10, wherein the DMD mirror configuration is varied by iterating through a preset pattern of mirror configurations.

16. The system of claim 10, wherein the DMD mirror configuration is varied by moving mirrors corresponding to specular reflection locations detected by the detection device into a deflected position.

17. The system of claim 10, wherein the DMD mirror configuration is varied according to a binary iterative search algorithm.

18. A method of imaging using structured light, the method comprising: directing full range light towards a target object; receiving full range light reflected off the target object; and generating a structured light pattern for a structured light device based on the reflection of the full range light to minimize specular reflections.

19. The method of claim 18, wherein the structured light device comprises the device of claim 1.

20. The method of claim 18, wherein the structured light device comprises a laser.

21. The method of claim 18, wherein the structured light device comprises a digital micromirror device (“DMD”), wherein the DMD mirror configuration is variedaccording to feedback received by the detection device, and transmitted to the structured light device.

22. The method of claim 21 , wherein the DMD mirror configuration is varied by iterating through a preset pattern of mirror configurations.

23. The method of claim 21 , wherein the DMD mirror configuration is varied by moving mirrors corresponding to specular reflection locations detected by the detection device into a deflected position.

24. The method of claim 21 , wherein the DMD mirror configuration is dynamically varied to maximize resolution of identifiable features of a target object.

25. The method of claim 21 , wherein the DMD mirror configuration corresponds to a sequence of X image projections to maximize the detection of orthogonal features.

26. The method of claim 21 , wherein the DMD mirror configuration is varied according to a binary iterative search algorithm.

Images