Adjusting camera exposure for three-dimensional depth sensing and two-dimensional imaging

Abstract

This application relates to adjusting camera exposure for three-dimensional depth sensing and two-dimensional imaging. A method is disclosed, comprising: instructing, by a processing system of a distance sensor, a pattern projector of the distance sensor to project a pattern of light onto an object during a first timing frame; instructing, by the processing system, a camera of the distance sensor to acquire a first image of the object during the first timing frame, wherein the first image is acquired with an exposure time of the camera set to a first value; instructing, by the processing system, the pattern projector to project a pattern of light onto the object during a second timing frame after the first timing frame; instructing, by the processing system, the camera to acquire a second image of the object during the second timing frame, wherein the second image is acquired with the exposure time of the camera set to the first value; instructing, by the processing system, an illumination source of the distance sensor to illuminate the object during a third timing frame after the second timing frame; and instructing, by the processing system, the camera to acquire a third image of the object during the third timing frame, wherein the third image is acquired with the exposure time of the camera set to the first value.

Description

[0001] This application is a divisional application of the national patent application No. 201980033874.8, which was filed on March 14, 2019, and is entitled "Adjusting Camera Exposure for Three-Dimensional Depth Sensing and Two-Dimensional Imaging".

[0002] Cross-references to related applications

[0003] This application claims priority to U.S. Provisional Patent Application Serial No. 62 / 645,190, filed on March 20, 2018, which is incorporated herein by reference in its entirety. Background Technology

[0004] U.S. Patent Application Serials 14 / 920,246, 15 / 149,323, and 15 / 149,429 describe various configurations of distance sensors. Such distance sensors could be useful in a wide range of applications, including security, gaming, control of autonomous vehicles, and others.

[0005] Distance sensors described in these applications include: a projection system (e.g., including a laser, diffractive optics, and / or other cooperating components) that projects a beam of light at a wavelength substantially invisible to the human eye (e.g., infrared light) into a field of view. The beam is diffused to create a pattern (a pattern of dots, dashes, or other artifacts) that can be detected by a suitable light-receiving system (e.g., a lens, an image-capturing device, and / or other components). When the pattern is incident on an object in the field of view, the distance from the sensor to the object can be calculated based on the appearance of the pattern (e.g., the positional relationship of dots, dashes, or other artifacts) in one or more images of the field of view that can be captured by the sensor's light-receiving system. The shape and size of the object can also be determined.

[0006] For example, the appearance of a pattern can change with distance from the object. As an example, if the pattern includes a pattern of dots, the dots can appear closer together when the object is closer to the sensor, and farther apart when the object is farther away from the sensor. Summary of the Invention

[0007] An example method includes: setting the exposure time of a camera on a distance sensor to a first value; instructing the camera to acquire a first image of an object in the camera's field of view, wherein the first image is acquired simultaneously with setting the exposure time to the first value; instructing a pattern projector on the distance sensor to project a pattern of light onto the object; setting the camera's exposure time to a second value different from the first value; and instructing the camera to acquire a second image of the object, wherein the second image includes the pattern of light, and wherein the second image is acquired simultaneously with setting the exposure time to the second value.

[0008] In another example, a non-transitory machine-readable storage medium is encoded with instructions executable by a processor. When executed, the instructions cause the processor to perform operations including: setting the exposure time of a camera of a distance sensor to a first value; instructing the camera to acquire a first image of an object in the camera's field of view, wherein the first image is acquired simultaneously with setting the exposure time to the first value; instructing a pattern projector of the distance sensor to project a pattern of light onto the object; setting the camera's exposure time to a second value different from the first value; and instructing the camera to acquire a second image of the object, wherein the second image includes the pattern of light, and wherein the second image is acquired simultaneously with setting the exposure time to the second value.

[0009] In another example, the distance sensor includes: a pattern projector configured to project a pattern of light onto an object; a camera; a controller configured to set the camera's exposure time to a first value when the pattern projector does not project a pattern of light onto the object, and to set the camera's exposure time to a second value when the pattern projector projects a pattern of light onto the object; and a processor configured to calculate the distance from the distance sensor to the object based on a first image captured when the exposure time is set to the first value and a second image captured when the exposure time is set to the second value. Attached Figure Description

[0010] Figure 1 This is a block diagram illustrating an example distance sensor of this disclosure;

[0011] Figure 2 This is a flowchart illustrating an example of a method for adjusting camera exposure of a distance sensor for three-dimensional depth sensing and two-dimensional image capture according to the present disclosure;

[0012] Figure 3 This is an exemplary timing diagram illustrating the relationship between the frame rate and exposure of a distance sensor camera and the distance projection used for 3D distance measurement, wherein a single light source emits light during separate exposures for 3D distance information and 2D image acquisition, and wherein 3D distance measurement and 2D image acquisition alternate every other frame.

[0013] Figure 4 This is a block diagram illustrating an example distance sensor of this disclosure;

[0014] Figure 5 This is a flowchart illustrating an example of a method for adjusting camera exposure of a distance sensor for three-dimensional depth sensing and two-dimensional image capture according to the present disclosure;

[0015] Figure 6This is an exemplary timing diagram illustrating the relationship between the frame rate and exposure of the distance sensor camera, the distance projection for 3D distance measurement, and the light emission for 2D image acquisition, wherein a first light source emits light at or near the time of 3D data acquisition, and a second, separate light source emits light at the time of 2D image acquisition, and the 3D distance measurement and 2D image acquisition alternate every other frame.

[0016] Figure 7 This is an exemplary timing diagram illustrating the relationship between the frame rate and exposure of the distance sensor camera, the distance projection for 3D distance measurement, and the light emission for 2D image acquisition, wherein a first light source emits light at or near the time of 3D data acquisition, and a second, separate light source emits light at the time of 2D image acquisition, and 3D distance measurement and 2D image acquisition alternate every predetermined number of frames.

[0017] Figure 8 This is a block diagram illustrating an example distance sensor of this disclosure;

[0018] Figure 9 This is an exemplary timing diagram illustrating the relationship between the frame rate and exposure of a distance sensor camera (e.g., a camcorder), the distance projection for 3D distance measurement, and the light emission for 2D image acquisition, where two light projection systems (e.g., used for 3D distance data acquisition) are used, and the exposure duration for 3D data acquisition and 2D image capture is the same.

[0019] Figure 10 This is a flowchart illustrating an example of a method for adjusting camera exposure of a distance sensor for three-dimensional depth sensing and two-dimensional image capture according to the present disclosure;

[0020] Figure 11 This is an exemplary timing diagram illustrating the relationship between the frame rate and exposure of a distance sensor camera (e.g., a camcorder), the distance projection for 3D distance measurement, and the light emission for 2D image acquisition, wherein information about the shutter speed at the time of 2D image acquisition is fed back to the timing for 3D distance data acquisition; and

[0021] Figure 12 A high-level block diagram of an example electronic device used to calculate the distance from a sensor to an object is depicted. Detailed Implementation

[0022] This disclosure broadly describes apparatus, methods, and non-transitory computer-readable media for adjusting camera exposure of a distance sensor for three-dimensional depth sensing and two-dimensional image capture. As discussed above, distance sensors such as those described in U.S. Patent Application Serials 14 / 920,246, 15 / 149,323, and 15 / 149,429 determine the distance to an object (and potentially, the shape and size of the object) by projecting a beam of light that spreads out to create a pattern (e.g., a pattern of dots, dashes, or other artifacts) in a field of view that includes the object. The beam can be projected from one or more laser sources that emit light having wavelengths that are substantially invisible to the human eye but visible to a suitable detector (e.g., a suitable detector in a light-receiving system). The three-dimensional distance to the object can then be calculated based on the appearance of the pattern to the detector.

[0023] In some cases, a two-dimensional image of the object can also be captured (e.g., by a camera of a light-receiving system), and this image can be used to improve three-dimensional distance measurements. For example, reference markers can be attached to the object. Then, when the amount of available three-dimensional information (e.g., the number of dots in the pattern) is insufficient for accurate distance measurements, information from the two-dimensional image of the object (including the reference markers) can be used to supplement the three-dimensional information. It is also possible to determine environmental characteristics such as external brightness, object reflectivity, etc., from the two-dimensional image. This information can be used to adjust the projected beam (and therefore, the projected pattern) to improve three-dimensional distance measurements.

[0024] However, various factors make it difficult to rapidly and continuously acquire 3D object data (e.g., including pattern data) and 2D object images using the same camera. For example, noise introduced by ambient light can make it difficult for the detector to clearly detect patterns formed by the beam. One way to mitigate the influence of ambient light is to incorporate a narrow-bandpass filter into the sensor's light-receiving system, for example, where the filter only allows infrared light to pass through. However, if the amount of ambient light is very large (e.g., perhaps outdoors), the brightness difference between the pattern and the ambient light may be very small. Furthermore, if the exposure time of the light-receiving system is not set properly, the relationship between the exposure value and the sensor latitude can cause unwanted saturation. In either case, the detector may still have difficulty distinguishing the pattern formed by the beam from the ambient light, even with the use of a narrow-bandpass filter. For example, saturation may occur when both the image of the pattern formed by the beam and the ambient light exceed the sensor latitude. However, by reducing the exposure time so that the amount of light input to the light-receiving system is within the sensor latitude, the image of the pattern can become clearer.

[0025] Alternatively or further, when the amount of ambient light is high, if the brightness of the patterning beam is increased relative to the ambient light, the pattern may be more easily distinguishable by the detector. However, from a safety perspective, increasing the beam brightness may pose some risks, as exposure to a brighter beam could be harmful to the human eye. Therefore, the emission time of the laser can be shortened to minimize these risks, and the exposure time of the light receiving system can also be shortened to reduce ambient light.

[0026] While increasing pattern brightness and reducing the exposure time of the light-receiving system can improve the detector's ability to acquire 3D information, these modifications may also impair the camera's ability to capture useful 2D images. For example, a 2D image captured with a shortened exposure time is likely to be dark. Typically, a longer exposure time may be needed to capture a sharper 2D image.

[0027] Therefore, in summary, the optimal camera exposure time for detecting 3D projected patterns and the optimal camera exposure time for capturing 2D images can be very different. This makes it difficult to use the same camera to simultaneously detect 3D projected patterns and capture 2D images within a relatively short time period (e.g., less than one second).

[0028] Examples of this disclosure provide a distance sensor capable of rapidly and continuously acquiring three-dimensional information (e.g., from a pattern of projected light) and two-dimensional images using a single camera. In one example, the light source used to provide illumination for the two-dimensional image acquisition has the same wavelength as the light source used to project the pattern for the three-dimensional information acquisition. This eliminates the need for a bandpass filter in the distance sensor's light-receiving system.

[0029] Figure 1 This is a block diagram illustrating an example distance sensor 100 of the present disclosure. The distance sensor 100 can be used to detect a distance d of an object 114. In one example, the distance sensor 100 shares many components with the distance sensors described in U.S. Patent Application Serial Nos. 14 / 920,246, 15 / 149,323, and 15 / 149,429. For example, in one example, the distance sensor includes: a camera (or other image capturing device) 102, a processor 104, a controller 106, and a pattern projector 108.

[0030] In one example, camera 102 can be a still camera or a video camera. Camera 102 can be capable of capturing three-dimensional distance data. For example, camera 102 can include a detector capable of detecting a pattern of light projected onto object 114, wherein the projected light has wavelengths that are substantially invisible to the human eye (e.g., infrared). Camera 102 can also be capable of capturing two-dimensional red, green, and blue (RGB) images of object 114. Thus, in one example, camera 102 can be a red, green, and blue infrared (RGBIR) camera. In this case, the infrared light emitted for three-dimensional distance sensing can be input to the pixels of camera 102 only if it has an IR filter, while light of other wavelengths can be recognized as a color image by one or more pixels on the RGB filter. Therefore, the camera's detector can detect red, green, blue, and infrared simultaneously, can detect only infrared, or can detect only red, green, and blue. Because three-dimensional distance sensing depends on the intensity of the projected light pattern, and two-dimensional imaging depends on external brightness, the optimal exposure times for the IR and RGB portions of camera 102 will be different. Camera 102 may have a fisheye lens and may be configured to capture image data with a field of view of up to 180 degrees.

[0031] Camera 102 can send the captured image data to processor 104. Processor 104 can be configured to process the captured image data (e.g., three-dimensional distance data and two-dimensional image data) to calculate the distance to object 114. For example, the distance can be calculated according to the methods described in U.S. Patent Application Serial Nos. 14 / 920,246, 15 / 149,323, and 15 / 149,429.

[0032] Controller 106 can be configured to control the operation of other components of the distance sensor, such as camera 102, processor 104, and pattern projector 108. For example, controller 106 can control the exposure time of camera 102 (e.g., the duration the camera shutter is open) and the timing of camera 102 capturing images (including an image of object 114). As discussed in further detail below, controller 106 can set two separate exposure durations for camera 102: a first exposure duration during which an image of object 114 is captured while pattern projector 108 projects a pattern onto object 114 (e.g., for 3D distance sensing); and a second exposure duration during which an image of object 114 is captured when pattern projector 108 is not projecting a pattern onto object 114 (e.g., for 2D image acquisition). In one example, controller 106 can alternate between the first and second exposure durations.

[0033] The controller 106 can also control the duration for which the pattern projector 108 projects a pattern of light onto the object 114, and the timing of the projection. For example, the controller 106 can control the duration of pulses emitted by the light source of the pattern projector 108, as discussed in further detail below.

[0034] Pattern projector 108 may include various optical devices configured to project a pattern of light onto object 114. For example, pattern projector 108 may include a laser source, such as a vertical-cavity surface-emitting laser (VCSEL) 110 and a diffractive optical element (DOE) 112. VCSEL 110 may be configured to emit a laser beam under the guidance of controller 106 (e.g., where controller 106 controls the duration of the laser pulse). DOE 112 may be configured to split the beam projected by VCSEL 110 into multiple beams. These multiple beams may fan out or diverge, such that each beam creates a different spot of light (e.g., a dot, a dash, an x, etc.) in the field of view of a camera. In summary, the different spots of light created by the multiple beams form a pattern. The distance to object 114 can be calculated based on the appearance of the pattern on object 114.

[0035] Figure 2 This is a flowchart illustrating an example of a method 200 for adjusting camera exposure of a distance sensor for three-dimensional depth sensing and two-dimensional image capture according to this disclosure. Method 200 can be, for example, by... Figure 1 The processor 104 shown in the diagram performs the operation. For the sake of example, method 200 is described as being performed by the processing system.

[0036] Method 200 may begin with step 202. In step 204, the processing system may set the exposure time of the camera to a first value. This first value may define the duration of the exposure (e.g., the first time window when the camera shutter is open to acquire image data).

[0037] In step 206, the processing system may instruct the camera to acquire a first image of the object in the field of view of the distance sensor. In one example, the first image is a two-dimensional image (which does not include data from the projection pattern of light). Therefore, in one example, the exposure time used to acquire the first image is equal to a first value.

[0038] In step 208, the processing system may instruct a pattern projector of the distance sensor (e.g., a system of optics including a laser source and diffractive optical elements) to project a pattern of light onto the object. In one example, the pattern of light may include light emitted at wavelengths substantially invisible to the human eye (e.g., infrared). In one example, instructions sent to the pattern projector may include instructions on when to begin projecting the pattern of light and for how long to project the pattern of light (e.g., the timing and duration of a laser pulse).

[0039] In step 210, the processing system can set the camera's exposure time to a second value. This second value can define the duration of the exposure (e.g., a second time window when the camera's shutter is open to acquire image data). In one example, the second value is less than the first value.

[0040] In step 212, the processing system may instruct the camera to acquire a second image of the object, wherein the second image also includes a pattern of light projected onto the object by a pattern projector. Therefore, in one example, the exposure time used to acquire the second image is equal to a second value.

[0041] In step 214, the processing system can instruct the pattern projector to stop projecting the light pattern onto the object. For example, the instruction sent to the pattern projector can instruct the pattern projector to turn off the laser.

[0042] In step 216, the processing system may determine whether to stop imaging the object. For example, if enough data (e.g., from the first and second images) has been acquired to calculate the distance to the object, imaging of the object may stop. If the processing system concludes in step 216 that imaging should not be stopped, method 200 may return to step 204 and proceed as described above to capture additional images of the object.

[0043] Alternatively, if the processing system concludes in step 216 that imaging should be stopped, method 200 may proceed to step 218. In step 218, the processing system may process the first and second images to determine the distance to the object. For example, any method described in U.S. Patent Application Serials 14 / 920,246, 15 / 149,323, and 15 / 149,429 can be used to calculate the distance. Alternatively, the processing system may send the first and second images to a remote processing system for distance calculation.

[0044] Method 200 can end with step 220.

[0045] Figure 3is an exemplary timing diagram that illustrates the relationship between the frame rate and exposure of a distance sensor camera and the distance projection for three-dimensional distance measurement, where a single light source emits light during separate exposures for three-dimensional distance information and two-dimensional image acquisition, and where three-dimensional distance measurement and two-dimensional image acquisition alternate every other frame. For example, Figure 3 the timing diagram of Figure 1 the distance sensor 100 can illustrate

[0046] Specifically, Figure 3 seven frames f1 - f7 of the timing diagram are shown. In one example, three-dimensional distance measurement and two-dimensional image acquisition are alternately performed every other frame. That is, during the first frame f1, a first camera exposure with a first duration d1 can be employed to obtain information for three-dimensional distance measurement. Then, during the subsequent second frame f2, a second camera exposure with a second duration d2 (longer than the duration of the first camera exposure, i.e., d2 > d1) can be used to obtain a two-dimensional image. During the third frame f3 and subsequent odd-numbered frames f5, f7, etc., the first duration d1 is again used for exposure to obtain additional information for three-dimensional distance measurement. During the fourth frame f4 and subsequent even-numbered frames f6, etc., the second duration d2 is again used for exposure to obtain additional two-dimensional images, and so on.

[0047] In one example, laser (or projection light source) pulses with a third, fixed duration p1 can be emitted every other frame. In one example, the third duration p1 is greater than the first duration d1 but less than the second duration d2 (i.e., d1 < p1 < d2). In one example, a laser pulse is emitted simultaneously with each camera exposure having the first duration d1 (e.g., each odd-numbered frame). In other words, at the start of each odd-numbered frame, a laser pulse with a duration p1 is emitted, and the camera shutter is open for a window with a duration d1. Thus, the laser pulse can be used to project a pattern from which the distance sensor can obtain information for three-dimensional distance measurement.

[0048] It can also be seen from Figure 3 that each laser pulse with the third duration p1 is associated with one camera exposure having the first duration d1 and one camera exposure having the second duration d2. That is, one camera exposure having the first duration d1 and one camera exposure having the second duration d2 (in that order) occur between each pair of laser pulses having the third duration p1. Subsequently, the images obtained for three-dimensional distance measurement and the two-dimensional images can be processed separately and differently.

[0049] Figure 4This is a block diagram illustrating an example distance sensor 400 of the present disclosure. The distance sensor 400 can be used to detect a distance d of an object 414. In one example, the distance sensor 400 shares many components with the distance sensors described in U.S. Patent Application Serials 14 / 920,246, 15 / 149,323, and 15 / 149,429. For example, in one example, the distance sensor includes: a camera (or other image capturing device) 402, a processor 404, a controller 406, and a pattern projector 408. However, with... Figure 1 Unlike the distance sensor 100, the distance sensor 400 additionally includes a light-emitting diode (LED) 416 or other type of illumination component that emits light at wavelengths visible to the human eye (e.g., white). Alternatively, the emission wavelength of the LED 416 may be the same as that of the VCSEL 410.

[0050] In one example, camera 402 can be a still camera or a video camera. Camera 402 can be capable of capturing three-dimensional distance data. For example, camera 402 can include a detector capable of detecting patterns of light projected onto object 414, where the projected light has wavelengths that are substantially invisible to the human eye (e.g., infrared). Camera 402 can also be capable of capturing two-dimensional red, green, and blue (RGB) images of object 414. Thus, in one example, camera 402 can be a red, green, and blue infrared (RGBIR) camera. In this case, the infrared light emitted for three-dimensional distance sensing can be input to the pixels of camera 402 only if it has an IR filter, while light of other wavelengths can be recognized as a color image by one or more pixels on the RGB filter. Because three-dimensional distance sensing depends on the intensity of the projected pattern of light, and two-dimensional imaging depends on external brightness, the optimal exposure times for the IR and RGB portions of camera 402 will be different. Camera 402 can have a fisheye lens and can be configured to capture image data with a field of view of up to 180 degrees.

[0051] Camera 402 can send the captured image data to processor 404. Processor 404 can be configured to process the captured image data (e.g., three-dimensional distance data and two-dimensional image data) to calculate the distance to object 414. For example, the distance can be calculated according to the methods described in U.S. Patent Application Serial Nos. 14 / 920,246, 15 / 149,323, and 15 / 149,429.

[0052] Controller 406 can be configured to control the operation of other components of the distance sensor, such as camera 402, processor 404, pattern projector 408, and LED 416. For example, controller 406 can control the exposure time of camera 402 (e.g., the duration the camera shutter is open) and the timing of camera 402 capturing images (including an image of object 414). As discussed in further detail below, controller 406 can set two separate exposure durations for camera 402: a first exposure duration during which an image of object 414 is captured while pattern projector 408 projects a pattern onto object 414 (e.g., for 3D distance sensing); and a second exposure duration during which an image of object 414 is captured while pattern projector 408 is not projecting a pattern onto object 414 but LED 416 illuminates object 414 (e.g., for 2D image acquisition). In one example, controller 406 can alternate between the first and second exposure durations.

[0053] The controller 406 can also control the duration for which the pattern projector 408 projects a pattern of light onto the object 414, and the timing of the projection. For example, the controller 406 can control the duration of pulses emitted by the light source of the pattern projector 408, as discussed in further detail below.

[0054] The controller 406 can also control the duration of the illumination of object 414 by LED 416 and the timing of the illumination. For example, the controller 406 can control the duration of the pulses emitted by LED 416, as discussed in further detail below.

[0055] Pattern projector 408 may include various optical devices configured to project a pattern of light onto object 414. For example, pattern projector 408 may include a laser source, such as a vertical-cavity surface-emitting laser (VCSEL) 410 and a diffractive optical element (DOE) 412. VCSEL 410 may be configured to emit a laser beam under the guidance of controller 406 (e.g., where controller 406 controls the duration of the laser pulse). DOE 412 may be configured to split the beam projected by VCSEL 410 into multiple beams. These multiple beams may fan out or diverge, such that each beam creates a different spot of light (e.g., a dot, a dash, an x, etc.) in the field of view of a camera. In summary, the different spots of light created by the multiple beams form a pattern. The distance to object 414 can be calculated based on the appearance of the pattern on object 414.

[0056] LED 416 may include one or more light-emitting diodes or other light sources capable of emitting light at a wavelength visible to the human eye (e.g., white) under the guidance of controller 406 (e.g., where controller 406 controls the duration of LED pulses). Alternatively, the emission wavelength of LED 416 may be the same as that of VCSEL 410. The illumination provided by LED 416 can be used to acquire a two-dimensional image of object 414, as discussed in further detail below.

[0057] Figure 5 This is a flowchart illustrating an example of a method 500 for adjusting camera exposure of a distance sensor for three-dimensional depth sensing and two-dimensional image capture according to this disclosure. Method 500 can be, for example, by... Figure 4 The processor 404 shown in the diagram is executed. For the sake of example, method 500 is described as being executed by the processing system.

[0058] Method 500 may begin with step 502. In step 504, the processing system may set the exposure time of the camera to a first value. This first value may define the duration of the exposure (e.g., the first time window when the camera shutter is open to acquire image data).

[0059] In step 506, the processing system may instruct a light source (e.g., an LED) of the distance sensor to illuminate an object in the distance sensor's field of view. In one example, the light emitted to illuminate the object may include light at wavelengths visible to the human eye. Alternatively, the emission wavelength of the light source may be the same as the wavelength of the pattern projector of the distance sensor. In one example, the instructions sent to the light source may include instructions on when to begin emitting light and for how long to emit the light (e.g., the timing and duration of LED pulses).

[0060] In step 508, the processing system may instruct the camera to acquire a first image of the object. In one example, the first image is a two-dimensional image (which does not include data on the projected pattern from the light). Therefore, in one example, the exposure time used to acquire the first image is equal to a first value.

[0061] In step 510, the processing system can instruct the lighting source to stop illuminating the object. For example, the instruction sent to the lighting source can instruct the pattern projector to turn off the LEDs.

[0062] In step 512, the processing system may instruct a pattern projector of the distance sensor (e.g., a system of optics including a laser source and diffractive optical elements) to project a pattern of light onto the object. In one example, the pattern of light may include light emitted at a wavelength substantially invisible to the human eye (e.g., infrared). In one example, instructions sent to the pattern projector may include instructions on when to begin projecting the pattern of light and for how long to project the pattern of light (e.g., the timing and duration of a laser pulse).

[0063] In step 514, the processing system can set the camera's exposure time to a second value. The second value can define the duration of the exposure (e.g., a second time window during which the camera shutter opens to acquire image data). In one example, the second value is less than the first value.

[0064] In step 516, the processing system may instruct the camera to acquire a second image of the object, wherein the second image also includes a pattern of light projected onto the object by a pattern projector. Therefore, in one example, the exposure time used to acquire the second image is equal to a second value.

[0065] In step 518, the processing system can instruct the pattern projector to stop projecting the light pattern onto the object. For example, the instruction sent to the pattern projector can instruct the pattern projector to turn off the laser.

[0066] In step 520, the processing system may determine whether to stop imaging the object. For example, if enough data (e.g., from the first and second images) has been acquired to calculate the distance to the object, imaging of the object may stop. If the processing system concludes in step 520 that imaging should not be stopped, method 500 may return to step 504 and proceed as described above to capture additional images of the object.

[0067] Alternatively, if the processing system concludes in step 520 that imaging should be stopped, method 500 may proceed to step 522. In step 522, the processing system may process the first and second images to determine the distance to the object. For example, any method described in U.S. Patent Application Serials 14 / 920,246, 15 / 149,323, and 15 / 149,429 can be used to calculate the distance. Alternatively, the processing system may send the first and second images to a remote processing system for distance calculation.

[0068] Method 500 can end with step 524.

[0069] Figure 6is an exemplary timing diagram that illustrates the relationship between the frame rate and exposure of a distance sensor camera (e.g., a video camera), the distance projection for three-dimensional distance measurement, and the light emission for two-dimensional image acquisition, where a first light source emits light at or near the time of three-dimensional data acquisition, and a second, separate light source emits light at the time of two-dimensional image acquisition, and the three-dimensional distance measurement and two-dimensional image acquisition alternate every other frame.

[0070] In particular, Figure 6 seven frames f1 - f7 of the timing diagram are shown. As in the example of Figure 3 , three-dimensional distance measurement and two-dimensional image acquisition are alternately performed every other frame. That is, during the first frame f1, a first camera exposure with a first duration d1 can be used to acquire information for three-dimensional distance measurement. Then, during the subsequent second frame f2, a second camera exposure with a second duration d2 (longer than the duration of the first camera exposure, i.e., d2 > d1) can be used to acquire a two-dimensional image. During the third frame f3 and subsequent odd-numbered frames f5, f7, etc., the first duration d1 is again used for exposure to acquire additional information for three-dimensional distance measurement. During the fourth frame f4 and subsequent even-numbered frames f6, etc., the second duration d2 is again used for exposure to acquire additional two-dimensional images, and so on.

[0071] As in the example of Figure 3 , laser (or projection light source) pulses with a third, fixed duration p1 can be emitted every other frame. In one example, the third duration p1 is greater than the first duration d1 but less than the second duration d2 (i.e., d1 < p1 < d2). In one example, a laser pulse is emitted simultaneously with the start of each camera exposure with the first duration d1 (e.g., at the start of each odd-numbered frame). In other words, at the start of each odd-numbered frame, a laser pulse with a duration p1 is emitted, and the camera shutter is open for a window with a duration d1. Thus, the laser pulse can be used to project a pattern from which the distance sensor can acquire information for three-dimensional distance measurement.

[0072] It can also be seen from Figure 6 that each laser pulse with the third duration p1 is associated with one camera exposure with the first duration d1 and one camera exposure with the second duration d2. That is, one camera exposure with the first duration d1 and one camera exposure with the second duration d2 (in that order) occur between each pair of laser pulses with the third duration p1.

[0073] In one example, pulses of a light-emitting diode (LED) (or illumination source) having a fourth, fixed duration p2 can also be emitted alternately with laser pulses having a third duration p1. In one example, the fourth duration p2 is the largest of the first duration d1, the second duration d2, and the third duration p1 (i.e., d1 < p1 < d2 < p2). In one example, the LED pulses overlap with the frames; that is, the LED pulses can start at the end of one frame (e.g., more than half way) and can end near the middle of a subsequent frame. For example, referring to Figure 6 , after the laser pulse having the third duration p1 has ended, the LED pulse having the fourth duration p2 can start in frame f1. The same LED pulse can end in the middle of a subsequent frame f2 (during which no laser pulse may occur). In one example, an LED pulse is emitted恰好before each camera exposure having the second duration d2 (e.g.,恰好before the start of each even-numbered frame). In other words, an LED pulse having a duration p2 is emitted恰好before the start of each even-numbered frame, and the camera shutter is open for a window having a duration d2 that ends in the middle of the (even-numbered) frame. Thus, the LED pulses can be used to provide illumination that the distance sensor can utilize to obtain a two-dimensional image of an object.

[0074] It can also be seen from Figure 6 that each LED pulse having the fourth duration p2 is associated with one camera exposure having the second duration d2 and one camera exposure having the first duration d1. That is, one camera exposure having the second duration d2 and one camera exposure having the first duration d1 (in that order) occur between each pair of LED pulses having the fourth duration p2. Subsequently, the images acquired for three-dimensional distance measurement and the two-dimensional images can be processed separately and differently.

[0075] In another example, steps 508 and 516 of Figure 5 can be modified such that the processing system instructs the camera to capture a first plurality (e.g., n) of images and a second plurality (e.g., n) of images, respectively. Thus, multiple images can be captured during each pulse or emission of the illumination source or pattern projection.

[0076] For example, Figure 7is an exemplary timing diagram that illustrates the relationship between the frame rate and exposure of a distance sensor camera (e.g., a video camera), the distance projection for three-dimensional distance measurement, and the light emission for two-dimensional image acquisition, where a first light source emits light at or near the time of three-dimensional data acquisition, and a second, separate light source emits light at the time of two-dimensional image acquisition, and the three-dimensional distance measurement and two-dimensional image acquisition alternate every predetermined number of frames.

[0077] In particular, Figure 7 seven frames f1 - f7 of the timing diagram are shown. Different from the examples in Figure 3 and Figure 6 where the three-dimensional distance measurement and two-dimensional image acquisition alternate every other frame, in Figure 7 , the three-dimensional distance measurement and two-dimensional image acquisition alternate every predetermined (and configurable) number of n frames. In the specific example illustrated in Figure 7 , n = 3. That is, during the first three frames f1, f2, and f3, a first camera exposure with a first duration d1 can be employed to acquire information for three-dimensional distance measurement. Then, during the subsequent three frames f4, f5, and f6, a second camera exposure with a second duration d2 (longer than the duration of the first camera exposure, i.e., d2 > d1) can be used to acquire a two-dimensional image. During the subsequent three frames starting with f7, the first duration d1 is again used for exposure to acquire additional information for three-dimensional distance measurement, and so on.

[0078] As in the example of Figure 3 , a laser (or projection light source) pulse with a third, fixed duration p1 can be emitted at the start of each frame in which three-dimensional distance measurement data is acquired. In one example, the third duration p1 is greater than the first duration d1 but less than the second duration d2 (i.e., d1 < p1 < d2). In one example, the laser pulse is emitted simultaneously with the start of each camera exposure with a first duration d1 (e.g., each time at the start of a frame in a set of n subsequent frames). In other words, at the start of each frame in a set of n frames designated for three-dimensional data acquisition, a laser pulse with a duration p1 is emitted, and the camera shutter is open for a window with a duration d1. Thus, the laser pulse can be used to project a pattern from which the distance sensor can acquire information for three-dimensional distance measurement.

[0079] It can also be seen from Figure 7 that each laser pulse with a third duration p1 is associated with one camera exposure with a first duration d1. That is, one camera exposure with a first duration d1 occurs between each pair of laser pulses with a third duration p1.

[0080] In one example, a light emitting diode (LED) (or illumination source) pulse having a fifth, fixed duration p3 can also be emitted at the start of each set of n frames for which a two-dimensional image has been acquired. In one example, the fifth duration p2 is the largest of the first duration d1, the second duration d2, the third duration p1, and the fourth duration p2 (i.e., d1 < p1 < d2 < p2 < p3). In one example, the LED pulse overlaps the frames: that is, the LED pulse can start at the end (e.g., more than half way) of one frame and can end near the middle of the frame following the n frames. For example, referring Figure 7 to, after a laser pulse having a third duration p1 has ended, an LED pulse having a fifth duration p3 can start in frame f3. This same LED pulse can end in the middle of the frame n frames later (i.e., frame f6). In one example, the LED pulse is emitted just before the first camera exposure having a second duration d2 (e.g., where n camera exposures having a second duration d2 occur consecutively). In other words, just before the start of the first of n subsequent frames designated for two-dimensional image acquisition, an LED pulse having a duration p3 is emitted and, during a duration d3, the camera shutter is opened three times consecutively for a window having a duration d2 (the window ending in the middle of each of the n frames). Thus, the LED pulse can be used to provide illumination that the distance sensor can utilize to acquire a two-dimensional image of an object.

[0081] From Figure 7 it can also be seen that each LED pulse having a fifth duration p3 is associated with n camera exposures having a second duration d2. That is, during each LED pulse having a fifth duration p3, n camera exposures having a second duration d2 occur. Subsequently, the images acquired for three-dimensional distance measurement and the two-dimensional images can be processed separately and differently.

[0082] Figure 8 is a block diagram illustrating an example distance sensor 800 of the present disclosure. The distance sensor 800 can be used to detect the distance d to an object 814. In one example, the distance sensor 800 shares many components of the distance sensors described in U.S. Patent Application Serial Numbers 14 / 920,246, 15 / 149,323, and 15 / 149,429. For example, in one example, the distance sensor includes: a camera (or other image capture device) 802, a processor 804, a controller 806, and multiple pattern projectors 8081 - 8082 (hereinafter individually referred to as "pattern projector 808" or collectively as "pattern projectors 808"). Thus, different from Figure 1 and Figure 4The distance sensor 100 and the distance sensor 800 include more than one pattern projector.

[0083] In one example, camera 802 can be a still camera or a video camera. Camera 802 can be capable of capturing three-dimensional distance data. For example, camera 802 can include a detector capable of detecting patterns of light projected onto object 814, where the projected light has wavelengths that are substantially invisible to the human eye (e.g., infrared). Camera 802 can also be capable of capturing two-dimensional red, green, and blue (RGB) images of object 814. Thus, in one example, camera 802 can be a red, green, and blue infrared (RGBIR) camera. In this case, the infrared light emitted for three-dimensional distance sensing can be input to the pixels of camera 802 only if it has an IR filter, while light of other wavelengths can be recognized as a color image by one or more pixels on the RGB filter. Because three-dimensional distance sensing depends on the intensity of the projected pattern of light, and two-dimensional imaging depends on external brightness, the optimal exposure times for the IR and RGB portions of camera 802 will be different. Camera 802 can have a fisheye lens and can be configured to capture image data with a field of view of up to 180 degrees.

[0084] Camera 802 can send the captured image data to processor 804. Processor 804 can be configured to process the captured image data (e.g., three-dimensional distance data and two-dimensional image data) to calculate the distance to object 814. For example, the distance can be calculated according to the methods described in U.S. Patent Application Serial Nos. 14 / 920,246, 15 / 149,323, and 15 / 149,429.

[0085] Controller 806 can be configured to control the operation of other components of the distance sensor, such as camera 802, processor 804, pattern projector 808, and LED 816. For example, controller 806 can control the exposure time of camera 802 (e.g., the duration of the camera shutter opening) and the timing of camera 802 capturing images (including an image of object 814). As discussed in further detail below, controller 806 can set two separate exposure durations for camera 802: a first exposure duration during which an image of object 814 is captured while at least one of the pattern projectors 808 projects a pattern onto object 814 (e.g., for 3D distance sensing); and a second exposure duration during which an image of object 814 is captured at times when pattern projector 808 does not project a pattern onto object 814 but LED 816 illuminates object 814 (e.g., for 2D image acquisition). In one example, controller 806 can alternate between the first and second exposure durations.

[0086] The controller 806 can also control the duration for which the pattern projector 808 projects a pattern of light onto the object 814, and the timing of this projection. For example, the controller 806 can control the duration of pulses emitted by the light source of the pattern projector 808, as discussed in further detail below. In a particular example, the controller 806 can control the pattern projector 808 to project a pattern of light onto separate portions of the camera's field of view at separate times.

[0087] The controller 806 can also control the duration of the illumination of object 814 by the LED 816, as well as the timing of the illumination of object 814 by the LED 816. For example, the controller 806 can control the duration of the pulses emitted by the LED 816, as discussed in further detail below.

[0088] Pattern projector 808 may include various optical devices configured to project a pattern of light onto object 814. For example, each pattern projector 808 may include a corresponding laser source, such as a corresponding vertical-cavity surface-emitting laser (VCSEL) 8101 or 8102 (hereinafter also individually referred to as "VCSEL 810" or collectively "VCSEL 810") and a corresponding diffractive optical element (DOE) 8121 or 8122 (hereinafter individually referred to as "DOE 812" or collectively "DOE 812"). VCSEL 810 may be configured to emit a laser beam under the guidance of controller 806 (e.g., where controller 806 controls the duration of the laser pulse). DOE 812 may be configured to split the beam projected by the corresponding VCSEL 810 into a corresponding plurality of beams. The plurality of beams may fan out or diverge, such that each beam creates a different spot of light (e.g., a dot, a dash, an x, etc.) in the field of view of a camera. In summary, the different light spots created by multiple light beams form a corresponding pattern. The distance to object 814 can be calculated based on the appearance of the pattern on object 814.

[0089] LED 816 may include one or more light-emitting diodes or other light sources capable of emitting light at a wavelength visible to the human eye (e.g., white) under the guidance of controller 806 (e.g., where controller 806 controls the duration of LED pulses). Alternatively, the emission wavelength of LED 816 may be the same as that of VCSEL 810. The illumination provided by LED 816 can be used to acquire a two-dimensional image of object 814, as discussed in further detail below.

[0090] Figure 9is an exemplary timing diagram that illustrates the relationship between the frame rate and exposure of a distance sensor camera (e.g., a camera), the distance projection for three-dimensional distance measurement, and the light emission for two-dimensional image acquisition, where two light projection systems (e.g., used for three-dimensional distance data acquisition) are used, and the exposure durations for three-dimensional data acquisition and two-dimensional image capture are the same.

[0091] Specifically, Figure 9 seven frames f1 - f7 of the timing diagram are shown. In one example, two three-dimensional distance measurements and one two-dimensional image acquisition are alternately performed every three frames. That is, during the first frame f1, a first camera exposure with a first duration d1 can be employed to acquire information for three-dimensional distance measurement. During the subsequent second frame f2, a second camera exposure with the first duration d1 can be used to acquire information for three-dimensional distance measurement. Then, during the subsequent third frame f3, a third camera exposure with the first duration d1 can be used to acquire a two-dimensional image. During the fourth frame f4 and the fifth frame f5, the first duration d1 is again used for exposure to acquire additional information for three-dimensional distance measurement. During the sixth frame f6, the first duration d1 is again used for exposure to acquire an additional two-dimensional image, and so on.

[0092] As Figure 9 illustrated, a first laser (or projection light source) pulse with a third, fixed duration p1 can be emitted every three frames. In one example, the third duration p1 is greater than the first duration d1 (i.e., d1 < p1). In one example, the first laser pulse is emitted at the start of every third frame. In other words, at the start of every third frame, a first laser pulse with a duration p1 is emitted, and the camera shutter is open for a window with a duration d1. Thus, the first laser pulse can be used to project a pattern from which the distance sensor can obtain information for three-dimensional distance measurement.

[0093] Similarly, a second laser (or projection light source) pulse with a third, fixed duration p1 can also be emitted every three frames. In one example, the second laser pulse is emitted at the start of every third frame, but at the start of a frame after each first laser pulse. In other words, at the start of each frame after the frame in which the first laser pulse occurs, a second laser pulse with a duration p1 is emitted, and the camera shutter is open for a window with a duration d1. Thus, the second laser pulse can be used to project a pattern from which the distance sensor can obtain information for three-dimensional distance measurement.

[0094] From Figure 9It can also be seen that each laser pulse having a third duration p1 (whether it is the first laser pulse or the second laser pulse) is associated with three camera exposures having a first duration d1. That is, three camera exposures having a first duration d1 occur between each pair of (first or second) laser pulses having a third duration p1.

[0095] In one example, after each second laser pulse having a third duration p1, a light emitting diode (LED) (or illumination source) pulse having a fourth, fixed duration p2 can also be emitted. In one example, the fourth duration p2 is the largest of the first duration d1 and the third duration p1 (i.e., d1 < p1 < p2). In one example, the LED pulse overlaps with a frame: that is, the LED pulse can start at the end (e.g., more than half) of one frame and can end near the middle of a subsequent frame. For example, referring to Figure 9 , after the second laser pulse having a third duration p1 has ended, the LED pulse having a fourth duration p2 can start in frame f2. This same LED pulse can end in the middle of a subsequent frame f3 (during which no laser pulse may occur). In one example, an LED pulse is emitted恰好before each camera exposure (e.g.,恰好before the start of every third frame), and two-dimensional image acquisition is performed during that camera exposure. In other words,恰好before the start of every third frame, an LED pulse having a duration p2 is emitted, and the camera shutter is open for a window having a duration d1 that ends in the middle of a subsequent frame. Thus, the LED pulse can be used to provide illumination, and the distance sensor can utilize this illumination to acquire a two-dimensional image of an object.

[0096] [[ID=%]]From Figure 9 It can also be seen that each LED pulse having a fourth duration p2 is associated with three camera exposures having a first duration d1. That is, three camera exposures having a first duration d1 occur between each pair of LED pulses having a fourth duration p2. Subsequently, the images acquired for three-dimensional distance measurement and the two-dimensional images can be processed separately and differently.

[0097] Figure 10 is a flowchart illustrating an example of a method 1000 for adjusting camera exposures of a distance sensor for three-dimensional depth sensing and two-dimensional image capture according to the present disclosure. Method 1000 can be performed, for example, by Figure 1 , Figure 4 and Figure 8 Note: There is an unclear expression "恰好" in the original Chinese text. It is translated as "恰好" in the English text for now, but it may need to be adjusted according to the specific context for a more accurate translation.Any of the processors 104, 404, or 804 shown in the diagram are implemented. However, in this case, the processor additionally performs analysis of the two-dimensional image data captured by the camera and sends the analysis back to the controller to control exposure control and illumination control. For the sake of example, method 1000 is described as being implemented by a processing system.

[0098] Method 1000 may begin with step 1002. In step 1004, the processing system may set the exposure time of the camera to a first value. This first value may define the duration of the exposure (e.g., the first time window when the camera shutter is open to acquire image data).

[0099] In step 1006, the processing system may instruct a light source (e.g., an LED) of the distance sensor to illuminate an object in the distance sensor's field of view. In one example, the light emitted to illuminate the object may be light at a wavelength visible to the human eye (e.g., white). In one example, the instructions sent to the light source may include instructions on when to begin emitting light and for how long to emit the light (e.g., timing and duration of LED pulses).

[0100] In step 1008, the processing system may instruct the camera to acquire a first image of the object. In one example, the first image is a two-dimensional image (which does not include data on the projection pattern from the light). Therefore, in one example, the exposure time used to acquire the first image is equal to a first value.

[0101] In step 1010, the processing system can instruct the lighting source to stop illuminating the object. For example, the instruction sent to the lighting source can instruct the pattern projector to turn off the LEDs.

[0102] In step 1012, the processing system may determine a second value of the camera exposure time and the projection time of the pattern projector of the distance sensor (e.g., a system of optical devices including laser light sources and diffractive optical elements) based on the analysis of the first image of the object.

[0103] In step 1014, the processing system may instruct the pattern projector of the distance sensor to project a pattern of light onto the object. In one example, the pattern of light may include light emitted at a wavelength substantially invisible to the human eye (e.g., infrared). In one example, the instructions sent to the pattern projector may include instructions on when to begin projecting the pattern of light and for how long to project the pattern of light (e.g., the timing and duration of a laser pulse).

[0104] In step 1016, the processing system can set the camera's exposure time to a second value. This second value can define the duration of the exposure (e.g., a second time window when the camera's shutter is open to acquire image data). In one example, the second value is less than the first value.

[0105] In step 1018, the processing system may instruct the camera to acquire a second image of the object, wherein the second image also includes a pattern of light projected onto the object by a pattern projector. Therefore, in one example, the exposure time used to acquire the second image is equal to a second value.

[0106] In step 1020, the processing system can instruct the pattern projector to stop projecting the light pattern onto the object. For example, the instruction sent to the pattern projector can instruct the pattern projector to turn off the laser.

[0107] In step 1022, the processing system may determine whether to stop imaging the object. For example, if sufficient data (e.g., from the first and second images) has been acquired to calculate the distance to the object, imaging of the object may be stopped. If the processing system concludes in step 1022 that imaging should not be stopped, method 1000 may return to step 1004 and proceed as described above to capture additional images of the object.

[0108] Alternatively, if the processing system concludes in step 1022 that imaging should be stopped, method 1000 may proceed to step 1024. In step 1024, the processing system may process the first and second images to determine the distance to the object. For example, any method described in U.S. Patent Application Serials 14 / 920,246, 15 / 149,323, and 15 / 149,429 can be used to calculate the distance. Alternatively, the processing system may send the first and second images to a remote processing system for distance calculation.

[0109] Method 1000 can end with step 1026.

[0110] Figure 11 This is an exemplary timing diagram illustrating the relationship between the frame rate and exposure of a distance sensor camera (e.g., a camcorder), the distance projection for 3D distance measurement, and the light emission for 2D image acquisition, wherein information about the shutter speed at the time of 2D image acquisition is fed back to the timing for 3D distance data acquisition. That is, during 3D distance data acquisition, the camera's exposure time and the projection time of the light pattern can be based on an analysis of the object based on the 2D image of the object.

[0111] It should be noted that, although not explicitly specified, some boxes, functions, or operations of methods 200, 500, and 1000 described above may include application-specific storage, display, and / or output. In other words, depending on the specific application, any data, records, fields, and / or intermediate results discussed in methods 200, 500, and 1000 may be stored, displayed, and / or output to another device. Additionally, Figure 2 , Figure 5 and Figure 10 The description of a defined operation or a framework, function, or operation involving a decision does not mean that both branches of the defined operation are performed. In other words, depending on the outcome of the defined operation, one branch of the defined operation may not be performed.

[0112] Figure 12 A high-level block diagram depicts an example electronic device 1100 for calculating the distance from a sensor to an object. Accordingly, the electronic device 2300 can be implemented as a processor of an electronic device or system, such as a distance sensor (e.g., as...). Figure 1 , Figure 4 and Figure 8 The processor in it is 104, 404 or 804.

[0113] As in Figure 12 The electronic device 1200 described herein includes: a hardware processor element 1202 (e.g., a central processing unit (CPU), microprocessor, or multi-core processor), a memory 1204 (e.g., random access memory (RAM) and / or read-only memory (ROM)), a module 1205 for calculating the distance from the sensor to the object, and various input / output devices 1206 (e.g., storage devices (including but not limited to tape drives, floppy disk drives, hard disk drives, or compact disk drives), receivers, transmitters, displays, output ports, input ports, and user input devices such as keyboards, keypads, mice, microphones, cameras, laser light sources, LED light sources, etc.).

[0114] Although a single processor element is shown, it should be noted that the electronic device 1200 may employ multiple processor elements. Furthermore, although one electronic device 1200 is shown in the figure, if the methods(one or more) discussed above are implemented in a distributed or parallel manner for a particular illustrative example—that is, if a block or the entire method(one or more) of the above methods is implemented across multiple or parallel electronic devices—then the electronic device 1200 in this figure is intended to represent each of those multiple electronic devices.

[0115] It should be noted that this disclosure may be implemented by machine-readable instructions and / or by a combination of machine-readable instructions and hardware, for example, using application-specific integrated circuits (ASICs), programmable logic arrays (PLAs), including field-programmable gate arrays (FPGAs), or state machines deployed on hardware devices, general-purpose computers, or any other hardware equivalent. For example, computer-readable instructions related to the methods(one or more) discussed above may be used to configure a hardware processor to perform the blocks, functions, and / or operations of the methods(one or more) disclosed above.

[0116] In one example, instructions and data (e.g., machine-readable instructions) for this module or process 1205 to calculate the distance from the sensor to the object can be loaded into memory 1204 and executed by hardware processor element 1202 to implement the blocks, functions, or operations discussed above in conjunction with methods 200, 500, and 1000. Additionally, when the hardware processor executes instructions to perform an "operation," this can include the hardware processor directly performing the operation and / or facilitating, directing, or cooperating with another hardware device or component (e.g., a coprocessor, etc.) to perform those operations.

[0117] A processor executing machine-readable instructions related to the methods(s) described above can be considered a programmed processor or a dedicated processor. Accordingly, the present module 1205 for calculating the distance from a sensor to an object can be stored on a tangible or physical (broadly non-transitory) computer-readable storage device or medium (e.g., volatile memory, non-volatile memory, ROM memory, RAM memory, magnetic or optical drives, devices or disks, etc.). More specifically, the computer-readable storage device may include any physical device that provides the ability to store information such as data and / or instructions that will be accessed by a processor or electronic device (such as a computer or controller of a security sensor system).

[0118] It will be understood that the variations disclosed above, as well as other features and functions or alternatives thereof, can be combined into many other different systems or applications. Various alternatives, modifications, or variations that are not currently foreseeable or anticipated can then be made therein, which are also intended to be covered by the appended claims.

Claims

1. A method for acquiring an image of an object, comprising: The distance sensor's processing system instructs the distance sensor's pattern projector to project a pattern of light onto the object for a duration p1 during the first time frame; The processing system instructs the camera of the distance sensor to acquire a first image of the object during a duration p1 in the first time frame, wherein the first image is acquired when the camera's exposure time is set to a first value d1, where d1 <p1; The processing system instructs the pattern projector to project a pattern of light onto the object for a duration p1 during a second time frame following the first time frame; The processing system instructs the camera to acquire a second image of the object during a duration p1 in a second temporal frame, wherein the second image is acquired when the camera's exposure time is set to a first value d1, where d1 <p1; The processing system instructs the illumination source of the distance sensor to illuminate the object for a duration p2 from the end of the second time frame until the end of the third time frame; and The processing system instructs the camera to acquire a third image of the object during a duration p2 in the third time frame, wherein the third image is acquired when the camera's exposure time is set to a first value d1, where d1 <p1<p2。 2. The method of claim 1, wherein the pattern projector is inactive during the third time frame, such that the pattern of light is not present in the third image.

3. The method according to claim 2, wherein the third image comprises a two-dimensional image.

4. The method according to claim 1, further comprising: The processing system calculates the distance from the distance sensor to the object based on at least one of the first image or the second image.

5. The method according to claim 1, wherein the light source of the pattern projector is of a different type than the light source of the illumination source.

6. The method of claim 5, wherein the light source of the illumination source comprises a light-emitting diode.

7. The method of claim 6, wherein the light source of the pattern projector comprises a laser light source.

8. The method of claim 7, wherein the laser source emits light pulses having a first fixed duration during the first time frame and the second time frame.

9. The method of claim 8, wherein the value of the first fixed duration is greater than the first value d1.

10. The method of claim 9, wherein the light-emitting diode emits a light pulse having a second fixed duration during the third timing frame.

11. The method of claim 10, wherein the value of the second fixed duration is greater than the value of the first fixed duration.

12. The method of claim 10, wherein the light-emitting diode begins emitting the light pulse during the end of the second timing frame and stops emitting the light pulse during the third timing frame.

13. The method of claim 12, wherein the light-emitting diode begins emitting the light pulse more than halfway through the second timing frame.

14. The method of claim 12, wherein the light-emitting diode begins to emit the light pulse before the camera shutter opens during the third time frame.

15. The method of claim 1, further comprising: The processing system instructs the pattern projector to project the pattern of light onto the object for a duration p1 during a fourth time frame following the third time frame; The processing system instructs the camera to acquire a fourth image of the object during a duration p1 in the fourth temporal frame, wherein the fourth image is acquired when the camera's exposure time is set to a first value d1, where d1 <p1; The processing system instructs the pattern projector to project the pattern of light onto the object for a duration p1 during a fifth time frame following the fourth time frame; The processing system instructs the camera to acquire a fifth image of the object during a duration p1 in the fifth time frame, wherein the fifth image is acquired when the camera's exposure time is set to a first value d1, where d1 <p1; The processing system instructs the illumination source of the distance sensor to illuminate the object for a duration p2 from the end of the fifth time frame until the sixth time frame; and The processing system instructs the camera to acquire a sixth image of the object during a duration p2 in the sixth time frame, wherein the sixth image is acquired when the camera's exposure time is set to a first value d1, where d1 <p1<p2。 16. The method of claim 15, wherein the pattern projector is inactive during the third and sixth time frames.

17. The method of claim 15, wherein the pattern projector emits a pattern of light in the form of pulses, the pulses lasting for a first fixed duration during the first time frame, the second time frame, the fourth time frame, and the fifth time frame.

18. The method of claim 15, wherein three camera exposures having a duration of the first value d1 occur between the pulse emitted during the first timing frame and the pulse emitted during the fourth timing frame.

19. A non-transitory machine-readable storage medium having instructions encoded thereon that are executable by a processing system of a distance sensor, wherein, when executed, the instructions cause the processing system to perform an operation comprising: The pattern projector of the distance sensor projects a pattern of light onto the object for a duration p1 during the first time frame; The camera, instructing the distance sensor, acquires a first image of the object during a duration p1 in the first time frame, wherein the first image is acquired when the camera's exposure time is set to a first value d1, where d1 <p1; The pattern projector is instructed to project a pattern of light onto the object for a duration p1 during a second time frame following the first time frame; The camera is instructed to acquire a second image of the object during a duration p1 in the second time frame, wherein the second image is acquired when the camera's exposure time is set to a first value d1, where d1 <p1; The illumination source of the distance sensor illuminates the object for a duration p2 from the beginning of the second timing frame until the end of the third timing frame; and The camera is instructed to acquire a third image of the object during a duration p2 in the third time frame, wherein the third image is acquired when the camera's exposure time is set to a first value d1, where d1 <p1<p2。 20. A distance sensor, comprising: A pattern projector configured to project a pattern of light onto an object using a laser light source; A lighting source, separate from the laser light source, configured to illuminate the object; camera; Processing system; and A non-transitory machine-readable storage medium encoded with instructions executable by the processing system, wherein, when executed, the instructions cause the processing system to perform operations including: The pattern projector is instructed to project a pattern of light onto the object for a duration p1 during the first time frame; The camera is instructed to acquire a first image of the object during a duration p1 in the first temporal frame, wherein the first image is acquired when the camera's exposure time is set to a first value d1, where d1 <p1; The pattern projector is instructed to project a pattern of light onto the object for a duration p1 during a second time frame following the first time frame; The camera is instructed to acquire a second image of the object during a duration p1 in the second time frame, wherein the second image is acquired when the camera's exposure time is set to a first value d1, where d1 <p1; The illumination source is instructed to illuminate the object for a duration p2 from the end of the second timing frame until the end of the third timing frame; and The camera is instructed to acquire a third image of the object during a duration p2 in the third time frame, wherein the third image is acquired when the camera's exposure time is set to a first value d1, where d1 <p1<p2。

Images