Distance measuring device

Abstract

A ranging device of one embodiment of the present disclosure comprises a light source, an optical detector, and a circuit. The circuit causes a first signal to be generated by: causing the light source to emit an optical pulse towards an object; and causing the optical detector to detect a reflected optical pulse which is from the object and which is produced as a result of the optical pulse, such detection performed during a first duration that includes at least a part of a rising duration from a first point in time where the reflected optical pulse starts to increase in intensity to a second point in time where the increase concludes, and that includes the first point in time. The circuit causes a second signal to be generated by causing the optical detector to detect the reflected optical pulse during a second duration after the conclusion of the first duration, the second duration including at least a part of a falling duration from a third point in time where the reflected optical pulse starts to decrease in intensity to a fourth point in time where the decrease concludes, and including the fourth point in time. The circuit generates and outputs data indicative of the distance from the optical detector to the object on the basis of the first signal and the second signal.

Description

[Technical field]

[0001] The present disclosure relates to a distance measuring device. [Background technology]

[0002] Conventionally, a distance measurement method called Time of Flight (TOF) has been proposed, which estimates the distance to an object by using the flight time of light. In the TOF method, a light pulse is emitted from a light source toward the object, and the return light is received by a pulse-driven sensor. At this time, the amount of light detected by the sensor varies depending on the time it takes for the return light to reach the sensor. Therefore, the flight time of light can be indirectly estimated from the variation in the amount of detected light, and the distance to the object can be calculated. Various methods have been proposed to improve the accuracy of distance estimation in the TOF method. For example, Patent Document 1 discloses a method of improving distance measurement accuracy by detecting the duty shift of the light emission waveform in advance and correcting the amount of shift. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2017-32342 A Summary of the Invention [Problem to be solved by the invention]

[0004] The present disclosure provides a technique for further improving the accuracy of distance measurement by a TOF method using optical pulses. [Means for solving the problem]

[0005] A distance measuring device according to an embodiment of the present disclosure includes a light source, a photodetector, and a circuit for controlling the light source and the photodetector. The circuit causes the light source to emit a light pulse toward an object. The circuit causes the photodetector to detect the reflected light pulse in a first period including at least a part of a rising period from a first time point when the intensity of a reflected light pulse from the object caused by the light pulse starts to increase to a second time point when the increase ends and including the first time point, thereby generating a first signal. The circuit causes the photodetector to detect the reflected light pulse in a second period after the end of the first period, including at least a part of a falling period from a third time point when the intensity of the reflected light pulse starts to decrease to a fourth time point when the decrease ends and including the fourth time point, thereby generating a second signal. The circuit generates and outputs data indicating a distance from the photodetector to the object based on the first signal and the second signal.

[0006] A distance measuring device according to another aspect of the present disclosure includes a light source, a photodetector, and a circuit for controlling the light source and the photodetector. The circuit causes the light source to emit a first light pulse and a second light pulse toward an object. The circuit causes the photodetector to detect the first reflected light pulse in a first period including at least a part of a rise period from a first time point when the intensity of the first reflected light pulse from the object caused by the first light pulse starts to increase to a second time point when the increase ends and including the first time point, thereby generating a first signal. The circuit causes the photodetector to detect the second reflected light pulse in a second period including at least a part of a fall period from a third time point when the intensity of the second reflected light pulse from the object caused by the second light pulse starts to decrease to a fourth time point when the decrease ends and including the fourth time point, thereby generating a second signal. The circuit generates and outputs data indicating a distance from the photodetector to the object based on the first signal and the second signal.

[0007] A comprehensive or specific aspect of the present disclosure may be realized in a system, an apparatus, a method, an integrated circuit, a computer program, or a recording medium such as a computer-readable recording disk, or may be realized in any combination of a system, an apparatus, a method, an integrated circuit, a computer program, and a recording medium. The computer-readable recording medium may include a non-volatile recording medium such as a CD-ROM (Compact Disc-Read Only Memory). An apparatus may be composed of one or more devices. When an apparatus is composed of two or more devices, the two or more devices may be arranged in one device, or may be arranged separately in two or more separate devices. In this specification and the claims, "apparatus" may mean not only one device, but also a system consisting of multiple devices. Effect of the Invention

[0008] According to the present disclosure, the distance to an object can be measured with higher accuracy than before. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a schematic diagram showing how the distance measuring device of the first embodiment captures an image of an object. [Diagram 2] FIG. 2 is a diagram illustrating an example of the configuration of an image sensor. [Figure 3A] FIG. 3A is a diagram showing an example of a light emission pulse, a reflected light pulse, and shutter timing. [Figure 3B] FIG. 3B is a diagram showing another example of light emission pulses, reflected light pulses, and shutter timing. [Figure 3C] FIG. 3C is a diagram showing still another example of light emission pulses, reflected light pulses, and shutter timing. [Figure 3D] FIG. 3D is a diagram showing still another example of light emission pulses, reflected light pulses, and shutter timing. [Figure 4A] FIG. 4A is a flowchart showing an example of the operation of the electronic circuit. [Figure 4B] FIG. 4B is a flowchart showing another example of the operation of the electronic circuit. [Figure 4C] FIG. 4C is a flowchart showing yet another example of the operation of the electronic circuit. [Diagram 5] FIG. 5 is a diagram showing a driving method in the second embodiment. [Figure 6] FIG. 6 is a flowchart showing an example of the operation of the electronic circuit according to the second embodiment. [Figure 7] FIG. 7 is a diagram showing an example in which the distance measuring device is used for personal authentication of a driver. [Figure 8A] FIG. 8A is a diagram showing an example in which a mobile terminal equipped with a distance measuring device performs personal authentication. [Figure 8B] FIG. 8B is another diagram showing an example in which a portable terminal equipped with a distance measuring device performs personal authentication. [Figure 9] FIG. 9 is a diagram showing an example in which the distance measuring device detects components of light scattered inside the skin of a subject in addition to distance measurement. [Figure 10] FIG. 10 is a flowchart showing an example of the operation of the electronic circuit in the example shown in FIG. [Figure 11A] FIG. 11A is a diagram showing an example of personal authentication using distance measurement information and biometric information. [Figure 11B] FIG. 11B is another diagram showing an example of personal authentication using distance measurement information and biometric information. [Figure 12] FIG. 12 is a diagram showing the results of measuring the distance to a face in the embodiment. [Figure 13] FIG. 13 is a diagram showing the results of distance measurement of a face in the comparative example. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] (Item 1) A distance measuring device according to item 1 of the present disclosure includes a light source, a photodetector, and a circuit for controlling the light source and the photodetector. The circuit causes the light source to emit a light pulse toward an object. The circuit causes the photodetector to detect the reflected light pulse in a first period including at least a part of a rising period from a first time point when the intensity of a reflected light pulse from the object caused by the light pulse starts to increase to a second time point when the increase ends and including the first time point, thereby generating a first signal. The circuit causes the photodetector to detect the reflected light pulse in a second period after the end of the first period, including at least a part of a falling period from a third time point when the intensity of the reflected light pulse starts to decrease to a fourth time point when the decrease ends and including the fourth time point, thereby generating a second signal. The circuit generates and outputs data indicating a distance from the photodetector to the object based on the first signal and the second signal.

[0011] (Item 2) In the distance measuring device according to item 1, the first time period ends before a midpoint between the first time point and the fourth time point; The second period may begin after the midpoint.

[0012] (Item 3) In the distance measuring device according to item 1 or 2, The pulse width of the light pulse may be not less than 3 ns and not more than 30 ns.

[0013] (Item 4) In the distance measuring device according to any one of items 1 to 3, Let t1 be the time from the start of emission of the light pulse to the end of the first period, t2 be the time from the start of emission of the light pulse to the start of the second period, and T0 be the pulse width of the light pulse. T0 / 3<(t2-t1) <T0 may be satisfied.

[0014] (Item 5) In the distance measuring device according to any one of items 1 to 4, the first time period ends before the second time point; The second time period may begin after the third time point.

[0015] (Item 6) In the distance measuring device according to any one of items 1 to 5, The timing at which the first period begins may be different from the timing at which the emission of the light pulse begins, or The timing at which the second period starts may be different from the timing at which the emission of the light pulse ends.

[0016] (Item 7) A distance measuring device according to item 7 of the present disclosure includes a light source, a photodetector, and a circuit for controlling the light source and the photodetector. The circuit causes the light source to emit a first light pulse and a second light pulse toward an object. The circuit causes the photodetector to detect the first reflected light pulse in a first period including at least a part of a rise period from a first time point when the intensity of the first reflected light pulse from the object caused by the first light pulse starts to increase to a second time point when the increase ends and including the first time point, thereby generating a first signal. The circuit causes the photodetector to detect the second reflected light pulse in a second period including at least a part of a first fall period from a third time point when the intensity of the second reflected light pulse from the object caused by the second light pulse starts to decrease to a fourth time point when the decrease ends and including the fourth time point, thereby generating a second signal. The circuit generates and outputs data indicative of a distance from the photodetector to the object based on the first signal and the second signal.

[0017] (Item 8) 8. The distance measuring device according to claim 7, the first time period ends before a midpoint between the first time point and a time point at which the decrease in intensity of the first reflected light pulse ends; The second time period may start later than a midpoint between a point in time at which the increase in intensity of the second reflected light pulse begins and the fourth point in time.

[0018] (Item 9) In the distance measuring device according to item 7 or 8, The pulse width of each of the first light pulse and the second light pulse may be not less than 3 ns and not more than 30 ns.

[0019] (Item 10) In the distance measuring device according to any one of items 7 to 9, Let t1 be the time from the start of emission of the first light pulse to the end of the first period, t2 be the time from the start of emission of the second light pulse to the start of the second period, and T0 be the pulse width of each of the first light pulse and the second light pulse. T0 / 3<(t2-t1) <T0 may be satisfied.

[0020] (Item 11) In the distance measuring device according to any one of items 7 to 10, the first time period ends before the second time point; The second time period may begin after the third time point.

[0021] (Item 12) In the distance measuring device according to any one of items 7 to 11, The timing at which the first period begins may be different from the timing at which the emission of the first light pulse begins; or The timing at which the second period starts may be different from the timing at which emission of the second light pulse ends.

[0022] (Item 13) In the distance measuring device according to any one of items 7 to 12, The object is a living body, The light source further emits a third light pulse toward the object; the optical detector further detects a third reflected optical pulse resulting from the third optical pulse returning from the object; the circuit causes the photodetector to detect the third reflected light pulse and output a third signal during a third period including at least a part of a second falling period from a fifth time point when the intensity of the third reflected light pulse starts to decrease to a sixth time point when the decrease ends and including the sixth time point; A length from the start of the third period to the sixth time point is shorter than a length from the start of the second period to the fourth time point; The circuit may generate and output biological information of the object based on the third signal.

[0023] (Item 14) A signal processing device according to item 14 of the present disclosure includes one or more processors and a memory that stores a computer program executed by the one or more processors. The one or more processors execute the computer program to: A light source is caused to emit a light pulse toward the target object; generating a first signal by causing a photodetector to detect the reflected light pulse during a first period including at least a portion of a rise period from a first time point at which an intensity of the reflected light pulse from the object caused by the light pulse starts to increase to a second time point at which the increase ends and including the first time point; generating a second signal by causing the photodetector to detect the reflected light pulse during a second period following the end of the first period, the second period including at least a part of a falling period from a third time point at which the intensity of the reflected light pulse starts to decrease to a fourth time point at which the decrease ends and including the fourth time point; Based on the first signal and the second signal, data indicative of the distance from the photodetector to the object is generated and output.

[0024] (Item 15) A signal processing device according to item 15 of the present disclosure includes one or more processors and a memory that stores a computer program executed by the one or more processors. The one or more processors execute the computer program to: causing a light source to emit a first light pulse and a second light pulse toward a target; generating a first signal by causing a photodetector to detect the first reflected light pulse during a first period including at least a portion of a rise period from a first time point at which an intensity of a first reflected light pulse from the object caused by the first light pulse starts to increase to a second time point at which the increase ends and including the first time point; generating a second signal by causing the photodetector to detect the second reflected light pulse in a second period including at least a part of a falling period from a third time point at which an intensity of the second reflected light pulse from the object caused by the second light pulse starts to decrease to a fourth time point at which the decrease ends and including the fourth time point; Based on the first signal and the second signal, data indicative of the distance from the photodetector to the object is generated and output.

[0025] (Item 16) The computer program according to item 16 of the present disclosure comprises: causing a light source to emit a light pulse toward a target; generating a first signal by causing a photodetector to detect the reflected light pulse during a first time period including at least a part of a rise time period from a first time point at which an intensity of the reflected light pulse from the object caused by the light pulse starts to increase to a second time point at which the increase ends and including the first time point; generating a second signal by causing the photodetector to detect the reflected light pulse during a second time period after the end of the first time period, the second time period including at least a part of a falling period from a third time point at which the intensity of the reflected light pulse starts to decrease to a fourth time point at which the decrease ends and including the fourth time point; and generating and outputting data indicative of a distance from the photodetector to the object based on the first signal and the second signal; The method causes a computer to perform operations including:

[0026] (Item 17) The computer program according to item 17 of the present disclosure comprises: causing a light source to emit a first light pulse and a second light pulse toward a target; generating a first signal by causing a photodetector to detect the first reflected light pulse during a first period including at least a part of a rise period from a first time point at which an intensity of a first reflected light pulse from the object caused by the first light pulse starts to increase to a second time point at which the increase ends and including the first time point; generating a second signal by causing the photodetector to detect the second reflected light pulse in a second time period including at least a part of a falling period from a third time point at which an intensity of the second reflected light pulse from the object caused by the second light pulse starts to decrease to a fourth time point at which the decrease ends and including the fourth time point; and generating and outputting data indicative of a distance from the photodetector to the object based on the first signal and the second signal; The method causes a computer to perform operations including:

[0027] (Item 18) A computer-readable non-transitory recording medium according to item 18 of the present disclosure records the computer program according to item 16.

[0028] (Item 19) A computer-readable non-transitory recording medium according to item 19 of the present disclosure records the computer program according to item 17.

[0029] The embodiments described below are all comprehensive or specific examples. The numerical values, shapes, materials, components, the arrangement and connection forms of the components, steps, and the order of steps shown in the following embodiments are examples and are not intended to limit the technology of the present disclosure. Among the components in the following embodiments, components that are not described in the independent claims showing the highest concept are described as optional components. Each figure is a schematic diagram and is not necessarily illustrated strictly. Furthermore, in each figure, substantially the same or similar components are given the same reference numerals. Duplicate descriptions may be omitted or simplified.

[0030] In the present disclosure, all or part of a circuit, unit, device, member or part, or all or part of a functional block in a block diagram may be implemented by one or more electronic circuits including, for example, a semiconductor device, a semiconductor integrated circuit (IC), or an LSI (large scale integration). The LSI or IC may be integrated into one chip, or may be configured by combining multiple chips. For example, functional blocks other than memory elements may be integrated into one chip. Here, although it is called an LSI or an IC, the name may change depending on the degree of integration, and it may be called a system LSI, a VLSI (very large scale integration), or an ULSI (ultra large scale integration). A Field Programmable Gate Array (FPGA), which is programmed after the manufacture of the LSI, or a reconfigurable logic device, which can reconfigure the junction relationship inside the LSI or set up the circuit partition inside the LSI, can also be used for the same purpose.

[0031] Furthermore, all or part of the functions or operations of a circuit, unit, device, member, or part can be executed by software processing. In this case, the software is recorded in one or more non-transitory recording media such as ROMs, optical disks, hard disk drives, etc., and when the software is executed by a processor, the functions specified in the software are executed by the processor and peripheral devices. The system or device may include one or more non-transitory recording media on which the software is recorded, a processor, and necessary hardware devices, such as interfaces.

[0032] First, an overview of the embodiment of the present disclosure will be described.

[0033] A distance measuring device according to an exemplary embodiment of the present disclosure includes a light source, a photodetector, and a circuit for controlling the light source and the photodetector, the circuit performing the following operations (a) to (d): (a) causing the light source to emit a light pulse; (b) generating a first signal by causing the optical detector to detect the reflected optical pulse during a first time period that includes at least a portion of a rise time period from a first time point at which an intensity of a reflected optical pulse from an object caused by the optical pulse begins to increase to a second time point at which the increase ends and that includes the first time point; (c) generating a second signal by causing the photodetector to detect the reflected light pulse during a second period after the end of the first period, the second period including at least a portion of a falling period from a third time point at which the intensity of the reflected light pulse starts to decrease to a fourth time point at which the decrease ends and including the fourth time point; (d) generating and outputting data indicating a distance to the object based on the first signal and the second signal;

[0034] According to the above configuration, the photodetector generates a first signal indicating a component of a front-end portion including a front end of the reflected light pulse that includes at least a first time point at which the rising period of the reflected light pulse starts, and a second signal indicating a component of a rear-end portion including a rear end of the reflected light pulse that includes at least a fourth time point at which the falling period of the reflected light pulse ends. Then, the circuit calculates or estimates the distance to the object based on the first signal and the second signal. By such an operation, as will be described in detail later, the ranging accuracy can be improved.

[0035] The first period may end before the midpoint between the first time point and the fourth time point. The second period may start after the midpoint. Thereby, the component of the midpoint where the intensity of the reflected light pulse is high is not included in the first signal and the second signal. For this reason, the ranging accuracy can be further improved.

[0036] The pulse width of the optical pulse can be set, for example, to be 3 ns or more and 30 ns or less. Here, the pulse width of the optical pulse means the time length from the start point of the rise of the optical pulse to the end point of the fall. The pulse width may be less than 3 ns or more than 30 ns.

[0037] When the time from the start of the emission of the optical pulse to the end of the first period is t1, the time from the start of the emission of the optical pulse to the start of the second period is t2, and the pulse width of the optical pulse is T0, it may satisfy T0 / 3 < (t2 - t1) < T0. Alternatively, it may satisfy T0 / 2 < (t2 - t1) < T0. When such conditions are satisfied, the ranging accuracy can be further improved.

[0038] The ranging device according to another embodiment of the present disclosure includes a light source, a photodetector, and a circuit that controls the light source and the photodetector. The circuit performs the operations from (a') to (d') below. (a') Cause the light source to emit a first optical pulse and a second optical pulse. (b') generating a first signal by having the optical detector detect the first reflected light pulse during a first period that includes at least a portion of a rise period from a first point in time when the intensity of a first reflected light pulse from an object caused by the first light pulse begins to increase to a second point in time when the increase ends and that includes the first point in time; (c') generating a second signal by having the optical detector detect the second reflected optical pulse during a second period that includes at least a portion of a falling period from a third time point at which the intensity of the second reflected optical pulse from the object caused by the second optical pulse begins to decrease to a fourth time point at which the decrease ends, and that includes the fourth time point. (d') generating and outputting data indicating a distance to the object based on the first signal and the second signal;

[0039] According to the above configuration, the photodetector generates a first signal indicating a component of a leading end portion of the first reflected light pulse including at least a first time point when the rising period of the first reflected light pulse starts, and a second signal indicating a component of a trailing end portion of the second reflected light pulse including at least a fourth time point when the falling period of the second reflected light pulse ends. Then, the circuit calculates or estimates the distance to the object based on the first signal and the second signal. This operation can also improve the accuracy of distance measurement, as described below.

[0040] The first period may end before the midpoint between the first time point and the time point at which the intensity of the first reflected light pulse ends decreasing. The second period may start after the midpoint between the time point at which the intensity of the second reflected light pulse starts increasing and the fourth time point. This prevents the first signal and the second signal from including a component at the midpoint where the intensity of the reflected light pulse is high. This can further improve the accuracy of distance measurement.

[0041] The pulse width of each of the first optical pulse and the second optical pulse can be set, for example, to 3 ns or more and 30 ns or less. The pulse width of each optical pulse may be set to a value outside the above range.

[0042] When the time from the start of emission of the first optical pulse to the end of the first period is t1, the time from the start of emission of the second optical pulse to the start of the second period is t2, and the pulse width of the optical pulse is T0, it may be satisfied that T0 / 3 < (t2 - t1) < T0. Alternatively, it may be satisfied that T0 / 2 < (t2 - t1) < T0. When such conditions are satisfied, the ranging accuracy can be further improved.

[0043] The object can be, for example, a living body. Living bodies include humans and animals. In one example, the object is a human head, and the circuit generates data indicating the distance distribution of the human face.

[0044] The light source may further emit a third optical pulse toward the object. The photodetector may further detect a third reflected optical pulse caused by the third optical pulse returned from the object. The circuit causes the photodetector to detect the third reflected optical pulse and output a third signal in at least a part of the falling period from the fifth time point when the intensity of the third reflected optical pulse starts to decrease to the sixth time point when the decrease ends, and in a third period including the sixth time point. The length from the start of the third period to the sixth time point can be set to a value shorter than the length from the start of the second period to the fourth time point. The circuit may generate and output biological information of the object based on the third signal.

[0045] According to the above configuration, not only the distance information of the subject but also biological information can be acquired with high accuracy based on the component at the trailing end of the third reflected optical pulse. The component at the trailing end of the reflected optical pulse includes the component of the light scattered and returned inside the skin of the subject. By detecting such a component, biological information such as the distribution of blood vessels of the subject can be acquired.

[0046] A signal processing device according to another embodiment of the present disclosure includes one or more processors and a memory that stores a computer program executed by the one or more processors. The processor may execute the computer program to perform the functions of the circuit in any of the above examples.

[0047] The present disclosure includes computer programs that define the functionality of the aforementioned circuits, and control methods executed by the aforementioned circuits.

[0048] Hereinafter, more specific embodiments of the present disclosure will be described with reference to the drawings.

[0049] (Embodiment 1) [1. Configuration] First, the configuration of the distance measuring device 100 according to the first embodiment will be described.

[0050] FIG. 1 is a schematic diagram showing a distance measuring device 100 according to this embodiment. The distance measuring device 100 includes a light source 110, an image sensor 120, and an electronic circuit 130. The image sensor 120 includes a photoelectric conversion unit 122 and a charge storage unit 124. The electronic circuit 130 includes a control circuit 132, a signal processing circuit 134, and a memory 136. FIG. 1 also shows an object 50 for which the distance is measured by the distance measuring device 100. In this embodiment, the object 50 is a human head. The object 50 is not limited to a human head, and may be any object.

[0051] [1-1.Light source 110] The light source 110 irradiates light onto the object 50. The light irradiated from the light source 110 and reaching the object 50 is reflected by the object 50, and a part of the light reaches the image sensor 120. The light source 110 generates light pulses multiple times at a predetermined time interval or timing.

[0052] In the distance measuring device 100 of this embodiment, since the target 50 is a human, a light source 110 designed in consideration of the effect on the retina may be used. For example, a light source 110 that satisfies Class 1 of the laser safety standards established in various countries may be used. In this case, light with such low illuminance that the exposed emission limit (AEL) is below 1 mW is irradiated onto the target 50. However, the light source 110 itself does not have to satisfy Class 1. For example, a diffuser or ND filter may be placed in front of the light source 110 to diffuse or attenuate the light, thereby satisfying Class 1 of the laser safety standards.

[0053] The light source 110 may be, for example, a light source using a general-purpose semiconductor laser. When a general-purpose semiconductor laser is driven at a low voltage, if the pulse width is too short, it becomes difficult to follow the driving of turning on and off the light. For this reason, the light emission waveform differs for each pulse emission, which is likely to exhibit unstable behavior and cause variation in distance measurement results. In order to obtain a stable waveform using a general-purpose semiconductor laser, the light source 110 may be controlled to emit a light pulse with a pulse width of, for example, 3 ns or more. Alternatively, for further stabilization, the light source 110 may emit a light pulse with a pulse width of 5 ns or more, or even 10 ns or more. On the other hand, if the pulse width is too large, the light outflow to the charge accumulation unit 124 at the time of shutter off, that is, the parasitic light sensitivity (PLS), may become large, which may cause distance measurement errors. Therefore, the light source 110 may be controlled to generate a light pulse with a pulse width of, for example, 50 ns or less. Alternatively, the light source 110 may emit a light pulse with a pulse width of 30 ns or less, or even 20 ns or less.

[0054] A pattern with a uniform intensity distribution in the irradiation area may be selected as the irradiation pattern of the light source 110. In that case, the object 50 can be irradiated with light with a spatially equal illuminance, and it becomes easy to keep the intensity of the detection signal within the dynamic range in any pixel of the image sensor 120.

[0055] [1-2.Image sensor 120] The image sensor 120 is an example of a photodetector that receives light emitted from the light source 110 and reflected from the object 50. The image sensor 120 has a plurality of photodetection cells arranged two-dimensionally, and can acquire two-dimensional information of the object 50 at once. The image sensor 120 can be any imaging element such as a CCD image sensor or a CMOS image sensor. The photodetector in the present disclosure is not limited to a sensor having a plurality of photodetection cells arranged two-dimensionally, but may be, for example, a sensor having a plurality of photodetection cells arranged one-dimensionally. In addition, in applications where it is sufficient to acquire distance information of only one point, a photodetector having a single photodetection element such as a photodetector may be used.

[0056] The image sensor 120 in this embodiment includes an electronic shutter. The electronic shutter is a circuit that controls the timing of exposure. The electronic shutter controls one signal accumulation period during which the received light is converted into a valid electrical signal and accumulated, and a period during which the signal accumulation is stopped. The signal accumulation period can also be called an "exposure period." The time from the end of one exposure period to the start of the next exposure period is sometimes called a "non-exposure period." Hereinafter, the exposed state may be expressed as "OPEN," and the stopped exposure state may be expressed as "CLOSE."

[0057] The image sensor 120 can adjust the exposure period and non-exposure period to sub-nanoseconds, for example, in the range of 30 ps to 1 ns, by using an electronic shutter. The control circuit 132 causes the light source 110 to emit a light pulse multiple times, and in synchronization therewith, causes the image sensor 120 to expose multiple times. This allows the signal to be integrated multiple times, improving sensitivity.

[0058] An example of the configuration of the image sensor 120 will now be described.

[0059] The image sensor 120 includes a plurality of photodetection cells arranged two-dimensionally on an imaging surface. In this specification, the photodetection cells are also called "pixels." Each pixel includes a photoelectric conversion element, such as a photodiode, and one or more charge storage units.

[0060] FIG. 2 is a diagram showing an example of the configuration of the image sensor 120. In FIG. 2, an area surrounded by a two-dot chain line frame corresponds to one pixel 201. The pixel 201 includes one photodiode (not shown). Although only four pixels arranged in two rows and two columns are shown in FIG. 2, a larger number of pixels may be arranged in practice. Each pixel 201 includes four floating diffusion layers 204, 205, 206, and 207. Signals accumulated in the four floating diffusion layers 204, 205, 206, and 207 are treated as if they were signals from four pixels of a general CMOS image sensor, and are output from the image sensor 120.

[0061] Each pixel 201 has four signal detection circuits. Each signal detection circuit includes a source follower transistor 309, a row selection transistor 308, and a reset transistor 310. Each transistor is, for example, but not limited to, a field effect transistor formed on a semiconductor substrate. As shown in the figure, one of the input terminal and the output terminal of the source follower transistor 309 is connected to one of the input terminal and the output terminal of the row selection transistor 308. The one of the input terminal and the output terminal of the source follower transistor 309 is typically a source. The one of the input terminal and the output terminal of the row selection transistor 308 is typically a drain. The gate, which is the control terminal of the source follower transistor 309, is connected to a photodiode (not shown). The signal charge by the holes or electrons generated by the photodiode is accumulated in the floating diffusion layer 204, which is a charge accumulation portion between the photodiode and the source follower transistor 309.

[0062] Although not shown in Fig. 2, the floating diffusion layers 204 to 207 are connected to a photodiode (not shown). A switch may be provided between the photodiode and each of the floating diffusion layers 204 to 207. This switch switches the conduction state between the photodiode and each of the floating diffusion layers 204 to 207 in response to a control signal from the control circuit 132. This controls the start and stop of accumulation of signal charges in each of the floating diffusion layers 204 to 207. The electronic shutter in this embodiment has a mechanism for such exposure control.

[0063] The signal charge accumulated in each floating diffusion layer is read out by turning on the gate of the row selection transistor 308 by the row selection circuit 302. At this time, the current flowing from the source follower power supply 305 to the source follower transistor 309 and the source follower load 306 is amplified according to the signal potential of the floating diffusion layer. An analog signal due to this current read out from the vertical signal line 304 is converted into digital signal data by an analog-digital (AD) conversion circuit 307 connected to each column. This digital signal data is read out for each column by the column selection circuit 303 and output from the image sensor 120. After reading out one row, the row selection circuit 302 and the column selection circuit 303 read out the next row, and so on in the same manner, reading out information on the signal charges of the floating diffusion layers of all rows. After reading out all the signal charges, the control circuit 132 resets all the floating diffusion layers by turning on the gate of the reset transistor 310. This completes the imaging of one frame. Similarly, high-speed imaging of frames is repeated thereafter, until imaging of a series of frames by the image sensor 120 is completed.

[0064] In this embodiment, an example of the CMOS type image sensor 120 has been described, but the image sensor 120 may be another type of imaging element. The image sensor 120 may be, for example, a CCD type, a single photon counting element, or an amplified image sensor such as an EMCCD or ICCD. Also, a sensor having a single pixel may be used instead of the image sensor 120. When a single-pixel sensor is used, distance measurement is possible for only one point, but distance detection at a high rate is possible.

[0065] [1-3.Electronic circuit 130] The electronic circuit 130 includes a control circuit 132, a signal processing circuit 134, and a recording medium such as a memory 136. The control circuit 132 adjusts the time difference between the emission timing of the light pulse of the light source 110 and the shutter timing of the image sensor 120. In this specification, the time difference is sometimes referred to as a "phase difference." The "emission timing" of the light source 110 refers to the timing when the light pulse emitted from the light source 20 starts to rise. The "shutter timing" refers to the timing when exposure starts.

[0066] The control circuit 132 may be configured to remove an offset component from a signal detected by each pixel of the image sensor 120. The offset component is a signal component due to ambient light such as sunlight or fluorescent light, or disturbance light. The offset component due to ambient light or disturbance light is estimated by detecting a signal by the image sensor 120 in a state in which the driving of the light source 110 is turned off and no light is emitted from the light source 110.

[0067] The control circuit 132 may be, for example, a processor such as a central processing unit (CPU), or an integrated circuit such as a microcontroller incorporating a processor and memory. The control circuit 132 adjusts, for example, emission timing and shutter timing by, for example, causing the processor to execute a computer program recorded in the memory 136.

[0068] The signal processing circuit 134 is a circuit that processes the image signal output from the image sensor 120. The signal processing circuit 134 performs arithmetic processing such as image processing. The signal processing circuit 134 can be realized by, for example, a digital signal processor (DSP), a programmable logic device (PLD) such as a field programmable gate array (FPGA), or a central processing unit (CPU) or an image processing arithmetic processor (GPU). The signal processing circuit 134 performs processing described below by the processor executing a computer program stored in the memory 136. The control circuit 132 and the signal processing circuit 134 may be an integrated single circuit, or may be separate individual circuits. The signal processing circuit 134 may be a component of an external device such as a server provided in a remote location. In this case, the external device such as the server transmits and receives data to and from the distance measuring device including the light source 110, the image sensor 120, and the control circuit 132 by wireless communication or wired communication. The signal processing circuit 134 executes a process to obtain a detailed distance to the object 50 located within a predetermined distance range from the distance measuring device 100, based on the signal output from the image sensor 120. Prior to this process, the signal processing circuit 134 may estimate an offset component due to ambient light and remove the offset component.

[0069] [2.Distance calculation] An example of a distance measurement calculation method executed by the electronic circuit 130 in this embodiment will be described below.

[0070] FIG. 3A is a diagram showing an example of a light emission pulse, a reflected light pulse, and a shutter timing. In the example shown in FIG. 3A, the distance measuring device 100 emits a rectangular light emission pulse Ie from the light source 110. To form a rectangular pulse, as the light source 110, for example, a light source such as a laser diode (LD) with a steep time response characteristic can be used. Thereby, for example, a light emission pulse Ie and a reflected light pulse Ir with a rise period and a fall period of 2 ns or less can be emitted. Also, a light emission pulse Ie and a reflected light pulse Ir with a rise period and a fall period of 1 ns or less can be emitted. By emitting a rectangular light emission pulse Ie, the relationship between the change in the detected light amount and the distance change can be made linear. The light emission pulse Ie is reflected by the object 50 and then reaches the image sensor 120 as the reflected light pulse Ir. The control circuit 132 controls the image sensor 120 to perform exposure with two different exposure periods while the reflected light pulse Ir is incident on the image sensor 120. The first exposure period starts before the reflected light pulse Ir reaches the image sensor 120 and stops after the start of the charge accumulation in the front-end portion including the front end of the reflected light pulse Ir. The second exposure period starts after the reflected light pulse Ir starts to reach the image sensor 120 and stops after the rear end of the reflected light pulse Ir reaches the image sensor 120. In this example, the first exposure period and the second exposure period do not overlap. That is, after the first exposure period ends, the second exposure period starts. Here, let the time width of the light emission pulse and the reflected light pulse be T0, the time from when the light source 110 starts emitting the light pulse until the first exposure period ends be t1, and the time from when the light source 110 starts emitting the light pulse until the second exposure period starts be t2. In the present embodiment, 0 < t2 - t1 < T0 holds. The control circuit 132 sends a control signal for starting the emission of the light emission pulse Ie to the light source 110 and sends a control signal for starting the first exposure period to the image sensor 120. At this time, there is a possibility that one control signal becomes noise of the other control signal. Therefore, for suppressing the noise, the timing of sending the control signal for starting the emission of the light emission pulse Ie and the timing of sending the control signal for starting the first exposure period to the image sensor 120 may be made different.In other words, the control circuit 132 may control the light source 110 and the image sensor 120 so that the timing at which the first exposure period starts is different from the timing at which the emission of the light emitting pulse starts. Similarly, the control circuit 132 may control the light source 110 and the image sensor 120 so that the timing at which the second exposure period starts is different from the timing at which the emission of the light emitting pulse ends.

[0071] By the above operation, in the first exposure period, charges generated by exposure of the front end portion including the front end of the reflected light pulse Ir are accumulated. On the other hand, in the second exposure period, charges generated by exposure of the rear end portion including the rear end of the reflected light pulse Ir are accumulated. A signal indicating the amount of charge accumulated in the first exposure period is S1, and a signal indicating the amount of charge accumulated in the second exposure period is S2. The intensities of the signals S1 and S2 vary according to the distance z between the object 50 and the image sensor 120. The distance z can be estimated using the following (Equation 1).

number

[0072] As can be seen from (Equation 1), the sum of S1 and S2 in the denominator is constant with respect to the phase change of the reflected light pulse. By using (Equation 1), the distance z can be changed linearly with respect to the change in S2. In this embodiment, the timing of the first and second exposure periods is set so as to acquire the components of the front end portion and the rear end portion of the reflected light pulse Ir, respectively. This maximizes the changes in S1 and S2 with respect to the change in distance of the object 50, and improves the distance resolution. From (Equation 1), the distance measurement error σz in this method can be calculated by the following (Equation 2) using the detection errors σs1 and σs2 of S1 and S2.

number

[0073] It can be seen from (Equation 2) that the distance measurement error σz can be reduced by reducing T0-(t2-t1). In this embodiment, the first exposure period and the second exposure period are set so as to obtain the components of the front end portion and the rear end portion of the reflected light pulse Ir, respectively, thereby reducing T0-(t2-t1) and reducing the distance measurement error.

[0074] Here, "obtaining a component of the front end portion of the reflected light pulse Ir" means detecting at least a part of the component of the rising edge period of the reflected light pulse Ir. Also, "obtaining a component of the rear end portion of the reflected light pulse Ir" means detecting at least a part of the component of the falling edge period of the reflected light pulse Ir. In this embodiment, the time point at which the intensity of the reflected light pulse Ir that has reached the photodetector starts to increase is called the "first time point", the time point at which the intensity of the reflected light pulse Ir stops increasing is called the "second time point", the time point at which the intensity of the reflected light pulse Ir starts to decrease is called the "third time point", and the time point at which the intensity of the reflected light pulse Ir stops decreasing is called the "fourth time point". The first exposure period includes the first time point, and the second exposure period includes the fourth time point. Thus, in this embodiment, the first exposure period includes the time point at which the reflected light pulse Ir starts to rise, and the second exposure period includes the time point at which the reflected light pulse Ir ends to fall. By controlling each exposure period in this manner, the accuracy of distance measurement can be improved.

[0075] The distance measurement calculation shown in (Equation 2) is particularly effective when the detection errors σs1 and σs2 are caused by illuminance fluctuations of the light source 110. This is because the dependence on the signals S1 and S2 on the right side of (Equation 2) is lower and the dependence on T0-(t2-t1) is higher than an error proportional to the square root of S1 and S2 such as shot noise. This is because the error caused by illuminance fluctuations is proportional to S1 and S2.

[0076] Therefore, in this embodiment, the control circuit 132 controls the image sensor 120 so as to satisfy 0 < t2 - t1 < T0 in order to acquire the components of the front end portion and the rear end portion of the reflected light pulse Ir. The control circuit 132 may control the image sensor 120 so as to satisfy T0 / 3 < t2 - t1 < T0, or T0 / 2 < t2 - t1 < T0. By such control, the ranging accuracy can be further improved. Thus, from the perspective of ranging resolution, it is effective that both S1 and S2 are the components of the end portion of the reflected light pulse Ir. Therefore, each of the first exposure period and the second exposure period can be set so as not to include the midpoint or the peak of the reflected light pulse Ir. Here, the midpoint refers to a point between the rising start point and the falling end point of the reflected light pulse Ir. By accumulating the charges of the components of both end portions of the reflected light pulse Ir so as not to include the midpoint or the peak, even higher-precision ranging is possible.

[0077] Although reducing the pulse width T0 also has the effect of improving the ranging accuracy, there is a limit to reducing T0 because the pulse shape becomes difficult to stabilize when the emission pulse is shortened. The pulse width T0 can be set to, for example, 3 ns or more, and in one example, 5 ns or more. Increasing the time difference (t2 - t1) between the first and second exposure periods rather than reducing the pulse width T0 enhances the followability of the electronic shutter and enables stable driving of the image sensor 120.

[0078] The signal S1 indicating the component of the front end portion of the reflected light pulse Ir is likely to saturate because the detection signal value increases due to both factors of increased illuminance and increased light amount due to phase shift when the distance to the object 50 approaches. Therefore, the first and second exposure periods may be set so that S1 < S2. In this case, the components of the rear end portion of the reflected light pulse Ir cancel each other out because the illuminance change and the phase change have opposite characteristics, and it is difficult for a change in S2 with respect to distance variation to occur. Therefore, S2 is likely to be maintained within the dynamic range even when the distance changes compared to S1, and thus the risk of saturation is small.

[0079] 3B is a diagram showing an example of another driving method of the distance measuring device 100. In this example, the distance measuring device 100 drives the shutter once for each reflected light pulse Ir. This allows a circuit design specialized for increasing the steepness of the front and rear end portions of the reflected light pulse Ir, thereby further improving distance measurement accuracy. The distance measuring device 100 may include two light sources 110 corresponding to the first and second exposure periods, respectively.

[0080] 3C is a diagram showing yet another example of the light emission pulse, the reflected light pulse, and the shutter timing. In this example, each of the first and second exposure periods is shorter than the pulse width of the reflected light pulse Ir. In this way, by shortening each of the first and second exposure periods, it is possible to reduce dark current noise and fixed pattern noise.

[0081] FIG. 3D is a diagram showing yet another example of the light emission pulse, the reflected light pulse, and the shutter timing. In this example, the light source 110 emits a light emission pulse Ie having a peak. The first exposure period starts before the start of the rising period of the reflected light pulse Ir and ends in the middle of the rising period. The second exposure period starts in the middle of the falling period of the reflected light pulse Ir and ends after the falling period ends. That is, the first exposure period ends at the inclined portion on the front end side of the reflected light pulse Ir, and the second exposure period starts at the inclined portion on the rear end side of the reflected light pulse Ir. In other words, the first exposure period ends before the second time point, and the second exposure period starts after the third time point. By such control, the change in the light amount detected with respect to the phase change of the reflected light pulse Ir accompanying the change in the distance of the object 50 becomes sensitive, and the distance detection sensitivity is improved. The steeper the inclination, the greater the rate of change in the light amount. Such control is particularly effective when the dominant factor of the noise component is shot noise or emission illuminance fluctuation, not phase fluctuation. This is because shot noise and emission illuminance fluctuation are uncorrelated with the shift in shutter timing. In the example shown in FIG. 3D, the control circuit 132 also adjusts the first and second shutter open periods so that the detected signal includes both end components of the reflected light pulse Ir and does not include the peak component. This can improve the distance resolution. In this example, the intensity change at the front end and rear end of the reflected light pulse Ir does not need to be steeper than in the example of FIG. 3A. For this reason, a light source such as an LED whose pulse rises and falls are not steep can be used as the light source 110. For example, using an LED as the light source 110 provides benefits such as increased emission intensity, improved safety, and reduced costs. In the example of FIG. 3D, the light emission pulse Ie and the reflected light pulse Ir have waveforms close to triangular waves, but these waveforms may have other shapes. For example, the light emission pulse Ie and the reflected light pulse Ir may have waveforms close to trapezoidal waves.

[0082] In each example of FIG. 3A to FIG. 3D, the first exposure period includes the time immediately after the rising edge of the reflected light pulse Ir, and the second exposure period includes the time when the falling edge of the reflected light pulse Ir ends. By controlling each exposure period in this manner, the change in the light amount of S1 and S2 with respect to the change in the distance of the object 50 can be kept large. For example, assume that the distance from the distance measuring device 100 to the object 50 is shortened from the reference distance. In this case, the reflected light pulse Ir arrives at the image sensor 120 earlier, so the waveform of the reflected light pulse Ir shown in FIG. 3A to FIG. 3D shifts to the left. At this time, if the first exposure period does not include the time when the rising edge of the reflected light pulse Ir from the reference distance starts, but is set after the rising edge starts, the change in the light amount of S1 does not occur or the change in the light amount becomes small. As a result, the distance resolution decreases. The same can be said about the falling edge. Therefore, in this embodiment, the first exposure period includes the start point of the rising period of the reflected light pulse Ir from the reference distance, and the first exposure period includes the end point of the falling period of the reflected light pulse Ir. This makes the change in the detection signal value sensitive to the change in distance to the object 50, improving the distance resolution.

[0083] 3. Operation of Electronic Circuit 130 FIG. 4A is a flowchart showing an example of the operation of the electronic circuit 130 in this embodiment. This flowchart corresponds to the operation shown in FIG. 3A. In this example, each pixel of the image sensor 120 includes a first charge storage section and a second charge storage section. The control circuit 132 in the electronic circuit 130 first causes the light source 110 to emit a light pulse (step S101). The time length of this light pulse is T0. After the emission of the light pulse, the control circuit 132 causes the electronic shutter to start accumulating charges in the first charge storage section of each pixel at a first timing (step S102). The first timing is before the rising edge start point of the reflected light pulse Ir returning from the object 50 when it is assumed that the object 50 exists at a reference distance set in advance from the distance measuring device 100. Next, the control circuit 132 causes the electronic shutter to stop accumulating charges in the first charge storage section of each pixel at a second timing (step S103). The second timing is after the rising edge start point of the reflected light pulse Ir. The operations of steps S101 to S103 detect the components of the reflected light pulse that have reached each pixel of the image sensor 120 between the first timing and the second timing. Then, the control circuit 132 causes the electronic shutter to start charge accumulation in the second charge accumulation section of each pixel at the third timing (step S104). The third timing is before the start of the fall of the reflected light pulse Ir. Next, the control circuit 132 causes the electronic shutter to stop charge accumulation in the second charge accumulation section of each pixel at the fourth timing (step S105). The fourth timing is after the end of the fall of the reflected light pulse Ir. The operations of steps S103 to S105 detect the components of the reflected light pulse that have reached each pixel of the image sensor 120 between the third timing and the fourth timing. Then, the control circuit 132 determines whether the number of times signal accumulation has been performed in each charge accumulation section has reached a predetermined number (step S106). If the result of this determination is No, steps S101 to S105 are repeated until the result is Yes. This repetition is performed when the amount of light is insufficient for one signal accumulation. The number of repetitions is set to an appropriate number according to the required S / N ratio.

[0084] If the result of the determination in step S106 is Yes, the control circuit 132 causes the signal processing circuit 134 to generate a first intensity map based on the signal charge stored in the first charge storage section of each pixel (step S107). Furthermore, the control circuit 132 causes the signal processing circuit 134 to generate a second intensity map based on the signal charge stored in the second charge storage section of each pixel (step S108). Each of the first intensity map and the second intensity map is image data having a pixel value that differs according to the flight time of the reflected light pulse. Next, the signal processing circuit 134 performs the arithmetic processing shown in the above-mentioned (Equation 1) from the data of the first and second intensity maps to obtain the distance of each pixel and generate a distance map (step S109). The distance map is image data in which each pixel has a distance value. The signal processing circuit 134 records the generated distance map in a recording medium such as a memory.

[0085] In step S109, the signal processing circuit 134 may determine the distance of each pixel by referring to a table stored in advance in a recording medium such as the memory 136, instead of calculating the distance by the operation shown in (Equation 1). Such a table may define, for example, the relationship between the range of values ​​of S2 / (S1+S2) and the distance corresponding to that range. The table may also define the relationship between the range of values ​​of S1 / (S1+S2) and the distance corresponding to that range.

[0086] The operations of steps S101 to S109 generate a distance map for one frame. The electronic circuit 130 may generate distance maps for multiple frames by repeating the operations of steps S101 to S109. The above operations allow the distance map, i.e., the distance image, of the object 50 to be generated with high accuracy.

[0087] FIG. 4B is a flowchart showing another example of the operation of the electronic circuit 130 in this embodiment. This flowchart corresponds to the operation shown in FIG. 3B. In this example, the charge accumulation in the first charge accumulation section and the charge accumulation in the second charge accumulation section are performed based on different light emission pulses, which is different from the example in FIG. 4A. The operations from steps S201 to S203 are the same as the operations from S101 to S103 in FIG. 4A. After step S203, in step S204, the control circuit 132 causes the light source 110 to emit another light pulse. Then, in steps S205 to S207, the same operations as S104 to S106 in FIG. 4A are performed. In step S207, when it is determined that a predetermined number of signal accumulations have been completed in each of the first charge accumulation section and the second charge accumulation section, the control circuit 132 causes the signal processing circuit 134 to perform the operations from steps S208 to S210. The operations in steps S208 to S210 are the same as the operations in steps S107 to S109 in FIG. 4A, respectively.

[0088] The operations of steps S201 to S210 generate a distance map for one frame. The electronic circuit 130 may generate distance maps for multiple frames by repeating the operations of steps S201 to S210. The above operations allow the distance map of the object 50 to be generated with high accuracy.

[0089] Note that multiple light emission and charge accumulation are not essential and are performed as necessary. Moreover, instead of allocating the accumulation of two types of signal charges for generating the first and second intensity maps to two charge accumulation units, it may be allocated to two different frames. For example, the shutter timing may be made different for each frame, and the first intensity map may be generated in odd frames and the second intensity map may be generated in even frames. It is also possible to make the shutter timing different for each pixel. In this way, it is possible to allocate and acquire the first intensity map and the second intensity map for each pixel. For example, the first intensity map may be acquired in a checkered pattern on the pixels, and the second intensity map may be acquired in the same checkered pattern on the remaining pixels. The first intensity map and the second intensity map may be allocated and acquired for each row of pixels. For example, the first intensity map may be acquired in odd rows of pixels, and the second intensity map may be acquired in even rows of pixels. A spatial interpolation process may be performed on the thinned pixels in each of the first and second intensity maps, and then a distance map may be generated by a calculation process based on the first and second intensity maps.

[0090] FIG. 4C is a flowchart showing an example of a process for generating a different intensity map for each frame. In this example, the control circuit 132 first causes the light source 110 to emit a light pulse for a predetermined time (step S301). The control circuit 132 causes the electronic shutter to start accumulating charges in the charge accumulation unit at a first timing (step S302). Next, the control circuit 132 causes the electronic shutter to stop accumulating charges in the charge accumulation unit at a second timing (step S303). Through these operations, a signal charge corresponding to a component of the front end portion of the reflected light pulse is accumulated in the charge accumulation unit. Next, the control circuit 132 determines whether or not the number of times the above signal accumulation has been performed reaches a predetermined number (step S304). If this determination is No, steps S301 to S303 are repeated until the determination is Yes. If the determination is Yes in step S304, the control circuit 132 causes the signal processing circuit 134 to generate a first intensity map based on the signal charge accumulated in the charge accumulation unit, and resets the signal charge in the charge accumulation unit (step S305). Next, the control circuit 132 causes the light source 110 to emit a light pulse for a predetermined time (step S306). The control circuit 132 causes the electronic shutter to start accumulating charges in the charge accumulation unit at a third timing (step S307). Next, the control circuit 132 causes the electronic shutter to stop accumulating charges in the charge accumulation unit at a fourth timing (step S308). Through these operations, a signal charge corresponding to the component of the trailing end portion of the reflected light pulse is accumulated in the charge accumulation unit. Next, the control circuit 132 determines whether or not the number of times the above signal accumulation has been performed reaches a predetermined number (step S309). If this determination is No, steps S306 to S308 are repeated until the determination is Yes. If the determination is Yes in step S309, the control circuit 132 causes the signal processing circuit 134 to generate a second intensity map based on the signal charge accumulated in the charge accumulation unit, and resets the signal charge in the charge accumulation unit (step S310). Next, the signal processing circuit 134 generates a distance map from the data of the first and second intensity maps by the method described above (step S311).

[0091] The operations from step S301 to S311 can generate a distance map of the object 50 with high accuracy. Although FIG. 4C shows the operations for two frames, the operations shown in FIG. 4C may be repeated to obtain distance maps for more frames. In the example of FIG. 4C, a distance map is generated once every two frames, but a distance map can also be generated for each frame. For example, a step similar to step S311 may be added between step S305 and step S306. In that case, in the added step, a distance map can be generated based on the first intensity map obtained in step S305 and the second intensity map obtained in the previous frame.

[0092] (Embodiment 2) Next, a distance measuring device 100 according to a second embodiment will be described.

[0093] The distance measuring device 100 of this embodiment differs from that of the first embodiment in that it has a driving mode for setting the first and second exposure periods so as to detect the ends of the reflected light pulse Ir. The following description will focus on the differences from the first embodiment, and a description of overlapping matters will be omitted.

[0094] The distance measuring device 100 drives the image sensor 120 to detect both ends of the reflected light pulse Ir in order to improve the distance measuring accuracy. However, since the timing at which the reflected light pulse Ir returns changes depending on the distance between the object 50 and the distance measuring device 100, there are cases where the ends of the reflected light pulse Ir cannot be detected with the preset shutter timing. In particular, the smaller T0-(t2-t1) is made to improve the distance measuring accuracy, the smaller the distance measuring range, i.e., the range of distances that can be measured, becomes, and either the first or second exposure period becomes more likely to fall outside the reflected light pulse Ir.

[0095] Therefore, in this embodiment, as shown in FIG. 5, an operation in a wide range mode, which is a distance measurement mode with a wide distance measurement range, is executed before the high-precision distance measurement mode in which only the components at both ends of the reflected light pulse Ir are detected. The electronic circuit 130 calculates an approximate distance in the wide range mode, and reflects the result in the adjustment of the shutter phase in the high-precision distance measurement mode. In the wide range mode, for example, as shown in FIG. 5, the image sensor 120 is driven so that either the first or second exposure period includes the midpoint of the reflected light pulse Ir. Here, the midpoint of the reflected light pulse Ir means the central time between the rising start time and the falling end time of the reflected light pulse Ir. In the wide range mode, the end time of the first exposure period coincides with the start time of the second exposure period. In the wide range mode, the distance measurement range depends on the pulse width T0. For example, when the pulse width T0 is 5 ns, a distance measurement range of about 75 cm can be secured. To further expand the distance measurement range, the pulse width T0 may be lengthened only in the wide range mode. In the wide range mode, the distance z is calculated by (Equation 3).

number

[0096] FIG. 6 is a flow chart showing an outline of the operation of adjusting the shutter timing within the distance measurement range. First, the control circuit 132 judges whether the end of the reflected light pulse can be detected in both the first and second exposure periods (step S401). Whether the end of the reflected light pulse can be detected in each exposure period is judged based on the intensity of the signals S1 and S2. If the judgment in step S401 is Yes, proceed to step S404. If the judgment in step S401 is No, proceed to step S402. If one or both of the signals S1 and S2 do not contain signal components other than those caused by the ambient light, the end cannot be detected and the judgment is No. Also, if one or both of the signals S1 and S2 have a value much larger than the expected value, it is considered that the center part of the reflected light pulse, not the end part, has been detected, and the judgment is No. If the judgment in step S401 is No, the control circuit 132 transitions to the wide range mode. In the wide range mode, the control circuit 132 drives the shutter at a timing when one of the two exposure periods includes the center time point of the reflected light pulse. The control circuit 132 calculates the distance to the object 50 based on (Equation 3), and calculates the amount of deviation of the phase difference between each exposure period and the reflected light pulse from the appropriate value based on the distance (step S402). Furthermore, based on the calculated amount of phase deviation, the control circuit 132 finely adjusts the relative timing between the light emission pulse and each exposure period, and detects both end components of the reflected light pulse Ir in the first and second exposure periods (S403). The appropriate phase difference changes depending on the distance to the object 50. Therefore, in order to finely adjust the relative timing, for example, a table that specifies the relationship between the distance and the appropriate phase difference may be held in advance. The control circuit 132 can adjust the light emission timing of the light emission pulse relative to the reference clock or the timing of each exposure period so as to obtain an appropriate phase difference for the detected distance by referring to the table. After step S403, or if it is determined that the answer is Yes in step S401, the signal processing circuit 134 calculates the distance based on Equation 1 from the detection signal values ​​obtained in the first and second exposure periods (step S404).

[0097] By the above operation, when the end of the reflected light pulse Ir cannot be detected with the preset shutter timing, the shutter timing can be adjusted so that the end of the reflected light pulse Ir can be detected. This makes it possible to obtain distance information stably and with high accuracy even if the distance between the object 50 and the distance measuring device 100 changes over time.

[0098] (Embodiment 3) The third embodiment relates to an application using the distance measuring device 100.

[0099] FIG. 7 is a diagram showing an example in which the distance measuring device 100 in the first or second embodiment is installed in the cabin of a car. In the example shown in FIG. 7, the distance measuring device 100 is provided in the cabin of a car. The distance measuring device 100 can acquire a 3D texture of the driver, who is the object 50 of distance measurement. By checking the acquired 3D texture data against a database, it is possible to identify whether the driver is the owner of the car. Instead of a car key, the distance measuring device 100 may identify the driver and start the car. During automatic driving, the distance measuring device 100 may identify the passenger, and the identification result may be fed back to the automatic driving control system to perform driving customized according to the passenger. For example, customization is possible, such as gentle braking for an elderly passenger or a passenger with a sore back, or exhilarating driving for a passenger who likes driving. Such feedback enables automatic driving that matches individual preferences.

[0100] 8A and 8B are diagrams showing an example in which the distance measuring device 100 in the first or second embodiment is applied to a mobile terminal. The mobile terminal is, for example, a smartphone, a tablet, or a personal computer. An image sensor built into the mobile terminal may be used as the image sensor 120 in the distance measuring device 100. An individual can be authenticated from 3D data of the user's face obtained from the distance measuring device 100. FIG. 8A shows an example of a screen of the mobile terminal during authentication. FIG. 8B shows an example of a screen of the mobile terminal when authentication is successful. Such an authentication mechanism is effective not only for login authentication to a mobile terminal, but also for login authentication to an online account. In this embodiment, by using 3D data of the user's face, the amount of information increases compared to conventional two-dimensional images, and therefore the identification accuracy is improved. This makes it possible to eliminate impersonation such as having a user's photo read in front of a camera, and to realize authentication with high security. In addition, in this embodiment, precise distance measurement by detecting the components at both ends of the reflected light pulse Ir allows for the formation of detailed 3D face data, making it possible to further improve the authentication accuracy compared to conventional 3D authentication. Therefore, the technology of the present embodiment is suitable for applications that require extremely high authentication accuracy, such as electronic payments using mobile terminals. The technology disclosed herein may be used as one of multi-step authentication methods for high-value payments, and may become a key technology in a cashless society.

[0101] The distance measuring device 100 may be applied to applications other than automobiles or mobile terminals, such as identifying account holders at ATMs installed in banks or convenience stores, unlocking the entrance to a house, or managing entrance and exit to an office.

[0102] The distance measuring device 100 is capable of not only measuring distance but also acquiring information about the inside of the skin of the object 50. An example of such a distance measuring device 100 will be described below. In the following description, the information about the inside of the skin of the object 50 will be referred to as "biometric information."

[0103] FIG. 9 is a diagram for explaining the operation of the distance measuring device 100 for acquiring biometric information of the object 50 in addition to distance information. In this example, in addition to the first and second exposure periods for distance measurement, a third exposure period for acquiring biometric information is provided. Each pixel of the image sensor 120 has three or more charge storage units. In the first exposure period, a signal S1 indicating a component of the front end portion of the reflected light pulse Ir is acquired. In the second exposure period, a signal S2 indicating a component of the rear end portion of the reflected light pulse Ir is acquired. The third exposure period starts at a timing that is further delayed from the start of the light emission pulse to the second exposure period. As a result, in the third exposure period, a signal S3 indicating a component in a falling period, which is a period from the start to the end of the decrease in the intensity of the reflected light pulse Ir, is acquired. By providing the third exposure period, for example, it is possible to detect a component of light with a relatively long optical path length scattered inside the skin. As a result, it is possible to clearly detect, for example, the blood vessel distribution centered on the vein.

[0104] FIG. 10 is a flowchart showing the operation of the electronic circuit 130 in the example shown in FIG. 9. In this flowchart, the operations from step S501 to S506 are the same as the operations from step S201 to S206 shown in FIG. 4B. In this embodiment, after step S506, the control circuit 132 causes the light source 110 to emit a light pulse (step S507). Next, the control circuit 132 causes the image sensor 120 to start charge accumulation in the third charge accumulation unit at a fifth timing (step S508). Thereafter, the control circuit 132 causes the image sensor 120 to stop charge accumulation in the third charge accumulation unit at a sixth timing (step S509). The operations from step S501 to S509 are repeated until it is determined in step S510 that a predetermined number of signal accumulations in each charge accumulation unit have been completed. When it is determined in step S510 that the signal accumulation is completed a predetermined number of times, the control circuit 132 causes the signal processing circuit 134 to generate first to third intensity maps based on the signal charges accumulated in the first to third charge accumulation units, respectively (steps S511 to S513). Then, similar to the above-described embodiment, the signal processing circuit 134 generates a distance map based on the first and second intensity maps (step S514). Furthermore, the signal processing circuit 134 generates a bioinformation map based on the third intensity map (step S515). The bioinformation map is image data generated by processing based on the signal S3 of each pixel. The bioinformation map can be, for example, image data showing blood vessel distribution.

[0105] By combining the blood vessel distribution obtained by the above process with 3D texture information based on the distance map, it becomes possible to identify individuals with such high accuracy that it is possible to distinguish between identical twins, for example. In addition, the blood vessel distribution may be displayed in three dimensions based on the distance information.

[0106] 11A and 11B show examples of images of vascular distribution displayed three-dimensionally. As shown in FIG. 11A, even if a face is photographed from a direction other than the front, it can be converted into an image of vascular distribution viewed from the front as shown in FIG. 11B by projective transformation using three-dimensional information. Such transformation makes it possible to improve the stability of matching between the vascular distribution acquired by photographing and a standard database stored in advance on a recording medium. As a result, it is possible to reduce the rejection rate of the person or the acceptance rate of other people. In addition, even if a user unintentionally photographs the face from an angle different from that of the database, it is possible to automatically correct and recognize the image on the terminal side, thereby increasing the recognition speed.

[0107] In the example of FIG. 9, three types of exposure periods are provided, but blood vessel information inside the skin may be acquired in the second exposure period instead of the third exposure period. By using near-infrared light of about 650 nm to 1000 nm that easily penetrates subcutaneously, it is possible to acquire a certain amount of blood vessel information regardless of the shutter timing. By acquiring information on internal blood vessels in the second exposure period, the number of shutter types can be reduced to two, and the frame rate can be improved. The database of blood vessel distribution stored in advance in the recording medium may be two-dimensional data or three-dimensional data. Alternatively, a database combining two-dimensional data of blood vessels and 3D texture data of the face may be used.

[0108] (Example) FIG. 12 shows the results of measuring the distance to the face of a mannequin of the same size as a human being using the distance measuring device 100. The distance between the distance measuring device 100 and the mannequin's face was 30 cm. The distance measuring device 100 was equipped with a lens with F2.5 and a horizontal angle of view of ±16.3°, and the number of pixels of the image sensor 120 was 320 × 240 pixels. The pulse width T0 of the light source 110, that is, the time from the start of the rise of the light emission pulse to the end, was set to 5 ns. The components of the reflected light pulse Ir from the reference distance were detected at 10% each on both ends, that is, the period from the start of the rise of the reflected light pulse Ir to 0.5 ns and the period 0.5 ns before the end of the fall. In this case, the distance measuring range was ±7.5 cm. Note that this distance measuring range is usually sufficient for detecting a 3D image of a face, but since there are individual differences in the size of the face, the shutter timing may be set to detect the inside of the reflected light pulse Ir by about 1 ns to allow for a margin.

[0109] 13 shows the results of a comparative example in which the same mannequin was actually measured using a conventional distance measuring method. In this comparative example, the distance z was calculated based on the following (Equation 4) instead of (Equation 1).

number

[0110] In the face image shown in Fig. 13 acquired by the conventional distance measurement method, the unevenness of each facial organ cannot be distinguished. In contrast, in the distance measurement method of this embodiment, as shown in Fig. 12, the surface shape and depth distribution of the face can be precisely acquired, and the effectiveness of the technology disclosed herein was confirmed. The variation in distance measurement calculated at 3σ was 0.12 mm, demonstrating that the technology disclosed herein can achieve high distance measurement accuracy. [Industrial Applicability]

[0111] The distance measuring device according to the present disclosure can obtain information on the distance to an object or the unevenness of its surface with high accuracy without contact, and is therefore useful in a wide range of fields such as object recognition, line inspection, and authentication. [Explanation of symbols]

[0112] 50 Objects 100 Rangefinder 110 Light source 120 Image Sensor 122 Photoelectric conversion unit 124 Charge storage section 130 Electronic circuits 132 Control circuit 134 Signal Processing Circuit

Claims

1. A light source; A photodetector; a circuit for controlling the light source and the light detector; Equipped with The circuit comprises: causing the light source to emit a light pulse toward a target; generating a first signal by causing the photodetector to detect the reflected light pulse during a first time period including at least a portion of a rise time period from a first time point at which an intensity of the reflected light pulse from the object caused by the light pulse starts to increase to a second time point at which the increase ends, the first time point being included and ending before the second time point; generating a second signal by causing the photodetector to detect the reflected light pulse in a second period following the end of the first period, the second period including at least a part of a falling period from a third time point at which the intensity of the reflected light pulse starts to decrease to a fourth time point at which the decrease ends, and including the fourth time point and starting after the third time point; generating and outputting data indicating a distance from the photodetector to the object based on the first signal and the second signal; Ranging device.

2. the first time period ends before a midpoint between the first time point and the fourth time point; the second time period begins after the midpoint.

2. A distance measuring device according to claim 1.

3. The pulse width of the light pulse is 3 ns or more and 30 ns or less.

3. A distance measuring device according to claim 1 or 2.

4. Let t1 be the time from the start of emission of the light pulse to the end of the first period, t2 be the time from the start of emission of the light pulse to the start of the second period, and T0 be the pulse width of the light pulse. T0 / 3<(t2-t1)<T0 Fulfilling 4. A distance measuring device according to claim 1.

5. A timing at which the first period starts is different from a timing at which the emission of the light pulse starts; or A timing at which the second period starts is different from a timing at which the emission of the light pulse ends.

5. A distance measuring device according to claim 1.

6. A ranging device as described in any one of claims 1 to 5, wherein the third point in time is different from the second point in time.

7. A ranging device as described in any one of claims 1 to 5, wherein the circuit causes the photodetector to detect the reflected light pulse so that the value of the second signal is greater than the value of the first signal.

8. A light source; A photodetector; a circuit for controlling the light source and the light detector; Equipped with The circuit comprises: causing the light source to emit a first light pulse and a second light pulse toward an object; generating a first signal by causing the photodetector to detect the first reflected light pulse during a first time period including at least a portion of a rise time period from a first time point at which an intensity of a first reflected light pulse from the object caused by the first light pulse starts to increase to a second time point at which the increase ends, the first time point being included and ending before the second time point; generating a second signal by causing the photodetector to detect the second reflected light pulse in a second time period including at least a part of a first falling period from a third time point at which an intensity of the second reflected light pulse from the object caused by the second light pulse starts to decrease to a fourth time point at which the decrease ends, and including the fourth time point and starting after the third time point; The circuit generates and outputs data indicating a distance from the photodetector to the object based on the first signal and the second signal. Ranging device.

9. the first time period ends before a midpoint between the first time point and a time point at which the decrease in intensity of the first reflected light pulse ends; the second period starts after a midpoint between a point in time at which the intensity of the second reflected light pulse starts to increase and the fourth point in time.

9. A distance measuring device according to claim 8.

10. The pulse width of each of the first light pulse and the second light pulse is not less than 3 ns and not more than 30 ns.

10. A distance measuring device according to claim 8 or 9.

11. Let t1 be the time from the start of emission of the first light pulse to the end of the first period, t2 be the time from the start of emission of the second light pulse to the start of the second period, and T0 be the pulse width of each of the first light pulse and the second light pulse. T0 / 3<(t2-t1)<T0 Fulfilling A distance measuring device according to any one of claims 8 to 10.

12. A timing at which the first period begins is different from a timing at which the emission of the first light pulse begins; or a start timing of the second period is different from a finish timing of emission of the second light pulse; 12. A distance measuring device according to claim 8.

13. The object is a living body, The light source further emits a third light pulse toward the object; the light detector further detects a third reflected light pulse resulting from the third light pulse returning from the object; the circuit causes the photodetector to detect the third reflected light pulse and output a third signal during a third period including at least a part of a second falling period from a fifth time point when the intensity of the third reflected light pulse starts to decrease to a sixth time point when the decrease ends and including the sixth time point; a length from the start of the third period to the sixth time point is shorter than a length from the start of the second period to the fourth time point; The circuit generates and outputs biological information of the object based on the third signal.

13. A distance measuring device according to claim 8.

14. one or more processors; a memory for storing a computer program to be executed by said one or more processors; Equipped with The one or more processors execute the computer program to: A light source is caused to emit a light pulse toward the target object; generating a first signal by causing a photodetector to detect the reflected light pulse during a first time period including at least a portion of a rise time period from a first time point at which an intensity of the reflected light pulse from the object caused by the light pulse starts to increase to a second time point at which the increase ends, the first time point being included and ending before the second time point; generating a second signal by causing the photodetector to detect the reflected light pulse in a second period following the end of the first period, the second period including at least a part of a falling period from a third time point at which the intensity of the reflected light pulse starts to decrease to a fourth time point at which the decrease ends, and including the fourth time point and starting after the third time point; generating and outputting data indicating a distance from the photodetector to the object based on the first signal and the second signal; Signal processing device.

15. one or more processors; a memory for storing a computer program to be executed by said one or more processors; Equipped with The one or more processors execute the computer program to: causing a light source to emit a first light pulse and a second light pulse toward a target; generating a first signal by causing a photodetector to detect the first reflected light pulse during a first time period including at least a portion of a rise time period from a first time point at which an intensity of a first reflected light pulse from the object caused by the first light pulse starts to increase to a second time point at which the increase ends, the first time point being included and ending before the second time point; generating a second signal by causing the photodetector to detect the second reflected light pulse in a second time period including at least a part of a falling period from a third time point at which an intensity of the second reflected light pulse from the object caused by the second light pulse starts to decrease to a fourth time point at which the decrease ends, and including the fourth time point and starting after the third time point; generating and outputting data indicating a distance from the photodetector to the object based on the first signal and the second signal; Signal processing device.

16. causing a light source to emit a light pulse toward a target; generating a first signal by causing a photodetector to detect the reflected light pulse during a first time period including at least a portion of a rise time period from a first time point at which an intensity of the reflected light pulse from the object caused by the light pulse starts to increase to a second time point at which the increase ends, the first time point being included and ending before the second time point; generating a second signal by causing the photodetector to detect the reflected light pulse during a second period following the end of the first period, the second period including at least a part of a falling period from a third time point at which the intensity of the reflected light pulse starts to decrease to a fourth time point at which the decrease ends, and including the fourth time point and starting after the third time point; and generating and outputting data indicative of a distance from the photodetector to the object based on the first signal and the second signal; causing a computer to perform operations including Computer program.

17. causing a light source to emit a first light pulse and a second light pulse toward a target; generating a first signal by causing a photodetector to detect the first reflected light pulse during a first time period including at least a portion of a rise time period from a first time point at which an intensity of a first reflected light pulse from the object caused by the first light pulse starts to increase to a second time point at which the increase ends, the first time point including the first time point and ending before the second time point; generating a second signal by causing the photodetector to detect the second reflected light pulse during a second time period including at least a part of a falling period from a third time point at which an intensity of the second reflected light pulse from the object caused by the second light pulse starts to decrease to a fourth time point at which the decrease ends, and including the fourth time point and starting after the third time point; and generating and outputting data indicative of a distance from the photodetector to the object based on the first signal and the second signal; causing a computer to perform operations including Computer program.

Images