Distance measuring device

Abstract

A distance measuring device according to a technical solution of the present disclosure includes a light source, a light detector, and a circuit. The circuit causes the light source to emit a light pulse toward an object, and causes the light detector to generate a first signal by detecting a reflected light pulse from the object generated by the light pulse during a first period, wherein the first period includes at least a portion of a rising period from a first time point when the intensity of the reflected light pulse starts to increase to a second time point when the increase ends, and includes the first time point. The circuit causes the light detector to generate a second signal by detecting the reflected light pulse during a second period, wherein the second period is a period after the first period ends, and includes at least a portion of a falling period from a third time point when the intensity of the reflected light pulse starts to decrease to a fourth time point when the decrease ends, and includes the fourth time point. The circuit generates and outputs data representing the distance from the light detector to the object based on the first signal and the second signal.

Description

Technical Field

[0001] The present disclosure relates to a distance measuring device. Background Art

[0002] In the past, a distance measurement method called Time of Flight (TOF) has been proposed, which uses the flight time of light to infer the distance to an object. In the TOF method, a light pulse is emitted from a light source toward an object, and a sensor driven by the pulse receives the return light. At this time, the amount of light detected by the sensor changes according to the time until the return light reaches the sensor. Therefore, the flight time of light can be indirectly inferred based on the change in the detected light amount, and the distance to the object can be calculated. Various methods have been proposed to improve the accuracy of distance estimation under the TOF method. For example, Patent Document 1 discloses a method for improving the distance measurement accuracy by detecting the duty cycle deviation of the light emitting waveform in advance and correcting the deviation.

[0003] Prior art literature

[0004] Patent Literature

[0005] Patent Document 1: Japanese Patent Application Publication No. 2017-32342 Summary of the invention

[0006] Problems to be solved by the invention

[0007] An object of the present disclosure is to provide a technology for further improving the accuracy of distance measurement based on the TOF method using light pulses.

[0008] Means used to solve problems

[0009] A distance measuring device according to a technical solution of the present disclosure includes: a light source; a photodetector; and a circuit for controlling the light source and the photodetector. The circuit causes the light source to emit a light pulse toward an object. The circuit causes the photodetector to generate a first signal by detecting a reflected light pulse from the object generated by the light pulse during a first period, wherein the first period includes at least a portion of a rising period from a first time point when the intensity of the reflected light pulse starts to increase to a second time point when the increase ends, and includes the first time point. The circuit causes the photodetector to generate a second signal by detecting the reflected light pulse during a second period, wherein the second period is a period after the first period ends, and includes at least a portion of a falling period from a third time point when the intensity of the reflected light pulse starts to decrease to a fourth time point when the decrease ends, and includes the fourth time point. The circuit generates and outputs data indicating the distance from the photodetector to the object based on the first signal and the second signal.

[0010] A distance measuring device according to another technical solution of the present disclosure includes: a light source; a photodetector; and a circuit for controlling the light source and the photodetector. The circuit causes the light source to emit a first light pulse and a second light pulse toward an object. The circuit causes the photodetector to generate a first signal by detecting a first reflected light pulse from the object generated by the first light pulse during a first period, wherein the first period includes at least a portion of a rising period from a first time point when the intensity of the first reflected light pulse starts to increase to a second time point when the increase ends, and includes the first time point. The circuit causes the photodetector to generate a second signal by detecting a second reflected light pulse from the object generated by the second light pulse during a second period, wherein the second period includes at least a portion of a falling period from a third time point when the intensity of the second reflected light pulse starts to decrease to a fourth time point when the decrease ends, and includes the fourth time point. The circuit generates and outputs data indicating the distance from the photodetector to the object based on the first signal and the second signal.

[0011] The inclusive or specific form of the present disclosure may also be implemented by a system, device, method, integrated circuit, computer program or computer-readable recording disk or other recording medium, or may be implemented by any combination of systems, devices, methods, integrated circuits, computer programs and recording media. Computer-readable recording media may include, for example, non-volatile recording media such as CD-ROM (Compact Disc-Read Only Memory). A device may also be composed of more than one device. In the case where a device is composed of more than two devices, the two or more devices may be configured in one device or separately configured in two or more separate devices. In this specification and claims, "device" can refer not only to one device but also to a system composed of multiple devices.

[0012] Effects of the Invention

[0013] According to the present disclosure, the distance to an object can be measured with higher accuracy than before. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 This is a schematic diagram showing a situation in which the distance measuring device according to Embodiment 1 captures an image of an object.

[0015] Figure 2 This is a diagram showing an example of the structure of an image sensor.

[0016] Figure 3A This is a diagram showing an example of a light emission pulse, a reflected light pulse, and a shutter timing.

[0017] Figure 3BThis is a diagram showing another example of light emission pulses, reflected light pulses, and shutter timing.

[0018] Figure 3C This is a diagram showing still another example of the light emission pulse, the reflected light pulse, and the shutter timing.

[0019] Figure 3D This is a diagram showing still another example of the light emission pulse, the reflected light pulse, and the shutter timing.

[0020] Figure 4A This is a flowchart showing an example of the operation of the electronic circuit.

[0021] Figure 4B This is a flowchart showing another example of the operation of the electronic circuit.

[0022] Figure 4C This is a flowchart showing still another example of the operation of the electronic circuit.

[0023] Figure 5 This is a diagram showing a driving method according to the second embodiment.

[0024] Figure 6 This is a flowchart showing an example of the operation of the electronic circuit according to the second embodiment.

[0025] Figure 7 This is a diagram showing an example in which a distance measuring device is used for personal authentication of a driver.

[0026] Fig. 8A This is a diagram showing an example of personal authentication performed by a portable terminal equipped with a distance measuring device.

[0027] Figure 8B This is another diagram showing an example of personal authentication performed by a portable terminal equipped with a distance measuring device.

[0028] Fig. 9 1 is a diagram showing an example in which the distance measuring device detects components of light scattered inside the skin of a subject in addition to distance measurement.

[0029] Fig.10 Yes means Fig. 9 A flowchart of an example of the operation of the electronic circuit in the example shown.

[0030] Fig.11A This figure shows an example of personal authentication using distance measurement information and biometric information.

[0031] Fig. 11B This is another diagram showing an example of personal authentication using distance measurement information and biometric information.

[0032] Fig.12 It is a figure which shows the distance measurement result of the face in Example.

[0033] Fig.13 It is a diagram showing the face distance measurement result in the comparative example. DETAILED DESCRIPTION

[0034] (Item 1)

[0035] The distance measuring device of item 1 of the present disclosure comprises: a light source; a light detector; and a circuit for controlling the light source and the light detector. The circuit causes the light source to emit a light pulse toward an object. The circuit causes the light detector to generate a first signal by detecting a reflected light pulse from the object generated by the light pulse during a first period, wherein the first period includes at least a part of a rising period from a first time point when the intensity of the reflected light pulse starts to increase to a second time point when the increase ends, and includes the first time point. The circuit causes the light detector to generate a second signal by detecting the reflected light pulse during a second period, wherein the second period is a period after the end of the first period, and includes at least a part of a falling period from a third time point when the intensity of the reflected light pulse starts to decrease to a fourth time point when the decrease ends, and includes the fourth time point. The circuit generates and outputs data representing the distance from the light detector to the object based on the first signal and the second signal.

[0036] (Item 2)

[0037] In the distance measuring device described in item 1, the first period may end before a midpoint between the first time point and the fourth time point, and the second period may start after the midpoint.

[0038] (Item 3)

[0039] In the distance measuring device described in item 1 or 2, the pulse width of the optical pulse may be greater than or equal to 3 ns and less than or equal to 30 ns.

[0040] (Item 4)

[0041] In the distance measuring device described in any one of items 1 to 3, when the time from the start of emission of the light pulse to the end of the first period is t1, the time from the start of emission of the light pulse to the start of the second period is t2, and the pulse width of the light pulse is T0, T0 / 3<(t2-t1) is satisfied. <T0。

[0042] (Item 5)

[0043] In the distance measuring device described in any one of items 1 to 4, the first period may end before the second time point, and the second period may start after the third time point.

[0044] (Item 6)

[0045] In the distance measuring device described in any one of items 1 to 5, the timing at which the first period starts may be different from the timing at which the emission of the light pulse starts; or the timing at which the second period starts may be different from the timing at which the emission of the light pulse ends.

[0046] (Item 7)

[0047] The distance measuring device according to item 7 of the present disclosure comprises: a light source; a photodetector; and a circuit for controlling the light source and the photodetector. The circuit causes the light source to emit a first light pulse and a second light pulse toward an object. The circuit causes the photodetector to generate a first signal by detecting a first reflected light pulse from the object generated by the first light pulse during a first period, wherein the first period includes at least a portion of a rising period from a first time point when the intensity of the first reflected light pulse starts to increase to a second time point when the increase ends, and includes the first time point. The circuit causes the photodetector to generate a second signal by detecting a second reflected light pulse from the object generated by the second light pulse during a second period, wherein the second period includes at least a portion of a first falling period from a third time point when the intensity of the second reflected light pulse starts to decrease to a fourth time point when the decrease ends, and includes the fourth time point. The circuit generates and outputs data indicating the distance from the photodetector to the object based on the first signal and the second signal.

[0048] (Item 8)

[0049] In the ranging device recorded in Item 7, the first period may end earlier than the midpoint between the first time point and the time point at which the intensity of the first reflected light pulse ends decreasing; and the second period may start later than the midpoint between the time point at which the intensity of the second reflected light pulse starts increasing and the fourth time point.

[0050] (Item 9)

[0051] In the distance measuring device described in item 7 or 8, each of the pulse widths of the first light pulse and the second light pulse may be greater than or equal to 3 ns and less than or equal to 30 ns.

[0052] (Item 10)

[0053] In the distance measuring device described in any one of items 7 to 9, when the time from the start of emission of the first light pulse to the end of the first period is t1, the time from the start of emission of the second light pulse to the start of the second period is t2, and the pulse width of each of the first light pulse and the second light pulse is T0, T0 / 3<(t2-t1) is satisfied. <T0。

[0054] (Item 11)

[0055] In the distance measuring device described in any one of items 7 to 10, the first period may end before the second time point, and the second period may start after the third time point.

[0056] (Item 12)

[0057] In the distance measuring device described in any one of items 7 to 11, the timing at which the first period starts may be different from the timing at which the emission of the first light pulse starts; or the timing at which the second period starts may be different from the timing at which the emission of the second light pulse ends.

[0058] (Item 13)

[0059] In the distance measuring device described in any one of items 7 to 12, it may also be that the above-mentioned object is a biological body; the above-mentioned light source also emits a third light pulse toward the above-mentioned object; the above-mentioned light detector also detects a third reflected light pulse caused by the above-mentioned third light pulse returning from the above-mentioned object; the above-mentioned circuit causes the above-mentioned light detector to detect the above-mentioned third reflected light pulse in a third period and output a third signal, and the above-mentioned third period includes at least a part of the second falling period from the fifth time point when the intensity of the above-mentioned third reflected light pulse begins to decrease to the sixth time point when the above-mentioned decrease ends, and includes the above-mentioned sixth time point; the length from the start of the above-mentioned third period to the above-mentioned sixth time point is shorter than the length from the start of the above-mentioned second period to the above-mentioned fourth time point; and the above-mentioned circuit generates and outputs the biological information of the above-mentioned object based on the above-mentioned third signal.

[0060] (Item 14)

[0061] The signal processing device related to Item 14 of the present disclosure comprises: one or more processors; and a memory storing a computer program executed by the one or more processors; the one or more processors, by executing the computer program, cause a light source to emit a light pulse toward an object; cause a light detector to generate a first signal by detecting a reflected light pulse from the object generated by the light pulse during a first period, wherein the first period includes at least a portion of a rising period from a first time point when the intensity of the reflected light pulse starts to increase to a second time point when the increase ends, and includes the first time point; cause the light detector to generate a second signal by detecting the reflected light pulse during a second period, wherein the second period is a period after the end of the first period, includes at least a portion of a falling period from a third time point when the intensity of the reflected light pulse starts to decrease to a fourth time point when the decrease ends, and includes the fourth time point; and based on the first signal and the second signal, generate and output data representing the distance from the light detector to the object.

[0062] (Item 15)

[0063] The signal processing device related to item 15 of the present disclosure comprises: one or more processors; and a memory storing a computer program executed by the one or more processors; the one or more processors, by executing the computer program, cause a light source to emit a first light pulse and a second light pulse toward an object; cause a light detector to generate a first signal by detecting a first reflected light pulse from the object generated by the first light pulse during a first period, the first period including at least a portion of a rising period from a first time point when the intensity of the first reflected light pulse starts to increase to a second time point when the increase ends, and includes the first time point; cause the light detector to generate a second signal by detecting a second reflected light pulse from the object generated by the second light pulse during a second period, the second period including at least a portion of a falling period from a third time point when the intensity of the second reflected light pulse starts to decrease to a fourth time point when the decrease ends, and includes the fourth time point; and generate and output data indicating a distance from the light detector to the object based on the first signal and the second signal.

[0064] (Item 16)

[0065] The computer program related to Item 16 of the present disclosure causes a computer to execute an action, which includes: causing a light source to emit a light pulse toward an object; causing a light detector to generate a first signal during a first period by detecting a reflected light pulse from the object caused by the light pulse, wherein the first period includes at least a portion of a rising period from a first time point when the intensity of the reflected light pulse starts to increase to a second time point when the increase ends, and includes the first time point; causing the light detector to generate a second signal during a second period by detecting the reflected light pulse, wherein the second period is a period after the end of the first period, includes at least a portion of a falling period from a third time point when the intensity of the reflected light pulse starts to decrease to a fourth time point when the decrease ends, and includes the fourth time point; and generating and outputting data representing the distance from the light detector to the object based on the first signal and the second signal.

[0066] (Item 17)

[0067] The computer program related to Item 17 of the present disclosure causes a computer to execute an action, the action including: causing a light source to emit a first light pulse and a second light pulse toward an object; causing a light detector to generate a first signal by detecting a first reflected light pulse from the object generated by the first light pulse during a first period, the first period including at least a portion of a rising period from a first time point when the intensity of the first reflected light pulse starts to increase to a second time point when the increase ends, and including the first time point; causing the light detector to generate a second signal by detecting a second reflected light pulse from the object generated by the second light pulse during a second period, the second period including at least a portion of a falling period from a third time point when the intensity of the second reflected light pulse starts to decrease to a fourth time point when the decrease ends, and including the fourth time point; based on the first signal and the second signal, generating and outputting data representing the distance from the light detector to the object.

[0068] (Item 18)

[0069] The computer-readable non-transitory recording medium according to Item 18 of the present disclosure records the computer program according to Item 16 .

[0070] (Item 19)

[0071] The computer-readable non-transitory recording medium according to Item 19 of the present disclosure records the computer program according to Item 17 .

[0072] The embodiments described below all represent inclusive or specific examples. The numerical values, shapes, materials, constituent elements, configuration positions and connection forms of constituent elements, steps, the order of steps, etc. shown in the following embodiments are examples and are not intended to limit the present disclosure. Among the constituent elements of the following embodiments, constituent elements that are not recorded in the independent claims representing the highest concept are described as arbitrary constituent elements. Each figure is a schematic diagram and is not necessarily a strict illustration. Furthermore, in each figure, the same reference numerals are given to substantially the same or similar constituent elements. There are cases where repeated descriptions are omitted or simplified.

[0073] In the present disclosure, all or part of a circuit, unit, device, component or part, or all or part of a functional block in a block diagram may be performed by one or more electronic circuits including a semiconductor device, a semiconductor integrated circuit (IC) or an LSI (large scale integration). An LSI or an IC may be integrated into one chip or may be composed by combining multiple chips. For example, functional blocks other than storage elements may be integrated into one chip. Here, it is referred to as LSI or IC, but the name may change depending on the degree of integration, and may also be referred to as system LSI, VLSI (very large scale integration) or ULSI (ultra large scale integration). A Field Programmable Gate Array (FPGA) that can be programmed after the manufacture of the LSI, or a reconfigurable logic device that can reconfigure the bonding relationship within the LSI or set the circuit division within the LSI may also be used for the same purpose.

[0074] Furthermore, all or part of the functions or operations of a circuit, unit, device, component or part can be executed by software processing. In this case, the software is recorded in one or more non-temporary recording media such as ROMs, optical disks, hard disk drives, etc. When the software is executed by a processor, the functions determined by the software are executed by the processor and peripheral devices. The system or device may also have one or more non-temporary recording media with software recorded, a processor, and necessary hardware devices, such as interfaces.

[0075] First, an overview of the embodiments of the present disclosure will be described.

[0076] The distance measuring device according to the exemplary embodiment of the present disclosure includes a light source, a photodetector, and a circuit for controlling the light source and the photodetector. The circuit performs the following operations (a) to (d).

[0077] (a) The light source is caused to emit light pulses.

[0078] (b) causing the light detector to generate a first signal during a first period by detecting a reflected light pulse from an object caused by the light pulse, wherein the first period includes at least a portion of a rising period from a first time point when the intensity of the reflected light pulse starts to increase to a second time point when the increase ends, and includes the first time point.

[0079] (c) causing the light detector to generate a second signal by detecting the reflected light pulse during a second period, wherein the second period is a period after the first period ends, including at least a portion of a falling period from a third time point when the intensity of the reflected light pulse begins to decrease to a fourth time point when the decrease ends, and including the fourth time point.

[0080] (d) generating and outputting data indicating the distance to the object based on the first signal and the second signal.

[0081] According to the above structure, the photodetector generates: a first signal representing a component of a front end portion including a front end of a reflected light pulse, the front end of the reflected light pulse including at least a first time point when a rising period of the reflected light pulse begins; and a second signal representing a component of a rear end portion including a rear end of the reflected light pulse, the rear end of the reflected light pulse including at least a fourth time point when a falling period of the reflected light pulse ends. Furthermore, the above circuit calculates or infers the distance to the object based on the first signal and the second signal. By such an operation, as described in detail later, the distance measurement accuracy can be improved.

[0082] The first period may end earlier than the midpoint between the first time point and the fourth time point. The second period may start later than the midpoint. Thus, the component of the midpoint with high intensity of the reflected light pulse will not be included in the first signal and the second signal. Therefore, the ranging accuracy can be further improved.

[0083] The pulse width of the light pulse can be set to be, for example, 3 ns or more and 30 ns or less. Here, the pulse width of the light pulse refers to the time length from the start time point of the rise to the end time point of the fall of the light pulse. The pulse width can also be less than 3 ns or more than 30 ns.

[0084] When the time from the start of the emission of the above-mentioned optical pulse to the end of the above-mentioned first period is set as t1, the time from the start of the emission of the above-mentioned optical pulse to the start of the above-mentioned second period is set as t2, and the pulse width of the above-mentioned optical pulse is T0, it is also possible to satisfy T0 / 3 < (t2 - t1) < T0. Alternatively, it is also possible to satisfy T0 / 2 < (t2 - t1) < T0. When such conditions are satisfied, the ranging accuracy can be further improved.

[0085] The ranging device according to another embodiment of the present disclosure includes a light source, a light detector, and a circuit that controls the above-mentioned light source and the above-mentioned light detector. The above-mentioned circuit performs the following operations (a') to (d').

[0086] (a') Cause the above-mentioned light source to emit a first optical pulse and a second optical pulse.

[0087] (b') Cause the above-mentioned light detector to generate a first signal by detecting a first reflected optical pulse from an object due to the above-mentioned first optical pulse during a first period. The above-mentioned first period includes at least a part of a rising period from a first time point when the intensity of the above-mentioned first reflected optical pulse starts to increase to a second time point when the increase ends, and includes the above-mentioned first time point.

[0088] (c') Cause the above-mentioned light detector to generate a second signal by detecting a second reflected optical pulse from the above-mentioned object due to the above-mentioned second optical pulse during a second period. The above-mentioned second period includes at least a part of a falling period from a third time point when the intensity of the above-mentioned second reflected optical pulse starts to decrease to a fourth time point when the decrease ends, and includes the above-mentioned fourth time point.

[0089] (d') Generate and output data representing the distance to the above-mentioned object based on the above-mentioned first signal and the above-mentioned second signal.

[0090] According to the above structure, the light detector generates: a first signal representing a component of the front end portion of the above-mentioned first reflected optical pulse including at least the first time point at the start of the rising period of the above-mentioned first reflected optical pulse; and a second signal representing a component of the rear end portion of the above-mentioned second reflected optical pulse including at least the fourth time point at the end of the falling period of the above-mentioned second reflected optical pulse. And, the above-mentioned circuit calculates or infers the distance to the object based on the first signal and the second signal. Through such operations, the ranging accuracy can be improved as described later.

[0091] The above-mentioned first period may also end earlier than the midpoint between the above-mentioned first time point and the time point when the decrease in the intensity of the above-mentioned first reflected light pulse ends. The above-mentioned second period may also start later than the midpoint between the time point when the increase in the intensity of the above-mentioned second reflected light pulse starts and the above-mentioned fourth time point. Thus, the component of the midpoint with a high intensity of the reflected light pulse is not included in the first signal and the second signal. Therefore, the ranging accuracy can be further improved.

[0092] The pulse width of each of the above-mentioned first light pulse and the above-mentioned second light pulse can be set, for example, to be 3 ns or more and 30 ns or less. The pulse width of each light pulse can also be set to a value outside the above-mentioned range.

[0093] When the time from the start of the emission of the above-mentioned first light pulse to the end of the above-mentioned first period is t1, the time from the start of the emission of the above-mentioned second light pulse to the start of the above-mentioned second period is t2, and the pulse width of the above-mentioned light pulse is T0, it may also satisfy T0 / 3 < (t2 - t1) < T0. Alternatively, it may also satisfy T0 / 2 < (t2 - t1) < T0. When such conditions are satisfied, the ranging accuracy can be further improved.

[0094] The above-mentioned object may be a living body, for example. Living bodies include humans and animals. In an example, the object is a human head, and the circuit generates data representing the distance distribution of the human face.

[0095] The above-mentioned light source may also emit a third light pulse toward the above-mentioned object. The above-mentioned light detector may also detect a third reflected light pulse resulting from the above-mentioned third light pulse returning from the above-mentioned object. The above-mentioned circuit may also cause the above-mentioned light detector to detect the above-mentioned third reflected light pulse during a third period and output a third signal. The third period includes at least a part of the descending period from the fifth time point when the intensity of the above-mentioned third reflected light pulse starts to decrease to the sixth time point when the decrease ends, and includes the above-mentioned sixth time point. The length from the start of the above-mentioned third period to the above-mentioned sixth time point can be set to a value shorter than the length from the start of the above-mentioned second period to the above-mentioned fourth time point. The above-mentioned circuit may also generate and output the biological information of the above-mentioned object based on the above-mentioned third signal.

[0096] According to the above structure, it is possible to obtain biological information with high accuracy not only based on the distance information of the object person but also based on the component at the rear end of the third reflected light pulse. The component at the rear end of the reflected light pulse includes the component of the light scattered inside the skin of the object person. By detecting such a component, biological information such as the distribution of blood vessels of the object person can be obtained.

[0097] A signal processing device according to another embodiment of the present disclosure includes one or more processors and a memory storing a computer program executed by the one or more processors. The processor may also execute the function of the circuit in any of the above examples by executing the computer program.

[0098] The present disclosure includes a computer program that specifies the functions of the above-mentioned circuit and a control method executed by the above-mentioned circuit.

[0099] Hereinafter, more specific embodiments of the present disclosure will be described with reference to the drawings.

[0100] (Implementation Method 1)

[0101] [1.Structure]

[0102] First, the configuration of the distance measuring device 100 according to the first embodiment will be described.

[0103] Figure 1 1 is a schematic diagram showing a distance measuring device 100 according to the present embodiment. The distance measuring device 100 includes a light source 110, an image sensor 120, and an electronic circuit 130. The image sensor 120 includes a photoelectric conversion unit 122 and a charge storage unit 124. The electronic circuit 130 includes a control circuit 132, a signal processing circuit 134, and a memory 136. Figure 1 2 also shows an object 50 for distance measurement by the distance measuring device 100. In this embodiment, the object 50 is a human head. The object 50 is not limited to a human head, and may be any object.

[0104] [1-1. Light source 110]

[0105] The light source 110 irradiates light toward the object 50. The light irradiated from the light source 110 and reaching the object 50 is reflected by the object 50, and a part of the light reaches the image sensor 120. The light source 110 generates light pulses a plurality of times at predetermined time intervals or timings.

[0106] Since the object 50 of the distance measuring device 100 of this embodiment is a person, a light source 110 designed in consideration of the effect on the retina can be used. For example, a light source 110 that satisfies the level 1 of the laser safety standard established by various countries can be used. In this case, low-illuminance light with a radiation emission limit (AEL) of less than 1 mW is irradiated to the object 50. However, the light source 110 itself does not have to satisfy level 1. For example, the level 1 of the laser safety standard can be satisfied by placing a diffuser or an ND filter in front of the light source 110 to diffuse or attenuate the light.

[0107] The light source 110 can be, for example, a light source formed by a general-purpose semiconductor laser. When a general-purpose semiconductor laser is driven at a low voltage, if the pulse width is too short, it is difficult to follow the driving of the light on and off. Therefore, the light emission waveform becomes different according to each pulse emission, which is easy to present unstable dynamics and easily cause deviations in the distance measurement results. In order to obtain a stable waveform using a general-purpose semiconductor laser, for example, the light source 110 can be controlled so that a light pulse with a pulse width of more than 3ns is emitted. Alternatively, in order to further stabilize it, the light source 110 can also emit a light pulse with a pulse width of more than 5ns, and then more than 10ns. On the other hand, when the pulse width is too large, the light outflow to the charge accumulation part 124 when the shutter is closed, that is, the parasitic light sensitivity (PLS) becomes larger, which may cause distance measurement errors. Therefore, the light source 110 can be controlled so that it generates, for example, a light pulse with a pulse width of less than 50ns. Alternatively, the light source 110 can also emit a light pulse with a pulse width of less than 30ns, and then less than 20ns.

[0108] In addition, a pattern with uniform intensity distribution in the irradiation area may be selected as the irradiation pattern of the light source 110. This is because, in this case, light can be irradiated to the object 50 with spatially equal illuminance, and the intensity of the detection signal can be included in the dynamic range in any pixel of the image sensor 120.

[0109] [1-2. Image sensor 120]

[0110] The image sensor 120 is an example of a light detector that receives light emitted from the light source 110 and reflected from the object 50. The image sensor 120 has a plurality of light detection units arranged two-dimensionally, and can obtain two-dimensional information of the object 50 at one time. The image sensor 120 can be, for example, any imaging element such as a CCD image sensor or a CMOS image sensor. The light detector disclosed in the present invention is not limited to a sensor having a plurality of light detection units arranged two-dimensionally, and can also be a sensor having a plurality of light detection units arranged one-dimensionally, for example. In addition, in applications where it is sufficient to obtain distance information of only one point, a light detector having a single light detection element such as a photodetector can also be used.

[0111] The image sensor 120 of this embodiment is equipped with an electronic shutter. The electronic shutter is a circuit that controls the timing of exposure. The electronic shutter controls a signal accumulation period in which the received light is converted into an effective electrical signal and accumulated, and a period in which the signal accumulation is stopped. The signal accumulation period can also be referred to as an "exposure period". There is a case where the time from the end of one exposure period to the beginning of the next exposure period is referred to as a "non-exposure period". In the following, there is a case where the state of exposure is expressed as "OPEN (open)", and the state of exposure is stopped is expressed as "CLOSE (closed)".

[0112] The image sensor 120 can adjust the exposure period and the non-exposure period in the sub-nanosecond range, for example, 30ps to 1ns, by means of an electronic shutter. The control circuit 132 causes the light source 110 to emit light pulses multiple times, and synchronously causes the image sensor 120 to expose multiple times. Thus, the signal is accumulated over multiple times, and the sensitivity can be improved.

[0113] Hereinafter, a configuration example of the image sensor 120 will be described.

[0114] The image sensor 120 includes a plurality of light detection units arranged two-dimensionally on an imaging surface. In this specification, a light detection unit is also referred to as a “pixel.” Each pixel includes a photoelectric conversion element such as a photodiode and one or more charge storage units.

[0115] Figure 2 1 is a diagram showing an example of the structure of the image sensor 120. Figure 2 In FIG. 2 , a region surrounded by a two-dot chain line corresponds to one pixel 201. The pixel 201 includes one photodiode (not shown). Figure 2 Only four pixels arranged in two rows and two columns are shown, but more pixels can be configured in practice. Each pixel 201 includes four floating diffusion layers 204, 205, 206, and 207. The signals accumulated in the four floating diffusion layers 204, 205, 206, and 207 are processed like the signals of four pixels of a normal CMOS image sensor and output from the image sensor 120.

[0116] Each pixel 201 has four signal detection circuits. Each signal detection circuit includes a source follower transistor 309, a row selection transistor 308 and a reset transistor 310. Each transistor is, for example, a field effect transistor formed on a semiconductor substrate, but is not limited to this. As shown in the figure, one of the input terminal and the output terminal of the source follower transistor 309 is connected to one of the input terminal and the output terminal of the row selection transistor 308. The above-mentioned one of the input terminal and the output terminal of the source follower transistor 309 is typically a source. The above-mentioned one of the input terminal and the output terminal of the row selection transistor 308 is typically a drain. The gate of the source follower transistor 309, which serves as a control terminal, is connected to a photodiode not shown. The signal charge brought by the holes or electrons generated by the photodiode is accumulated in the floating diffusion layer 204, which serves as a charge accumulation unit between the photodiode and the source follower transistor 309.

[0117] Although in Figure 2 Although not shown in the figure, the floating diffusion layers 204 to 207 are connected to photodiodes not shown. A switch may be provided between the photodiodes and each of the floating diffusion layers 204 to 207. The switch switches the conduction state between the photodiodes and each of the floating diffusion layers 204 to 207 according to a control signal from the control circuit 132. Thus, the start and stop of the accumulation of signal charges to each of the floating diffusion layers 204 to 207 are controlled. The electronic shutter of this embodiment has a mechanism for such exposure control.

[0118] The signal charge accumulated in each floating diffusion layer is read out by turning on the gate of the row selection transistor 308 by the row selection circuit 302. At this time, the current flowing from the source follower power supply 305 to the source follower transistor 309 and the source follower load 306 is amplified according to the signal potential of the floating diffusion layer. The analog signal formed by the current read out from the vertical signal line 304 is converted into digital signal data by the analog-to-digital (AD) conversion circuit 307 connected to each column. The digital signal data is read out by each column by the column selection circuit 303 and output from the image sensor 120. After reading out one row, the row selection circuit 302 and the column selection circuit 303 read out the next row, and similarly read out the information of the signal charge of the floating diffusion layers of all rows. After reading out all the signal charges, the control circuit 132 resets all the floating diffusion layers by turning on the gate of the reset transistor 310. Thus, the shooting of one frame is completed. Similarly, by repeating the high-speed imaging of frames, a series of frame imaging by the image sensor 120 is completed.

[0119] In this embodiment, an example of a CMOS type image sensor 120 is described, but the image sensor 120 may be another type of imaging element. The image sensor 120 may be, for example, a CCD type, a single photon counting element, or an amplified image sensor such as an EMCCD or ICCD. In addition, a sensor having a single pixel may be used instead of the image sensor 120. When a single-pixel sensor is used, distance measurement can only be performed on one point, but distance detection at a high speed is possible.

[0120] [1-3. Electronic circuit 130]

[0121] The electronic circuit 130 includes a control circuit 132, a signal processing circuit 134, and a recording medium such as a memory 136. The control circuit 132 adjusts the time difference between the emission timing of the light pulse of the light source 110 and the shutter timing of the image sensor 120. In this specification, this time difference is sometimes referred to as a "phase difference". The "emission timing" of the light source 110 refers to the timing when the light pulse emitted from the light source 20 starts to rise. The "shutter timing" refers to the timing when exposure starts.

[0122] The control circuit 132 may also be configured to remove an offset component from a signal detected by each pixel of the image sensor 120. The offset component is a signal component caused by ambient light such as sunlight or fluorescent light, or interference light. The offset component caused by ambient light or interference light is estimated by detecting a signal with the image sensor 120 while the drive of the light source 110 is turned off and light is not emitted from the light source 110.

[0123] The control circuit 132 may be, for example, a processor such as a central processing unit (CPU) or an integrated circuit such as a microcontroller having a built-in processor and memory. The control circuit 132 adjusts, for example, the emission timing and the shutter timing by, for example, executing a computer program recorded in the memory 136 through the processor.

[0124] The signal processing circuit 134 is a circuit for processing the image signal output from the image sensor 120. The signal processing circuit 134 performs arithmetic processing such as image processing. The signal processing circuit 134 can be implemented by, for example, a programmable logic device (PLD) such as a digital signal processor (DSP), a field programmable gate array (FPGA), or a central processing unit (CPU) or a graphics processing unit (GPU). The signal processing circuit 134 performs the processing described below by executing a computer program stored in the memory 136 by the processor. The control circuit 132 and the signal processing circuit 134 can be either an integrated circuit or a separate circuit. The signal processing circuit 134 can also be a component of an external device such as a server located at a remote location. In this case, the external device such as the server transmits and receives data to and from the distance measuring device including the light source 110, the image sensor 120 and the control circuit 132 through wireless communication or wired communication. The signal processing circuit 134 performs processing to obtain a detailed distance to the object 50 located within a predetermined distance range of the distance measuring device 100 based on the signal output from the image sensor 120. The signal processing circuit 134 may estimate and remove an offset component caused by disturbance light before this processing.

[0125] [2. Ranging calculation]

[0126] An example of a distance measurement calculation method executed by the electronic circuit 130 of the present embodiment will be described below.

[0127] Figure 3A is a diagram showing an example of a light emitting pulse, a reflected light pulse, and a shutter timing. Figure 3AIn the example shown, the distance measuring device 100 emits a rectangular light emission pulse Ie from the light source 110. In order to form a rectangular pulse, for example, a light source such as a laser diode (LD) with a steep time response characteristic can be used as the light source 110. Thus, for example, a light emission pulse Ie and a reflected light pulse Ir with a rise period and a fall period of 2 ns or less can be emitted. In addition, a light emission pulse Ie and a reflected light pulse Ir with a rise period and a fall period of 1 ns or less can be emitted. By emitting a rectangular light emission pulse Ie, the relationship between the change in the detected light amount and the change in distance can be made linear. The light emission pulse Ie is reflected by the object 50 and then reaches the image sensor 120 as the reflected light pulse Ir. During the period when the reflected light pulse Ir is incident on the image sensor 120, the control circuit 132 controls the image sensor 120 to perform exposure during two different exposure periods. The first exposure period starts before the reflected light pulse Ir reaches the image sensor 120 and stops after the start of charge accumulation in the front end portion including the front end of the reflected light pulse Ir. The second exposure period starts after the reflected light pulse Ir reaches the image sensor 120 and stops after the rear end of the reflected light pulse Ir reaches the image sensor 120. In this example, the first exposure period and the second exposure period do not overlap. That is, the second exposure period starts after the end of the first exposure period. Here, let the time width of the light emission pulse and the reflected light pulse be T0, the time from the start of the light pulse emission from the light source 110 to the end of the first exposure period be t1, and the time from the start of the light pulse emission from the light source 110 to the start of the second exposure period be t2. In the present embodiment, 0 < t2 - t1 < T0 holds. The control circuit 132 sends a control signal for starting the emission of the light emission pulse Ie to the light source 110 and a control signal for starting the first exposure period to the image sensor 120. At this time, it is possible that one control signal becomes noise of the other control signal. Therefore, in order to suppress this noise, the timing of sending the control signal for starting the emission of the light emission pulse Ie may be made different from the timing of sending the control signal for starting the first exposure period to the image sensor 120. In other words, the control circuit 132 may also control the light source 110 and the image sensor 120 so that the timing of starting the first exposure period is different from the timing of starting the emission of the light emission pulse. Similarly, the control circuit 132 may also control the light source 110 and the image sensor 120 so that the timing of starting the second exposure period is different from the timing of the end of the emission of the light emission pulse.

[0128] Through the above-mentioned operation, the charge generated by the exposure of the front end portion including the front end of the reflected light pulse Ir is accumulated during the first exposure period. On the other hand, the charge generated by the exposure of the rear end portion including the rear end of the reflected light pulse Ir is accumulated during the second exposure period. Let the signal indicating the amount of charge accumulated during the first exposure period be S1, and let the signal indicating the amount of charge accumulated during the second exposure period be S2. The intensity of each of the signals S1 and S2 varies according to the distance z between the object 50 and the image sensor 120. The distance z can be estimated using the following (Formula 1).

[0129] [Formula 1]

[0130]

[0131] Here, c(≈3.0×10 8 m / s) represents the speed of light.

[0132] According to (Formula 1), the sum of S1 and S2 in the denominator is constant with respect to the phase change of the reflected light pulse. By using (Formula 1), the distance z can be changed linearly with respect to the change of S2. In this embodiment, the timing of the first and second exposure periods is set to obtain the components of the front end and the rear end of the reflected light pulse Ir, respectively. Thus, the change of S1 and S2 with respect to the distance change of the object 50 is maximized, and the distance resolution can be improved. According to (Formula 1), the distance measurement error σz of this method can be calculated using the detection errors σs1 and σs2 of S1 and S2 by the following calculation of (Formula 2).

[0133] [Formula 2]

[0134]

[0135] According to (Formula 2), by reducing T0-(t2-t1), the distance measurement error σz can be reduced. In this embodiment, by setting the first exposure period and the second exposure period to obtain the components of the front end and the rear end of the reflected light pulse Ir, T0-(t2-t1) can be reduced, thereby reducing the distance measurement error.

[0136] Here, "obtaining the component of the front end portion of the reflected light pulse Ir" means detecting at least a part of the rising period of the reflected light pulse Ir. Further, "obtaining the component of the rear end portion of the reflected light pulse Ir" means detecting at least a part of the falling period of the reflected light pulse Ir. In the present embodiment, the time point at which the intensity of the reflected light pulse Ir reaching the photodetector starts to increase is referred to as "the first time point", the time point at which the intensity of the reflected light pulse Ir ends to increase is referred to as "the second time point", the time point at which the intensity of the reflected light pulse Ir starts to decrease is referred to as "the third time point", and the time point at which the intensity of the reflected light pulse Ir ends to decrease is referred to as "the fourth time point". The first exposure period includes the first time point, and the second exposure period includes the fourth time point. Thus, in the present embodiment, the first exposure period includes the rising start time point of the reflected light pulse Ir, and the second exposure period includes the falling end time point of the reflected light pulse Ir. By controlling each exposure period in this way, the ranging accuracy can be improved.

[0137] The ranging operation shown in (Equation 2) is particularly effective when the detection errors σs1 and σs2 are caused by the illuminance variation of the light source 110. This is because, compared with an error proportional to the square root of S1 and S2 such as shot noise, the dependence on the signals S1 and S2 on the right side of (Equation 2) decreases, and the dependence on T0-(t2-t1) increases. This is due to the fact that the error caused by the illuminance variation is proportional to S1 and S2.

[0138] Therefore, in the present embodiment, the control circuit 132 controls the image sensor 120 to satisfy 0 < t2 - t1 < T0 in order to obtain the components of the front end portion and the rear end portion of the reflected light pulse Ir. The control circuit 132 may also control the image sensor 120 to satisfy T0 / 3 < t2 - t1 < T0 or T0 / 2 < t2 - t1 < T0. By such control, the ranging accuracy can be further improved. Thus, from the viewpoint of ranging resolution, it is effective that both S1 and S2 are the components of the end portions of the reflected light pulse Ir. Therefore, the first exposure period and the second exposure period can be set respectively so as not to include the midpoint or the peak value of the reflected light pulse Ir. Here, the midpoint refers to the point between the rising start point and the falling end point of the reflected light pulse Ir. By accumulating the charges of the components of the two end portions of the reflected light pulse Ir in a manner that does not include the midpoint or the peak value, a higher-precision ranging can be performed.

[0139] By reducing the pulse width T0, there is also an effect of improving the ranging accuracy. However, if the emission pulse is made shorter, it is difficult to stabilize the pulse shape, so reducing T0 is limited. The pulse width T0 can be set to 3 ns or more, and in one example, it can be set to 5 ns or more. Compared with reducing the pulse width T0, when the time difference (t2 - t1) between the first and second exposure periods is increased, the followability of the electronic shutter is higher, and the image sensor 120 can be driven more stably.

[0140] The signal S1 representing the component of the front end portion of the reflected light pulse Ir has a higher detected signal value due to both the increase in illuminance and the increase in light amount due to phase shift when the distance of the object 50 approaches, so it is likely to saturate. Therefore, the first and second exposure periods can also be set such that S1 < S2. In this case, since the components of the rear end portion of the reflected light pulse Ir have opposite characteristics due to the illuminance change and the phase change, they cancel each other out, and it is less likely that S2 changes with respect to the distance change. Therefore, S2 has a high possibility of remaining within the dynamic range even when the distance changes compared to S1, so the risk of saturation is small.

[0141] Figure 3B FIG. is an example showing another driving method of the ranging device 100. In this example, the ranging device 100 drives the shutter once for each reflected light pulse Ir. Therefore, a circuit design can be made to improve the sharpness of each of the front end portion and the rear end portion of the reflected light pulse Ir, and the ranging accuracy can be further improved. The ranging device 100 may also include two light sources 110 corresponding to the first and second exposure periods, respectively.

[0142] Figure 3C FIG. is another example showing the emission pulse, the reflected light pulse, and the shutter timing. In this example, the first and second exposure periods are each shorter than the pulse width of the reflected light pulse Ir. In this way, by shortening the first and second exposure periods respectively, the dark current noise and the fixed pattern noise can be reduced.

[0143] Figure 3D: is a diagram showing another example of a light emitting pulse, a reflected light pulse and shutter timing. The light source 110 in this example emits a light emitting pulse Ie with a peak value. The first exposure period starts before the start of the rising period of the reflected light pulse Ir and ends in the middle of the rising period. The second exposure period starts in the middle of the falling period of the reflected light pulse Ir and ends after the falling period ends. That is, the first exposure period ends at the inclined portion on the front end side of the reflected light pulse Ir, and the second exposure period starts at the inclined portion on the rear end side of the reflected light pulse Ir. In other words, the first exposure period ends before the second time point, and the second exposure period starts after the third time point. Through such control, the change in the detected light amount becomes sensitive to the phase change of the reflected light pulse Ir accompanying the change in the distance to the object 50, and the distance detection sensitivity becomes higher. The steeper the inclination, the greater the rate of change of the light amount. Such control is particularly effective when the dominant cause of the noise component is not phase change but shot noise or light emitting illumination change. This is because shot noise and changes in luminous intensity are not correlated with the deviation in shutter timing. Figure 3D In the example shown, the control circuit 132 also adjusts the first and second shutter opening periods so that the detected signal includes the components at both ends of the reflected light pulse Ir but does not include the components at the peak portion. This can improve the distance resolution. Figure 3A Compared with the example of the reflected light pulse Ir, the intensity change at the front end and the rear end does not need to be sharp. Therefore, a light source such as an LED whose pulse rise and fall are not sharp can be used as the light source 110. For example, by using an LED as the light source 110, advantages such as increased luminous intensity, improved safety, and reduced costs are generated. In addition, Figure 3D In the example of FIG. 1 , the light emitting pulse Ie and the reflected light pulse Ir have waveforms close to a triangular wave, but their waveforms may also be other shapes. For example, the light emitting pulse Ie and the reflected light pulse Ir may also have waveforms close to a trapezoidal wave.

[0144] exist FIG. 3A to FIG. 3D In each example, the first exposure period includes the time point immediately after the reflected light pulse Ir rises, and the second exposure period includes the time point at which the reflected light pulse Ir ends falling. By controlling each exposure period in this way, the light amount changes of S1 and S2 with respect to the distance change of the object 50 can be kept large. For example, suppose that the distance from the distance measuring device 100 to the object 50 is shortened relative to the reference distance. In this case, since the reflected light pulse Ir reaches the image sensor 120 earlier, FIG. 3A to FIG. 3DThe waveform of the reflected light pulse Ir shown is shifted to the left. At this time, if the first exposure period does not include the rising start time point of the reflected light pulse Ir from the reference distance and is set after the rise start, the light amount change of S1 does not occur or the light amount change becomes smaller. As a result, the distance resolution decreases. The same can be said about the decline. Therefore, in the present embodiment, the first exposure period includes the starting time point of the rising period of the reflected light pulse Ir from the reference distance, and the first exposure period includes the ending time point of the falling period of the reflected light pulse Ir. As a result, the change of the detection signal value relative to the distance change of the object 50 becomes sensitive, and the distance resolution is improved.

[0145] [3. Operation of the electronic circuit 130]

[0146] Figure 4A is a flowchart showing an example of the operation of the electronic circuit 130 of this embodiment. Figure 3AThe action shown corresponds to the above. In this example, each pixel of the image sensor 120 has a first charge accumulation unit and a second charge accumulation unit. The control circuit 132 in the electronic circuit 130 first causes the light source 110 to emit a light pulse (step S101). The time length of the light pulse is T0. After the emission of the light pulse, the control circuit 132 causes the electronic shutter to start accumulating charges in the first charge accumulation unit of each pixel at a first timing (step S102). Assuming that the object 50 exists at a predetermined reference distance from the distance measuring device 100, the first timing is earlier than the rising start time point of the reflected light pulse Ir returned from the object 50. Next, the control circuit 132 causes the electronic shutter to stop accumulating charges in the first charge accumulation unit of each pixel at a second timing (step S103). The second timing is later than the rising start time point of the above-mentioned reflected light pulse Ir. Through the operations of steps S101 to S103, the components of the reflected light pulses that have reached each pixel of the image sensor 120 during the period from the first timing to the second timing are detected. Next, the control circuit 132 causes the electronic shutter to start accumulating charges in the second charge accumulation section of each pixel at the third timing (step S104). The third timing is earlier than the falling start time point of the above-mentioned reflected light pulse Ir. Next, the control circuit 132 causes the electronic shutter to stop accumulating charges in the second charge accumulation section of each pixel at the fourth timing (step S105). The fourth timing is later than the falling end time point of the above-mentioned reflected light pulse Ir. Through the operations of steps S103 to S105, the components of the reflected light pulses that have reached each pixel of the image sensor 120 during the period from the third timing to the fourth timing are detected. Next, the control circuit 132 determines whether the number of times the above-mentioned signal accumulation to each charge accumulation section has been performed has reached a prescribed number of times (step S106). If the determination result is "No", steps S101 to S105 are repeated until the determination result is "Yes". This repetition is performed when the light amount is insufficient in one signal accumulation. The number of repetitions is set to an appropriate number according to the required SN ratio.

[0147] If it is determined as "yes" in step S106, the control circuit 132 causes the signal processing circuit 134 to generate a first intensity map based on the signal charge accumulated in the first charge accumulation section of each pixel (step S107). Furthermore, the control circuit 132 causes the signal processing circuit 134 to generate a second intensity map based on the signal charge accumulated in the second charge accumulation section of each pixel (step S108). The first intensity map and the second intensity map are image data having different pixel values ​​according to the flight time of the reflected light pulse. Next, the signal processing circuit 134 calculates the distance of each pixel and generates a distance map (step S109) by performing the calculation processing shown in (Formula 1) above based on the data of the first and second intensity maps. The distance map is image data in which each pixel has a distance value. The signal processing circuit 134 records the generated distance map in a recording medium such as a memory.

[0148] In addition, in step S109, the signal processing circuit 134 may determine the distance of each pixel by referring to a table stored in advance in a recording medium such as the memory 136 instead of calculating the distance of each pixel by the operation shown in (Formula 1). Such a table specifies, for example, the relationship between the range of values ​​of S2 / (S1+S2) and the distance corresponding to the range. The table may also specify the relationship between the range of values ​​of S1 / (S1+S2) and the distance corresponding to the range.

[0149] Through the operations of steps S101 to S109, a distance map for one frame is generated. The electronic circuit 130 may also generate distance maps for multiple frames by repeatedly performing the operations of steps S101 to S109. Through the above operations, a distance map, that is, a distance image, of the object 50 can be generated with high accuracy.

[0150] Figure 4B is a flowchart showing another example of the operation of the electronic circuit 130 of this embodiment. Figure 3B In this example, the charge accumulation in the first charge accumulation section and the charge accumulation in the second charge accumulation section are performed based on different light emission pulses. Figure 4A The actions of steps S201 to S203 are the same as Figure 4A The operations of S101 to S103 in the above are respectively the same. After step S203, in step S204, the control circuit 132 causes the light source 110 to emit another light pulse. And, in steps S205 to S207, the same operations as those in step S206 are performed. Figure 4A In step S207, if it is determined that the signal accumulation to the first charge accumulation unit and the second charge accumulation unit for the prescribed number of times is completed, the control circuit 132 causes the signal processing circuit 134 to execute the operations of steps S208 to S210. The operations of steps S208 to S210 are the same as those of steps S209 to S211. Figure 4A The actions of steps S107 to S109 are respectively the same.

[0151] Through the operations from step S201 to S210, a distance map for one frame is generated. The electronic circuit 130 may also generate distance maps for multiple frames by repeatedly performing the operations from step S201 to S210. Through the above operations, a distance map of the object 50 can be generated with high accuracy.

[0152] In addition, multiple times of light emission and charge accumulation are not necessary, but are performed as needed. In addition, the accumulation of two types of signal charges used to generate the first and second intensity maps can be allocated to two different frames instead of being allocated to two charge accumulation units. For example, the shutter timing can be different for each frame, and the first intensity map is generated in the odd frame and the second intensity map is generated in the even frame. In addition, the shutter timing can be different for each pixel. In this way, the first intensity map and the second intensity map can be obtained by allocating them to each pixel. For example, the first intensity map can be obtained in a checkerboard pattern on the pixel, and the second intensity map can be obtained in a checkerboard pattern on the remaining pixels. The first intensity map and the second intensity map can also be obtained by allocating them to each row of pixels. For example, the first intensity map can be obtained in the odd rows of pixels, and the second intensity map can be obtained in the even rows of pixels. In each of the first and second intensity maps, after the pixels that are spaced out are interpolated in space, a distance map can be generated by calculation processing based on the first and second intensity maps.

[0153] Figure 4C1 is a flowchart showing an example of a process of generating a different intensity map for each frame. In this example, the control circuit 132 first causes the light source 110 to emit a light pulse for a predetermined time (step S301). The control circuit 132 causes the electronic shutter to start accumulating charges in the charge accumulation section at the first timing (step S302). Next, the control circuit 132 causes the electronic shutter to stop accumulating charges in the charge accumulation section at the second timing (step S303). Through these actions, a signal charge corresponding to a component of the front end portion of the reflected light pulse is accumulated in the charge accumulation section. Next, the control circuit 132 determines whether the number of times the above-mentioned signal accumulation has been performed has reached a predetermined number of times (step S304). In the case where the determination is "no", steps S301 to S303 are repeated until the determination is "yes". If the determination is "yes" in step S304, the control circuit 132 causes the signal processing circuit 134 to generate a first intensity map based on the signal charge accumulated in the charge accumulation section, and resets the signal charge of the charge accumulation section (step S305). Next, the control circuit 132 causes the light source 110 to emit a light pulse at a predetermined time (step S306). The control circuit 132 causes the electronic shutter to start accumulating charges in the charge accumulation section at the third timing (step S307). Next, the control circuit 132 causes the electronic shutter to stop accumulating charges in the charge accumulation section at the fourth timing (step S308). Through these actions, the signal charge corresponding to the component of the rear end portion of the reflected light pulse is accumulated in the charge accumulation section. Next, the control circuit 132 determines whether the number of times the above-mentioned signal accumulation has been performed has reached a predetermined number of times (step S309). In the case where the determination is "no", steps S306 to S308 are repeated until the determination is "yes". If the determination is "yes" in step S309, the control circuit 132 causes the signal processing circuit 134 to generate a second intensity map based on the signal charge accumulated in the charge accumulation section, and resets the signal charge of the charge accumulation section (step S310). Next, the signal processing circuit 134 generates a distance map using the above-described method based on the data of the first and second intensity maps (step S311 ).

[0154] Through the operations of steps S301 to S311 , a distance map of the object 50 can be generated with high accuracy. Figure 4C This shows the action for two frames, but you can repeat it to get a distance map for more frames. Figure 4C The action shown. Figure 4C In the example of , the distance map is generated once every two frames, but the distance map may be generated for each frame. For example, a step similar to step S311 may be added between step S305 and step S306. In this case, in the added step, the distance map can be generated based on the first intensity map obtained in step S305 and the second intensity map obtained in the previous frame.

[0155] (Implementation Method 2)

[0156] Next, the distance measuring device 100 according to the second embodiment will be described.

[0157] The distance measuring device 100 of this embodiment is different from Embodiment 1 in that it has a driving mode for setting the first and second exposure periods so as to detect the end of the reflected light pulse Ir. The following description will focus on the differences from Embodiment 1, and description of overlapping matters will be omitted.

[0158] In order to improve the distance measurement accuracy, the distance measuring device 100 drives the image sensor 120 so as to detect both ends of the reflected light pulse Ir. However, since the timing of the reflected light pulse Ir returning varies depending on the distance between the object 50 and the distance measuring device 100, the ends of the reflected light pulse Ir may not be detected at the preset shutter timing. In particular, as T0-(t2-t1) is made smaller to improve the distance measurement accuracy, the distance measurement range, i.e., the range of distance that can be measured, becomes smaller, and one of the first and second exposure periods is likely to deviate to the outside of the reflected light pulse Ir.

[0159] Therefore, in this embodiment, if Figure 5 As shown in FIG. 1 , before the high-precision distance measurement mode that detects only the components at both ends of the reflected light pulse Ir, an operation based on a distance measurement mode with a large distance measurement range, i.e., a long-range mode, is performed. The electronic circuit 130 calculates an approximate distance in the long-range mode, and the result is reflected in the adjustment of the shutter phase in the high-precision distance measurement mode. In the long-range mode, for example, Figure 5 As shown, the image sensor 120 is driven so that one of the first and second exposure periods includes the midpoint of the reflected light pulse Ir. Here, the midpoint of the reflected light pulse Ir refers to the time point at the center of the rising start time point and the falling end time point of the reflected light pulse Ir. In the wide range mode, the end time point of the first exposure period coincides with the start time point of the second exposure period. In the wide range mode, the ranging range depends on the pulse width T0. For example, when the pulse width T0 is 5ns, a ranging range of approximately 75cm can be ensured. In order to further expand the ranging range, the pulse width T0 can also be lengthened only in the wide range mode. In the wide range mode, the distance z is calculated by (Formula 3).

[0160] [Formula 3]

[0161]

[0162] Figure 61 is a flowchart schematically showing the operation of adjusting the shutter timing to within the ranging range. First, the control circuit 132 determines whether the end of the reflected light pulse is detected during both the first and second exposure periods (step S401). Whether the end of the reflected light pulse is detected during each exposure period can be determined based on the intensity of the signals S1 and S2. If the determination in step S401 is "yes", the process proceeds to step S404. If the determination in step S401 is "no", the process proceeds to step S402. In the case where one or both of the signals S1 and S2 do not contain signal components other than components caused by ambient light, the end is not detected and the determination is "no". In addition, in the case where one or both of the signals S1 and S2 have a value much larger than the assumed value, it is considered that the center portion is detected instead of the end of the reflected light pulse, so the determination is "no". In the case where the determination in step S401 is "no", the control circuit 132 shifts to the wide range mode. In the wide range mode, the control circuit 132 drives the shutter at a timing at a time point when one of the two exposure periods includes the center of the reflected light pulse. The control circuit 132 calculates the distance to the object 50 based on (Formula 3), and based on the distance, calculates the deviation of the phase difference between each exposure period and the reflected light pulse relative to an appropriate value (step S402). Furthermore, according to the calculated phase deviation, the relative timing of the light pulse and each exposure period is fine-tuned, and the components at both ends of the reflected light pulse Ir are detected during the first and second exposure periods (S403). The appropriate phase difference varies depending on the distance to the object 50. Therefore, in order to fine-tune the relative timing, for example, a table of the relationship between a specified distance and an appropriate phase difference can be maintained in advance. The control circuit 132 can refer to the table to adjust the light emission timing of the light emission pulse relative to the reference clock or the timing of each exposure period so as to achieve an appropriate phase difference relative to the detected distance. After step S403 or if it is determined to be YES in step S401 , the signal processing circuit 134 calculates the distance based on (Formula 1) from the detection signal values ​​obtained in the first and second exposure periods (step S404 ).

[0163] Through the above operation, when the end of the reflected light pulse Ir cannot be detected at the preset shutter timing, the shutter timing can be adjusted so that the end of the reflected light pulse Ir can be detected. Thus, even when the distance between the object 50 and the distance measuring device 100 changes with the passage of time, distance information can be obtained stably and accurately.

[0164] (Implementation method 3)

[0165] Embodiment 3 relates to an application using the distance measuring device 100 .

[0166] Figure 7FIG. 1 is a diagram schematically showing an example in which the distance measuring device 100 according to Embodiment 1 or 2 is mounted in the interior of a car. Figure 7 In the example shown, the distance measuring device 100 is set in the cabin of the car. Through the distance measuring device 100, the 3D texture of the driver, which is the object 50 of distance measurement, can be obtained. By comparing the data of the obtained 3D texture with the database, it is possible to identify whether the driver is the owner of the car. The distance measuring device 100 can also replace the car key to identify the driver and start the car. The distance measuring device 100 can also identify the passenger during automatic driving, and feed back the identification result to the control system of automatic driving, so that driving customized according to the passenger can be performed. For example, the following customization can be performed: if the passenger is an elderly person or a passenger with back pain, the brakes are softer; if the passenger likes driving, the driving is smooth, etc. Through such feedback, automatic driving suitable for personal preferences can be performed.

[0167] Fig. 8A and Figure 8B 1 is a diagram schematically showing an example of applying the distance measuring device 100 of embodiment 1 or 2 to a mobile terminal. The mobile terminal is, for example, a smart phone, a tablet computer, or a personal computer. As the image sensor 120 in the distance measuring device 100, an image sensor built into the mobile terminal can also be used. An individual can be authenticated based on the 3D data of the user's face obtained from the distance measuring device 100. Fig. 8A An example of a screen showing a mobile terminal being authenticated. Figure 8B An example of a screen of a mobile terminal when authentication is successful. Such an authentication mechanism is effective for login authentication to an online account, in addition to login authentication to a mobile terminal. In the present embodiment, by using 3D data of the user's face, the amount of information is increased compared to the previous two-dimensional image, so the recognition accuracy is improved. As a result, it is possible to exclude counterfeiting such as reading a user's photo in front of a camera, and achieve highly secure authentication. In addition, in the present embodiment, high-precision ranging is achieved by detecting the components at both ends of the reflected light pulse Ir, and fine 3D face data can be formed, so the authentication accuracy can be further improved compared to the previous 3D authentication. Therefore, the technology of the present embodiment is suitable for applications that require very high authentication accuracy, such as electronic settlement using a mobile terminal. The technology disclosed in the present invention can also be used as one of the multi-level authentications for high-value settlements, and can become a key technology for a cashless society.

[0168] The distance measuring device 100 may be used not only in automobiles and mobile terminals but also in, for example, identification of account holders of ATMs installed in banks and convenience stores, unlocking of residential entrances, and entry and exit management of offices.

[0169] The distance measuring device 100 can not only measure the distance, but also obtain information about the inside of the skin of the object 50. An example of such a distance measuring device 100 will be described below. In the following description, the information about the inside of the skin of the object 50 is referred to as "biological information."

[0170] Fig. 9 1 is a diagram for explaining the operation of the distance measuring device 100 for acquiring biological information of the object 50 in addition to the distance information. In this example, in addition to the first exposure period and the second exposure period for distance measurement, a third exposure period for acquiring biological information is provided. Each pixel of the image sensor 120 has three or more charge accumulation units. During the first exposure period, a signal S1 representing a component of the front end portion of the reflected light pulse Ir is obtained. During the second exposure period, a signal S2 representing a component of the rear end portion of the reflected light pulse Ir is obtained. The third exposure period starts at a timing later than the second exposure period with respect to the start time point of the light emission pulse. Thus, during the third exposure period, a signal S3 representing a component of a falling period is obtained, and the falling period is a period from the start to the end of the decrease in the intensity of the reflected light pulse Ir. By providing the third exposure period, for example, a component of light scattered inside the skin and having a relatively long optical path length can be detected. Thus, for example, the blood vessel distribution centered on the vein can be clearly detected.

[0171] Fig.10 Yes means Fig. 9 1 is a flowchart of the operation of the electronic circuit 130 in the example shown in FIG. In this flowchart, the operation of steps S501 to S506 is the same as that of steps S502 to S506. Figure 4BThe operations of steps S201 to S206 shown in the figure are the same. In the present embodiment, after step S506, the control circuit 132 causes the light source 110 to emit a light pulse (step S507). Next, the control circuit 132 causes the image sensor 120 to start accumulating charges to the third charge accumulation unit at the fifth timing (step S508). Then, the control circuit 132 causes the image sensor 120 to stop accumulating charges to the third charge accumulation unit at the sixth timing (step S509). The operations of steps S501 to S509 are repeated until it is determined in step S510 that the signal accumulation to each charge accumulation unit for a specified number of times is completed. If it is determined in step S510 that the signal accumulation for a specified number of times is completed, the control circuit 132 causes the signal processing circuit 134 to generate the first to third intensity maps based on the signal charges accumulated in the first to third charge accumulation units, respectively (steps S511 to S513). Next, similar to the above-mentioned embodiment, the signal processing circuit 134 generates a distance map based on the first and second intensity maps (step S514). Furthermore, the signal processing circuit 134 generates a biological information map based on the third intensity map (step S515). The biological information map is image data generated by processing the signal S3 based on each pixel. The biological information map may be, for example, image data indicating blood vessel distribution.

[0172] By combining the blood vessel distribution obtained by the above-mentioned processing with the 3D texture information based on the distance map, it is possible to perform highly accurate individual identification to the extent that, for example, identical twins can be distinguished. In addition, the blood vessel distribution can also be three-dimensionally displayed based on the distance information.

[0173] Fig.11A and Fig. 11B An example of an image showing blood vessel distribution in three dimensions is shown. Fig.11A As shown in FIG. 1 , even when the face is photographed from a direction other than the front, it is possible to use the projection transformation using the three-dimensional information to obtain the face image. Fig. 11B The transformation shown is an image of the blood vessel distribution observed from the front. By such a transformation, the stability of the comparison between the blood vessel distribution obtained by shooting and the standard database pre-stored in the recording medium can be improved. As a result, the rejection rate of the person or the acceptance rate of others can be reduced. In addition, if the user accidentally shoots at an angle different from the database, it can be automatically corrected and recognized on the terminal side, which can improve the recognition speed.

[0174] exist Fig. 9In the example of , three exposure periods are provided, but the blood vessel information inside the skin can be obtained during the second exposure period instead of the third exposure period. By using near-infrared light of about 650nm to 1000nm that easily penetrates subcutaneously, a lot of blood vessel information can be obtained regardless of shutter timing. By also obtaining the information of the internal blood vessels during the second exposure period, the types of shutters can be reduced to two, and the frame rate can be increased. The database of blood vessel distribution pre-stored in the recording medium can be either two-dimensional data or three-dimensional data. Alternatively, a database that adds the two-dimensional data of blood vessels to the data of the 3D texture of the face can be used.

[0175] (Example)

[0176] Fig.12 The actual measurement result of measuring the distance of the face of a human model of the same size as a human being using the distance measuring device 100 is shown. The distance between the distance measuring device 100 and the face of the human model is set to 30 cm. A lens with an F2.5 and a horizontal field angle of ±16.3° is set in the distance measuring device 100, and the number of pixels of the image sensor 120 is set to 320×240 pixels. The pulse width T0 of the light source 110, that is, the time from the rise to the end of the light pulse, is set to 5 ns. One tenth of each end of the reflected light pulse Ir from the reference distance, that is, the component from the rise to 0.5 ns of the reflected light pulse Ir and the component from 0.5 ns before the end of the fall, is detected. In this case, the distance measuring range is ±7.5 cm. In addition, this distance measuring range is usually not a problem for detecting a 3D image of the face, but since the size of the face varies from person to person, the shutter timing can also be set so that a margin of about 1 ns is left to detect the inside of the reflected light pulse Ir.

[0177] on the other hand, Fig.13 The following is a result of a comparative example in which the same human body model is actually measured using a conventional distance measurement method. In this comparative example, the distance z is calculated based on the following (Formula 4) instead of (Formula 1).

[0178] [Formula 4]

[0179]

[0180] Here, unlike the embodiment, the signal S1 is obtained by adjusting the shutter timing of exposure so as to detect the entire reflected light pulse Ir. The signal S2 is obtained by the same method as the embodiment.

[0181] The distance measurement method obtained by the previous Fig.13 In the face image shown in FIG. 1 , the concavity and convexity of each organ of the face cannot be distinguished. In contrast, in the distance measurement method of this embodiment, Fig.12As shown, the surface shape and depth distribution of the face can be obtained in detail, confirming the effectiveness of the technology disclosed in this disclosure. If the distance measurement deviation is calculated with 3σ, it is 0.12mm, which verifies that a high distance measurement accuracy can be achieved through the technology disclosed in this disclosure.

[0182] Industrial Applicability

[0183] The distance measuring device of the present disclosure can obtain the distance to an object or information on the concavo-convex shape of the surface with high accuracy in a non-contact manner. Therefore, the distance measuring device of the present disclosure is useful in a wide range of fields such as object recognition, line inspection, and authentication.

[0184] Description of symbols

[0185] 50 Objects

[0186] 100 Distance measuring device

[0187] 110 Light Source

[0188] 120 Image Sensor

[0189] 122 Photoelectric conversion unit

[0190] 124 Charge storage unit

[0191] 130 Electronic Circuits

[0192] 132 Control Circuit

[0193] 134 Signal Processing Circuit

Claims

1. A distance measuring device, wherein: have: light source; Light detector; as well as A circuit for controlling the light source and the light detector; The circuit causes the light source to emit light pulses toward the object; The circuit causes the photodetector to generate a first signal by detecting a reflected light pulse from the object generated by the light pulse during a first period, wherein the first period includes at least a portion of a rising period from a first time point when the intensity of the reflected light pulse starts to increase to a second time point when the increase ends, and includes the first time point. The circuit causes the photodetector to generate a second signal by detecting the reflected light pulse during a second period, the second period being a period after the first period ends, including at least a portion of a falling period from a third time point when the intensity of the reflected light pulse starts to decrease to a fourth time point when the decrease ends, and including the fourth time point. The circuit generates and outputs data indicating the distance from the photodetector to the object based on the first signal and the second signal; The first period ends earlier than the second time point; The second period starts later than the third time point.

2. The distance measuring device according to claim 1, wherein: The first period ends before the midpoint between the first time point and the fourth time point; The second period starts later than the midpoint.

3. The distance measuring device according to claim 1 or 2, wherein: The pulse width of the light pulse is greater than or equal to 3 ns and less than or equal to 30 ns.

4. The distance measuring device according to claim 1 or 2, wherein: When the time from the start of the emission of the light pulse to the end of the first period is t1, the time from the start of the emission of the light pulse to the start of the second period is t2, and the pulse width of the light pulse is T0, T0 / 3<(t2-t1) <T0。 5. The distance measuring device according to claim 1 or 2, wherein: The timing at which the first period starts is different from the timing at which the emission of the light pulse starts; or The timing at which the second period starts is different from the timing at which the emission of the light pulse ends.

6. A distance measuring device, wherein: have: light source; a light detector; and A circuit for controlling the light source and the light detector; The circuit causes the light source to emit a first light pulse and a second light pulse toward an object; The circuit causes the photodetector to generate a first signal by detecting a first reflected light pulse from the object generated by the first light pulse during a first period, wherein the first period includes at least a portion of a rising period from a first time point when the intensity of the first reflected light pulse starts to increase to a second time point when the increase ends, and includes the first time point. The circuit causes the photodetector to generate a second signal by detecting a second reflected light pulse from the object generated by the second light pulse during a second period, wherein the second period includes at least a portion of the first falling period from a third time point when the intensity of the second reflected light pulse starts to decrease to a fourth time point when the intensity decrease ends, and includes the fourth time point. The circuit generates and outputs data indicating the distance from the photodetector to the object based on the first signal and the second signal; The first period ends earlier than the second time point; The second period starts later than the third time point.

7. The distance measuring device according to claim 6, wherein: The first period ends before a midpoint between the first time point and a time point at which the intensity of the first reflected light pulse ends decreasing; The second period starts later than a midpoint between a time point at which the intensity of the second reflected light pulse starts to increase and the fourth time point.

8. The distance measuring device according to claim 6 or 7, wherein: The pulse width of each of the first light pulse and the second light pulse is greater than or equal to 3 ns and less than or equal to 30 ns.

9. The distance measuring device according to claim 6 or 7, wherein: When the time from the start of emission of the first light pulse to the end of the first period is t1, the time from the start of emission of the second light pulse to the start of the second period is t2, and the pulse width of each of the first light pulse and the second light pulse is T0, T0 / 3<(t2-t1) <T0。 10. The distance measuring device according to claim 6 or 7, wherein: The timing at which the first period starts is different from the timing at which the emission of the first light pulse starts; or The timing at which the second period starts is different from the timing at which the emission of the second light pulse ends.

11. The distance measuring device according to claim 6 or 7, wherein: The above-mentioned object is a living organism; The light source further emits a third light pulse toward the object; The photodetector further detects a third reflected light pulse resulting from the third light pulse returning from the object; The circuit causes the photodetector to detect the third reflected light pulse during a third period and output a third signal, wherein the third period includes at least a portion of the second falling period from a fifth time point when the intensity of the third reflected light pulse starts to decrease to a sixth time point when the decrease ends, and includes the sixth time point. The length from the start of the third period to the sixth time point is shorter than the length from the start of the second period to the fourth time point; The circuit generates and outputs biological information of the object based on the third signal.

12. A signal processing device, wherein: have: More than 1 processor; and A memory storing a computer program executed by the one or more processors; The one or more processors execute the computer program. causing the light source to emit a light pulse toward the object; causing the photodetector to generate a first signal by detecting a reflected light pulse from the object generated by the light pulse during a first period, wherein the first period includes at least a portion of a rising period from a first time point when the intensity of the reflected light pulse starts to increase to a second time point when the increase ends, and includes the first time point; causing the photodetector to generate a second signal by detecting the reflected light pulse during a second period, wherein the second period is a period after the first period ends, includes at least a portion of a falling period from a third time point when the intensity of the reflected light pulse starts to decrease to a fourth time point when the decrease ends, and includes the fourth time point; generating and outputting data indicating a distance from the photodetector to the object based on the first signal and the second signal; The first period ends earlier than the second time point; The second period starts later than the third time point.

13. A signal processing device, wherein: have: More than 1 processor; and A memory storing a computer program executed by the one or more processors; The one or more processors execute the computer program. causing the light source to emit a first light pulse and a second light pulse toward the object; causing the photodetector to generate a first signal by detecting a first reflected light pulse from the object generated by the first light pulse during a first period, wherein the first period includes at least a portion of a rising period from a first time point when the intensity of the first reflected light pulse starts to increase to a second time point when the increase ends, and includes the first time point; causing the photodetector to generate a second signal by detecting a second reflected light pulse from the object generated by the second light pulse during a second period, wherein the second period includes at least a portion of a falling period from a third time point when the intensity of the second reflected light pulse starts to decrease to a fourth time point when the intensity decrease ends, and includes the fourth time point; generating and outputting data indicating a distance from the photodetector to the object based on the first signal and the second signal; The first period ends earlier than the second time point; The second period starts later than the third time point.

14. A computer program product comprising a computer program, wherein: The computer program causes the computer to perform actions, including: causing the light source to emit a light pulse toward the object; causing the photodetector to generate a first signal by detecting a reflected light pulse from the object generated by the light pulse during a first period, wherein the first period includes at least a portion of a rising period from a first time point when the intensity of the reflected light pulse starts to increase to a second time point when the increase ends, and includes the first time point; causing the photodetector to generate a second signal by detecting the reflected light pulse during a second period, wherein the second period is a period after the first period ends, includes at least a portion of a falling period from a third time point when the intensity of the reflected light pulse starts to decrease to a fourth time point when the decrease ends, and includes the fourth time point; and generating and outputting data indicating a distance from the photodetector to the object based on the first signal and the second signal; The first period ends earlier than the second time point; The second period starts later than the third time point.

15. A computer program product comprising a computer program, wherein: The computer program causes the computer to perform actions, including: causing the light source to emit a first light pulse and a second light pulse toward the object; causing the photodetector to generate a first signal by detecting a first reflected light pulse from the object generated by the first light pulse during a first period, wherein the first period includes at least a portion of a rising period from a first time point when the intensity of the first reflected light pulse starts to increase to a second time point when the increase ends, and includes the first time point; causing the photodetector to generate a second signal by detecting a second reflected light pulse from the object generated by the second light pulse during a second period, wherein the second period includes at least a portion of a falling period from a third time point when the intensity of the second reflected light pulse starts to decrease to a fourth time point when the intensity decrease ends, and includes the fourth time point; generating and outputting data indicating a distance from the photodetector to the object based on the first signal and the second signal; The first period ends earlier than the second time point; The second period starts later than the third time point.

Images