Distance measuring device and distance measuring method

Abstract

A distance measuring device according to an embodiment comprises: a determining unit that determines a first minimum value from a first integrated histogram; and a processing unit that integrates the first integrated histogram and a first histogram and subtracts the first minimum value to generate a second integrated histogram.

Description

Distance measuring device and distance measuring method

[0001] The present disclosure relates to a distance measuring device and a distance measuring method.

[0002] In recent years, distance measuring devices that measure distance using the ToF (Time-of-Flight) method have been attracting attention. Examples of distance measuring devices include ToF sensors that measure the distance to an object using multiple light-receiving elements arranged on a plane. When using SPADs (Single Photon Avalanche Diodes) as light-receiving elements, ToF sensors typically set up multiple observation periods and integrate the histograms obtained during each observation. Peak signals caused by reflected light appear on the integrated histogram, and subsequent signal processing accurately estimates their temporal positions to determine the distance to the object.

[0003] In addition to light reflected from the object, photons incident on the ToF sensor also include those resulting from ambient light that is uncorrelated with time. The ambient light component is primarily thermal radiation from the sun. When sunlight is bright and the object has high reflectance, strong ambient light enters the sensor, increasing the overall count value (count number) of the resulting histogram. In extreme cases, the count value of the ambient light component may exceed the data range set in the histogram (potential for saturation). One method for avoiding this saturation is to subtract a specific value from the histogram each time the histogram is integrated (see, for example, Patent Documents 1 and 2).

[0004] JP 2018-169384 A International Publication No. 2021 / 145134

[0005] However, it is difficult to appropriately set the subtraction value, which is the specific value mentioned above. For example, if the average or minimum value of the histogram before integration is set as the subtraction value, false detection or failure to perform subtraction may occur, resulting in a decrease in distance measurement accuracy.

[0006] Therefore, the present disclosure provides a distance measuring device and a distance measuring method that can suppress a decrease in distance measurement accuracy.

[0007] The distance measuring device according to the embodiment includes a determination unit that determines a first minimum value from a first integrated histogram, and a processing unit that performs an integration of the first integrated histogram and the first histogram and subtracts the first minimum value to generate a second integrated histogram.

[0008] A distance measurement method according to an embodiment includes a computer determining a first minimum value from a first integrated histogram, integrating the first integrated histogram with the first histogram and subtracting the first minimum value from the first integrated histogram to generate a second integrated histogram.

[0009] 1 is a diagram illustrating an example of the configuration of a ToF sensor according to an embodiment; FIG. 2 is a diagram illustrating an example of the configuration of an optical system of a ToF sensor according to an embodiment; FIG. 3 is a diagram illustrating an example of the configuration of a light receiving unit according to an embodiment; FIG. 4 is a circuit diagram illustrating an example of the configuration of a SPAD pixel according to an embodiment; FIG. 5 is a diagram illustrating an example of the configuration of a SPAD adder according to an embodiment; FIG. 6 is a diagram illustrating an example of the configuration of a calculation unit according to an embodiment; FIG. 7 is a diagram illustrating an example of the processing of histogram integration and subtraction according to an embodiment; FIG. 8 is a diagram illustrating an example of the processing of histogram integration and subtraction according to an embodiment; FIG. 9 is a graph illustrating the relationship between time bins and count values ​​according to an embodiment and a comparative example; FIG. 10 is a graph illustrating the relationship between the amount of incident light and the average value of ambient light components according to an embodiment and a comparative example; FIG. 11 is a diagram illustrating an example of the configuration of a calculation unit of a first modified example according to an embodiment; FIG. 12 is a diagram illustrating an example of the configuration of a calculation unit of a second modified example according to an embodiment;

[0010] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. The embodiments include examples and modifications. Note that the technology according to the present disclosure is not limited to the embodiments. Furthermore, in the embodiments, essentially identical components are designated by the same reference numerals, and redundant explanations will be omitted.

[0011] The present disclosure will be described in the following order: 1. Embodiment 1-1. Configuration example of ToF sensor 1-2. Configuration example of optical system 1-3. Configuration example of light receiving unit 1-4. Configuration example of SPAD pixel 1-5. Configuration example of SPAD adder 1-6. Configuration example of calculation unit 1-7. Processing example of histogram accumulation and subtraction 1-8. Differences between this embodiment and comparative example 1-9. Modifications 1-9-1. Modification 1 1-9-2. Modification 2 1-9-3. Modification 3 1-10. Actions and effects 2. Other embodiments 3. Application examples 4. Supplementary notes

[0012] <1. Embodiment> <1-1. Configuration Example of ToF Sensor> An example configuration of a ToF sensor 1 according to this embodiment will be described with reference to Fig. 1. Fig. 1 is a diagram showing an example configuration of a ToF sensor 1 according to this embodiment. The ToF sensor 1 is an example of a distance measuring device.

[0013] As shown in FIG. 1 , the ToF sensor 1 includes a control unit 11 , a light emitting unit 12 , a light receiving unit 13 , a calculation unit 14 , and an external I / F (interface) 15 .

[0014] The control unit 11 controls each unit of the ToF sensor 1. The control unit 11 is configured by an information processing device such as a CPU (Central Processing Unit), for example.

[0015] The external I / F 15 is, for example, a communication adapter that establishes communication with an external host 80 via a communication network. The communication network may be, for example, a wireless local area network (LAN) or a wired LAN, or may be a communication network conforming to any standard such as a controller area network (CAN), a local interconnect network (LIN), or FlexRay (registered trademark).

[0016] For example, when the ToF sensor 1 is mounted on a moving body such as an automobile, the host 80 may be an ECU (Engine Control Unit) mounted on the moving body. Furthermore, when the ToF sensor 1 is mounted on an autonomous moving body such as a domestic pet robot, a robot vacuum cleaner, an unmanned aerial vehicle, or a follow-up transport robot, the host 80 may be a control device that controls the autonomous moving body.

[0017] The light-emitting unit 12 includes, for example, one or more semiconductor laser diodes as a light source, and emits pulsed laser light (also referred to as irradiation light) L1 with a predetermined duration at a predetermined cycle (also referred to as light emission cycle). The light-emitting unit 12 emits laser light L1 with a duration of 1 ns at a cycle of 1 MHz, for example. If an object (measurement target) 90 is present within the distance measurement range, the laser light L1 emitted from the light-emitting unit 12 is reflected by the object 90 and enters the light-receiving unit 13 as reflected light L2.

[0018] The light receiving unit 13 includes, for example, a plurality of SPAD (Single Photon Avalanche Diode) pixels arranged in a two-dimensional lattice pattern, and outputs information (e.g., equivalent to the number of detection signals described below) regarding the number of SPAD pixels that detect incident photons (also referred to as the detection number) after light emission by the light emitting unit 12. For example, the light receiving unit 13 detects incident photons at a predetermined sampling period for each light emission by the light emitting unit 12, and outputs the detection number.

[0019] The calculation unit 14 counts the detection counts output from the light receiving unit 13 for each of multiple SPAD pixels (or multiple macro pixels), and creates a histogram with the horizontal axis representing the time of flight and the vertical axis representing the cumulative pixel value based on the pixel values ​​obtained by the counting. Note that one macro pixel is composed of a predetermined number of SPAD pixels.

[0020] For example, the calculation unit 14 calculates pixel values ​​by tallying the number of detections at a predetermined sampling frequency for each light emission from the light-emitting unit 12, and repeats this process for multiple light emissions from the light-emitting unit 12. Then, the calculation unit 14 creates a histogram in which the horizontal axis (histogram bins) represents the sampling period corresponding to the time of flight, and the vertical axis represents the cumulative pixel value obtained by accumulating the pixel values ​​calculated at each sampling period.

[0021] The calculation unit 14 also performs a predetermined filtering process on the created histogram, and then identifies, for example, the time of flight when the cumulative pixel value peaks from the filtered histogram. The calculation unit 14 then calculates the distance L from the ToF sensor 1 or a device equipped with the ToF sensor 1 to an object 90 present within the ranging range based on the identified time of flight. Information about the distance L calculated by the calculation unit 14 may be output to the host 80 or the like via the external I / F 15, for example.

[0022] Here, the aforementioned calculation unit 14 and the like may be configured with both hardware and / or software, and the configuration is not particularly limited. The calculation unit 14 may be realized by an integrated circuit such as an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or a CPLD (Complex Programmable Logic Device). Furthermore, the calculation unit 14 may be realized by a computer such as a CPU or an MPU (Micro Control Unit) executing a program stored in advance in a ROM (Read Only Memory) using a RAM (Random Access Memory) or the like as a work area.

[0023] <1-2. Configuration Example of Optical System> A configuration example of the optical system of the ToF sensor 1 according to this embodiment will be described with reference to Fig. 2. Fig. 2 is a diagram showing a configuration example of the optical system of the ToF sensor 1 according to this embodiment.

[0024] 2 illustrates a so-called scanning type optical system that horizontally scans the angle of view of the light receiving unit 13. Note that the ToF sensor 1 may be, for example, a so-called flash type ToF sensor in which the angle of view of the light receiving unit 13 is fixed.

[0025] 2, the ToF sensor 1 includes, as an optical system, a light source 31, a collimator lens 32, a half mirror 33, a galvanometer mirror 35, a light-receiving lens 36, and a SPAD array 37. The light source 31, the collimator lens 32, the half mirror 33, and the galvanometer mirror 35 are included in, for example, the light-emitting unit 12 in FIG. 1. The light-receiving lens 36 and the SPAD array 37 are included in, for example, the light-receiving unit 13 in FIG. 1.

[0026] In the configuration shown in FIG. 2 , laser light L1 emitted from a light source 31 is converted by a collimator lens 32 into a rectangular parallel beam whose cross-sectional intensity spectrum is elongated in the vertical direction, and then enters a half mirror 33. The half mirror 33 reflects a portion of the incident laser light L1. The laser light L1 reflected by the half mirror 33 enters a galvanometer mirror 35. The galvanometer mirror 35 is vibrated in the horizontal direction around a predetermined rotation axis as the vibration center by a driver 34, which operates, for example, based on control from the control unit 11. This causes the laser light L1 to be horizontally scanned such that the angle of view SR of the laser light L1 reflected by the galvanometer mirror 35 horizontally scans the distance measurement range AR back and forth. Note that the driver 34 may be a microelectromechanical system (MEMS) or a micromotor.

[0027] The laser light L1 reflected by the galvanometer mirror 35 is reflected by an object 90 present within the distance measurement range AR and enters the galvanometer mirror 35 as reflected light L2. A portion of the reflected light L2 that enters the galvanometer mirror 35 passes through the half mirror 33 and enters the light-receiving lens 36, which then forms an image on a used SPAD array 37a in the SPAD array 37. The used SPAD array 37a is, for example, a region that is used in the SPAD array 37, and may be a portion of the SPAD array 37 or the entire SPAD array 37.

[0028] <1-3. Configuration Example of Light Receiving Unit> A configuration example of the light receiving unit 13 according to this embodiment will be described with reference to Fig. 3. Fig. 3 is a diagram showing a configuration example of the light receiving unit 13 according to this embodiment.

[0029] As shown in FIG. 3, the light receiving unit 13 includes a SPAD array 37 , a timing control circuit 41 , a drive circuit 42 , and an output circuit 43 .

[0030] The SPAD array 37 includes a plurality of SPAD pixels 20 arranged in a two-dimensional lattice pattern. Pixel drive lines LD (vertical in FIG. 3 ) are connected to the plurality of SPAD pixels 20 for each column, and output signal lines LS (horizontal in FIG. 3 ) are connected to the plurality of SPAD pixels 20 for each row. One end of the pixel drive line LD is connected to an output terminal of the drive circuit 42 corresponding to each column, and one end of the output signal line LS is connected to an input terminal of the output circuit 43 corresponding to each row.

[0031] In this embodiment, the reflected light L2 is detected using part or all of the SPAD array 37. The used SPAD array 37a, which is the area used in the SPAD array 37, may be a rectangle that is long in the vertical direction, the same as the image of the reflected light L2 that is formed on the SPAD array 37 when the entire laser light L1 is reflected as the reflected light L2. However, the area is not limited to this, and may be variously modified, such as being larger or smaller than the image of the reflected light L2 that is formed on the SPAD array 37.

[0032] The timing control circuit 41 includes a timing generator that generates various timing signals, etc. The timing control circuit 41 controls the drive circuit 42 and the output circuit 43 based on the various timing signals generated by the timing generator.

[0033] The drive circuit 42 includes a shift register, an address decoder, etc., and drives each SPAD pixel 20 of the SPAD array 37 simultaneously or column by column. For example, the drive circuit 42 includes a circuit that applies a quench voltage V_QCH (described later) to each SPAD pixel 20 in a selected column in the SPAD array 37, and a circuit that applies a selection control voltage V_SEL (described later) to each SPAD pixel 20 in the selected column. The drive circuit 42 applies the selection control voltage V_SEL to a pixel drive line LD corresponding to a column to be read out, thereby selecting, column by column, the SPAD pixels 20 to be used for detecting incident photons.

[0034] A signal (called a detection signal) V_OUT output from each SPAD pixel 20 in a column selected and scanned by the drive circuit 42 is input to the output circuit 43 through each output signal line LS. The output circuit 43 outputs the detection signal V_OUT input from each SPAD pixel 20.

[0035] 1-4. Configuration Example of SPAD Pixel> A configuration example of the SPAD pixel 20 according to this embodiment will be described with reference to Fig. 4. Fig. 4 is a circuit diagram showing a configuration example of the SPAD pixel 20 according to this embodiment.

[0036] 4, the SPAD pixel 20 includes a photodiode 21 as a light receiving element and a readout circuit 22 that detects that a photon has been incident on the photodiode 21. When a photon is incident on the photodiode 21 while a reverse bias voltage V_SPAD equal to or greater than the breakdown voltage is applied between the anode and cathode of the photodiode 21, the photodiode 21 generates an avalanche current.

[0037] The photodiode 21 is a SPAD. A SPAD is an avalanche photodiode that operates in Geiger mode when a reverse bias voltage equal to or greater than the breakdown voltage is applied between its anode and cathode, and is capable of detecting the incidence of a single photon.

[0038] The read circuit 22 includes a quench resistor 23 , a selection transistor 24 , a digital converter 25 , an inverter 26 , and a buffer 27 .

[0039] The quench resistor 23 is configured, for example, by an N-type MOSFET (Metal Oxide Semiconductor Field Effect Transistor), i.e., an NMOS transistor. The drain of this NMOS transistor is connected to the anode of the photodiode 21, and the source is grounded via the selection transistor 24. A quench voltage V_QCH, which is set in advance, is applied to the gate of the NMOS transistor that configures the quench resistor 23 from the drive circuit 42 via the pixel drive line LD to cause the NMOS transistor to function as a quench resistor.

[0040] The selection transistor 24 is, for example, an NMOS transistor, and its drain is connected to the source of the NMOS transistor that constitutes the quench resistor 23, and its source is grounded. The selection transistor 24 is connected to the drive circuit 42, and when a selection control voltage V_SEL from the drive circuit 42 is applied to the gate of the selection transistor 24 via the pixel drive line LD, the selection transistor 24 changes from an OFF state to an ON state.

[0041] The digital converter 25 includes a resistor 25 a and an NMOS transistor 25 b. The drain of the NMOS transistor 25 b is connected to the power supply voltage VDD via the resistor 25 a, and the source of the NMOS transistor 25 b is grounded. The voltage at the connection point N1 between the anode of the photodiode 21 and the quench resistor 23 is applied to the gate of the NMOS transistor 25 b.

[0042] The inverter 26 includes P-type MOSFETs, i.e., a PMOS transistor 26a and an NMOS transistor 26b. The drain of the PMOS transistor 26a is connected to the power supply voltage VDD, and the source is connected to the drain of the NMOS transistor 26b. The drain of the NMOS transistor 26b is connected to the source of the PMOS transistor 26a, and the source is grounded. The voltage at the connection point N2 between the resistor 25a and the drain of the NMOS transistor 25b is applied to the gates of the PMOS transistor 26a and NMOS transistor 26b, respectively. The output of the inverter 26 is input to a buffer 27.

[0043] The buffer 27 is a circuit for impedance conversion, and when the output signal from the inverter 26 is input, the buffer 27 converts the impedance of the input output signal and outputs the result as a detection signal V_OUT.

[0044] 4 operates, for example, as follows. First, while the selection control voltage V_SEL is applied from the drive circuit 42 to the selection transistor 24, turning the selection transistor 24 on, a reverse bias voltage V_SPAD equal to or greater than the breakdown voltage is applied to the photodiode 21. This allows the photodiode 21 to operate (for example, perform detection).

[0045] On the other hand, when the selection control voltage V_SEL is not applied from the drive circuit 42 to the selection transistor 24 and the selection transistor 24 is in the off state, the reverse bias voltage V_SPAD is not applied to the photodiode 21, and therefore the operation of the photodiode 21 is prohibited.

[0046] When a photon is incident on the photodiode 21 while the selection transistor 24 is in the ON state, an avalanche current is generated in the photodiode 21. This causes the avalanche current to flow through the quench resistor 23, increasing the voltage at the connection point N1. When the voltage at the connection point N1 becomes higher than the ON voltage of the NMOS transistor 25b, the NMOS transistor 25b turns ON, and the voltage at the connection point N2 changes from the power supply voltage VDD to 0 V. When the voltage at the connection point N2 changes from the power supply voltage VDD to 0 V, the PMOS transistor 26a changes from the OFF state to the ON state, and the NMOS transistor 26b changes from the ON state to the OFF state, causing the voltage at the connection point N3 to change from 0 V to the power supply voltage VDD. As a result, a high-level detection signal V_OUT is output from the buffer 27.

[0047] Thereafter, as the voltage at the connection point N1 continues to rise, the voltage applied between the anode and cathode of the photodiode 21 becomes smaller than the breakdown voltage, thereby stopping the avalanche current and decreasing the voltage at the connection point N1. When the voltage at the connection point N1 becomes lower than the on-voltage of the NMOS transistor 25b, the NMOS transistor 25b is turned off, and the output of the detection signal V_OUT from the buffer 27 stops (low level).

[0048] In this way, the readout circuit 22 outputs a high-level detection signal V_OUT during the period from the time when a photon is incident on the photodiode 21, generating an avalanche current and thereby turning on the NMOS transistor 25b, to the time when the avalanche current stops and the NMOS transistor 25b turns off.

[0049] <1-5. Example of the Configuration of the SPAD Adder> An example of the configuration of the SPAD adder 44 according to this embodiment will be described with reference to Fig. 5. Fig. 5 is a diagram showing an example of the configuration of the SPAD adder 44 according to this embodiment. Note that the SPAD adder 44 may be included in the light receiving unit 13 or may be included in the calculation unit 14.

[0050] 5 , the SPAD adder 44 includes, for example, a pulse shaping unit 44 a and a light-receiving number counting unit 44 b. This SPAD adder 44 is provided, for example, for each SPAD pixel 20 (or macro pixel). For each SPAD pixel 20 (or macro pixel), the detection signal V_OUT output from the readout circuit 22 is input to the SPAD adder 44 via the output circuit 43.

[0051] The pulse shaping unit 44 a shapes the pulse waveform of the detection signal V_OUT input from the SPAD pixel 20 (or macro pixel) of the SPAD array 37 via the output circuit 43 into a pulse waveform with a time width according to the operating clock of the SPAD addition unit 44 .

[0052] The light reception counting unit 44b counts the detection signal V_OUT input from the SPAD pixel 20 (or macro pixel) for each sampling period, thereby counting the number of SPAD pixels 20 (detection number) at which incident photons are detected for each sampling period, and outputs this count value as the pixel value of the SPAD pixel 20 (or macro pixel).

[0053] (Sampling Period) The sampling period is a period for measuring the time (time of flight) from when the light-emitting unit 12 emits the laser light L1 until the light-receiving unit 13 detects the incidence of a photon. This sampling period is set to be shorter than the light-emitting period of the light-emitting unit 12. For example, by shortening the sampling period, it becomes possible to calculate the time of flight of a photon emitted from the light-emitting unit 12 and reflected by the object 90 with higher time resolution. This means that by increasing the sampling frequency, it becomes possible to calculate the distance L to the object 90 with higher ranging resolution.

[0054] For example, if the flight time from when the light-emitting unit 12 emits laser light L1 to when this laser light L1 is reflected by the object 90 and when the reflected light L2 is incident on the light-receiving unit 13 is t, the speed of light C is constant (C≈300,000,000 m / s), and therefore the distance L to the object 90 can be calculated from the formula L=C×t / 2. If the sampling frequency is 1 GHz, the sampling period is 1 ns. In this case, one sampling period corresponds to 15 cm. This means that the ranging resolution when the sampling frequency is 1 GHz is 15 cm. Furthermore, if the sampling frequency is doubled to 2 GHz, the sampling period becomes 0.5 ns, and therefore one sampling period corresponds to 7.5 cm. This means that doubling the sampling frequency can reduce the ranging resolution by half. In this way, by increasing the sampling frequency and shortening the sampling period, it is possible to calculate the distance L to the object 90 with higher accuracy.

[0055] <1-6. Configuration Example of Calculation Unit> A configuration example of the calculation unit 14 according to this embodiment will be described with reference to Figs. 6 to 8. Fig. 6 is a diagram showing a configuration example of the calculation unit 14 according to this embodiment. Figs. 7 and 8 each show a histogram (for example, a histogram H ｉ , cumulative histogram H' ｉ－１ , cumulative histogram H' ｉ ) is a diagram for explaining an example of a process of accumulating and subtracting.

[0056] 6, the calculation unit 14 includes a memory 51, a processing unit 52, and a minimum value determination unit 53. In the example of FIG. 6, each SPAD addition unit 44 is connected to the processing unit 52. These SPAD addition units 44 calculate the histogram H ｉ It functions as a generator that generates the histogram H ｉ is an acquired histogram generated based on the output signals from the light receiving unit 13 (each SPAD pixel 20).

[0057] The memory 51 stores, for example, a past cumulative histogram H'. ｉ－１ The memory 51 stores data such as the above. The memory 51 may store other data as needed. The memory 51 is connected to the processing unit 52 (for example, the integrating unit 52 a) and transmits the stored data to the processing unit 52.

[0058] The processing unit 52 includes an accumulating unit 52a and a subtracting unit 52b. ｉ and the past cumulative histogram H' ｉ－１ The subtraction unit 52b subtracts the subtraction value B from the generated histogram. ｉ－１ is subtracted, and the integrated histogram H' is obtained. ｉ This cumulative histogram H' is generated. ｉ is stored in the memory 51 as a past integrated histogram for the next integration. For example, when the above-described integration and subtraction are repeated a predetermined number of times, the integrated histogram H' ｉ is passed to the distance measurement process (output).

[0059] The minimum value determination unit 53 determines the cumulative histogram H' ｉThen, when the subtraction unit 52b performs subtraction, the minimum value determination unit 53 searches for and determines the minimum value from the minimum value B. ｉ－１ The minimum value may be determined every time an accumulation is performed, or may be determined once for multiple accumulations. In the latter case, the minimum value is determined as B ｉ may be set to 0.

[0060] Here, the cumulative histogram H' ｉ and subtraction value (minimum value) B ｉ is expressed by the following equations (1) and (2): ｉ is the i-th acquired histogram. ｉ The initial value of B ０ may be set to 0, for example.

[0061] As shown in FIG. 7, the i-th acquired histogram H ｉ is the past integrated histogram H′ stored in the memory 51. ｉ－１ In detail, the i-th acquired histogram H ｉ The count value (count number) of each bin of the i-1th integrated histogram H' ｉ－１ are added to the respective count values ​​of each bin of H to generate a new histogram. ｉ is a histogram shaped with the horizontal axis representing bins and the vertical axis representing count values ​​(cumulative pixel values). ｉ－１ is a histogram obtained by accumulating multiple histograms.

[0062] Then, a predetermined subtraction value B is subtracted from the new histogram. ｉ－１ = min(H' ｉ－１ ) is subtracted. ｉ－１ is the past cumulative histogram H' ｉ－１ The minimum value determining unit 53 determines the minimum value in advance from the past integrated histogram H'. ｉ－１ to the minimum value min(H' ｉ－１) is determined and stored, and the i-th acquired histogram H ｉ Subtraction value B when subtracting from ｉ－１ is set as

[0063] Then, subtract value B from the new histogram. ｉ－１ When the subtraction is performed, the cumulative histogram H' is obtained as shown in FIG. ｉ This cumulative histogram H' is generated. ｉ is stored in the memory 51 and is passed to the minimum value determination unit 53 for searching for the minimum value. ｉ to the minimum value min(H' ｉ ) is determined and stored, and the i+1th acquired histogram H ｉ＋１ Subtraction value B when subtracting from ｉ is set as

[0064] After acquiring a desired number N of histograms, the calculation unit 14 performs distance measurement processing based on the N histograms H'_N, and calculates the distance L (or an estimated value of the distance L) to the object 90. For example, as the distance measurement processing, the calculation unit 14 identifies the bin number at which the count value (cumulative pixel value) reaches a peak value in the N histograms H'_N, and calculates the distance L to the object 90 by converting the identified bin number into a flight time.

[0065] For example, the calculation unit 14 detects mountain peaks by repeatedly comparing the magnitude of each count value of each adjacent bin number in the histogram H'_N N times, and sets multiple mountain peaks with large peak values ​​as candidates, obtains the bin number of the rising edge of each mountain, and calculates the distance L to the object 90 from the flight time of the reflected light L2. At this time, multiple mountain peaks may be detected, but the host 80 calculates the final distance measurement value by referring to information on surrounding pixels, so information on the distance measurement values ​​of the multiple reflected light candidates is sent to the host 80 via the external I / F 15.

[0066] The conversion from bin number to flight time may be performed using a conversion table stored in advance in a predetermined memory, or a conversion formula for converting bin number to flight time may be stored in advance and the conversion may be performed using this conversion formula.

[0067] In addition, various methods can be used to identify the bin number at which the cumulative pixel value peaks, such as a method of identifying the bin number of the bin with the largest value, or a method of fitting a histogram and identifying the bin number at which the cumulative pixel value peaks from the function curve obtained thereby.

[0068] According to the above-described histogram integration, by accumulating pixel values ​​obtained from multiple light emissions, it is possible to increase the difference between the cumulative pixel value of pixel values ​​that detect reflected light L2 and the cumulative pixel value caused by noise such as ambient light L0. This makes it possible to more reliably distinguish between reflected light L2 and noise, thereby making it possible to more accurately calculate (or estimate) the distance L to the object 90. In addition to the reflected light L2 that is reflected and returned by the object 90, ambient light L0 that is reflected and scattered by the object 90, the atmosphere, etc., is also incident on the light receiving unit 13.

[0069] The main component of the disturbance light L0 is thermal radiation from the sun. When the sunlight is bright and the object 90 has high reflectivity, strong disturbance light L0 is incident on the light-receiving unit 13, resulting in an increase in the overall count value (count number) of the resulting histogram. In extreme cases, the count value of the disturbance light component may exceed the data range set in the histogram (possibility of saturation). One method for avoiding this saturation is to subtract a specific value from the histogram each time the histogram is integrated. However, if the subtraction value is set too large, problems occur when the count value of a histogram bin is smaller than the subtraction value. Therefore, the subtraction value is set based on the average value of the histogram before integration or the minimum value of the histogram before integration.

[0070] For example, if the average value of the acquired histogram before integration is acquired and used as the subtraction value, if the count value of a bin in the histogram is less than the subtraction value, the count value is treated as 0, resulting in an overestimation of the count value. This results in the removal of the downward fluctuations that occur in the count value of the disturbance light component, resulting in an overestimation of the count value of the disturbance light component on the histogram. If the peak count value caused by reflected light is small relative to the disturbance light component fluctuations, for example, because the subject object 90 is located far away, erroneous detection may occur. On the other hand, according to this embodiment, the minimum value is used as the subtraction value, so the above-mentioned overestimation does not occur in principle.

[0071] Furthermore, for example, if the minimum value is extracted from the acquired histogram before integration and that minimum value is used as the subtraction value, if there is one or more bins with a zero count across all bins of the acquired histogram before integration, the subtraction value will be 0, and the subtraction will not be performed. This may prevent the original purpose of preventing histogram saturation from being achieved. A histogram has a data size of, for example, approximately 1,000 bins (approximately 10 bits per bin). To set a subtraction value of 1 or greater, there must be no bins with a zero count among the 1,000 bins of the acquired histogram before integration. The probability of a sufficient amount of subtraction being performed under this condition is sufficiently small.

[0072] On the other hand, according to this embodiment, the minimum value is determined from the histogram after integration and subtraction, and this minimum value is used as the subtraction value. As integration progresses, the probability that the minimum value of the histogram after integration and subtraction will be equal to or greater than 1 increases, so it is possible to avoid the above-mentioned phenomenon in which no subtraction occurs. Note that it is also possible to predict a phenomenon in which almost no subtraction occurs even when integration proceeds, but in such a case, the ambient light L0 is weak and there is no need to avoid saturation, so this phenomenon does not pose a problem.

[0073] By taking these two points into consideration, this embodiment makes it possible to avoid setting the subtraction value (subtraction amount) too high or too low, and to achieve sufficient subtraction of ambient light components in the histogram, thereby preventing a decrease in distance measurement accuracy.

[0074] <1-7. Example of Histogram Integration and Subtraction Processing> An example of histogram integration and subtraction processing (example of accumulation processing) according to this embodiment will be described with reference to Fig. 9. Fig. 9 is a flowchart showing the flow of the histogram integration and subtraction processing according to this embodiment.

[0075] As shown in FIG. 9, the processing unit 52 generates a histogram H ｉ The following process is executed for each bin, for example, starting from the bin with the smallest number.

[0076] The processing unit 52 uses the acquired histogram H ｉ The count value of the desired bin is acquired from the past integrated histogram H' in the memory 51. ｉ－１ The count value corresponding to the desired bin is read from (step S12).

[0077] The processing unit 52 accumulates the count value for each desired bin and subtracts a predetermined value B ｉ－１ For example, the integrating unit 52a integrates the acquired count value of the desired bin with the count value of the desired bin read from the memory, and the subtracting unit 52b subtracts a predetermined subtraction value B from the integrated value of these count values. ｉ－１ Subtract.

[0078] The processing unit 52 calculates a predetermined subtraction value B ｉ－１ The integrated value of the desired bin from which the subtraction is made is stored in the memory 51 as the count value of the desired bin, and the minimum value determination unit 53 determines whether the stored count value of the desired bin is the minimum value among the count values ​​of all the bins (step S14).

[0079] The processing unit 52 determines whether the count values ​​of all the bins have been acquired (step S15). If the processing unit 52 determines that the count values ​​of all the bins have not been acquired (step S15: No), the processing unit 52 returns to step S12. If the processing unit 52 determines that the count values ​​of all the bins have been acquired (step S15: Yes), the processing unit 52 proceeds to step S16.

[0080] The minimum value determination unit 53 calculates the count value of the desired bin, which is the minimum value among the count values ​​of all the bins, by subtracting the count value B ｉ (Step S16) For example, the minimum value determination unit 53 checks the count value of each bin in turn and updates the smallest value.

[0081] The processing unit 52 determines whether or not the histogram has been accumulated the desired number of times N (step S17). If the processing unit 52 determines that the histogram has not been accumulated the desired number of times N (step S17: No), the processing unit 52 returns to step S11. If the processing unit 52 determines that the histogram has been accumulated the desired number of times N (step S17: Yes), the processing unit 52 proceeds to step S18.

[0082] The processing unit 52 reads out the histogram for which the desired number of times of integration N has been completed from the memory 51 (step S18), and subtracts the subtraction value B from the read out histogram. Ｎ (step S19), and the histogram after the subtraction is output to an external device (for example, a processing unit for distance measurement processing at a later stage) (step S20).

[0083] In step S13, the order of addition and subtraction may be reversed. ｉ－１ -B ｉ－１ +H ｉ The calculation result of the above is sufficient. The configuration in which the order of the summation and subtraction is reversed will be described in detail later. Also, the process of searching for the minimum value (determining the minimum value) does not have to be performed for each summation. The subtraction value B when the minimum value search is not performed is ｉ is set to 0, for example.

[0084] In step S19, a subtraction value B is applied to the read histogram to set the minimum value of the read histogram to 0. ＮHowever, the present invention is not limited to this. For example, if it is not necessary to set the minimum value of the read histogram to 0, step S19 may be omitted.

[0085] 1-8. Differences Between the Present Embodiment and the Comparative Example The differences between the present embodiment and the comparative example will be described with reference to Fig. 10 and Fig. 11. Fig. 10 is a graph showing the relationship between time bins and count values ​​according to the present embodiment and the comparative example. Fig. 11 is a graph showing the relationship between the amount of incident light and the average value of disturbance light components (final output) according to the present embodiment and the comparative example.

[0086] 10 , the relationship between the time bins and the count values ​​according to this embodiment is shown by waveform A1, and the relationship between the time bins and the count values ​​according to the comparative example is shown by waveforms A2 and A3. Waveform A1 shows a histogram when subtraction of the minimum value according to this embodiment is performed. Waveform A2 shows a histogram when subtraction is not performed. Waveform A3 shows a histogram when subtraction of a specific value other than the minimum value is performed.

[0087] From waveforms A1, A2, and A3, it can be seen that the histogram of waveform A1 has a lower count value than the histograms of waveforms A2 and A3. The count value of the histogram of waveform A1 is about 1 / 9 of the count value of the histogram of waveform A2, and about 1 / 3 of the count value of the histogram of waveform A3. Therefore, by subtracting the minimum value according to this embodiment, saturation can be suppressed compared to the comparative example.

[0088] 11 , the relationship between the amount of incident light and the average value of disturbance light components according to the present embodiment is shown by waveform B1, and the relationship between the amount of incident light and the average value of disturbance light components according to the comparative example is shown by waveform B2. Waveform B1 shows the average value of disturbance light components relative to the amount of incident light in a histogram after subtraction of the minimum value. Waveform B2 shows the average value of disturbance light components relative to the amount of incident light in a histogram after subtraction of a specific value other than the minimum value.

[0089] From these waveforms B1 and B2, it can be seen that the average value of the disturbance light components of waveform B1 is smaller than the average value of the disturbance light components of waveform B2. When the amount of incident light is 8 (Counts / bin / pix / itr), the average value of the disturbance light components of waveform B1 is about 1 / 7 of the average value of the disturbance light components of waveform B2. Therefore, by performing the subtraction according to this embodiment, the disturbance light components of the histogram after subtraction can be suppressed compared to the comparative example.

[0090] <1-9. Modifications> <1-9-1. Modification 1> Modification 1 of the calculation unit 14 according to the present embodiment will be described with reference to Fig. 12. Fig. 12 is a diagram showing an example of the configuration of the calculation unit 14 of Modification 1.

[0091] As shown in Fig. 12, the processing unit 52 according to the first modification performs subtraction and then accumulation. In the example of Fig. 12, the positions of the accumulation unit 52a and the subtraction unit 52b are interchanged with those in Fig. 6, and the order of accumulation and subtraction is reversed compared to Fig. 6. For example, the accumulation unit 52a generates an accumulated histogram H' after a predetermined number of subtractions and accumulations. ｉ is passed to the distance measurement process which is the subsequent process (output). In this modified example 1, as in the above, it is possible to suppress a decrease in distance measurement accuracy.

[0092] <1-9-2. Modification 2> Modification 2 of the calculation unit 14 according to the present embodiment will be described with reference to Fig. 13. Fig. 13 is a diagram showing an example of the configuration of the calculation unit 14 of Modification 2.

[0093] 13 , the processing unit 52 according to the second modification performs subtraction and then accumulation, similar to the first modification. For example, the subtraction unit 52 b passes (outputs) the accumulated histogram after a predetermined number of accumulations and subtractions to a distance measurement process, which is a subsequent process. In the second modification, as described above, it is possible to suppress a decrease in distance measurement accuracy.

[0094] <1-9-3. Modification 3> Modification 3 of the calculation unit 14 according to the present embodiment will be described with reference to Fig. 14. Fig. 14 is a diagram showing an example of the configuration of the calculation unit 14 of Modification 3.

[0095] As shown in Fig. 14, the calculation unit 14 according to the third modification includes a calculation unit 62 in addition to the configuration shown in Fig. 6. The calculation unit 62 calculates the total subtraction value (ΣBｉ ) is held (saved), and the held total subtraction value is added to the histogram and output. In subsequent signal processing or the like, the total subtraction amount is added to all bins of the histogram, thereby completely reproducing the histogram that would have been obtained without subtraction. In this modification 3, as in the above, it is possible to suppress a decrease in distance measurement accuracy.

[0096] <1-10. Actions and Effects> As described above, the distance measuring device according to the embodiment (for example, the ToF sensor 1) generates a first integrated histogram (for example, a past integrated histogram H' ｉ－１ ) to the first minimum value (for example, the subtraction value B ｉ－１ a determination unit (for example, a minimum value determination unit 53) that determines a second integrated histogram (for example, an integrated histogram H' ｉ ) (see FIGS. 6 to 8, etc.). As a result, the first minimum value of the first integrated histogram is used as the subtraction value, so that the count value is not overestimated and it is possible to suppress the occurrence of erroneous detection. Furthermore, since the subtraction value is unlikely to become 0 and subtraction is performed reliably, it is possible to realize sufficient subtraction of ambient light components in the histogram. As a result, it is possible to suppress a decrease in distance measurement accuracy.

[0097] The processing unit 52 may also include an integrating unit 52 a that integrates the first integrated histogram and the first histogram, and a subtracting unit 52 b that subtracts the first minimum value from the histogram obtained by integrating the first integrated histogram and the first histogram (see FIG. 6 ). This makes it possible to reliably prevent a decrease in distance measurement accuracy.

[0098] The subtractor 52b may also pass the second integrated histogram to the distance measurement process, which is a subsequent process (see FIG. 6), thereby reliably preventing a decrease in distance measurement accuracy.

[0099] The processing unit 52 may also include a subtraction unit 52b that subtracts the first minimum value from the first integrated histogram, and an integration unit 52a that integrates the first integrated histogram from which the first minimum value has been subtracted and the first histogram (see FIGS. 12 and 13). This makes it possible to reliably prevent a decrease in distance measurement accuracy.

[0100] The integrating unit 52a may also pass the second integrated histogram to the distance measurement process, which is a subsequent process (see FIG. 12), thereby reliably preventing a decrease in distance measurement accuracy.

[0101] The subtractor 52b may also pass the first integrated histogram from which the first minimum value has been subtracted to the subsequent distance measurement process (see FIG. 13), thereby reliably preventing a decrease in distance measurement accuracy.

[0102] Furthermore, the processing unit 52 may perform the integration of the first integrated histogram and the first histogram and the subtraction of the first minimum value for each bin (see, for example, FIGS. 6 to 8 ). This allows the integration and subtraction to be performed for each bin, thereby reliably suppressing a decrease in distance measurement accuracy.

[0103] Furthermore, the determination unit (for example, the minimum value determination unit 53) may hold the first minimum value (see, for example, FIGS. 6 to 8 ). This makes it possible to hold the first minimum value, thereby reliably suppressing a decrease in distance measurement accuracy.

[0104] The distance measuring device (e.g., the ToF sensor 1) may further include a memory 51 for storing the first and second integrated histograms (see FIG. 6 ). This allows the first and second integrated histograms to be stored, thereby reliably preventing a decrease in distance measurement accuracy.

[0105] Furthermore, a determination unit (e.g., minimum value determination unit 53) determines the nth minimum value from the nth integrated histogram, and processing unit 52 integrates the nth integrated histogram and the nth histogram and subtracts the nth minimum value from them to generate the (n+1)th integrated histogram, where n is a natural number that may increase by 1 (see, for example, Figures 6 to 8). This makes it possible to repeatedly perform integration and subtraction, thereby reliably suppressing degradation of ranging accuracy.

[0106] The distance measuring device (e.g., the ToF sensor 1) may further include a calculation unit 62 that calculates a total sum (total subtraction) of the minimum values ​​(see FIG. 14). This makes it possible to add the total subtraction, which is the total sum of the minimum values, to all bins of the histogram in subsequent signal processing, etc., thereby reproducing the histogram that would have been obtained without subtraction.

[0107] 2. Other Embodiments The configurations and processes according to the above-described embodiments (including examples and modified examples) may be implemented in various different forms other than the above-described embodiments. For example, the configurations and processes are not limited to the above-described examples and may be implemented in various forms. Furthermore, for example, the configurations, processing procedures, specific names, or information including various data and parameters shown in the above documents and drawings may be changed arbitrarily unless otherwise specified.

[0108] Furthermore, the components and processes according to the above-described embodiments (including examples and modifications) do not necessarily have to be physically configured as shown in the drawings. In other words, the specific forms of distribution and integration of the components and processes are not limited to those shown in the drawings, and all or part of them may be functionally or physically distributed or integrated in any unit depending on various loads, usage conditions, etc.

[0109] Furthermore, the configurations and processes of the above-described embodiments (including examples and modified examples) may be combined as appropriate. For example, at least a part of an embodiment may be combined as appropriate with at least a part of another embodiment. Furthermore, the effects of the embodiments are merely examples and are not intended to be limiting, and other effects may also be obtained.

[0110] 3. Application Examples The technology according to the present disclosure can be applied to various products. For example, the technology according to the present disclosure may be realized as a device mounted on any type of moving body, such as an automobile, an electric vehicle, a hybrid electric vehicle, a motorcycle, a bicycle, personal mobility, an airplane, a drone, a ship, a robot, construction machinery, or agricultural machinery (tractor). Furthermore, for example, the technology according to the present disclosure may be realized as a device mounted on an endoscopic surgery system, a microsurgery system, or the like.

[0111] 15 is a block diagram showing a schematic configuration example of a vehicle control system 7000, which is an example of a mobile object control system to which the technology according to the present disclosure can be applied. The vehicle control system 7000 includes a plurality of electronic control units connected via a communication network 7010. In the example shown in FIG. 15 , the vehicle control system 7000 includes a drive system control unit 7100, a body system control unit 7200, a battery control unit 7300, an outside-vehicle information detection unit 7400, an inside-vehicle information detection unit 7500, and an integrated control unit 7600. The communication network 7010 connecting these multiple control units may be an in-vehicle communication network conforming to any standard, such as a Controller Area Network (CAN), a Local Interconnect Network (LIN), a Local Area Network (LAN), or FlexRay (registered trademark).

[0112] Each control unit includes a microcomputer that performs arithmetic processing according to various programs, a memory unit that stores the programs executed by the microcomputer or parameters used in various calculations, and a drive circuit that drives various devices to be controlled. Each control unit includes a network I / F for communicating with other control units via a communication network 7010, and a communication I / F for communicating with devices or sensors inside and outside the vehicle via wired or wireless communication. Figure 15 illustrates the functional configuration of the integrated control unit 7600, including a microcomputer 7610, a general-purpose communication I / F 7620, a dedicated communication I / F 7630, a positioning unit 7640, a beacon receiving unit 7650, an in-vehicle device I / F 7660, an audio / video output unit 7670, an in-vehicle network I / F 7680, and a memory unit 7690. Similarly, the other control units also include a microcomputer, a communication I / F, a memory unit, and the like.

[0113] The drivetrain control unit 7100 controls the operation of devices related to the drivetrain of the vehicle in accordance with various programs. For example, the drivetrain control unit 7100 functions as a control device for a drive force generating device for generating drive force for the vehicle, such as an internal combustion engine or a drive motor, a drive force transmission mechanism for transmitting drive force to the wheels, a steering mechanism for adjusting the steering angle of the vehicle, and a braking device for generating braking force for the vehicle. The drivetrain control unit 7100 may also function as a control device for an ABS (Antilock Brake System) or an ESC (Electronic Stability Control), etc.

[0114] A vehicle state detection unit 7110 is connected to the drivetrain control unit 7100. The vehicle state detection unit 7110 includes at least one of a gyro sensor that detects the angular velocity of the axial rotational motion of the vehicle body, an acceleration sensor that detects the acceleration of the vehicle, or a sensor that detects the amount of operation of the accelerator pedal, the amount of operation of the brake pedal, the steering angle of the steering wheel, the engine rotation speed, the rotation speed of the wheels, etc. The drivetrain control unit 7100 performs arithmetic processing using signals input from the vehicle state detection unit 7110, and controls the internal combustion engine, the drive motor, the electric power steering device, the brake device, etc.

[0115] The body system control unit 7200 controls the operation of various devices equipped in the vehicle body according to various programs. For example, the body system control unit 7200 functions as a control device for a keyless entry system, a smart key system, a power window device, or various lamps such as headlamps, backup lamps, brake lamps, turn signals, and fog lamps. In this case, radio waves transmitted from a portable device that serves as a key or signals from various switches can be input to the body system control unit 7200. The body system control unit 7200 receives these radio waves or signals and controls the vehicle's door lock device, power window device, lamps, etc.

[0116] The battery control unit 7300 controls the secondary battery 7310, which is the power supply source for the drive motor, in accordance with various programs. For example, information such as battery temperature, battery output voltage, or remaining battery capacity is input to the battery control unit 7300 from a battery device equipped with the secondary battery 7310. The battery control unit 7300 performs arithmetic processing using these signals, and controls the temperature regulation of the secondary battery 7310 or a cooling device or the like equipped in the battery device.

[0117] The outside vehicle information detection unit 7400 detects information outside the vehicle equipped with the vehicle control system 7000. For example, at least one of an imaging unit 7410 and an outside vehicle information detection unit 7420 is connected to the outside vehicle information detection unit 7400. The imaging unit 7410 includes at least one of a time-of-flight (ToF) camera, a stereo camera, a monocular camera, an infrared camera, and other cameras. The outside vehicle information detection unit 7420 includes at least one of an environmental sensor for detecting the current weather or climate, or a surrounding information detection sensor for detecting other vehicles, obstacles, pedestrians, etc. around the vehicle equipped with the vehicle control system 7000.

[0118] The environmental sensor may be, for example, at least one of a raindrop sensor that detects rain, a fog sensor that detects fog, a sunshine sensor that detects the degree of sunshine, and a snow sensor that detects snowfall. The surrounding information detection sensor may be at least one of an ultrasonic sensor, a radar device, and a LIDAR (Light Detection and Ranging, Laser Imaging Detection and Ranging) device. The imaging unit 7410 and the outside vehicle information detection unit 7420 may each be provided as an independent sensor or device, or may be provided as a device in which multiple sensors or devices are integrated.

[0119] 16 shows an example of the installation positions of the imaging unit 7410 and the vehicle exterior information detection unit 7420. The imaging units 7910, 7912, 7914, 7916, and 7918 are provided, for example, at least one of the front nose, side mirrors, rear bumper, back door, and upper part of the windshield inside the vehicle cabin of the vehicle 7900. The imaging unit 7910 provided on the front nose and the imaging unit 7918 provided on the upper part of the windshield inside the vehicle cabin mainly acquire images of the front of the vehicle 7900. The imaging units 7912 and 7914 provided on the side mirrors mainly acquire images of the sides of the vehicle 7900. The imaging unit 7916 provided on the rear bumper or back door mainly acquires images of the rear of the vehicle 7900. The imaging unit 7918 provided on the upper part of the windshield inside the vehicle cabin is mainly used to detect leading vehicles, pedestrians, obstacles, traffic lights, traffic signs, lanes, etc.

[0120] 16 shows an example of the imaging ranges of the imaging units 7910, 7912, 7914, and 7916. Imaging range a indicates the imaging range of the imaging unit 7910 provided on the front nose, imaging ranges b and c indicate the imaging ranges of the imaging units 7912 and 7914 provided on the side mirrors, respectively, and imaging range d indicates the imaging range of the imaging unit 7916 provided on the rear bumper or back door. For example, by overlaying the image data captured by the imaging units 7910, 7912, 7914, and 7916, a bird's-eye view image of the vehicle 7900 viewed from above can be obtained.

[0121] The outside vehicle information detection units 7920, 7922, 7924, 7926, 7928, and 7930 provided on the front, rear, sides, corners, and above the windshield inside the vehicle cabin of the vehicle 7900 may be, for example, ultrasonic sensors or radar devices. The outside vehicle information detection units 7920, 7926, and 7930 provided on the front nose, rear bumper, back door, and above the windshield inside the vehicle cabin of the vehicle 7900 may be, for example, LIDAR devices. These outside vehicle information detection units 7920 to 7930 are mainly used to detect preceding vehicles, pedestrians, obstacles, etc.

[0122] Returning to FIG. 15 , the explanation will be continued. The outside-vehicle information detection unit 7400 causes the imaging unit 7410 to capture an image outside the vehicle and receives the captured image data. The outside-vehicle information detection unit 7400 also receives detection information from the connected outside-vehicle information detection unit 7420. If the outside-vehicle information detection unit 7420 is an ultrasonic sensor, a radar device, or a LIDAR device, the outside-vehicle information detection unit 7400 emits ultrasonic waves or electromagnetic waves and receives information on the received reflected waves. Based on the received information, the outside-vehicle information detection unit 7400 may perform object detection processing or distance detection processing for people, vehicles, obstacles, signs, text on the road, etc. Based on the received information, the outside-vehicle information detection unit 7400 may also perform environmental recognition processing for recognizing rainfall, fog, road conditions, etc. Based on the received information, the outside-vehicle information detection unit 7400 may also calculate the distance to an object outside the vehicle.

[0123] The outside vehicle information detection unit 7400 may also perform image recognition processing or distance detection processing to recognize people, vehicles, obstacles, signs, or characters on the road surface based on the received image data. The outside vehicle information detection unit 7400 may perform processing such as distortion correction or alignment on the received image data, and may also generate an overhead image or a panoramic image by combining image data captured by different image capturing units 7410. The outside vehicle information detection unit 7400 may also perform viewpoint conversion processing using image data captured by different image capturing units 7410.

[0124] The interior information detection unit 7500 detects information inside the vehicle. A driver state detection unit 7510 that detects the driver's state is connected to the interior information detection unit 7500, for example. The driver state detection unit 7510 may include a camera that captures an image of the driver, a biosensor that detects the driver's biometric information, or a microphone that collects sound from within the vehicle cabin. The biosensor is provided, for example, on the seat or steering wheel, and detects the biometric information of a passenger sitting in the seat or the driver gripping the steering wheel. The interior information detection unit 7500 may calculate the driver's level of fatigue or concentration based on the detection information input from the driver state detection unit 7510, or may determine whether the driver is dozing off. The interior information detection unit 7500 may perform processing such as noise canceling on the collected audio signal.

[0125] The integrated control unit 7600 controls the overall operation of the vehicle control system 7000 according to various programs. An input unit 7800 is connected to the integrated control unit 7600. The input unit 7800 may be implemented by a device that can be operated by a passenger, such as a touch panel, a button, a microphone, a switch, or a lever. Data obtained by voice recognition of a voice input through a microphone may be input to the integrated control unit 7600. The input unit 7800 may be, for example, a remote control device using infrared or other radio waves, or an externally connected device such as a mobile phone or a personal digital assistant (PDA) that can operate the vehicle control system 7000. The input unit 7800 may be, for example, a camera, in which case the passenger can input information using gestures. Alternatively, data obtained by detecting the movement of a wearable device worn by the passenger may be input. Furthermore, the input unit 7800 may include, for example, an input control circuit that generates an input signal based on information input by the passenger using the input unit 7800 and outputs the input signal to the integrated control unit 7600. Passengers and the like operate this input unit 7800 to input various data to the vehicle control system 7000 and to instruct processing operations.

[0126] The storage unit 7690 may include a ROM (Read Only Memory) that stores various programs executed by the microcomputer, and a RAM (Random Access Memory) that stores various parameters, calculation results, sensor values, etc. The storage unit 7690 may also be realized by a magnetic storage device such as an HDD (Hard Disc Drive), a semiconductor storage device, an optical storage device, a magneto-optical storage device, or the like.

[0127] The general-purpose communication I / F 7620 is a general-purpose communication I / F that mediates communication with various devices present in the external environment 7750. The general-purpose communication I / F 7620 may implement a cellular communication protocol such as GSM (Global System of Mobile communications), WiMAX (registered trademark), LTE (Long Term Evolution), or LTE-Advanced (LTE-A), or other wireless communication protocols such as a wireless LAN (also referred to as Wi-Fi (registered trademark)) or Bluetooth (registered trademark). The general-purpose communication I / F 7620 may connect to a device (e.g., an application server or a control server) present on an external network (e.g., the Internet, a cloud network, or an operator-specific network) via, for example, a base station or an access point. In addition, the general-purpose communication I / F 7620 may connect to a terminal located near the vehicle (e.g., a terminal of a driver, pedestrian, or store, or an MTC (Machine Type Communication) terminal) using, for example, P2P (Peer To Peer) technology.

[0128] The dedicated communication I / F 7630 is a communication I / F that supports a communication protocol designed for use in vehicles. The dedicated communication I / F 7630 may implement a standard protocol such as WAVE (Wireless Access in Vehicle Environment), which is a combination of a lower layer IEEE 802.11p and an upper layer IEEE 1609, DSRC (Dedicated Short Range Communications), or a cellular communication protocol. The dedicated communication I / F 7630 typically performs V2X communication, which is a concept including one or more of vehicle-to-vehicle communication, vehicle-to-infrastructure communication, vehicle-to-home communication, and vehicle-to-pedestrian communication.

[0129] The positioning unit 7640 performs positioning by receiving, for example, GNSS signals from GNSS (Global Navigation Satellite System) satellites (for example, GPS signals from GPS (Global Positioning System) satellites), and generates position information including the latitude, longitude, and altitude of the vehicle. Note that the positioning unit 7640 may identify the current position by exchanging signals with a wireless access point, or may obtain position information from a terminal such as a mobile phone, PHS, or smartphone that has a positioning function.

[0130] The beacon receiving unit 7650 receives, for example, radio waves or electromagnetic waves transmitted from radio stations or the like installed on the road, and acquires information such as the current location, congestion, road closures, required travel time, etc. The function of the beacon receiving unit 7650 may be included in the dedicated communication I / F 7630 described above.

[0131] The in-vehicle device I / F 7660 is a communication interface that mediates connections between the microcomputer 7610 and various in-vehicle devices 7760 present in the vehicle. The in-vehicle device I / F 7660 may establish wireless connections using wireless communication protocols such as wireless LAN, Bluetooth (registered trademark), NFC (Near Field Communication), or WUSB (Wireless USB). Furthermore, the in-vehicle device I / F 7660 may establish a wired connection such as USB (Universal Serial Bus), HDMI (High-Definition Multimedia Interface), or MHL (Mobile High-Definition Link) via a connection terminal (and a cable, if necessary) not shown. The in-vehicle device 7760 may include, for example, at least one of a mobile device or a wearable device owned by a passenger, or an information device carried into or attached to the vehicle. The in-vehicle device 7760 may also include a navigation device that searches for a route to an arbitrary destination. The in-vehicle device I / F 7660 exchanges control signals or data signals with these in-vehicle devices 7760.

[0132] The in-vehicle network I / F 7680 is an interface that mediates communication between the microcomputer 7610 and the communication network 7010. The in-vehicle network I / F 7680 transmits and receives signals in accordance with a predetermined protocol supported by the communication network 7010.

[0133] The microcomputer 7610 of the integrated control unit 7600 controls the vehicle control system 7000 in accordance with various programs based on information acquired via at least one of the general-purpose communication I / F 7620, the dedicated communication I / F 7630, the positioning unit 7640, the beacon receiving unit 7650, the in-vehicle device I / F 7660, and the in-vehicle network I / F 7680. For example, the microcomputer 7610 may calculate control target values ​​for the driving force generating device, the steering mechanism, or the braking device based on the acquired information inside and outside the vehicle, and output control commands to the drivetrain control unit 7100. For example, the microcomputer 7610 may perform cooperative control aimed at realizing functions of an Advanced Driver Assistance System (ADAS), including vehicle collision avoidance or impact mitigation, following driving based on the following distance, vehicle speed maintenance driving, vehicle collision warning, vehicle lane departure warning, etc. In addition, the microcomputer 7610 may perform cooperative control for the purpose of autonomous driving, in which the vehicle travels autonomously without relying on driver operation, by controlling a driving force generating device, a steering mechanism, a braking device, etc. based on information acquired about the vehicle's surroundings.

[0134] The microcomputer 7610 may generate three-dimensional distance information between the vehicle and objects such as surrounding structures and people, and create local map information including information about the vicinity of the vehicle's current location, based on information acquired via at least one of the general-purpose communication I / F 7620, the dedicated communication I / F 7630, the positioning unit 7640, the beacon receiving unit 7650, the in-vehicle device I / F 7660, and the in-vehicle network I / F 7680. Furthermore, the microcomputer 7610 may predict dangers, such as a vehicle collision, the approach of a pedestrian, or entry into a closed road, based on the acquired information, and generate a warning signal. The warning signal may be, for example, a signal for generating a warning sound or turning on a warning lamp.

[0135] The audio / image output unit 7670 transmits at least one audio and / or image output signal to an output device capable of visually or audibly notifying vehicle occupants or the outside of the vehicle of information. In the example of FIG. 15 , an audio speaker 7710, a display unit 7720, and an instrument panel 7730 are illustrated as examples of the output devices. The display unit 7720 may include, for example, at least one of an on-board display and a head-up display. The display unit 7720 may have an AR (Augmented Reality) display function. The output device may also be other devices, such as headphones, a wearable device such as an eyeglass-type display worn by the occupant, a projector, or a lamp. When the output device is a display device, the display device visually displays results obtained by various processes performed by the microcomputer 7610 or information received from other control units in various formats, such as text, images, tables, and graphs. When the output device is an audio output device, the audio output device converts audio signals, such as reproduced audio data or acoustic data, into analog signals and audibly outputs the analog signals.

[0136] In the example shown in FIG. 15 , at least two control units connected via the communication network 7010 may be integrated into a single control unit. Alternatively, each control unit may be composed of multiple control units. Furthermore, the vehicle control system 7000 may include another control unit not shown. In the above description, some or all of the functions performed by one of the control units may be assigned to another control unit. In other words, as long as information is transmitted and received via the communication network 7010, predetermined arithmetic processing may be performed by one of the control units. Similarly, a sensor or device connected to one of the control units may be connected to another control unit, and multiple control units may transmit and receive detection information to each other via the communication network 7010.

[0137] A computer program for realizing each function of the ToF sensor 1 according to this embodiment described with reference to FIG. 1 can be implemented in any control unit or the like. A computer-readable recording medium storing such a computer program can also be provided. Examples of the recording medium include a magnetic disk, an optical disk, a magneto-optical disk, and a flash memory. The computer program may also be distributed, for example, via a network without using a recording medium.

[0138] In the vehicle control system 7000 described above, the ToF sensor 1 according to the present embodiment described using Fig. 1 can be applied to the integrated control unit 7600 of the application example shown in Fig. 15. For example, the control unit 11, the calculation unit 14, and the external I / F 15 of the ToF sensor 1 correspond to the microcomputer 7610, the storage unit 7690, and the in-vehicle network I / F 7680 of the integrated control unit 7600. However, without being limited thereto, the vehicle control system 7000 or the integrated control unit 7600 may correspond to the host 80 in Fig. 1.

[0139] Furthermore, at least some of the components of the ToF sensor 1 according to the present embodiment described with reference to Fig. 1 may be realized in a module (for example, an integrated circuit module configured on a single die) for the integrated control unit 7600 shown in Fig. 15. Alternatively, at least some of the components of the ToF sensor 1 according to the present embodiment described with reference to Fig. 1 may be realized by a plurality of control units of the vehicle control system 7000 shown in Fig. 15.

[0140] <4. Supplementary Notes> Note that the present technology may also have the following configurations. (1) A ranging device comprising: a determination unit that determines a first minimum value from a first integrated histogram; and a processing unit that generates a second integrated histogram by integrating the first integrated histogram and the first histogram and subtracting the first minimum value. (2) The ranging device according to (1), wherein the processing unit has: an integration unit that integrates the first integrated histogram and the first histogram; and a subtraction unit that subtracts the first minimum value from a histogram obtained by integrating the first integrated histogram and the first histogram. (3) The ranging device according to (2), wherein the subtraction unit passes the second integrated histogram to a ranging process that is a subsequent process. (4) The ranging device according to (1), wherein the processing unit includes: a subtraction unit that subtracts the first minimum value from the first integrated histogram; and an integration unit that integrates the first integrated histogram and the first histogram from which the first minimum value has been subtracted. (5) The ranging device according to (4), wherein the integration unit passes the second integrated histogram to a subsequent processing step, that is, a ranging process. (6) The ranging device according to (4), wherein the subtraction unit passes the first integrated histogram from which the first minimum value has been subtracted to a subsequent processing step, that is, a ranging process. (7) The ranging device according to any one of (1) to (6), wherein the processing unit performs the integration of the first integrated histogram and the first histogram and the subtraction of the first minimum value for each bin. (8) The ranging device according to any one of (1) to (7), wherein the determination unit holds the first minimum value. (9) The distance measuring device according to any one of (1) to (8), further comprising a memory that stores the first integrated histogram and the second integrated histogram. (10) The distance measuring device according to any one of (1) to (9), wherein the determination unit determines an nth minimum value from the nth integrated histogram, the processing unit generates an (n+1)th integrated histogram by integrating the nth integrated histogram and the nth histogram and subtracting the nth minimum value, and n is a natural number that increases by one.(11) The distance measuring device according to (10), further comprising a calculation unit that calculates a total sum of the minimum values. (12) A distance measuring method, including: a computer determining a first minimum value from a first integrated histogram; and generating a second integrated histogram by integrating the first integrated histogram with the first histogram and subtracting the first minimum value. (13) An electronic device comprising the distance measuring device according to any one of (1) to (11). (14) A mobile object comprising the distance measuring device according to any one of (1) to (11). (15) A distance measuring method using the distance measuring device according to any one of (1) to (11).

[0141] 1 ToF sensor 11 Control unit 12 Light emitting unit 13 Light receiving unit 14 Calculation unit 15 External I / F 20 SPAD pixel 21 Photodiode 22 Readout circuit 23 Quench resistor 24 Selection transistor 25 Digital converter 25a Resistor 25b NMOS transistor 26 Inverter 26a PMOS transistor 26b NMOS transistor 27 Buffer 31 Light source 32 Collimator lens 33 Half mirror 34 Driving unit 35 Galvanometer mirror 36 Light receiving lens 37 SPAD array 37a SPAD array used 41 Timing control circuit 42 Driving circuit 43 Output circuit 44 SPAD adding unit 44a Pulse shaping unit 44b Light receiving number counting unit 51 Memory 52 Processing unit 52a Integration unit 52b Subtraction unit 53 Minimum value determination unit 62 Calculation unit 80 Host 90 Object B ｉ Subtraction value H ｉ Histogram H' ｉ cumulative histogram

Claims

1. A distance measuring device comprising: a determination unit that determines a first minimum value from a first integrated histogram; and a processing unit that performs integration of the first integrated histogram and a first histogram and subtracts the first minimum value to generate a second integrated histogram.

2. The distance measuring device according to claim 1, wherein the processing unit has: an integrating unit that integrates the first integrated histogram and the first histogram; and a subtracting unit that subtracts the first minimum value from the histogram obtained by integrating the first integrated histogram and the first histogram.

3. The distance measuring device according to claim 2, wherein the subtraction unit passes the second integrated histogram to a distance measuring process which is a subsequent process.

4. The distance measuring device of claim 1, wherein the processing unit has: a subtraction unit that subtracts the first minimum value from the first integrated histogram; and an integration unit that integrates the first integrated histogram from which the first minimum value has been subtracted and the first histogram.

5. The distance measuring device according to claim 4, wherein the integrating section passes the second integrated histogram to a distance measuring process which is a subsequent process.

6. The distance measuring device according to claim 4, wherein the subtraction unit passes the first integrated histogram from which the first minimum value has been subtracted to a distance measuring process that is a subsequent process.

7. The distance measuring device according to claim 1, wherein the processing unit performs the integration of the first integrated histogram and the first histogram and the subtraction of the first minimum value for each bin.

8. The distance measuring device according to claim 1, wherein the determination unit holds the first minimum value.

9. The distance measuring device according to claim 1, further comprising a memory for storing the first integrated histogram and the second integrated histogram.

10. The distance measuring device of claim 1, wherein the determination unit determines the nth minimum value from the nth integrated histogram, the processing unit integrates the nth integrated histogram with the nth histogram and subtracts the nth minimum value to generate the (n+1)th integrated histogram, and n is a natural number that increases by one.

11. The distance measuring device according to claim 10, further comprising a calculation unit that calculates a total sum of the minimum values.

12. A distance measurement method, comprising: a computer determining a first minimum value from a first integrated histogram; and generating a second integrated histogram by integrating the first integrated histogram with the first histogram and subtracting the first minimum value.

Images