Distance measuring device and distance measuring method

Abstract

The present invention relates to a ranging device and a ranging method, which effectively use memory resources according to situations. A distance measuring device is provided with: a control register (21) that maintains a pixel mode; an array unit (143) in which a plurality of light-receiving elements for detecting incidence of photons are arranged; a reading unit (22) that reads a detection signal from each of the light-receiving elements at a predetermined sampling period; a pixel value generation unit (15, 161, 162) that generates a pixel value of a pixel at each sampling period by summing up the values of the detection signals read from the light-receiving elements by pixel units; and a memory (17) for storing, for each pixel, a histogram of pixel values for each sampling period, in which a region having a first size in the memory is allocated to each pixel when a first pixel mode is maintained in the control register, and a region having a second size in the memory is allocated to each pixel when a second pixel mode is maintained in the control register. A region in the memory having a second size different from the first size is allocated to each pixel.

Description

[0001] This application is a divisional application with application date of October 18, 2019, application number 201980009616.6, and invention name “Distance measuring device and distance measuring method”, all contents of which are incorporated herein by reference. Technical Field

[0002] The present application relates to a distance measuring device and a distance measuring method. Background Art

[0003] In recent years, distance image sensors (hereinafter referred to as ToF sensors) that measure distance based on the ToF (time of flight) method have attracted increasing attention. For example, ToF sensors are known to be produced using CMOS (complementary metal oxide semiconductor) semiconductor integrated circuit technology, and use multiple SPADs (single photon avalanche diodes) arranged in a planar manner to measure the distance to a target object.

[0004] In a ToF sensor using SPAD, the time period spanning between light emitted by a light source and reflected light incident on the SPAD (hereinafter referred to as flight time) is measured multiple times as a physical quantity, and the distance to the target object is identified based on a histogram of the physical quantity generated according to the measurement results.

[0005] Reference List

[0006] Patent Literature

[0007] Patent Document 1: Japanese Patent Publication No. 2010-091378

[0008] Patent Document 2: Japanese Patent Publication No. 2016-176750 Summary of the invention

[0009] Technical issues

[0010] Herein, based on the constant speed of light principle, the interval count of the histogram is equivalent to the distance to the target object. Therefore, in the case of ranging over a short distance, the necessary and sufficient interval count of the histogram is different from that of ranging over a long distance. That is, in the case of ranging over a short distance, it is sufficient to have a small interval count. On the other hand, in the case of ranging over a long distance, a larger interval count is required.

[0011] Furthermore, the bin count of the histogram corresponds to the size (also called depth) of the memory area storing the histogram. Thus, in the case of ranging over short distances, it is sufficient to have a shallow memory area. On the other hand, in the case of ranging over long distances, it is necessary to have a deeper memory area.

[0012] However, conventionally, only a memory area of ​​a certain size is used regardless of the range of the ranging target. Therefore, when a deep memory area is not needed in a case such as ranging over a short distance, some memory areas remain almost unused. This leads to the problem that memory resources are not used in an efficient manner.

[0013] In this regard, in the present application, a distance measuring device and a distance measuring method capable of effectively using memory resources according to circumstances are proposed.

[0014] Solution to the problem

[0015] In order to solve the above-mentioned problems, a distance measuring device according to one aspect of the present disclosure has: a control register that holds one pixel pattern among a plurality of pixel patterns as a set value; an array unit that includes an arrangement of a plurality of light receiving elements, each light receiving element detecting incidence of photons; a reading unit that reads a detection signal from each light receiving element at a predetermined sampling period; a pixel value generating unit that records the number of detection signals read from each of the plurality of light receiving elements at a sampling period in units of pixels, and generates a pixel value for each pixel for each sampling period; and a memory that stores, for each pixel, a histogram of pixel values ​​in each sampling period calculated by the pixel value generating unit, wherein each pixel is composed of one or more light receiving elements among the plurality of light receiving elements, and when a first pixel pattern is held in the control register, an area having a first size in the memory is allocated to each pixel, and when a second pixel pattern is held in the control register, an area having a second size different from the first size in the memory is allocated to each pixel. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 is a block diagram showing an example of a schematic configuration of a ToF sensor serving as a distance measuring device according to the first embodiment.

[0017] Figure 2 is a block diagram showing an example of a schematic configuration of a light receiving unit according to the first embodiment.

[0018] Figure 3 is a block diagram showing an example of a schematic configuration of a SPAD array according to the first embodiment.

[0019] Figure 4 is a circuit diagram showing an example of a schematic configuration of a SPAD pixel according to the first embodiment.

[0020] Figure 5 is a block diagram showing an example of a more detailed configuration of the adding unit according to the first embodiment.

[0021] Figure 6is a block diagram showing an example of a schematic configuration of a histogram processing unit according to the first embodiment.

[0022] Figure 7 : is a block diagram showing an example of a more detailed configuration of the pixel count variable parallel adder (in the case of the first pixel mode) according to the first embodiment.

[0023] Figure 8 is a block diagram showing an example of a more detailed configuration of the pixel count variable parallel adder (in the case of the second pixel mode) according to the first embodiment.

[0024] Fig. 9 is a block diagram showing an example of a more detailed configuration of the pixel count variable parallel adder (in the case of the third pixel mode) according to the first embodiment.

[0025] Fig.10 is a block diagram showing an example of a more detailed configuration of the pixel count variable parallel adder (in the case of the fourth pixel mode) according to the first embodiment.

[0026] Fig.11 is a block diagram showing an example of a schematic configuration of a matrix transposition unit according to the first embodiment.

[0027] Fig.12 is a block diagram for explaining the operation performed by the histogram generation unit in a single sampling instance according to the first embodiment.

[0028] Fig.13 is a block diagram for explaining operations performed by a histogram generating unit in a single sampling instance (in the case of the first pixel mode) according to the first embodiment by focusing on a single histogram generating unit.

[0029] Fig.14 is a timing chart for explaining the operation performed by the histogram generation unit in a single sampling instance (in the case of the first pixel mode) according to the first embodiment.

[0030] Fig.15 is a block diagram for explaining the operation performed by the histogram generation unit in a single sampling instance (in the case of the second pixel mode) according to the first embodiment.

[0031] Fig.16 is a timing chart for explaining the operation performed by the histogram generation unit in a single sampling instance (in the case of the second pixel mode) according to the first embodiment.

[0032] Fig.17is a block diagram for explaining the operation performed by the histogram generation unit in a single sampling instance (in the case of the third pixel mode) according to the first embodiment.

[0033] Fig.18 2 is a timing chart for explaining the operation performed by the histogram generation unit in a single sampling instance (in the case of the third pixel mode) according to the first embodiment.

[0034] Fig.19 is a block diagram for explaining the operation performed by the histogram generation unit in a single sampling instance (in the case of the fourth pixel mode) according to the first embodiment.

[0035] Fig. 20 2 is a timing chart for explaining the operation performed by the histogram generation unit in a single sampling instance (in the case of the fourth pixel mode) according to the first embodiment.

[0036] Fig.21 2 is a diagram for explaining operations performed in the histogram generating unit according to the first embodiment.

[0037] Fig. 22 is a flowchart for explaining an example of the overall operation performed by the ToF sensor according to the first embodiment.

[0038] Fig.23 It is used to illustrate Fig. 22 0 is a flowchart of an example of a histogram generation operation performed by the histogram generation unit during the first type of histogram operation in step S15 shown in FIG.

[0039] Fig.24 It is used to illustrate Fig. 22 0 is a flowchart of an example of the histogram generation operation performed by the histogram generation unit during the second type histogram operation in step S19 shown in FIG.

[0040] Fig.25 It is used to illustrate Fig. 22 Flowchart of the histogram generation operation performed by the histogram generation unit during the third type of histogram operation in step S23 shown in FIG.

[0041] Fig.26 It is used to illustrate Fig. 22 Flowchart of the histogram generation operation performed by the histogram generation unit during the fourth type of histogram operation in step S26 shown in FIG.

[0042] Fig. 27 is a diagram showing an exemplary histogram created in a memory in response to a first light emission example from a light emission unit according to the first embodiment.

[0043] Fig.28 : is a diagram showing an example of a pixel value at each sampling number obtained in response to the second light emission example from the light emitting unit according to the first embodiment.

[0044] Fig.29 is a diagram showing an exemplary histogram created in a memory in response to a second light emission example from a light emission unit according to the first embodiment.

[0045] Fig.30 : is a diagram showing an example of a pixel value at each sampling number obtained in response to the third light emission example from the light emitting unit according to the first embodiment.

[0046] Fig.31 is a diagram illustrating an exemplary histogram created in a memory in response to a third light emission example from a light emission unit according to the first embodiment.

[0047] Fig.32 is a schematic diagram showing an example of a schematic configuration of a ToF sensor serving as a distance measuring device according to the second embodiment.

[0048] Fig.33 is a block diagram showing an example of a schematic configuration of a vehicle control system.

[0049] Fig.34 2 is an explanatory diagram showing an example of installation positions of the vehicle exterior information detection portion and the imaging portion. DETAILED DESCRIPTION

[0050] The preferred embodiments of the present application are described in detail below with reference to the accompanying drawings. In the embodiments described below, similar components are represented by the same reference numerals, and no repeated description thereof will be given.

[0051] The present application will be described in the following order of items.

[0052] 1. Get Started

[0053] 2. First Implementation

[0054] 2.1 Distance measuring device (ToF sensor)

[0055] 2.2 Light receiving unit

[0056] 2.3 SPAD Array

[0057] 2.4SPAD pixels

[0058] 2.5 Example of overall operation of SPAD pixel

[0059] 2.6 Addition Unit

[0060] 2.7 Sampling period

[0061] 2.8 Histogram Processing Unit

[0062] 2.8.1 Parallel adder with variable pixel count

[0063] 2.8.1.1 Pixel mode = 0 (first pixel mode)

[0064] 2.8.1.2 Pixel mode = 1 (second pixel mode)

[0065] 2.8.1.3 Pixel mode = 2 (third pixel mode)

[0066] 2.8.1.4 Pixel mode = 3 (fourth pixel mode)

[0067] 2.8.2 Matrix Transposition Unit

[0068] 2.8.3 Example of Schematic Configuration of Histogram Generating Unit

[0069] 2.8.3.1 First Pixel Mode (Pixel Mode = 0)

[0070] 2.8.3.1.1 Operations performed in a single sampling instance

[0071] 2.8.3.1.2 Operations in response to a single emission instance

[0072] 2.8.3.2 Second Pixel Mode (Pixel Mode = 1)

[0073] 2.8.3.2.1 Operations performed in a single sampling instance

[0074] 2.8.3.2.2 Operations in response to a single light instance

[0075] 2.8.3.3 Third Pixel Mode (Pixel Mode = 2)

[0076] 2.8.3.3.1 Operations performed in a single sampling instance

[0077] 2.8.3.3.2 Operations in response to a single light instance

[0078] 2.8.3.4 Fourth Pixel Mode (Pixel Mode = 3)

[0079] 2.8.3.4.1 Operations performed in a single sampling instance

[0080] 2.8.3.4.2 Operations in response to a single light instance

[0081] 2.9 Example of Circuit for Histogram Processing

[0082] 2.10 Operation Examples

[0083] 2.10.1 The first type of histogram generation operation

[0084] 2.10.2 The Second Type of Histogram Generation Operation

[0085] 2.10.3 The third type of histogram generation operation

[0086] 2.10.4 The fourth type of histogram generation operation

[0087] 2.11 Specific example of histogram creation

[0088] 2.12 Distance Estimation / Calculation Unit

[0089] 2.13 Actions / Effects

[0090] 3. Second Implementation

[0091] 3.1 Distance measuring device (ToF sensor)

[0092] 3.2 Action / Effect

[0093] 4. Application examples

[0094] 1. Get Started

[0095] As described above, in the case of ranging over a short distance, the required depth of the memory area is different from the case of ranging over a long distance.

[0096] Furthermore, in the case of performing distance measurement over a short distance, the necessary and sufficient resolution required in the distance measurement image obtained by the ToF sensor is also different from the case of performing distance measurement over a long distance.

[0097] For example, a ToF sensor using SPADs includes a SPAD array having a plurality of SPADs arranged in a two-dimensional grid. The SPADs in the SPAD array are grouped into a plurality of macro pixels (macro pixels), each of which is composed of one or more SPADs. A single macro pixel corresponds to a single pixel in a ranging image. Therefore, when the number of SPADs constituting a single macro pixel is determined and the shape of the region of the macro pixel is determined, the number of macro pixels in the entire ToF sensor is determined, thereby determining the resolution of the ranging image. In this regard, for example, if the number of SPADs in a single macro pixel is kept small, it becomes possible to enhance the resolution of the ranging image obtained by the ToF sensor.

[0098] However, if the number of SPADs in a single macropixel is kept small, the dynamic range of the pixel value obtained from the single macropixel becomes narrower. Therefore, any weak reflected light from a distant object is buried in the ambient light (also called background light) with a roughly constant intensity, and it is difficult to distinguish the reflected light from the ambient light.

[0099] To this end, in order to enhance the detection performance of distant objects or objects with low reflectivity, it is necessary to widen the dynamic range of pixel values ​​by increasing the number of SPADs in a single macropixel.

[0100] In this way, a single SPAD array needs to be used to achieve two contradictory usage states: namely, for a ToF sensor using SPADs, a memory is used with a small interval count while a high-resolution usage state is achieved by reducing the number of SPADs in a single macro-pixel; and for a ToF sensor using SPADs, a memory is used with a large interval count while a wide dynamic range usage state is achieved by increasing the number of SPADs in a single macro-pixel.

[0101] In that regard, in the embodiments described below, a ranging device and a ranging method are explained with reference to specific examples, which can keep the resolution and interval count variable, thereby achieving efficient use of memory resources according to the situation, and can rebalance the resolution and dynamic range according to the situation.

[0102] 2. First Implementation

[0103] The first embodiment is described in detail below with reference to the drawings. In the first embodiment, a distance measuring device called a flash type distance measuring device in which SPAD pixels are arranged in a two-dimensional grid and a wide-angle distance measuring image is obtained at once is explained with reference to an example.

[0104] 2.1 Distance measuring device (ToF sensor)

[0105] Figure 1 1 is a block diagram showing an example of a schematic configuration of a ToF sensor used as a distance measuring device according to the first embodiment. Figure 1 As shown, the ToF sensor 1 includes a control unit 11 , a control register 12 , a light emitting unit 13 , a light receiving unit 14 , an adding unit 15 , a histogram processing unit 16 , a distance estimating / calculating unit 18 , and an external interface (I / F) 19 .

[0106] The control unit 11 is configured using an information processing device such as a CPU (Central Processing Unit), for example, and controls constituent elements of the ToF sensor 1. Herein, the control unit 11 controls the constituent elements of the ToF sensor 1 according to a pixel pattern stored in a control register 12 as a setting value.

[0107] The ToF sensor 1 has a plurality of pixel modes in which the resolution and the distance measuring range of the ToF sensor 1 are different, and can switch between the pixel modes based on an instruction from the external host 80 or based on a determination made by the control unit 11. Regarding the setting of the pixel mode, for example, the control unit 11 can set a value input from the host 80 via the external I / F 19 to a set value corresponding to the pixel mode in the control register 12, or the control unit 11 can perform determination and change the set value of the pixel mode in the control register 12.

[0108] For example, when the ToF sensor 1 is implemented in a car, the host 80 may be an ECU (engine control unit) installed in the car. Alternatively, when the ToF sensor 1 is installed in an autonomous mobile robot (such as a household robot), or in an autonomous mobile object (such as a robot vacuum cleaner, or an unmanned aerial vehicle, or a target-guided transport robot), the host 80 may be a control device that controls the autonomous mobile object.

[0109] The control register 12 stores therein, for example, a pixel mode of the ToF sensor 1 as a set value. Although described in detail below with reference to an example, in this description, the pixel mode of the ToF sensor 1 includes pixel mode 0 to pixel mode 3. For example, in pixel mode 0, the resolution is the highest among pixel modes 0 to 3, and the ranging range is narrow. In pixel mode 3, the ranging range is the widest among pixel modes 0 to 3 and the resolution is low. In pixel modes "1" and "2", the resolution and ranging range are between those of pixel mode 0 and pixel mode 3.

[0110] The light emitting unit 13 is configured using, for example, one or more semiconductor laser diodes, and emits pulsed laser light L1 having a predetermined duration at a predetermined cycle (also referred to as a light emitting cycle). Herein, the light emitting unit 13 emits the laser light L1 within an angle range at least equal to or greater than the field angle of the light receiving unit 14. In addition, for example, the light emitting unit 13 emits the laser light L1 having a duration of 1 ns (nanoseconds) at a cycle of 1 GHz (gigahertz). For example, when an object 90 is present within the ranging range, the laser light L1 emitted from the light emitting unit 13 is reflected from the object 90 and falls on the light receiving unit 14 as reflected light L2.

[0111] Although detailed description is made later, for example, the light receiving unit 14 includes a plurality of SPAD pixels arranged in a two-dimensional grid, and outputs information related to the number of SPAD pixels (hereinafter, referred to as detection count) that detects the incidence of photons on the light receiving unit after light is emitted from the light emitting unit 13 (for example, this information is equivalent to the number of detection signals as described later). For example, in response to a single light emission instance from the light emitting unit 13, the light receiving unit 14 detects the incidence of photons at a predetermined sampling period, and outputs the detection count.

[0112] An adding unit 15 totals the detection counts output from the light receiving unit 14 for each of a plurality of SPAD pixels (equivalent to one or more macro pixels (described later)), and outputs each count value as a pixel value to a histogram processing unit 16 .

[0113] The histogram processing unit 16 creates a histogram based on the pixel values ​​obtained for each of one or more macro pixels, wherein the horizontal axis represents the flight time (e.g., equal to a number indicating a sampling order (hereinafter, referred to as a sampling number)), and wherein the vertical axis represents the cumulative pixel value. For example, the histogram is created in the memory 17 installed in the histogram processing unit 16. As the memory 17, for example, an SRAM (static random access memory) can be used. However, this is not the only feasible case, and alternatively, various types of memories such as DRAM (dynamic RAM) can be used as the memory 17.

[0114] The distance estimation / calculation unit 18 identifies the flight time corresponding to the peak of the cumulative pixel value from the histogram created by the histogram processing unit 16. Furthermore, based on the identified flight time, the distance estimation / calculation unit 18 estimates or calculates the distance from the ToF sensor 1 or from the device on which the ToF sensor 1 is mounted to the object 90 present within the ranging range. Then, the distance estimation / calculation unit 18 outputs information on the estimated / calculated distance to the host computer 80 via, for example, the external I / F 19.

[0115] 2.2 Light receiving unit

[0116] Figure 2 1 is a block diagram showing an example of a schematic configuration of a light receiving unit according to the first embodiment. Figure 2 As shown, the light receiving unit 14 includes a timing control circuit 141 , a driving circuit 142 , a SPAD array 143 , and an output circuit 144 .

[0117] The SPAD array 143 includes a plurality of SPAD pixels 20 arranged in a two-dimensional grid. A pixel drive line LD is connected to each column of SPAD pixels 20 (see Figure 2, in the vertical direction), and an output signal line LS is connected to each row of SPAD pixels 20 (refer to Figure 2 , in the horizontal direction). One end of the pixel driving line LD is connected to the output end of the driving circuit 142 corresponding to each column of the SPAD pixel 20. Similarly, one end of the output signal line LS is connected to the input end of the output circuit 144 corresponding to each row of the SPAD pixel 20.

[0118] The driving circuit 142 includes a shift register and an address decoder, and drives the SPAD pixels 20 of the SPAD array 143 in a full-pixel parallel manner or in units of columns. In this regard, the driving circuit 142 includes at least: a circuit for applying a quenching voltage V_QCH (described later) to the SPAD pixels 20 in the selected column of the SPAD array 143, and a circuit for applying a selection control voltage V_SEL (described later) to the SPAD pixels 20 in the selected column. The driving circuit 142 applies the selection control voltage V_SEL to the pixel driving line LD corresponding to the target column for reading, and selects the SPAD pixels 20 for detecting the incidence of photons in units of columns.

[0119] Herein, a signal (referred to as a detection signal) V_OUT output from each SPAD pixel 20 in a column selected and scanned by the driving circuit 142 is input to the output circuit 144 through the corresponding output signal line LS. The output circuit 144 outputs the detection signal V_OUT input from the SPAD pixel 20 to the adding unit 15 (see FIG. 1 ) arranged with respect to each macro pixel (described later). Figure 1 ).

[0120] The timing control circuit 141 includes a timing generator for generating various timing signals, and controls the driving circuit 142 and the output circuit 144 based on the various timing signals generated by the timing generator.

[0121] 2.3 SPAD Array

[0122] Figure 3 1 is a schematic diagram showing an example of a schematic configuration of a SPAD array according to the first embodiment. Figure 3 As shown, in the configuration of the SPAD array 143, for example, a plurality of SPAD pixels 20 are arranged in a two-dimensional grid. The SPAD pixels 20 are grouped into a plurality of macro pixels 30 consisting of a predetermined number of SPAD pixels 20 arranged in a column direction and / or a row direction. In each macro pixel 30, a region formed by joining the outer edges of the SPAD pixels 20 located at the outermost periphery of the macro pixel 30 has a predetermined shape (e.g., a rectangular shape).

[0123] Meanwhile, the plurality of macro pixels 30 arranged in the column direction are divided into a plurality of pixel groups 31 each having a predetermined number of macro pixels 30. Figure 3 In the example shown in FIG. 1 , 48 macro pixels 30 arranged in the column direction from macro pixel #0 to macro pixel #47 are divided into 6 pixel groups 31 corresponding to j=0 to j=5. Thus, in each pixel group 31, 8 macro pixels 30 corresponding to i=0 to i=7 are included.

[0124] 2.4SPAD pixels

[0125] Figure 4 1 is a circuit diagram showing an example of a schematic configuration of a SPAD pixel according to the first embodiment. Figure 4 As shown, the SPAD pixel 20 includes a photodiode 21 used as a light receiving element, and includes a reading circuit 22 for detecting photons incident on the photodiode 21. In a state where a reverse bias voltage V_SPAD equal to or greater than a breakdown voltage is applied between the anode and cathode of the photodiode 21, when a photon falls on the photodiode 21, an avalanche current is generated in the photodiode.

[0126] The reading circuit 22 includes a quenching resistor 23, a digital converter 25, an inverter 26, a buffer 27, and a selection transistor 24. For example, the quenching resistor 23 is configured using, for example, an N-type MOSFET (metal oxide semiconductor field effect transistor, hereinafter referred to as an NMOS transistor). In the quenching resistor 23, the drain is connected to the anode of the photodiode 21, and the source is grounded via the selection transistor 24. In addition, the gate of the NMOS transistor constituting the quenching resistor 23 is applied with a quenching voltage V_QCH by the driving circuit 142 via the pixel driving line LD, and the quenching voltage V_QCH is preset for the purpose of making the NMOS transistor act as a quenching resistor.

[0127] In the first embodiment, the photodiode 21 represents a SPAD. The SPAD is an avalanche photodiode that operates in a Geiger mode when a reverse bias voltage equal to or greater than a breakdown voltage is applied between its anode and cathode, and is capable of detecting the incidence of a single photon.

[0128] The digital converter 25 includes a resistor 251 and an NMOS transistor 252. The drain of the NMOS transistor 252 is connected to the power supply voltage VDD via the resistor 251, and the source is grounded. In addition, the gate of the NMOS transistor 252 is applied with the voltage of the connection point N1 between the anode of the photodiode 21 and the quenching resistor 23.

[0129] The inverter 26 includes a P-type MOSFET (hereinafter, referred to as a PMOS transistor) 261 and an NMOS transistor 262. The drain of the PMOS transistor 261 is connected to the power supply voltage VDD, and the source is connected to the drain of the NMOS transistor 262. The drain of the NMOS transistor 262 is connected to the source of the PMOS transistor 261, and its source is grounded. The gate of the PMOS transistor 261 and the gate of the NMOS transistor 262 are applied with a voltage of a connection point N2 between the resistor 251 and the drain of the NMOS transistor 252. The output of the inverter 26 is input to the buffer 27.

[0130] The buffer 27 is a circuit for impedance conversion, and when receiving the output signal from the inverter 26 , performs impedance conversion on the output signal and outputs the result as a detection signal V_OUT.

[0131] The selection transistor 24 is, for example, an NMOS transistor. The drain of the selection transistor 24 is connected to the source of the NMOS transistor constituting the quenching resistor 23, and the source is grounded. The selection transistor 24 is connected to the drive circuit 142, and when the selection control voltage V_SEL from the drive circuit 142 is applied to its gate via the pixel drive line LD, it changes from the off state to the on state.

[0132] 2.5 Example of overall operation of SPAD pixel

[0133] For example, Figure 4 The reading circuit 22 shown in FIG. 1 operates in the following manner. First, when the selection transistor 24 is in an on state due to application of the selection control voltage V_SEL from the driving circuit 142, a reverse bias voltage V_SPAD greater than or equal to the breakdown voltage is applied to the photodiode 21. Thus, the operation of the photodiode 21 is allowed.

[0134] On the other hand, although the selection transistor 24 is in an off state because the selection control voltage V_SEL from the drive circuit 142 is not applied, since the reverse bias voltage V_SPAD is not applied to the photodiode 21 , the operation of the photodiode 21 is not allowed.

[0135] Although the selection transistor 24 is in the on state, when a photon falls on the photodiode 21, an avalanche current is generated in the photodiode 21. Therefore, the avalanche current flows to the quenching resistor 23, so that the voltage of the connection point N1 increases. If the voltage of the connection point N1 exceeds the on voltage of the NMOS transistor 252, the NMOS transistor 252 switches to the on state, and the voltage of the connection point N2 changes from the power supply voltage VDD to 0V. Once the voltage of the connection point N2 changes from the power supply voltage VDD to 0V, the PMOS transistor 261 switches from the off state to the on state, the NMOS transistor switches from the on state to the off state, and the voltage of the connection point N3 changes from 0V to the power supply voltage VDD. Therefore, a high-level detection signal V_OUT is output from the buffer 27.

[0136] If the voltage at the connection point N1 continues to increase, the voltage applied between the anode and cathode of the photodiode 21 drops below the breakdown voltage. Therefore, the avalanche current stops and the voltage at the connection point N1 drops. If the voltage at the connection point N1 drops below the turn-on voltage of the NMOS transistor 252, the NMOS transistor 252 switches to the cut-off state, and the output of the detection signal V_OUT from the buffer 27 stops (indicating a low level).

[0137] In this way, during the period from the time when the NMOS transistor 252 switches to the on state due to the avalanche current generated by the incidence of photons on the photodiode 21 to the time when the NMOS transistor 252 switches to the off state due to the cessation of the avalanche current, the reading circuit outputs the detection signal V_OUT of the high level. Then, the detection signal V_OUT is input to the adding unit 15 corresponding to each macro pixel 30 via the output circuit 144 (see Figure 1 ). Therefore, each adding unit 15 receives an input of a detection signal V_OUT equal in number to the number of SPAD pixels 20 that are detected to have photon incidence among the SPAD pixels 20 constituting a single macro-pixel 30 (ie, equal to the detection count).

[0138] 2.6 Addition Unit

[0139] Figure 5 1 is a block diagram showing an example of a more detailed configuration of an adding unit according to the first embodiment. Figure 5 As shown, the adding unit 15 includes, for example, a pulse shaping unit 151 and a light reception instance counting unit 152 .

[0140] The pulse shaping unit 151 shapes the pulse waveform of the detection signal V_OUT input from the SPAD array 143 via the output circuit 144 into a pulse waveform having a duration corresponding to the operation clock of the adding unit 15 .

[0141] The light receiving instance counting unit 152 counts the detection signal V_OUT input from the corresponding macropixel 30 in each sampling cycle, totals the number of SPAD pixels 20 on which photon incidence is detected in each sampling cycle (i.e., records the detection count), and outputs the count value as the pixel value d of the macropixel 30.

[0142] At the same time, Figure 5 In the pixel values ​​d[i][j][8:0] shown in FIG. 1 , [i] represents an identifier for identifying the macro pixel 30 in each pixel group 31. In this example, [i] takes a value from "0" to "7" (see Figure 3 ). In addition, [j] represents an identifier of the enabled pixel group 31. In this example, [j] takes a value from "0" to "5" (see Figure 3 ). In addition, [8:0] represents the bit count of the pixel value d[i][j]. The same is true for the pixel value D[i][j] (described later). Therefore, in Figure 5 , it is shown that a 9-bit pixel value that can take a value from “0” to “511” is generated based on the detection signal V_OUT input from the macro pixel 30 identified by the identifiers i and j.

[0143] 2.7 Sampling period

[0144] The sampling period means the periodicity for measuring the time period (flight time) from the emission of the laser light L1 by the light emitting unit 13 to the detection of the photon on the light receiving unit 14. The sampling period is set to be shorter than the light emission period of the light emitting unit 13. For example, since the sampling period is shortened, the flight time of the photon that has been emitted from the light emitting unit 13 and has been reflected from the object 90 can be estimated or calculated with a higher time resolution. This means that since a higher sampling frequency is set, the distance to the object 90 can be estimated or calculated with a higher ranging resolution.

[0145] For example, if t represents the flight time from the time when the light emitting unit 13 emits the laser L1 to the time when the reflected light L2 obtained when the laser L1 is reflected from the object 90 is incident on the light receiving unit 14, since the speed of light C is constant (where C≈300,000,000 m (meters) / s (seconds)), the distance L to the object 90 can be estimated or calculated as given in the following equation (1).

[0146] L=C×t / 2 (1)

[0147] In this regard, if the sampling frequency is set to be equal to 1 GHz, the sampling period becomes equal to 1 ns (nanoseconds). In this case, a single sampling period is equal to 15 cm (centimeter). This indicates that when the sampling frequency is set to be equal to 1 GHz, the ranging resolution is equal to 15 cm. If the sampling frequency is doubled to 2 GHz, since the sampling period becomes equal to 0.5 ns (nanoseconds), the single sampling period becomes equal to 7.5 cm (centimeter). This indicates that when the sampling frequency is doubled, the ranging resolution can be halved. In this way, since the sampling frequency is increased to shorten the sampling period, the distance to the object 90 can be estimated or calculated with higher accuracy.

[0148] 2.8 Histogram Processing Unit

[0149] The histogram processing unit 16 according to the first embodiment will be described in detail below with reference to the drawings.

[0150] Figure 6 : is a block diagram showing an example of a schematic configuration of a histogram processing unit according to the first embodiment. Figure 6 , for clarity, the configuration of the SPAD array 143 is also shown while showing the adding unit 15. On the other hand, for convenience, the memory 17 is not shown, but it is assumed that the memory is included in each of the plurality of histogram generating units 1630 to 1637.

[0151] like Figure 6 As shown, the adding units 15 are arranged in one-to-one correspondence with the macro pixels 30 in the SPAD array 143. In addition, Figure 1 The histogram processing unit 16 shown in FIG. 1 includes: a pixel count variable-parallel adder 161 arranged in one-to-one correspondence with the pixel groups 31 in the SPAD array 143; a matrix transposition unit 162 arranged together with the pixel count variable-parallel adder 161; and histogram generation units 1630 to 1637 (however, in the case where the histogram generation units 1630 to 1637 are not distinguished, they are referred to as the histogram generation unit 163). Herein, the addition unit 15, the pixel count variable-parallel adder 161, and the matrix transposition unit 162 represent, for example, a pixel value generation unit.

[0152] 2.8.1 Variable Pixel Count Parallel Adder

[0153] Figures 7 to 10 : is a block diagram showing an example of a more detailed configuration of the pixel count variable parallel adder according to the first embodiment. Figure 7 An example of a schematic configuration of the pixel count variable parallel adder 161 when the pixel mode is set to "0" is shown in FIG. Figure 8An example of a schematic configuration of the pixel count variable parallel adder 161 when the pixel mode is set to "1" is shown in FIG. Fig. 9 An example of a schematic configuration of the pixel count variable parallel adder 161 when the pixel mode is set to "2" is shown in FIG. Fig.10 An example of a schematic configuration of the pixel count variable parallel adder 161 when the pixel mode is set to "3" is shown in FIG.

[0154] 2.8.1.1 Pixel mode = 0 (first pixel mode)

[0155] First, if Figure 7 As shown, when the pixel mode "0" is set by the control unit 11, in other words, when the pixel mode with the highest resolution and the narrowest ranging range (hereinafter, referred to as the first pixel mode) is set among all the pixel modes, the pixel count variable parallel adder 161 outputs the eight 8-bit pixel values ​​d[i][j][8:0] (i=0 to 7) without modification after receiving them from the addition unit 15, as eight 12-bit pixel values ​​D[i][j][11:0] (i=0 to 7). Therefore, eight 12-bit pixel values ​​D[i][j][11:0] (i=0 to 7) are output in parallel from the pixel count variable parallel adder 161. In this case, since a single macro pixel is equivalent to a single pixel in the ranging image, the resolution of the ToF sensor 1 becomes equal to the number of macro pixels 30 per unit area.

[0156] 2.8.1.2 Pixel mode = 1 (second pixel mode)

[0157] like Figure 8 As shown, when the pixel mode "1" is set by the control unit 11, in other words, when the pixel mode with the second high resolution and the second narrow ranging range (hereinafter, referred to as the second pixel mode) is set in all pixel modes, four adders 41 are implemented in the pixel count variable parallel adder 161 to add the pixel value of the pixel value d[2k][j][8:0] from i=2k (where k is an integer between 0 and 3, including 0 and 3) to the pixel value d[2k+1][j][8:0] of i=2k+1. Herein, the adder 41 can be implemented by, for example, executing a predetermined program.

[0158] The adder 41 receives inputs of the pixel value d[0][j][8:0] of i=0 and the pixel value d[1][j][8:0] of i=1, adds those 8-bit pixel values, and generates a 12-bit pixel value D. Then, the generated 12-bit pixel value D is output from the pixel count variable parallel adder 161 as the pixel value D[0][j][11:0] of i=0 and the pixel value D[1][j][11:0] of i=1. Therefore, the pixel value D[0][j][11:0] and the pixel value D[1][j][11:0] maintain the same value.

[0159] In the same manner, the adder 41 receives inputs of the pixel value d[2][j][8:0] of i=2 and the pixel value d[3][j][8:0] of i=3, adds those 8-bit pixel values, and generates a 12-bit pixel value D. Then, the generated 12-bit pixel value D is output as the pixel value D[2][j][11:0] and the pixel value D[3][j][11:0] that maintain the same value. In addition, the adder 41 receives inputs of the pixel value d[4][j][8:0] of i=4 and the pixel value d[5][j][8:0] of i=5, adds those 8-bit pixel values, and generates a 12-bit pixel value D. Then, the generated 12-bit pixel value D is output as the pixel value D[4][j][11:0] and the pixel value D[5][j][11:0] that maintain the same value. Furthermore, the adder 41 receives input of the pixel value d[6][j][8:0] of i=6 and the pixel value d[7][j][8:0] of i=7, adds these values, and generates a 12-bit pixel value D. Then, the generated 12-bit pixel value D is output as the pixel value D[6][j][11:0] and the pixel value D[7][j][11:0] which maintain the same value.

[0160] In this way, in the second pixel mode, the values ​​obtained from the two macro pixels 30 are added to generate two pixel values ​​D[2k][j][11:0] and D[2k+1][j][11:0] that maintain the same value. This means that in the second pixel mode, the two macro pixels 30 correspond to a single pixel in the ranging image. Therefore, in the second pixel mode, the resolution of the ToF sensor 1 becomes, for example, equal to half of the resolution in the first pixel mode.

[0161] At the same time, also in this case, eight 12-bit pixel values ​​D[i][j][11:0] (i=0 to 7) are output in parallel from the pixel count variable parallel adder 161 .

[0162] 2.8.1.3 Pixel mode = 2 (third pixel mode)

[0163] like Fig. 9As shown, when the pixel mode "2" is set by the control unit 11, in other words, when the pixel mode with the third high resolution and the third narrow ranging range (hereinafter referred to as the third pixel mode) is set in all pixel modes, two adders 45 are implemented in the pixel count variable parallel adder 161 to add the pixel value d[4k][j][8:0] of i=4k (where k is equal to 0 or 1), the pixel value d[4k+1][j][8:0] of i=4k+1, the pixel value d[4k+2][j][8:0] of i=4k+2 and the pixel value d[4k+3][j][8:0] of i=4k+3.

[0164] Each adder 45 can be further implemented by installing two adders 42, each of which adds two of the four outputs from the four adders 41 implemented in the second pixel mode. In the same manner as the adder 41, the adder 42 can be implemented by, for example, executing a predetermined program.

[0165] The adder 45 receives input of the pixel values ​​d[0][j][8:0] to d[3][j][8:0] of i=0 to 3, adds these 8-bit pixel values ​​and generates a 12-bit pixel value D. Then, the generated 12-bit pixel value D is output from the pixel count variable parallel adder 161 as the pixel values ​​D[0][j][11:0] to D[3][j][11:0] of i=0 to 3. Therefore, the pixel values ​​D[0][j][11:0] to D[3][j][11:0] of i=0 to 3 maintain the same value.

[0166] In the same manner, the adder 45 receives input of the pixel values ​​d[4][j][8:0] to d[7][j][8:0] of i=4 to 7, adds these 8-bit pixel values, and generates a 12-bit pixel value D. Then, the generated 12-bit pixel value D is output as the pixel values ​​D[4][j][11:0] to D[7][j][11:0] maintaining the same value.

[0167] In this way, in the third pixel mode, the values ​​obtained from the four macro pixels 30 are added to generate four pixel values ​​D[4k][j][11:0], D[4k+1][j][11:0], D[4k+2][j][11:0], and D[4k+3][j][11:0] that maintain the same value. This means that in the third pixel mode, the four macro pixels 30 correspond to a single pixel in the ranging image. Therefore, in the third pixel mode, the resolution of the ToF sensor 1 becomes equal to, for example, one quarter of the resolution in the first pixel mode.

[0168] At the same time, also in this case, eight 12-bit pixel values ​​D[i][j][11:0] (i=0 to 7) are output in parallel from the pixel count variable parallel adder 161 .

[0169] 2.8.1.4 Pixel mode = 3 (fourth pixel mode)

[0170] like Fig.10 As shown, when the pixel mode "3" is set by the control unit 11, in other words, when the pixel mode having the lowest resolution and the widest ranging range (hereinafter, referred to as the fourth pixel mode) is set among all the pixel modes, the adder 46 is implemented in the pixel count variable parallel adder 161 to add all the pixel values ​​d[i][j][8:0] (where i=0 to 7)

[0171] The adder 46 can be further realized by installing the adder 43 which adds the outputs of the two adders 42 installed additionally in the third pixel mode. For example, in the same manner as the adders 41 and 42, the adder 43 can be realized by, for example, executing a predetermined program.

[0172] The adder 46 adds all the input pixel values ​​d[i][j][8:0] (i=0 to 7) and generates a 12-bit pixel value D. The generated pixel value D is output from the pixel count variable parallel adder 161 as pixel values ​​D[0][j][11:0] to D[7][j][11:0] of i=0 to 7. Therefore, the pixel values ​​D[0][j][11:0] to D[7][j][11:0] maintain the same value.

[0173] In this way, in the fourth pixel mode, the values ​​obtained from the eight macro pixels 30 are added to generate eight pixel values ​​D[i][j][11:0] (i=0 to 7) maintaining the same value. This means that in the fourth pixel mode, the eight macro pixels 30 correspond to a single pixel in the ranging image. Therefore, in the fourth pixel mode, the resolution of the ToF sensor 1 becomes equal to, for example, one eighth of the resolution in the first pixel mode.

[0174] At the same time, also in this case, eight 12-bit pixel values ​​D[i][j][11:0] (i=0 to 7) are output in parallel from the pixel count variable parallel adder 161 .

[0175] In this way, in the first embodiment, the number of pixel values ​​d added by the pixel count variable parallel adder 161 varies according to the pixel mode.

[0176] 2.8.2 Matrix Transposition Unit

[0177] Fig.111 is a block diagram showing an example of a schematic configuration of a matrix transposition unit according to the first embodiment. Fig.11 As shown, from each of the six pixel count variable parallel adders 161 corresponding to the pixel groups 31 of j=0 to 5, eight pixel values ​​D[i][j] (i=0 to 7, j=0 to 5) are input to the matrix transposition unit 162 (hereinafter, referred to as pixel value sequences 480 to 485). That is, from the six pixel count variable parallel adders 161, 48 pixel values ​​D[i][j] (hereinafter, referred to as pixel value matrices) arranged in 8 rows and 6 columns are input to the matrix transposition unit 162. The matrix transposition unit 162 performs matrix transposition on the input pixel value matrix of 8 rows and 6 columns, and generates a pixel value matrix of 6 rows and 8 columns.

[0178] Then, the matrix transposition unit 162 inputs the post-matrix transposed pixel value matrix column by column to the histogram generation unit 163 arranged at the subsequent stage. Herein, the histogram generation units 1630 to 1637 are arranged to have a one-to-one correspondence with the post-matrix transposed pixel value sequences 490 to 497. Therefore, the pixel value sequence 490 is input to the histogram generation unit 1630; the pixel value sequence 491 is input to the histogram generation unit 1631; the pixel value sequence 492 is input to the histogram generation unit 1632; the pixel value sequence 493 is input to the histogram generation unit 1633; the pixel value sequence 494 is input to the histogram generation unit 1634; the pixel value sequence 495 is input to the histogram generation unit 1635; the pixel value sequence 496 is input to the histogram generation unit 1636; and the pixel value sequence 497 is input to the histogram generation unit 1637. Meanwhile, in the following description, without distinguishing the pixel value sequences 490 to 497, they are referred to as the pixel value sequence 49.

[0179] 2.8.3 Example of Schematic Configuration of Histogram Generating Unit

[0180] Fig.12 , Fig.13 , Fig.15 , Fig.17 and Fig.19 : is a block diagram for explaining the operation performed by the histogram generation unit in one sampling example according to the first embodiment. Fig.14 , Fig.16 , Fig.18 and Fig. 20 is a timing diagram for explaining the operation performed by the histogram generation unit in response to a single emission instance from the emission unit according to the first embodiment. Fig.12 , Fig.13 and Fig.14 is a diagram for explaining the histogram generation unit when the pixel pattern “0” is set. Fig.15and Fig.16 is a diagram for explaining the histogram generation unit when the pixel mode “1” is set. Fig.17 and Fig.18 is a diagram for explaining the histogram generation unit when the pixel mode "2" is set. Fig.19 and Fig. 20 3 is a diagram for explaining the histogram generation unit when the pixel mode “3” is set.

[0181] 2.8.3.1 First Pixel Mode (Pixel Mode = 0)

[0182] First, description will be given regarding the case where the first pixel mode (pixel mode=0) is set in the histogram generating unit 163 .

[0183] 2.8.3.1.1 Operations performed in a single sampling instance

[0184] like Fig.12 As shown, when the pixel mode is set to 0, that is, when the first pixel mode is set by the control unit 11, each of the histogram generation units 1630 to 1637 adds the value of the corresponding pixel value sequence 49 input from the matrix transposition unit 162 to the value of the interval corresponding to the current sampling number in each histogram, and each histogram is stored in a corresponding memory in the memories 170 to 177.

[0185] This operation will be described in detail by mainly describing the histogram generation unit 1630. Fig.13 As shown, the memory 170 in the histogram generation unit 1630 includes memories 1700 to 1705, and the number of memories 1700 to 1705 is equal to the number of pixel groups 31. Each of the memories 1700 to 1705 has a one-to-one correspondence with each of the pixel groups 31 of j=0 to 5, and is used to store a histogram based on the pixel values ​​obtained from the macro pixels 30 in the corresponding pixel group 31.

[0186] In each of the memories 1700 to 1705, each bin number of the histogram (also written as BINIDX) corresponds to, for example, a memory address in the memory. Furthermore, in each of the histograms, a single bin corresponds to a single sampling period. Thus, in the present example, due to the light emission from the light emitting unit 13, a smaller bin number corresponds to a sampling performed at an earlier time, i.e., to a sampling with a lower number of samples. At the same time, in Fig.13 2 shows a case where 256 sampling instances are performed in response to a single lighting instance from the lighting unit 13.

[0187] The histogram generation unit 1630 adds the pixel value D[0][0] of the pixel group 31 having the identifier j equal to "0" from the pixel value sequence 490 input thereto (wherein the identifier i of the macro pixel is set to "0") to the value of the interval corresponding to the current sampling number in the histogram stored in the memory 1700 corresponding to the pixel group 31 of j=0 (hereinafter referred to as the memory of j=0). For example, if the current sampling number corresponds to BINIDX=50, the histogram generation unit 1630 adds the pixel value D[0][0] to the value in the interval of BINIDX=50 in the histogram stored in the memory 1700.

[0188] In the same manner, the histogram generation unit 1630 adds each of the pixel values ​​D[0][1] to D[0][5] obtained from the macropixel 30 in the pixel group 31 of j=1 to 5 to the value of the interval corresponding to the current sampling number in the histogram in the corresponding memory in the memories 1701 to 1705 of j=1 to 5.

[0189] As the above-described operations are performed in each of the histogram generating units 1630 to 1637, the pixel value D[i][j] (where i=0 to 7, j=0 to 5) is added to the value of the corresponding interval in each histogram in the corresponding memory in the memories 1700 to 1705. Therefore, in the first pixel mode, in the memory 17, a histogram is created whose number (48 in this example) is equal to the result of multiplying the number Smax of macro pixels 30 in a single pixel group 31 (8 in this example) by the number Nc of pixel groups 31 (6 in this example).

[0190] Meanwhile, for example, information indicating the interval corresponding to the current sampling number may be input from the control unit 11 to the histogram processing unit 16. Alternatively, the current sampling number may be identified in the histogram processing unit 16 or the adding unit 15. Still alternatively, the time elapsed since the light emitting unit 13 emits light may be monitored, and the histogram generating unit 163 may identify the current sampling number based on the monitored value.

[0191] 2.8.3.1.2 Operations in response to a single emission instance

[0192] like Fig.14 As shown, repeatedly perform reference Fig.12 and Fig.13 The sampling operation described above reaches a predetermined sampling count (in this example, 256 times). During the period of time during which the predetermined number of samplings are performed (in Fig.14During the time period between the sampling numbers P0 to P255 shown in FIG. 1 ), the enable signals EN0 to EN7 for respectively enabling or disabling writing to the memories 170 to 177 are maintained at a high level (enable level). Therefore, Nc memories (in each of the memories 170 to 177) are enabled to be written. Fig.13 In the example shown, writing is performed in memory 1700 to 1705) and, ultimately, 48 histograms are created in memory 17 in response to a single instance of light emission from light emitting unit 13.

[0193] 2.8.3.2 Second Pixel Mode (Pixel Mode = 1)

[0194] Next, a case where the second pixel mode (pixel mode=1) is set in the histogram generating unit 163 is described.

[0195] 2.8.3.2.1 Operations performed in a single sampling instance

[0196] like Fig.15 As shown, when the pixel mode "1" is set, that is, the second pixel mode is set by the control unit 11, due to the addition of the pixel value D by the pixel count variable parallel adder 161 (see Figure 8 ), in the pixel value sequence 49, the pixel values ​​D[0][j] and D[1][j] of i=0 and 1 maintain the same value, the pixel values ​​D[2][j] and D[3][j] of i=2 and 3 maintain the same value, the pixel values ​​D[4][j] and D[5][j] of i=4 and 5 maintain the same value, and the pixel values ​​D[6][j] and D[7][j] of i=6 and 7 maintain the same value.

[0197] Therefore, in the second pixel mode, the histogram generation unit 163 creates a single histogram using 2 pixel values ​​D that maintain the same value. More specifically, the histogram generation unit 163 creates a single histogram using pixel values ​​D[0][j] and D[1][j], creates a single histogram using pixel values ​​D[2][j] and D[3][j], creates a single histogram using pixel values ​​D[4][j] and D[5][j], and creates a single histogram using pixel values ​​D[6][j] and D[7][j].

[0198] In other words, in the second pixel mode, the histogram generation unit 163 creates a histogram using a pair of memories 170 and 171 storing 2 pixel values ​​D that keep the same value, creates a histogram using a pair of memories 172 and 173 storing 2 pixel values ​​D that keep the same value, creates a histogram using a pair of memories 174 and 175 storing 2 pixel values ​​D that keep the same value, and creates a histogram using a pair of memories 176 and 177 storing 2 pixel values ​​D that keep the same value. At this time, one of the 2 memories in each pair (that is, memory 170, memory 172, memory 174 and memory 176) is assigned to the interval in the first half of the corresponding histogram. In addition, the other memories in each pair (that is, memory 171, memory 173, memory 175 and memory 177) are assigned to the interval in the second half of the corresponding histogram. For example, each of the memory 170, the memory 172, the memory 174, and the memory 176 is allocated to the interval of BINIDX=0 to 255 in the corresponding histogram, and each of the memory 171, the memory 173, the memory 175, and the memory 177 is allocated to the interval of BINIDX=256 to 511 in the corresponding histogram. Therefore, the number of intervals in each histogram can be doubled, thereby enabling the ranging range to be widened.

[0199] More specifically, first, in the histogram generating unit 163, each of the histogram generating units 1630, 1632, 1634 and 1636 adds the pixel value D[2k][j] (wherein j=0 to 5) of i=2k (wherein k is an integer between 0 and 3, inclusive) in the pixel value sequence 49 input from the matrix transposition unit 162 to the value of the interval corresponding to the current sampling number in each histogram, and the histogram is stored in the corresponding memory in the memories 170, 172, 174 and 176. Then, each of the remaining histogram generating units 1631, 1633, 1635 and 1637 adds the pixel value D[2k+1][j] (wherein j=0 to 5) of i=2k+1 to the value of the interval corresponding to the current sampling number in each histogram, and the histogram is stored in the corresponding memory in the memories 171, 173, 175 and 177.

[0200] Therefore, Nc (6 in this example) histograms with twice the number of intervals are created in memories 170 and 171; Nc (6 in this example) histograms with twice the number of intervals are created in memories 172 and 173; Nc (6 in this example) histograms with twice the number of intervals are created in memories 174 and 175; and Nc (6 in this example) histograms with twice the number of intervals are created in memories 176 and 177.

[0201] 2.8.3.2.2 Operations in response to a single light instance

[0202] like Fig.16 As shown, repeatedly perform reference Fig.15 The sampling operation performed reaches a predetermined sampling count, such as twice the sampling count in the first pixel mode (i.e., 512 times in this example). At this time, in the second pixel mode, during the time period for creating the first half of the second pixel mode histogram (e.g., during the time period spanning between the sampling numbers P0 to P255), the high-level enable signals EN0, EN2, EN4, and EN6 for allowing writing are sent only to the memories 170, 172, 174, and 176, respectively, for the first half; and the low-level enable signals EN1, EN3, EN5, and EN7 for prohibiting writing are sent to other memories 171, 173, 175, and 177, respectively, for the second half. Therefore, during the time period spanning between the sampling numbers P0 to P255, the first half of the histogram (e.g., equivalent to the interval of BINIDX=0 to 255) is created in the memories 170, 172, 174, and 176 using the pixel values ​​D of i=0, 2, 4, and 6.

[0203] On the other hand, during the time period for creating the second half of the histogram (e.g., during the time period spanning between the sampling numbers P256 and P511), the low-level enable signals EN0, EN2, EN4, and EN6 for prohibiting writing are sent to the memories 170, 172, 174, and 176, respectively, for the first half; and the high-level enable signals EN1, EN3, EN5, and EN7 for allowing writing are sent to the memories 171, 173, 175, and 177, respectively, for the second half. Therefore, during the time period spanning between the sampling numbers P256 and P511, the pixel values ​​D of i=1, 3, 5, and 7 are used to create the second half of the histogram in the memories 171, 173, 175, and 177 (e.g., the interval equivalent to BINIDX=256 to 511).

[0204] Therefore, Nc (6 in this example) histograms are created in memories 170 and 171 using pixel values ​​D of i=0 and 1; Nc (6 in this example) histograms are created in memories 172 and 173 using pixel values ​​D of i=2 and 3; Nc (6 in this example) histograms are created in memories 174 and 175 using pixel values ​​D of i=4 and 5; and Nc (6 in this example) histograms are created in memories 176 and 177 using pixel values ​​D of i=6 and 7. Therefore, in this example, finally, Smax / 2×Nc (24 in this example) histograms are created in response to a single light-emission instance from the light-emitting unit 13.

[0205] 2.8.3.3 Third Pixel Mode (Pixel Mode = 2)

[0206] Next, a case where the third pixel mode (pixel mode=2) is set in the histogram generating unit 163 is described.

[0207] 2.8.3.3.1 Operations performed in a single sampling instance

[0208] like Fig.17 As shown, when the pixel mode "2" is set, that is, the third pixel mode is set by the control unit 11, due to the addition of the pixel value D by the pixel count variable parallel adder 161 (see Fig. 9 ), in the pixel value sequence 49, the pixel values ​​D[0][j] to D[3][j] of i=0 to 3 maintain the same value, and the pixel values ​​D[4][j] to D[7][j] of i=4 to 7 maintain the same value.

[0209] Therefore, in the third pixel mode, the histogram generation unit 163 creates a single histogram using 4 pixel values ​​D that maintain the same value. More specifically, the histogram generation unit 163 creates a single histogram using pixel values ​​D[0][j] to D[3][j], and creates a single histogram using pixel values ​​D[4][j] to D[7][j].

[0210] In other words, in the third pixel mode, the histogram generation unit 163 creates a histogram using 4 memories 170 to 173 storing 4 pixel values ​​D that maintain the same value, and creates a histogram using 4 memories 174 to 177 storing 4 pixel values ​​D that maintain the same value. At this time, in each group of 4 memories, the initial memory (i.e., memory 170 and memory 174) is allocated to the interval in the first quarter of the corresponding histogram; the second memory (i.e., memory 171 and memory 175) is allocated to the interval in the second quarter of the corresponding histogram; the third memory (i.e., memory 172 and memory 176) is allocated to the interval in the third quarter of the corresponding histogram; and the last memory (i.e., memory 173 and memory 177) is allocated to the interval in the fourth quarter of the corresponding histogram. Therefore, the number of intervals in each histogram can be quadrupled, so that the ranging range can be further widened.

[0211] More specifically, first, in the histogram generating unit 163, each of the histogram generating units 1630 and 1634 adds the pixel value D[4k][j] of i=4k (where k is an integer equal to 0 or 1) in the pixel value sequence 49 input from the matrix transposition unit 162 to the value of the interval corresponding to the current sampling number in each histogram, and the histogram is stored in the corresponding memory in the memories 170 and 174. Then, each of the histogram generating units 1631 and 1635 adds the pixel value D[4k+1][j] of i=4k+1 to the value of the interval corresponding to the current sampling number in each histogram, and the histogram is stored in the corresponding memory in the memories 171 and 175. Subsequently, each of the histogram generating units 1632 and 1636 adds the pixel value D[4k+2][j] of i=4k+2 to the value of the interval corresponding to the current sampling number in each histogram, and the histogram is stored in the corresponding memory in the memories 172 and 176. Finally, each of the histogram generating units 1633 and 1637 adds the pixel value D[4k+3][j] of i=4k+3 to the value of the interval corresponding to the current sampling number in each histogram, and the histogram is stored in the corresponding memory in the memories 173 and 177.

[0212] Therefore, Nc (6 in this example) histograms having four times the number of bins are created in memories 170 to 173 ; and Nc (6 in this example) histograms having four times the number of bins are created in memories 174 to 177 .

[0213] 2.8.3.3.2 Operations in response to a single emission instance

[0214] like Fig.18 As shown, repeatedly perform reference Fig.17 The sampling operation is performed up to a predetermined sampling count, such as four times the sampling count in the first pixel mode (i.e., 1024 times in this example). At this time, in the third pixel mode, during the time period for creating the first quarter of the histogram (e.g., during the time period spanning between the sampling numbers P0 to P255), the high-level enable signals EN0 and EN4 for allowing writing are sent only to the memories 170 and 174, respectively, for the first quarter; and the low-level enable signals EN1, EN2, EN3, EN5, EN6, and EN7 for prohibiting writing are sent to the other memories 171, 172, 173, 175, 176, and 177, respectively, for the second quarter. Therefore, during the time period spanning the sampling numbers P0 to P255, the first quarter of the histogram (e.g., equivalent to the interval of BINIDX=0 to 255) is created in the memories 170 and 174 using the pixel values ​​D of i=0 and 4.

[0215] Subsequently, during the time period for creating the second quarter of the histogram (e.g., during the time period spanning between the sampling numbers P256 and P511), the high-level enable signals EN1 and EN5 for enabling writing are sent only to the memories 171 and 175, respectively, for the second quarter; and the low-level enable signals EN0, EN2, EN3, EN4, EN6, and EN7 for prohibiting writing are sent to the other memories 170, 172, 173, 174, 176, and 177, respectively, for the second quarter. Therefore, during the time period spanning between the sampling numbers P256 and P511, the second quarter of the histogram (e.g., an interval equal to BINIDX=256 to 511) is created in the memories 171 and 175 using the pixel values ​​D of i=1 and 5.

[0216] Subsequently, during the time period for creating the third quarter of the histogram (e.g., during the time period spanning between the sampling numbers P512 and P767), the high-level enable signals EN2 and EN6 for enabling writing are sent only to the memories 172 and 176, respectively, for the third quarter, and the low-level enable signals EN0, EN1, EN3, EN4, EN5, and EN7 for prohibiting writing are sent to the other memories 170, 171, 173, 174, 175, and 177, respectively. Therefore, during the time period spanning between the sampling numbers P512 and P767, the third quarter of the histogram (e.g., equivalent to the interval of BINIDX=512 to 767) is created in the memories 172 and 176 using the pixel values ​​D of i=2 and 6.

[0217] Subsequently, during the time period for creating the fourth quarter of the histogram (e.g., during the time period spanning between the sampling numbers P768 and P1023), the high-level enable signals EN3 and EN7 for allowing writing are sent only to the memories 173 and 177, respectively, for the fourth quarter, and the low-level enable signals EN0, EN1, EN2, EN4, EN5, and EN6 for prohibiting writing are sent to the other memories 170, 171, 172, 174, 175, and 176, respectively, for the fourth quarter. Therefore, during the time period spanning between the sampling numbers P768 and P1023, the fourth quarter of the histogram (e.g., an interval equal to BINIDX=768 to 1023) is created in the memories 173 and 177 using the pixel values ​​D of i=3 and 7.

[0218] Therefore, Nc (6 in this example) histograms are created in memories 170 to 173 using pixel values ​​D of i=0 to 3, and Nc (6 in this example) histograms are created in memories 174 and 177 using pixel values ​​D of i=4 to 7. Therefore, in the present embodiment, finally, Smax / 4×Nc (12 in this example) histograms are created in response to a single light emission instance from the light emitting unit 13.

[0219] 2.8.3.4 Fourth Pixel Mode (Pixel Mode = 3)

[0220] Next, a description will be given of a case where the fourth pixel mode (pixel mode=3) is set in the histogram generating unit 163 .

[0221] 2.8.3.4.1 Operations performed in a single sampling instance

[0222] like Fig.19 As shown, when the pixel mode "3" is set, that is, the fourth pixel mode is set by the control unit 11, due to the addition of the pixel value D by the pixel count variable parallel adder 161 (see Fig. 9 ), in the pixel value sequence 49, all pixel values ​​D[0][j] to D[7][j] with i=0 to 7 maintain the same value.

[0223] Therefore, in the fourth pixel mode, each histogram generation unit 163 creates a single histogram using all (8) pixel values ​​D that maintain the same value. In other words, in the fourth pixel mode, each histogram generation unit 163 creates a single histogram using 8 memories 170 to 177 storing 8 pixel values ​​D. At this time, the initial memory 170 is allocated to the interval of the initial eighth part of the corresponding histogram; the second memory 171 is allocated to the interval of the second eighth part of the corresponding histogram; the third memory 172 is allocated to the interval of the third eighth part of the corresponding histogram; the fourth memory 173 is allocated to the interval of the fourth eighth part of the corresponding histogram; the fifth memory 174 is allocated to the interval of the fifth eighth part of the corresponding histogram; the sixth memory 175 is allocated to the interval of the sixth eighth part of the corresponding histogram; the seventh memory 176 is allocated to the interval of the seventh eighth part of the corresponding histogram; and the last memory 177 is allocated to the interval of the last eighth part of the corresponding histogram. Therefore, the number of bins in each histogram can be increased eightfold, thereby further widening the ranging range.

[0224] More specifically, first, in the histogram generating unit 163, the histogram generating unit 1630 adds the pixel value D[0][j] of i=0 in the pixel value sequence 49 input from the matrix transposition unit 162 to the value of the interval corresponding to the current sampling number in the histogram stored in the memory 170. Subsequently, in the same manner, the histogram generating units 1631 to 1637 add the pixel values ​​D[1][j] to D[7][j] of i=1 to 7 to the value of the interval corresponding to the current sampling number in the histogram stored in the memories 171 to 177, respectively.

[0225] Therefore, Nc (6 in this example) histograms having 8 times the number of bins are created in the memories 170 to 177 .

[0226] 2.8.3.4.2 Operations in response to a single emission instance

[0227] like Fig. 20 As shown, repeatedly perform reference Fig.19 The sampling operation performed reaches a predetermined sampling count, such as 8 times the sampling count in the first pixel mode (i.e., 2048 times in this example). At this time, in the fourth pixel mode, the time period for sending the high-level enable signals EN0 to EN7 that allow writing is sequentially changed in the order of the memories 170 to 177.

[0228] Thus, during the time period spanning between sample numbers P0 and P255, an initial eighth of the histogram is created in memory 170 using pixel values ​​D of i=0 (e.g., equivalent to the interval of BINIDX=0 to 255). During the time period spanning between sample numbers P256 and P511, a second eighth of the histogram is created in memory 171 using pixel values ​​D of i=1 (e.g., equivalent to the interval of BINIDX=256 to 511). During the time period spanning between sample numbers P512 and P767, a third eighth of the histogram is created in memory 172 using pixel values ​​D of i=2 (e.g., equivalent to the interval of BINIDX=512 to 767). During the time period spanning between sample numbers P768 and P1023, a fourth eighth of the histogram is created in memory 173 using pixel values ​​D of i=3 (e.g., equivalent to the interval of BINIDX=768 to 1023). During the time period spanning between sample numbers P1024 and P1279, the fifth eighth portion of the histogram is created in memory 174 using pixel values ​​D with i=4 (e.g., equivalent to the interval of BINIDX=1024 to 1279). During the time period spanning between sample numbers P1280 and P1535, the sixth eighth portion of the histogram is created in memory 175 using pixel values ​​D with i=5 (e.g., equivalent to the interval of BINIDX=1280 to 1535). During the time period spanning between sample numbers P1536 and P1791, the seventh eighth portion of the histogram is created in memory 176 using pixel values ​​D with i=6 (e.g., equivalent to the interval of BINIDX=1536 to 1791). During the time period spanning between the sampling numbers P1792 to P2047, the last one-eighth portion of the histogram (e.g., corresponding to the interval of BINIDX=1792 to 2047) is created in the memory 177 using the pixel value D of i=7. Therefore, using the pixel values ​​D of i=0 to 7, Nc (6 in this example) histograms are created in the memories 170 to 177. Therefore, in this example, finally, in response to a single lighting instance from the lighting unit 13, Smax / 8×Nc (6 in this example) histograms are created.

[0229] 2.9 Example of Circuit for Histogram Processing

[0230] Fig.21 1 is a diagram for explaining the operation performed in the histogram generation unit according to the first embodiment. Fig.21 It is shown in Fig.13The histogram generation unit 1630 shown in FIG. 1 is an example of a flow when creating a histogram in the memory 1700. However, the same operation can also be applied to other memories 1701 to 1705 and to other histogram generation units 1631 to 1637. That is, Fig.21 The configuration shown is arranged in a corresponding manner to each memory for storing a histogram individually (in Fig.13 In the example shown, memories 1700 to 1705). For example, regarding Fig.21 , the histogram generation unit 1630 includes 6 such configurations that have a one-to-one correspondence with the memories 1700 to 1705.

[0231] like Fig.21 As shown, the histogram generating unit 1630 includes selectors 51 and 64, an adder (+1) 52, D-flip-flop (FF) circuits 53, 54, 56, 61, 62, and 65 as synchronization circuits; the adder 63, and an SRAM 55 used as the memory 1700. Herein, the SRAM 55 to which the read address READ_ADDR is input and the SRAM 55 to which the write address WRITE_ADDR is input are the same SRAM (memory 1700).

[0232] In this configuration, the D-FF circuits 61 and 62 , the adder 63 , the selector 64 , and the D-FF circuit 65 enclosed by the dotted line operate as the adder circuit 60 for calculating the cumulative pixel value.

[0233] In the histogram generation operation performed in response to the first sampling instance, first, the start address (assumed to be "0") of the SRAM 55 is input to the selector 51, and the start address of the SRAM 55 is equivalent to the initial BIN number of the histogram. In addition, in the histogram generation operation in response to the first sampling instance, a control signal that causes the selector 51 to output the start address "0" has been input to the selector 51 by a control line (not shown). Therefore, the input start address "0" is output from the selector 51 as the address ADDR. Then, the address ADDR is input to the D-FF circuit 53 and the adder (+1) 52.

[0234] The pixel value D[i][j] (in this description, i=0 and j=0) obtained in the initial sampling instance (eg, sampling number P0) is input to the D-FF circuit 61 representing the initial stage of the adder circuit 60 .

[0235] The output of the D-FF circuit 53 is synchronized with the output of the D-FF circuit 61. Therefore, when the D-FF circuit 53 outputs the address ADDR, the D-FF circuit 61 outputs the pixel value D[i][j].

[0236] The address ADDR output from the D-FF circuit 53 is input to the D-FF circuit 54, and is also input as a read address READ_ADDR to the SRAM 55. The pixel value D[i][j] output from the D-FF circuit 61 is input to the D-FF circuit 62.

[0237] The output of the D-FF circuit 54, the output of the SRAM 55, and the output of the D-FF circuit 62 are synchronized. Therefore, when the D-FF circuit 54 outputs the address ADDR, the accumulated pixel value READ_DATA[i] stored in the interval corresponding to the read address READ_ADDR is output from the SRAM 55, and the pixel value D[i][j] is output from the D-FF circuit 62.

[0238] The address ADDR output from the D-FF circuit 54 is input to the D-FF circuit 56. In addition, the accumulated pixel value READ_DATA[i] read from the SRAM 55 is input to the adder 63. In addition, the pixel value D[i][j] output from the D-FF circuit 62 is input to the adder 63 and the selector 64.

[0239] In the histogram generation operation performed in response to the first sampling instance, a control signal for causing the selector 64 to output the pixel value D[i][j] has been input to the selector 64 via a control line (not shown). Therefore, the input pixel value D[i][j] is output from the selector 64 to the D-FF circuit 65 as the write data BIN_DATA[j].

[0240] The outputs of the D-FF circuit 56 and the D-FF circuit 65 are synchronized. Therefore, when the D-FF circuit 56 outputs the address ADDR, the D-FF circuit 65 outputs the write data BIN_DATA[j].

[0241] The address ADDR output from the D-FF circuit 56 is input as the write address WRITE_ADDR to the SRAM 55. In addition, the write data BIN_DATA[j] is also input to the SRAM 55 in synchronization with the write address WRITE_ADDR. Therefore, the write data BIN_DATA[j] input to the SRAM 55 is stored as the accumulated pixel value in the section specified in the write address WRITE_ADDR.

[0242] Subsequently, in the histogram generation operation performed in response to the second sampling instance and subsequent instances, a control signal for causing the adder (+1) 52 to output an input value is input to the selector 51. The adder (+1) 52 outputs a value obtained by adding 1 to the address ADDR output from the selector 51. Therefore, in the histogram generation operation performed in response to the second sampling instance and subsequent instances, the next address ADDR obtained by adding 1 to the previous address ADDR is output from the selector 51. The output address ADDR is then input to the D-FF circuit 53 and the adder (+1) 52.

[0243] The pixel values ​​D[i][j] (in this description, i=0 and j=0) obtained as a result of the second sampling instance and subsequent instances (e.g., corresponding to the later sampling number P1) are input to the D-FF circuit 61 representing the initial stage of the adder circuit 60.

[0244] Since the output of the D-FF circuit 53 and the output of the D-FF circuit 61 are synchronized, the D-FF circuit 61 outputs the pixel value D[i][j] at the same time as the D-FF circuit 53 outputs the address ADDR.

[0245] Subsequently, since the output of the D-FF circuit 54, the output of the SRAM 55 and the output of the D-FF circuit 62 are synchronized, when the D-FF circuit 54 outputs the address ADDR, the accumulated pixel value READ_DATA[i] stored in the interval corresponding to the read address READ_ADDR is output from the SRAM 55, and the pixel value D[i][j] is output from the D-FF circuit 62.

[0246] In the histogram generation operation performed in response to the second sampling instance and subsequent instances, a control signal for causing the selector 64 to output the value input from the adder 63 is input to the selector 64. The adder 63 outputs a value obtained by adding the cumulative pixel value READ_DATA[i] read from the SRAM 55 to the pixel value D[i][j] output from the D-FF circuit 62. Therefore, in the histogram generation operation performed in response to the second sampling instance and subsequent instances, a value obtained by adding the pixel D[i][j] of the current instance to the cumulative pixel value READ_DATA[i] until that point in time is output as the write data BIN_DATA[j] from the selector 64. Therefore, in the section of the SRAM 55 corresponding to the current sampling instance (current sampling number), the cumulative pixel value obtained by adding the pixel D[i][j] of the current instance to the cumulative pixel value READ_DATA[i] is stored.

[0247] Since the above operation is repeatedly performed by a number equal to the predetermined sampling count, a histogram corresponding to a single instance of light emission from the light emitting unit 13 is created in the memory 17. In addition, when the above operation is repeatedly performed by a number equal to the predetermined light emission count, a histogram is created for estimating or calculating the distance to the object 90 existing in the ranging range.

[0248] 2.10 Operation Examples

[0249] Fig. 22 is a flowchart for explaining an example of the overall operation performed by the ToF sensor according to the first embodiment.

[0250] like Fig. 22 As shown, in operation, first, the control unit 11 obtains the pixel mode set in the control register 12 (step S11). Then, the control unit 11 determines whether the pixel mode is set to "0", that is, whether the first pixel mode is set (step S12). If the pixel mode is set to "0" (yes in step S12), the control unit 11 sets N to the sampling count Nmax in response to a single light-emitting instance from the light-emitting unit 13 (step S13); and sets the pixel mode to =0 in the pixel count variable parallel adder 161 and the histogram generation unit 163 (step S14). Subsequently, the control unit 11 causes the histogram processing unit 16 to perform a first type of histogram operation (for example, see Figure 7 , Fig.11 , Fig.12 , Fig.13 ,and Fig.14 )(Step S15). Then, the system control proceeds to step S27.

[0251] On the other hand, if the pixel mode is not set to "0" (No in step S12), the control unit 11 determines whether the pixel mode is set to "1", that is, whether the second pixel mode is set (step S16). If the pixel mode is set to "1" (Yes in step S16), the control unit 11 sets 2N as the sampling count Nmax (step S17), and sets the pixel mode = 1 in the pixel count variable parallel adder 161 and the histogram generation unit 163 (step S18). Subsequently, the control unit 11 causes the histogram processing unit 16 to perform the second type of histogram operation (for example, see Figure 8 , Fig.11 , Fig.15 ,and Fig.16 )(Step S19). Then, the system control proceeds to step S27.

[0252] On the other hand, if the pixel mode is not set to "1" (No in step S16), the control unit 11 determines whether the pixel mode is set to "2", that is, whether the third pixel mode is set (step S20). If the pixel mode is set to "2" (Yes in step S20), the control unit 11 sets 4N as the sampling count Nmax (step S21), and sets the pixel mode = 2 in the pixel count variable parallel adder 161 and the histogram generation unit 163 (step S22). Subsequently, the control unit 11 causes the histogram processing unit 16 to perform the third type of histogram operation (for example, see Fig. 9 , Fig.11 , Fig.17 ,and Fig.18 ) (Step S23). Then, the system control proceeds to Step S27.

[0253] On the other hand, if the pixel mode is not set to "2" (No in step S20), the control unit 11 determines that the pixel mode is set to "3" and sets 8N as the sampling count Nmax (step S24), and sets the pixel mode = 3 in the pixel count variable parallel adder 161 and the histogram generation unit 163 (step S25). Subsequently, the control unit 11 causes the histogram processing unit 16 to perform the fourth type of histogram operation (for example, see Fig.10 , Fig.11 , Fig.19 ,and Fig. 20 )(Step S26). Then, the system control proceeds to step S27.

[0254] In step S27, based on the histogram generated according to any one of the first to fourth histogram operations (step S15, or step S19, or step S23, or step S26), the control unit 11 estimates or calculates the distance to the object 90 present in the range of measurement. Then, the control unit 11 determines whether to end the current operation (step S28). If the current operation is to be ended (yes in step S28), the control unit 11 ends the current operation. On the other hand, if the current operation is not to be ended (no in step S28), the system control returns to step S11, and the control unit 11 performs subsequent operations again.

[0255] 2.10.1 The first type of histogram generation operation

[0256] Fig.23 It is used to illustrate Fig. 22Flow chart of the example of the histogram generation operation performed by the histogram generation unit during the first type of histogram operation in the step S15 shown in FIG. In the following description, for ease of explanation, emphasis is placed on the histogram generation unit 1630 in the histogram generation unit 163. However, the same operation is also applicable to other histogram generation units 1631 to 1637. In addition, in the following description, for clarity, it is assumed that the bin number (BINIDX) is identical to the sampling number (N).

[0257] like Fig.23 As shown, in the first type of histogram generation operation, first, the histogram generation unit 1630 sets "1" in the light count M for identifying the current number of light-emitting instances from the light-emitting unit 13 (step S101), and resets "0" in the sampling number N for identifying the current sampling number (step S102). Then, the histogram generation unit 1630 inputs the enable signals EN0 to EN7 of the enable level to the memories 170 to 177, respectively, and allows writing to all memories 170 to 177 (step S103). Herein, it is assumed that a counter (not shown) in the histogram generation unit 163 is used to manage the sampling number N.

[0258] Then, the histogram generation unit 1630 receives an input of a pixel value sequence D (pixel value sequence 49) calculated based on the detection signal read from the SPAD pixel 20 of the SPAD array 143 from the matrix transposition unit 162 (step S104), and adds the pixel value D[i][j]=D[0][0] to D[0][5] to the value of the interval with BINIDX=N of the histogram for each "j" stored in the memory 170 (step S105). Meanwhile, as described above, it is assumed that the detection signal reading from the SPAD pixel 20 is repeatedly performed at a predetermined sampling period.

[0259] Then, the histogram generation unit 1630 determines whether the number of samples N has reached the sampling count Nmax representing the maximum value (step S106). If the number of samples N has reached the sampling count Nmax (yes in step S106), the system control proceeds to step S108. However, if the number of samples N has not yet reached the sampling count Nmax (no in step S106), the histogram generation unit 1630 adds 1 to the number of samples N (step S107). Then, the system control returns to step S104, and the histogram generation unit 1630 performs subsequent operations again.

[0260] In step S108, the histogram generation unit 1630 determines whether the number of light emission instances from the light emitting unit 13, for example, has reached a predetermined count Mmax. If the number of light emission instances from the light emitting unit 13 has reached the predetermined count Mmax (yes in step S108), the histogram generation unit 1630 ends the first type of histogram generation operation. However, if the number of light emission instances from the light emitting unit 13 has not reached the predetermined count Mmax (no in step S108), the histogram generation unit 1630 adds 1 to the light emission count M of the light emitting unit 13 (step S109). Then, the system control returns to step S101, and the histogram generation unit 1630 performs subsequent operations again. In this way, since a histogram can be generated based on M times of light emission from the light emitting unit 13, the distance to the object 90 can be estimated or calculated with high accuracy.

[0261] 2.10.2 The Second Type of Histogram Generation Operation

[0262] Fig.24 It is used to illustrate Fig. 22 Flowchart of an example of a histogram generation operation performed by a histogram generation unit during the second type of histogram operation in step S19 shown in FIG. In the following description, for ease of explanation, the focus is placed on the histogram generation unit 1630 in the histogram generation unit 163. However, the same operation also applies to the other histogram generation units 1631 to 1637. In addition, in Fig.24 In the operations shown, the same step numbers are used to indicate Fig.23 The operations shown are the same operations, and their detailed descriptions are omitted.

[0263] like Fig.24 As shown, in the second type of histogram generation operation, first, the histogram generation unit 1630 sets "1" in the emission count M for identifying the current number of emission instances from the emission unit 13 (step S101), sets "1" in the maximum value Qmax of the variable Q (step S201), and resets "0" in the variable Q (step S202). Fig.23 In the same manner as step S102 shown in FIG. 1 , the histogram generation unit 1630 resets “0” in the sampling number N (step S102 ).

[0264] Then, the histogram generation unit 1630 allows the enable signal of the level (in Fig.16 In the example shown, the set of enable signals EN0, EN2, EN4 and EN6 or the set of enable signals EN1, EN3, EN5 and EN7) is input to the 2k+Qth memory (in Fig.15In the example shown, the set of memories 170, 172, 174 and 176 or the set of memories 171, 173, 175 and 177), thereby allowing writing in the 2k+Qth memory (step S203).

[0265] Then, the histogram generation unit 1630 performs the same Fig.23 The same operations as the operations of steps S104 to S107 shown in FIG. 1 are performed until the sampling number N reaches Nmax, and the first half or the second half of each histogram is created.

[0266] Subsequently, the histogram generation unit 1630 determines whether the variable Q has reached the maximum value Qmax (step S204). If the variable Q has reached the maximum value Qmax (yes in step S204), the system control proceeds to step S108. However, if the variable Q has not reached the maximum value Qmax (no in step S204), the histogram generation unit 1630 adds 1 to the variable Q (step S205). Then, the system control returns to step S102, and the histogram generation unit 1630 performs subsequent operations again to create the remaining half of each histogram.

[0267] 2.10.3 The third type of histogram generation operation

[0268] Fig.25 It is used to illustrate Fig. 22 Flowchart of the histogram generation operation performed by the histogram generation unit during the third type of histogram operation in step S23 shown in FIG. In the following description, for ease of explanation, the focus is placed on the histogram generation unit 1630 in the histogram generation unit 163. However, the same operation also applies to the other histogram generation units 1631 to 1637. In addition, in Fig.25 In the operations shown, the same step numbers are used to indicate Fig.23 or Fig.24 The operations shown are the same operations, and their detailed descriptions are omitted.

[0269] exist Fig.25 In the third type of histogram generation operation shown in Fig.24 The second type of histogram generation operation shown in the figure is the same as the operation in the previous example, using Fig.25 The operations in step S301 and step S302 shown in FIG. Fig.24 The operations in step S201 and step S203 shown in FIG.

[0270] In step S301 , the histogram generation unit 1630 sets “3” in the maximum value Qmax of the variable Q.

[0271] In step S302, the histogram generation unit 1630 allows the enable signal (at Fig.18 In the example shown, any one of a pair of enable signals EN0 and EN4, a pair of enable signals EN1 and EN5, a pair of enable signals EN2 and EN6, or a pair of enable signals EN3 and EN7) is output to the 4k+Qth memory (in Fig.17 In the example shown, any pair of memories 170 and 174, or a pair of memories 171 and 175, or a pair of memories 172 and 176, or a pair of memories 173 and 177) is provided, thereby allowing writing in the 4k+Qth memory.

[0272] Subsequently, with Fig.24 In the same manner as the operations of steps S104 to S205 shown in , for each value of the variable Q, the histogram generation unit 1630 performs the same operations as steps S102 to S107 until the sampling number N reaches Nmax from 0, and thus creates each quarter of each histogram.

[0273] Then, the system control proceeds to step S108 , and the histogram generation unit 1630 performs the above-described operation in a repetitive manner until the light emission count M of the light emission unit 13 reaches a predetermined count Mmax.

[0274] 2.10.4 The fourth type of histogram generation operation

[0275] Fig.26 It is used to illustrate Fig. 22 Flowchart of the histogram generation operation performed by the histogram generation unit during the fourth type of histogram operation in step S26 shown in FIG. In the following description, for ease of explanation, the focus is placed on the histogram generation unit 1630 in the histogram generation unit 163. However, the same operation also applies to the other histogram generation units 1631 to 1637. In addition, in Fig.26 In the operations shown, the same step numbers are used to indicate Figure 23 to Figure 25 The operations shown in any of the steps are the same operations, and their detailed descriptions will not be repeated.

[0276] exist Fig.25 In the third type of histogram generation operation shown in Fig.24 The second type of histogram generation operation shown in Fig.25 The same operations as those in the third type of histogram generation operation shown in FIG. Fig.26 The operations in step S401 and step S402 shown in FIG. Fig.24 The operations in step S201 and step S203 shown in Fig.25The operations in step S301 and step S302 shown in FIG.

[0277] In step S401 , the histogram generation unit 1630 sets “7” in the maximum value Qmax of the variable Q.

[0278] In step S402, the histogram generation unit 1630 allows the enable signal (at Fig. 20 In the example shown in FIG. 1 , the enable signal EN0, or the enable signal EN1, or the enable signal EN2, or the enable signal EN3, or the enable signal EN4, or the enable signal EN5, or the enable signal EN6, or the enable signal EN7) is output to the 8k+Qth memory (in the 8k+Qth memory) among the memories 170 to 177. Fig.19 In the example shown, memory 170, or memory 171, or memory 172, or memory 173, or memory 174, or memory 175, or memory 176, or memory 177), thereby allowing writing in the 8k+Qth memory.

[0279] Subsequently, with Fig.24 In the same manner as the operations from step S104 to step S205 shown in , for each value of the variable Q, the histogram generation unit 1630 performs the same operations as the operations from step S102 to step S107 until the sampling number N reaches Nmax from 0, and thus creates each one-eighth portion of each histogram.

[0280] Then, the system control proceeds to step S108. The histogram generation unit 1630 performs the above-described operation in a repetitive manner until the light emission count M of the light emission unit 13 reaches a predetermined count Mmax.

[0281] 2.11 Specific example of histogram creation

[0282] Regarding the creation of a histogram based on the above configuration and operation, a description is given below with reference to a specific example. The following description is given of the case of setting pixel mode=2 (ie, setting the third pixel mode). However, the same description also applies to other pixel modes.

[0283] Fig. 27 is a diagram showing an exemplary histogram created in a memory in response to a first lighting instance (M=1) from a lighting unit. Fig.28 is a diagram showing an example of a pixel value at each sampling number obtained in response to the second light emission example (M=2) from the light emission unit. Fig.29 is a diagram showing an exemplary histogram created in a memory in response to a second lighting instance (M=2) from a lighting unit. Fig.30: is a diagram showing an example of a pixel value at each sampling number obtained in response to the third light emission example (M=3) from the light emission unit. Fig.31 is a diagram showing an exemplary histogram created in the memory in response to the third light emission instance (M=3) from the light emitting unit. Herein, it is assumed that pixel values ​​D[i][j] to D[i+3][j] having the same value for i and j are created in the memory. Figures 27 to 31 The histogram is shown in .

[0284] First, if Fig. 27 As shown, in response to the first light emission instance (M=1) from the light emission unit 13, a histogram is created in the memory 17, wherein pixel values ​​D of each sampling number N obtained by sampling in response to a single light emission instance are stored in corresponding intervals.

[0285] Then, when the second light emission instance (M=2) from the light emitting unit 13 is obtained as shown in FIG. Fig.28 When the histogram shown in Fig.29 As shown, a histogram is created in the memory 17, wherein the value of each bin of the histogram obtained in response to the second lighting instance (M=2) is added to the value of each bin of the histogram obtained in response to the first lighting instance (M=1).

[0286] In the same manner, when the third light emission instance (M=3) is obtained in response to the light emission from the light emitting unit 13, Fig.30 When the histogram shown in FIG. 1 is obtained, in each interval of the histogram stored in the memory 17, the cumulative value of the pixel values ​​obtained from the first light-emitting instance (M=1) to the third light-emitting instance (M=3) (i.e., the cumulative pixel value) is stored, as shown in FIG. Fig.31 shown.

[0287] In this way, it is possible to increase the difference between the cumulative pixel value of the pixel value of the detected reflected light L2 and the cumulative pixel value caused by noise (such as ambient light L0) by adding the pixel values ​​D obtained in response to a plurality of light emission instances from the light emitting unit 13. This makes it possible to enhance the reliability of distinguishing between the reflected light L2 and the noise, and thus makes it possible to more accurately estimate or calculate the distance to the object 90.

[0288] 2.12 Distance Estimation / Calculation Unit

[0289] The distance estimation / calculation unit 18 according to the first embodiment is described below. The distance estimation / calculation unit 18 estimates or calculates the distance to the object 90 based on the histogram created in the memory 17 by the histogram processing unit 16. For example, the distance estimation / calculation unit 18 identifies the number of bins in each histogram when the cumulative pixel value reaches a peak, converts the identified number of bins into flight time or distance information, and estimates or calculates the distance to the object 90 accordingly.

[0290] For example, in the first pixel mode, the distance estimation / calculation unit 18 identifies such a number of bins in the histogram stored in the memories 170 to 177 when the cumulative pixel value is at a peak value, and estimates or calculates the distance to the object 90 based on the identified number of bins. In the second pixel mode, the distance estimation / calculation unit 18 identifies such a number of bins in the histogram configured using a pair of memories 170 and 171, a pair of memories 172 and 173, a pair of memories 174 and 175, and a pair of memories 176 and 177 when the cumulative pixel value is at a peak value, and estimates or calculates the distance to the object 90 based on the identified number of bins. In the third pixel mode, the distance estimation / calculation unit 18 identifies such a number of bins in the histogram configured using a set of memories 170 to 173 and a set of memories 174 to 177 when the cumulative pixel value is at a peak value, and estimates or calculates the distance to the object 90 based on the identified number of bins. In the fourth pixel mode, the distance estimation / calculation unit 18 identifies such bin numbers in the histogram configured using the memories 170 to 177 when the cumulative pixel value is at a peak, and estimates or calculates the distance to the object 90 based on the identified bin numbers.

[0291] Meanwhile, the conversion from the section number to the flight time or distance information may be performed using a conversion table stored in advance in a memory, or a conversion formula for converting the section number to the flight time or distance information may be stored in advance and the conversion may be performed using the conversion formula.

[0292] In addition, various methods can be implemented to identify the number of intervals where the cumulative pixel value is at a peak, such as a method of identifying the number of intervals with the highest value, or a method of fitting a histogram and identifying the number of intervals where the cumulative pixel value is at a peak based on a function curve obtained due to the fitting.

[0293] 2.13 Actions / Effects

[0294] As described above, according to the first embodiment, the usage mode of memories 170 to 177 can be changed according to the pixel mode. For example, in the first pixel mode, 48 histograms are created in memories 170 to 177. In the second pixel mode, 24 histograms are created by forming 4 pairs of 2 memories from memories 170 to 177. In the third pixel mode, 12 histograms are created by forming 2 groups of 4 memories from memories 170 to 177. In the fourth pixel mode, 6 histograms are created by combining all memories 170 to 177. Therefore, when a deep storage area is not required (for example, in the case of ranging at a close distance) or when a deep storage area is required (for example, in the case of ranging at a long distance), the usage mode of memory 17 can be changed, so that storage resources can be effectively used according to the situation.

[0295] Furthermore, according to the first embodiment, a corresponding enable signal that does not allow writing can be supplied from the enable signals EN0 to EN7 to a memory that does not perform writing among the memories 170 to 177, and the memory can be switched to a sleep state. This enables reduction in power consumption.

[0296] Furthermore, according to the first embodiment, the number of macro pixels 30 corresponding to a single pixel in the ranging image can be changed according to the pixel mode. Therefore, the resolution of the ToF sensor 1 can also be changed according to the situation. For example, in the case where ranging is performed at a short distance and a wide dynamic range is not required, a detailed ranging image can be obtained by enhancing the resolution. On the other hand, in the case where ranging is performed at a long distance and a wide dynamic range is required, a ranging image can be obtained in a wide dynamic range by reducing the resolution. Therefore, the resolution can be effectively changed according to the situation.

[0297] 3. Second Embodiment

[0298] In the first embodiment, a flash type distance measuring device is described with reference to an example. In the second embodiment, a scanning type distance measuring device is described with reference to an example. In the following description, the same reference numerals are used to represent the same configuration as the first embodiment, and redundant descriptions are not repeated.

[0299] 3.1 Distance measuring device (ToF sensor)

[0300] Fig.32 1 is a schematic diagram showing an example of a schematic configuration of a ToF sensor used as a distance measuring device according to the second embodiment. Figure 1As shown, the ToF sensor 2 includes a control device 200, a focusing lens 201, a half mirror 202, a micro mirror 203, a light receiving lens 204, a scanning unit 205, a light emitting unit 213, and a light receiving unit 214. The micro mirror 203 and the scanning unit 205 represent the scanning unit mentioned in the claims, and the scanning unit, for example, scans the light falling on the array unit (e.g., equivalent to the SPAD array 143). At the same time, the scanning unit may further include at least one of the focusing lens 201, the half mirror 202 and the light receiving lens 204.

[0301] The light receiving unit 214 has a structure in which the macro pixels 30 according to the first embodiment are arranged in a vertical direction (corresponding to a column direction). Figure 3 Some columns (one or more columns) of the SPAD array 143 shown in FIG. 1 configure the light receiving unit 214. In the following, those columns of the SPAD array 143 are referred to as macro pixel sequences 243. Therefore, referring to Figure 3 In the SPAD array 143 shown in FIG. 2 , the macropixel sequence 243 has a structure in which 48 macropixels 30 from macropixel #0 to macropixel #47 arranged in the column direction are divided into 6 pixel groups 31 of j=0 to 6.

[0302] In the same manner as the light emitting unit 13 according to the first embodiment, the light emitting unit 213 is configured using, for example, one or more semiconductor laser diodes, and emits pulsed laser light L1 having a predetermined duration at a predetermined cycle (also referred to as a light emitting cycle). In addition, for example, the light emitting unit 13 emits laser light L1 having a duration of 1 ns (nanoseconds) at a cycle of 1 GHz (gigahertz).

[0303] The condenser lens 201 focuses the laser light L1 emitted from the light emitting unit 213. For example, the condenser lens 201 focuses the laser light L1 in such a manner that the spread of the laser light L1 is comparable to the field angle of the light receiving unit 214.

[0304] The half mirror 202 reflects at least a portion of the incident laser light L1 toward the micro mirror 203. On the other hand, the half mirror 202 may be replaced with an optical device such as a polarizer that reflects a portion of the light and transmits the rest of the light.

[0305] The micro-reflector 203 is attached to the scanning unit 205 in such a manner that the angle of the micro-reflector 203 can be changed around the center of the reflection surface. The scanning unit 205 causes the micro-reflector 203 to oscillate or vibrate in the horizontal direction so that, for example, the image SA of the laser light L1 that has been reflected from the micro-reflector 203 moves back and forth in the horizontal direction within the predetermined scanning area AR. For example, the scanning unit 205 causes the micro-reflector 203 to oscillate or vibrate in the horizontal direction in such a manner that the image SA of the laser light L1 moves back and forth for 1 ms (milliseconds) within the predetermined scanning area AR. Herein, a stepping motor or a piezoelectric element may be used to cause the micro-reflector 203 to oscillate or vibrate.

[0306] The reflected light L2 obtained from the reflection of the laser light L1 from the object 90 existing in the distance measurement range falls on the micro mirror 203 from the direction opposite to the laser light L1 and along the incident axis of the same optical axis as the emission axis of the laser light L1. When falling on the micro mirror 203, the reflected light L2 falls on the half mirror 202 along the same optical axis as the optical axis of the laser light L1, and a part of the reflected light passes through the half mirror 202.

[0307] An image of the reflected light L2 that has passed through the half mirror 202 passes through the light receiving lens 204 , so that an image of the macro pixel sequence 243 is formed on the light receiving unit 214 .

[0308] For example, the control device 200 includes a control unit 11, a control register 12, an adding unit 15, a histogram processing unit 16, a memory 17, a distance estimation / calculation unit 18, and an external I / F 19. These constituent elements may be, for example, the same as those according to the first embodiment. However, except for the control register 12, the adding unit 15, the histogram processing unit 16, the memory 17, the distance estimation / calculation unit 18, and the external I / F 19. Figure 1 In addition to the constituent elements shown in FIG. 2 , the control unit 11 according to the second embodiment also controls the scanning unit 205 .

[0309] In this configuration, in the same manner as the first embodiment, the adding unit 15 receives an input of the detection signal V_OUT output from each SPAD pixel 20 in each macropixel 30 of the macropixel sequence 243 at a predetermined sampling period. Then, in the same manner as the first embodiment, the adding unit 15 totals the detection signal V_OUT output from the light receiving unit 214 for each of one or more macropixels, and outputs each count value as a pixel value to the histogram processing unit 16.

[0310] The histogram processing unit 16 creates a histogram in the memory 17 from the pixel pattern in the same manner as the first embodiment. Furthermore, based on the histogram created in the memory 17, the distance estimation / calculation unit 18 estimates or calculates the distance to the object 90 in the same manner as the first embodiment.

[0311] 3.2 Action / Effect

[0312] As described above, the usage mode of the memories 170 to 177 can be changed according to the pixel mode not only in the flash type ToF sensor 1 but also in the scanning type ToF sensor 2. Therefore, when a deep storage area is not required (for example, in the case of performing ranging at a close distance) or when a deep storage area is required (for example, in the case of performing ranging at a long distance), the usage mode of the memory 17 can be changed, thereby enabling efficient use of storage resources according to the situation.

[0313] Meanwhile, the remaining configuration, remaining operation, and remaining effects may be the same as those of the first embodiment. Therefore, the same detailed description will not be repeated.

[0314] 4. Application examples

[0315] The technology disclosed in this application can be applied to various products. For example, the technology disclosed in this application can be implemented as a device installed in any type of mobile object, such as a car, an electric vehicle, a hybrid electric vehicle, a motorcycle, a bicycle, a personal mobile body, an airplane, a drone, a ship, a robot, a construction equipment, or an agricultural machine (tractor).

[0316] Fig.33 7000 is a block diagram showing an example of a schematic configuration of a vehicle control system 7000, which is an example of a mobile body control system to which the technology according to the embodiment of the present disclosure can be applied. The vehicle control system 7000 includes a plurality of electronic control units connected to each other via a communication network 7010. Fig.33 In the example shown, the vehicle control system 7000 includes a drive system control unit 7100, a body system control unit 7200, a battery control unit 7300, an external information detection unit 7400, an internal information detection unit 7500, and an integrated control unit 7600. The communication network 7010 that connects the plurality of control units to each other may be an in-vehicle communication network conforming to any standard, such as a controller area network (CAN), a local interconnect network (LIN), a local area network (LAN), FlexRay (registered trademark), and the like.

[0317] Each control unit includes: a microcomputer that performs arithmetic processing according to various programs; a storage unit that stores programs performed by the microcomputer, parameters for various operations, etc.; and a drive circuit that drives various control target devices. Each control unit further includes: a network interface (I / F) for performing communication with other control units via the communication network 7010; and a communication I / F for performing communication with devices, sensors, etc. inside and outside the vehicle through wired communication or radio communication. Fig.33The functional configuration of the integrated control unit 7600 shown includes a microcomputer 7610, a general communication I / F 7620, a dedicated communication I / F 7630, a positioning unit 7640, a beacon receiving unit 7650, an in-vehicle device I / F 7660, a sound / image output unit 7670, an in-vehicle network I / F 7680, and a storage unit 7690. Other control units also similarly include a microcomputer, a communication I / F, a storage unit, and the like.

[0318] The drive system control unit 7100 controls the operation of the devices related to the drive system of the vehicle according to various programs. For example, the drive system control unit 7100 is used as a control device to control: a drive force generating device for generating the drive force of the vehicle, such as an internal combustion engine, a drive motor, etc.; a drive force transmission mechanism for transmitting the drive force to the wheels; a steering mechanism for adjusting the steering angle of the vehicle; a brake device for generating the braking force of the vehicle, etc. The drive system control unit 7100 may have the function of a control device for an anti-lock braking system (ABS), an electronic stability control (ESC), etc.

[0319] The drive system control unit 7100 is connected to a vehicle state detection unit 7110. The vehicle state detection unit 7110 includes, for example, at least one of the following items: a gyro sensor for detecting the angular velocity of the axial rotational motion of the vehicle body, an acceleration sensor for detecting the acceleration of the vehicle, and a sensor for detecting the operation amount of the accelerator pedal, the operation amount of the brake pedal, the steering angle of the steering wheel, the engine speed or the rotation speed of the wheel, etc. The drive system control unit 7100 performs calculation processing using the signal input from the vehicle state detection unit 7110 to control the internal combustion engine, the drive motor, the electric power steering device, the brake device, etc.

[0320] The body system control unit 7200 controls the operation of various devices installed on the body according to various programs. For example, the body system control unit 7200 is used as a control device to control: a keyless entry system, a smart key system, a power window device, or various lights such as headlights, reverse lights, brake lights, turn signals, fog lights, etc. In this case, the body system control unit 7200 can receive radio waves transmitted from a mobile device that replaces the key or signals from various switches as input. The body system control unit 7200 receives these input radio waves or signals to control the vehicle's door lock devices, power window devices, lights, etc.

[0321] The battery control unit 7300 controls the secondary battery 7310 used as a power source for driving the motor according to various programs. For example, the battery control unit 7300 receives information about battery temperature, battery output voltage, remaining battery power, etc. from a battery device including the secondary battery 7310. The battery control unit 7300 uses these signals to perform calculation processing, perform temperature adjustment control of the secondary battery 7310, or control a cooling device of the battery device, etc.

[0322] The vehicle exterior information detection unit 7400 detects information outside the vehicle including the vehicle control system 7000. For example, the vehicle exterior information detection unit 7400 is connected to at least one of the imaging unit 7410 and the vehicle exterior information detection unit 7420. The imaging unit 7410 includes at least one of a time-of-flight (ToF) camera, a stereo camera, a monocular camera, an infrared camera, and other cameras. The vehicle exterior information detection unit 7420 may include at least one of the following items: an environmental sensor for detecting current atmospheric conditions or weather conditions, and a peripheral information detection sensor for detecting other vehicles, obstacles, pedestrians, etc. around the vehicle including the vehicle control system 7000.

[0323] The environmental sensor may be, for example, at least one of the following: a raindrop sensor for detecting rain, a fog sensor for detecting fog, a sunshine sensor for detecting the degree of sunshine, and a snow sensor for detecting snowfall. The peripheral information detection sensor may be at least one of the following: an ultrasonic sensor, a radar device, and a LIDAR device (a light detection and ranging device, or a laser imaging detection and ranging device). Each of the imaging unit 7410 and the vehicle exterior information detection unit 7420 may be configured as an independent sensor or device, or may be configured as a device in which a plurality of sensors or devices are integrated.

[0324] Fig.34 An example of the installation position of the imaging unit 7410 and the vehicle exterior information detection unit 7420 is shown. The imaging units 7910, 7912, 7914, 7916 and 7918 may be arranged at the positions of the front nose, side mirrors, rear bumper, rear door and the upper part of the windshield inside the vehicle 7900. The imaging unit 7910 arranged at the front nose and the imaging unit 7918 arranged at the upper part of the windshield inside the vehicle mainly obtain the image in front of the vehicle 7900. The imaging units 7912 and 7914 arranged at the side mirrors mainly obtain the image of the side of the vehicle 7900. The imaging unit 7916 arranged at the rear bumper or rear door mainly obtains the image of the rear of the vehicle 7900. The imaging unit 7918 arranged at the upper part of the windshield inside the vehicle is mainly used to detect the front vehicle, pedestrians, obstacles, signals, traffic signs, lanes, etc.

[0325] By the way, Fig.34An example of the shooting range of each imaging unit 7910, 7912, 7914 and 7916 is shown. Imaging range a represents the imaging range of the imaging unit 7910 arranged at the front nose. Imaging ranges b and c respectively represent the imaging ranges of the imaging units 7912 and 7914 arranged at the side mirrors. Imaging range d represents the imaging range of the imaging unit 7916 arranged at the rear bumper or the rear door. For example, a bird's-eye view image of the vehicle 7900 observed from above can be obtained by superimposing image data imaged by the imaging units 7910, 7912, 7914 and 7916.

[0326] The vehicle exterior information detection units 7920, 7922, 7924, 7926, 7928, and 7930 arranged at the front, rear, side, and corner of the vehicle 7900 and the upper portion of the windshield inside the vehicle may be ultrasonic sensors or radar devices. The vehicle exterior information detection units 7920, 7926, and 7930 arranged at the front nose of the vehicle 7900, the rear bumper of the vehicle 7900, the rear door, and the upper portion of the windshield inside the vehicle may be LIDAR devices. These vehicle exterior information detection units 7920 to 7930 are mainly used to detect vehicles ahead, pedestrians, obstacles, and the like.

[0327] Back to Fig.33 , continue to describe. The vehicle exterior information detection unit 7400 causes the imaging unit 7410 to image the image outside the vehicle and receive the imaged image data. In addition, the vehicle exterior information detection unit 7400 receives detection information from the vehicle exterior information detection unit 7420 connected to the vehicle exterior information detection unit 7400. When the vehicle exterior information detection unit 7420 is an ultrasonic sensor, a radar device, or a LIDAR device, the vehicle exterior information detection unit 7400 causes ultrasonic waves, electromagnetic waves, etc. to be transmitted, and receives information about the received reflected waves. Based on the received information, the vehicle exterior information detection unit 7400 may perform a process of detecting an object (such as a person, vehicle, obstacle, sign, symbol, etc. on the road), or perform a process of detecting the distance of the object. The vehicle exterior information detection unit 7400 may perform an environment recognition process based on the received information to identify rainfall, fog, road conditions, etc. The vehicle exterior information detection unit 7400 may calculate the distance to an object outside the vehicle based on the received information.

[0328] In addition, based on the received image data, the vehicle exterior information detection unit 7400 may perform image recognition processing for identifying objects (such as people, vehicles, obstacles, signs, symbols, etc. on the road), or perform processing for detecting the distance of the object. The vehicle exterior information detection unit 7400 may perform processing such as distortion correction, alignment, etc. on the received image data, and generate a bird's-eye view image or a panoramic image by combining image data imaged by a plurality of different imaging units 7410. The vehicle exterior information detection unit 7400 may perform viewpoint conversion processing using image data imaged by different imaging units 7410.

[0329] The in-vehicle information detection unit 7500 detects information inside the vehicle. The in-vehicle information detection unit 7500 may be connected to a driver state detection unit 7510 that detects the driver's state. The driver state detection unit 7510 may include a camera for photographing the driver, a biosensor for detecting the driver's biometric information, a microphone for collecting sounds inside the vehicle, and the like. The biosensor may be arranged on a seat surface, a steering wheel, and the like, and detects biometric information of a passenger sitting in a seat or a driver holding the steering wheel. Based on the detection information input from the driver state detection unit 7510, the in-vehicle information detection unit 7500 may calculate the driver's fatigue level or the driver's concentration level, or may distinguish whether the driver is dozing off. The in-vehicle information detection unit 7500 may perform processing such as noise elimination processing on the audio signal obtained by sound collection.

[0330] The integrated control unit 7600 controls the overall operation within the vehicle control system 7000 according to various programs. The integrated control unit 7600 is connected to the input unit 7800. The input unit 7800 is a device that can be operated by the passenger, such as a touch panel, a button, a microphone, a switch, a joystick, etc. The integrated control unit 7600 can receive data obtained by voice recognition of the voice input via the microphone. The input unit 7800 can be a remote control device using infrared or other radio waves, or can be an external connection device such as a mobile phone, a personal digital assistant (PDA), etc. that supports the operation of the vehicle control system 7000. The input unit 7800 can be a camera. In this case, the passenger can input information through gestures. Alternatively, data obtained by detecting the movement of a wearable device worn by the passenger can be input. In addition, the input unit 7800 may include an input control circuit, etc., which generates an input signal based on the information input by the passenger, etc. using the above-mentioned input unit 7800, and outputs the generated input signal to the integrated control unit 7600. Passengers and the like can input various data into the vehicle control system 7000 through the operation input unit 7800 to process operation instructions.

[0331] The storage unit 7690 may include a read-only memory (ROM) storing various programs executed by the microcomputer and a random access memory (RAM) storing various parameters, operation results, sensor values, etc. In addition, the storage unit 7690 may be a magnetic storage device such as a hard disk drive (HDD), a semiconductor storage device, an optical storage device, a magneto-optical storage device, etc.

[0332] The general communication I / F 7620 is a widely used communication I / F that mediates communication with various devices existing in the external environment 7750. The general communication I / F 7620 can implement: cellular communication protocols such as Global System for Mobile Communications (GSM (registered trademark)), Worldwide Interoperability for Microwave Access (WiMAX (registered trademark)), Long Term Evolution (LTE (registered trademark)), LTE Advanced (LTE-A), etc., or other wireless communication protocols such as wireless LAN (also known as Wireless Fidelity (Wi-Fi (registered trademark)), Bluetooth (registered trademark), etc. The general communication I / F 7620 can be connected to a device (e.g., an application server or a control server) existing on an external network (e.g., the Internet, a cloud network, or a company-specific network) via a base station or an access point. In addition, the general communication I / F 7620 can be connected to a terminal (e.g., a terminal of a driver, a pedestrian, or a store, or a machine type communication (MTC) terminal) existing near the vehicle using a peer-to-peer (P2P) technology.

[0333] The dedicated communication I / F 7630 is a communication I / F that supports a communication protocol developed for vehicle use. The dedicated communication I / F 7630 can implement: a standard protocol, for example, Wireless Access in Vehicular Environments (WAVE) (which is a combination of the Institute of Electrical and Electronics Engineers (IEEE) 802.11p as a lower layer and IEEE 1609 as an upper layer), a dedicated short-range communication (DSRC), or a cellular communication protocol. The dedicated communication I / F 7630 generally performs V2X communication including one or more of the following concepts: communication between vehicles and vehicles (vehicle-to-vehicle), communication between roads and vehicles (vehicle-to-infrastructure), communication between vehicles and homes (vehicle-to-home), and communication between pedestrians and vehicles (vehicle-to-pedestrian).

[0334] The positioning unit 7640 can perform positioning by receiving a global navigation satellite system (GNSS) signal from a GNSS satellite (e.g., a GPS signal from a global positioning system (GPS) satellite) to generate position information including the latitude, longitude, and altitude of the vehicle. Incidentally, the positioning unit 7640 can identify the current position by exchanging signals with a wireless access point, and can also obtain position information from a terminal such as a mobile phone, a personal handyphone system (PHS), or a smart phone with a positioning function.

[0335] The beacon receiving unit 7650 can receive radio waves or electromagnetic waves transmitted from a radio station installed on a road or the like, thereby obtaining information on the current position, congestion, road closure, required time, etc. Incidentally, the function of the beacon receiving unit 7650 may be included in the above-mentioned dedicated communication I / F 7630.

[0336] The in-vehicle device I / F 7660 is a communication interface that mediates the connection between the microcomputer 7610 and various in-vehicle devices 7760 present in the vehicle. The in-vehicle device I / F 7660 can establish a wireless connection using wireless communication protocols such as wireless LAN, Bluetooth (registered trademark), near field communication (NFC), or wireless universal serial bus (WUSB). In addition, the in-vehicle device I / F 7660 can establish a wired connection through a universal serial bus (USB), a high-definition multimedia interface (HDMI (registered trademark)), a mobile high-definition link (MHL), etc. via a connection terminal (and a cable, if necessary) not shown in the figure. The in-vehicle device 7760 may include at least one of the following items: a mobile device and a wearable device owned by a passenger and an information device loaded into or attached to the vehicle. The in-vehicle device 7760 may also include a navigation device that searches for a path to an arbitrary destination. The in-vehicle device I / F 7660 exchanges control signals or data signals with these in-vehicle devices 7760.

[0337] The in-vehicle network I / F 7680 is an interface that mediates communication between the microcomputer 7610 and the communication network 7010. The in-vehicle network I / F 7680 transmits and receives signals and the like in accordance with a predetermined protocol supported by the communication network 7010.

[0338] The microcomputer 7610 of the integrated control unit 7600 controls the vehicle control system 7000 according to various programs based on information obtained via at least one of the general communication I / F 7620, the dedicated communication I / F 7630, the positioning unit 7640, the beacon receiving unit 7650, the in-vehicle device I / F 7660, and the vehicle network I / F 7680. For example, the microcomputer 7610 may calculate a control target value for a driving force generating device, a steering mechanism, or a braking device based on the obtained vehicle interior or vehicle exterior related information, and output a control command to the drive system control unit 7100. For example, the microcomputer 7610 may perform cooperative control aimed at realizing functions of an advanced driver assistance system (ADAS), including collision avoidance or impact buffering for a vehicle, following driving based on a vehicle-to-vehicle distance, vehicle speed keeping driving, a vehicle collision warning, a vehicle lane departure warning, and the like. In addition, the microcomputer 7610 can control the drive force generation device, the steering mechanism, the braking device based on the information obtained about the vehicle's surroundings, thereby performing cooperative control intended for automatic driving, etc. that does not depend on the driver's operation.

[0339] The microcomputer 7610 may generate three-dimensional distance information between the vehicle and objects such as surrounding structures and people, and generate local map information including information about the surrounding environment in which the vehicle is currently located, based on information obtained via at least one of the general communication I / F 7620, the dedicated communication I / F 7630, the positioning unit 7640, the beacon receiving unit 7650, the in-vehicle device I / F 7660, and the in-vehicle network I / F 7680. In addition, the microcomputer 7610 may predict dangers such as collision of the vehicle, approach of pedestrians, etc., entering a closed road, etc., based on the obtained information, and generate an alarm signal. The alarm signal may be a signal for generating a warning sound or lighting an alarm lamp.

[0340] The sound / image output unit 7670 transmits an output signal of at least one of sound and image to an output device, which can visually or auditorily notify information to the passengers of the vehicle or the outside of the vehicle. Fig.33 In the example, the audio speaker 7710, the display unit 7720 and the instrument panel 7730 are shown as output devices. The display unit 7720 may include at least one of an on-board display and a head-up display. The display unit 7720 may have an augmented reality (AR) display function. The output device may be other devices other than these devices, such as headphones, wearable devices such as eyeglass displays worn by passengers, projectors, lights, etc. In the case where the output device is a display device, the display device visually displays the results obtained by various processes performed by the microcomputer 7610, or displays information received from other control units in various forms (such as text, images, tables, graphs, etc.). In addition, in the case where the output device is an audio output device, the audio output device converts an audio signal composed of played audio data or sound data into an analog signal and outputs the analog signal in an auditory manner.

[0341] By the way, in Fig.33 In the example shown, at least two control units connected to each other via the communication network 7010 may be integrated into one control unit. Alternatively, each individual control unit may include multiple control units. In addition, the vehicle control system 7000 may include other control units not shown in the figure. In addition, part or all of the functions performed by one of the control units in the above description may be assigned to another control unit. That is, a predetermined operation process may be performed by any control unit as long as the information is transmitted and received via the communication network 7010. Similarly, a sensor or device connected to one of the control units may be connected to another control unit, and multiple control units may transmit and receive detection information to each other via the communication network 7010.

[0342] At the same time, for realizing the respective references according to the implementation mode Figure 1 and Fig.32 The computer program for explaining the functions of the ToF sensor 1 and the ToF sensor 2 can be installed in any control unit. In addition, a computer-readable recording medium storing the computer program can be provided. Examples of the recording medium include a magnetic disk, an optical disk, a magneto-optical disk, and a flash memory. Alternatively, instead of using a recording medium, the computer program can be distributed, for example, via a network.

[0343] In the vehicle control system 7000, the Fig.33 In the integrated control unit 7600 of the application example shown in FIG. 1 , the respective references according to the embodiment are implemented Figure 1 and Fig.32 The ToF sensor 1 and ToF sensor 2 are described. For example, the control unit 11, the control register 12, the adding unit 15, the histogram processing unit 16, the memory 17, the distance estimation / calculation unit 18, and the external I / F 19 in the ToF sensor 1 and the ToF sensor 2 are equivalent to the microcomputer 7610, the storage unit 7690, and the vehicle network I / F 7680 of the integrated control unit 7600. However, this is not the only possible case. Alternatively, the vehicle control system 7000 may be equivalent to Figure 1 The host 80 is shown in FIG.

[0344] At the same time, Fig.33 The modules for integrating the control unit 7600 shown in FIG. 7 (eg, an integrated circuit module using a single die configuration) are implemented in accordance with the embodiments, respectively referring to Figure 1 and Fig.32 At least some of the components of the ToF sensor 1 and ToF sensor 2 described above. Alternatively, a Fig.33 The multiple control units of the vehicle control system 7000 shown in the figure implement the respective references according to the embodiment. Figure 1 and Fig.32 ToF sensor 1 and ToF sensor 2 are illustrated.

[0345] Although the present application is described in detail in the form of embodiments with reference to the accompanying drawings, the technical scope of the present application is not limited to the above embodiments. That is, the present application should be interpreted as embodying all variations that can be thought of by those skilled in the art and fairly fall within the basic teachings set forth herein, such as other embodiments, additions, alternative configurations, and deletions.

[0346] The effects described in the embodiments described in this written description are only illustrative and exemplary, and the scope is not limited. That is, the technology disclosed in this application can achieve other effects that can be thought of by those skilled in the art.

[0347] In addition, the configurations described below also fall within the technical scope of the present application.

[0348] (1) A distance measuring device comprising:

[0349] A control register for maintaining one of the plurality of pixel modes as a set value;

[0350] an array unit including an arrangement of a plurality of light receiving elements, each light receiving element detecting the incidence of a photon;

[0351] a reading unit that reads a detection signal from each light receiving element at a predetermined sampling period;

[0352] a pixel value generating unit that records the number of detection signals read from each of the plurality of light receiving elements in a sampling period in units of pixels and generates a pixel value of each pixel for each sampling period; and

[0353] A memory is used to store, for each pixel, a histogram of pixel values ​​calculated by a pixel value generating unit in each sampling period, wherein:

[0354] Each pixel includes one or more light receiving elements among a plurality of light receiving elements.

[0355] When the first pixel mode is maintained in the control register, a region having a first size in the memory is allocated to each pixel, and

[0356] When the second pixel mode is maintained in the control register, an area in the memory having a second size different from the first size is allocated to each pixel.

[0357] (2) The distance measuring device according to (1), wherein:

[0358] The pixel value generating unit includes:

[0359] an adding unit that records the number of detection signals output from each of the plurality of light receiving elements for a number of light receiving elements equal to a first predetermined count and outputs a first type pixel value, and

[0360] a variable parallel adding unit that outputs a pixel value of each pixel based on the first type of pixel value output from the adding unit,

[0361] When the first pixel mode is maintained in the control register, the variable parallel adding unit outputs a first type of pixel value as a pixel value of each pixel; and

[0362] When the second pixel mode is held in the control register, the variable parallel addition unit outputs a value obtained by adding at least two of the first category pixel values ​​as a pixel value of each pixel.

[0363] (3) The distance measuring device according to (2), wherein:

[0364] grouping the plurality of light receiving elements in the array unit into a plurality of macro pixels consisting of a first predetermined count of light receiving elements,

[0365] grouping the plurality of macro-pixels into a third predetermined count of pixel groups, each of the third predetermined count of pixel groups consisting of a second predetermined count of macro-pixels, and

[0366] When the second pixel mode is maintained in the control register, the variable parallel addition unit calculates the pixel value of each pixel by adding at least two of the first type pixel values, the first type pixel values ​​being obtained by at least two macro pixels belonging to the same pixel group.

[0367] (4) A ranging device according to (3), wherein, when the second pixel mode is maintained in the control register, the variable parallel addition unit outputs pixel values ​​obtained by adding the first category pixel values ​​so that the number of the pixel values ​​is equal to the number of the first category pixel values ​​used in the addition.

[0368] (5) The distance measuring device according to any one of (2) to (4), wherein:

[0369] the memory includes storage areas having a one-to-one correspondence with the light receiving elements in number equal to the first predetermined count,

[0370] When the first pixel mode is maintained in the control register, a histogram of a single pixel is stored in a single storage area, and

[0371] When the second pixel pattern is maintained in the control register, a histogram of a single pixel is stored in at least two storage areas.

[0372] (6) The distance measuring device according to (4), wherein:

[0373] the memory includes storage areas having a one-to-one correspondence with the light receiving elements in number equal to the first predetermined count,

[0374] When the first pixel mode is maintained in the control register, a histogram of a single pixel is stored in a single storage area,

[0375] When the second pixel pattern is maintained in the control register, a histogram of single pixel values ​​is stored in at least two storage areas, and

[0376] Pixel values ​​having the same value output from the variable parallel adding unit are stored in storage areas different from each other in at least two storage areas in which histograms of single pixels are stored.

[0377] (7) The distance measuring device according to (3) or (4), wherein:

[0378] The pixel value generating unit outputs the pixel value of each pixel as a pixel value matrix having a number of elements in a row direction equal to a third predetermined count and a number of elements in a column direction equal to a second predetermined count, and

[0379] The pixel value generating unit further includes a matrix transposition unit for performing matrix transposition on the pixel value matrix.

[0380] (8) The distance measuring device according to (6), wherein:

[0381] the memory includes a first type of memory in an amount equal to a third predetermined count,

[0382] Each of the first type of memories includes a number of storage areas equal to a second predetermined count,

[0383] The pixel value generating unit outputs the pixel value of each pixel as a pixel value matrix having a number of elements in the row direction equal to the third predetermined count and a number of elements in the column direction equal to the second predetermined count, and

[0384] The pixel value generating unit further includes a matrix transposition unit for performing matrix transposition on the pixel value matrix.

[0385] (9) The distance measuring device according to any one of (1) to (8), wherein:

[0386] A single memory address of the memory corresponds to a single bin of the histogram, and

[0387] The distance measuring device further includes a histogram generating unit that creates a histogram by writing the pixel values ​​calculated by the pixel value generating unit in the memory in the order of storage addresses of the memory for each sampling period.

[0388] (10) The distance measuring device according to (9), wherein the histogram generating unit adds the pixel value calculated by the pixel value generating unit to the value stored in the storage address indicating the writing destination for each sampling period.

[0389] (11) The distance measuring device according to (9) or (10), further comprising a light emitting unit, which emits pulsed laser light within a range at least including the field of view angle of the array unit, wherein:

[0390] When the first pixel mode is maintained in the control register, the histogram generation unit creates a histogram using pixel values ​​whose number is equal to the first sampling count, and

[0391] When the second pixel pattern is maintained in the control register, the histogram generation unit creates a histogram using pixel values ​​having a number equal to a second sampling count, the second sampling count being different from the first sampling count.

[0392] (12) The distance measuring device according to any one of (1) to (11), further including a distance estimation / calculation unit that estimates or calculates the distance to the object based on the histogram stored in the memory.

[0393] (13) The distance measuring device according to any one of (1) to (12), wherein the light receiving element is a SPAD (Single Photon Avalanche Diode).

[0394] (14) The distance measuring device according to any one of (1) to (13), wherein:

[0395] The ranging range in the second pixel mode is wider than the ranging range in the first pixel mode, and

[0396] The resolution in the first pixel mode is higher than the resolution in the second pixel mode.

[0397] (15) The distance measuring device according to any one of (1) to (14), wherein the distance measuring device is a flash type in which a plurality of light receiving elements are arranged in a two-dimensional grid.

[0398] (16) The distance measuring device according to any one of (1) to (15), further including a scanning unit that scans the light falling on the array unit.

[0399] (17) A distance measurement method, comprising:

[0400] causing a control register to hold one of a plurality of pixel modes as a set value;

[0401] totaling the number of detection signals read from the plurality of light receiving elements at a predetermined sampling period in units of pixels, and generating a pixel value of each pixel for each sampling period; and

[0402] A histogram of pixel values ​​in each sampling period is stored in a memory for each pixel, where

[0403] Each pixel is composed of one or more light receiving elements among a plurality of light receiving elements.

[0404] When the first pixel mode is maintained in the control register, a region having a first size in the memory is allocated to each pixel, and

[0405] When the second pixel mode is maintained in the control register, an area in the memory having a second size different from the first size is allocated to each pixel.

[0406] Reference Symbols List

[0407] 1.2ToF sensor

[0408] 11Control Unit

[0409] 12 Control Registers

[0410] 13, 213 light emitting unit

[0411] 14, 214 optical receiving unit

[0412] 15Addition Unit

[0413] 16 histogram processing units

[0414] 17 Memory

[0415] 18 distance estimation / calculation unit

[0416] 19External I / F

[0417] 20SPAD pixels

[0418] 21 Photodiode

[0419] 22 Reading circuit

[0420] 23 Quenching resistor

[0421] 24 Select transistor

[0422] 25Digital Converter

[0423] 26 Inverter

[0424] 27 Buffer

[0425] 30 macro pixels

[0426] 31 pixel group

[0427] 480 to 485, 490 to 497 pixel value sequence

[0428] 51, 64 selectors

[0429] 52 Adder (+1)

[0430] 53, 54, 56, 61, 62, 65D-FF circuit

[0431] 55SRAM

[0432] 63 Adder

[0433] 80 Host

[0434] 90 objects

[0435] 141 Timing Control Circuit

[0436] 142 drive circuit

[0437] 143SPAD Array

[0438] 144 Output Circuit

[0439] 151 Pulse Shaping Unit

[0440] 152 light receiving instance counting unit

[0441] 161 Pixel Count Variable Parallel Adder

[0442] 162 Matrix transposition unit

[0443] 163, 1630 to 1637 histogram generation unit

[0444] 170 to 177, 1700 to 1705 memory

[0445] 200 control devices

[0446] 201 focusing lens

[0447] 202 half mirror

[0448] 203 micro mirror

[0449] 204 light receiving lens

[0450] 205 Scanning Unit

[0451] 243 macro pixel sequence

[0452] 251 resistor

[0453] 252, 262 NMOS transistors

[0454] 261PMOS transistor

[0455] AR Scan Area

[0456] SA Images

[0457] L0 Ambient Light

[0458] L1 Laser

[0459] L2 reflected light

[0460] LD pixel drive line

[0461] LS output signal line

[0462] P0 to P2047 sampling number.

Claims

1. A distance measuring device, comprising: a control register for holding one of a plurality of pixel modes as a set value; a light receiving unit including a plurality of SPAD pixels arranged in a two-dimensional array; a pixel value generating unit that aggregates the number of detection signals read from each of the plurality of SPAD pixels in units of macro pixels and generates a pixel value of each macro pixel, wherein each macro pixel includes one or more of the plurality of SPAD pixels; wherein, when the first pixel mode is maintained in the control register, an area having a first size is allocated to each macro pixel; When a second pixel mode is held in the control register, an area having a second size different from the first size is allocated to each macropixel; and When the third pixel mode is held in the control register, a third size having different sizes from the first size and the second size is allocated to each macro pixel.

2. The distance measuring device according to claim 1, wherein: The number of SPAD pixels constituting one macro pixel in the second pixel mode is twice that in the first pixel mode.

3. The distance measuring device according to claim 1, wherein: The number of SPAD pixels constituting one macro pixel in the third pixel mode is four times that in the first pixel mode.

4. The distance measuring device according to claim 1, wherein: The distance measuring device is of a flash type in which a plurality of light receiving cells are arranged in a two-dimensional grid.

5. The distance measuring device according to claim 1, wherein: The ranging range in the second pixel mode is wider than the ranging range in the first pixel mode, and the ranging range in the third pixel mode is wider than the ranging range in the second pixel mode.

6. The distance measuring device according to claim 1, wherein: The resolution in the first pixel mode is higher than the resolution in the second pixel mode, and the resolution in the second pixel mode is higher than the resolution in the third pixel mode.

7. The distance measuring device according to claim 1, wherein: The memory is further configured to store, for each macro pixel, a histogram of the pixel values ​​in each sampling period calculated by the pixel value generating unit. A single memory address of the memory corresponds to a single bin of the histogram, and The distance measuring device further includes a histogram generating unit that creates the histogram by writing the pixel values ​​calculated by the pixel value generating unit in the memory in order of storage addresses of the memory for each sampling period.

Images