Optical detection device, imaging device, and ranging device
Patent Information
- Application Number
- JP2021189721
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2021-11-22
- Publication Date
- 2024-11-01
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Conventional photodetectors using avalanche photodiodes (APDs) are limited by intra-pixel leakage and capacitance variations, restricting the frame rate due to the need for A/D conversion, especially in applications with high readout frequencies.
Incorporating a pulse response section, an analog counter section, and a reset section to generate and reset count values based on pulse signals, allowing for the use of both analog and digital counters to expand the bit depth and reduce frame rate limitations.
This approach enables increased bit depth and reduced frame rate limitations by allowing the analog counter to handle lower bits and the digital counter to handle upper bits, thereby optimizing the readout process.
Abstract
Description
Technical Field
[0001] The present disclosure relates to an optical detection device, an imaging device, and a distance measurement device using an avalanche photodiode (APD).
Background Art
[0002] Conventionally, in an optical detection device using an APD, an analog counter has been provided at the subsequent stage of the APD. However, in such a case, due to pixel leakage and capacitance variation, for example, only a signal of about 7 to 9 bits can be used as the output signal of a 4V analog counter. Therefore, in applications with a large number of read times, there has been a problem that the frame rate is limited due to the time required for A / D conversion. To address this problem, for example, it is conceivable to use a 1-bit digital counter in addition to the analog counter to expand the number of bits of the output signal by 1 bit (see Patent Document 1).
Prior Art Documents
Patent Documents
[0003]
Patent Document 1
Summary of the Invention
Problems to be Solved by the Invention
[0004] However, in the method described in Patent Document 1, the analog value is read multiple times at the read timing, and the limitation of the frame rate cannot be essentially solved. Therefore, it is desirable to provide an optical detection device, an imaging device, and a distance measurement device that can reduce the limitation of the frame rate while using an analog counter.
Means for Solving the Problems
[0005] A photodetector according to the first aspect of this disclosure comprises a pulse response unit, an analog counter unit, and a reset unit. The pulse response unit generates a pulse signal in response to incident light. The analog counter unit generates a first analog count value by performing counting processing based on the pulse signal. The reset unit resets the first count value when the first count value exceeds a first threshold.
[0006] An imaging device relating to a second aspect of this disclosure comprises a plurality of pixels arranged in a matrix, and a signal processing unit that generates image data based on pixel data obtained from each pixel. Each pixel has a pulse response unit, an analog counter unit, and a reset unit. The pulse response unit generates a pulse signal in response to incident light. The analog counter unit generates a first analog count value by performing count processing based on the pulse signal. The reset unit resets the first count value when the first count value exceeds a first threshold.
[0007] A distance measuring device relating to a third aspect of this disclosure comprises a photodetector and a signal processing circuit that calculates the distance from the output signal of the photodetector to an object to be measured. The photodetector includes a pulse response unit, an analog counter unit, and a reset unit. The pulse response unit generates a pulse signal in response to incident light. The analog counter unit generates a first analog count value by performing counting processing based on the pulse signal. The reset unit resets the first count value when the first count value exceeds a first threshold.
[0008] In the photodetector according to the first aspect of this disclosure, the imaging device according to the second aspect of this disclosure, and the distance measuring device according to the third aspect of this disclosure, the count value of the analog counter is reset when the count value of the analog counter exceeds a predetermined threshold. This makes it possible, for example, to count the lower bits with the analog counter and the higher bits with another counter (e.g., a digital counter). [Brief explanation of the drawing]
[0009] [Figure 1] This figure shows an example of a functional block of an imaging device according to the first embodiment of this disclosure. [Figure 2] This figure shows a schematic example of the configuration of a solid-state image sensor. [Figure 3] Figure 2 shows an example of a functional block of pixels and peripheral circuits. [Figure 4] This figure shows an example of the circuit configuration of the detection unit in Figure 3. [Figure 5] This figure shows an example of a signal waveform within the detection unit in Figure 4. [Figure 6] Figures 3 and 4 show an example of the count value output from the pixels. [Figure 7] This diagram illustrates the concept of the signal readout operation from the pixel array section in Figure 2. [Figure 8] This figure shows a modified example of the circuit configuration of the pixels and peripheral circuits shown in Figure 2. [Figure 9] This figure shows a modified example of the circuit configuration of the detection unit in Figure 3. [Figure 10] This figure shows an example of a signal waveform within the detection unit shown in Figure 9. [Figure 11] This figure shows a modified example of the circuit configuration of the detection unit in Figure 3. [Figure 12] Figures 3 and 8 show examples of circuit configurations that can be used to replace the A / D conversion section. [Figure 13] Figures 3 and 8 show examples of circuit configurations that can be used to replace the A / D conversion section. [Figure 14] Figures 4, 9, and 11 show examples of capacitor placement within analog counters. [Figure 15] Figures 4, 9, and 11 show examples of capacitor placement within analog counters. [Figure 16] Figures 4, 9, and 11 show examples of planar configurations of capacitances within analog counters. [Figure 17] Figure 2 shows a modified example of the pixel and peripheral circuit functional block. [Figure 18]FIG. 2 is a diagram showing a modified example of the functional blocks of the pixel and the peripheral circuit. [Figure 19] FIG. 2 is a diagram showing a modified example of the functional blocks of the pixel and the peripheral circuit. [Figure 20] FIG. 19 is a diagram showing an example of the count value output from the pixel. [Figure 21] FIG. 2 is a diagram showing a modified example of the functional blocks of the pixel and the peripheral circuit. [Figure 22] FIG. 21 is a diagram showing an example of the change over time of the count value obtained from the pixel. [Figure 23] FIG. 2 is a diagram showing a modified example of the functional blocks of the pixel and the peripheral circuit. [Figure 24] FIG. 23 is a diagram showing an example of the reference resistance in the OVF control unit. [Figure 25] FIG. 23 is a diagram showing an example of the change over time of the count value obtained from the pixel. [Figure 26] FIG. 1 is a diagram showing how the imaging device is configured in one chip. [Figure 27] FIG. 1 is a diagram showing how the imaging device is configured by stacking two chips. [Figure 28] FIG. 2 is a diagram showing a modified example of the circuit configuration in the pixel array unit. [Figure 29] FIG. 2 is a diagram showing a modified example of the circuit configuration in the pixel array unit. [Figure 30] FIG. is a diagram showing an example of the functional blocks of the distance measuring device according to the second embodiment of the present disclosure. [Figure 31] FIG. 30 is a diagram showing a schematic configuration example of the light detection unit. [Figure 32] FIG. is a block diagram showing an example of the schematic configuration of the vehicle control system. [Figure 33] FIG. is an explanatory diagram showing an example of the installation positions of the external vehicle information detection unit and the imaging unit.
DETAILED DESCRIPTION OF THE INVENTION
[0010] The forms for implementing this disclosure will be described in detail below with reference to the drawings. The explanation will be given in the following order. 1. First Embodiment (Figures 1-7)... Imaging Device 2. Variations (Figures 8-29) 3. Second embodiment (Figures 30, 31)... Distance measuring device 4. Application Examples (Figures 32, 33)
[0011] <1. First Embodiment> [composition] Figure 1 shows an example of the functional block of an imaging device 100 according to the first embodiment of this disclosure. The imaging device 100 is a device that acquires image data by imaging, as shown in Figure 1, and comprises an optical system 110, a solid-state image sensor 120, a control unit 130, and a communication unit 140.
[0012] The optical system 110 focuses incident light and guides it to the solid-state image sensor 120. The solid-state image sensor 120 acquires image data through imaging, and outputs the image data obtained through imaging to the outside via the communication unit 140. The solid-state image sensor 120 has a plurality of pixels 10, which will be described later. The plurality of pixels 10 are arranged in two dimensions in the effective pixel area. The communication unit 140 is an interface for communicating with external devices and outputs the image data obtained by the solid-state image sensor 120 to the external devices.
[0013] The control unit 130 controls the solid-state image sensor 120 to acquire image data through imaging. The control unit 130, for example, simultaneously selects multiple pixels 10 (pixel rows) arranged in a row direction, thereby causing the solid-state image sensor 120 to hold multiple pixel data obtained from the selected pixel rows. The control unit 130 further, for example, sequentially outputs the held multiple pixel data to the communication unit 140. The control unit 130, for example, sequentially selects multiple pixel rows at a predetermined period and sequentially outputs the multiple pixel data obtained therefrom to the communication unit 140. In this way, the control unit 130 outputs the multiple pixel data obtained by the solid-state image sensor 120 as image data from the solid-state image sensor 120 to the communication unit 140.
[0014] Figure 2 shows a schematic example of the solid-state image sensor 120. The solid-state image sensor 120 includes, for example, a pixel array section 121, a row drive section 122, an A / D conversion section 123, a digital count section 124, and an interface section 125, as shown in Figure 2.
[0015] The pixel array unit 121 has a plurality of pixels 10 that perform photoelectric conversion. The plurality of pixels 10 are arranged in a matrix of s × m (s, m are positive integers) in the effective pixel area. In the pixel array unit 121, two horizontal signal lines HSL1 and SHL2 are wired along the row direction for each pixel row, and two vertical signal lines VSL1 and VSL2 are wired along the column direction for each pixel column. The horizontal signal lines HSL1 and SHL2 are wires for controlling the timing of reading signals from the pixels 10. One end of the horizontal signal lines HSL1 and SHL2 is connected to the row drive unit 122. The vertical signal lines VSL1 and VSL2 are wires for reading signals from the pixels 10. One end of the vertical signal line VSL1 is connected to the A / D conversion unit 123, and one end of the vertical signal line VSL2 is connected to the digital count unit 124.
[0016] Each pixel 10 has, for example, an optical pulse response unit 11 and a detection unit 12, as shown in Figure 3.
[0017] (Optical pulse response unit 11) The optical pulse response unit 11 generates a pulse signal in response to incident light. The optical pulse response unit 11 includes, for example, a light receiving unit 11b and a quenching unit 11a, as shown in Figure 3.
[0018] The light-receiving section 11b includes an avalanche photodiode (APD). In a Geiger-mode APD, when a voltage above the breakdown voltage is applied between the terminals, the avalanche phenomenon occurs with the incidence of a single photon. An APD that multiplies a single photon by the avalanche phenomenon is called a single-photon avalanche diode (SPAD). In each pixel 10, the light-receiving section 11b includes, for example, an SPAD. The quench section 11a has the function of stopping the avalanche phenomenon (quenching) by lowering the voltage applied to the light-receiving section 11b to the breakdown voltage. The quench section 11a further has the function of enabling the light-receiving section 11b to detect photons again by making the voltage applied to the light-receiving section 11b a bias voltage above the breakdown voltage. The quench section 11a includes, for example, a MOS transistor. The quench section 11a may also be, for example, a resistor.
[0019] One end of the quenching unit 11a (e.g., the source of the MOS transistor) is connected to a power line to which a fixed voltage Vspad is applied. On the other hand, the other end of the quenching unit 11a (e.g., the drain of the MOS transistor) is connected to one end of the light receiving unit 11b (e.g., the cathode of the SPAD). Figure 3 shows N1 as the connection point (connection node) between the quenching unit 11a and the light receiving unit 11b. The other end of the light receiving unit 11b (e.g., the anode of the SPAD) is connected to a power line to which, for example, a reference voltage Van is applied. The values of the fixed voltage Vspad and the reference voltage Van are set so that a voltage greater than or equal to the breakdown voltage is applied to the light receiving unit 11b. The connection node N1 is connected to the analog counter 12a (described later) of the detection unit 12.
[0020] (Detection unit 12) The detection unit 12 includes, for example, an analog counter 12a, an OVF (overflow) control unit 12b, and switch units 12c and 12d, as shown in Figure 3. The detection unit 12 performs counting based on the pulse signal input from the optical pulse response unit 11. The detection unit 12 outputs an analog count voltage Vout2 and a digital count voltage Vout1 obtained by the counting process. For example, the detection unit 12 outputs the count voltage Vout1 to the vertical signal line VSL1 and the count voltage Vout2 to the vertical signal line VSL2. The count value Cnt2 obtained based on the count voltage Vout2 is used as the low bit side, and the count value Cnt1 obtained based on the count voltage Vout1 is used as the high bit side, and the count value Cnt1 and count value Cnt2 are combined to form the pixel data.
[0021] The analog counter 12a outputs an analog voltage (count voltage Vcnt) corresponding to the number of pulses input from the optical pulse response unit 11. The analog counter 12a is, for example, a countdown counter in which the count voltage Vcnt decreases in steps from the initial voltage each time a pulse signal is input from the optical pulse response unit 11. The analog counter 12a may also be, for example, a count-up counter in which the count voltage Vcnt increases in steps from the initial voltage each time a pulse signal is input from the optical pulse response unit 11. The count voltage Vcnt output from the analog counter 12a is input to the OVF control unit 12b.
[0022] The analog counter 12a resets its output (count voltage Vcnt) based on the inverted signal of the comparison voltage Vcmp input from the OVF control unit 12b. By resetting its output, the analog counter 12a outputs an initial voltage as the count voltage Vcnt.
[0023] The OVF control unit 12b detects whether or not the analog counter 12a is saturated and outputs the result as the saturated voltage Vovf. The OVF control unit 12b outputs the saturated voltage Vovf to the switch unit 12c and also outputs an inverted signal of the comparison voltage Vcmp to the analog counter 12a. When the switch unit 12c is ON, it outputs the saturated voltage Vovf input to the input terminal to the vertical signal line VSL1 via the output terminal. When the switch unit 12c is OFF, it makes the input terminal floating. In other words, the switch unit 12c outputs only the voltage of the saturated voltage Vovf during the period when the switch unit 12c is ON as the count voltage Vout1 to the vertical signal line VSL1. When the switch unit 12d is ON, it outputs the count voltage Vcnt input to the input terminal to the vertical signal line VSL2 via the output terminal. When the switch unit 12d is OFF, it makes the input terminal floating. In other words, the switch unit 12d outputs only the voltage of the count voltage Vcnt during the period when the switch unit 12d is ON as the count voltage Vout2 to the vertical signal line VSL1.
[0024] Next, the circuit configuration of the analog counter 12a and the OVF control unit 12b will be described in detail. Figure 4 shows an example of the circuit configuration of the analog counter 12a and the OVF control unit 12b. Figure 5 shows an example of the change in the signal waveform over time in the detection unit 12.
[0025] (Analog counter 12a) The analog counter 12a has, for example, a charge extraction unit 12-1, a holding unit 12-2, and a reset unit 12-3, as shown in Figure 4.
[0026] The charge extraction unit 12-1 is a circuit that has a capacitor C1 and stores charge in the capacitor C1 according to the voltage of the pulse signal input from the optical pulse response unit 11. Since there is a relationship V = Q / C between the capacitance C of the capacitor C1, the stored charge Q, and the voltage V extracted from the capacitor C1, the analog counter 12a can hold a voltage corresponding to the number of input pulses.
[0027] The holding unit 12-2 is a circuit that has a capacitor C2 and reduces the initial voltage held in capacitor C2 by the amount of the voltage of capacitor C1 held in charge extraction unit 12-1. Each time a pulse signal is input from the optical pulse response unit 11 to the charge extraction unit 12-1, the voltage of capacitor C2 (count voltage Vcnt) decreases in a stepwise manner from the initial voltage.
[0028] The reset unit 12-3 has a switch element, and when an inverted signal of the comparison voltage Vcmp is input from the OVF control unit 12b as a signal to turn on the switch element, it turns on the switch element and sets the voltage of capacitor C2 (count voltage Vcnt) to the initial voltage (power supply voltage Vdd). In other words, when the reset unit 12-3 receives an inverted signal of the comparison voltage Vcmp from the OVF control unit 12b as a signal to turn on the switch element, it resets the count voltage Vcnt. When the reset unit 12-3 receives an inverted signal of the comparison voltage Vcmp from the OVF control unit 12b as a signal to turn off the switch element, it turns off the switch element.
[0029] (OVF control unit 12b) The OVF control unit 12b includes, for example, a comparator, an inverter, an RS flip-flop, and an AND circuit, as shown in Figure 4. The OVF control unit 12b compares the count voltage Vcnt input to the comparator with the reference voltage Vref. As a result, when the count voltage Vcnt falls below the reference voltage Vref (i.e., when the count voltage Vcnt crosses from being above the reference voltage Vref to being below it), the OVF control unit 12b outputs a Hi voltage as the comparison voltage Vcmp to the RS flip-flop and inverter. The inverter outputs an inverted signal of the input comparison voltage Vcmp to the reset unit 12-3. At this time, the OVF control unit 12b resets the count voltage Vcnt. When the count voltage Vcnt exceeds the reference voltage Vref (i.e., when the count voltage Vcnt crosses from being below the reference voltage Vref to being above it), the OVF control unit 12b outputs a Lo voltage as the comparison voltage Vcmp to the RS flip-flop and inverter.
[0030] The RS flip-flop is used as memory. In the RS flip-flop, the S terminal is connected to the output terminal of the comparator, the Q terminal is connected to the input terminal of the switch section 12c and one of the input terminals of the AND circuit, and the R terminal is connected to the output terminal of the AND circuit. In the RS flip-flop, when S = 0 (Lo) → 1 (Hi), Q = 0 (Lo) → 1 (Hi). At this time, the RS flip-flop is in the ON state, and even if S subsequently becomes 1 (Hi) → 0 (Lo), Q remains at 1 (Hi). When the RS flip-flop is in the ON state, if R = 0 (Lo) → 1 (Hi), Q = 1 (Hi) → 0 (Lo). At this time, the RS flip-flop is in the OFF state.
[0031] In the AND circuit, one input terminal is connected to the Q terminal of the RS flip-flop, the other input terminal is connected to the wiring that supplies the control signal Sel1 to turn the switch section 12c on and off, and the output terminal is connected to the R terminal of the RS flip-flop.
[0032] When a comparison voltage Vcmp, which shifts from a low voltage to a high voltage, is input to the S terminal of the RS flip-flop, the RS flip-flop outputs the high voltage as a saturation signal Vovf from the Q terminal. At this time, the count voltage Vcnt is reset in the reset section 12-3, and a comparison voltage Vcmp, which shifts from a high voltage to a low voltage, is input to the S terminal. However, since the RS flip-flop is in the ON state, the RS flip-flop continues to output the high voltage as a saturation signal Vovf from the Q terminal.
[0033] When the control signal Sel1 reaches a voltage that turns on the switch unit 12c (a high voltage), the saturation signal Vovf of the high voltage is output to the vertical signal line VSL1 as the count voltage Vout1. At this time, both input terminals of the AND circuit have a high voltage, so the high voltage is input to the R terminal. As a result, the RS flip-flop turns off and outputs a low voltage as the saturation signal Vovf from the Q terminal. At this time, the voltage on the vertical signal line VSL1 (count voltage Vout1) transitions from a high voltage to a low voltage. Subsequently, when the control signal Sel1 reaches a voltage that turns off the switch unit 12c (a low voltage), the switch unit 12c turns off, and the voltages at the output terminal of the AND circuit and the R terminal of the RS flip-flop transition from high voltage to low voltage.
[0034] (Peripheral circuits) Next, the peripheral circuits of the pixel array section 121 in the solid-state image sensor 120 will be described. The peripheral circuits consist of a row drive section 122, an A / D conversion section 123, a digital count section 124, and an interface section 125.
[0035] The row drive unit 122 controls the timing of reading signals from multiple pixels 10. The row drive unit 122 outputs control signals to multiple horizontal signal lines HSL1 and HSL2 to select multiple pixels 10 for each pixel row. As a result, signals from multiple pixels 10 are sequentially output to multiple vertical signal lines VSL for each pixel row.
[0036] The A / D conversion unit 123 is connected to the vertical signal line VSL2. The vertical signal line VSL2 is connected to the output terminal of the analog counter 12a via the switch unit 12d. The A / D conversion unit 123 performs A / D conversion on the voltage at the output terminal of the analog counter 12a (count voltage Vcnt) via the vertical signal line VSL2, thereby generating a digital count value Cnt2 (nk bits) corresponding to the count voltage Vcnt.
[0037] The digital counting unit 124 is connected to the vertical signal line VSL1. The digital counting unit 124 reads a digital signal (count voltage Vout1) via the vertical signal line VSL1 and generates a digital count value Cnt1 (k bits) by performing a counting operation on the read count voltage Vout1. As shown in Figure 3, the digital counting unit 124 is configured to include a digital adder 124A and a memory array 124B.
[0038] The digital adder 124A initializes the memory array 124B at the start of the exposure period. Then, the digital adder 124A reads the count voltage Vout1 of a certain pixel 10 (hereinafter referred to as "the pixel 10 to be counted") via the vertical signal line VSL1 at predetermined intervals. The digital adder 124A further reads the count voltage Vout1 of the pixel 10 to be counted that was previously written to the memory array 124B from the memory array 124B. The digital adder 124A adds the two read count voltages Vout1 together and writes (overwrites) the resulting value to the memory array 124B as the count voltage Vout1 of the pixel 10 to be counted. The digital adder 124A generates a digital count value Cnt1 (k bits) by repeatedly performing the above addition process (counting process) each time it reads the count voltage Vout1 from the pixel 10 to be counted.
[0039] The interface unit 125 generates a count value Cnt by combining count value Cnt1 (low bit side) and count value Cnt2 (high bit side) (see Figure 6). The interface unit 125 outputs the generated count value Cnt to the outside as pixel data.
[0040] Next, the signal readout operation from the pixel array unit 121 will be described. Figure 7 is a conceptual diagram of an example of the signal readout operation from the pixel array unit 121.
[0041] The control unit 130 controls, for example, the pixel array unit 121 to simultaneously turn on each switch 12c of a plurality (m) of pixels 10 (row lines) arranged in a row direction. As a result, the signal processing unit 122 reads the count voltage Vout1 from each pixel 10 of the row line selected by the control unit 130 and generates a digital count value Cnt1 by performing a counting process on the read count voltage Vout1.
[0042] Here, the count voltage Vout1 is a 1-bit digital data that takes the value of "1" or "0". Suppose, for example, 3 bits are assigned to the count value Cnt1. In this case, the signal processing unit 122 adds the digital count voltage Vout1 to the count value Cnt1. If the count value Cnt1 is at its initial value (000), the signal processing unit 122 adds the read count voltage Vout1 ("1" or "0") to the count value Cnt1, so that the count value Cnt1 becomes 001 or 000.
[0043] The control unit 130 performs the above control for each row line, for example, for all (s) row lines. As a result, the signal processing unit 122 reads multiple (m) count voltages Vout1 from each row line and generates a digital count value Cnt1 by performing a counting process on the read count voltages Vout1. This gives the count value Cnt1 for one frame. The signal processing unit 122 stores the acquired count value Cnt1 for one frame in memory.
[0044] The control unit 130, for example, repeatedly acquires the count value Cnt1 for one frame as described above during the exposure period, and at the timing when the exposure period ends, acquires the count value Cnt2 for one frame at the same time as acquiring the count value Cnt1 for one frame as described above.
[0045] The control unit 130 controls the pixel array unit 121 to simultaneously turn on each switch 12d included in the row line, for example, at the timing when the exposure period ends. As a result, the signal processing unit 122 reads the analog count voltage Vout2 from each pixel 10 of the row line selected by the control unit 130, performs AD conversion on the read count voltage Vout2, and thereby generates a digital count value Cnt2 corresponding to the count voltage Vout2. Here, let's assume that, for example, 4 bits are assigned to the count value Cnt2. In this case, the count value Cnt2 is, for example, 0101.
[0046] The control unit 130 performs the above control for each row line, for example, for all (s) row lines. As a result, the signal processing unit 122 reads multiple (m) count voltages Vout2 from each row line, performs AD conversion on the read count voltages Vout2, and thereby generates a digital count value Cnt2 corresponding to the count voltages Vout2. This gives the count value Cnt2 for one frame. The signal processing unit 122 stores the acquired count value Cnt2 for one frame in memory.
[0047] The signal processing unit 122 uses the count value Cnt1 for one frame read from one memory and the count value Cnt2 for one frame read from the other memory to generate pixel data (i.e., image data) for one frame.
[0048] [effect] Next, the effects of the imaging device 100 will be explained.
[0049] In this embodiment, an OVF control unit 12b is provided downstream of the analog counter 12a, which resets the count voltage Vcnt of the analog counter 12a when the count value cnt1 of the analog counter 12a exceeds a predetermined threshold. This makes it possible, for example, to count the lower bits with the analog counter 12a and the upper bits with the digital counter 122a. As a result, the number of analog value readouts per pixel 10 can be reduced to one, and the number of bits can be increased by the amount of the digital counter 122a provided. Therefore, it is possible to reduce the frame rate limitation while using an analog counter.
[0050] In this embodiment, a digital count value Cnt1 is generated by performing a counting process on the digital signal (count voltage Vout1) output from the OVF control unit 12b. Furthermore, a digital count value Cnt2 is generated by performing an A / D conversion on the analog count voltage Vout2 output from the analog counter 12a. This makes it possible, for example, to count the lower bits with the analog counter 12a and the upper bits with the digital counter 122a. As a result, the number of analog value readouts per pixel 10 can be reduced to one, and the number of bits can be increased by the amount of the digital counter 122a provided. Therefore, the frame rate limitation can be reduced while using an analog counter.
[0051] In this embodiment, the pixel array unit 121 is provided with a vertical signal line VSL1 that transmits the digital count voltage Vout1 to the digital summer 124A, and a vertical signal line VSL2 that transmits the analog count voltage Vout2 to the A / D converter 123. This allows the digital summer 124A and the A / D converter 123 to be located outside the pixel array unit 121, thereby reducing the size of the pixels 10. <2. Variant> Next, a modified example of the imaging device 100 according to the above embodiment will be described.
[0052] [Differentiation A] In the above embodiment, for example, as shown in Figure 8, a digital counter array 124C may be provided instead of the digital adder 124A and the memory array 124B.
[0053] The digital counter array 124C has a digital counter corresponding to each pixel 10. The digital counter array 124C is initialized at the start of the exposure period. Thereafter, the digital counter array 124C reads the count voltage Vout1 (digital count value) of the pixel 10 to be counted via the vertical signal line VSL1 at a predetermined period, and generates a digital count value Cnt1 (k bits) by performing count processing on the read count voltage Vout1. The digital counter array 124C generates a digital count value Cnt1 (k bits) by repeatedly performing the above count processing each time the count voltage Vout1 (digital count value) is read from the pixel 10 to be counted. When the digital counter array 124C is provided, the occupied area of the digital count unit 124 is slightly larger compared to the above embodiment, but in other respects, the same effects as the above embodiment can be obtained.
[0054] [Variation B] In the above embodiment, the OVF control unit 12b may have a digital counter instead of the RSFF and AND circuit, for example, as shown in Figure 9. In this case, the digital counter is a 1-bit digital counter, and for example, as shown in Figure 10, the output of the digital counter (saturation voltage Vovf) changes from "1", "0", "1", etc. each time the comparison voltage Vcmp becomes a high voltage. Even in this case, the same count values Cnt1 and Cnt2 as in the above embodiment can be obtained. Therefore, as in the above embodiment, the frame rate limitation can be reduced while using an analog counter.
[0055] [Differentiation C] In the above embodiment, the OVF control unit 12b may have an inverter instead of a comparator, for example, as shown in Figure 11. In this case, when a count voltage Vcnt below the inverter threshold is input to the inverter, the inverter outputs a Hi voltage as the output signal Vinv to the RS flip-flop and the inverter. Also, when a count voltage Vcnt above the inverter threshold is input to the inverter, the inverter outputs a Lo voltage as the output signal Vinv to the RS flip-flop and the inverter. Even in this case, the same functions as in the above embodiment can be realized. Therefore, as in the above embodiment, the frame rate limitation can be reduced while using an analog counter.
[0056] [Differentiation D] In the above embodiment, the A / D conversion unit 122b may be configured to include a comparator and a counter, for example, as shown in Figure 12. In this case, the comparator receives the voltage of the vertical signal line VSL2 (count voltage Vout2) and the reference voltage Vref_adc as inputs. The comparator compares the count voltage Vout2 and the reference voltage Vref_adc, and outputs a high-level voltage when the count voltage Vout2 is lower than the reference voltage Vref_adc. The comparator outputs a low-level voltage when, for example, the count voltage Vout2 is higher than the reference voltage Vref_adc. The counter counts the number of pulses input from the comparator and thereby generates a count value Cnt2 (nk bits).
[0057] [Differentiation Example E] In the above embodiment, the A / D conversion unit 124c may be configured to include a comparator and a latch, for example, as shown in Figure 13. In this case, the comparator receives the voltage of the vertical signal line VSL2 (count voltage Vout2) and the reference voltage Vref_adc as inputs. The comparator compares the count voltage Vout2 and the reference voltage Vref_adc, and outputs a high-level voltage when the count voltage Vout2 falls below the reference voltage Vref_adc. The comparator outputs a low-level voltage when, for example, the count voltage Vout2 exceeds the reference voltage Vref_adc. The latch records (holds) the time code when the comparator output becomes high-level.
[0058] [Modification F] In the above embodiment and its modified form, the capacitors C1 and C2 of the analog counter 12a may be composed of MIM (Metal-Insulator-Metal). In this case, the capacitors C1 and C2 may be formed on a chip (sensor chip 100A) separate from the logic chip 100B on which the signal processing unit 122 and interface unit 123 are formed, for example, as shown in Figure 14. The sensor chip 100A has a pixel array unit 121 formed on it. The sensor chip 100A and the logic chip 100B are stacked by, for example, bonding Cu pads formed on the sensor chip 100A and Cu pads formed on the logic chip 100B to each other.
[0059] Capacitors C1 and C2 may be formed on the logic chip 100B, for example, as shown in Figure 15. In this case, the portion of each pixel 10 excluding capacitors C1 and C2 may be formed on the sensor chip 100A. By doing so, the sensor chip 100A can be made smaller because it is not necessary to form the area-consuming capacitors C1 and C2 on the sensor chip 100A.
[0060] Figure 16 shows an example of a planar configuration of capacitors C1 and C2. Capacitors C1 and C2 have a comb-like shape, as shown in Figure 16. In this configuration, capacitors C1 and C2 can be formed in the same layer with a capacitance not significantly different from that of capacitors C1 and C2 stacked in the thickness direction.
[0061] [Differentiation G] In the above embodiment and its modified form, each pixel 10 may have a switching unit 13, for example, as shown in Figure 17. The switching unit 13 switches between outputting a saturation signal Vovf to the vertical signal line VSL and outputting a count voltage Vcnt to the vertical signal line VSL. In this case, the peripheral circuit may have a switching unit 126, for example, as shown in Figure 17. The switching unit 126 connects the vertical signal line VSL to the digital summer 124A when the saturation signal Vovf is applied to the vertical signal line VSL, and connects the vertical signal line VSL to the A / D converter 123 when the count voltage Vcnt is applied to the vertical signal line VSL. In this case, the number of vertical signal lines VSL can be reduced, so the area of the pixel array 121 can be reduced.
[0062] [Modified form H] In the above embodiment and its modified form, for example, as shown in Figure 18, the A / D conversion unit 123 may be omitted. In this case, an A / D conversion unit 12e with the same function as the A / D conversion unit 123 may be provided in the detection unit 12 of each pixel 10, and a readout unit 127 for reading the voltage of the vertical signal line VSL2 may be provided in the peripheral circuit. In this case, each pixel 10 can output two digital signals (saturation signal Vovf and count voltage Vcnt) to the vertical signal lines VSL1 and VSL2, thereby realizing an imaging device 100 that is resistant to external noise.
[0063] [Modification I] In the above embodiment and its modified form, the detection unit 12 may have multiple stages of an analog block consisting of a pair of analog counters 12a and an OVF control unit 12b, as shown in Figure 19, for example.
[0064] At this point, in the first-stage analog block, the analog counter 12a generates an analog count value Cnt1 (count voltage Vcnt1). Furthermore, in the first-stage analog block, the OVF control unit 12b outputs a Hi voltage as a saturation signal Vovf when the count voltage Vcnt1 generated by the analog counter 12a falls below the reference voltage Vref (i.e., when the count voltage Vcnt1 crosses from above the reference voltage Vref to below it). The OVF control unit 12b also outputs a Lo voltage as a saturation signal Vovf when, for example, the count voltage Vcnt1 exceeds the reference voltage Vref (i.e., when the count voltage Vcnt1 crosses from above the reference voltage Vref to below it).
[0065] In the second and subsequent analog blocks, the analog counter 12a generates an analog count value Cnt2 (count voltage Vcnt2) by performing counting based on the saturation signal Vovf output from the OVF control unit 12b in the preceding analog block. Furthermore, in the second and subsequent analog blocks, the OVF control unit 12b outputs a Hi voltage as the saturation signal Vovf when the count voltage Vcnt2 generated by the analog counter 12a falls below the reference voltage Vref (i.e., when the count voltage Vcnt2 crosses from above the reference voltage Vref to below it). The OVF control unit 12b also outputs a Lo voltage as the saturation signal Vovf when, for example, the count voltage Vcnt2 exceeds the reference voltage Vref (i.e., when the count voltage Vcnt2 crosses from above the reference voltage Vref to below it).
[0066] In this modified example, the peripheral circuitry includes a digital counting unit 124 and A / D conversion units 123 and 128. The digital summing unit 124A is connected to the vertical signal line VSL1. The vertical signal line VSL1 is connected to the second stage OVF control unit 12b via the switch unit 12c. The digital summing unit 124A reads the second stage saturation signal Vovf via the vertical signal line VSL1 and generates a digital count value Cnt1 (k1 bits) by performing a counting operation on the read saturation signal Vovf.
[0067] The A / D conversion unit 123 is connected to the vertical signal line VSL2. The vertical signal line VSL2 is connected to the output terminal of the first-stage analog counter 12a via the switch unit 12d. The A / D conversion unit 123 performs A / D conversion on the voltage at the output terminal of the first-stage analog counter 12a (count voltage Vcnt2) via the vertical signal line VSL2, thereby generating a digital count value Cnt2 (k2 bits) corresponding to the count voltage Vcnt2.
[0068] The A / D conversion unit 128 is connected to the vertical signal line VSL3. The vertical signal line VSL3 is connected to the output terminal of the second-stage analog counter 12a via the switch unit 12f. The A / D conversion unit 128 performs A / D conversion on the voltage at the output terminal of the second-stage analog counter 12a (count voltage Vcnt3) via the vertical signal line VSL3, thereby generating a digital count value Cnt3 (n-k1-k2 bits) corresponding to the count voltage Vcnt3.
[0069] The interface unit 125 generates the count value Cnt by combining the count values Cnt1, Cnt2, and Cnt3 obtained from the first and second stage analog counters 12a (count values Cnt2, Cnt3) as the low-bit side and the count value obtained from the digital counter 122a (count Cnt1) as the high-bit side (see Figure 20). Note that count value Cnt2 is on the lower bit side than count value Cnt3. The interface unit 125 outputs the generated count value Cnt to the outside as pixel data.
[0070] In this way, by arranging multiple analog blocks consisting of a pair of analog counters 12a and OVF control units 12b, it becomes possible to handle larger numbers of bits. This makes it possible to reduce the frame rate limitations while still using analog counters.
[0071] [Modification J] In the above embodiment and its modified form, the detection unit 12 may have a digital counter 12g after the OVF control unit 12b, for example, as shown in Figure 21. In this case, the digital counting unit 124 is omitted, and a reading unit 129 for reading the voltage of the vertical signal line VSL1 may be provided in the peripheral circuit.
[0072] The digital counter 12g generates a digital count value Cnt1 by performing a counting process on the digital signal output from the OVF control unit 12b. The reading unit 122f reads out the digital count value Cnt1 via the vertical signal line VSL1.
[0073] Figure 22 shows an example of the time-dependent change in the count value Cnt obtained from pixel 10. At low illumination levels, the analog counter 12a may not saturate, and counting may not occur with the digital counter 12g. At medium and high illumination levels, when the analog counter 12a saturates, counting also occurs with the digital counter 12g. At low illumination levels, the time-dependent change in the count value Cnt is gradual, but as the illumination level increases, the time-dependent change in the count value Cnt becomes steeper.
[0074] [Differentiation K] In the modified example J described above, the detection unit 12 may have a determination unit 12h, for example, as shown in Figure 23. The determination unit 12h sets the reference voltage Vref based on the count value Cnt1 generated by the digital counter 12g. The determination unit 12h generates a control signal Scode for setting the reference voltage Vref based on the count value Cnt1, for example, and outputs it to the OVF control unit 12b.
[0075] In the OVF control unit 12b, for example, as shown in Figure 24(A), multiple reference voltage lines are connected to one input terminal of the comparator via a switching unit. The switching unit selects one reference voltage line from the multiple reference voltage lines based on the control signal Scode input from the determination unit 12h.
[0076] In the OVF control unit 12b, for example, as shown in Figure 24(B), multiple constant current sources may be connected to one input terminal of the comparator via a switching unit. In this case, the switching unit selects one reference voltage line from among multiple reference voltage lines based on the control signal Scode input from the determination unit 12h.
[0077] Figure 25 shows an example of the time-dependent change in the count value Cnt obtained from pixel 10. As shown by the dashed lines in Figure 25, multiple thresholds are set for the count value Cnt. At low illumination, the count value Cnt does not reach the lowest threshold, and the reference voltage line to which the lowest reference voltage Vref1 is applied is selected. At medium and high illumination, when the count value Cnt reaches the lowest threshold, the reference voltage line to which a reference voltage Vref2 higher than the lowest reference voltage line Vref1 is selected. Furthermore, when the count value Cnt reaches the second lowest threshold, the reference voltage line to which a reference voltage Vref3 higher than reference voltage Vref2 is applied is selected.
[0078] In this way, by setting the reference voltage value according to the magnitude of the count value Cnt, the time change of the count value Cnt can be kept approximately constant regardless of the illuminance.
[0079] [Modified version L] In the above modified embodiment and its modified form, the solid-state image sensor 120 may be formed on a single semiconductor substrate 100C, for example, as shown in Figure 26. In this case, an effective pixel region is formed on the surface of the semiconductor substrate 100C, in which a plurality of pixels 10 are arranged in a matrix. Around the effective pixel region on the surface of the semiconductor substrate 100C, for example, a row drive unit 122, an A / D conversion unit 123, a digital count unit 124, and an interface unit 125 are mounted.
[0080] [Differentiation M] In the above modified embodiment and its modified form, the solid-state image sensor 120 may be constructed by stacking a plurality of semiconductor substrates (for example, two semiconductor substrates 100D and 100E), as shown in Figure 27. In this case, a plurality of light-receiving portions 11b may be arranged in a matrix on the surface of semiconductor substrate 100D, and a plurality of quench portions 11a may be arranged in a matrix on the surface of semiconductor substrate 100E. The semiconductor substrates 100D and 100E are bonded together, for example, by joining a pad portion 11c made of Cu provided on the semiconductor substrate 100D side and a pad portion 11d made of Cu provided on the semiconductor substrate 100E side. In this case, since the light-receiving portions 11b and quench portions 11a can be stacked on top of each other in each pixel 10, the planar size of the pixel 10 can be reduced compared to the case where the light-receiving portions 11b and quench portions 11a are formed on the surface of a single semiconductor substrate.
[0081] [Differentiation N] In the above modified embodiment and its modified form, for example, as shown in Figure 28, an adder 20 may be connected to the output of multiple pixels 10. The adder 20 adds the count values Cnt input from the multiple pixels 10. This makes it possible to obtain a highly sensitive image.
[0082] [Modification O] In the above modified embodiment and its modified form, for example, as shown in Figure 29, the detection unit 12 may be omitted in each pixel 10. In this case, an adder 20 may be connected to the outputs of a plurality of optical pulse response units 11, and the detection unit 12 may be connected to the output of the adder 20. The adder 20 counts the number of pulses input from the plurality of optical pulse response units 11 and outputs the resulting analog count value to the detection unit 12. Even in this case, a highly sensitive image can be obtained.
[0083] <3. Second Embodiment> [composition] Figure 30 shows an example of the functional block of a distance measuring device 200 according to a second embodiment of the present disclosure. The distance measuring device 200 is a ToF (Time Of Flight) sensor that emits light and detects reflected light reflected by an object to be detected. The distance measuring device 200 comprises a light-emitting unit 210, an optical system 220, a light detection unit 230, a control unit 240, a signal processing unit 250, and a communication unit 260.
[0084] The light-emitting unit 210 emits light pulses L0 toward the object to be detected based on instructions from the control unit 240. The light-emitting unit 210 emits light pulses L0 by performing a light-emitting operation that alternately repeats the cycle of emitting and not emitting light based on instructions from the control unit 240. The light-emitting unit 210 has a light source that emits infrared light, for example. This light source is configured using, for example, a laser light source or an LED (Light Emitting Diode).
[0085] The optical system 220 includes a lens that forms an image on the light-receiving surface of the light detection unit 230. Light pulses (reflected light pulses L1) emitted from the light-emitting unit 210 and reflected by the object to be detected are incident on this optical system 220.
[0086] The light detection unit 230 detects the reflected light pulse L1 based on instructions from the control unit 240. The signal processing unit 250 generates distance image data based on the detection result from the light detection unit 230 and outputs the generated distance image data to the outside via the communication unit 250. The light detection unit 230 has the same configuration as the solid-state image sensor 120 according to the above embodiment and its modified form, for example, as shown in Figure 31.
[0087] The control unit 240 controls the operation of the distance measuring device 200 by supplying control signals to the light-emitting unit 210 and the light-detecting unit 230 and controlling their operation.
[0088] [effect] In this embodiment, similar to the above embodiment, an OVF control unit 12b is provided after the analog counter 12a, which resets the count voltage Vcnt of the analog counter 12a when the count value cnt1 of the analog counter 12a exceeds a predetermined threshold. This makes it possible, for example, to count the lower bits with the analog counter 12a and the upper bits with the digital counter 124b. As a result, the number of analog value readouts per pixel 10 can be reduced to one, and the number of bits can be increased by the amount of the digital counter 124b provided. Therefore, it is possible to reduce the frame rate limitation while using an analog counter.
[0089] In this embodiment, a digital count value Cnt1 is generated by performing a counting process on the digital signal (count voltage Vout1) output from the OVF control unit 12b. Furthermore, a digital count value Cnt2 is generated by performing an A / D conversion on the analog count voltage Vout2 output from the analog counter 12a. This makes it possible, for example, to count the lower bits with the analog counter 12a and the upper bits with the digital counter 124b. As a result, the number of analog value readouts per pixel 10 can be reduced to one, and the number of bits can be increased by the amount of the digital counter 124b provided. Therefore, the frame rate limitation can be reduced while using an analog counter.
[0090] In this embodiment, the pixel array unit 121 is provided with a vertical signal line VSL1 that transmits the digital count voltage Vout1 to the digital summer 124A, and a vertical signal line VSL2 that transmits the analog count voltage Vout2 to the A / D converter 123. This allows the digital summer 124A and the A / D converter 123 to be located outside the pixel array unit 121, thereby reducing the size of the pixels 10.
[0091] <4. Application Examples> The technology disclosed herein can be applied to a variety of products. For example, the technology disclosed herein may be implemented as a device mounted on any type of mobile vehicle, such as automobiles, electric vehicles, hybrid electric vehicles, motorcycles, bicycles, personal mobility devices, airplanes, drones, ships, robots, construction machinery, or agricultural machinery (tractors).
[0092] Figure 32 is a block diagram showing a schematic configuration example of a vehicle control system 7000, which is an example of a mobile control system to which the technology described herein can be applied. The vehicle control system 7000 comprises a plurality of electronic control units connected via a communication network 7010. In the example shown in Figure 32, the vehicle control system 7000 comprises a drive system control unit 7100, a body system control unit 7200, a battery control unit 7300, an external information detection unit 7400, an internal information detection unit 7500, and an integrated control unit 7600. The communication network 7010 connecting these plurality of control units may be an in-vehicle communication network compliant with any standard such as CAN (Controller Area Network), LIN (Local Interconnect Network), LAN (Local Area Network), or FlexRay®.
[0093] Each control unit comprises a microcomputer that performs calculations according to various programs, a storage unit that stores programs executed by the microcomputer or parameters used in various calculations, and a drive circuit that drives various controlled devices. Each control unit is equipped with a network interface for communication with other control units via the communication network 7010, and a communication interface for communication with devices or sensors inside or outside the vehicle via wired or wireless communication. Figure 32 illustrates the functional configuration of the integrated control unit 7600, which includes a microcomputer 7610, a general-purpose communication interface 7620, a dedicated communication interface 7630, a positioning unit 7640, a beacon receiver 7650, an in-vehicle equipment interface 7660, an audio / image output unit 7670, an in-vehicle network interface 7680, and a storage unit 7690. Other control units similarly include a microcomputer, a communication interface, and a storage unit.
[0094] The drivetrain control unit 7100 controls the operation of devices related to the vehicle's drivetrain according to various programs. For example, the drivetrain control unit 7100 functions as a control device for generating driving force for the vehicle, such as an internal combustion engine or a drive motor; a driving force transmission mechanism for transmitting driving force to the wheels; a steering mechanism for adjusting the steering angle of the vehicle; and a braking device for generating braking force for the vehicle. The drivetrain control unit 7100 may also function as a control device such as ABS (Antilock Brake System) or ESC (Electronic Stability Control).
[0095] A vehicle state detection unit 7110 is connected to the drivetrain control unit 7100. The vehicle state detection unit 7110 includes, for example, a gyro sensor for detecting the angular velocity of the vehicle's axial rotational motion, an acceleration sensor for detecting the vehicle's acceleration, or at least one of the sensors for detecting the amount of operation of the accelerator pedal, the amount of operation of the brake pedal, the steering angle of the steering wheel, the engine speed, or the rotational speed of the wheels. The drivetrain control unit 7100 performs calculations using signals input from the vehicle state detection unit 7110 and controls the internal combustion engine, drive motor, electric power steering system, brake system, etc.
[0096] The body system control unit 7200 controls the operation of various devices mounted on the vehicle body according to various programs. For example, the body system control unit 7200 functions as a control device for a keyless entry system, a smart key system, a power window system, or various lamps such as headlights, reverse lights, brake lights, turn signals, or fog lights. In this case, the body system control unit 7200 may receive radio waves transmitted from a portable device that replaces a key or signals from various switches. The body system control unit 7200 receives these radio waves or signals and controls the vehicle's door lock system, power window system, lamps, etc.
[0097] The battery control unit 7300 controls the secondary battery 7310, which is the power source for the drive motor, according to various programs. For example, the battery control unit 7300 receives information such as battery temperature, battery output voltage, or remaining battery capacity from the battery device equipped with the secondary battery 7310. The battery control unit 7300 uses these signals to perform calculations and controls the temperature of the secondary battery 7310 or the cooling device provided in the battery device.
[0098] The external information detection unit 7400 detects information from outside the vehicle equipped with the vehicle control system 7000. For example, at least one of the imaging unit 7410 and the external information detection unit 7420 is connected to the external information detection unit 7400. The imaging unit 7410 includes at least one of the following: a ToF (Time Of Flight) camera, a stereo camera, a monocular camera, an infrared camera, and other cameras. The external information detection unit 7420 includes at least one of the following: an environmental sensor for detecting the current weather or climate, or an ambient information detection sensor for detecting other vehicles, obstacles, or pedestrians around the vehicle equipped with the vehicle control system 7000.
[0099] The environmental sensor may be at least one of the following: a raindrop sensor for detecting rain, a fog sensor for detecting fog, a sunshine sensor for detecting the degree of sunlight, and a snow sensor for detecting snowfall. The ambient information detection sensor may be at least one of the following: an ultrasonic sensor, a radar device, and a LIDAR (Light Detection and Ranging, Laser Imaging Detection and Ranging) device. These imaging unit 7410 and external information detection unit 7420 may be provided as independent sensors or devices, or as a device in which multiple sensors or devices are integrated.
[0100] Here, Figure 33 shows an example of the installation location of the imaging unit 7410 and the external information detection unit 7420. The imaging units 7910, 7912, 7914, 7916, and 7918 are installed, for example, at least one of the following locations on the vehicle 7900: the front nose, side mirrors, rear bumper, back door, and the upper part of the windshield inside the passenger compartment. The imaging unit 7910 installed on the front nose and the imaging unit 7918 installed on the upper part of the windshield inside the passenger compartment mainly acquire images of the front of the vehicle 7900. The imaging units 7912 and 7914 installed on the side mirrors mainly acquire images of the sides of the vehicle 7900. The imaging unit 7916 installed on the rear bumper or back door mainly acquires images of the rear of the vehicle 7900. The imaging unit 7918 installed on the upper part of the windshield inside the passenger compartment is mainly used for detecting preceding vehicles, pedestrians, obstacles, traffic lights, traffic signs, or lanes.
[0101] Figure 33 shows an example of the imaging range of each imaging unit 7910, 7912, 7914, and 7916. Imaging range a shows the imaging range of imaging unit 7910 located on the front nose, imaging ranges b and c show the imaging ranges of imaging units 7912 and 7914 located on the side mirrors, respectively, and imaging range d shows the imaging range of imaging unit 7916 located on the rear bumper or back door. For example, by superimposing the image data captured by imaging units 7910, 7912, 7914, and 7916, an overhead view image of the vehicle 7900 can be obtained.
[0102] The external information detection units 7920, 7922, 7924, 7926, 7928, and 7930, which are installed on the front, rear, sides, corners, and the upper part of the windshield inside the vehicle 7900, may be, for example, ultrasonic sensors or radar devices. The external information detection units 7920, 7926, and 7930, which are installed on the front nose, rear bumper, back door, and the upper part of the windshield inside the vehicle 7900, may be, for example, LIDAR devices. These external information detection units 7920 to 7930 are mainly used to detect preceding vehicles, pedestrians, or obstacles.
[0103] Returning to Figure 32, the explanation continues. The external information detection unit 7400 causes the imaging unit 7410 to capture images of the area outside the vehicle and receives the captured image data. The external information detection unit 7400 also receives detection information from the connected external information detection unit 7420. If the external information detection unit 7420 is an ultrasonic sensor, radar device, or LIDAR device, the external information detection unit 7400 emits ultrasonic waves or electromagnetic waves and receives information on the received reflected waves. Based on the received information, the external information detection unit 7400 may perform object detection processing such as detecting people, vehicles, obstacles, signs, or characters on the road surface, or distance detection processing. Based on the received information, the external information detection unit 7400 may perform environmental recognition processing to recognize rainfall, fog, or road surface conditions. Based on the received information, the external information detection unit 7400 may calculate the distance to an object outside the vehicle.
[0104] Furthermore, the external information detection unit 7400 may perform image recognition processing or distance detection processing to recognize people, vehicles, obstacles, signs, or characters on the road surface based on the received image data. The external information detection unit 7400 may perform distortion correction or alignment processing on the received image data, and may also synthesize image data captured by different imaging units 7410 to generate an overhead view image or a panoramic image. The external information detection unit 7400 may also perform viewpoint transformation processing using image data captured by different imaging units 7410.
[0105] The in-vehicle information detection unit 7500 detects information inside the vehicle. The in-vehicle information detection unit 7500 is connected to, for example, a driver status detection unit 7510 that detects the driver's state. The driver status detection unit 7510 may include a camera that images the driver, a biosensor that detects the driver's biometric information, or a microphone that collects sounds inside the vehicle. The biosensor is installed, for example, on the seat or steering wheel and detects the biometric information of a passenger sitting in the seat or a driver holding the steering wheel. Based on the detection information input from the driver status detection unit 7510, the in-vehicle information detection unit 7500 may calculate the driver's level of fatigue or concentration, or determine whether the driver is dozing off. The in-vehicle information detection unit 7500 may perform processing such as noise cancellation on the collected audio signals.
[0106] The integrated control unit 7600 controls the overall operation of the vehicle control system 7000 according to various programs. An input unit 7800 is connected to the integrated control unit 7600. The input unit 7800 is implemented by a device that can be operated by the passenger, such as a touch panel, buttons, a microphone, a switch, or a lever. The integrated control unit 7600 may also receive data obtained by voice recognition of voice input from the microphone. The input unit 7800 may be a remote control device using infrared or other radio waves, or an external device such as a mobile phone or PDA (Personal Digital Assistant) that is compatible with the operation of the vehicle control system 7000. The input unit 7800 may be a camera, in which case the passenger can input information by gesture. Alternatively, data obtained by detecting the movement of a wearable device worn by the passenger may be input. Furthermore, the input unit 7800 may include, for example, an input control circuit that generates an input signal based on the information input by the passenger using the above input unit 7800 and outputs it to the integrated control unit 7600. Passengers and others can input various data or instruct the vehicle control system 7000 to perform processing operations by operating this input unit 7800.
[0107] The memory unit 7690 may include a ROM (Read Only Memory) for storing various programs executed by a microcomputer, and a RAM (Random Access Memory) for storing various parameters, calculation results, or sensor values. The memory unit 7690 may also be implemented using a magnetic storage device such as an HDD (Hard Disk Drive), a semiconductor storage device, an optical storage device, or a magneto-optical storage device.
[0108] The general-purpose communication interface 7620 is a general-purpose communication interface that mediates communication between the vehicle and various devices present in the external environment 7750. The general-purpose communication interface 7620 may implement cellular communication protocols such as GSM (Global System of Mobile communications), WiMAX (registered trademark), LTE (registered trademark) (Long Term Evolution), or LTE-A (LTE-Advanced), or other wireless communication protocols such as wireless LAN (also known as Wi-Fi (registered trademark)) or Bluetooth (registered trademark). The general-purpose communication interface 7620 may connect to devices (e.g., application servers or control servers) located on an external network (e.g., the Internet, a cloud network, or a carrier-specific network) via, for example, a base station or access point. The general-purpose communication interface 7620 may also connect to terminals located near the vehicle (e.g., terminals for drivers, pedestrians, or shops, or MTC (Machine Type Communication) terminals) using, for example, P2P (Peer To Peer) technology.
[0109] The Dedicated Communication I / F 7630 is a communication interface that supports communication protocols developed for use in vehicles. The Dedicated Communication I / F 7630 may implement standard protocols such as WAVE (Wireless Access in Vehicle Environment), DSRC (Dedicated Short Range Communications), or cellular communication protocols, which are combinations of lower-layer IEEE 802.11p and upper-layer IEEE 1609. The Dedicated Communication I / F 7630 typically performs V2X communication, a concept that includes one or more of the following: vehicle-to-vehicle communication, vehicle-to-infrastructure communication, vehicle-to-home communication, and vehicle-to-pedestrian communication.
[0110] The positioning unit 7640 performs positioning by receiving GNSS signals from GNSS (Global Navigation Satellite System) satellites (for example, GPS signals from GPS (Global Positioning System) satellites) and generates location information including the vehicle's latitude, longitude, and altitude. The positioning unit 7640 may also determine its current location by exchanging signals with a wireless access point, or it may acquire location information from a terminal such as a mobile phone, PHS, or smartphone that has a positioning function.
[0111] The beacon receiver 7650 receives radio waves or electromagnetic waves transmitted from, for example, a radio station installed on a road, and obtains information such as the current location, traffic congestion, road closures, or travel time. The functions of the beacon receiver 7650 may also be included in the dedicated communication interface 7630 described above.
[0112] The In-Vehicle Equipment I / F 7660 is a communication interface that mediates connections between the microcomputer 7610 and various in-vehicle equipment 7760 located inside the vehicle. The In-Vehicle Equipment I / F 7660 may establish a wireless connection using wireless communication protocols such as Wi-Fi, Bluetooth®, NFC (Near Field Communication), or WUSB (Wireless USB). Furthermore, the in-vehicle equipment I / F 7660 may establish a wired connection such as USB (Universal Serial Bus), HDMI (Registered Trademark) (High-Definition Multimedia Interface), or MHL (Mobile High-Definition Link) via connection terminals (and, if necessary, cables) not shown. The in-vehicle equipment 7760 may include, for example, at least one of the following: a mobile device or wearable device owned by a passenger, or an information device brought into or installed in the vehicle. The in-vehicle equipment 7760 may also include a navigation device that performs route searching to any destination. The in-vehicle equipment I / F 7660 exchanges control signals or data signals with these in-vehicle equipment 7760s.
[0113] The in-vehicle network interface 7680 is an interface that mediates communication between the microcomputer 7610 and the communication network 7010. The in-vehicle network interface 7680 transmits and receives signals and other data in accordance with a predetermined protocol supported by the communication network 7010.
[0114] The microcomputer 7610 of the integrated control unit 7600 controls the vehicle control system 7000 according to various programs based on information acquired via at least one of the general-purpose communication I / F 7620, dedicated communication I / F 7630, positioning unit 7640, beacon receiver 7650, in-vehicle equipment I / F 7660, and in-vehicle network I / F 7680. For example, the microcomputer 7610 may calculate control target values for the drive force generator, steering mechanism, or braking device based on acquired in-vehicle and out-of-vehicle information and output control commands to the drive system control unit 7100. For example, the microcomputer 7610 may perform coordinated control aimed at realizing ADAS (Advanced Driver Assistance System) functions, including vehicle collision avoidance or impact mitigation, following driving based on distance between vehicles, maintaining vehicle speed, vehicle collision warning, or vehicle lane departure warning. Furthermore, the microcomputer 7610 may perform cooperative control for purposes such as autonomous driving, where the vehicle drives autonomously without driver intervention, by controlling the drive force generating device, steering mechanism, or braking device, etc., based on the acquired information about the vehicle's surroundings.
[0115] The microcomputer 7610 may generate three-dimensional distance information between the vehicle and surrounding structures, people, and other objects based on information acquired via at least one of the general-purpose communication I / F 7620, dedicated communication I / F 7630, positioning unit 7640, beacon receiver 7650, in-vehicle equipment I / F 7660, and in-vehicle network I / F 7680, and create local map information including surrounding information of the vehicle's current location. Furthermore, the microcomputer 7610 may predict dangers such as vehicle collision, proximity of pedestrians, or entry into a closed road based on the acquired information, and generate a warning signal. The warning signal may, for example, be a signal to generate a warning sound or illuminate a warning lamp.
[0116] The audio-image output unit 7670 transmits at least one of audio and image output signals to an output device capable of visually or audibly notifying the vehicle's occupants or those outside the vehicle. In the example in Figure 32, the output devices are exemplified as an audio speaker 7710, a display unit 7720, and an instrument panel 7730. The display unit 7720 may include, for example, at least one of an onboard display and a head-up display. The display unit 7720 may also have an AR (Augmented Reality) display function. The output device may be other devices besides these, such as headphones, wearable devices such as glasses-type displays worn by occupants, projectors, or lamps. If the output device is a display device, the display device visually displays the results obtained from various processes performed by the microcomputer 7610 or information received from other control units in various formats such as text, images, tables, and graphs. If the output device is an audio output device, the audio output device converts the audio signal, consisting of reproduced audio data or sound data, into an analog signal and outputs it audibly.
[0117] In the example shown in Figure 32, at least two control units connected via the communication network 7010 may be integrated into a single control unit. Alternatively, each control unit may be composed of multiple control units. Furthermore, the vehicle control system 7000 may include other control units not shown. Also, in the above description, some or all of the functions performed by one control unit may be assigned to other control units. In other words, as long as information is transmitted and received via the communication network 7010, predetermined calculation processing may be performed by any of the control units. Similarly, a sensor or device connected to one control unit may be connected to another control unit, and multiple control units may transmit and receive detection information to each other via the communication network 7010.
[0118] Furthermore, computer programs for realizing the functions of the imaging device 100, as described using Figures 1 to 29, and the functions of the distance measuring device 200, as described using Figures 30 and 31, can be implemented in any of the control units. A computer-readable recording medium containing such computer programs can also be provided. Examples of recording media include magnetic disks, optical disks, magneto-optical disks, and flash memory. Alternatively, the computer programs may be distributed via a network, for example, without using a recording medium.
[0119] In the vehicle control system 7000 described above, the imaging device 100 described using Figures 1 to 29, etc., and the distance measuring device 200 described using Figures 30, 31, etc., can be used, for example, as the light source steering unit of a LIDAR as an environmental sensor. Furthermore, image recognition in the imaging unit can also be performed using an optical computing unit that utilizes the imaging device 100 described using Figures 1 to 29, etc., and the distance measuring device 200 described using Figures 30, 31, etc. When the imaging device 100 described using Figures 1 to 29, etc., and the distance measuring device 200 described using Figures 30, 31, etc., are used as high-efficiency, high-brightness projection devices, lines and characters can be projected onto the ground. Specifically, lines can be displayed so that people outside the vehicle can see the vehicle's position when it is reversing, or a crosswalk can be displayed with light when yielding to pedestrians.
[0120] Furthermore, at least some of the components of the imaging device 100 described using Figures 1 to 29 may be implemented in a module for the integrated control unit 7600 shown in Figure 32 (for example, an integrated circuit module consisting of a single die). Alternatively, the imaging device 100 described using Figures 1 to 29 may be implemented by multiple control units of the vehicle control system 7000 shown in Figure 32.
[0121] Furthermore, at least some of the components of the distance measuring device 200 described using Figures 30, 31, etc., may be implemented in a module for the integrated control unit 7600 shown in Figure 32 (for example, an integrated circuit module consisting of a single die). Alternatively, the distance measuring device 200 described using Figures 30, 31, etc., may be implemented by multiple control units of the vehicle control system 7000 shown in Figure 32.
[0122] The present disclosure has been described above with reference to embodiments, modifications thereof, and application examples. However, the present disclosure is not limited to the above embodiments, and various modifications are possible. The effects described herein are merely illustrative. The effects of the present disclosure are not limited to those described herein. The present disclosure may have effects other than those described herein.
[0123] Furthermore, for example, this disclosure can take the following configuration. (1) A pulse response unit that generates a pulse signal in response to light incidence, An analog counter unit that generates an analog first count value by performing counting processing based on the pulse signal, A reset unit that resets the first count value when the first count value exceeds a first threshold, Equipped with Light detection device. (2) The reset unit outputs a digital signal indicating when the first count value exceeds the first threshold. The light detection device is A first digital counter generates a second digital count value by performing a counting process on the digital signal output from the reset unit, An A / D conversion unit generates a third digital count value by performing A / D conversion on the first count value. It is further equipped with (1) The light detection device described above. (3) The system further includes a switch that controls the output of the digital signal to the first digital counter and the output of the first count value to the A / D conversion unit by separate control signals. The light detection device described in (1) or (2). (4) The reset unit holds the digital signal when the first count value exceeds the first threshold, and resets the first count value when the digital signal is output. (3) The light detection device described above. (5) The reset unit includes a second digital counter that holds and outputs the digital signal. (3) The light detection device described above. (6) The first digital counter has a digital adder and a memory, The digital adder adds the digital signal output from the reset unit and the count value read from the memory, and overwrites the count value in the memory with the resulting value as the second count value. (2) The light detection device described above. (7) The pulse response unit includes an avalanche photodiode and a quench circuit connected in series with the avalanche photodiode. A light detection device as described in any one of (1) to (6). (8) The device further includes an output unit that generates and outputs a fourth count value by combining the third count value and the second count value, with the third count value on the low-bit side and the second count value on the high-bit side. (2) The light detection device described above. (9) Multiple analog blocks are provided, each consisting of a set of the aforementioned analog counter unit and reset unit. In the first analog block, The reset unit outputs a digital signal indicating that the first count value generated by the analog counter unit exceeds the first threshold value. In the analog blocks from the second stage onward, The analog counter unit generates a fifth analog count value by performing counting processing based on the digital signal output from the reset unit in the preceding analog block. The reset unit outputs a digital signal indicating that the fifth count value input to the reset unit exceeds the second threshold value. A light detection device as described in any one of (2) to (8). (10) The system further includes a threshold setting unit that sets the first threshold based on the second count value generated by the digital counter. A light detection device as described in any one of (2) to (8). (11) Multiple pixels arranged in a matrix, A signal processing unit that generates image data based on pixel data obtained from each of the aforementioned pixels. Equipped with, Each of the aforementioned pixels is A pulse response unit that generates a pulse signal in response to light incidence, An analog counter unit that generates an analog first count value by performing counting processing based on the pulse signal, A reset unit that resets the first count value when the first count value exceeds a first threshold, has Imaging device. (12) The reset unit outputs a digital signal indicating when the first count value exceeds the first threshold. The signal processing unit, A digital counter that generates a second digital count value by performing a counting process on the digital signal output from the reset unit, An A / D conversion unit generates a third digital count value by performing A / D conversion on the first count value. has The imaging device described in (11). (13) The system further includes a switch that controls the output of the digital signal to the first digital counter and the output of the first count value to the A / D conversion unit by separate control signals. The imaging device described in (11). (14) The reset unit holds the digital signal when the first count value exceeds the first threshold, and resets the first count value when the digital signal is output. The imaging device described in (13). (15) The reset unit includes a second digital counter that holds and outputs the digital signal. The imaging device described in (13). (16) A first wiring that transmits the digital signal to the digital counter, A second wiring that transmits the first count value to the A / D conversion unit and It is further equipped with The imaging device described in any one of (11) to (15). (17) Each of the aforementioned pixels further includes a first switching unit that switches between the output of the digital signal and the output of the first count value, The imaging device further includes a second switching unit that inputs the output of the switching unit to either the digital counter or the A / D conversion unit. The imaging device described in any one of (11) to (16). (18) Each of the aforementioned pixels has multiple stages of analog blocks, each consisting of a set of analog counter units and reset units. In the first analog block, The reset unit outputs a digital signal indicating that the first count value generated by the analog counter unit exceeds the first threshold value. In the analog blocks from the second stage onward, The analog counter unit generates a fifth analog count value by performing counting processing based on the digital signal output from the reset unit in the preceding analog block. The reset unit outputs a digital signal indicating that the fifth count value input to the reset unit exceeds the second threshold value. The imaging device described in any one of (11) to (17). (19) The reset unit outputs a digital signal indicating when the first count value exceeds the first threshold. Each of the aforementioned pixels further includes a digital counter that generates a second digital count value by performing a counting operation on the digital signal output from the reset unit. The imaging device described in (11). (20) The system further includes a threshold setting unit that sets the first threshold based on the second count value generated by the digital counter. The imaging device described in (19). (twenty one) The pulse response unit includes an avalanche photodiode and a quench circuit connected in series with the avalanche photodiode. A light detection device as described in any one of (11) to (20). (twenty two) The capacitance of the aforementioned analog counter section is composed of MIM (Metal-Insulator-Metal). The imaging device described in any one of (11) to (21). (twenty three) Among the analog counters, a first chip having a capacitance formed by the MIM, The pulse response unit and the second chip on which the circuits of the analog counter, excluding the MIM, are formed are Prepared The imaging device described in (22). (twenty four) Light detection device and A signal processing circuit that calculates the distance from the output signal of the light detector to the object to be measured. Equipped with, The aforementioned light detection device is A pulse response unit that generates a pulse signal in response to light incidence, An analog counter unit that generates an analog first count value by performing counting processing based on the pulse signal, A reset unit that resets the first count value when the first count value exceeds a first threshold, has Ranging device.
[0124] In the photodetector according to the first aspect of this disclosure, the imaging device according to the second aspect of this disclosure, and the distance measuring device according to the third aspect of this disclosure, a reset of the count value of the analog counter unit occurs when the count value of the analog counter unit exceeds a predetermined threshold. This makes it possible, for example, to count the lower bits with the analog counter unit and the higher bits with another counter (e.g., a digital counter). In this way, by using a multi-stage counter, it is possible to reduce the frame rate limitations caused by the analog counter while still using an analog counter. [Explanation of symbols]
[0125] 10,10i…Pixel, 11…Optical pulse response unit, 11a…Quench unit, 11b…Light receiving unit, 11c,11d…Pad unit, 12…Detection unit, 12-1…Charge extraction unit, 12-2…Holding unit, 12-3…Reset unit, 12a…Analog counter, 12b…OVF control unit, 12c,12d,12f…Switch unit, 12e…A / D conversion unit, 12g…Digital counter, 12h…Determination unit, 13…Switching unit, 20…Addition unit, 100…Imaging device, 100A…Sensor chip, 100B…Logic chip, 100C,100D,100E…Semiconductor substrate, 110…Optical system, 120…Solid-state image sensor, 121…Pixel array unit, 122…Row drive unit, 123…A / D conversion unit, 124…Digital count unit, 124A…Digital Adding unit, 124B...memory array, 124C...digital counter array, 125...interface unit, 126...switching unit, 127...reading unit, 128...A / D conversion unit, 129...reading unit, 130...control unit, 140...communication unit, 200...distance measuring device, 210...light emission unit, 220...optical system, 230...light detection unit, 240...control unit, 250...signal processing unit, 260...communication unit, C1, C2...capacitance, HSL, HSL1, HSL2, HSL3...horizontal signal line, L0...light pulse, L1...reflected light pulse, N1, N2, N3...connection node, Vcnt...count voltage, Vout1, Vout2...count voltage, Vref, Vref1, Vref2, Vref3...reference voltage, VSL1, VSL2, VSL3...vertical signal line.
Claims
1. A pulse response unit that generates a pulse signal in response to incident light; an analog counter unit that generates an analog first count value by performing count processing based on the pulse signal; a reset unit that resets the first count value when the first count value exceeds a first threshold value; Equipped with Light detection device.
2. the reset unit outputs a digital signal indicating that the first count value exceeds the first threshold value, The light detection device comprises: a first digital counter that generates a second digital count value by performing a count process on the digital signal output from the reset unit; an A / D conversion unit that generates a third digital count value by A / D converting the first count value; Further equipped 2. The optical detection device according to claim 1.
3. The output of the digital signal to the first digital counter and the output of the first count value to the A / D converter are controlled by different control signals.
3. The optical detection device according to claim 2.
4. The reset unit holds the digital signal when the first count value exceeds the first threshold value, and resets the first count value when the digital signal is output.
4. The optical detection device according to claim 3.
5. The reset unit includes a second digital counter that holds and outputs the digital signal.
4. The optical detection device according to claim 3.
6. the first digital counter includes a digital adder and a memory; The digital addition unit adds the digital signal output from the reset unit and the count value read from the memory, and overwrites the count value in the memory with the value obtained by adding the digital signal output from the reset unit and the count value read from the memory as the second count value.
3. The optical detection device according to claim 2.
7. The pulse response section includes an avalanche photodiode and a quench circuit connected in series to the avalanche photodiode.
2. The optical detection device according to claim 1.
8. A plurality of pixels arranged in a matrix; a signal processing unit that generates image data based on pixel data obtained from each of the pixels; Equipped with Each of the pixels is A pulse response unit that generates a pulse signal in response to incident light; an analog counter unit that generates an analog first count value by performing count processing based on the pulse signal; a reset unit that resets the first count value when the first count value exceeds a first threshold value; have Imaging device.
9. the reset unit outputs a digital signal indicating that the first count value exceeds the first threshold value, The signal processing unit includes: a first digital counter that generates a second digital count value by performing a count process on the digital signal output from the reset unit; an A / D conversion unit that generates a third digital count value by A / D converting the first count value; have The imaging device according to claim 8.
10. The output of the digital signal to the first digital counter and the output of the first count value to the A / D converter are controlled by different control signals. The imaging device according to claim 9.
11. The reset unit holds the digital signal when the first count value exceeds the first threshold value, and resets the first count value when the digital signal is output. The imaging device according to claim 10.
12. The reset unit includes a second digital counter that holds and outputs the digital signal. The imaging device according to claim 10.
13. a first line for transmitting the digital signal to the first digital counter; a second wiring for transmitting the first count value to the A / D conversion unit; Further equipped The imaging device according to claim 9.
14. Each of the pixels further includes a first switching unit that switches between outputting the digital signal and outputting the first count value, The imaging device further includes a second switching unit that inputs an output of the first switching unit to either the first digital counter or the A / D conversion unit. The imaging device according to claim 9.
15. Each of the pixels has a set of the analog counter unit and the reset unit in a plurality of stages as an analog block, In the first stage analog block, the reset unit outputs a digital signal indicating that the first count value generated by the analog counter unit exceeds the first threshold value, In the analog blocks in the second and subsequent stages, the analog counter unit generates an analog fifth count value by performing a count process based on the digital signal output from the reset unit in the analog block at a previous stage; When the fifth count value input to the reset unit exceeds a second threshold, the reset unit outputs a digital signal indicating that. The imaging device according to claim 8.
16. the reset unit outputs a digital signal indicating that the first count value exceeds the first threshold value, Each of the pixels further includes a first digital counter that generates a second digital count value by performing a count process on the digital signal output from the reset unit. The imaging device according to claim 8.
17. The digital counter further includes a threshold setting unit that sets the first threshold based on the second count value generated by the first digital counter. The imaging device according to claim 16.
18. The pulse response section includes an avalanche photodiode and a quench circuit connected in series to the avalanche photodiode. The imaging device according to claim 8.
19. A photodetector; a signal processing circuit that calculates the distance to the object to be measured from the output signal of the light detection device; Equipped with The light detection device includes: A pulse response unit that generates a pulse signal in response to incident light; an analog counter unit that generates an analog first count value by performing count processing based on the pulse signal; a reset unit that resets the first count value when the first count value exceeds a first threshold value; have Ranging device.
