Solid-state imaging device, imaging apparatus, and control method for solid-state imaging device

Abstract

A solid-state imaging device, an imaging apparatus, and a control method of a solid-state imaging device are disclosed. In a solid-state imaging device that detects an address event, the present invention controls the detection sensitivity of the address event to an appropriate value. This solid-state imaging device is provided with a pixel array unit and a control unit. In the solid-state imaging device, a plurality of pixel circuits are arranged in the pixel array unit, each of which detects a change in luminance of incident light occurring outside a predetermined dead zone as an address event. The control unit controls the width of the dead zone in accordance with the number of times that a detection address event has been detected in the pixel array unit within a given unit period.

Description

[0001] This application is a divisional application. Its parent application has the application number 2018800875809, the application date is December 14, 2018, and the invention title is "Solid-state imaging element, imaging device and control method of solid-state imaging element". Technical Field

[0002] This technology relates to a solid-state imaging element, an imaging device, and a method for controlling the solid-state imaging element. Specifically, this technology relates to a solid-state imaging element for detecting brightness changes, an imaging device, and a method for controlling the solid-state imaging element. Background Technology

[0003] Traditionally, synchronous solid-state imaging elements have been used in imaging devices to capture image data (frames) in sync with a synchronization signal such as a vertical synchronization signal. Using such a general synchronous solid-state imaging element, image data can be acquired only at each synchronization signal cycle (e.g., 1 / 60th of a second). Therefore, it is difficult to handle the requirements for higher-speed processing in fields such as transportation and robotics. Against this backdrop, asynchronous solid-state imaging elements that detect brightness changes as address events in real time for each pixel address have been proposed (see, for example, Patent Document 1). Such a solid-state imaging element that detects address events for each pixel is called a dynamic vision sensor (DVS).

[0004] Reference List

[0005] Patent documents

[0006] Patent document 1: Japanese Patent Application Publication No. 2016-501495 Summary of the Invention

[0007] The problem to be solved by the present invention

[0008] The aforementioned asynchronous solid-state imaging element (i.e., DVS) can generate and output data at a much higher speed than synchronous solid-state imaging elements. Therefore, for example, in the transportation sector, image processing for recognizing people or obstacles can be performed at high speed, improving safety. However, in the aforementioned solid-state imaging element, it is difficult to control the detection sensitivity for address events to an appropriate value. For example, if the detection sensitivity for address events is too low, obstacle detection may fail in image recognition. On the other hand, if the detection sensitivity for address events is too high, address events may be over-detected when the brightness of all pixels changes due to variations in brightness, etc., and power consumption may increase.

[0009] Given this situation, this technique has been developed, and the purpose of this technique is to control the detection sensitivity for address events to an appropriate value in a solid-state imaging element for detecting address events.

[0010] Solution to the problem

[0011] This technology was developed to address the aforementioned problems, and its first aspect is a solid-state imaging element and a method for controlling the solid-state imaging element, the solid-state imaging element comprising: a pixel array unit in which a plurality of pixel circuits are arranged, each pixel circuit detecting a brightness change of incident light occurring outside a predetermined dead band as an address event; and a control unit controlling the width of the dead band according to the number of times the address event is detected in the pixel array unit within a fixed unit period. This results in the effect that address events are detected outside a dead band having a width corresponding to the number of detections.

[0012] Furthermore, in the first aspect, the control unit can widen the dead zone as the number of detections increases. This results in address events being detected outside the wider dead zone as the number of detections increases.

[0013] Furthermore, in the first aspect, each pixel circuit in the plurality of pixel circuits can compare each of the upper and lower limits of the dead zone with the amount of brightness change, and detect address events based on the comparison results. The effect of this is that address events are detected based on the comparison results between each of the upper and lower limits of the dead zone and the amount of brightness change.

[0014] Furthermore, in the first aspect, if the number of detections exceeds a predetermined allowable range, the control unit can control the width of the dead zone. This results in the detection of address events outside a dead zone with a width corresponding to the number of detections.

[0015] Furthermore, in the first aspect, the pixel array unit can be divided into multiple regions, and the control unit can control the width of the dead zone for each of the multiple regions. This results in the effect that address events are detected outside the dead zone, which has a width controlled for each region.

[0016] Furthermore, in the first aspect, each of the plurality of pixel circuits has a photoelectric conversion element and a current-to-voltage conversion circuit. The photoelectric conversion element performs photoelectric conversion on the incident light to generate a photocurrent, and the current-to-voltage conversion circuit converts the photocurrent into a voltage. The photoelectric conversion element can be arranged on a light receiving chip, and the current-to-voltage conversion circuit can be arranged on a detection chip laminated on the light receiving chip. This results in the detection of address events by circuits distributed across each of the light receiving chip and the detection chip.

[0017] Furthermore, a second aspect of this technology is an imaging apparatus comprising: a pixel array unit in which a plurality of pixel circuits are arranged, each pixel circuit detecting brightness changes of incident light occurring outside a predetermined dead zone as address events; a control unit controlling the width of the dead zone based on the number of times an address event is detected in the pixel array unit within a fixed unit period; and a recording unit recording data obtained from the detection results of the address events. This results in the detection of address events outside a dead zone with a width corresponding to the number of detections, and the recording of data obtained from the detection results.

[0018] Effects of the present invention

[0019] According to this technology, in solid-state imaging elements for detecting address events, excellent results can be obtained in controlling the detection sensitivity for address events to an appropriate value. Note that this is not necessarily limited to the effects described herein, and can be any of the effects described in this disclosure. Attached Figure Description

[0020] [ Figure 1 [ ] is a block diagram illustrating a configuration example of an imaging apparatus according to a first embodiment of the present technology.

[0021] [ Figure 2 [Illustration] is an example of a laminated structure of a solid-state imaging element according to a first embodiment of the present technology.

[0022] [ Figure 3 [Illustration 1] is a block diagram illustrating a configuration example of a solid-state imaging element according to a first embodiment of the present technology.

[0023] [ Figure 4 [Illustration 1] is a block diagram illustrating a configuration example of a solid-state imaging element according to a first embodiment of the present technology.

[0024] [ Figure 5 [Illustration] is a circuit diagram illustrating a configuration example of a current-to-voltage conversion circuit according to a first embodiment of the present technology.

[0025] [ Figure 6 [Illustration] is a circuit diagram illustrating a configuration example of a buffer, subtractor, and digital converter according to a first embodiment of the present technology.

[0026] [ Figure 7 [Illustration 1] is a block diagram illustrating a configuration example of a signal processing unit according to a first embodiment of the present technology.

[0027] [ Figure 8 [ ] is a line graph illustrating an example of the changes in the voltage signal, differential signal, and detection signal before widening the dead zone in the first embodiment of this technology.

[0028] [ Figure 9[ ] is a line graph illustrating an example of the changes in the voltage signal, differential signal, and detection signal after widening the dead zone in a first embodiment of the present technology.

[0029] [ Figure 10 [Illustration] is a diagram illustrating an example of the number of detections before and after changing the dead zone width in a first embodiment of this technology.

[0030] [ Figure 11 [ ] is a flowchart illustrating an example of the operation of a solid-state imaging element according to a first embodiment of the present technology.

[0031] [ Figure 12 [ ] is a block diagram illustrating a configuration example of a solid-state imaging element according to a second embodiment of the present technology.

[0032] [ Figure 13 [Illustration] is a diagram illustrating an example of information stored in a memory according to a second embodiment of the present technology.

[0033] [ Figure 14 [Illustration 1] is a block diagram illustrating a schematic configuration example of a vehicle control system.

[0034] [ Figure 15 [Illustration] is an explanatory diagram showing an example of the mounting location of the imaging unit. Detailed Implementation

[0035] The following describes the modes used to implement this technology (hereinafter referred to as implementation methods). The descriptions will be given in the following order.

[0036] 1. First Implementation Method (Example of controlling dead zone width based on the number of detections)

[0037] 2. Second Implementation Method (An Example of Controlling Dead Zone Width Based on the Number of Detections for Each Region)

[0038] 3. Examples of application to movable subjects

[0039] <1. First Implementation Method>

[0040] [Imaging Device Configuration Examples]

[0041] Figure 1 This is a block diagram illustrating a configuration example of an imaging device 100 according to a first embodiment of the present technology. The imaging device 100 includes an imaging lens 110, a solid-state imaging element 200, a recording unit 120, and an imaging control unit 130. It is assumed that a camera mounted on an industrial robot, a vehicle-mounted camera, or the like is used as the imaging device 100.

[0042] Imaging lens 110 collects incident light and guides it to solid-state imaging element 200. Solid-state imaging element 200 performs photoelectric conversion on the incident light to detect address events and performs predetermined processing such as object recognition based on the detection results. Solid-state imaging element 200 supplies data representing the execution results to recording unit 120.

[0043] Recording unit 120 records data from solid-state imaging element 200. Imaging control unit 130 controls solid-state imaging element 200 and causes solid-state imaging element 200 to start detecting address events.

[0044] [Configuration Examples of Solid-State Imaging Elements]

[0045] Figure 2 This is a diagram illustrating an example of the laminated structure of a solid-state imaging element 200 according to a first embodiment of the present technology. The solid-state imaging element 200 includes a detection chip 202 and a light-receiving chip 201 laminated on the detection chip 202. These chips are electrically connected via connection components such as through-holes. Note that in addition to through-holes, Cu-Cu bonds or bumps can also be used for connection.

[0046] Figure 3 This is a block diagram illustrating a configuration example of a solid-state imaging element 200 according to a first embodiment of the present technology. The solid-state imaging element 200 includes a row driving circuit 211, a bias voltage supply unit 212, a pixel array unit 213, a column driving circuit 214, a signal processing unit 220, and a memory 215.

[0047] Furthermore, in the pixel array unit 213, multiple pixel circuits 300 are arranged in a two-dimensional grid shape. Hereinafter, a group of pixel circuits 300 arranged in the horizontal direction is referred to as a "row", and a group of pixel circuits 300 arranged in the direction perpendicular to the row is referred to as a "column".

[0048] Pixel circuit 300 detects brightness changes occurring outside a predetermined dead zone as address events and generates a detection signal representing the detection result. Here, the dead zone represents the range of brightness changes for which no address event is detected. A bias voltage Vbon, representing the upper limit of the dead zone, and a bias voltage Vboff, representing the lower limit of the dead zone, are supplied to each of the plurality of pixel circuits 300.

[0049] Additionally, address events include on events and off events. An on event is detected when the amount of brightness change exceeds the upper limit of the dead zone (Vb_bon). Conversely, an off event is detected when the amount of brightness change is less than the lower limit of the dead zone (Vb_off). The detection signal includes a 1-bit signal representing the detection result of the on event and a 1-bit signal representing the detection result of the off event. Note that although the pixel circuit 300 detects both on and off events, it can detect only one of them.

[0050] The row driving circuit 211 drives each element in each row to generate a detection signal. When the pixel circuit 300 in the driven row detects an address event, the pixel circuit 300 sends a request to send a detection signal to the column driving circuit 214.

[0051] The column drive circuit 214 processes each request for the column and returns a response based on the processing result. The pixel circuit 300, having received a response, supplies a detection signal to the signal processing unit 220.

[0052] The signal processing unit 220 performs predetermined image processing, such as image recognition, on the detected signal. The signal processing unit 220 supplies data representing the processing result to the recording unit 120.

[0053] Furthermore, for each fixed unit cycle, the signal processing unit 220 counts the number of detections (i.e., the number of times an address event is detected in the pixel array unit 213 within the cycle) and stores the number of detections in the memory 215. When both an on-event and an off-event are present, the number of detections is counted each time either an on-event or off-event is detected. For example, if an on-event is detected in 10 pixels, an off-event is detected in 15 pixels, and no address event is detected in the remaining pixels within the unit cycle, the number of detections is 25.

[0054] Then, the signal processing unit 220 reads the number of detections from the memory 215 and controls the difference (i.e., the dead zone width) between the bias voltages Vbon and Vboff by sending a control signal based on the number of detections. For example, the signal processing unit 220 widens the dead zone as the number of detections increases. Here, the control signal is a signal used to instruct the bias voltage supply unit 212 to increase or decrease each of the bias voltages Vbon and Vboff. Note that the signal processing unit 220 is an example of the control unit described in the claims.

[0055] The bias voltage supply unit 212 generates bias voltages Vbon and Vboff based on the control signal from the signal processing unit 220 and supplies them to all pixel circuits 300. The memory 215 stores the number of detections and the upper and lower limits of the dead zone.

[0056] [Pixel Circuit Configuration Example]

[0057] Figure 4 This is a block diagram illustrating a configuration example of the pixel circuit 300 according to a first embodiment of the present technology. The pixel circuit 300 includes a photoelectric conversion element 301, a current-to-voltage conversion circuit 310, a buffer 320, a subtractor 330, a digital converter 340, and a transmission circuit 350.

[0058] The photoelectric conversion element 301 performs photoelectric conversion on the incident light to generate an optical signal. The photoelectric conversion element 301 supplies the generated photocurrent to the current-to-voltage conversion circuit 310.

[0059] The current-to-voltage conversion circuit 310 converts the photocurrent from the photoelectric conversion element 301 into a logarithmic voltage signal. The current-to-voltage conversion circuit 310 supplies the voltage signal to the buffer 320.

[0060] Buffer 320 corrects the voltage signal from current-to-voltage conversion circuit 310. Buffer 320 outputs the corrected voltage signal to subtractor 330.

[0061] Subtractor 330 reduces the level of the voltage signal from buffer 320 according to the row drive signal from row drive circuit 211. Subtractor 330 supplies the signal with the reduced level as a differential signal to digital converter 340.

[0062] The digital converter 340 quantizes the differential signal from the subtractor 330 into a digital signal and outputs it as a detection signal to the transmission circuit 350.

[0063] The transmission circuit 350 transmits the detection signal from the digitizer 340 to the signal processing unit 220. When an address event is detected, the transmission circuit 350 supplies a request for sending the detection signal to the column driver circuit 214. Then, when the transmission circuit 350 receives a response to the request from the column driver circuit 214, the transmission circuit 350 supplies the detection signal to the signal processing unit 220.

[0064] [Configuration Example of Current-to-Voltage Conversion Circuit]

[0065] Figure 5This is a circuit diagram illustrating a configuration example of a current-to-voltage conversion circuit 310 according to a first embodiment of the present technology. The current-to-voltage conversion circuit 310 includes N-type transistors 311 and 313 and a P-type transistor 312. For example, metal-oxide-semiconductor (MOS) transistors are used as these transistors.

[0066] N-type transistor 311 has a source connected to photoelectric conversion element 301 and a drain connected to a power supply terminal. N-type transistors 312 and 313 are connected in series between the power supply terminal and the ground terminal. Furthermore, the connection point of P-type transistor 312 and N-type transistor 313 is connected to the gate of N-type transistor 311 and the input terminal of buffer 320. Additionally, a predetermined bias voltage Vbias is applied to the gate of P-type transistor 312.

[0067] The drains of N-type transistors 311 and 313 are connected to the power supply side, and such a circuit is called a source follower. These two source followers connected in the loop convert the photocurrent from the photoelectric conversion element 301 into a logarithmic voltage signal. Additionally, P-type transistor 312 supplies a constant current to N-type transistor 313.

[0068] Furthermore, in each pixel circuit of the pixel circuit 300, a photoelectric conversion element 301 is arranged on the light receiving chip 201. On the other hand, circuits and components other than the photoelectric conversion element 301 are arranged on the detection chip 202.

[0069] [Configuration examples of buffers, subtractors, and digital converters]

[0070] Figure 6 This is a circuit diagram illustrating a configuration example of the buffer 320, subtractor 330, and digital converter 340 of the first embodiment of the present technology.

[0071] Buffer 320 includes P-type transistors 321 and 322 connected in series between the power supply and ground terminals. For example, MOS transistors are used as these transistors. The gate of P-type transistor 322 on the ground side is connected to the current-to-voltage conversion circuit 310, and a bias voltage Vbsf is applied to the gate of P-type transistor 321 on the power supply side. Additionally, the junction of P-type transistors 321 and 322 is connected to subtractor 330. This connection is used to perform impedance conversion on the voltage signal from the current-to-voltage conversion circuit 310.

[0072] Subtractor 330 includes capacitors 331 and 333, P-type transistors 332 and 334, and N-type transistor 335. For example, MOS transistors are used as these transistors.

[0073] One end of capacitor 331 is connected to buffer 320, and the other end is connected to one end of capacitor 333 and the gate of P-type transistor 334. The gate of P-type transistor 332 receives the input of the horizontal drive signal from horizontal drive circuit 211, and the source and drain of P-type transistor 332 are connected to the two ends of capacitor 333. N-type transistors 334 and 335 are connected in series between the power supply terminal and the ground terminal. Additionally, the other end of capacitor 333 is connected to the junction of P-type transistor 334 and N-type transistor 335. A bias voltage Vba is applied to the gate of N-type transistor 335 on the ground side, and the junction of P-type transistor 334 and N-type transistor 335 is also connected to digitizer 340. Using this connection, a differential signal representing the amount of brightness change is generated and output to digitizer 340.

[0074] The digital converter 340 includes P-type transistors 341 and 343, and N-type transistors 342 and 344. For example, MOS transistors are used as these transistors.

[0075] P-type transistor 341 and N-type transistor 342 are connected in series between the power supply terminal and the ground terminal, and P-type transistor 343 and N-type transistor 344 are also connected in series between the power supply terminal and the ground terminal. Additionally, the gates of P-type transistors 341 and 343 are connected to subtractor 330. A bias voltage Vbon is applied to the gate of N-type transistor 342, and a bias voltage Vboff is applied to the gate of N-type transistor 344.

[0076] The connection point of P-type transistor 341 and N-type transistor 342 is connected to the transmission circuit 350, and the voltage at the connection point is output as the detection signal VCH. The connection point of P-type transistor 343 and N-type transistor 344 is also connected to the transmission circuit 350, and the voltage at the connection point is output as the detection signal VCL. Using this connection, the digital converter 340 outputs a high-level detection signal VCH when the differential signal exceeds the bias voltage Vboff, and outputs a low-level detection signal VCL when the differential signal drops below the bias voltage Vboff. The detection signal VCH represents the result of an on-event detection, and the detection signal VCL represents the result of an off-event detection.

[0077] Note that although only the photoelectric conversion element 301 is arranged on the light receiving chip 201 and other components are arranged on the detection chip 202, the circuitry to be arranged on each chip is not limited to this configuration. For example, the photoelectric conversion element 301 and N-type transistors 311 and 313 can be arranged on the light receiving chip 201, and others can be arranged on the detection chip 202. For example, the photoelectric conversion element 301 and N-type transistors 311 and 310 can be arranged on the light receiving chip 201, and others can be arranged on the detection chip 202. Alternatively, the photoelectric conversion element 301, the current-to-voltage conversion circuit 310, and the buffer 320 can be arranged on the light receiving chip 201, and others can be arranged on the detection chip 202. Alternatively, the photoelectric conversion element 301, the current-to-voltage conversion circuit 310, the buffer 331, and the capacitor 311 can be arranged on the light receiving chip 201, and others can be arranged on the detection chip 202. Alternatively, the photoelectric conversion element 301, the current-to-voltage conversion circuit 310, the buffer 340, the subtractor 330, and the digital converter 340 can be arranged on the photodetector chip 201, and others can be arranged on the detection chip 202.

[0078] [Configuration Example of Signal Processing Unit]

[0079] Figure 7 This is a block diagram illustrating a configuration example of a signal processing unit 220 according to a first embodiment of the present technology. The signal processing unit 220 includes an image processor 221, a detection counter 222, and a bias controller 223.

[0080] Image processor 221 performs predetermined processing, such as object recognition, on image data including detection signals from pixel array unit 213. Image processor 221 supplies the processing results to recording unit 120. Note that image data processing can be performed by a digital signal processor (DSP) other than solid-state imaging element 200, instead of image processor 221.

[0081] For each fixed unit cycle, the detection counter 222 counts the number of times an address event is detected in the pixel array unit 213 within the cycle. At the beginning of each unit cycle, the detection counter 222 sets the number of detections in the memory 215 to an initial value. Then, whenever an address event is detected, the detection counter 222 increments the number of detections and updates the number of detections with the incremented value. That is, the detection counter 222 counts upwards. Note that although the detection counter 222 counts upwards, it can also count downwards.

[0082] The bias controller 223 controls the bias voltage based on the number of detections. When the imaging control unit 130 issues an instruction to start detecting the address event, the bias controller 223 sets the upper and lower limits of the dead zone in the memory 215 to initial values. Then, for each unit cycle, the bias controller 223 reads the number of detections from the memory 215 at the end of the cycle and determines whether the number of detections is within a predetermined allowable range.

[0083] If the number of detections exceeds the allowable range and is greater than the upper limit of the range, the bias controller 223 widens the dead zone. For example, the bias controller 223 raises the upper limit of the dead zone by a predetermined value and lowers the lower limit of the dead zone by a predetermined value, and updates the memory 215 using the changed values. In addition, the bias controller 223 controls the bias voltages Vbon and Vboff to values ​​corresponding to the updated upper and lower limits of the dead zone using control signals.

[0084] On the other hand, if the number of detections is less than the lower limit of the allowable range, the bias controller 223 narrows the dead zone. For example, the bias controller 223 lowers the upper limit of the dead zone by a predetermined value, raises the lower limit of the dead zone by a predetermined value, and updates the memory 215 using the changed values. In addition, the bias controller 223 controls the bias voltages Vbon and Vboff to values ​​corresponding to the updated upper and lower limits of the dead zone using control signals.

[0085] In addition, if the number of detections is within the allowable range, the bias controller 223 does not control the width of the dead zone and maintains the current value.

[0086] Furthermore, if the number of detections is outside the allowable range, the bias controller 223 can control only the width of the dead zone, or it can widen the dead zone as the number of detections increases without providing an allowable range.

[0087] In addition, the bias controller 223 increases or decreases both the upper and lower limits of the dead zone, and can control the width of the dead zone by increasing or decreasing only one of them.

[0088] Furthermore, the bias controller 223 does not limit the control amount of the dead zone width, but it can control the dead zone width within a certain control range. For example, if the dead zone width reaches the upper limit of the control range, the bias controller 223 will not further widen the dead zone even if the number of detections exceeds the upper limit of the allowable range. Conversely, if the dead zone width reaches the lower limit of the control range, the bias controller 223 will not further narrow the dead zone even if the number of detections is less than the lower limit of the allowable range.

[0089] Figure 8 This is a line graph illustrating an example of the changes in the voltage signal, differential signal, and detection signal before widening the dead zone in the first embodiment of this technology. Here, Figure 8 Figure 'a' is a line graph showing an example of changes in the voltage signal of a pixel. Figure 8 b is a line graph showing an example of the variation in the differential signal of a pixel. Here, Figure 8 c is a line graph showing an example of changes in the detection signal of a pixel. Figure 8 In diagram a, the vertical axis represents the level of the voltage signal from the current-to-voltage conversion circuit 310, and the horizontal axis represents time. Figure 8 In diagram b, the vertical axis represents the level of the differential signal from subtractor 330, while the horizontal axis represents time. Figure 8 In diagram c, the vertical axis represents the level of the detection signal from the digital converter 340, and the horizontal axis represents time. Figure 8 In 'c', the upward arrow represents the detection signal when an on event is detected, and the downward arrow represents the detection signal when an off event is detected.

[0090] When the brightness of light incident on a pixel changes, the voltage signal changes accordingly. Additionally, the differential signal representing the amount of brightness change also changes. Then, for example, at times T0 and T1, the level of the differential signal drops below the lower limit of the dead zone. Furthermore, at times T2, T3, and T4, the level of the differential signal exceeds the upper limit of the dead zone. Therefore, a disconnection event is detected at times T0 and T1, and a connection event is detected at times T2, T3, and T4. Furthermore, when the level of the differential signal is within the dead zone, no address event is detected.

[0091] Here, it is assumed that the number of detections exceeds the upper limit of the allowable range, and the bias controller 223 widens the dead zone.

[0092] Figure 9 This is a line graph illustrating an example of the changes in the voltage signal, differential signal, and detection signal after widening the dead zone in the first embodiment of this technology. Here, Figure 8 Figure 'a' is a line graph showing an example of the change in the voltage signal of a pixel, and Figure 8 b is a line graph showing an example of the variation in the differential signal of a pixel. Here, Figure 8 Figure c is a graph showing an example of changes in the detection signal of a pixel. Figure 9 In diagram 'a', the vertical axis represents the voltage signal level, and the horizontal axis represents time. Figure 9 In diagram b, the vertical axis represents the level of the differential signal, and the horizontal axis represents time. Figure 9 In 'c', the vertical axis represents the level of the detected signal, while the horizontal axis represents time.

[0093] Suppose that after changing the dead-band width, a similar brightness change occurs as before the change. After the change, at time T0, the differential signal level drops below the lower limit of the dead-band. Additionally, at times T1 and T2, the differential signal level exceeds the upper limit of the dead-band. Therefore, an off event is detected at time T0, and an on event is detected at times T1 and T2. As described above, since the detection sensitivity for address events is lower than before widening the dead-band, the number of address events detected becomes less.

[0094] Figure 10 This is a diagram illustrating an example of the number of detections before and after changing the dead zone width in the first embodiment of this technology. Here, Figure 10 'a' is a histogram showing the number of detections per unit period before and after widening the dead zone. Here, Figure 10 b is a histogram showing the number of times detected in each unit cycle before and after the narrowing dead zone.

[0095] When the number of detections exceeds the upper limit of the allowable range, the bias controller 223 widens the dead zone. This reduces the detection sensitivity for address events, and the number of detections becomes less than the number of detections before the dead zone width was changed.

[0096] On the other hand, when the number of detections is less than the lower limit of the allowable range, the bias controller 223 narrows the dead zone. Therefore, the detection sensitivity for address events is improved, and the number of detections becomes greater than the number of detections before the dead zone width was changed.

[0097] As described above, since the bias controller 223 increases or decreases the width of the dead zone according to the number of detections, the width of the dead zone can be set to an appropriate range.

[0098] For example, consider a scenario where the brightness of the entire screen changes due to brightness variations. In this case, since the brightness of all pixels changes, if the dead zone is too narrow, address events may be detected in all pixels. As the number of address event detections increases, the load on the circuitry transmitting and processing the detection signals increases, potentially increasing the power consumption of the solid-state imaging element 200 as a whole. However, because the bias controller 223 widens the dead zone as the number of detections increases, over-detection of address events can be suppressed, and power consumption can be reduced.

[0099] Furthermore, consider the case where brightness changes occur in some pixels among all pixels, and the amount of change is small. In this case, if the dead zone is too wide, there is a possibility that address events cannot be detected in the pixels where the change occurred, and thus the address events are lost. However, since the bias controller 223 narrows the dead zone as the number of detections decreases, address event loss can be prevented.

[0100] [Operational Examples of Solid-State Imaging Elements]

[0101] Figure 11 This is a flowchart illustrating an example of the operation of a solid-state imaging element 200 according to a first embodiment of the present technology. Operation begins when a predetermined application for detecting address events is executed.

[0102] The signal processing unit 220 in the solid-state imaging element 200 initializes the upper and lower limits of the dead zone and the number of detections (step S901). Then, the signal processing unit 220 counts the number of detection address events (step S902) and determines whether a unit cycle has elapsed (step S903). If no unit cycle has elapsed (step S903: No), the signal processing unit 220 repeats step S902.

[0103] On the other hand, if a unit cycle has been completed (step S903: Yes), the signal processing unit 220 determines whether the number of detections exceeds the upper limit of the allowable range (step S904). If the number of detections exceeds the upper limit of the allowable range (step S904: Yes), the signal processing unit 220 raises the upper limit of the dead zone and lowers the lower limit of the dead zone to widen the dead zone (step S905).

[0104] If the number of detections is equal to or less than the upper limit of the allowable range (step S904: No), the signal processing unit 220 determines whether the number of detections is less than the lower limit of the allowable range (step S906). If the number of detections is less than the lower limit of the allowable range (step S906: Yes), the signal processing unit 220 lowers the upper limit of the dead zone and raises the lower limit of the dead zone to narrow the dead zone (step S907).

[0105] If the number of detections is within the allowable range (step S906: No), the signal processing unit 220 initializes the number of detections (step S908) and repeats step S902 and subsequent steps. Additionally, the signal processing unit 220 executes step S908 after step S905 or S907.

[0106] As described above, according to the first embodiment of this technology, since the signal processing unit 220 controls the width of the dead zone based on the number of times the address event is detected, the detection sensitivity for the address event can be controlled to an appropriate value.

[0107] <2. Second Implementation Method>

[0108] In the first embodiment described above, the signal processing unit 220 controls the bias voltage of all pixels to the same value. However, with this configuration, the detection sensitivity for address events may be inappropriate for certain scenarios. For example, if the brightness of a portion of the pixel array unit 213 changes due to brightness variations, address events may be over-detected in that portion. The solid-state imaging element 200 of the second embodiment differs from the first embodiment in that the pixel array unit 213 is divided into multiple regions and the bias voltage is controlled for each region.

[0109] Figure 12 This is a block diagram illustrating a configuration example of a solid-state imaging element 200 according to a second embodiment of the present technology. The solid-state imaging element 200 of the second embodiment differs from that of the first embodiment in that the pixel array unit 213 is divided into M (M is an integer of 2 or greater) unit regions 305. In each unit region 305, pixel circuits 300 are arranged in I rows × J columns (I and J are integers).

[0110] Furthermore, the signal processing unit 220 of the second embodiment counts the number of detections in each unit region 305 and controls the bias voltage based on the number of detections. Additionally, the bias voltage supply unit 212 of the second embodiment supplies bias voltage Vbon1 to VbonM and bias voltage Vboff1 to VboffM. Bias voltages Vbonm and Vboffm (where m is an integer from 1 to M) are supplied to the m-th unit region 305.

[0111] Figure 13 This is a diagram illustrating an example of information stored in memory 215 according to a second embodiment of the present technology. Memory 215 stores the number of detections, upper dead zone limit, and lower dead zone limit for each of the M cell regions 305.

[0112] For example, suppose that the number of detections in a unit period for the region with region identification number "01" used for identification unit region 305 is "15", and the number of detections is within the allowable range. Alternatively, suppose that the number of detections in a unit period for the region with region identification number "02" is "0", and the number of detections is less than the lower limit of the allowable range. In this case, the signal processing unit 220 does not change the upper and lower limits of the dead zone for the region with region identification number "01". On the other hand, for the region with region identification number "02", the signal processing unit 220 lowers the upper dead zone limit "U02" and raises the lower dead zone limit "L02".

[0113] As described above, according to the second embodiment of this technology, since the signal processing unit 220 controls the dead zone width based on the number of address events detected in each cell region, the detection sensitivity for each cell region can be controlled to an appropriate value.

[0114] <3. Examples of application to movable subjects>

[0115] The technology disclosed herein (the Technology) can be applied to a variety of products. For example, the Technology disclosed herein can be implemented as a device installed on any type of mobile body, including automobiles, electric vehicles, hybrid electric vehicles, motorcycles, bicycles, personal mobility devices, airplanes, drones, ships, robots, etc.

[0116] Figure 14 This is a block diagram illustrating a schematic configuration example of a vehicle control system, which is an example of a mobility control system to which the technology according to this disclosure can be applied.

[0117] The vehicle control system 12000 includes multiple electronic control units connected via a communication network 12001. Figure 14 In the example shown, the vehicle control system 12000 includes a drive system control unit 12010, a main system control unit 12020, an external information detection unit 12030, an internal information detection unit 12040, and an integrated control unit 12050. Furthermore, as part of the functional configuration of the integrated control unit 12050, a microcomputer 12051, an audio / video output unit 12052, and an in-vehicle network interface (I / F) 12053 are shown.

[0118] The drive system control unit 12010 controls the operation of devices related to the vehicle's drive system according to various processes. For example, the drive system control unit 12010 acts as a controller for devices including: a drive force generating device for generating drive force for the vehicle, such as an internal combustion engine or drive motor; a drive force transmission mechanism for transmitting drive force to the wheels; a steering mechanism for adjusting the vehicle's steering angle; and a braking device for generating braking force for the vehicle.

[0119] The main system control unit 12020 controls the operation of various devices equipped on the vehicle body according to various processes. For example, the main system control unit 12020 acts as a controller for a keyless entry system, a smart key system, power windows, or various lights such as headlights, taillights, brake lights, hazard lights, or fog lights. In this case, the main system control unit 12020 can receive inputs of radio waves or signals from various switches transmitted from a portable device that replaces the key. The main system control unit 12020 receives these radio wave or signal inputs and controls the vehicle's door locking devices, power windows, lights, etc.

[0120] The external information detection unit 12030 detects information about the exterior of the vehicle equipped with the vehicle control system 12000. For example, the imaging unit 12031 is connected to the external information detection unit 12030. The external information detection unit 12030 causes the imaging unit 12031 to capture images of the exterior of the vehicle and receives the captured images. The external information detection unit 12030 can perform object detection processing or distance detection processing based on the received images, such as detecting people, vehicles, obstacles, signs, characters on the road surface, etc.

[0121] Imaging unit 12031 is an optical sensor that receives light and outputs an electrical signal corresponding to the amount of light received. Imaging unit 12031 can output the electrical signal as an image or as distance measurement information. Furthermore, the light received by imaging unit 12031 can be visible light or invisible light such as infrared light.

[0122] The interior information detection unit 12040 detects information about the interior of the vehicle. For example, a driver state detection unit 12041, which detects the driver's state, is connected to the interior information detection unit 12040. For example, the driver state detection unit 12041 includes a camera for capturing images of the driver, and the interior information detection unit 12040 can calculate the driver's fatigue level or level of attention, or determine whether the driver is asleep, based on the detection information input from the driver state detection unit 12041.

[0123] The microcomputer 12051 can calculate control target values ​​for the drive force generation device, steering mechanism, or braking device based on external or internal information about the vehicle obtained by the external information detection unit 12030 or the internal information detection unit 12040, and output control commands to the drive system control unit 1210. For example, the microcomputer 12051 can perform coordinated control to realize the functions of an advanced driver assistance system (ADAS), including collision avoidance or impact mitigation, distance-based following, vehicle speed maintenance, collision warning, lane departure warning, etc.

[0124] In addition, the microcomputer 12051 can control the drive force generation device, steering mechanism, braking device, etc. based on information about the vehicle's surroundings obtained by the external information detection unit 12030 or the internal information detection unit 12040, so as to perform coordinated control for autonomous driving that is intended to drive automatically without relying on the driver's operation.

[0125] Additionally, the microcomputer 12051 can output control commands to the body system control unit 12020 based on external information about the vehicle obtained by the external information detection unit 12030. For example, the microcomputer 12051 can control the headlights based on the position of the vehicle in front or oncoming vehicles detected by the external information detection unit 12030, and perform coordinated control aimed at preventing glare, such as switching from high beam to low beam.

[0126] The audio-visual output unit 12052 sends an output signal of at least one of audio or visual signals to an external output device capable of visually or audibly notifying passengers or the vehicle of information. Figure 14 In the examples, as output devices, an audio speaker 12061, a display unit 12062, and an instrument panel 12063 are shown. For example, the display unit 12062 may include at least one of an onboard display or a head-up display.

[0127] Figure 15 This is a diagram showing an example of the mounting location of the imaging unit 12031.

[0128] exist Figure 15 In the imaging unit 12031, imaging units 12101, 12102, 12103, 12104 and 12105 are included.

[0129] For example, imaging units 12101, 12102, 12103, 12104, and 12105 are disposed in locations such as the front nose, side mirrors, rear bumper, rear door, and upper part of the windshield inside the vehicle 12100. Imaging unit 12101 disposed on the front nose inside the vehicle and imaging unit 12105 disposed on the upper part of the windshield mainly acquire images of the front of the vehicle 12100. Imaging units 12102 and 12103 disposed on the side mirrors mainly acquire images of the sides of the vehicle 12100. Imaging unit 12104 disposed on the rear bumper or rear door mainly acquires images of the rear of the vehicle 12100. Imaging unit 12105 disposed on the upper part of the windshield inside the vehicle is mainly used to detect vehicles or pedestrians, obstacles, traffic lights, traffic signs, lanes, etc. in front.

[0130] Notice, Figure 15Examples of the imaging ranges of imaging units 12101 to 12104 are shown. Imaging range 12111 represents the imaging range of imaging unit 12101 located on the front nose, imaging ranges 12112 and 12113 represent the imaging ranges of imaging units 12102 and 12103 located on the respective side mirrors, and imaging range 12114 represents the imaging range of imaging unit 12104 located on the rear bumper or rear door. For example, by superimposing the individual image data captured by imaging units 12101 to 12104, a bird's-eye view of the vehicle 12100 viewed from above can be obtained.

[0131] At least one of the imaging units 12101 to 12104 may have the function of acquiring distance information. For example, at least one of the imaging units 12101 to 12104 may be a stereo camera including multiple imaging elements, or may be an imaging element having pixels for phase difference detection.

[0132] For example, based on distance information obtained from imaging units 12101 to 12104, microcomputer 12051 can measure the distance to each three-dimensional object in the imaging range 12111 to 12114 and the time change of that distance (relative speed relative to vehicle 12100) to extract the closest three-dimensional object on the driving path of vehicle 12100 (specifically, a three-dimensional object traveling in approximately the same direction as vehicle 12100 at a predetermined speed (e.g., above 0 km / h)) as the vehicle ahead. Furthermore, microcomputer 12051 can pre-set the vehicle-to-vehicle distance to be maintained relative to the vehicle ahead and execute automatic braking control (including follow-stop control), automatic acceleration control (including follow-start control), etc. As described above, coordinated control for autonomous driving can be performed without relying on driver operation, etc.

[0133] For example, based on distance information obtained from imaging units 12101 to 12104, microcomputer 12051 can extract three-dimensional object data about three-dimensional objects by classifying the data into three-dimensional objects such as two-wheeled vehicles, ordinary vehicles, large vehicles, pedestrians, and telephone poles, and use this data for automatic obstacle avoidance. For example, microcomputer 12051 identifies obstacles around vehicle 12100 as obstacles visible to the driver of vehicle 12100 and obstacles that are almost invisible to the driver of vehicle 12100. Then, microcomputer 12051 can determine a collision risk representing the degree of risk of colliding with each obstacle, and when the collision risk is above a set value and there is a possibility of collision, microcomputer 12051 can output a warning to the driver through audio speaker 12061 or display unit 12062 or perform forced deceleration or evasive steering through drive system control unit 1210 to perform driving assistance for collision avoidance.

[0134] At least one of the imaging units 12101 to 12104 may be an infrared camera that detects infrared light. For example, the microcomputer 12051 can identify a pedestrian by determining whether a pedestrian exists in the images captured by the imaging units 12101 to 12104. This pedestrian recognition is performed, for example, by a process of extracting feature points from the images captured by the imaging units 12101 to 12104, which are infrared cameras, and by performing pattern matching processing on a series of feature points representing the outline of an object to determine whether the object is a pedestrian. When the microcomputer 12051 determines that a pedestrian exists in the images captured by the imaging units 12101 to 12104 and identifies the pedestrian, the audio-visual output unit 12052 causes the display unit 12062 to overlay a square outline for emphasizing the identified pedestrian. In addition, the audio-visual output unit 12052 can cause the display unit 12062 to display an icon or the like representing a pedestrian at a desired location.

[0135] An example of a vehicle control system to which the technology of this disclosure can be applied has been described above. For example, in the above configuration, the technology according to this disclosure can be applied to the imaging unit 12031. Specifically, Figure 1 The imaging device 100 can be applied to the imaging unit 12031. By applying the technology according to this disclosure to the imaging unit 12031, the detection sensitivity for address events can be controlled to an appropriate value and the reliability of the vehicle control system can be improved.

[0136] Note that the above embodiments are examples illustrating the present technology, and the content of the embodiments corresponds to the content of the invention specified in the "claims". Similarly, the content of the invention specified in the "claims" corresponds to the content with the same name in the embodiments of the present technology. Note, however, that the present invention is not limited to these embodiments and various modified embodiments may be implemented without departing from the spirit of the invention.

[0137] Furthermore, the processing described in the above embodiments can be considered as a method comprising a series of such processes, or as a program for causing a computer to execute a series of such processes or a stored program on a recording medium. Examples of recording media include, for example, compact optical discs (CDs), miniature disks (MDs), digital versatile optical discs (DVDs), memory cards, Blu-ray discs (registered trademark), etc.

[0138] Note that the effects described in this specification are merely illustrative and not limiting, and other effects may be obtained.

[0139] Note that this technology can also be configured as follows.

[0140] (1) A solid-state imaging element, comprising:

[0141] A pixel array unit is configured with multiple pixel circuits, each pixel circuit detecting brightness changes of incident light occurring outside a predetermined dead zone as address events; and

[0142] The control unit controls the width of the dead zone based on the number of address events detected in the pixel array unit within a fixed unit cycle.

[0143] (2) The solid-state imaging element according to (1) above, wherein

[0144] As the number of tests increases, the control unit widens the dead zone.

[0145] (3) The solid-state imaging element according to (1) or (2) above, wherein

[0146] Each pixel circuit in the multiple pixel circuits compares the upper and lower limits of the dead zone with the amount of brightness change, and detects address events based on the comparison results.

[0147] (4) A solid-state imaging element according to any one of (1) to (3) above, wherein

[0148] If the number of tests exceeds the predetermined allowable range, the control unit controls the width of the dead zone.

[0149] (5) A solid-state imaging element according to any one of (1) to (4) above, wherein

[0150] The pixel array unit is divided into multiple regions, and

[0151] The control unit controls the width of the dead zone for each of the multiple zones.

[0152] (6) A solid-state imaging element according to any one of (1) to (5) above, wherein

[0153] Each pixel circuit in a plurality of pixel circuits includes

[0154] A photoelectric conversion element converts incident light into photocurrent through photoelectric conversion.

[0155] A current-to-voltage conversion circuit converts photocurrent into voltage.

[0156] The photoelectric conversion element is arranged on the optical receiver chip, and

[0157] The current-to-voltage conversion circuit is arranged on the detection chip stacked on the optical receiver chip.

[0158] (7) An imaging device includes:

[0159] The pixel array unit is configured with multiple pixel circuits, each of which detects the brightness change of incident light occurring outside a predetermined dead zone as an address event.

[0160] The control unit controls the width of the dead zone based on the number of address events detected in the pixel array unit within a fixed unit cycle; and

[0161] The recording unit records the data obtained from the detection results of the address event.

[0162] (8) A method for controlling a solid-state imaging element, comprising:

[0163] A counting process that counts the number of address events detected in a pixel array cell within a fixed unit period. Multiple pixel circuits are arranged in the pixel array cell, and each pixel circuit detects brightness changes of incident light occurring outside a predetermined dead zone as address events; and

[0164] The control process adjusts the dead zone width based on the number of detections.

[0165] Reference Mark List

[0166] 100 Imaging Device

[0167] 110 Imaging Lens

[0168] 120 Recording Unit

[0169] 130 Imaging Control Unit

[0170] 200 solid-state imaging elements

[0171] 201 Optical Receiver Chip

[0172] 202 Detection Chip

[0173] 211 Horizontal Drive Circuit

[0174] 212 Bias Voltage Supply Unit

[0175] 213 pixel array unit

[0176] 214-column drive circuit

[0177] 215 Memory

[0178] 220 Signal Processing Unit

[0179] 221 Image Processor

[0180] 222 Detection Counter

[0181] 223 Bias Controller

[0182] 300-pixel circuit

[0183] 301 Photoelectric Conversion Element

[0184] 305 Unit Area

[0185] 310 Current-to-Voltage Conversion Circuit

[0186] 311, 313, 335, 342, and 344N type transistors

[0187] 312, 321, 322, 332, 334, 341, 343P-type transistors

[0188] 320 buffer

[0189] 330 Subtractor

[0190] 331 and 333 capacitors

[0191] 340 digital converter

[0192] 350 transmission circuit

[0193] 12031 Imaging Unit.

Claims

1. A light detection device comprising: a pixel array including a plurality of regions, each of the regions including at least one pixel circuit that detects a change in luminance of incident light occurring outside a dead zone as an address event; and a signal processing unit configured to control a width of the dead zone in accordance with a number of times the address event is detected by the at least one pixel circuit within a fixed unit period, wherein the signal processing unit is configured to widen the dead zone as the number of detections increases.

2. The light detection device according to claim 1, wherein the signal processing unit is configured to compare each of an upper limit and a lower limit of the dead zone with an amount of the change in luminance, and detect the address event based on a comparison result.

3. The light detection device according to claim 1, wherein the signal processing unit is configured to control the width of the dead zone in a case where the number of detections exceeds a predetermined allowable range.

4. The light detection device according to claim 1, wherein the signal processing unit is configured to control the width of the dead zone for each of the plurality of regions.

5. The light detection device according to claim 1, wherein the at least one pixel circuit includes: a photoelectric conversion element that photoelectrically converts the incident light to generate a photoelectric current, and a current-voltage conversion circuit that converts the photoelectric current into a voltage, wherein the photoelectric conversion element is disposed on a light-receiving chip, and the current-voltage conversion circuit is disposed on a detection chip laminated on the light-receiving chip.

6. An imaging device including the light detection device according to claim 1.

7. A light detection device comprising: a pixel circuit that detects a change in luminance of incident light occurring outside a dead zone as an address event; and a signal processing unit configured to control a width of the dead zone in accordance with a number of times the address event is detected in the pixel circuit within a fixed unit period, wherein the signal processing unit is configured to widen the dead zone as the number of detections increases.

8. The light detection device according to claim 7, wherein the signal processing unit is configured to compare each of an upper limit and a lower limit of the dead zone with an amount of the change in luminance, and detect the address event based on a comparison result.

9. The light detection device according to claim 7, wherein the signal processing unit is configured to control the width of the dead zone in a case where the number of detections exceeds a predetermined allowable range.

10. The light detection device according to claim 7, wherein the pixel circuit includes: a photoelectric conversion element that photoelectrically converts the incident light to generate a photoelectric current, and a current-voltage conversion circuit that converts the photoelectric current into a voltage, wherein the photoelectric conversion element is disposed on a light-receiving chip, and the current-voltage conversion circuit is disposed on a detection chip laminated on the light-receiving chip. ​ ​ 11. An imaging device comprising the light detecting device according to claim 7.

Images