Photoelectric conversion device and equipment

Abstract

To provide technique advantageous for detecting an event in response to a change in a detection environment.SOLUTION: A photoelectric conversion device has a plurality of pixel blocks each including a plurality of first pixels, a plurality of event detection pixels, a determination unit for performing event detection, and an interface unit for outputting signals from the plurality of first pixels in response to the event detection. Each of the plurality of event detection pixels has a first event detection unit for outputting a signal in response to incidence of light, and a first counter unit for counting signals output from the first event detection unit per first unit time. The determination unit performs the event detection on the basis of the sum of the output from the first counter units of the plurality of event detection pixels.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present invention relates to a photoelectric conversion device and an apparatus. [Background technology]

[0002] Patent document 1 describes an imaging device that detects whether a specified address event has occurred based on whether the amount of change in the amount of incident light exceeds a specified threshold, and when an address event has occurred, counts the number of incident photons and outputs a pixel signal indicating the counted value. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2020-96347 A Summary of the Invention [Problem to be solved by the invention]

[0004] An object of the present invention is to provide a technique that is advantageous for detecting an event in response to a change in the detection environment. [Means for solving the problem]

[0005] A photoelectric conversion device according to one embodiment of the present invention is a photoelectric conversion device in which a plurality of pixel blocks are arranged, each pixel block including a plurality of first pixels, a plurality of event detection pixels, a judgment unit that performs event detection, and an interface unit that outputs signals from the plurality of first pixels in response to the event detection, wherein each of the plurality of event detection pixels has a first event detection unit that outputs a signal in response to incidence of light, and a first counter unit that counts the signals output from the first event detection unit per first unit time, and the judgment unit performs the event detection based on a sum of outputs of the first counter units of the plurality of event detection pixels. Effect of the Invention

[0006] According to the present invention, it is possible to provide a technique that is advantageous for detecting an event in response to a change in the detection environment. [Brief description of the drawings]

[0007] [Figure 1] FIG. 1 is a diagram showing a schematic configuration of a photoelectric conversion device according to an embodiment. [Diagram 2] 2 is a schematic diagram showing an example of pixel arrangement in a pixel block according to the embodiment; [Diagram 3] 1 is a circuit diagram showing an example of the configuration of a pixel according to an embodiment; [Figure 4] FIG. 2 is a functional block diagram showing an example of the arrangement of a pixel block according to the embodiment. [Diagram 5] 13 illustrates a modified example of an event detection operation according to the embodiment. [Figure 6] FIG. 1 is a schematic diagram showing an example of the configuration of a photoelectric conversion device according to an embodiment. [Figure 7] FIG. 13 is a diagram showing the control of the event detection unit. [Figure 8] 1 shows application examples of the photoelectric conversion device according to the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] Hereinafter, the embodiments will be described in detail with reference to the attached drawings. Note that the following embodiments do not limit the invention according to the claims. Although the embodiments describe a number of features, not all of these features are essential to the invention, and the features may be combined in any manner. Furthermore, in the attached drawings, the same reference numbers are used for the same or similar configurations, and duplicated descriptions are omitted.

[0009] In the following embodiments of the present invention, an image capture device capable of detecting an event will be described as an example of a photoelectric conversion device. However, the photoelectric conversion device in each embodiment is not limited to an image capture device, and may be applied to other photodetection devices based on photoelectric conversion. Examples of other photodetection devices include a distance measuring device and a focus detection device.

[0010] (Embodiment) A schematic configuration of a photoelectric conversion device 1 according to an embodiment of the present invention will be described with reference to Fig. 1. The photoelectric conversion device 1 has a pixel array 10, a pixel control unit 11, a post-processing unit 12, and an output interface (IF) unit 13. In the pixel array 10, a plurality of pixel blocks 100 are arranged to form a plurality of rows and a plurality of columns. A plurality of pixels are arranged in the pixel block 100, as will be described later. The pixel control unit 11 can drive the pixel array and control reading of signals from the pixels. The post-processing unit 12 can perform correction processing and sorting processing of the imaging signals read out from the pixels. The output interface unit 13 can output the imaging signals and the event detection signals to the outside of the pixel array 10.

[0011] An example of the arrangement of pixels physically arranged in the pixel block 100 will be briefly described with reference to Figs. 2(a) to 2(d). Fig. 2(a) shows an example in which the pixel block 100 is composed of 16 pixels arranged in 4 rows and 4 columns. In Fig. 2(a), R, G, and B indicate imaging pixels 200 that capture images. The imaging pixels 200 are imaging pixels R, G, and B that perform photoelectric conversion mainly on red light. ij , imaging pixel G that mainly converts green light into an electric signal ij , imaging pixel B which mainly converts blue light into an electric signal ij Here, the subscripts ij of R, G, and B are row number i and column number j, which indicate the position in the pixel block 100. For example, B 24 indicates an imaging pixel that mainly photoelectrically converts blue light, and is located in the second row and fourth column of a four-row, four-column pixel block 100. The imaging pixel 200 that photoelectrically converts each color can separate the color of incident light by the on-chip red, green, and blue color filters provided in the imaging pixel 200.

[0012] The imaging pixels 200 indicated by R, G, and B can output imaging signals capable of expressing gradations. In contrast, the pixel indicated by E is an event detection pixel 201 that detects changes in light intensity. Color separation is not necessarily required for the event detection pixel 201. A transparent color filter may be provided for the event detection pixel 201. Note that a transparent color filter may also be provided for the imaging pixel 200. When a transparent color filter is provided, color information is lost, but an improvement in the signal-to-noise ratio and resolution may be achieved.

[0013] The wavelength region of the color filter is not limited to the visible light region, and may be a filter for photoelectric conversion of ultraviolet light or infrared light. Also, an on-chip microlens may be arranged in each pixel. By arranging an on-chip microlens in a pixel, an improvement in the signal-to-noise ratio can be achieved. FIG. 2(b) shows an example of a pixel block in which pixels are arranged in 8 rows and 8 columns. In this example, the pixel block includes 16 event detection pixels E. FIG. 2(c) shows another example of a pixel block in which pixels are arranged in 8 rows and 8 columns. In the example of FIG. 2(c), four event detection pixels E, indicated by E, are arranged in the pixel block. FIG. 2(d) shows an example of a pixel block in which pixels are arranged in 4 rows and 2 columns. The number of rows and the number of columns in which pixels are arranged may be any number as long as they are natural numbers, and the number of rows and the number of columns do not have to be the same as in the example of FIG. 2(d).

[0014] The circuit configuration of the pixels arranged in the pixel block 100 will be briefly described with reference to the pixel section 30 shown in FIG. 3. The photoelectric conversion element 300 may be an avalanche amplification type diode. A reverse bias voltage of a magnitude equal to or greater than the breakdown voltage can be applied to the photoelectric conversion element 300. In this embodiment, the photoelectric conversion element 300 is set to operate in the Geiger mode. Specifically, a first power supply voltage Vg is applied to the anode side of the photoelectric conversion element 300 from a first power supply line connected to the photoelectric conversion element 300, and a second power supply voltage Vd is applied to the cathode side of the photoelectric conversion element 300 from a second power supply line connected to the photoelectric conversion element 300. The voltage difference between the first power supply voltage Vg and the second power supply voltage Vd is set to be equal to or greater than the breakdown voltage. For example, the first power supply voltage Vg may be −20V, and the second power supply voltage Vd may be 3.3V.

[0015] The reset unit 301 can reset the photoelectric conversion element 300 in accordance with the amount of charge generated in the photoelectric conversion element 300. The reset unit 301 may be a quenching element. The quenching element may be a PMOS transistor. A bias voltage V bias When the quenching element is a PMOS transistor, the bias voltage V applied to the gate terminal of the PMOS transistor is bias is typically 0 to 2V.

[0016] The connection point between the photoelectric conversion element 300 and the reset unit 301 is connected to the input of the inverter circuit 302. The photoelectric conversion element 300, the reset unit 301, and the inverter circuit 302 are components of an event detection unit that can detect that light has been incident on the photoelectric conversion element 300.

[0017] Next, the operation of the photoelectric conversion element 300 and the reset unit 301 will be described. Here, a case where the photoelectric conversion element 300 is a single photon avalanche diode (SINGLE PHOTON AVALANCHE DIODE (SPAD)) will be described. When one photon is incident on the photoelectric conversion element 300, one electron-hole pair is generated. After that, the one electron (and hole) is accelerated by an electric field, and a current due to multiple electrons (and holes) is generated. In other words, the photoelectric conversion element 300 amplifies the photocurrent by avalanche amplification.

[0018] A current obtained by the amplified electrons flows from the node of the second power supply voltage Vd to the first power supply voltage Vg via the photoelectric conversion element 300 and the reset unit 301. This causes the potential of the cathode of the photoelectric conversion element to drop, and the operating region of the photoelectric conversion element 300 moves out of the Geiger mode. This stops the avalanche amplification of the photoelectric conversion element 300. After that, the second power supply voltage Vd is supplied to the cathode of the photoelectric conversion element 300 via the reset unit 301, and the voltage drop caused by the reset unit 301 returns to normal. In other words, the operating region of the photoelectric conversion element 300 returns to the Geiger mode.

[0019] Here, the role of the reset unit 301 is to lower the potential of the cathode of the photoelectric conversion element 300 after avalanche amplification occurs, and then to return the operating region of the photoelectric conversion element 300 to the Geiger mode again. Through the above operations, the reset unit 301 can reset the photoelectric conversion element 300 in which avalanche amplification has occurred.

[0020] In this way, when the photoelectric conversion element 300 is the above-mentioned SPAD, the reset unit 301 resets the photoelectric conversion element according to the multiplication amount of one generated charge. Therefore, the reset unit 301 can control whether to perform a reset depending on whether a charge is generated by the incidence of a photon or whether no charge is generated.

[0021] When an avalanche amplification type diode is used for the photoelectric conversion element 300, the reset unit 301 resets the state of the photoelectric conversion element 300 in response to the charge generated in the photoelectric conversion element 300 by one photon. In contrast, instead of using an avalanche amplification type diode for the photoelectric conversion element 300, a charge accumulation type photodiode may be used. In this case, the reset unit 301 may be configured to reset the photoelectric conversion element in response to the generation of a predetermined amount of charge. The predetermined amount may be set to an amount less than the saturated charge amount of the photodiode. The reset unit 301 does not perform the reset operation while the amount of charge generated in the photoelectric conversion element 300 is less than the predetermined amount. On the other hand, when a charge exceeding the predetermined amount is generated in the photoelectric conversion element 300, the reset unit 301 resets the photoelectric conversion element 300. By such control, the photoelectric conversion element 300 can be reset in response to the amount of charge generated.

[0022] In addition to the method of resetting according to the amount of generated charge, a method of resetting the photoelectric conversion element 300 when a predetermined time has elapsed may be used. In this method, the reset unit 301 resets the photoelectric conversion element 300 when a predetermined time has elapsed, regardless of whether or not charge has been generated in the photoelectric conversion element 300. Therefore, the method of resetting according to the elapsed time has a different effect from resetting according to the amount of generated charge. In this embodiment, the control of resetting the photoelectric conversion element 300 according to the generation of a predetermined amount of charge and the control of resetting the photoelectric conversion element 300 when a predetermined time has elapsed may be used in combination.

[0023] The potential of the cathode of the photoelectric conversion element 300 is input to the inverter circuit 302. The inverter circuit 302 can invert the input potential and output it. The inverter circuit 302 can shape the change in potential caused by the presence or absence of a photon incident on the photoelectric conversion element 300 into a detection pulse. As described above, when a photon is incident on the photoelectric conversion element 300, the potential of the cathode of the photoelectric conversion element 300 decreases. Since the cathode of the photoelectric conversion element 300 is connected to the inverter circuit 302, when the potential of the cathode is higher than the threshold of the inverter circuit 302, the output of the inverter circuit 302 becomes a low level. On the other hand, when the potential of the cathode is lower than the threshold of the inverter circuit 302, the output of the inverter circuit 302 becomes a high level. In other words, the input potential can be binarized and output by the inverter circuit 302. As a result, a rectangular detection pulse can be output from the inverter circuit 302 in response to the incidence of light on the photoelectric conversion element 300.

[0024] A counter unit that counts the number of detection pulses is provided at the output of the inverter circuit 302. In FIG. 3, the counter unit is shown as a counter 303. The counter 303 counts the number of detection pulses output from the inverter circuit 302 for a certain period of time and outputs the accumulated count value to an output line. In other words, the counter 303 is a count unit that changes the count value when it receives a detection pulse from the inverter circuit 302. Here, as described above, the inverter circuit 302 outputs a detection pulse in response to the incidence of light on the photoelectric conversion element 300. In other words, a signal generated by the photoelectric conversion element 300 is input to the counter 303 via the inverter circuit 302. The counter 303 performs a counting operation that changes the count value in response to a signal from the photoelectric conversion element. As a result, the counter 303 can count periodically, and therefore the counter can count the number of times photons are incident on the photoelectric conversion element 300 per unit time.

[0025] The counter 303 is controlled to count or not depending on whether light is incident on the photoelectric conversion element 300. Such an operation can change the count value depending on the signal from the photoelectric conversion element. The count value of the counter 303 does not change when no charge is generated in the photoelectric conversion element 300. The count value obtained by counting by the counter 303 for a predetermined period of time can be used as an imaging signal or an event detection signal.

[0026] If the photoelectric conversion element is a SPAD, the counter 303 can change the count value when a single photon is incident on the photoelectric conversion element. If the photoelectric conversion element is a charge-accumulation diode, the counter can change the count value when a predetermined number of charges are generated in the photoelectric conversion element by the incidence of light. This type of operation is generally called photon counting. With photon counting, the amount of charge multiplied by the amount of signal charge generated by the incidence of light can be made very large, so that the signal-to-noise ratio of the output signal to the incident signal can be improved. In addition, since A / D conversion is no longer necessary, signal calculation processing becomes easier.

[0027] The number of bits of the counter 303 can be N bits (N is an integer equal to or greater than 2). The counter 303 can output binary code. For example, when the number of bits of the counter is 3 bits, the count value changes according to the count as "000", "001", "010", "011", and so on.

[0028] Data processing in the pixel block 100 according to this embodiment will be described with reference to Fig. 4. The pixel block 100 may include a pixel unit 30, a determination unit 40 that performs event detection, and an output interface (IF) unit 406. The determination unit 40 may include an adder unit 401, a memory unit 400, a subtractor unit 403, a first control unit 402, a second control unit 404, and a comparator unit 405.

[0029] The adder 401 can add up the count values ​​output by the counters 303 of multiple pixel units 30 and output the sum. The memory unit 400 can store a reference value for event detection and the sum added by the adder. The subtracter 403 subtracts the reference value stored in the memory unit 400 from the output from the counter 303 of the event detection pixel 201. The subtracter 403 can also subtract the reference value from the sum stored in the memory unit 400. The reference value may be, for example, the count value of the counter 303 of the event detection pixel 201 when a change in the event immediately before the current event detection was detected.

[0030] The first control unit 402 controls the adder unit 401 and the pixel unit 30, and can control the stopping of addition and the reading of data from the pixel unit 30. The second control unit 404 stores a threshold value for event detection and can change the threshold value. The comparator unit 405 can compare the output of the subtractor unit 403 with the threshold value from the second control unit 404 to detect an event and output an event detection signal. The output IF unit 406 can output the imaging signal from the pixel unit 30 and the event detection signal from the comparator unit 405 from the pixel block 100. Also, it can control the output of the imaging signal in response to the event detection signal.

[0031] Next, the event detection operation according to this embodiment will be described with reference to Fig. 4. There are two modes for event detection. The first mode is a mode in which event detection is performed without adding the count value from the event detection pixel 201. The second mode is a mode in which event detection is performed by adding the count value from the event detection pixel 201. The processing for each mode is performed in the pixel block 100.

[0032] The first mode will be described. The event detection pixel 201 is controlled by an event synchronization signal. The counter 303 of the event detection pixel 201 counts during the period of the event synchronization signal, and the count result is read out under the control of the event synchronization signal. The read output of the counter 303 is input to a subtraction unit 403. The subtraction unit 403 calculates the difference between a reference value stored in the storage unit 400 and the output of the counter 303 of the event detection pixel 201. Difference data corresponding to the difference is input to a comparison unit 405 and compared with a threshold value output from a second control unit 404, and the comparison result is output from the comparison unit 405.

[0033] Here, the count value output by the counter 303 of the event detection pixel 201 when the immediately preceding event was detected can be used as the reference value. In this case, the reference value can be updated to the count value of the event detection pixel 201 corresponding to the newly detected event every time an event is detected. As described above, the reference value can be updated after the calculation by the subtraction unit 403 is performed.

[0034] The imaging pixel 200 in the pixel unit 30 is controlled by an imaging synchronization signal, and the output of the counter 303 of the imaging pixel 200 is input to the output IF unit 406. The count value of the counter 303 of the imaging pixel 200 can be used as an imaging signal. The imaging synchronization signal is a signal used to read out a signal from the imaging pixel 200, and it may have a speed that can realize a typical frame rate of a moving image, such as 60 FPS or 120 FPS.

[0035] The event synchronization signal is a signal that reads out a count value for event detection from the event detection pixel 201. The event synchronization signal can be a signal corresponding to a cycle of event detection. The frequency of event detection should be higher than the frame rate. For this purpose, the event synchronization signal should be faster than the imaging synchronization signal, and may be 1000 FPS. In this case, the first unit time during which the counter 303 of the event detection pixel 201 counts the detection pulses may be shorter than the second unit time during which the counter 303 of the imaging pixel 200 counts the detection pulses. In this case, the counters 303 of the imaging pixel 200 and the event detection pixel 201 may be counters with the same bit width. Alternatively, when the counter 303 of the imaging pixel is set as a first counter and the counter 303 of the event detection pixel 201 is set as a second counter, the bit width that can be counted by the second counter may be smaller than the bit width that can be counted by the first counter.

[0036] The second control unit 404 may store multiple thresholds. Usually, by having two thresholds, a positive threshold and a negative threshold, it is possible to determine the light / dark direction of the luminance change corresponding to the event change. For example, it is possible to detect that the subject has become brighter when the threshold is greater than the positive threshold, and that the subject has become darker when the threshold is less than the negative threshold. The output of the comparison unit 405 is output to the post-processing unit 12 via the output IF unit 406. In addition to the two thresholds, positive and negative, multiple thresholds can be set and the thresholds can be changed to set the event detection sensitivity corresponding to the target of the event detection.

[0037] When the comparison unit 405 detects a luminance change exceeding a threshold, it is determined that an event has occurred, and the comparison unit 405 outputs an event detection signal. The output IF unit 406 receives the event detection signal and can output the output of the imaging pixel 200 from the pixel block 100. The reference value in the storage unit 400 is updated by the second control unit 404. The update is performed, for example, by replacing the previous reference value with the value of the event detection signal when the event is detected. The reference value can also be updated in a different manner. For example, every time an event detection operation controlled by an event synchronization signal is performed, the previous reference value may be replaced with the count value output from the counter 303 of the event detection pixel 201 in association with the current event detection, and updated.

[0038] Next, a second mode in which event detection is performed by adding up count values ​​from the event detection pixels 201 will be described. The first control unit 402 controls the adder unit 401 to add up the outputs from the pixel counters 303. Specifically, since there are four event detection pixels 201 in the example of FIG. 2(a), the addition is performed by adding up the count values ​​output from each of the four event detection pixels 201. The addition may be performed not only for the event detection pixels 201 but also for the imaging pixels 200. The result of the addition can be stored in the memory unit 400. The addition of the imaging pixels 200 is performed for the imaging pixel R shown in FIG. 2(a). IJ , G IJ , B IJ The event detection can be performed for each color of the event detection pixel E. IJ In the case of , it can be performed for the event detection pixels arranged in the pixel block 100. The addition is performed by adding the output of the counter of each pixel in the adder. The addition is performed by adding the result of addition of pixels arranged with the same color filter as X ADD The output of the counter for the pixel in row i and column j is Y ij In the case of the arrangement shown in FIG. 2(a), the result of addition of the pixels with red filters is R ADD =R 11 +R 13 +R 31 +R 33 Similarly, the result of adding the pixels with green filters is G ADD=G 12 +G 14 +G 32 +G 34 The result of adding the pixels with blue filters is B ADD =B 22 +B 24 +B 42 +B 44 Similarly, the result of adding the event detection pixels is E ADD =E 21 +E 23 +E 41 +E 43 As shown in the figure.

[0039] When signals from four event detection pixels are added, the data width increases by up to 2 bits, so if necessary, the result of the addition may be shifted 2 bits to the right and averaged by dividing by 4. In this case, the same value can be used whether or not the reference value is added. The signal-to-noise ratio can be improved by adding multiple pixels. Imaging pixel 200R IJ , G IJ , B IJ For the event detection pixel E, we have reduced the roughness and noise of the image signal. IJ In addition, the effect of the addition can be improved by excluding the count values ​​from defective pixels during the addition.

[0040] Next, conditions for performing addition by the first control unit 402 will be described. Performing addition is advantageous in the following cases. The first condition is when the subject has low illuminance. Addition improves the signal-to-noise ratio, and can reduce noise in the case of an image for capture, and can reduce false event detection in the case of an event detection pixel. The second condition is when the photoelectric conversion element becomes hot. Addition can reduce the number of circuits driven inside the subtraction unit 403 and the comparison unit 405, thereby suppressing heat generation associated with operation. Furthermore, in addition to the same effect as the first condition, the second condition can also achieve a reduction in the dark count rate (DCR) when the photoelectric conversion element is a SPAD.

[0041] The third condition is when the frequency of event occurrence is low. By reducing false detections through addition and performing event-driven operation that operates only when an event occurs, low power consumption can be achieved. The fourth condition is when the frequency of event occurrence is high, and power can be reduced by reducing the number of circuits driven through addition. In addition, the frame rate can be increased to improve subject tracking. The fifth condition is when the contrast of the incident light from the subject is low, and the problem of edges in high contrast areas becoming dull through addition can be reduced. The sixth condition is due to changes in the power supply environment of the photoelectric conversion device. For example, when the battery level is low or the external power supply is lost, the number of circuits driven through addition can be reduced to extend the operating time.

[0042] The control of the adder 401 may be performed collectively for all pixel blocks 100 arranged in the photoelectric conversion device, or may be performed for each pixel block 100. Regarding the first condition to the fifth condition, the first control unit 402 in the pixel block 100 may have a determination function, or the determination may be performed by a pixel control unit 11 outside the pixel block. In addition, the pixel control unit 11 may have a function to exclude defective pixels from the addition target when such pixels are present.

[0043] Next, an effective use of the memory capacity of the storage unit 400 will be described. When one pixel block is made up of 4×4 pixels as shown in Fig. 2(a), in the first mode in which addition is not performed, a memory for at least four pixels is required for the reference values ​​for event detection in each of the four event detection pixels 201. This is because the reference values ​​are updated when an event is detected, and therefore each different event detection pixel requires its own reference value.

[0044] On the other hand, in the second mode in which the imaging pixel 200 and the event detection pixel 201 are added, one pixel of memory is required for a reference value for event detection corresponding to the event detection pixel after addition. In addition, four pixels of memory are required to store the results of adding the imaging pixels 200 indicated by R, G, and B and the event detection pixel 201 indicated by E. Therefore, a total of at least five pixels of memory are required. In the case of the pixel arrangement shown in FIG. 2(a), the memory capacities required for the first mode and the second mode are four pixels and five pixels, respectively, and since the required capacities are close to each other, it can be said that the memory utilization efficiency is good.

[0045] 2(b), when one pixel block is made up of 16 × 16 pixels, the memory required in the second mode is the same as in the 4 × 4 example, requiring four pixels for the imaging pixels 200 of each color and the event detection pixel 201. In addition, one pixel is required for the reference value of the event detection pixel 201, so a total of five pixels' worth of memory is required.

[0046] In the example of Fig. 2(b), there are 16 event detection pixels 201, so in the first mode, a memory for 16 pixels is required to store reference values ​​for each of the 16 event detection pixels 201. For this reason, more memory is unused in the first mode than in the second mode, and memory utilization efficiency is reduced. Taking these factors into consideration, in terms of memory utilization efficiency, when the pixel block is 16 x 16 pixels, it is recommended that the event detection pixels in one pixel block be four pixels as shown in Fig. 2(c), and that a memory for five pixels be prepared.

[0047] Furthermore, the length of the data from the event detection pixels 201 can be made shorter in accordance with the event detection cycle relative to the length of the data from the imaging pixels 200. Therefore, by making the bit width of the storage unit that holds the data from the event detection pixels 201 smaller relative to the bit width of the storage unit that holds the data from the imaging pixels 200, it is possible to further improve memory utilization efficiency.

[0048] Next, the use of the remaining memory in the second mode will be described with reference to Fig. 5. When one pixel block is composed of 4 × 2 pixels as shown in Fig. 2(d), the second mode requires a memory for five pixels as in Fig. 2(b), whereas the first mode requires only a memory for two pixels for the thresholds of the two event detection pixels 200 indicated by E. This results in a remaining memory of three pixels, which is five pixels minus two pixels. By using this remaining memory and writing the counted value of the event detection signal into the remaining memory at a cycle three times that of the event synchronization signal to operate the event detection, it is possible to improve the time resolution of the event detection.

[0049] An example of utilizing the remaining memory of three pixels is described below. A counter unit having multiple counters that count detection pulses per first unit time at different timings is provided in one event detection pixel 201. Three counters a, b, and c are provided to perform three counts with different timings. A counter unit that counts detection pulses per second unit time is provided in the imaging pixel 200. The second unit time should be a time equivalent to the period of an imaging synchronization signal. The count value counted by the counter 303 in the second unit time can become an imaging signal.

[0050] In this example, the first unit time is 1 / 3 of the second unit time. The first unit time is the period of the event synchronization signal. Counter a, counter b, and counter c count detection pulses at different timings over the first unit time. When an event is detected according to the count value of any of counter a, counter b, and counter c, an event detection signal is output from the comparison unit 405. The output IF unit 406 to which the event detection signal is input outputs an imaging signal from the pixel block 100 at a timing corresponding to the detected event. By utilizing the remaining memory in the second mode and performing event detection at a period shorter than the imaging period and at a shifted timing, the temporal resolution of event detection can be improved.

[0051] Even when a counter unit having multiple counters is arranged in one event detection pixel 201, the counters of each counter unit count detection pulses at the same timing for the multiple event detection pixels 201 arranged in the pixel block 100. In the second mode, it is preferable to add up the values ​​counted at the same timing.

[0052] Next, the configuration of the photoelectric conversion device of this embodiment will be described with reference to FIG. 6. The photoelectric conversion device may be configured by stacking a plurality of substrates. In this case, the photoelectric conversion device may be divided into a substrate 501 including a photoelectric conversion element and a substrate 502 including other circuit parts, and the substrates 501 and 502 may be stacked. By arranging the photoelectric conversion element on one side of the substrate 501 and arranging the counter, pixels, and circuits common to the entire device on the other substrate 502, it is possible to increase the light receiving area of ​​the photoelectric conversion element in a plan view while realizing high speed circuit operation. The number of stacked substrates is not limited to two, but may be three or more, and the functions assigned to each substrate are not limited to the configuration of this embodiment.

[0053] (Control of photoelectric conversion element) An example of controlling the operation of the photoelectric conversion element 300 when the photoelectric conversion element 300 is configured as an avalanche photodiode will be described mainly with reference to FIG. 7. In this example, a PMOS transistor is shown as the quenching element of the reset unit 301. The first control unit 402 applies a predetermined fixed bias potential V bias or a count disable signal CNT_DIS. The count disable signal CNT_DIS is a signal of a potential that can cut off the quenching element of the reset unit 301. When the first control unit 402 supplies the CNT_DIS signal to the quenching element, the quenching element is turned off, and control is performed so that no current is supplied from the second power supply voltage Vd to the avalanche photodiode. This control makes it possible to disable the operation of the photoelectric conversion element and thin out the pixels from which signals are read out.

[0054] Power consumption can be reduced by thinning out pixels from which signals are read out and not reading or processing signals from the thinned out pixels. Control to reduce power consumption by thinning out pixels is effective when the temperature of the photoelectric conversion element becomes high, when it is desired to reduce power consumption when the frequency of event occurrence is low or high, when the battery level is low, or when the external power supply is lost. In addition to the above effects, the precision of event detection can be improved by prioritizing thinning out defective pixels.

[0055] (Application of photoelectric conversion devices to equipment) Hereinafter, an apparatus 1000 including a semiconductor device 1100 including a package 1020 on which a semiconductor chip 1110 including a semiconductor integrated circuit is mounted as shown in FIG. 8 will be described. The semiconductor chip 1110 is accommodated in the package 1020 and mounted on the apparatus 1000. In the configuration shown in FIG. 8, the semiconductor chip 1110 includes a photoelectric conversion device according to the above-mentioned embodiment. The semiconductor device 1100 can include a package 1020 including a base 1010 to which the semiconductor chip 1110 is fixed and a light-transmitting member 1030 such as glass facing the semiconductor chip 1110. In the package 1020, a bonding member such as a wire or a bump that connects an inner lead provided on the base 1010 to a terminal such as a pad electrode provided on the semiconductor chip 1110 can be arranged.

[0056] The device 1000 may include at least one of an optical device 1040, a control device 1050, a processing device 1060, a display device 1070, a storage device 1080, and a mechanical device 1090. The optical device 1040 is, for example, a lens, a shutter, or a mirror. The control device 1050 controls a semiconductor chip 1110. The control device 1050 is, for example, a semiconductor device such as an ASIC.

[0057] The processing device 1060 processes an output signal from a photoelectric conversion device included in the semiconductor chip 1110. The processing device 1060 is a semiconductor device such as a CPU or ASIC for configuring an AFE analog front end or a DFE digital front end. For example, an image may be generated based on an imaging signal when an event is detected. The display device 1070 is an EL display device or a liquid crystal display device that displays an information image obtained by the semiconductor chip 1110. The storage device 1080 is a magnetic device or a semiconductor device that stores the information image obtained by the semiconductor chip 1110. The storage device 1080 is a volatile memory such as an SRAM or a DRAM, or a non-volatile memory such as a flash memory or a hard disk drive.

[0058] The mechanical device 1090 has a moving part or a propulsion part such as a motor or an engine. In the device 1000, the signal output from the semiconductor chip 1110 is displayed on the display device 1070, or transmitted to the outside by a communication device (not shown) included in the device 1000. For this purpose, the device 1000 may further include a storage device 1080 and a processing device 1060 in addition to the memory circuit and arithmetic circuit included in the semiconductor chip 1110. The mechanical device 1090 may be controlled based on the signal output from the semiconductor chip 1110.

[0059] The device 1000 is suitable for electronic devices such as information terminals having a photographing function, such as smartphones, wearable terminals, and cameras, such as interchangeable lens cameras, compact cameras, video cameras, and surveillance cameras. The mechanical device 1090 in the camera may be a device capable of driving components of the optical device 1040 for zooming, focusing, and shutter operation. Alternatively, the mechanical device 1090 in the camera may be a device capable of moving the optical device 1040 for vibration isolation operation.

[0060] The device 1000 may be a transport device such as a vehicle, a ship, or an aircraft. The mechanical device 1090 in the transport device may be used as a moving device. The device 1000 as a transport device is suitable for transporting the semiconductor chip 1110 or for assisting and / or automating driving by using a photographing function. The processing device 1060 for assisting and / or automating driving can perform processing for operating the mechanical device 1090 as a moving device based on information obtained by the semiconductor chip 1110. Alternatively, the device 1000 may be a medical device such as an endoscope, a measuring device such as a distance measuring sensor, an analytical device such as an electron microscope, an office machine such as a copier, or an industrial device such as a robot.

[0061] (Other embodiments) (Item 1) A plurality of first pixels; A plurality of event detection pixels; A determination unit that detects an event; an interface unit that outputs signals from the plurality of first pixels in response to the event detection, each of the plurality of event detection pixels includes a first event detection unit that outputs a signal in response to incidence of light, and a first counter unit that counts the signal output from the first event detection unit per first unit time; The photoelectric conversion device, wherein the determination unit performs the event detection based on a sum of outputs from the first counter units of the plurality of event detection pixels. (Item 2) The photoelectric conversion device described in item 1, characterized in that each of the multiple first pixels has a second event detection unit that outputs a signal in response to incidence of light, and a second counter unit that counts the signals output from the second event detection unit per second unit time. (Item 3) 3. The photoelectric conversion device according to item 2, wherein the first unit time is shorter than the second unit time. (Item 4) 4. The photoelectric conversion device according to item 2 or 3, wherein each of the plurality of first pixels is provided with a red color filter, a blue color filter, or a green color filter. (Item 5) The photoelectric conversion device described in item 4, characterized in that, in response to the event detection, the outputs of the second counter units of the pixels in which a color filter of the same color is arranged among the first pixels are summed and output to the interface unit. (Item 6) 6. The photoelectric conversion device according to item 5, wherein the determination unit further includes a memory unit configured to store a sum of outputs of the first counter units of the plurality of event detection pixels, a sum of outputs of second counter units of pixels among the first pixels in which a color filter of the same color is arranged, and a reference value for the event detection. (Item 7) 7. The photoelectric conversion device according to any one of items 1 to 6, wherein four of the event detection pixels are arranged in the pixel block. (Item 8) The photoelectric conversion device described in any one of items 1 to 7, characterized in that the first counter unit has a plurality of counters, and each of the plurality of counters counts the signal from the first event detection unit at a different timing. (Item 9) 9. The photoelectric conversion device according to any one of items 1 to 8, characterized in that the output of the first counter unit of a predetermined event detection pixel among the plurality of event detection pixels is excluded from the total. (Item 10) 10. The photoelectric conversion device according to any one of items 1 to 9, further comprising a control unit, the control unit controlling the device to stop the summing. (Item 11) The photoelectric conversion device described in item 10, characterized in that the control unit controls to stop the summation depending on at least one of the frequency of incident light, the illuminance of the incident light, the temperature of the first event detection unit, the contrast of the incident light, and the status of the supplied power supply. (Item 12) A plurality of first pixels; A plurality of event detection pixels; A determination unit that detects an event; a control unit for controlling an operation of each of the plurality of event detection pixels; and Each of the plurality of event detection pixels includes an avalanche photodiode. The photoelectric conversion device according to the present invention, wherein the control unit is capable of disabling the operation of the avalanche photodiode. (Item 13) The photoelectric conversion device described in item 12, characterized in that each of the plurality of event detection pixels has the avalanche photodiode and a quenching element connected to the avalanche photodiode, and the control unit controls the bias voltage supplied to the quenching element. (Item 14) The photoelectric conversion device according to any one of items 1 to 13, and a processing device that processes an output signal from the photoelectric conversion device.

[0062] The invention is not limited to the above-described embodiments, and various modifications and variations are possible without departing from the spirit and scope of the invention. Accordingly, the following claims are appended to apprise the public of the scope of the invention. [Explanation of symbols]

[0063] 100 pixel block section, 30 pixel section, 40 decision section, 400 memory section, 401 adder section, 402 first control section, 403 subtracter section, 404 second control section, 405 comparator section, 406 output IF section

Claims

1. A plurality of first pixels; A plurality of event detection pixels; A determination unit that detects an event; an interface unit that outputs signals from the first pixels in response to the event detection, each of the plurality of event detection pixels includes a first event detection unit that outputs a signal in response to incidence of light, and a first counter unit that counts the signal output from the first event detection unit per first unit time; The photoelectric conversion device according to claim 1, wherein the determination unit performs the event detection based on a sum of outputs from the first counter units of the plurality of event detection pixels.

2. 2. The photoelectric conversion device according to claim 1, wherein each of the plurality of first pixels has a second event detection unit that outputs a signal in response to incidence of light, and a second counter unit that counts the signal output from the second event detection unit per second unit time.

3. 3. The photoelectric conversion device according to claim 2, wherein the first unit time is shorter than the second unit time.

4. 3. The photoelectric conversion device according to claim 2, wherein each of the plurality of first pixels is provided with a red color filter, a blue color filter, or a green color filter.

5. The photoelectric conversion device according to claim 4, characterized in that, in response to the event detection, the outputs of the second counter units of the pixels among the first pixels in which a color filter of the same color is arranged are summed and output to the interface unit.

6. The photoelectric conversion device according to claim 5, characterized in that the judgment unit further has a memory unit that stores a sum of outputs of the first counter units of the plurality of event detection pixels, a sum of outputs of the second counter units of pixels among the first pixels in which a color filter of the same color is arranged, and a reference value for the event detection.

7. The photoelectric conversion device according to claim 6 , wherein the plurality of event detection pixels are arranged in a pixel block by four.

8. 2. The photoelectric conversion device according to claim 1, wherein the first counter section includes a plurality of counters, each of which counts the signal from the first event detection section at a different timing.

9. 2 . The photoelectric conversion device according to claim 1 , wherein an output of the first counter unit of a predetermined event detection pixel among the plurality of event detection pixels is excluded from the total.

10. The photoelectric conversion device according to claim 1 , further comprising a control unit, the control unit controlling the device to stop the summing.

11. The photoelectric conversion device according to claim 10, characterized in that the control unit controls to stop the summation depending on at least one of the frequency of incidence of light, the illuminance of the incident light, the temperature of the first event detection unit, the contrast of the incident light, and the status of the supplied power supply.

12. A plurality of first pixels; A plurality of event detection pixels; A determination unit that detects an event; a control unit for controlling an operation of each of the plurality of event detection pixels; and Each of the plurality of event detection pixels includes an avalanche photodiode. The photoelectric conversion device according to the present invention, wherein the control unit can disable the operation of the avalanche photodiode.

13. The photoelectric conversion device according to claim 12, wherein each of the plurality of event detection pixels has the avalanche photodiode and a quenching element connected to the avalanche photodiode, and the control unit controls a bias voltage supplied to the quenching element.

14. The photoelectric conversion device according to claim 1 or 12, and a processing device that processes an output signal from the photoelectric conversion device.

Images