Image pickup element and image pickup device

Abstract

An imaging element and an imaging device are provided that are capable of estimating the brightness of a pixel even in the case of a dark pixel. [Solution] An imaging element is provided that includes a photoelectric conversion unit that converts light into electric charges, a counting unit that counts the number of times the accumulated amount of electric charges converted by the photoelectric conversion unit exceeds a threshold, and a control unit that, if the counting result read out from the counting unit does not meet a predetermined value, calculates the intensity of light incident on the photoelectric conversion unit based on the number of times the counting result is read out from the counting unit until the predetermined value is met.

Description

[Technical Field]

[0001] The present invention relates to an imaging element and an imaging device. [Background technology]

[0002] 2. Description of the Related Art In an image sensor having a plurality of pixels, a configuration is known in which a pulse is output when the accumulated amount of charge generated in response to incident light on a pixel reaches a threshold value (see, for example, FIG. 1 of Patent Document 1). Patent Document 1 Patent No. 5269456

[0003] In the above-described image sensor, the amount of accumulated charge in a dark pixel may not reach a threshold value, making it difficult to estimate the brightness of the pixel. Summary of the Invention

[0004] In a first aspect of the present invention, an imaging element is provided that includes a photoelectric conversion unit that converts light into electric charges, a counting unit that outputs first information based on the number of times the accumulated amount of electric charges converted by the photoelectric conversion unit exceeds a threshold, and a control unit that outputs second information calculated when the accumulated amount of electric charges converted by the photoelectric conversion unit does not exceed the threshold.

[0005] In a second aspect of the present invention, there is provided an imaging device comprising an imaging element of the first aspect, and a generation unit that generates information of a subject using the first information output from the imaging element and the second information output from the imaging element.

[0006] The above summary of the invention does not list all of the necessary features of the present invention, and subcombinations of these features may also constitute inventions. [Brief explanation of the drawings]

[0007] [Figure 1] 1 is a diagram showing an overview of an image sensor 100 according to an embodiment of the present invention. [Figure 2]2 is a block diagram showing an example of the configuration of an individual processing unit 212 and a control unit 300 according to the embodiment of the present invention. FIG. [Figure 3A] FIG. 2 is a diagram illustrating an example of a circuit configuration of an individual processing unit 212 according to the embodiment of the present invention. [Figure 3B] FIG. 10 is a diagram showing another example of the circuit configuration of the individual processing unit 212 according to the embodiment of the present invention. [Figure 4] 3 is a diagram illustrating an example of the operation of the individual processing unit 212 illustrated in FIG. 2. [Figure 5] 3 is a diagram illustrating an example of the operation of the control unit 300 illustrated in FIG. 2. [Figure 6] 1 is a diagram showing an example of a cross section of an image sensor 100 according to an embodiment of the present invention. [Figure 7] FIG. 1 is a diagram showing an overview of an image sensor 100 according to another embodiment of the present invention. [Figure 8] FIG. 10 is a diagram showing an example of the circuit configuration of a photoelectric conversion unit 202 and an individual processing unit 212 according to yet another embodiment of the present invention. [Figure 9] FIG. 1 is a block diagram showing an example of the configuration of an imaging device 500 according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0008] The present invention will be described below through embodiments of the invention, but the following embodiments do not limit the scope of the invention according to the claims. Furthermore, not all of the combinations of features described in the embodiments are necessarily essential to the solution of the invention.

[0009] 1 is a diagram showing an overview of an image sensor 100 according to an embodiment of the present invention. The image sensor 100 generates image data according to incident light from a subject. The image sensor 100 includes a light receiving unit 200, a signal processing unit 210, and a control unit 300.

[0010] The light receiving unit 200 has a plurality of photoelectric conversion units 202. In this example, the plurality of photoelectric conversion units 202 are arranged in a row and column direction. Each photoelectric conversion unit 202 generates an electric charge in response to incident light from a subject. The photoelectric conversion unit 202 has a photoelectric conversion element such as a photodiode.

[0011] The signal processing unit 210 generates a pixel signal according to the amount of charge generated by each photoelectric conversion unit 202. The signal processing unit 210 has a plurality of individual processing units 212. An individual processing unit 212 is provided for each photoelectric conversion unit 202.

[0012] Each individual processing unit 212 generates a digital pixel signal according to the amount of charge generated by the corresponding photoelectric conversion unit 202. The multiple individual processing units 212 may read out the analog amounts of charge generated by the multiple photoelectric conversion units 202 in parallel. This allows a global shutter to be realized.

[0013] The control unit 300 controls each of the individual processing units 212 from outside the individual processing units 212. In this example, one control unit 300 is provided for each of the individual processing units 212, and is connected to each of the individual processing units 212 via a bus.

[0014] As will be described in detail later, the image sensor 100 has a layered structure in which multiple chips are stacked. Fig. 1 shows the functional relationships between the light receiving unit 200, signal processing unit 210, and control unit 300 in the image sensor 100, but does not necessarily correspond to the positional relationships in the layered structure. For example, at least some of the components of the individual processing unit 212 are arranged on a different layer from the photoelectric conversion unit 202, and the control unit 300 is arranged on an even different layer from the multiple photoelectric conversion units 202 and individual processing units 212.

[0015] 2 is a block diagram showing an example of the configuration of an individual processing unit 212 and a control unit 300 according to an embodiment of the present invention. The individual processing unit 212 of this example has a detection unit 214, a reset unit 216, a counting unit 218, a memory 220, and a flag 222. When the photoelectric conversion unit 202 receives incident light, an electric charge is generated in accordance with the incident light. The electric charge generated by the photoelectric conversion unit 202 is accumulated in the photoelectric conversion unit 202, etc.

[0016] The detection unit 214 detects the charge accumulation state in each photoelectric conversion unit 202. The detection unit 214 of this example detects that the amount of accumulated charge in the corresponding photoelectric conversion unit 202 has exceeded a threshold. The detection unit 214 of this example receives an accumulation voltage Vin indicating the amount of accumulated charge in the corresponding photoelectric conversion unit 202 and a threshold voltage Vth corresponding to the threshold. When the accumulation voltage Vin becomes smaller than the threshold voltage Vth, the detection unit 214 of this example determines that the amount of accumulated charge has exceeded the threshold and outputs a detection signal. The detection unit 214 of this example resets, via the reset unit 216, the accumulation state of a photoelectric conversion unit 202 whose accumulation state satisfies a predetermined condition, and maintains the accumulation state of a photoelectric conversion unit 202 whose accumulation state does not satisfy the predetermined condition.

[0017] When a detection signal is input from the detection unit 214, the reset unit 216 resets the amount of accumulated charge in the corresponding photoelectric conversion unit 202. For example, the reset unit 216 resets the amount of accumulated charge by connecting the region that accumulates the charge generated by the corresponding photoelectric conversion unit 202 to a reference potential. Even after the reset, the amount of accumulated charge increases again due to the charge generated by the photoelectric conversion unit 202. In other words, the amount of accumulated charge is repeatedly increased and reset. On the other hand, the reset unit 216 does not reset the amount of accumulated charge unless a detection signal is input from the detection unit 214, and therefore the accumulated state of the photoelectric conversion unit 202 is maintained.

[0018] The counting unit 218 counts the number of times the amount of accumulated charge in the corresponding photoelectric conversion unit 202 exceeds a threshold value. In this example, the counting unit 218 counts the number of times a detection signal is input from the detection unit 214. The counting unit 218 operates based on a synchronization pulse signal of a clock 304, which will be described later. In this example, the counting unit 218 outputs information related to the count value to the memory 220 at each elapse of a readout period. The readout period is a period in which the adjustment unit 302, which will be described later, reads out the count value.

[0019] The memory 220 stores information relating to the count values ​​as pixel signals of the corresponding photoelectric conversion units 202. One memory 220 may be provided for each of the individual processing units 212. The memory 220 may operate based on a synchronization pulse signal of a clock 304, which will be described later. For example, the memory 220 may output information relating to the stored count values ​​to the control unit 300 at the end of each readout period.

[0020] Flag 222 is set when a detection signal is input to counting unit 218. For example, flag 222 has an initial value of 0, and is set to 1 when a detection signal is input from detection unit 214 to counting unit 218 and the count value is incremented. Flag 222 in this example may be a 1-bit borrow flag. Alternatively, instead of providing flag 222, 1 bit of memory 220 may be used as the flag.

[0021] The control unit 300 of this example includes an adjustment unit 302 , a clock 304 , a period counter 306 , an upper memory 308 , and a lower memory 310 .

[0022] The clock 304 generates a synchronization pulse signal. The clock 304 outputs the generated synchronization pulse signal to the counting unit 218 and the period counter 306. The clock 304 may output the generated synchronization pulse signal to the memory 220.

[0023] The cycle counter 306 counts the number of elapsed readout cycles determined in advance from the start of charge accumulation. A readout cycle is a cycle in which the adjustment unit 302 reads out the counted value. In this example, the cycle counter 306 counts the number of elapsed readout cycles in accordance with a synchronization pulse signal input from the clock 304. The number of elapsed readout cycles is equal to the number of times the adjustment unit 302 has read out the counted value from the start of charge accumulation. The cycle counter 306 outputs the number of elapsed readout cycles to the upper memory 308 each time a readout cycle elapses.

[0024] The upper memory 308 stores the number of elapsed readout cycles input from the cycle counter 306. The lower memory 310 acquires information relating to the count value of the detection signal from the memory 220 of each photoelectric conversion unit 202 at the elapse of each readout cycle, and stores the information relating to the count value for each photoelectric conversion unit 202. The lower memory 310 may be provided for each photoelectric conversion unit 202.

[0025] The adjustment unit 302 reads out the count values ​​of each photoelectric conversion unit 202 in accordance with a predetermined readout cycle. In this example, the adjustment unit 302 acquires information about the count values ​​of each photoelectric conversion unit 202 from the lower memory 310 at each readout cycle. Alternatively, the adjustment unit 302 may acquire information about the count values ​​from the memory 220 of each photoelectric conversion unit 202 at each readout cycle. The adjustment unit 302 calculates the intensity of light incident on each photoelectric conversion unit 202 based on the readout count values.

[0026] Furthermore, the adjustment unit 302 of this example adjusts the period of the synchronization pulse signal output by the clock 304 in accordance with information related to the count value obtained from the memory 220. The adjustment unit 302 may adjust the period of the synchronization pulse signal by directly obtaining information stored in the upper memory 308 and the lower memory 310. The adjustment unit 302 may also reset the count value of the counter 218 and clear the value of the flag 222, and may control the reset unit 216 to reset the amount of accumulated charge.

[0027] 3A is a diagram showing an example of the circuit configuration of the individual processing unit 212 according to the embodiment of the present invention, including the photoelectric conversion unit 202.

[0028] The detection unit 214 of this example has a CMOS inverter. The CMOS inverter has a PMOS transistor 232 and an NMOS transistor 234. The input terminal of the CMOS inverter is connected to the output terminal of the photoelectric conversion unit 202. The charge generated by the photoelectric conversion unit 202 is accumulated in a parasitic capacitance or the like between the photoelectric conversion unit 202 and the detection unit 214. An accumulation voltage Vin corresponding to the accumulated charge is input to the CMOS inverter.

[0029] The output of the CMOS inverter transitions from a first logic value (logic L in this example) to a second logic value (logic H in this example) when the accumulated voltage Vin becomes smaller than the threshold voltage of the CMOS inverter (i.e., when the amount of accumulated charge exceeds the threshold). The edge at which the output of the detection unit 214 transitions from logic L to logic H functions as the detection signal described above. The threshold voltage of the CMOS inverter corresponds to the threshold voltage Vth described above.

[0030] The reset unit 216 has a reset transistor 230. The source and drain of the reset transistor 230 are provided between the output terminal of the photoelectric conversion unit 202 and a predetermined high-voltage reference potential. The gate of the reset transistor 230 is connected to the output terminal of the CMOS inverter. When the output of the CMOS inverter transitions from logic L to logic H, the reset transistor 230 connects the wiring between the photoelectric conversion unit 202 and the detection unit 214 to the high-voltage reference potential. This resets the accumulated charge, and the output of the detection unit 214 transitions to logic L.

[0031] When the output of the detection unit 214 transitions from logic L to logic H, the counter unit 218 increments the count value and sets the flag 222 to 1. At the end of each read cycle, the counter unit 218 stores the count value at that time in the memory 220 and clears the value of the flag 222. With this configuration, the individual processing unit 212 can be implemented with a small circuit scale.

[0032] 3B is a diagram showing another example of the circuit configuration of the individual processing unit 212 according to the embodiment of the present invention, which also shows the photoelectric conversion unit 202.

[0033] The detection unit 214 of this example has a comparator 236. The comparator 236 has a first input terminal connected to the output terminal of the photoelectric conversion unit 202, a second input terminal connected to a reference voltage line, and an output terminal connected to the reset unit 216 and the counter unit 218. An accumulation voltage Vin corresponding to the amount of accumulated charge generated by the photoelectric conversion unit 202 is input to the first input terminal. The comparator 236 compares the accumulation voltage Vin with a reference voltage Vref and outputs the comparison result.

[0034] Similar to the detection unit 214 of FIG. 3A, the detection unit 214 of this example outputs a binary signal of L logic or H logic depending on whether the accumulated voltage Vin is greater than the reference voltage Vref. The edge at which the output of the detection unit 214 transitions from L logic to H logic functions as the detection signal described above. The reference voltage Vref of the comparator 236 corresponds to the threshold voltage Vth described above. The operations of the reset unit 216, the counting unit 218, and the flag 222 that receive the output of the detection unit 214 are the same as those of FIG. 3A, and therefore will not be described here.

[0035] Fig. 4 is a diagram showing an example of the operation of individual processing unit 212 shown in Fig. 2. In Fig. 4, the horizontal axis represents time, and the vertical axis represents the intensity of accumulated voltage Vin. In this example, accumulated voltage Vin decreases as the amount of accumulated charge increases.

[0036] When light is incident on the corresponding photoelectric conversion unit 202, charge accumulation begins (t=0). While the photoelectric conversion unit 202 receives incident light, the accumulated voltage Vin decreases at a rate that corresponds to the intensity of the incident light. When the accumulated voltage Vin becomes equal to or lower than the threshold voltage Vth (t=t1), the detection unit 214 outputs a detection signal.

[0037] In response to the detection signal, the reset unit 216 resets the amount of charge accumulated in the corresponding photoelectric conversion unit 202. This returns the accumulated voltage Vin to a predetermined initial voltage. In addition, in response to the detection signal, the counter unit 218 increments the count value and sets the flag 222 to 1.

[0038] When a predetermined time has elapsed after the reset by the reset unit 216, the accumulation voltage Vin becomes a predetermined reset voltage Vrst. Then, the accumulation voltage Vin decreases again depending on the intensity of incident light at the photoelectric conversion unit 202. Then, when the accumulation voltage Vin becomes equal to or lower than the threshold voltage Vth, the detection unit 214 outputs a detection signal. This operation is repeated from the start of charge accumulation until a predetermined specified period has elapsed. The predetermined specified period may be set arbitrarily, for example, depending on the time that incident light is incident on the image sensor 100. The upper memory 308 may store the length of the specified period.

[0039] The amount of incident light within the specified period corresponds to the value obtained by multiplying the difference between the threshold voltage Vth and the reset voltage Vrst by the count value. The memory 220 may store this multiplied value, or may store the count value and the voltage difference in association with each other. Furthermore, the amount of incident light within the specified period can be converted to the amount of incident light per unit time by dividing it by the length of the specified period. The memory 220 may also store the amount of incident light per unit time. Furthermore, if the reset voltage Vrst is known, the threshold voltage Vth may be stored instead of the voltage difference.

[0040] FIG. 5 is a diagram illustrating an example of the operation of the control unit 300 shown in FIG. 2. Here, the upper row schematically illustrates the output status of the detection signal from the photoelectric conversion unit 202A when the intensity of incident light is high, and the lower row schematically illustrates the output status of the detection signal from the photoelectric conversion unit 202B when the intensity of incident light is low. The rectangles represent the detection signal output. The horizontal axis represents the elapsed time from the start of charge accumulation and the number of predetermined readout cycles. As an example, the readout cycle is set to 100 ms, the specified period is set to 1 s, and the minimum number of detectable charges (the amount of accumulated charge that can output a detection signal) is set to 10. Furthermore, the maximum count number of the counter 218 is set to 100. The adjustment unit 302 reads out each count value at each readout cycle, i.e., at t=T1 (100 ms), T2 (200 ms), ..., T10 (1 s).

[0041] First, we will explain the operation of the control unit 300 with respect to the photoelectric conversion unit 202A shown in the upper part of Fig. 5. As shown in the figure, in the first readout period (t = 0 to T1), the detection unit 214 outputs a detection signal 10 times. Therefore, at t = T1, the count value is 10, and the flag 222 is set to 1.

[0042] At t=T1, the upper memory 308 stores 1 as the number of elapsed read cycles input from the cycle counter 306. The lower memory 310 acquires information relating to the count value of the detection signal in the memory 220 and the value of the flag 222.

[0043] The adjustment unit 302 acquires the value of the flag 222. Since the value 1 of the flag 222 indicates that the count value is 1 or greater, the adjustment unit 302 reads out the count value 10 acquired by the lower memory 310. Alternatively, the adjustment unit 302 may read out the count value from the lower memory 310 without reading out the value of the flag 222.

[0044] If the count value at the time of readout is equal to or greater than a predetermined value, the adjustment unit 302 calculates the intensity of light incident on the photoelectric conversion unit 202 for each readout cycle. In this example, the predetermined value is set to 1, and the count value read out from the photoelectric conversion unit 202A at t=T1 is 10. Therefore, the adjustment unit 302 sets the count value 10 read out at t=T1 as the intensity of light incident on the photoelectric conversion unit 202A in the first readout cycle.

[0045] The adjustment unit 302 resets the count value of a photoelectric conversion unit whose read count value is equal to or greater than a predetermined value. In this example, the predetermined value is set to 1, and the adjustment unit 302 resets the count value of the counter 218 of the photoelectric conversion unit 202A whose read count value is equal to or greater than 1. In addition, the adjustment unit 302 clears the value of the flag 222. Furthermore, the adjustment unit 302 may or may not reset the amount of accumulated charge by controlling the reset unit 216.

[0046] Furthermore, the lower memory 310 stores the cumulative value of the read count values ​​and the intensity of incident light for each read cycle for each photoelectric conversion unit. In this example, the lower memory 310 stores the cumulative value 10 of the count values ​​of the photoelectric conversion unit 202A and the intensity 10 of incident light for the first read cycle at t=T1.

[0047] The control unit 300 repeats the same operation in subsequent readout cycles. In the example of Fig. 5, the count value 10 read out at t = T2 is also equal to or greater than the predetermined value. The adjustment unit 302 sets the count value 10 read out at t = T2 as the intensity of light incident on the photoelectric conversion unit 202A in the second readout cycle (t = T1 to T2). In this way, the adjustment unit 302 calculates the intensity of incident light for each readout cycle.

[0048] If the cumulative value stored in the lower memory 310 is 100 at the end of the specified period (t=T10), it means that the detection signal has been output 100 times. The brightness of a pixel is calculated from the number of charges output per second. For a pixel corresponding to the photoelectric conversion unit 202A, the number of charges output per second is 10 (minimum number of detected charges) × 100 (cumulative value) = 1,000.

[0049] In this way, if the read count value is equal to or greater than a predetermined value, the count value is reset every time a read cycle elapses, so that even a counter with a small number of digits will not overflow and the circuit can be made smaller.

[0050] Next, the operation of the control unit 300 with respect to the photoelectric conversion unit 202B shown in the lower part of Fig. 5 will be described. As shown in the figure, no detection signal is output during the first readout period (t = 0 to T1). Therefore, at t = T1, the count value is 0, and the value of the flag 222 remains 0.

[0051] At t=T1, the upper memory 308 stores 1 as the number of elapsed read cycles input from the cycle counter 306. The lower memory 310 acquires information relating to the count value of the detection signal in the memory 220 and the value of the flag 222.

[0052] The adjustment unit 302 acquires the value of the flag 222. Since the value 0 of the flag 222 indicates that the count value is 0, the adjustment unit 302 may omit reading the count value acquired by the lower memory 310. The flag 222 is maintained at 0.

[0053] If the count value does not reach a predetermined value during readout, the adjustment unit 302 calculates the intensity of light incident on the photoelectric conversion unit 202 based on the number of readouts until a count value equal to or greater than the predetermined value is read out and the cumulative value of the count values ​​read out until a count value equal to or greater than the predetermined value is read out. At t=T1, the measured value does not reach the predetermined value, so the intensity of light incident on the first readout cycle is not calculated at this point.

[0054] Furthermore, the adjustment unit 302 does not reset the accumulation amount of the photoelectric conversion unit 202 whose read count value does not reach a predetermined value, but maintains it at least until the next readout. The adjustment unit 302 does not reset the accumulation amount of the photoelectric conversion unit 202B whose read count value does not reach 1, but maintains it at least until the next readout (t=T2).

[0055] As shown in the figure, in the photoelectric conversion unit 202B, a detection signal is output once from the detection unit 214 in the second readout period (t=T1 to T2). Therefore, at t=T2, the count value is 1, and the flag 222 is set to 1.

[0056] At t=T2, the upper memory 308 stores 2 as the number of elapsed read cycles input from the cycle counter 306. The lower memory 310 acquires the count value of the detection signal in the memory 220 and the value of the flag 222.

[0057] The adjusting unit 302 acquires the value of the flag 222. The value 1 of the flag 222 indicates that the count value is 1 or greater, so the adjusting unit 302 reads out the count value 1 acquired by the lower memory 310.

[0058] The adjustment unit 302 calculates the intensity of light incident on the photoelectric conversion unit 202B based on the number of reads 2 until the count value 1 is read out and the cumulative value 1 of the count values ​​read out until the count value is read out. In this example, the adjustment unit 302 divides the cumulative value 1 of the count values ​​in the first read cycle and the second read cycle by the number of reads 2, and obtains 0.5 as the intensity of light incident on the photoelectric conversion unit 202B in the first read cycle and the second read cycle.

[0059] The adjustment unit 302 resets the count value of the counter 218 of the photoelectric conversion unit 202B whose read count value is equal to or greater than 1. At the same time, the adjustment unit 302 clears the value of the flag 222.

[0060] Furthermore, the lower memory 310 stores the cumulative value of the read count values ​​and the intensity of incident light for each readout cycle for each photoelectric conversion unit. In this example, at t=T2, the lower memory 310 stores the cumulative value of the count values ​​of the photoelectric conversion unit 202B, 1, and the intensity of incident light, 0.5, for each of the first readout cycle and the second readout cycle.

[0061] 5, after the count value is read out at the previous time t=T2, the count values ​​in the third readout period (t=T2 to T3), the fourth readout period (t=T3 to T4), and the fifth readout period (t=T4 to T5) are 0, 0, and 1, respectively. At t=T5, the adjustment unit 302 divides the cumulative value 1 of the count values ​​read out after t=T2 until the count value is read out by the number of readouts, 3, to obtain 0.33 as the intensity of light incident on the photoelectric conversion unit 202B in the third readout period, the fourth readout period, and the fifth readout period.

[0062] If the cumulative value stored in the lower memory 310 is 5 at the end of the specified period (t=T10), for example, it means that the detection signal has been output five times. The brightness of a pixel is calculated from the number of charges output per second. For the pixel corresponding to the photoelectric conversion unit 202B, the number of charges output per second is the minimum number of detected charges (10) × cumulative value (5) = 50.

[0063] As described above, for the photoelectric conversion unit 202B in which the count value read out at the end of a read cycle does not reach a predetermined value, the adjustment unit 302 in this example calculates the intensity of incident light for each read cycle based on the number of reads until a count value equal to or greater than the predetermined value is read out and the cumulative value of the count values ​​read out until a count value equal to or greater than the predetermined value is read out.

[0064] In a photoelectric conversion unit 202 in which the intensity of incident light is low, there may be readout cycles in which the count value is 0 during a specified period. For example, in the case of a moving image, if the count value during a readout cycle is treated as the intensity of incident light during that readout cycle, periods in which the intensity of incident light is 0 and periods in which it is not 0 may continue, resulting in flickering. Therefore, in this example, if the count value during readout does not reach a predetermined value, the adjustment unit 302 distributes the cumulative value of the count value evenly among the readout cycles, thereby preventing flickering of the moving image due to variations in the count value between readout cycles.

[0065] As described above, the adjustment unit 302 in this example does not reset the amount of accumulated charge in the photoelectric conversion unit 202B when the readout count value does not reach a predetermined value at time t=T1, but maintains the amount at least until the next readout (t=T2). When the intensity of incident light is low, as in the case of the photoelectric conversion unit 202B, even if the count value is 0 at the time of readout, the accumulated amount may not be 0, and a small amount of accumulated charge may still be present. If such a small amount of accumulated charge is periodically reset, it becomes even more difficult to make the accumulated amount reach the minimum number of detectable charges and output a detection signal.

[0066] In this example, such a small amount of accumulated charge can be maintained over multiple readout cycles without being periodically reset. This allows the intensity of incident light to be determined even when the intensity of the incident light is low. That is, for pixels with low incident light intensity that do not reach the signal charge amount required for pulse output within the specified readout time, a signal amount corresponding to the intensity can be detected by extending the accumulation time beyond the specified time, thereby expanding the dynamic range on the low-illuminance side. This allows an image corresponding to the intensity of the incident light to be acquired without causing so-called blackout. Note that if the intensity of the incident light is extremely low and the accumulated charge does not reach the threshold even after repeated readouts up to the maximum number of times, i.e., until the end of the specified period, the adjustment unit 302 may reset the accumulated charge at the end of the specified period.

[0067] In this example, the predetermined value is set to 1, but the predetermined value may be 2 or a larger value. For example, if the predetermined value is set to 2, the adjustment unit 302 does not reset the accumulation amount of a photoelectric conversion unit whose readout count value is 1, but maintains the accumulation amount at least until the next readout. Thereafter, when the readout count value becomes 2, the adjustment unit 302 resets the count value.

[0068] As described above, the brightness of a pixel can be calculated from the number of charges output per second. Therefore, the brightness of each pixel can be calculated based on the cumulative value of each count stored in the lower counter and the number of elapsed readout cycles and the length of the readout cycle stored in the upper counter. In this way, the adjustment unit 302 can calculate the brightness of the corresponding photoelectric conversion unit based on the cumulative value, the length of the readout cycle, and the number of elapsed readout cycles, and determine the brightness of the pixel.

[0069] The readout period is preferably set according to the strongest incident light to prevent overflow of the counting unit 218. The adjustment unit 302 may change the readout period during the specified period. For example, the adjustment unit 302 reads out the count value according to the readout period and controls the readout period according to the characteristics of the incident light. The adjustment unit 302 may lengthen or shorten the readout period according to the intensity of the incident light obtained from the detection result of the AE sensor.

[0070] Alternatively, the adjustment unit 302 may shorten the read cycle when the read count exceeds a predetermined number of times. The predetermined number of times may be the maximum count of the counter 218. For example, if the maximum count of the counter 218 is 100 and the count in one read cycle exceeds 100, the memory 220 sets a 1-bit carry flag. In response to the carry flag being set, the adjustment unit 302 may adjust the cycle of the synchronization pulse signal of the clock 304 to shorten the next read cycle so that the count in the next read cycle is 100 or less. In this way, the adjustment unit 302 in this example can autonomously control the read cycle.

[0071] 3B, the adjustment unit 302 may reduce the reference voltage Vref of the comparator 236 when the read count value exceeds a predetermined number of times. This reduces the detection sensitivity of the detection unit 214 and prevents overflow of the counting unit 218. Alternatively, the adjustment unit 302 may reduce the reference voltage Vref of the comparator in conjunction with shortening the read cycle.

[0072] 6 is a diagram showing an example of a cross section of an image sensor 100 according to an embodiment of the present invention. In this example, a back-illuminated image sensor 100 is shown, but the image sensor 100 is not limited to the back-illuminated type. The image sensor 100 of this example includes an image sensor chip 113 that outputs a signal corresponding to incident light, a signal processing chip 111 that processes the signal from the image sensor chip 113, and a memory chip 112 that stores image data processed by the signal processing chip 111. The image sensor chip 113, the signal processing chip 111, and the memory chip 112 are stacked and may be referred to as a first chip, a second chip, and a third chip, respectively.

[0073] The chips are bonded together by bonding between bumps 109 made of conductive material such as Cu formed on each chip and by bonding between oxide film layers formed on the top surfaces of the chips. The bonding between bumps 109 electrically connects the chips to each other.

[0074] The imaging element 100 is formed by bonding the imaging chip 113, the signal processing chip 111, and the memory chip 112 in a wafer state before they are separated into chips, and then dicing the bonded wafer.

[0075] When bonding wafers together, an activation device scans the wafer surfaces with plasma to activate the bonding surfaces of the wafers. The wafers with activated surfaces are bonded together by hydrogen bonds, van der Waals bonds, covalent bonds, and the like that are formed by contact, forming a laminated substrate. If the two wafers are hydrogen bonded by contact with each other, after forming the laminated substrate, the laminated substrate is placed in a heating device such as an annealing furnace and heated to form covalent bonds between the wafers.

[0076] The term "activation" includes treating the bonding surface of at least one of the substrates so that when the bonding surface of one wafer comes into contact with the bonding surface of another wafer, hydrogen bonds, van der Waals bonds, covalent bonds, etc. are generated, resulting in solid-state bonding without melting. In other words, activation includes generating dangling bonds (unbonded hands) on the wafer surface, thereby making it easier to form bonds.

[0077] More specifically, in an activation device, oxygen gas, which is a process gas, is excited to form plasma in a reduced-pressure atmosphere, and oxygen ions are irradiated onto the surfaces that will become the bonding surfaces of the two substrates. For example, if the wafer is a substrate with an SiO film formed on Si, the irradiation of this oxygen ion breaks the SiO bonds on the wafer surfaces that will become the bonding surfaces during lamination, forming dangling bonds of Si and O. The formation of such dangling bonds on the wafer surfaces is sometimes referred to as activation.

[0078] When a substrate with dangling bonds is exposed to the atmosphere, for example, moisture in the air binds to the dangling bonds, and the substrate surface becomes covered with hydroxyl groups (OH groups). The substrate surface becomes more likely to bond with water molecules, i.e., more likely to become hydrophilic. In other words, activation results in the substrate surface becoming more likely to become hydrophilic. Furthermore, in solid-state bonding, the presence of impurities such as oxides at the bonding interface, as well as defects at the bonding interface, affect the bonding strength. Therefore, cleaning the bonding surface can be considered part of the activation process.

[0079] Furthermore, the wafer may be activated by applying pure water or the like to the surface of the wafer to be bonded using a hydrophilizing device (not shown). This hydrophilization makes the wafer surface in a state where OH groups are attached, i.e., terminated with OH groups.

[0080] By heating the laminated substrate, the bumps 109 on each of the bonding surfaces of the two wafers are integrated with each other, forming an electrical connection between the wafers. By forming the bumps 109 from a material that melts at a low temperature, such as indium or a tin-silver alloy, the laminated substrate can be reflow-treated at a low temperature of 200°C or less. Alternatively, if the bumps 109 are made of a conductive metal such as copper, they expand during the heat treatment, causing the bumps 109 between the wafers to press together and bond by solid-state diffusion.

[0081] The imaging chip 113 is provided with a plurality of photoelectric conversion units 202. The signal processing chip 111 is provided with at least a portion of the configuration of the signal processing unit 210. For example, the imaging chip 113 is provided with a detection unit 214 and a reset unit 216 in each individual processing unit 212, the signal processing chip 111 is provided with a counting unit 218, a memory 220, and a flag 222 in each individual processing unit 212, and the memory chip 112 is provided with a control unit 300.

[0082] The memory 220 may be provided in the memory chip 112 or in the signal processing chip 111. The control unit 300 may be provided in a region of the signal processing chip 111 other than the regions vertically corresponding to the multiple photoelectric conversion units 202. Alternatively, the control unit 300 may be disposed externally without being stacked, and may be electrically connected only to the multiple individual processing units 212. In this way, by distributing the components other than the multiple photoelectric conversion units 202 mainly in the signal processing chip 111 or the memory chip 112, the area of ​​the conversion region that converts incident light into electric charges can be increased.

[0083] 3A, when the detection unit 214 is a CMOS inverter and the output of the photoelectric conversion unit 202 is directly connected to the input terminal of the CMOS inverter of the detection unit 214, providing the detection unit 214 on the imaging chip 113 can prevent the parasitic capacitance between the photoelectric conversion unit 202 and the detection unit 214 from becoming too large. "Direct connection" refers to, for example, a state in which there is no element, such as a transistor or buffer, that blocks the movement of charge between the output of the photoelectric conversion unit 202 and the input terminal of the CMOS inverter. This can prevent a decrease in the sensitivity of the imaging element 100.

[0084] As shown in the figure, incident light is mainly incident in the direction indicated by the white arrow. In this embodiment, the surface of the imaging chip 113 on which incident light is incident is referred to as the back surface. An example of the imaging chip 113 is a back-illuminated MOS image sensor. The imaging chip 113 corresponds to the light receiving unit 200. A PD (photodiode) layer 106 is arranged on the back surface side of the wiring layer 108. The PD layer 106 has a plurality of PD units 104 arranged two-dimensionally and accumulating charges according to the incident light, and transistors 105 provided corresponding to the PD units 104. The PD units 104 are an example of a photoelectric conversion unit 202.

[0085] A color filter 102 is provided on the incident light side of the PD layer 106 via a passivation film 103. The color filters 102 include multiple types that transmit different wavelength ranges, and have specific arrangements corresponding to the respective PD sections 104. The arrangement of the color filters 102 will be described later. A set of the color filter 102, the PD section 104, and multiple transistors 105 is included in one pixel. The light reception start timing (reset timing) of each pixel is controlled by controlling the on / off of the multiple transistors 105.

[0086] A microlens 101 is provided corresponding to each pixel on the incident light side of the color filter 102. The microlens 101 condenses the incident light toward the corresponding PD section 104.

[0087] The wiring layer 108 has wiring 107 that transmits signals from the PD layer 106 to the signal processing chip 111. The wiring 107 may be multi-layered, and may be provided with passive elements and active elements.

[0088] A plurality of bumps 109 are arranged on the surface of the wiring layer 108. The plurality of bumps 109 are aligned with a plurality of bumps 109 provided on the opposing surface of the signal processing chip 111, and the imaging chip 113 and the signal processing chip 111 are joined as described above, whereby the aligned bumps 109 are electrically connected to each other.

[0089] Similarly, a plurality of bumps 109 are arranged on the opposing surfaces of the signal processing chip 111 and the memory chip 112. These bumps 109 are aligned with each other, and the signal processing chip 111 and the memory chip 112 are joined as described above, whereby the aligned bumps 109 are electrically connected to each other.

[0090] The bonding between the bumps 109 is not limited to bonding between both the bumps and the oxide film as described above, but may also employ micro-bump bonding using solder melting. It is sufficient to provide about one bump 109 per unit block. Therefore, the size of the bumps 109 may be larger than the pitch of the PD sections 104. Furthermore, in a peripheral region other than the imaging region where the pixels are arranged, bumps larger than the bumps 109 corresponding to the imaging region may also be provided.

[0091] The signal processing chip 111 has through-silicon vias (TSVs) 110 that connect circuits provided on the front and back surfaces of the chip to each other. The TSVs 110 are preferably provided in the peripheral region. The TSVs 110 may also be provided in the peripheral region of the imaging chip 113 and the memory chip 112.

[0092] 7 is a diagram showing an overview of an image sensor 100 according to another embodiment of the present invention. The photoelectric conversion units 202 are divided into a plurality of blocks, and the control unit 300 may control readout for each block independently. For example, the photoelectric conversion units 202 are divided into M×N blocks G arranged in rows and columns. 1,1 ~G M,N Generally, adjacent pixels have similar brightness, so by controlling the readout for each block independently, the brightness of the pixel can be determined more accurately.

[0093] The control unit 300 may include a block control unit provided for each block, and each block control unit may control the photoelectric conversion unit belonging to the corresponding block. For example, the control unit 300 may include M×N block control units 300. 1,1 ~300 M,N Block control unit 300 1,1 ~300 M,N represents a block G of a plurality of photoelectric conversion units 202. 1,1 ~G M,N Each block control unit 300 1,1 ~300 M,N is the corresponding block G 1,1 ~G M,N The block control unit 300 controls the photoelectric conversion unit 202 belonging to the block control unit 300. 1,1 ~300 M,N The operation is the same as the operation of the control unit 300 described in relation to FIGS. 1 to 6, and therefore a description thereof will be omitted.

[0094] 8 is a diagram showing an example of the circuit configuration of a photoelectric conversion unit 202 and an individual processing unit 212 according to yet another embodiment of the present invention. The signal processing unit 210 of this example has one individual processing unit 212. That is, one individual processing unit 212 is provided for a plurality of photoelectric conversion units 202.

[0095] For example, each photoelectric conversion unit 202 has a transistor switch 204 between the output terminal and the input side of the individual processing unit 212. When a voltage is applied to the switch 204 of a selected photoelectric conversion unit 202, the selected photoelectric conversion unit 202 and the individual processing unit 212 are electrically connected, and an accumulation voltage Vin corresponding to the accumulated charge is input to the individual processing unit 212.

[0096] The individual processing unit 212 performs the operations described in relation to FIGS. 1 to 6 on the selected photoelectric conversion unit 202. However, the memory 220 stores a count value for each photoelectric conversion unit 202. When a predetermined period has elapsed since the start of charge accumulation for the selected photoelectric conversion unit 202, the reset unit 216 resets the amount of accumulated charge in the selected photoelectric conversion unit 202. When the accumulated amount is reset, a voltage is applied to the switch 204 of the next selected photoelectric conversion unit 202. When a voltage is applied to the switch 204 of the next selected photoelectric conversion unit 202, the next selected photoelectric conversion unit 202 and the individual processing unit 212 are electrically connected, and an accumulation voltage Vin corresponding to the accumulated charge is input to the individual processing unit 212. The individual processing unit 212 repeats the operations described above for all photoelectric conversion units 202.

[0097] In this way, the circuit can be made smaller by having multiple photoelectric conversion units 202 share one individual processing unit 212. Note that although the detection unit 214 in Fig. 8 is shown as a CMOS inverter as in Fig. 3A, it may also be a comparator as in Fig. 3B.

[0098] 9 is a block diagram showing an example of the configuration of an image capturing apparatus 500 according to an embodiment of the present invention. The image capturing apparatus 500 includes a photographing lens 520 as an image capturing optical system, and the photographing lens 520 guides a subject light beam incident along an optical axis OA to the image capturing element 100. The photographing lens 520 may be an interchangeable lens that can be attached to and detached from the image capturing apparatus 500. The image capturing apparatus 500 mainly includes the image capturing element 100, a system control unit 501, a drive unit 502, a photometry unit 503, a work memory 504, a recording unit 505, a display unit 506, and a drive unit 514.

[0099] The photographing lens 520 is composed of a group of optical lenses and focuses the subject light beam from the scene near its focal plane. Note that in Figure 9, the photographing lens 520 is represented by a single virtual lens placed near the pupil.

[0100] The driver 514 drives the photographing lens 520. More specifically, the driver 514 moves the optical lens group of the photographing lens 520 to change the focus position. The driver 514 also drives an iris diaphragm in the photographing lens 520 to control the amount of subject light entering the image sensor 100.

[0101] The drive unit 502 has a control circuit that executes charge accumulation control such as timing control and area control of the image sensor 100 in accordance with instructions from the system control unit 501. The drive unit 502 operates the light receiving unit 200 and signal processing unit 210 of the image sensor 100 as described with reference to Figures 1 to 7. In addition, the operation unit 508 accepts instructions from the photographer using a release button or the like.

[0102] The image sensor 100 is the same as the image sensor 100 described with reference to FIGS. 1 to 8. The image sensor 100 passes pixel signals to an image processing unit 511 in the system control unit 501. The image processing unit 511 generates image data by performing various image processes using a work memory 504 as a workspace. For example, when generating image data in the JPEG file format, a color video signal is generated from a signal obtained using a Bayer array, and then compression processing is performed. The generated image data is recorded in a recording unit 505 and converted into a display signal, which is then displayed on a display unit 506 for a preset time.

[0103] The photometry unit 503 detects the luminance distribution of a scene prior to a series of shooting sequences for generating image data. The photometry unit 503 includes, for example, an AE sensor with approximately one million pixels. The calculation unit 512 of the system control unit 501 receives the output of the photometry unit 503 and calculates the luminance of each region of the scene.

[0104] The calculation unit 512 determines the shutter speed, aperture value, and ISO sensitivity in accordance with the calculated luminance distribution. The image sensor 100 may also serve as the photometry unit 503. The calculation unit 512 also executes various calculations for operating the imaging device 500. Part or all of the drive unit 502 may be mounted on the image sensor 100. Part of the system control unit 501 may be mounted on the image sensor 100.

[0105] Although the present invention has been described above using embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments. It will be apparent to those skilled in the art that various modifications and improvements can be made to the above embodiments. It is clear from the claims that such modifications and improvements can also be included within the technical scope of the present invention.

[0106] It should be noted that the execution order of each process, such as operations, procedures, steps, and stages, in the devices, systems, programs, and methods shown in the claims, specifications, and drawings is not specifically stated as "before," "prior to," etc., and that the processes can be performed in any order unless the output of a previous process is used in a later process. Even if the operational flow in the claims, specifications, and drawings is described using "first," "next," etc. for convenience, this does not mean that the processes must be performed in this order. (Item 1) a plurality of photoelectric conversion units that generate charges in response to incident light; a counting unit that counts the number of times that the amount of accumulated charge in each of the plurality of photoelectric conversion units exceeds a threshold; a control unit that reads out a count value indicating the number of times at a predetermined cycle; Equipped with The control unit If the count value is equal to or greater than a predetermined value at the time of reading, the intensity of the light incident on the photoelectric conversion unit is calculated for each of the periods; If the count value does not reach the predetermined value when read, the image sensor calculates the intensity of light incident on the photoelectric conversion unit based on the number of reads until the count value reaches or exceeds the predetermined value and the cumulative value of the count values ​​read until the count value reaches or exceeds the predetermined value. (Item 2) Item 1. The image sensor according to item 1, wherein the control unit sets a value obtained by dividing the cumulative value of the count value by the number of readouts as the intensity per period. (Item 3) The control unit controls the length of the period in accordance with the intensity of the incident light. Item 1 or 2. The imaging element according to item 1 or 2. (Item 4) The control unit shortens the read cycle when the count value read exceeds a predetermined number of times. 4. The imaging device according to any one of items 1 to 3. (Item 5) the control unit further includes a memory that stores the length of the cycle and the number of elapsed cycles; the memory further stores an accumulated value of the read count values ​​for each photoelectric conversion unit; The control unit calculates the intensity of the light incident on the corresponding photoelectric conversion unit based on the cumulative value, the length of the period, and the number of elapsed periods. Item 5. The imaging device according to any one of items 1 to 4. (Item 6) The plurality of photoelectric conversion units are divided into a plurality of blocks, and the control unit controls the readout independently for each block. 6. The imaging device according to any one of items 1 to 5. (Item 7) The control unit includes a block control unit provided for each block, and the block control unit controls a photoelectric conversion unit belonging to the corresponding block. Item 7. The imaging device according to item 6. (Item 8) a first chip provided with the plurality of photoelectric conversion units; a second chip stacked on the first chip and having the counting unit; a third chip stacked on the second chip and having the control unit provided thereon; and 8. The imaging device according to any one of items 1 to 7, comprising: (Item 9) a plurality of photoelectric conversion units that accumulate charges according to incident light; a detection unit that detects an accumulation state of charge in each of the plurality of photoelectric conversion units; Equipped with The detection unit resets the accumulation state of a photoelectric conversion unit whose accumulation state satisfies a predetermined condition, and maintains the accumulation state of a photoelectric conversion unit whose accumulation state does not satisfy the predetermined condition. Image sensor. (Item 10) a plurality of photoelectric conversion units that generate charges in response to incident light; a counting unit that counts the number of times that the amount of accumulated charge in each of the plurality of photoelectric conversion units exceeds a threshold; a control unit that reads out a count value indicating the number of times; Equipped with the control unit resets the count value of the photoelectric conversion unit when the read count value is equal to or greater than a predetermined value; The control unit maintains the accumulation amount of the photoelectric conversion unit where the read count value does not reach the predetermined value at least until the next read. Image sensor. (Item 11) An imaging device comprising the imaging element according to any one of items 1 to 10. [Explanation of symbols]

[0107] 100 imaging element, 101 microlens, 102 color filter, 103 passivation film, 104 PD section, 105 transistor, 106 PD layer, 107 wiring, 108 wiring layer, 109 bump, 110 TSV, 111 signal processing chip, 112 memory chip, 113 imaging chip, 200 light receiving section, 202 photoelectric conversion section, 204 switch, 210 signal processing section, 212 individual processing section, 214 detection section, 216 reset section, 218 counting section, 220 memory, 222 flag, 230 reset transistor, 232 PMOS transistor, 234 NMOS transistor, 236 comparator, 300 control section, 300 1,1 ~300 M,N Block control unit, 302 adjustment unit, 304 clock, 306 period counter, 308 upper memory, 310 lower memory, 500 imaging device, 501 system control unit, 502 drive unit, 503 photometry unit, 504 work memory, 505 recording unit, 506 display unit, 508 operation unit, 511 image processing unit, 512 calculation unit, 514 drive unit, 520 photographing lens

Claims

[Claim 1] a photoelectric conversion unit that converts light into an electric charge; a counting unit that counts the number of times that the accumulated amount of electric charge converted by the photoelectric conversion unit exceeds a threshold; a control unit that, when the counting result read out from the counting unit does not reach a predetermined value, calculates the intensity of light incident on the photoelectric conversion unit based on the number of times the counting result is read out from the counting unit until the predetermined value is reached; An imaging element comprising:

Images