Photoelectric conversion device

Abstract

To provide a photoelectric conversion device that reduces the possibility of omission of detection of an event.SOLUTION: A photoelectric conversion device has a plurality of pixels each including a photoelectric conversion part generating electric charges according to incident light, an electric charge holding part, a first transfer transistor transferring the electric charges from the photoelectric conversion part to the electric charge holding part, a second transfer transistor transferring the electric charges from the electric charge holding part to a first node, and a third transfer transistor transferring the electric charges from the photoelectric conversion part to a second node. In one of the plurality of pixels, first read-out processing in which a gradation signal indicating a quantity of the incident light is read out based on the electric charges transferred to the first node, and second read-out processing in which an event signal indicating a change in a quantity of the incident light is read out based on the electric charges transferred to the second node, are performed in parallel.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to a photoelectric conversion device. [Background technology]

[0002] Patent Document 1 discloses a sensor device capable of generating an event signal and a grayscale signal. In the sensor device of Patent Document 1, selection of pixels to be read out from an event signal and selection of pixels to be read out from a grayscale signal are performed row-sequentially at different timings. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2021-129265 Summary of the Invention [Problem to be solved by the invention]

[0004] When the period during which an event can be detected is not sufficiently long, an event may go undetected. Therefore, in a photoelectric conversion device capable of reading out an event signal, it may be necessary to reduce the possibility of an event going undetected.

[0005] An object of the present invention is to provide a photoelectric conversion device in which the possibility of missing an event detection is reduced. [Means for solving the problem]

[0006] According to one disclosure of the present specification, there is provided a photoelectric conversion device having a plurality of pixels each including a photoelectric conversion unit that generates a charge in response to incident light, a charge holding unit, a first transfer transistor that transfers charge from the photoelectric conversion unit to the charge holding unit, a second transfer transistor that transfers charge from the charge holding unit to a first node, and a third transfer transistor that transfers charge from the photoelectric conversion unit to a second node, and characterized in that in one of the plurality of pixels, a first readout process is performed in parallel to read out a gradation signal indicating the amount of the incident light based on the charge transferred to the first node, and a second readout process is performed in parallel to read out an event signal indicating a change in the amount of the incident light based on the charge transferred to the second node. Effect of the Invention

[0007] According to the present invention, a photoelectric conversion device is provided in which the possibility of an event being missed is reduced. [Brief description of the drawings]

[0008] [Figure 1] 1 is a block diagram showing a schematic configuration of a photoelectric conversion device according to a first embodiment. [Diagram 2] 1 is a circuit diagram showing an example of the configuration of a pixel according to a first embodiment. [Diagram 3] 4 is a timing chart showing an outline of a method for driving the photoelectric conversion device according to the first embodiment. FIG. [Figure 4] 4 is a timing chart showing a method for driving the photoelectric conversion device according to the first embodiment. FIG. [Diagram 5] FIG. 4 is a schematic diagram showing a schematic configuration of a photoelectric conversion device according to a second embodiment. [Figure 6] FIG. 11 is a circuit diagram showing an example of the configuration of a pixel according to a third embodiment. [Figure 7] FIG. 13 is a circuit diagram showing an example of the configuration of a pixel according to a fourth embodiment. [Figure 8] FIG. 13 is a block diagram of an apparatus according to a fifth embodiment. [Figure 9] FIG. 13 is a block diagram of an apparatus according to a sixth embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. The same elements or corresponding elements in multiple drawings are denoted by the same reference numerals, and the description thereof may be omitted or simplified.

[0010] In the first to fourth embodiments described below, an image pickup device capable of detecting an event will be mainly described as an example of a photoelectric conversion device. However, the photoelectric conversion device in each embodiment is not limited to an image pickup device, and can be applied to other photodetection devices based on photoelectric conversion. Examples of other photodetection devices include a distance measuring device and a photometry device. The distance measuring device can be, for example, a focus detection device, a distance measuring device using TOF (Time-Of-Flight), etc. The photometry device can be a device that measures the amount of light incident on the device.

[0011] [First embodiment]

[0012] 1 is a block diagram showing a schematic configuration of a photoelectric conversion device according to this embodiment. The photoelectric conversion device includes a pixel array 10, a timing control circuit 11, a pixel control circuit 12, a grayscale signal readout circuit 13, and an event signal readout circuit 14.

[0013] The pixel array 10 has a plurality of pixels 100 arranged in a plurality of rows and a plurality of columns, each of which outputs a signal according to incident light by photoelectric conversion. Each of the plurality of pixels 100 has a photoelectric conversion unit that generates and accumulates a signal charge based on the incident light. A microlens and a color filter may be arranged on the pixel 100. Each of the plurality of pixels 100 can output a grayscale signal indicating the amount of incident light and an event signal indicating a change in the amount of incident light. The event signal is a signal generated when an event in which the amount of light changes suddenly in each pixel 100 is detected. This event may be, for example, a local change in the amount of light caused by a change in the position of the subject within the shooting range.

[0014] The timing control circuit 11 supplies the pixel control circuit 12, the grayscale signal read circuit 13, and the event signal read circuit 14 with control signals instructing the operation timing of each of these circuits.

[0015] The pixel control circuit 12 includes a shift register, a gate circuit, a buffer circuit, etc. Based on a control signal from the timing control circuit 11, the pixel control circuit 12 outputs a plurality of control signals for controlling the operation of a plurality of pixels 100 via control lines arranged for each row of the pixel array 10. The control lines are arranged to extend in a first direction (the horizontal direction in FIG. 1). Note that there may be a plurality of control lines per row. In this embodiment, there are six control lines per row.

[0016] In each column of the pixel array 10, output lines VL, VLp, and VLn are arranged extending in a second direction (vertical direction in FIG. 1) intersecting the first direction. Each of the output lines VL, VLp, and VLn is connected to the pixels 100 aligned in the second direction and forms a signal line common to these pixels 100. The output line VL of each column is connected to a grayscale signal readout circuit 13, and the output lines VLp and VLn of each column are connected to an event signal readout circuit 14.

[0017] 2 is a circuit diagram showing an example of the configuration of a pixel 100 according to this embodiment. The pixel 100 has a photoelectric conversion unit PD, a charge holding unit C1, a first transfer transistor M11, a second transfer transistor M12, a reset transistor M2, an amplification transistor M3, a selection transistor M4, and a charge discharging transistor M5. The pixel 100 also has a third transfer transistor M13, a logarithmic conversion unit 101, a subtraction unit 102, and a comparison unit 103.

[0018] Each transistor in the pixel 100 is composed of a MOS transistor. A control signal for controlling each of these transistors is input from the pixel control circuit 12 to the gate of each transistor via a control line.

[0019] The photoelectric conversion unit PD is, for example, a photodiode. The photoelectric conversion unit PD photoelectrically converts incident light and accumulates electric charges generated by the photoelectric conversion. The anode of the photoelectric conversion unit PD is connected to the ground node. The cathode of the photoelectric conversion unit PD is connected to the source of the first transfer transistor M11, the source of the third transfer transistor M13, and the source of the charge discharging transistor M5.

[0020] The drain of the first transfer transistor M11 is connected to the charge holding unit C1 and the source of the second transfer transistor M12. The charge holding unit C1 includes a capacitance component and has a function of holding the charge transferred from the photoelectric conversion unit PD. In Fig. 2, the capacitance component of the charge holding unit C1 is equivalently shown by the circuit symbol of a capacitive element.

[0021] The drain of the second transfer transistor M12 is connected to the source of the reset transistor M2 and the gate of the amplification transistor M3. A node (first node) to which the drain of the second transfer transistor M12, the source of the reset transistor M2, and the gate of the amplification transistor M3 are connected is a floating diffusion FD. The floating diffusion FD includes a capacitance component (floating diffusion capacitance) and has a function of holding the charge transferred from the photoelectric conversion unit PD. The floating diffusion capacitance includes the parasitic capacitance of the electrical path from the second transfer transistor M12 to the amplification transistor M3 via the floating diffusion FD.

[0022] The drain of the reset transistor M2, the drain of the amplifying transistor M3, and the drain of the charge discharging transistor M5 are connected to a power supply potential node Vdd to which a power supply potential is supplied. The source of the amplifying transistor M3 is connected to the drain of the selecting transistor M4. The source of the selecting transistor M4 is connected to an output line VL. A constant current source (not shown) is connected to the output line VL.

[0023] A control signal TX1 is input to the gate of the first transfer transistor M11. The first transfer transistor M11 is controlled based on the control signal TX1, and when turned on, transfers the charge accumulated in the photoelectric conversion unit PD to the charge holding unit C1. The charge holding unit C1 holds the charge transferred from the photoelectric conversion unit PD.

[0024] A control signal TX2(n) is input to the gate of the second transfer transistor M12. The second transfer transistor M12 is controlled by the control signal TX2(n) and, when turned on, transfers the charge held in the charge holding unit C1 to the floating diffusion FD. That is, the amplification transistor M3 functions as an amplifier, and the floating diffusion FD functions as an input node of the amplifier that receives the charge transferred from the charge holding unit C1. Note that the (n) suffixed to the symbol of the control signal indicates the row number of the pixel 100 to which the control signal is input.

[0025] A control signal SEL(n) is input to the gate of the selection transistor M4. The selection transistor M4 is controlled by the control signal SEL(n) and, when turned on, connects the source of the amplification transistor M3 to the output line VL. This causes the amplification transistor M3 and a constant current source (not shown) to function as a source follower. At this time, a grayscale signal based on the potential of the floating diffusion FD is output to the grayscale signal readout circuit 13 via the output line VL.

[0026] A control signal RES is input to the gate of the reset transistor M2. The reset transistor M2 is controlled by the control signal RES, and resets the potential of the floating diffusion FD by being turned on.

[0027] A control signal OFD is input to the gate of the charge discharging transistor M5. The charge discharging transistor M5 is controlled by the control signal OFD, and when turned on, discharges the charge accumulated in the photoelectric conversion unit PD to the power supply potential node Vdd.

[0028] The charge draining transistor M5 is controlled to be turned on or off collectively for all pixels. The charge draining transistor M5 is turned from on to off collectively for all pixels, thereby simultaneously terminating charge draining for all pixels. As a result, the charge draining transistor M5 has a function of controlling the start time of exposure in the global electronic shutter. The global electronic shutter is an operation in which the shutter start and shutter end of a plurality of effective pixels that receive incident light among a plurality of pixels are performed collectively for the plurality of effective pixels. However, when a light-shielded pixel is provided in addition to the plurality of effective pixels, the shutter start and shutter end may be performed collectively for the light-shielded pixel and the plurality of effective pixels. The light-shielded pixel includes both or either of an optical black pixel having a light-shielded photoelectric conversion element and a dummy pixel having no photoelectric conversion element and the same pixel circuit as the effective pixel.

[0029] The first transfer transistor M11 is controlled to be turned on or off collectively for all pixels. The first transfer transistor M11 has a function of transferring the charge accumulated in the photoelectric conversion unit PD to the charge holding unit C1 by turning on and then turning off collectively for all pixels, and controlling the end time of exposure in the global electronic shutter.

[0030] The drain of the third transfer transistor M13 is connected to the input node (second node) of the logarithmic conversion unit 101. A control signal TX3 is input to the gate of the third transfer transistor M13. The third transfer transistor M13 is controlled by the control signal TX3. When the third transfer transistor M13 is on, a photocurrent Ip based on charges generated by photoelectric conversion in the photoelectric conversion unit PD is transferred to the logarithmic conversion unit 101. The photocurrent Ip has a current amount corresponding to the amount of incident light.

[0031] The names of the source and drain of each transistor may differ depending on the conductivity type or the function of interest of each transistor. That is, some or all of the names of the source and drain may be reversed.

[0032] The logarithmic conversion unit 101 generates a potential according to the photocurrent Ip, and converts it into a logarithmic potential Vl by performing a potential conversion in the form of a logarithmic function. This expands the dynamic range. The logarithmic conversion unit 101 outputs the logarithmic potential Vl to the subtraction unit 102. Note that the logarithmic conversion process is not essential.

[0033] The subtraction unit 102 generates a differential potential Vd by subtracting a reference potential Vr (reference signal) from the logarithmic potential Vl. The differential potential Vd is output to the comparison unit 103.

[0034] The comparison unit 103 compares the differential potential Vd with a number of predetermined thresholds and outputs the comparison result as an event signal. The number of thresholds includes two: a positive threshold (first threshold) and a negative threshold (second threshold). When the differential potential Vd is greater than the positive threshold, or when the differential potential Vd is smaller than the negative threshold, an event occurrence is detected. When the differential potential Vd is greater than the positive threshold, the comparison unit 103 outputs an event signal to the event signal readout circuit 14 via the output line VLp. When the differential potential Vd is smaller than the negative threshold, the comparison unit 103 outputs an event signal to the event signal readout circuit 14 via the output line VLn. In this way, the comparison unit 103 operates to output an event signal when the amount of change in the amount of incident light is outside the range of the two thresholds (when an event occurs), and not to output an event signal when the amount of change is within the range of the two thresholds (when an event does not occur). In addition, by performing a comparison using a positive threshold value and a negative threshold value, the comparison unit 103 can output an event in which the amount of light increases and an event in which the amount of light decreases as separate output signals to the event signal readout circuit 14 via different output lines.

[0035] The event signal has a high time resolution because it is a signal obtained at a timing unrelated to the frame synchronization signal in a general frame-based image sensor. The event signal is output to the event signal readout circuit 14 and also to the subtraction unit 102, where it is used to update the reference potential Vr. The subtraction unit 102 holds the logarithmic potential Vl at the time of detecting the event and uses it as the reference potential Vr in the next subtraction process. That is, the subtraction unit 102 has a function of subtracting the reference potential Vr corresponding to the past logarithmic potential from the logarithmic potential Vl input at a certain point in time, thereby generating a difference potential Vd indicating the time change of the signal. With the above configuration, the logarithmic conversion unit 101, the subtraction unit 102, and the comparison unit 103 can detect the increase or decrease in the amount of light as a spike-like event signal.

[0036] As described above, the grayscale signal readout circuit 13 performs a first readout process to read out the grayscale signal output to the output line VL, and the event signal readout circuit 14 performs a second readout process to read out the event signal output to the output lines VLp and VLn. The grayscale signal readout circuit 13 performs signal processing such as analog-to-digital conversion on the grayscale signal, and outputs it to a signal processing circuit (not shown) in the photoelectric conversion device or a signal processing circuit outside the photoelectric conversion device. The event signal readout circuit 14 performs signal processing such as analog-to-digital conversion on the event signal, and outputs it to a signal processing circuit (not shown) in the photoelectric conversion device or a signal processing circuit outside the photoelectric conversion device.

[0037] Fig. 3 is a timing chart showing an outline of the method for driving the photoelectric conversion device according to this embodiment. Fig. 4 is a timing chart showing in more detail the method for driving the photoelectric conversion device according to this embodiment.

[0038] First, an outline of a method for driving a photoelectric conversion device will be described with reference to Fig. 3. Fig. 3 shows the timing at which the levels of control signals supplied to the pixels 100 in the first, second, third, fourth and nth rows in the pixel array 10 change, and the periods for charge accumulation and signal readout.

[0039] Time t1 is the time when charge accumulation for generating the gradation signal starts. After the reset transistor M2 and the charge discharge transistor M5 of each row are turned on, they are turned off at time t1. Time t2 is the time when charge accumulation for generating the gradation signal ends. That is, time t2 is the time when the first transfer transistor M11 of each row is turned off after being temporarily turned on for charge transfer. The period T1 between time t1 and time t2 indicates the charge accumulation period for generating the gradation signal. The charge accumulation period for generating the gradation signal is the same for each row, and the gradation signal is generated by the global electronic shutter.

[0040] After time t2, the second transfer transistor M12 is turned on in row sequence from the first row to the nth row, and a gradation signal is read out (first readout process) based on the charge accumulated in the period T1. Period T2 in FIG. 3 indicates the period for reading out the gradation signal. At time t3, the third transfer transistor M13 in each row is turned on, and event detection and readout of the event signal (second readout process) begins. Period T3 in FIG. 3 indicates the period for detecting the event and reading out the event signal. The period for detecting the event and reading out the event signal is the same for each row. Periods T2 and T3 at least partially overlap. That is, in at least one pixel 100, reading out the gradation signal and detecting the event and reading out the event signal are performed in parallel.

[0041] At time t4, the reset transistor M2 and the charge discharging transistor M5 of each row are turned on and then turned off. The period VD from time t1 to time t4 in FIG. 3 indicates one cycle in which the grayscale signal is read and the event signal is read. Similar operations are repeated after time t4. That is, similar operations are also performed in the period VD from time t4 to time t7 in FIG. 3, so a description thereof will be omitted.

[0042] Next, a method of driving the photoelectric conversion device will be described in more detail with reference to FIG. 4. FIG. 4 shows changes in the levels of each control signal in the vicinity of one period VD (time t1 to time t4) in FIG. 3. That is, FIG. 4 shows changes in the levels of the control signals RES, OFD, TX1, TX3, SEL(1) to SEL(n), and TX2(1) to TX2(n). It is assumed that when each control signal is at a high level, the corresponding transistor is in an on state, and when each control signal is at a low level, the corresponding transistor is in an off state. In addition, the control signals with row numbers (1) to (n) indicate the level of the control signal supplied to the pixel 100 of that row, and the control signals without row numbers indicate the level of the control signal supplied commonly to all rows. In addition, since the levels of the control signal RES and the control signal OFD are the same, the operations of these control signals are collectively shown as one "RES / OFD" in FIG. 4.

[0043] At time t11, the control signal RES goes high. This turns on the reset transistor M2 in the pixels 100 of each row, and the potential of the floating diffusion FD is reset. Also, at time t11, the control signal OFD goes high. This turns on the charge discharging transistor M5 in the pixels 100 of each row, and the charge accumulated in the photoelectric conversion unit PD is discharged to the power supply potential node Vdd. These operations also reset the photoelectric conversion unit PD.

[0044] At time t1, the control signals RES and OFD go to low level. This turns off the reset transistor M2 and the charge discharge transistor M5 in the pixels 100 of each row. This ends the reset of the potential of the floating diffusion FD and the discharge of the charge accumulated in the photoelectric conversion unit PD in the pixels 100 of each row. In this way, in the pixels 100 of each row, charge accumulation for generating a grayscale signal in the photoelectric conversion unit PD starts from time t1.

[0045] At time t12, the control signal TX1 goes to high level, which turns on the first transfer transistor M11 in the pixels 100 in each row, and starts transferring the charges stored in the photoelectric conversion units PD to the charge holding units C1.

[0046] At time t2, the control signal TX1 becomes low level. This turns off the first transfer transistor M11 in the pixels 100 in each row, and the transfer of charges from the photoelectric conversion unit PD to the charge holding unit C1 ends. In this manner, in the pixels 100 in each row, charge accumulation for generating a gradation signal in the photoelectric conversion unit PD ends at time t2. The period T1 between time t1 and time t2 is a charge accumulation period for generating a gradation signal. When the charge transfer is completed at time t2, the photoelectric conversion unit PD returns to its initial state.

[0047] At time t2, the charge holding unit C1 holds the charges transferred from the photoelectric conversion unit PD. During a period T2 from time t2 to time t17, grayscale signals based on the charges are read out row by row from the first row to the nth row.

[0048] At time t13, the control signal SEL(1) goes to high level, which turns on the selection transistor M4 in the first row, selecting the pixels 100 in the first row and enabling them to output grayscale signals.

[0049] At time t14, the control signal TX2(1) goes high. This turns on the second transfer transistor M12 in the first row, and charges are transferred from the charge storage unit C1 in the first row to the floating diffusion FD in the first row. This causes the potential of the output line VL to change according to the transferred charges.

[0050] At time t15, the control signal TX2(1) goes to low level. This turns off the second transfer transistor M12 in the first row, and the charge transfer ends. After that, the grayscale signal is read out from the pixel 100 in the first row to the grayscale signal readout circuit 13 via the output line VL.

[0051] At time t16, the control signal SEL(1) goes to low level. This turns off the selection transistor M4 in the first row, and the pixels 100 in the first row go into a non-selected state. In this manner, the grayscale signals are read from the pixels 100 in the first row. Note that a reset signal may also be read based on the reset state of the floating diffusion FD during the period from time t13 to time t14.

[0052] After time t16, the grayscale signals are read out from the pixels 100 in the second to nth rows in row sequence. The reading out of these grayscale signals is the same as that in the first row, so a description thereof will be omitted. At time t17, the reading out of the grayscale signals from the pixels 100 in the nth row ends. The period T2 from time t2 to time t17 is the period for reading out the grayscale signals.

[0053] At time t3, the control signal TX3 goes to high level. This turns on the third transfer transistor M13 in the pixels 100 in each row, and the photoelectric conversion unit PD and the logarithmic conversion unit 101 become conductive. Therefore, the pixels 100 in each row are in a state where the logarithmic conversion unit 101 can detect a photocurrent Ip corresponding to the charge generated by light incident on the photoelectric conversion unit PD. That is, after time t3, event detection and event signal readout start.

[0054] After time t3, the logarithmic conversion unit 101, the subtraction unit 102, and the comparison unit 103 perform the process of detecting an event and reading out an event signal as follows. The logarithmic conversion unit 101 generates a potential according to the photocurrent Ip and converts it into a logarithmic potential Vl. The subtraction unit 102 generates a differential potential Vd by subtracting a reference potential Vr from the logarithmic potential Vl. The comparison unit 103 compares the differential potential Vd with a positive threshold and a negative threshold. If the differential potential Vd is greater than the positive threshold, the comparison unit 103 outputs an event signal to the event signal readout circuit 14 via the output line VLp. If the differential potential Vd is less than the negative threshold, the comparison unit 103 outputs an event signal to the event signal readout circuit 14 via the output line VLn.

[0055] At time t17, the control signal TX3 goes to low level. As a result, in the pixels 100 of each row, the third transfer transistor M13 is turned off, and the photoelectric conversion unit PD and the logarithmic conversion unit 101 are rendered non-conductive. The period T3 from time t3 to time t17 is a period for detecting an event and reading out an event signal.

[0056] At time t18, the control signal RES goes high. This turns on the reset transistor M2 in the pixels 100 of each row, and the potential of the floating diffusion FD is reset. Also, at time t18, the control signal OFD goes high. This turns on the charge discharging transistor M5 in the pixels 100 of each row, and the charge accumulated in the photoelectric conversion unit PD is discharged to the power supply potential node Vdd. Therefore, the photoelectric conversion unit PD is also reset.

[0057] At time t4, the control signals RES and OFD go to low level. This turns off the reset transistor M2 and the charge discharging transistor M5 in the pixels 100 of each row, and starts charge accumulation for generating a grayscale signal. In this manner, the grayscale signal and the event signal are read out for one period VD from time t1 to time t4. Similar operations are repeated after time t4, so a description thereof will be omitted.

[0058] As described above, the photoelectric conversion device of this embodiment accumulates charges for generating a grayscale signal, reads out the grayscale signal, detects an event, and reads out the event signal, with the period VD from time t1 to time t4 being one cycle. Periods T2 and T3 at least partially overlap. That is, in the period from time t3 to time t17, reading out the grayscale signal, detecting an event, and reading out the event signal are performed in parallel. Therefore, it is possible to secure a long time during which an event can be detected within the period VD. As described above, according to this embodiment, a photoelectric conversion device is provided in which the possibility of missing an event detection is reduced.

[0059] 3 and 4, in this embodiment, the period during which an event can be detected can be made consistent for each pixel 100. This makes it possible to align the event detection periods within the angle of view in which imaging is performed, further reducing the possibility of an event being overlooked.

[0060] When capturing an image in a low-light environment such as a dark place, the amount of information in the grayscale signal may be insufficient. In such a case, image data may be generated by adding multiple signals acquired during multiple periods VD in an adder circuit inside or outside the photoelectric conversion device. This can improve the accuracy of capturing an image in a low-light environment.

[0061] In addition, multiple image data may be generated in which the ratio between the accumulation time of the grayscale signal in one period VD and the time for detecting the event and reading out the event signal is changed, or the image data may be generated with the ratio being variable. Also, the length of one period VD may be variable. With these configurations, appropriate image data according to the amount of light in the environment can be generated.

[0062] In the photoelectric conversion device of this embodiment, during the period in which an event is detected and an event signal is read out, the state is updated asynchronously within the photoelectric conversion device, and the event signal is read out as needed, so that the delay time from the change in light amount to the output of the event signal is short. Also, since an event is often detected in a pixel 100 that corresponds to the vicinity of the edge of a moving subject, the event signal is output sparsely within the pixel array 10. Therefore, signal processing at the time of outputting the event signal is performed sparsely when an event occurs, so that the average power consumption of the photoelectric conversion device of this embodiment is reduced.

[0063] Moreover, the photoelectric conversion unit PD is connected to the arithmetic circuit (logarithmic conversion unit 101, subtraction unit 102, and comparison unit 103) without going through a readout circuit. This allows for low-delay signal processing, and is less susceptible to speed reductions caused by the transmission speed of the readout circuit. In other words, even in an imaging environment where a large number of events are detected, processing delays in the photoelectric conversion device are unlikely to occur. Furthermore, the spatial and temporal resolution of the event signal are coarse due to processing in the arithmetic circuit, reducing the amount of information. Therefore, an interface with a relatively small communication capacity can be applied to transmit the event signal, and power consumption related to transmission is small.

[0064] In addition, the specific configuration of each of the logarithmic conversion unit 101, the subtraction unit 102, and the comparison unit 103 in this embodiment is not particularly limited as long as the same calculation result (including an approximate calculation result) can be obtained. For example, each of the logarithmic conversion unit 101, the subtraction unit 102, and the comparison unit 103 may be an analog computing unit, an asynchronous (clockless) digital computing unit, or a clock-synchronous digital computing unit. Alternatively, each of the logarithmic conversion unit 101, the subtraction unit 102, and the comparison unit 103 may be software-implemented by a processor. Each of the logarithmic conversion unit 101, the subtraction unit 102, and the comparison unit 103 may be a combination of the above-mentioned methods. In addition, the above-mentioned computing unit may be used in a time-division manner to be shared in a plurality of calculation processes. In addition, the order of calculation may be changed in the processing of the logarithmic conversion unit 101, the subtraction unit 102, and the comparison unit 103 shown in FIG. 2. 2, the process of subtracting the reference potential Vr from the logarithmic potential Vl may be replaced with a process of adding the reference potential Vr to a threshold value. Since these processes are equivalent, either one can be applied to this embodiment.

[0065] [Second embodiment] In this embodiment, an example will be described in which the photoelectric conversion device of the first embodiment is a stacked type. Descriptions of elements common to the first embodiment may be omitted or simplified as appropriate.

[0066] Fig. 5 is a schematic diagram showing a schematic configuration of a photoelectric conversion device according to this embodiment. The photoelectric conversion device according to this embodiment is a stacked type photoelectric conversion device having a structure in which a plurality of substrates are stacked. As shown in Fig. 5, the photoelectric conversion device according to this embodiment has a structure in which a first substrate 21 and a second substrate 22 are stacked.

[0067] The first substrate 21 is provided with a pixel array 10 including a plurality of pixels 100 arranged in a plurality of rows and a plurality of columns. The second substrate 22 is provided with a pixel control circuit 12, a grayscale signal readout circuit 13, and an event signal readout circuit 14. The grayscale signals and event signals output from the pixels 100 of the first substrate 21 are read out to the circuits of the second substrate 22. The configurations and operations of the pixels 100, the pixel control circuit 12, the grayscale signal readout circuit 13, and the event signal readout circuit 14 are the same as those of the first embodiment, and description thereof will be omitted. Each of the first substrate 21 and the second substrate 22 is typically a semiconductor substrate mainly containing silicon. Note that the present invention is not limited to this example, and each of the first substrate 21 and the second substrate 22 may be an SOI (Silicon On Insulator) substrate or a gallium arsenide substrate. The first substrate 21 and the second substrate 22 may be substrates mainly containing different materials. For example, the first substrate 21 may be a semiconductor substrate mainly containing silicon, and the second substrate 22 may be a gallium arsenide substrate.

[0068] The wiring layer of first substrate 21 is disposed on the surface bonded to second substrate 22, and the light incident surface of first substrate 21 is the surface opposite to the surface bonded to second substrate 22. That is, the photoelectric conversion device of this embodiment is a so-called back-illuminated type.

[0069] In this embodiment, a photoelectric conversion device is provided that can obtain the same effects as in the first embodiment. Furthermore, in this embodiment, by arranging circuits such as the pixel control circuit 12, the grayscale signal readout circuit 13, and the event signal readout circuit 14 on a substrate different from that of the pixel array 10, it is possible to reduce the area of ​​the photoelectric conversion device while maintaining the area of ​​the pixel array 10.

[0070] 5, the output lines VL, VLp, and VLn are not shown, but they may be arranged on the first substrate 21, or on the second substrate 22, or they may be arranged across both the first substrate 21 and the second substrate 22. In addition, in FIG. 5, the timing control circuit 11 is not shown, but they may be arranged on the first substrate 21, or on the second substrate 22, or they may be arranged outside the photoelectric conversion device.

[0071] [Third embodiment] In this embodiment, a modified example of the stacked photoelectric conversion device of the second embodiment will be described. In this embodiment, a part of the circuit constituting the pixel 100 is arranged on the first substrate 21, and the other part is arranged on the second substrate 22. Descriptions of elements common to the second embodiment may be omitted or simplified as appropriate.

[0072] 6 is a circuit diagram showing an example of the configuration of a pixel 100 according to this embodiment. The photoelectric conversion unit PD, the charge holding unit C1, the first transfer transistor M11, the second transfer transistor M12, the reset transistor M2, the amplification transistor M3, the selection transistor M4, and the charge discharging transistor M5 are arranged on a first substrate 21. The third transfer transistor M13, the logarithmic conversion unit 101, the subtraction unit 102, and the comparison unit 103 are arranged on a second substrate 22.

[0073] The first substrate 21 and the second substrate 22 are joined by a joint 23. The joint 23 electrically connects the cathode of the photoelectric conversion unit PD and the source of the third transfer transistor M13. The joint 23 can be, for example, a Cu-Cu joint.

[0074] The present embodiment also provides the same effects as those of the first and second embodiments. Moreover, the present embodiment can reduce the effect of crosstalk on the charge storage unit C1 caused by the switching operation of the third transfer transistor M13 or fluctuations in the current flowing through the third transfer transistor M13 when an event is detected. This can reduce the effect on the accuracy of the grayscale signal caused by the detection of an event.

[0075] The charge retention portion C1 does not have to be disposed on the first substrate 21, and may be formed in a wiring layer provided between the first substrate 21 and the second substrate 22. In this case, the charge retention portion C1 may be formed by a plurality of metal wirings in the same wiring layer sandwiching an insulating film. The charge retention portion C1 may also be formed by arranging a metal wiring in one wiring layer of a plurality of wiring layers and a metal wiring in another wiring layer so as to sandwich an insulating film therebetween.

[0076] [Fourth embodiment] In this embodiment, another modified example of the stacked photoelectric conversion device of the second embodiment will be described. In this embodiment, the circuits included in the first substrate 21 and the second substrate 22 are different from those in the third embodiment. Descriptions of elements common to the second and third embodiments may be omitted or simplified as appropriate.

[0077] 7 is a circuit diagram showing an example of the configuration of a pixel 100 according to this embodiment. In this embodiment, a charge discharging transistor M5 is disposed on the second substrate 22. The drain of the reset transistor M2 and the drain of the amplification transistor M3 are connected to a power supply potential node Vdd1 of the first substrate 21 to which a power supply potential is supplied. The drain of the charge discharging transistor M5 is connected to a power supply potential node Vdd2 of the second substrate 22 to which a power supply potential is supplied.

[0078] The present embodiment also provides the same effects as the first to third embodiments. Furthermore, the present embodiment can reduce the effect of crosstalk caused by fluctuations in the current flowing through the charge discharging transistor M5 on the charge holding unit C1. This can further reduce the effect on the accuracy of the grayscale signal caused by the detection of an event.

[0079] The combination of the circuits included in each of the first substrate 21 and the second substrate 22 and the position of the junction 23 are not limited to those described in the third and fourth embodiments. For example, the junction 23 may be provided inside the logarithmic conversion unit 101.

[0080] In the second to fourth embodiments, a configuration in which the first substrate 21 and the second substrate 22 are stacked is illustrated, but the number of substrates to be stacked is not limited to this. For example, a configuration in which three or more substrates are stacked may be used.

[0081] [Fifth embodiment] The photoelectric conversion device in the above-described embodiment can be applied to various devices. Examples of the devices include digital cameras, digital camcorders, camera heads, copiers, fax machines, mobile phones, vehicle-mounted cameras, observation satellites, and surveillance cameras. Fig. 8 shows a block diagram of a digital camera as an example of the device.

[0082] The device 70 shown in FIG. 8 includes a barrier 706, a lens 702, an aperture 704, and a photoelectric conversion device 700. The device 70 further includes a signal processing unit (processing device) 708, a timing generating unit 720, an overall control / calculation unit 718 (control device), a memory unit 710 (storage device), a recording medium control I / F unit 716, a recording medium 714, and an external I / F unit 712. At least one of the barrier 706, the lens 702, and the aperture 704 is an optical device corresponding to the device. The barrier 706 protects the lens 702, and the lens 702 forms an optical image of a subject on the photoelectric conversion device 700. The aperture 704 makes the amount of light passing through the lens 702 variable. The photoelectric conversion device 700 is configured as in the above-mentioned embodiment, and converts the optical image formed by the lens 702 into image data (image signal). The signal processing unit 708 performs various corrections, data compression, etc. on the image data output from the photoelectric conversion device 700. The timing generating unit 720 outputs various timing signals to the photoelectric conversion device 700 and the signal processing unit 708. The overall control and calculation unit 718 controls the entire digital camera, and the memory unit 710 temporarily stores image data. The recording medium control I / F unit 716 is an interface for recording or reading image data to the recording medium 714, and the recording medium 714 is a removable recording medium such as a semiconductor memory for recording or reading image data. The external I / F unit 712 is an interface for communicating with an external computer or the like. Timing signals and the like may be input from outside the device. The device 70 may further include a display device (monitor, electronic viewfinder, etc.) that displays information obtained by the photoelectric conversion device. The device includes at least a photoelectric conversion device. The device 70 further includes at least one of an optical device, a control device, a processing device, a display device, a storage device, and a mechanical device that operates based on information obtained by the photoelectric conversion device. The mechanical device is a movable part (for example, a robot arm) that operates by receiving a signal from the photoelectric conversion device.

[0083] Each pixel circuit may include a plurality of photoelectric conversion units (a first photoelectric conversion unit and a second photoelectric conversion unit). The signal processing unit 708 may be configured to process a pixel signal based on the charge generated in the first photoelectric conversion unit and a pixel signal based on the charge generated in the second photoelectric conversion unit, and to acquire distance information from the photoelectric conversion device 700 to a subject.

[0084] [Sixth embodiment] 9(a) and 9(b) are block diagrams of devices related to the vehicle-mounted camera in this embodiment. The device 80 has the photoelectric conversion device 800 of the above-mentioned embodiment and a signal processing device (processing device) that processes a signal from the photoelectric conversion device 800. The device 80 has an image processing unit 801 that performs image processing on a plurality of image data acquired by the photoelectric conversion device 800, and a parallax calculation unit 802 that calculates parallax (phase difference of parallax images) from a plurality of image data acquired by the device 80. The device 80 also has a distance measurement unit 803 that calculates a distance to an object based on the calculated parallax, and a collision determination unit 804 that determines whether or not there is a possibility of collision based on the calculated distance. Here, the parallax calculation unit 802 and the distance measurement unit 803 are examples of distance information acquisition means that acquire distance information to the object. That is, the distance information is information related to the parallax, the defocus amount, the distance to the object, and the like. The collision determination unit 804 may determine the possibility of collision using any of these distance information. The distance information acquisition means may be realized by dedicated hardware, a software module, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), or a combination of these.

[0085] The device 80 is connected to a vehicle information acquisition device 810, and can acquire vehicle information such as vehicle speed, yaw rate, and steering angle. In addition, the device 80 is connected to a control ECU 820, which is a control device that outputs a control signal to generate a braking force for the vehicle based on the judgment result of the collision judgment unit 804. In addition, the device 80 is also connected to an alarm device 830 that issues an alarm to the driver based on the judgment result of the collision judgment unit 804. For example, when the judgment result of the collision judgment unit 804 indicates that there is a high possibility of a collision, the control ECU 820 performs vehicle control to avoid a collision and reduce damage by applying the brakes, releasing the accelerator, suppressing engine output, etc. The alarm device 830 warns the user by sounding an alarm such as a sound, displaying alarm information on the screen of a car navigation system, etc., and applying vibrations to a seat belt or steering wheel. The device 80 functions as a control means that controls the operation of controlling the vehicle as described above.

[0086] In this embodiment, the surroundings of the vehicle, for example, the front or rear, are captured by the device 80. Fig. 9(b) shows the device when capturing an image of the area in front of the vehicle (imaging range 850). A vehicle information acquisition device 810, which serves as an imaging control means, sends an instruction to the device 80 or the photoelectric conversion device 800 to perform an imaging operation. This configuration can further improve the accuracy of distance measurement.

[0087] Although the above describes an example of control to prevent collision with other vehicles, the present invention can also be applied to control of automatic driving by following other vehicles, control of automatic driving to prevent deviation from lanes, etc. Furthermore, the device is not limited to vehicles such as automobiles, but can be applied to moving bodies (moving devices) such as ships, aircraft, artificial satellites, industrial robots, and consumer robots. In addition, the present invention can be applied to devices that use object recognition or biometric recognition, such as intelligent transport systems (ITS) and surveillance systems, in addition to moving bodies.

[0088] [Modified embodiment] The present invention is not limited to the above-described embodiments and can be modified in various ways. For example, an example in which a part of the configuration of any of the embodiments is added to another embodiment, or an example in which a part of the configuration of any of the embodiments is replaced with a part of the configuration of another embodiment, is also an embodiment of the present invention.

[0089] The disclosure of this specification includes the complement of the concepts described in this specification. In other words, if the specification states, for example, that "A is B" (A=B), the specification is deemed to disclose or suggest that "A is not B" even if the statement that "A is not B" (A≠B) is omitted. This is because when it states that "A is B," it is assumed that the case that "A is not B" is taken into consideration.

[0090] The disclosure of this specification includes the following configurations. (Configuration 1) A photoelectric conversion unit that generates charges in response to incident light; A charge holding portion; a first transfer transistor that transfers charges from the photoelectric conversion unit to the charge storage unit; a second transfer transistor that transfers charges from the charge storage unit to a first node; a third transfer transistor that transfers charges from the photoelectric conversion unit to a second node; a plurality of pixels each including In one of the plurality of pixels, a first readout process is performed in parallel to read out a grayscale signal indicating the amount of incident light based on the charge transferred to the first node, and a second readout process is performed in parallel to read out an event signal indicating a change in the amount of incident light based on the charge transferred to the second node. A photoelectric conversion device comprising: (Configuration 2) The second readout process is performed in parallel in each of the plurality of pixels. 2. The photoelectric conversion device according to configuration 1. (Configuration 3) The first readout process is performed in each of the plurality of pixels during a period different from each other. 3. The photoelectric conversion device according to configuration 1 or 2. (Configuration 4) In the one of the plurality of pixels, the second transfer transistor is turned on during a period in which the third transfer transistor is on. 4. The photoelectric conversion device according to any one of configurations 1 to 3. (Configuration 5) Each of the plurality of pixels further includes a charge drain transistor that drains a charge of the photoelectric conversion unit, A period from when the discharge of charges by the charge discharge transistor is completed to when the transfer of charges by the first transfer transistor is completed is common to each of the plurality of pixels. 5. The photoelectric conversion device according to any one of configurations 1 to 4. (Configuration 6) The pixels are arranged across a number of rows and a number of columns, Each of the plurality of pixels further includes a reset transistor that resets a potential of the first node, A period from when the reset of the potential of the first node is completed to when the transfer of charges by the first transfer transistor is completed is common to each of the plurality of pixels. 6. The photoelectric conversion device according to any one of configurations 1 to 5. (Configuration 7) Each of the plurality of pixels further includes an amplifying transistor that outputs a potential corresponding to the charge transferred to the first node. 7. The photoelectric conversion device according to any one of configurations 1 to 6. (Configuration 8) Each of the plurality of pixels further includes a comparison unit that compares a time change of a signal corresponding to the charge transferred to the second node with a threshold and outputs a comparison result as the event signal. 8. The photoelectric conversion device according to any one of configurations 1 to 7. (Configuration 9) The output signal of the comparator is output to different output lines when the time change is greater than a first threshold and when the time change is less than a second threshold that is smaller than the first threshold. 9. The photoelectric conversion device according to configuration 8. (Configuration 10) The second node further includes a logarithmic converter that outputs a logarithm of a signal based on the charge transferred to the second node. 10. The photoelectric conversion device according to configuration 8 or 9. (Configuration 11) The second node further includes a subtraction unit that calculates the time change based on a difference between a signal corresponding to the charge transferred to the second node and a reference signal based on a past event signal. 11. The photoelectric conversion device according to any one of configurations 8 to 10. (Configuration 12) The photoelectric conversion unit is disposed on a first substrate, The gray scale signal and the event signal are read out to a second substrate laminated on the first substrate. 12. The photoelectric conversion device according to any one of configurations 1 to 11. (Configuration 13) A control circuit for controlling the first readout process and the second readout process is disposed on the second substrate. 13. The photoelectric conversion device according to configuration 12. (Configuration 14) The photoelectric conversion unit is disposed on a first substrate, The third transfer transistor is disposed on a second substrate laminated with the first substrate. 12. The photoelectric conversion device according to any one of configurations 1 to 11. (Configuration 15) The charge retaining portion is disposed on the first substrate. 15. The photoelectric conversion device according to configuration 14. (Configuration 16) The charge retaining portion is provided in a wiring layer disposed between the first substrate and the second substrate. 15. The photoelectric conversion device according to configuration 14. (Configuration 17) A junction between the first substrate and the second substrate is disposed at a node between the photoelectric conversion unit and the third transfer transistor. 17. The photoelectric conversion device according to any one of configurations 14 to 16, (Configuration 18) Each of the plurality of pixels further includes a charge drain transistor that drains a charge of the photoelectric conversion unit, The charge draining transistor is disposed on the first substrate. 18. The photoelectric conversion device according to any one of configurations 14 to 17. (Configuration 19) Each of the plurality of pixels further includes a charge drain transistor that drains a charge of the photoelectric conversion unit, The charge draining transistor is disposed on the second substrate. 18. The photoelectric conversion device according to any one of configurations 14 to 17. (Configuration 20) The photoelectric conversion device according to any one of configurations 1 to 19, an optical device corresponding to the photoelectric conversion device; A control device for controlling the photoelectric conversion device; a processing device that processes a signal output from the photoelectric conversion device; a display device that displays information obtained by the photoelectric conversion device; A storage device that stores information obtained by the photoelectric conversion device; and and a mechanical device that operates based on information obtained by the photoelectric conversion device. An apparatus characterized by: (Configuration 21) The processing device processes the image signals generated by the plurality of photoelectric conversion units, and obtains distance information from the photoelectric conversion devices to a subject. 21. The device of claim 20.

[0091] The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) for implementing one or more of the functions.

[0092] It should be noted that the above-mentioned embodiments are merely examples of the implementation of the present invention, and the technical scope of the present invention should not be interpreted as being limited by these embodiments. In other words, the present invention can be implemented in various forms without departing from its technical concept or main features. [Explanation of symbols]

[0093] 100 pixels C1 Charge holding part M11 First transfer transistor M12 Second transfer transistor M13 Third transfer transistor PD Photoelectric conversion unit

Claims

1. a photoelectric conversion unit that generates charges in response to incident light; a charge holding portion; a first transfer transistor that transfers charges from the photoelectric conversion unit to the charge storage unit; a second transfer transistor that transfers charges from the charge storage unit to a first node; a third transfer transistor that transfers charges from the photoelectric conversion unit to a second node; a charge drain transistor that drains charges from the photoelectric conversion unit; a plurality of pixels each including a first readout process in which a gradation signal indicating the amount of incident light is read out based on the charge transferred to the first node, and a second readout process in which an event signal indicating a change in the amount of incident light is read out based on the charge transferred to the second node are performed in parallel in one of the plurality of pixels; a first timing at which the charge discharging transistor finishes discharging the charge is common to each of the plurality of pixels; a second timing at which the first transfer transistor finishes transferring the charge from the photoelectric conversion unit to the charge storage unit is common to each of the plurality of pixels; After the second timing, the third transfer transistors of the plurality of pixels are simultaneously turned on. A photoelectric conversion device characterized by:

2. The second readout process is performed in parallel in each of the plurality of pixels.

2. The photoelectric conversion device according to claim 1.

3. The first readout process is performed in each of the plurality of pixels in a different period.

2. The photoelectric conversion device according to claim 1.

4. In the one of the plurality of pixels, the second transfer transistor is turned on during a period in which the third transfer transistor is on.

2. The photoelectric conversion device according to claim 1.

5. The plurality of pixels are arranged across a plurality of rows and a plurality of columns, Each of the plurality of pixels further includes a reset transistor that resets the potential of the first node; The period from when the reset of the potential of the first node is completed to when the transfer of charges by the first transfer transistor is completed is common to each of the plurality of pixels.

2. The photoelectric conversion device according to claim 1.

6. Each of the plurality of pixels further includes an amplifying transistor that outputs a potential corresponding to the charge transferred to the first node.

2. The photoelectric conversion device according to claim 1.

7. Each of the plurality of pixels further includes a comparison unit that compares a time change in a signal corresponding to the charge transferred to the second node with a threshold and outputs the comparison result as the event signal.

2. The photoelectric conversion device according to claim 1.

8. The output signal of the comparator is output to different output lines when the time change is greater than a first threshold and when the time change is smaller than a second threshold that is smaller than the first threshold.

8. The photoelectric conversion device according to claim 7.

9. a logarithmic converter that outputs a logarithm of a signal based on the charge transferred to the second node; 8. The photoelectric conversion device according to claim 7.

10. The second node further includes a subtractor that calculates the time change based on a difference between a signal corresponding to the charge transferred to the second node and a reference signal based on a past event signal.

8. The photoelectric conversion device according to claim 7.

11. the photoelectric conversion unit is disposed on a first substrate, The gray scale signal and the event signal are read out to a second substrate stacked on the first substrate.

2. The photoelectric conversion device according to claim 1.

12. A control circuit for controlling the first readout process and the second readout process is disposed on the second substrate.

12. The photoelectric conversion device according to claim 11.

13. the photoelectric conversion unit is disposed on a first substrate, The third transfer transistor is disposed on a second substrate stacked on the first substrate.

2. The photoelectric conversion device according to claim 1.

14. The charge holding portion is disposed on the first substrate.

14. The photoelectric conversion device according to claim 13.

15. The charge retention portion is provided in a wiring layer disposed between the first substrate and the second substrate.

14. The photoelectric conversion device according to claim 13.

16. A junction between the first substrate and the second substrate is disposed at a node between the photoelectric conversion unit and the third transfer transistor.

14. The photoelectric conversion device according to claim 13.

17. Each of the plurality of pixels further includes a charge drain transistor that drains charge from the photoelectric conversion unit, The charge draining transistor is disposed on the first substrate.

14. The photoelectric conversion device according to claim 13.

18. Each of the plurality of pixels further includes a charge drain transistor that drains charge from the photoelectric conversion unit, The charge drain transistor is disposed on the second substrate.

14. The photoelectric conversion device according to claim 13.

19. The photoelectric conversion device according to any one of claims 1 to 18, an optical device corresponding to the photoelectric conversion device; a control device that controls the photoelectric conversion device; a processing device that processes a signal output from the photoelectric conversion device; a display device that displays information obtained by the photoelectric conversion device; a storage device that stores information obtained by the photoelectric conversion device; and and a mechanical device that operates based on information obtained by the photoelectric conversion device. A device characterized by:

20. The processing device processes the image signals generated by the plurality of photoelectric conversion units, and acquires distance information from the photoelectric conversion units to a subject.

20. The device of claim 19.

Images