Imaging device and electronic apparatus

Abstract

[Problem] To provide: an imaging device capable of suppressing deterioration in the accuracy of image-capture driving; and an electronic apparatus. [Solution] The present disclosure provides an imaging device comprising: a pixel array unit having a plurality of first pixel circuits which each output a detection signal indicating whether an output signal corresponding to the amount of incident light has changed due to exceeding a predetermined threshold value, and having a plurality of second pixel circuits which each output a gradation signal corresponding to the amount of incident light; and a control unit for controlling an image capture drive of the plurality of second pixel circuits on the basis of the detection signal.

Description

Imaging device and electronic device

[0001] The present disclosure relates to an imaging device and an electronic device.

[0002] Electronic devices such as digital cameras equipped with imaging devices are becoming more common. The number of pixel circuits in imaging devices is increasing, slowing down image processing speed. To address this issue, efforts are being made to speed up processing by limiting the imaging area that outputs signals from the pixel array of the imaging device.

[0003] Japanese Patent Application Laid-Open No. 2021-5846

[0004] However, since the position of the imaging target changes dynamically, there is a risk that the accuracy of imaging driving in the pixel circuit may decrease.

[0005] The present disclosure provides an imaging device and an electronic device that can suppress a decrease in accuracy of imaging drive.

[0006] In order to solve the above-mentioned problems, the present disclosure provides an imaging device including: a pixel array unit having a plurality of first pixel circuits that output detection signals indicating whether or not an output signal corresponding to the amount of incident light has changed beyond a predetermined threshold, and a plurality of second pixel circuits that output gradation signals corresponding to the amount of incident light; and a control unit that controls imaging drive of the plurality of second pixel circuits based on the detection signals.

[0007] The plurality of first pixel circuits and the plurality of second pixel circuits may be capable of simultaneously outputting the detection signals and the gradation signals.

[0008] Each of the plurality of first pixel circuits comprises a first pixel having a first photoelectric conversion element that generates a photoelectric conversion signal on which the output signal is based in accordance with the amount of light, and each of the plurality of second pixel circuits comprises a second pixel having a second photoelectric conversion element that generates a photoelectric conversion signal on which the gradation signal is based in accordance with the amount of light, the plurality of first pixels and the plurality of second pixels being arranged in a two-dimensional matrix on a light-receiving surface of the pixel array unit according to a predetermined rule, and the control unit may select a second pixel circuit to perform the imaging drive from among the plurality of second pixel circuits based on the detection signal of each of the plurality of first pixel circuits.

[0009] The image pickup device may further include a signal processing unit that selects the second pixel circuit that will perform the imaging drive based on the detection signal output by at least each of the plurality of first pixels, and the control unit may select the second pixel circuit that will perform the imaging drive based on information included in the output signal of the signal processing unit.

[0010] The control unit may select, from among the plurality of second pixels, the second pixels that are to perform the imaging drive as one or more regions, and perform different imaging drives on the second pixel circuits corresponding to the second pixels for each of the regions.

[0011] The control unit may select the region as one or more rectangular regions.

[0012] The control unit may select the region as a patch-unit region that is a combination of one or more of the second pixels.

[0013] The signal processing unit may select the second pixels that will perform the imaging drive as one or more regions, and based on the gradation signal, may be able to set at least one of exposure time, frame rate, gain, and resolution as imaging conditions for the second pixel circuit for each region, and may be able to set the threshold value of the first pixel circuit corresponding to the first pixels included in the region.

[0014] The plurality of first pixel circuits may be repeatedly driven to output the detection signals in a predetermined cycle, and the signal processing unit may change the imaging conditions in accordance with the predetermined cycle.

[0015] The pixel array may further include a plurality of AD converters corresponding to the plurality of second pixel circuits, respectively, and the control unit may reduce the power supplied to the AD converters corresponding to the second pixel circuits that are not selected as regions.

[0016] The signal processing unit may be capable of receiving an external signal as an input.

[0017] The signal processing unit may select the second pixels to perform imaging driving as one or more regions based on information included in the external signal, and may be capable of setting at least one of exposure time, frame rate, gain, and resolution as the imaging conditions for each region, and may be capable of setting the threshold value of the first pixel circuit corresponding to the first pixels included in the region.

[0018] The plurality of first pixel circuits may be repeatedly driven to output the detection signals in a predetermined cycle, and the signal processing unit may change the imaging conditions in accordance with the predetermined cycle.

[0019] The external signal may be updated less frequently than the predetermined period.

[0020] The control unit may be capable of selecting all of the second pixel circuits to be driven to perform the imaging drive and causing them to perform the imaging drive.

[0021] The image pickup device may further include an access control circuit that controls the imaging drive of the second pixel circuits based on the control of the control unit.

[0022] The access control circuit may control the second pixel circuits arranged in a matrix via row-direction control lines, and a column-direction control line may be connected to each of the second pixel circuits divided into a plurality of groups in the column direction.

[0023] The access control circuit may include: a first access control circuit that controls the second pixel circuits arranged in a matrix via row-directional control lines; and a second access control circuit that controls the second pixel circuits arranged in a matrix via column-directional control lines.

[0024] the second pixel circuit may include: a photoelectric conversion element that generates a photoelectric conversion signal in accordance with the amount of light; a first transfer transistor and a second transfer transistor connected in series between the photoelectric conversion element and a floating diffusion layer; a reset transistor having one end connected to the floating diffusion layer and the other end connected to a predetermined potential line; an amplification transistor having a gate connected to the floating diffusion layer and one end connected to a predetermined potential line; and a first selection transistor and a second selection transistor connected in series between the other end of the amplification transistor and a signal line, wherein one gate of the first transfer transistor and the second transfer transistor is connected to the first access control circuit and the other is connected to the second access control circuit; and one gate of the first selection transistor and the second selection transistor is connected to the first access control circuit and the other is connected to the second access control circuit.

[0025] The photoelectric conversion element may be configured on a pixel chip, and the access control circuit may be configured on a logic chip stacked on the pixel chip.

[0026] The column direction control line may be connected to the second pixel circuits arranged in a matrix in units of a plurality of columns.

[0027] The area information and the imaging conditions may be stored in the output data.

[0028] In order to solve the above-mentioned problems, according to the present disclosure, there is provided an electronic device comprising: an optical system that receives incident light; an imaging device that outputs a signal corresponding to the amount of incident light that has passed through the optical system; a first control unit that can control the imaging device; and a recording unit that records the signal output by the imaging device, wherein the imaging device comprises: a pixel array unit having a plurality of first pixel circuits that output a detection signal indicating whether or not an output signal corresponding to the amount of incident light has changed beyond a predetermined threshold, and a plurality of second pixel circuits that output a gradation signal corresponding to the amount of incident light; and a second control unit that controls imaging driving of the plurality of second pixel circuits based on the detection signal.

[0029] In order to solve the above problem, according to the present disclosure, there is provided an imaging device including: a pixel array unit having at least a plurality of second pixel circuits that output a gradation signal corresponding to the amount of incident light; a control unit that executes control to read out the gradation signals from at least two or more second pixel circuits among the plurality of second pixel circuits in a predetermined row order based on a detection signal that indicates whether or not the output signal corresponding to the amount of incident light has changed beyond a predetermined threshold; and a time information generation unit that generates time information related to the readout time for each row of the two or more second pixel circuits.

[0030] The time information may include information on the scanning speed of the two or more second pixel circuits.

[0031] The scanning speed may be a speed based on the reciprocal of the time required to read out the row range including the two or more second pixel circuits.

[0032] The time information may include at least one readout time within a row range for reading out the two or more second pixel circuits.

[0033] The readout time may be at least one of a readout start time, a readout end time, and a time between the readout start time and the readout end time in a row range in which the two or more second pixel circuits are read out.

[0034] The image sensor may further include: an AD conversion unit that converts the gradation signal in a range that reads out the second pixel circuit into digital data; and a signal processing unit that generates the digital data and the time information in a predetermined data format.

[0035] The pixel array section may further include a plurality of first pixel circuits that output the detection signal indicating whether or not an output signal corresponding to the amount of incident light has changed beyond a predetermined threshold value.

[0036] The time information generating section may be capable of generating time information relating to a readout time for each row of the first pixel circuits.

[0037] The image sensor may further include a storage unit that stores the time information, and the control unit may select a range for reading out the second pixel circuits based on the time information stored in the storage unit.

[0038] 1 is a block diagram showing an example configuration of an electronic device equipped with an imaging device according to a first embodiment. FIG. 1 is a block diagram showing a configuration of an imaging device according to the first embodiment. FIG. 2 is a diagram showing an example configuration of a pixel signal acquisition circuit. FIG. 3 is a diagram showing an example of color filters included in a plurality of gradation pixel circuits. FIG. 4 is a block diagram showing an example of control on the gradation pixel circuit side. FIG. 5 is a diagram showing an example in which a signal processing unit 0 is configured as a control unit (AP). FIG. 6 is a diagram showing an EVS pixel circuit and a gradation pixel circuit configured as four adjacent pixels. FIG. 7 is a diagram showing an example combination of an EVS pixel circuit and a gradation pixel circuit. FIG. 8 is a diagram showing an example combination of a second EVS pixel circuit and a gradation pixel circuit. FIG. 9 is a diagram showing an example combination of a third EVS pixel circuit and a gradation pixel circuit. Circuit diagram showing the configuration of an EVS pixel circuit. FIG. 10 is a diagram showing an example configuration of a gradation pixel circuit. Block diagram showing an example configuration of an AD conversion unit. FIG. 11 is a diagram showing an example of a stacked structure of a solid-state imaging element according to an embodiment of the present technology. FIG. 12 is a diagram showing an example of regions detected by a region detection unit. Time chart according to imaging parameters set for three regions. 30. A diagram schematically showing images read out for three regions and the entire region. A diagram showing an example of resolution. A diagram showing an example of the operation of a gradation pixel circuit in which only the region is set as the imaging region. A diagram showing an example of the frame configuration of one frame's worth of image data. A diagram showing an example of data stored in embedded data (EBD). A diagram showing an example of the configuration of embedded data (EBD) and image data. A diagram showing another example of data format. A diagram showing an example of pattern 1. A diagram showing an example of pattern 2. A diagram showing an example of pattern 3. A diagram showing an example of pattern 4. A diagram showing an example of a data format in which long packets are configured in units of rows. A diagram showing an example of a data format in units of imaging region (patch). A flowchart showing an example of control processing according to the present embodiment. A flowchart showing an example of the processing of part of step S16 in FIG. 30. A diagram showing an example of the configuration of a pixel circuit according to Modification 1 of the first embodiment. A diagram showing an example of an imaging mode of each pixel circuit. A diagram showing an example of the configuration of a pixel array unit according to Modification 2 of the first embodiment. A block diagram showing an example of the configuration of a signal processing unit of an imaging device according to a second embodiment. A time chart showing an example of imaging operation in accordance with imaging parameters set for three regions and the entire region. 1 is a diagram schematically showing a control range of a first access control circuit, a diagram schematically showing control ranges of a plurality of access control circuits, a diagram schematically showing control ranges of a plurality of access control circuits, and a diagram schematically showing a control example of the first access control circuit and the column-direction access control circuit.13. A diagram showing an example of the configuration of a gradation pixel circuit according to a fourth embodiment. A diagram showing an example of a stacked structure of a gradation pixel circuit according to an embodiment of the present technology. A diagram schematically showing an example of control of a first access control circuit and a column-direction access control circuit. A diagram schematically showing a non-drive range of an AD conversion unit of an imaging device according to a fifth embodiment. A flowchart showing an example of clock control of a control unit according to a sixth embodiment. A flowchart showing an example of clock control of a control unit according to a seventh embodiment. A diagram schematically showing control of changing a threshold of a control unit according to an eighth embodiment. A flowchart showing an example of control of changing a threshold of a control unit according to an eighth embodiment. A diagram showing an event filter circuit according to a ninth embodiment. A diagram showing an example of processing of an event filter circuit. A flowchart showing an example of processing of an event filter circuit. A diagram showing an example of a read operation in a case where there is a row to which an event has not been issued. A block diagram showing an example of the configuration of a signal processing unit according to an eleventh embodiment. A block diagram showing an example of the configuration of an EVS / CIS-related data processing unit. A diagram showing an example of a three-layer structure. A diagram showing another example of a three-layer structure. A diagram showing an example of a two-layer structure. A block diagram showing the configuration of an imaging device according to a thirteenth embodiment. A diagram showing an example of operation of a gradation pixel circuit according to a thirteenth embodiment. A diagram showing an example of the configuration of an event processing unit according to the thirteenth embodiment. 62 and 63. A block diagram showing the configuration of an imaging device 20 according to a fourteenth embodiment. A time chart showing an example of an imaging operation according to the imaging parameters that have been set. A diagram showing an example of reading out a line range of an area. A diagram showing example data in an area. Data showing an event signal corresponding to area A65. A diagram showing an example of reading out a patch area. A diagram showing an example of the configuration of a gradation signal processing unit 28 according to a fifteenth embodiment. An image showing a partial range of FIG. 65. A block diagram showing an example of the schematic configuration of a vehicle control system. An explanatory diagram showing an example of the installation positions of an outside vehicle information detection unit and an imaging unit.

[0039] Hereinafter, embodiments of an imaging device and a light detection method will be described with reference to the drawings. The following description will focus on the main components of the imaging device, but the imaging device may include components and functions that are not shown or described. The following description does not exclude components and functions that are not shown or described.

[0040] First Embodiment FIG. 1 is a block diagram showing an example of the configuration of an electronic device equipped with an imaging device according to a first embodiment.

[0041] 1 includes an imaging lens (optical system) 11, an imaging device (imaging element) 20, a recording unit 12, and a control unit (AP) 13. This electronic device 10 can be applied to, for example, a camera system mounted on a communication device such as a smartphone, a camera system mounted on an industrial robot, an in-vehicle camera system, and the like.

[0042] The imaging lens 11 captures incident light from a subject and forms an image on the imaging surface of the imaging device 20. The imaging device 20 photoelectrically converts the incident light captured by the imaging lens 11 on a pixel-by-pixel basis to obtain imaging data.

[0043] The imaging device 20 performs predetermined signal processing such as image recognition processing on the captured image data, and outputs data indicating the processing results to the recording unit 12. The recording unit 12 stores the data supplied from the imaging device 20 via the signal line 14.

[0044] The control unit (AP) 13 includes, for example, an application processor (AP). The control unit (AP) 13 controls, for example, the imaging operation of the imaging device 20. The control unit (AP) 13 is also capable of performing recognition processing to recognize more advanced imaging target categories using neural networks, more advanced area recognition, and more advanced imaging condition setting. For example, the control unit (AP) 13 sets imaging conditions using information included in the detection signal from the EVS side and at least information included in the detection signal from among the information on the gradation image. The imaging conditions include at least one of the exposure time, frame rate, gain, and resolution of the gradation pixel circuit. The imaging conditions can be changed depending on the recognized category of the imaging target. For example, the control unit (AP) 13 can change the threshold value of the EVS pixel circuit using information included in the detection signal from the EVS side and at least information included in the detection signal from among the information on the gradation image. The control unit (AP) 13 according to this embodiment corresponds to a first control unit.

[0045] Fig. 2 is a block diagram showing the configuration of an imaging device 20 according to the first embodiment. Fig. 3 is a diagram showing an example configuration of a pixel signal acquisition circuit. The imaging device 20 shown in Fig. 2 is a device capable of performing scanning imaging using EVS pixel circuits and still image imaging using gradation pixel circuits, each using independent control systems. This imaging device 20 includes a pixel array unit 21, an access control circuit 22, a control unit 23, an EVS readout circuit 24, an EVS signal processing unit 25, a timestamp generation circuit 26, an AD conversion unit 27, a gradation signal processing unit 28, output interfaces 31 and 32, and a signal processing unit 40 (see Fig. 5).

[0046] The pixel array unit 21 includes a plurality of pixel signal acquisition circuits 30 arranged two-dimensionally in a matrix. As shown in Fig. 3, the pixel signal acquisition circuit 30 is configured with an EVS pixel circuit 300 and a plurality of gradation pixel circuits 400 arranged in a predetermined arrangement. The EVS pixel circuit 300 according to this embodiment corresponds to a first pixel circuit, and the gradation pixel circuit 400 corresponds to a second pixel circuit.

[0047] FIG. 4 is a diagram showing an example of color filters included in a plurality of gradation pixel circuits 400. R is a red filter that transmits light in a band corresponding to red. In this embodiment, the gradation pixel circuit 400 having this red filter is referred to as gradation pixel circuit 400R. Similarly, G is a green filter that transmits light in a band corresponding to green. In this embodiment, the gradation pixel circuit 400 having this green filter is referred to as gradation pixel circuit 400G. Similarly, B is a blue filter that transmits light in a band corresponding to blue. In this embodiment, the gradation pixel circuit 400 having this blue filter is referred to as gradation pixel circuit 400B.

[0048] Each of the multiple EVS pixel circuits 300 and the multiple gradation pixel circuits 400R, G, and B has a photoelectric conversion element. These photoelectric conversion elements are arranged in a two-dimensional matrix in a predetermined arrangement. Furthermore, the region of photoelectric conversion elements arranged in a matrix on the light-receiving surface of the pixel array section 21 may be referred to as a pixel. That is, the multiple pixels are arranged in a matrix on the light-receiving surface of the pixel array section 21. In this case, the circuit elements of the EVS pixel circuit 300 and the multiple gradation pixel circuits 400R, G, and B, excluding the photoelectric conversion elements, may be configured without being arranged in a matrix. The EVS pixel circuit 300 and the gradation pixel circuit 400 will be described in detail below.

[0049] Each EVS pixel circuit 300 photoelectrically converts incident light and generates a voltage signal corresponding to the amount of incident light as an output signal. Based on the generated voltage signal, each EVS pixel circuit 300 outputs a detection signal indicating whether the luminance of the incident light has changed beyond a predetermined threshold. The detection signal is output row by row (for each row). Alternatively, the detection signal may be output column by column (for each column).

[0050] On the other hand, the gradation pixel circuit 400 outputs a gradation signal according to the amount of incident light. A gradation image is formed based on the gradation signal output by the gradation pixel circuit 400.

[0051] The access control circuit 22 is configured by, for example, a shift register, and has a first access control circuit 22a and a second access control circuit 22b.

[0052] The first access control circuit 22a controls the plurality of gradation pixel circuits 400. The first access control circuit 22a controls resetting of accumulated charges in each of the plurality of gradation pixel circuits 400, generation of gradation signals according to the accumulated amount of photoelectric conversion current, output of gradation signals, etc. The first access control circuit 22a also converts electrons accumulated by photoelectric conversion in each of the plurality of gradation pixel circuits 400 into voltage signals, and outputs the voltage signals to the AD converter 230 as gradation signals in order for each row. The control operation of the gradation pixel circuits 400 will be described in detail later.

[0053] The second access control circuit 22b controls each EVS pixel circuit 300 based on the control of the control unit 23. The second access control circuit 22b executes, for example, an auto-zero operation (AZ), an event detection operation (Det), and a read operation (Read) of the EVS pixel circuit 300.

[0054] The auto-zero operation (AZ) sets the reference voltage to zero by turning on the AZ switch in the pixel. The event detection operation (Det) enables a latch mounted in the logic circuit (Logic) in the pixel, and executes an operation of writing event information to the latch if the event detection threshold is exceeded at that time. The read operation (Read) executes an operation of transferring an event signal from the latch mounted in the logic circuit (Logic). The control operation of the EVS pixel circuit 300 will be described in detail later.

[0055] (EVS pixel circuit side control) The control unit 23 transmits a frame synchronization signal to each circuit in the imaging device 20. This frame synchronization signal is a signal that is transmitted at a period in which processing for one frame is repeatedly performed using each EVS pixel circuit 300 in the imaging device 20. Note that the control unit 23 according to this embodiment corresponds to a second control unit.

[0056] Furthermore, the control unit 23 identifies a detection pixel row in which an event has been detected, based on a detection signal input from the pixel array unit 21 via the second access control circuit 22b. The detection pixel row includes at least one active pixel circuit. An active pixel circuit is a pixel circuit in which a voltage value corresponding to the luminance of incident light exceeds or falls below the above-mentioned preset threshold. Note that, when the detection signal is output from the pixel array unit 21 on a column-by-column basis, the control unit 23 identifies a detection pixel column in which an event has been detected. The detection pixel column also includes at least one active pixel circuit.

[0057] The EVS readout circuit 24 reads out the detection signals of the detection pixel row identified by the control unit 23 from the pixel array unit 21. The EVS readout circuit 24 has, for example, a first mode and a second mode. The first mode is a mode in which the detection signals of all rows of the pixel array unit 21 are read out row by row in order.

[0058] The second mode is a mode in which detection signals of only detection pixel rows that include active pixels are read out, rather than reading out all rows. When detection signals are output from the pixel array unit 21 on a column-by-column basis, the EVS readout circuit 24 reads out detection signals of detection pixel columns identified by the control unit 23 from the pixel array unit 21. In other words, the EVS readout circuit 24 does not read out detection signals of all columns of the pixel array unit 21, but reads out detection signals of only detection pixel columns that include active pixels.

[0059] The EVS signal processing unit 25 performs predetermined signal processing on the detection signal read out by the EVS readout circuit 24. For example, the EVS signal processing unit 25 creates EVS data for each frame separated by a frame synchronization signal. The EVS data includes the address of the detected pixel row (or detected pixel column), a timestamp indicating the time information when the event was detected, and the like.

[0060] The timestamp generation circuit 26 generates a timestamp, which is time information indicating when a change in luminance of incident light is detected by the EVS pixel circuit 300 of the pixel array unit 21. The timestamp generation circuit 26 also outputs the generated timestamp to the EVS signal processing unit 25.

[0061] The output interface 31 outputs the EVS data to the recording unit 12 (see FIG. 1) as first output data of the EVS signal processing unit 25 .

[0062] (Gradation Pixel Circuit Side Control) Fig. 5 is a block diagram showing an example of control on the gradation pixel circuit side. As shown in Fig. 5, the signal processing unit 40 has a signal processing unit 40. This signal processing unit 40 has an area detection unit 41 and a parameter setting unit 42. The area detection unit 41 sets, for example, a plurality of imaging areas (sub) using at least one detection signal from among the detection signal of the detection pixel row read out by the EVS readout circuit 24 and the gradation image of the pixel array unit 21 generated by the AD conversion unit 27. The gradation image may be the entire pixel area or the imaging area (sub).

[0063] The signal processing unit 40 calculates the area in which the imaging target moves and the approximate moving speed based on the value of the detection signal (for example, "+1" or "-1" described below). As a result, the signal processing unit 40 sets, for example, multiple imaging areas (sub). The signal processing unit 40 also sets the range of these imaging areas (sub) based on the category of the imaging target using, for example, grayscale image data. For example, the range of the imaging area (sub) is changed depending on whether the imaging target is a stationary object such as a car, a person, or a tree. The category recognition process may be performed by the control unit (AP) 13, or the area detection unit 41 may perform the recognition process. A signal input from the control unit (AP) 13 or another processing unit may be referred to as an external signal.

[0064] Furthermore, the region detection unit 41 changes the position and range of the imaging region (sub) using a detection signal read out for each frame via the EVS readout circuit 24. In this embodiment, the imaging region (sub) may be referred to as a selected region or a region of interest (ROI).

[0065] The parameter setting unit 42 sets parameters such as the reset timing (shutter timing) of the accumulated charge of each gradation pixel circuit 400 for each imaging region (sub), the generation timing (read timing) of a gradation signal corresponding to the accumulated amount of photoelectric conversion current, and the output of the gradation signal. The interval between the shutter timing and read timing of the gradation pixel circuit 400 corresponds to the exposure time. The number of shutter timings and read timings within a predetermined time corresponds to the frame rate. The slope of the gradation curve of the gradation signal processing unit 28, which will be described later, corresponds to the gain. The number of additions of the gradation pixel circuits 400, or the predetermined number when selecting one output from a predetermined number of gradation pixel circuits 400, corresponds to the resolution. In other words, the resolution decreases as the number of additions or the predetermined number increases. The parameter setting unit 42 can set the imaging drive method for each imaging region (sub) for each frame as a parameter using the detection signal read out for each frame.

[0066] The control unit 23 executes imaging control based on the parameters set by the parameter setting unit 42 via the first access control circuit 22a of the access control circuit 22. Note that the configuration example of the signal processing unit 40 is not limited to this. Fig. 6 is a diagram showing an example in which the signal processing unit 40 is configured in the control unit (AP) 13. For example, the signal processing unit 40 can be configured in the control unit (AP) 13. This makes it possible to speed up recognition processing and the like.

[0067] The AD conversion unit 27 converts the analog luminance signal into a digital luminance signal using an AD converter provided for each column signal line VSL1 of the gradation pixel circuits 400 arranged in a matrix. The generated digital signal is supplied to the gradation signal processing unit 28. The AD conversion unit 27 will be described in detail later.

[0068] The gradation signal processing unit 28 performs predetermined signal processing such as gradation conversion processing, CDS (Correlated Double Sampling) processing, and image recognition processing on the digital signal from the AD converter 230. For example, this image recognition processing includes setting an imaging area and recognizing the category of the imaging object within the imaging area. Note that this image recognition processing can also be configured within the area detection unit 41 of the signal processing unit. The output interface 32 outputs the digital signal to the recording unit 12 (see FIG. 1) as second output data.

[0069] 7 to 10, the pixel signal acquisition circuit 30 can be any combination of the EVS pixel circuit 300 and the gradation pixel circuit 400. Fig. 7 shows an example in which the EVS pixel circuit 300 and the gradation pixel circuits 400R, G, and B are configured as four adjacent pixels.

[0070] FIG. 8 is a diagram showing an example of a configuration in which two EVS pixel circuits 300 are used, four gradation pixel circuits 400R are used, seven gradation pixel circuits 400G are used, and three gradation pixel circuits 400B are used. FIG. 9 is a diagram showing an example of a configuration in which four EVS pixel circuits 300 are used, three gradation pixel circuits 400R are used, six gradation pixel circuits 400G are used, and three gradation pixel circuits 400B are used. Four EVS pixel circuits 300 are arranged in the center. FIG. 10 is a diagram showing an example of a configuration in which 64 pixels are used as one unit of the pixel signal acquisition circuit 30. FIG. 11 is a diagram showing an example of a configuration in which four EVS pixel circuits 300 are used, 16 gradation pixel circuits 400R are used, 32 gradation pixel circuits 400G are used, and 12 gradation pixel circuits 400B are used.

[0071] [Configuration Example of EVS Pixel Circuit] Fig. 11 is a circuit diagram showing the configuration of an EVS pixel circuit 300. The EVS pixel circuit 300 shown in Fig. 11 is an example of an EVS pixel circuit, and has a logarithmic conversion circuit 310, a buffer circuit 320, a subtraction circuit 330, a quantization circuit 340, and a logic circuit 351. The buffer circuit 320, the subtraction circuit 330, the quantization circuit 340, and the logic circuit 351 configure an analog front end (AFE) for generating a detection signal.

[0072] The logarithmic conversion circuit 310 has a photoelectric conversion element 200, an N-channel MOS transistor 311, a P-channel MOS transistor 312, an N-channel MOS transistor 313, and an N-channel MOS transistor 314. The photoelectric conversion element 200 photoelectrically converts incident light to generate an electric charge whose amount corresponds to the amount of incident light. The photoelectric conversion element 200 is formed, for example, by a photodiode. The photoelectric conversion element 200 is disposed on a pixel chip 201 (see FIG. 14 described later). All elements other than the photoelectric conversion element 200 are disposed on a logic chip 202.

[0073] The photoelectric conversion element 200 is connected in series to a MOS transistor 314. Furthermore, the MOS transistor 312 is connected in series to a MOS transistor 313. Furthermore, the gate of the MOS transistor 311 is connected to the drain of the MOS transistor 312 and the drain of the MOS transistor 313. The logarithmic conversion circuit 310 converts the charge photoelectrically converted by the photoelectric conversion element 200 into a logarithmic output voltage signal Vlog.

[0074] The buffer circuit 320 has a P-channel MOS transistor 321 and a P-channel MOS transistor 322. The MOS transistor 321 is connected in series to a MOS transistor 522. The buffer circuit 320 outputs a source follower voltage signal Vsf obtained by performing impedance conversion on a voltage signal Vlog input to the gate of the MOS transistor 322.

[0075] The subtraction circuit 330 has a P-channel MOS transistor 331, a P-channel MOS transistor 332, an N-channel MOS transistor 333, a capacitor 334, a capacitor 335, and an AZ switch 335. The MOS transistor 332 is connected in series to the MOS transistor 333. The capacitor 334 is connected to the gate of the MOS transistor 332.

[0076] An AZ switch 335 is connected to the gate of the MOS transistor 331 and the gate of the MOS transistor 333. When the AZ switch 335 is connected to the bias potential AZ side by a BIAS_SW signal from the first access control circuit 22a, the charge in the capacitor 334 is reset. This causes the EVS pixel circuit 300 to enter a reset state in which an event cannot be detected. In this embodiment, this reset state is called auto-zero (AZ).

[0077] Furthermore, when the BIAS_SW signal from the first access control circuit 22 a connects the AZ switch 335 to the bias potential EVT side, the EVS pixel circuit 300 enters an event detection enabled state. The subtraction circuit 330 outputs a differential voltage signal of the source follower voltage signal Vsf before and after reset.

[0078] The quantization circuit 340 includes a P-channel MOS transistor 341 and an N-channel MOS transistor 342. The MOS transistor 341 is connected to the MOS transistor 342 in series.

[0079] The MOS transistor 341 and the MOS transistor 342 constitute a first comparator circuit that compares the differential voltage signal input to the gate of the MOS transistor 341 with an upper threshold. A second comparator circuit may be further constituted by connecting another MOS transistor and another MOS transistor in series, and comparing the differential voltage signal input to the gate of one of the MOS transistors with a lower threshold. For example, the output signal of the first comparator circuit corresponds to a digital detection signal. For example, if the differential voltage signal exceeds the upper threshold, the signal value of the detection signal is "+1." If the differential voltage signal Vdiff is less than the upper threshold, the signal value of the detection signal is "0."

[0080] The EVS pixel circuit 300 is not limited to the circuit shown in FIG. 11 , and may be any circuit capable of detecting changes in the luminance of incident light. For example, the quantization circuit 340 may further include a second comparator circuit that connects another MOS transistor and another MOS transistor in series and compares a differential voltage signal input to the gate of one of the MOS transistors with a lower threshold. For example, the output signals of the first comparator circuit and the second comparator circuit correspond to digital detection signals. For example, when the differential voltage signal exceeds an upper threshold, the signal value of the detection signal is "+1." When the differential voltage signal Vdiff is below a lower threshold, the signal value of the detection signal is "-1." When the differential voltage signal Vdiff is within the range between the upper and lower thresholds, the signal value of the detection signal is "0." In other words, a signal value of "+1" or "-1" indicates the occurrence of a so-called event.

[0081] The logic circuit 351 includes a latch circuit, and when an enable signal LATCH_EN is supplied from the first access control circuit 22a, the logic circuit 351 stores the signal value of the detection signal.

[0082] [Configuration Example of Gradation Pixel Circuit] Fig. 12 is a diagram showing a configuration example of the gradation pixel circuit 400. As shown in Fig. 12, the gradation pixel circuit 400 has a photoelectric conversion element 200, a transfer transistor 401, a reset transistor 402, an amplification transistor 403, a selection transistor 404, and a floating diffusion layer 405.

[0083] For example, an N-type MOS (Metal-Oxide-Semiconductor) transistor is used as the transfer transistor 401, reset transistor 402, amplification transistor 403, selection transistor 404, and transfer transistor 401. The photoelectric conversion element 200 is disposed on a pixel chip 201 (see FIG. 14 described later). For example, all elements other than the photoelectric conversion element 200 are disposed on a logic chip 202.

[0084] The photoelectric conversion element 200 photoelectrically converts incident light to generate electric charges. A reset signal RST is supplied from the second access control circuit 211b to the gate electrode of the reset transistor 402. When the reset signal RST is supplied, the electric charges in the floating diffusion layer 405 are reset.

[0085] After the charges in the floating diffusion layer 405 are reset, a transfer signal TRG is supplied from the second access control circuit 211b to the gate electrode of the selection transistor 401. When the transfer signal TRG is supplied, the charges photoelectrically converted by the photoelectric conversion element 200 are supplied from the photoelectric conversion element 200 to the floating diffusion layer 405 by the transfer transistor 401. The charges supplied from the photoelectric conversion element 200 are accumulated in the floating diffusion layer 405. The floating diffusion layer 405 generates a voltage signal having a voltage value corresponding to the amount of accumulated charge.

[0086] The amplifier transistor 403 is connected in series with the selection transistor 404 between the power supply line of the power supply voltage VDD and the vertical signal line VSL1. The amplifier transistor 403 amplifies the voltage signal converted from charge to voltage by the floating diffusion layer 405.

[0087] A selection signal SEL is supplied from the second access control circuit 211b to the gate electrode of the selection transistor 404. In response to the selection signal SEL, the selection transistor 404 outputs the voltage signal amplified by the amplification transistor 403 as a pixel signal SIG to the AD converter via the vertical signal line VSL1. Note that the configuration of the gradation pixel circuit 400 illustrated here is just an example, and the present invention is not limited to this configuration example.

[0088] An example configuration of the AD conversion unit 27 will be described with reference to FIG. 13 . FIG. 13 is a block diagram showing an example configuration of the AD conversion unit 27. The AD conversion unit 27 includes an AD converter 230 for each column of the gradation pixel circuits 400. The AD converter (ADC) 230 converts the analog gradation signal SIG supplied via the vertical signal line VSL1 into a digital signal. This digital signal is converted into a digital pixel signal having a larger number of bits than the gradation signal SIG1. For example, if the gradation signal SIG1 is 2 bits, the pixel signal is converted into a digital signal of 3 bits or more (e.g., 16 bits). The AD converter 230 supplies the generated digital signal to the gradation signal processing unit 28. Note that this embodiment may be described as an example in which the number of vertical signal lines VSL1 corresponds to the number of gradation pixel circuits 400 present in the same column. In this case, the same number of AD converters 230 as the number of vertical signal lines VSL1 are configured.

[0089] 14 is a diagram showing an example of a stacked structure in which the solid-state imaging device 20 according to the embodiment of the present technology is configured as a solid-state imaging device. The solid-state imaging device 20 includes a logic chip 202 and a pixel chip 201 stacked on the logic chip 202. These substrates are electrically connected via connecting portions such as vias. Note that, in addition to vias, the connections can also be made using Cu-Cu bonding or bumps.

[0090] Fig. 15 is a diagram showing an example of regions detected by the region detection unit 41. In Fig. 15, for example, three regions sub1, sub2, and sub3 are detected. The parameter setting unit 42 sets imaging parameters for the gradation pixel circuits 400 of these three regions sub1, sub2, and sub3. The parameter setting unit 42 also sets all-region imaging parameters for the entire image region (main frame) of the gradation pixel circuit 400 in addition to the imaging parameters for the three regions sub1, sub2, and sub3. For example, the all-region imaging parameters set conversion to digital data such as thinning and pixel addition (see Fig. 18 described below).

[0091] FIG. 16 is a time chart showing an example of imaging operation according to imaging parameters set in the three regions sub1, sub2, and sub3 of FIG. 15 . FIG. 16( a) is a diagram showing an example of operation of the gradation pixel circuit 400. The horizontal axis represents time, and the vertical axis represents row addresses in the pixel array section 21. FIG. 16( b) is a diagram showing an example of operation of the EVS pixel circuit 300. The horizontal axis represents time, and the vertical axis represents row addresses in the pixel array section 21. For ease of explanation, the three regions sub1, sub2, and sub3 are illustrated as if they are repeating the same shutter operation and read operation in the same manner, but this is not limited to this. For example, in actual imaging operation, the range of each region, the number of regions, the shutter operation of each region, the read operation of each region, and the like may be changed according to the processing cycle of the signal processing section 40.

[0092] 16(b), each EVS pixel circuit 300 repeats processing in a frame cycle, such as imaging processing t0 to t1, t2 to t3, and t4 to t5 for one frame, in accordance with a frame synchronization signal from the control unit 23. As shown in FIG. 16(b), at processing timings t0, t2, t4, etc., the second access control circuit 22b supplies an enable signal LATCH_EN to the logic circuits 352 of all EVS pixel circuits 300. As a result, an event detection operation (Det) is performed in which the signal value of the detection signal is latched in the latch circuit of the logic circuit 352.

[0093] Next, an auto-zero operation (AZ) is performed in which the AZ switches 335 of all EVS pixel circuits 300 are turned ON to reset the reference voltage of the capacitor 334 to zero. Subsequently, under the control of the second access control circuit 22b, an operation of transferring an event signal from a latch mounted in the logic circuit (Logic) is performed for each row of the pixel array unit 21. Then, when the transfer of all rows is completed, the AZ switches 335 of all EVS pixel circuits 300 are turned OFF to perform event detection. In this way, the signal value of the detection signal for each frame is associated with the address of each EVS pixel circuit 300 in accordance with the frame synchronization signal, and is supplied to the signal processing unit 40.

[0094] 16B, in the regions corresponding to the three regions sub1, sub2, and sub3 and the entire region (Main frame), a high-level reset signal RST and transfer signal TRG are supplied to the gate of the reset transistor 402 by a shutter operation for each row determined for each of the three regions sub1, sub2, and sub3, thereby resetting the floating diffusion layer 405 and the photoelectric conversion element 200. Then, a low-level reset signal RST and a high-level transfer signal TRG are supplied to the gates of the transfer transistor 401 and the reset transistor 402, thereby starting imaging. Charges corresponding to the photoelectric conversion of the photoelectric conversion element 200 are accumulated in the floating diffusion layer 405.

[0095] Next, in a read operation for each row defined for each of the three regions sub1, sub2, and sub3, a low-level transfer signal TRG is supplied to the gate of the transfer transistor 401, and upon completion of imaging, a high-level transfer signal TRG is supplied to the selection signal SEL. This causes the voltage signal amplified by the amplifier transistor 403 to be output as a pixel signal SIG via the vertical signal line VSL1 to the AD converter, where it is converted into a digital gradation signal. Note that the data readout in FIG. 16 is described as an example in which there is an AD converter 230 corresponding to each pixel circuit 400. This allows simultaneous readout from gradation pixel circuits 400 in different rows.

[0096] 17 is a diagram showing an image composed of digital grayscale signals read from three regions, sub1, sub2, and sub3, and the entire region (Main frame). Here, short exposure refers to a case where the interval between the shutter operation and the read operation for each row is shorter than the long exposure time. For example, exposure times are sometimes referred to as short exposure, medium exposure, and long exposure in ascending order of length.

[0097] The gain corresponds to the slope of the tone conversion curve of the tone signal processing unit 28 (see FIG. 2). The tone conversion curve defines the rules for converting pixel values ​​of input image data into output pixel values. As the gain of the tone conversion curve increases, the contrast of the tone image after conversion processing by the tone signal processing unit 28 increases. For example, the gain may be referred to as high gain, medium gain, or low gain in descending order of gain.

[0098] In this way, the control unit 23 uses the detection signal to set imaging parameters (region of interest setting, exposure time, gain setting, frame rate, etc.) for each region and executes pixel control corresponding to the set parameters. This enables high-precision selection of RGB output regions, while achieving high-speed, low-power imaging with settings appropriate for each region. Furthermore, use of the EVS pixel circuit 300 provides a wide dynamic range and enables high-speed detection, enabling robust, low-latency RGB output for moving objects.

[0099] FIG. 18 is a diagram showing an example of resolution. As described above, the resolution is changed by adding image data or thinning out the data during readout, which are setting parameters for the signal processing unit 40 according to this embodiment. The greater the number of pixels added, the lower the resolution becomes. For example, the resolution gradually decreases in the order of adding four adjacent pixels, adding eight adjacent pixels, and adding sixteen adjacent pixels. These addition processes are performed by the gradation signal processing unit 28. For example, the resolution may be referred to as high resolution, medium resolution, and low resolution, in descending order of resolution.

[0100] The frame rate (fps) corresponds to the number of times images are repeatedly captured within a given time period, and may be referred to as a high frame rate, a medium frame rate, or a low frame rate in descending order of the number of times images are captured.

[0101] Region sub1 is an example of a person category where a gradation image is generated with short exposure and high gain. Region sub2 is an example of a tree category where a gradation image is generated with long exposure and low gain. Region sub3 is an example of a car category where a gradation image is generated with short exposure and low gain. The entire region (Main frame) is an example of a gradation image generated with low resolution, full screen, and low frame rate.

[0102] In this way, it is possible to freely set imaging drive conditions (setting parameters) such as imaging time and readout time for each of the three regions sub1, sub2, sub3 and the entire region (Main frame). In this case, the signal value of the detection signal for each frame is associated with the address of each EVS pixel circuit 300 and supplied to the signal processing unit 40, and the setting parameters are changed sequentially.

[0103] FIG. 19 is a diagram showing an example of the operation of the gradation pixel circuit 400 in which the imaging area to be imaged is set to only area sub1. The horizontal axis represents time, and the vertical axis represents the row address of the pixel array unit 21. This shows an example in which, after imaging of the entire area (Main frame), the imaging target is set to only area sub1. As shown in FIG. 19 , the signal processing unit 40 can change the frame rate (fps) of the imaging area depending on the size and number of imaging areas. For example, the signal processing unit 40 increases the frame rate (fps) when the imaging area is reduced to one. This allows for more detailed information to be obtained. Furthermore, for example, the signal processing unit 40 increases the frame rate (fps) of the entire imaging area as the area of ​​the entire imaging area decreases. In this way, more detailed information can be obtained.

[0104] 20 is a diagram showing an example of the frame structure of one frame of image data transmitted from the output interface 32 to the recording unit 12 (see FIG. 1). As shown in FIG. 20, one frame of event data is stored in a plurality of long packets arranged in a line between a frame start FS, which is a short packet indicating the start of the frame, and a frame end FE, which is a short packet indicating the end of the frame. In the example shown in FIG. 20, a long packet storing embedded data (EBD) is arranged at the beginning of the long packet storing the image data.

[0105] Fig. 21 is a diagram showing an example of data stored in embedded data (EBD). Fig. 22 is a diagram showing an example of the configuration of embedded data (EBD) and image data. In Fig. 21, ROIs A to D are set as imaging regions. The embedded data (SROI Embedded Data Packet) stores the number of ROIs, which is the number of imaging regions, the number of ROI rows, which is the number of rows in the imaging regions, the coordinates of the origin of each imaging region, and the vertical length H and horizontal length W of the origin of each imaging region.

[0106] A long packet has a packet header PH and a packet footer PF. The packet header PH contains a data type DT that indicates the type of data stored in the long packet, and the data type DT can be used to distinguish whether embedded data or image data is stored. The data type DT may be placed in the packet header PH or at the beginning of the area in the long packet where data is stored.

[0107] 22, the image data for each row in which image data exists is stored in the long packet of image data. For example, the overlap between imaging area A and imaging area B is stored in imaging area A, which is the area on the origin side. In other words, the image data of the overlapping area is output only once.

[0108] FIG. 23 is a diagram showing another example of a data format. FIG. 23 shows four example imaging regions. As such, the imaging region is not limited to a rectangle, and any pixel circuit can be used as the imaging region. In this embodiment, the minimum unit size of a pixel circuit may be referred to as a patch. For example, this is the minimum unit size for readout (16x16, etc.). In such cases, a rectangle larger than the minimum size can be treated as a collection of patches. The transmission data format of such data will be explained as patterns 1 to 4 using FIGS. 24 to 27. Patterns 1 to 4 are selected by the user.

[0109] 24 is a diagram showing an example of pattern 1. Pattern 1 is an example in which data is arranged in the order of the imaging areas. That is, pattern 1 allows data of other classifications to be stored in a long packet.

[0110] 25 is a diagram showing an example of pattern 2. Pattern 2 is an example in which data is arranged in the order of the imaging areas, and in which data of other classifications is not allowed to be stored in a long packet. In other words, in pattern 2, data of other classifications is not allowed to be stored in a long packet.

[0111] 26 is a diagram showing an example of pattern 3. Pattern 3 is an example in which data is arranged in order from the top row, without distinguishing between the continuity of the imaging area.

[0112] 27 is a diagram showing an example of Pattern 4. Pattern 3 is an example in which data is arranged row by row from the top without distinguishing the continuity of the imaging area. In this way, transmission using various data formats is possible.

[0113] 28 is a diagram showing an example of a data format in which a long packet is configured in units of lines. The line header (LH) stores data indicating the start of a line of image data. The line footer (LF) stores data indicating the end of a line of image data. In this way, image data can be transmitted in units of lines.

[0114] Fig. 29 is a diagram showing an example of a data format configured in units of imaging areas (patches). The patch header (PH) stores data indicating the start of an imaging area. The line footer (LF) can store data indicating the end of a line of image data. In this way, image data can be transmitted in units of imaging areas (patches).

[0115] Fig. 30 is a flowchart showing an example of control processing according to the present embodiment. Here, an example of setting imaging parameters in accordance with the processing cycle of the signal processing unit 40 will be described. Fig. 31 is a flowchart showing an example of setting a frame rate. Fig. 31 is a flowchart showing an example of processing of part of step S16 in Fig. 30. For example, the processing cycle of the signal processing unit 40 can be set in accordance with the number of frame cycles of each EVS pixel circuit 300.

[0116] 30, first, the EVS readout circuit 24 detects a detection signal and supplies it to the signal processing unit 40 (step S10). Next, the area detection unit 41 of the signal processing unit 40 detects a target area based on the detection signal and the area where movement of the imaging target is detected (step S12).

[0117] Next, if the gradation signal processing unit 28 (see FIG. 2) is executing the setting of the imaging area and the recognition processing of the category of the imaging target within the imaging area, the area detection unit 41 of the signal processing unit 40 acquires this information. Next, referring to the range of the imaging area and the target area, the area detection unit 41 determines the range to be imaged as the selected area (imaging area) (step S14). Note that the area detection unit 41 can also set the range of the target area in the current frame by referring to the amount of movement from the range of the target area in the previous frame.

[0118] Next, the parameter setting unit 41 of the signal processing unit 40 sets the frame rate, shutter operation timing, read operation timing, resolution, gamma, etc. as imaging parameters for the selected region (imaging region) (step S16). Subsequently, the control unit 23 determines whether or not there has been a change in the imaging parameters (step S18). If there has been a change (Yes in step S18), the control unit 23 changes the pixel control of the first access control circuit 22a and executes control access to each pixel circuit 400 (step S22). On the other hand, if there has been no change (No in step S18), the control unit 23 maintains the pixel control of the first access control circuit 22a and executes control access to each pixel circuit 400 (step S22).

[0119] The control unit 23 determines whether or not to end the imaging (step S24), and if not to end it (No in step S24), repeats the processing from step S10. On the other hand, if to end it (Yes in step S24), the imaging control ends.

[0120] 31 , the parameter setting unit 41 of the signal processing unit 40 calculates the sum of the sizes of all regions of interest (all imaging regions) and determines whether the sum exceeds a predetermined value (step S30). If the sum does not exceed the predetermined value (Yes in step S30), the frame rate is set to a low rate (step S32) and the process ends. On the other hand, if the sum is equal to or greater than the predetermined value (Yes in step S30), the frame rate is set to a high rate (step S34) and the process ends. In this way, the frame rate can be set according to the size of all regions of interest (all imaging regions).

[0121] (Variation 1 of First Embodiment) The electronic device 10 according to Variation 1 of the first embodiment differs from the electronic device 10 according to the first embodiment in that the photoelectric conversion element 200 of the pixel circuit of the imaging device 20 is shared by the gradation pixel circuit and the EVS pixel circuit. The differences from the electronic device 10 according to the first embodiment will be described below.

[0122] 32 is a diagram showing an example configuration of a pixel circuit 350 according to Modification 1 of the first embodiment. The pixel circuit 350 includes a photoelectric conversion element 200, a switch 200a, an event pixel circuit 300a, and a gradation pixel circuit 400a. The switch 200a switches the connection between the event pixel circuit 300a and the gradation pixel circuit 400a under the control of the control unit 23 (see FIG. 2).

[0123] 11 is an example of an EVS pixel circuit, and includes a buffer circuit 320, a subtraction circuit 330, a quantization circuit 340, and a logic circuit 351. Furthermore, the event pixel circuit 300a has a circuit configuration excluding the photoelectric conversion element 200 in the logarithmic conversion circuit 310. A connection terminal of the switch 200a on the event pixel circuit 300a side is connected to the gate terminal of an N-channel MOS transistor 313.

[0124] 12, the transfer transistor 401, the reset transistor 402, the amplification transistor 403, the selection transistor 404, and the floating diffusion layer 405. The connection terminal of the switch 200a on the gradation pixel circuit 400a side is connected to one end of the transfer transistor 401.

[0125] Fig. 33 is a diagram showing an example of an imaging mode of each pixel circuit 350. As shown in Fig. 33, in the EVS mode, the switch 200a brings the photoelectric conversion element 200 and the gate terminal of the N-channel MOS transistor 313 into electrical connection. This forms a pixel circuit equivalent to the EVS pixel circuit 300.

[0126] On the other hand, in the gradation mode, the switch 200a conducts electricity between the photoelectric conversion element 200 and one end of the transfer transistor 401. This forms a pixel circuit equivalent to the gradation pixel circuit 400. In this way, since the pixel circuit 350 has the EVS mode and the gradation mode, it becomes possible to freely configure the arrangement of the EVS pixel circuit 300 and the gradation pixel circuit 400 within the pixel array unit 21.

[0127] (Modification 2 of First Embodiment) The electronic device 10 according to Modification 2 of the first embodiment differs from the electronic device 10 according to the first embodiment in that the imaging device 20 is composed only of gradation pixel circuits. The differences from the electronic device 10 according to the first embodiment will be described below.

[0128] FIG. 34 is a diagram showing an example of the configuration of a pixel array unit 21 according to Modification 2 of the first embodiment. As shown in FIG. 34, the pixel array unit 21 differs from the electronic device 10 according to the first embodiment in that it is composed only of gradation pixel circuits 400. The imaging device 20 also has a storage unit 27a and an event determination unit 27b. The storage unit 27a stores image data for each frame. The storage unit 27a stores image data for at least the previous frame and the current frame. The event determination unit 27b calculates a difference value between the image data of the previous frame and the current frame at the pixel circuit end.

[0129] For example, if the difference value exceeds the upper threshold, the event determination unit 27b sets the signal value of the detection signal to "+1." If the difference value is below the lower threshold, the event determination unit 27b sets the signal value of the detection signal to "-1." If the difference value is within the range between the upper threshold and the lower threshold, the event determination unit 27b sets the signal value of the detection signal to "0."

[0130] The event determination unit 27b supplies the detection signal to the event signal processing unit 25. By configuring the memory unit 27a and the event determination unit 27b in this way, it becomes possible to generate a detection signal even if the pixel array unit 21 is configured only with the gradation pixel circuits 400.

[0131] Second Embodiment The electronic device 10 according to the second embodiment differs from the electronic device 10 according to the first embodiment in that it has a signal processing unit 13a on the control unit (AP) side and a signal processing unit 40 on the imaging device 20 side. The differences from the electronic device 10 according to the first embodiment will be described below.

[0132] 35 is a block diagram showing an example of the configuration of the signal processing units 13a and 40 of the imaging device 20 according to the second embodiment. As shown in Fig. 35, the electronic device 10 differs from the electronic device 10 according to the first embodiment in that it has a signal processing unit 13a on the control unit (AP) side and a signal processing unit 40 on the imaging device 20 side.

[0133] The signal processing unit 13a on the control unit (AP) side performs more advanced processing than the signal processing unit 40. For example, the area detection unit 43 on the control unit (AP) side recognizes the category of the imaging object using, for example, a neural network. Furthermore, the area detection unit 43 on the control unit (AP) side recognizes the imaging area using, for example, a more accurate recognition process. The signal processing unit 13a on the control unit (AP) side sets reference imaging parameters based on the recognition processing result of the area detection unit 43.

[0134] The signal processing unit 40 on the imaging device 20 side changes the reference imaging parameters in response to changes in the imaging situation. For example, the area detection unit 41 of the signal processing unit 40 changes the reference area based on the detection signal. The parameter setting unit 42 of the signal processing unit 40 changes the reference imaging parameters in response to the area detected by the area detection unit 41. For example, if there is a change in the area detected by the area detection unit 41, the timing and rows of the shutter operation and read operation are reset.

[0135] FIG. 36 is a time chart showing an example of imaging operation according to imaging parameters set for the three regions sub1, sub2, and sub3 shown in FIG. 15 and the entire region (Main frame). FIG. 36( a) is a diagram showing an example of operation of the gradation pixel circuit 400. The horizontal axis represents time, and the vertical axis represents row addresses in the pixel array section 21. FIG. 36( b) is a diagram showing an example of operation of the EVS pixel circuit 300. The horizontal axis represents time, and the vertical axis represents row addresses in the pixel array section 21. For ease of explanation, the three regions sub1, sub2, and sub3 are illustrated as if they are repeating the same shutter operation and read operation in the same manner, but this is not limited to this. For example, in actual imaging operation, the range of each region, the number of regions, the shutter operation of each region, and the read operation of each region may be changed according to the processing cycle of the signal processing section 40.

[0136] 36(c) is a diagram showing the control period of the signal processing unit 13a on the control unit (AP) side, in which the control period is indicated by (A) and (B).

[0137] 36(d) is a diagram showing an example of timing at which the imaging parameters of the signal processing unit 40 are supplied to the control unit 23. #1 to #7 show examples of the supply timing.

[0138] 36(e) is a diagram showing an example of timing at which the imaging parameters of the signal processing unit 40 are supplied to the control unit 23. #1 to #6 correspond to supply timings #1 to #6 supplied from the signal processing unit 13a.

[0139] 36(c) and 36(d), information on the imaging area and imaging parameters set by the signal processing unit 13a in control cycle (A) is supplied to the signal processing unit 40 in the next control cycle (B) in sequence. Then, the signal processing unit 40 sets the imaging parameters at supply timings #1 to #6 while referencing the imaging parameters supplied from the signal processing unit 13a. The signal processing unit 40 can reflect information in the frame cycle of the EVS pixel circuit 300. For example, since the processing time of the signal processing unit 13a (on the AP side) includes communication time, updates are performed at a low frequency (e.g., 30 fps), and the signal processing unit 40 (inside the sensor) updates each area and parameter at a high frequency (e.g., 240 fps) based on the results.

[0140] 36( e), the imaging parameters set at supply timing #1 can reflect information on the detection signal of the EVS pixel circuit 300 one frame period earlier and gradation image data in the information at supply timing #1 supplied from the signal processing unit 40. In other words, the imaging drive methods (parameters) indicated at supply timings #1 to #6 can be dynamically updated.

[0141] In this way, the signal processing unit 40 can reflect the information on the detection signal and the gradation image data acquired at a timing closer to the timing #1 of generating the imaging parameters in the imaging parameters based on the advanced recognition processing supplied from the signal processing unit 13 a. This makes it possible to set imaging parameters that are closer to the current imaging situation based on the advanced recognition processing.

[0142] Third Embodiment The electronic device 10 according to the third embodiment differs from the electronic device 10 according to the first embodiment in that it has a plurality of access control circuits that control the gradation pixel circuits 400 of the imaging device 20. The differences from the electronic device 10 according to the first embodiment will be described below.

[0143] 37 is a diagram schematically showing the control range of the first access control circuit 22a of the imaging device 20 according to the first embodiment. As shown in Fig. 37, the first access control circuit 22a is configured to be able to control access to all gradation pixel circuits 300 in each row. Therefore, it has a redundant access timing time so that access control is possible even for gradation pixel circuits 300 outside the imaging region.

[0144] 38 is a diagram schematically illustrating the control ranges of the multiple access control circuits 220a-f of the imaging device 20 according to the third embodiment. As shown in FIG. 38, the imaging device 20 according to the third embodiment has multiple access control circuits 220a-f. The control ranges of the multiple access control circuits 220a-f are classified into #1 to #6. This makes it possible to control access to the next row without controlling access to the gradation conversion pixel circuits outside the imaging area of ​​each row, thereby increasing the processing speed.

[0145] (Variation 1 of the Third Embodiment) The imaging device 20 according to Variation 1 of the third embodiment differs from the electronic device 10 according to the third embodiment in that the positions of the multiple access control circuits 220a-f are separated and arranged on both sides of the pixel array unit 21. The differences from the electronic device 10 according to the third embodiment will be described below.

[0146] 39 is a diagram schematically showing the control ranges of the multiple access control circuits 220a-f of the imaging device 20 according to Modification 1 of the third embodiment. As shown in Fig. 39, in the imaging device 20 according to Modification 1 of the third embodiment, the multiple access control circuits 220a-c are arranged on the left side of the imaging device 20. The multiple access control circuits 220a-c control areas #1-3 on the left side of the imaging device 20.

[0147] On the other hand, the multiple access control circuits 220d-f are arranged on the right side of the imaging device 20. The multiple access control circuits 220d-f control regions #4-6 on the right side of the imaging device 20. This allows the wiring distance of the access control circuits 220a-f to be shortened, reducing unnecessary parasitic capacitance. Furthermore, access control to the next row is possible without controlling access to the gradation conversion pixel circuits outside the imaging region of each row, thereby increasing the processing speed.

[0148] (Fourth Embodiment) The electronic device 10 according to the fourth embodiment differs from the electronic device 10 according to the first embodiment in that the access control circuit that controls the gradation pixel circuits 400 of the imaging device 20 has a column-direction access control circuit 22c in the column direction. The differences from the electronic device 10 according to the first embodiment will be described below.

[0149] 40 is a diagram schematically illustrating an example of control by the first access control circuit 22a and the column access control circuit 22c of the imaging device 20 according to the fourth embodiment. As shown in FIG. 40, the imaging device 20 according to the fourth embodiment further includes a column access control circuit 22c. The column access control circuit 22c can select the gradation pixel circuit 400 that accesses the vertical signal line VSL1. This enables access control to only the gradation pixel circuits 400 that belong to the imaging region, further increasing the processing speed.

[0150] Fig. 41 is a diagram showing an example of the configuration of a gradation pixel circuit 400a according to the fourth embodiment. As shown in Fig. 41, the gradation pixel circuit 400a according to the fourth embodiment has a photoelectric conversion element 200, a transfer transistor 401, a reset transistor 402, an amplification transistor 403, a selection transistor 404, a floating diffusion layer 405, a second transfer transistor 406, and a second selection transistor 407.

[0151] For example, an N-type MOS (Metal-Oxide-Semiconductor) transistor is used as the transfer transistor 401, the reset transistor 402, the amplification transistor 403, the selection transistor 404, the floating diffusion layer 405, the second transfer transistor 406, and the second selection transistor 407. The photoelectric conversion element 200 is disposed on a pixel chip 201 (see FIG. 14 ). All elements other than the photoelectric conversion element 200 are disposed on a logic chip 202.

[0152] The photoelectric conversion element 200 photoelectrically converts incident light to generate electric charges. A reset signal RST is supplied from the first access control circuit 22 a to the gate electrode of the reset transistor 402. When the reset signal RST is supplied, the electric charges in the floating diffusion layer 405 are reset.

[0153] After the charge in the floating diffusion layer 405 is reset, a transfer signal TRG is supplied from the first access control circuit 22 a to the gate electrode of the selection transistor 401. In addition, a transfer signal TRG2 is supplied from the column-direction access control circuit 22 c to the gate electrode of the second transfer transistor 406. That is, imaging control is performed in the row direction by the transfer signal TRG, and imaging control is performed in the column direction by the transfer signal TRG2.

[0154] When the transfer signals TRG and TRG2 are supplied, the charges photoelectrically converted by the photoelectric conversion element 200 are supplied to the floating diffusion layer 405 by the transfer transistor 401 and the second transfer transistor 406 from the photoelectric conversion element 200. The charges supplied from the photoelectric conversion element 200 are accumulated in the floating diffusion layer 405. The floating diffusion layer 405 generates a voltage signal having a voltage value according to the amount of accumulated charge. This enables imaging control (shutter operation) in the row and column directions of the gradation pixel circuit 400a.

[0155] The amplifier transistor 403 is connected in series with the selection transistor 404 between the power supply line of the power supply voltage VDD and the vertical signal line VSL1. The amplifier transistor 403 amplifies the voltage signal converted from charge to voltage by the floating diffusion layer 405.

[0156] A selection signal SEL is supplied to the gate electrode of the selection transistor 404 from the first access control circuit 21a. Furthermore, a selection signal SEL2 is supplied to the gate electrode of the second selection transistor 407 from the column-direction access control circuit 22c. This enables row and column transfer control (read operation) of the gradation pixel circuit 400a. In response to the selection signals SEL and SEL2, the selection transistor 404 and the second selection transistor 407 output a voltage signal amplified by the amplification transistor 403 as a pixel signal SIG to the AD converter via the vertical signal line VSL1. Note that the configuration of the gradation pixel circuit 400 illustrated here is merely an example, and the present invention is not limited to this configuration example.

[0157] 42 is a diagram showing an example of a stacked structure of a gradation pixel circuit 400a according to an embodiment of the present technology. For example, in the gradation pixel circuit 400a, the photoelectric conversion element 200 is configured on a pixel chip 201. The transfer transistor 401, the reset transistor 402, the amplification transistor 403, the selection transistor 404, the floating diffusion layer 405, the second transfer transistor 406, and the second selection transistor 407 are configured on a logic chip 202 as selection logic. In this case, a reset signal RST, a transfer signal TRG, and a selection signal SEL are supplied from the second access control circuit 211b as row direction control signals. Meanwhile, a transfer signal TRG2 and a selection signal SEL2 are supplied from the second access control circuit 211b as column direction control signals. Here, the row direction control signals are control signals supplied from row direction signal lines in the pixel array unit 21, and the column direction control signals are control signals supplied from column direction signal lines in the pixel array unit 21.

[0158] 11 , in the EVS pixel circuit 300, the photoelectric conversion element 200 is configured on a pixel chip 201. The other circuits are configured on a logic chip 202. In this case, a bias signal BIAS_SW and an enable signal LATCH_EN are supplied from the first access control circuit 211a as row direction control signals. On the other hand, a read signal is supplied to the logic circuit 351 as a column direction control signal.

[0159] (Variation 1 of Fourth Embodiment) The electronic device 10 according to Variation 1 of the fourth embodiment differs from the electronic device 10 according to the fourth embodiment in that the column-direction access control circuit 22d that controls the gradation pixel circuits 400 of the imaging device 20 has a plurality of control ranges in the column direction. The differences from the electronic device 10 according to the fourth embodiment will be described below.

[0160] 43 is a diagram schematically illustrating an example of control by the first access control circuit 22a and the column-direction access control circuit 22d of the image pickup device 20 according to the first modification of the fourth embodiment. As shown in FIG. 43, the column-direction access control circuit 22d according to the first modification of the fourth embodiment controls access to the vertical signal line VSL1 (see FIG. 41) by dividing the control into a plurality of area ranges. For example, the control can be performed by dividing the plurality of area ranges into odd-numbered rows and even-numbered rows. This enables even faster readout.

[0161] Fifth Embodiment An electronic device 10 according to a fifth embodiment differs from the electronic device 10 according to the first embodiment in that the electronic device 10 controls the driving range of the gradation converter 230 in the AD conversion unit 27 of the imaging device 20. The differences from the electronic device 10 according to the first embodiment will be described below.

[0162] FIG. 44 is a diagram schematically illustrating a non-driving range of the AD conversion unit 27 of the imaging device 20 according to the fifth embodiment. As shown in FIG. 44, the control unit 23 stops driving the AD converters 230 connected to the vertical signal line VSL1 outside the imaging area. This reduces power consumption. This diagram is a diagram schematically illustrating an example of control by the first access control circuit 22a and the column-direction access control circuit 22c. As shown in FIG. 40, the imaging device 20 according to the fourth embodiment further includes a column-direction access control circuit 22c. The column-direction access control circuit 22c can select the gradation pixel circuits 400 that access the vertical signal line VSL1. This enables access control to only the gradation pixel circuits 400 that belong to the imaging area, further increasing the processing speed.

[0163] Sixth Embodiment An electronic device 10 according to a sixth embodiment differs from the electronic device 10 according to the first embodiment in that the control unit 23 of the imaging device 20 can change the clock frequency depending on the amount of output data from the imaging area. The differences from the electronic device 10 according to the first embodiment will be described below.

[0164] 45 is a flowchart showing an example of clock control (clock gating) by the control unit 23 according to the sixth embodiment. As shown in Fig. 45, first, the signal processing unit 40 sets an imaging area (selected area) and supplies imaging parameters including that information to the control unit 23 (step S40).

[0165] Next, the control unit 23 calculates the amount of force data in the imaging area and sets the clock frequency according to the amount of output data (step S41). If the amount of output data is less than a predetermined value (Yes in step S41), the control unit 23 sets the clock frequency to a high-speed clock (step S42). On the other hand, if the amount of output data is equal to or greater than the predetermined value (Yes in step S41), the control unit 23 sets the clock frequency to a low-speed clock (step S43).

[0166] Next, the control unit 23 ends the frame output (step S44) and determines whether or not to end the overall process (step S45). If not (No in step S45), the process is repeated from step S40. On the other hand, if it is to be ended (Yes in step S45), the overall process is ended. This makes it possible to suppress fluctuations in processing speed due to data volume.

[0167] Seventh Embodiment An electronic device 10 according to a seventh embodiment differs from the electronic device 10 according to the first embodiment in that the control unit 23 of the imaging device 20 is capable of performing clock gating. The differences from the electronic device 10 according to the first embodiment will be described below.

[0168] 46 is a flowchart showing an example of clock control (clock gating) by the control unit 23 according to the seventh embodiment. As shown in Fig. 46, first, the control unit 23 executes data output from the pixel array unit 21 (step S50).

[0169] Next, the control unit 23 determines whether or not the frame output has ended (step S51). If the control unit 23 determines that the frame output has not ended (No in step S51), the control unit 23 repeats the process from step S50.

[0170] On the other hand, if the control unit 23 determines that the frame output has ended (Yes in step S51), it executes clock gating, which stops the supply of clock signals to circuits related to data readout for a predetermined period of time (step S52). The control unit 23 then determines whether or not to end the overall process (step S53). If the overall process has ended (Yes in step S53), it executes the overall process. On the other hand, if the overall process has not ended (No in step S53), it repeats the process from step S50.

[0171] In this way, by performing clock gating, which stops the supply of clock signals to circuits related to data reading for a predetermined time, the power consumption of the imaging device 20 can be reduced.

[0172] Eighth Embodiment An electronic device 10 according to an eighth embodiment differs from the electronic device 10 according to the first embodiment in that the control unit 23 is capable of changing the threshold value of the EVS pixel circuit 300 of the imaging device 20. The differences from the electronic device 10 according to the first embodiment will be described below.

[0173] 47A and 47B are diagrams illustrating the threshold change control of the control unit 23 according to the eighth embodiment.

[0174] 47(b) is a diagram showing an example of setting the high-sensitivity threshold. The horizontal axis represents time, and the vertical axis represents the illuminance of the first imaging area (first area of ​​interest) Ar1 and the occurrence of an event. The positive side of the event occurrence represents "+1 (Pos.1)," which is the signal value of the detection signal when the illuminance exceeds the first upper threshold, and the negative side represents "-1 (Neg.1)," which is the signal value of the detection signal when the illuminance falls below the first lower threshold.

[0175] 47(c) is a diagram showing an example of setting the low sensitivity threshold. The horizontal axis represents time, and the vertical axis represents the illuminance of the 21st imaging area (first area of ​​interest) Ar2 and the occurrence of an event. The positive side of the event occurrence represents "+1 (Pos.1)," which is the signal value of the detection signal when the illuminance exceeds the second upper threshold, and the negative side represents "-1 (Neg.1)," which is the signal value of the detection signal when the illuminance falls below the second lower threshold. The absolute values ​​of the first upper threshold and the first lower threshold are set smaller than the absolute values ​​of the second upper threshold and the third lower threshold.

[0176] As shown in Figures 47(b) and (c), the frequency of occurrence of the signal value of the detection signal of the first imaging area (first area of ​​interest) Ar1, which is set with a first upper threshold and a first lower threshold, which are high-sensitivity thresholds, is higher than the frequency of occurrence of the signal value of the detection signal of the second imaging area (second area of ​​interest) Ar2, which is set with a second upper threshold and a second lower threshold, which are low-sensitivity thresholds.

[0177] FIG. 48 is a flowchart showing an example of threshold change control by the control unit 23 according to the eighth embodiment. As shown in FIG. 48 , first, the control unit 23 calculates the number of events occurring for each imaging region in the previous frame period and sets an event detection threshold corresponding to the number of events occurring (step S60). If the number of events occurring is equal to or greater than a predetermined value (Yes in step S60), the control unit 23 sets a low-sensitivity threshold and terminates processing. On the other hand, if the number of events occurring is less than the predetermined value (No in step S60), the control unit 23 sets a high-sensitivity threshold and terminates processing. In this way, since the threshold can be set according to the number of events for each imaging region, the number of events can be set according to the purpose of the signal processing units 13a and 40.

[0178] Ninth Embodiment An imaging device 20 of an electronic device 10 according to a ninth embodiment differs from the electronic device 10 according to the first embodiment in that it has an event filter circuit that filters the number of events. The differences from the electronic device 10 according to the first embodiment will be described below.

[0179] 49 is a diagram showing an event filter circuit 450 according to the ninth embodiment. The event filter circuit 450 is configured, for example, in the signal processing circuit 400 of the EVS readout circuit. The event filter circuit 450 randomly reduces the signal value of the detection signal output by the EVS pixel circuit 300 of the pixel array unit 21. The event filter circuit 450 supplies the signal value (event) of the randomly reduced (dropped) detection signal to the event signal processing unit 25.

[0180] 50 is a diagram showing an example of processing by the event filter circuit 450. The original data D100 shows the signal values ​​of the detection signals arranged in order. The original data D100 has five signals indicating "+1 (Pos. 1)" arranged in order.

[0181] Data D102 shows an example in which the signal values ​​of the detection signals are filtered using the low frequency setting. In data D102, the number of signals indicating "+1 (Pos. 1)" is reduced from five to four.

[0182] On the other hand, data D104 shows an example in which the signal values ​​of the detection signals are filtered using a high frequency setting. In data D104, the number of signals indicating "+1 (Pos. 1)" is reduced from five to two. In this way, the event filter circuit 450 randomly reduces the signal values ​​of the detection signals with a probability according to the frequency setting value. This allows the number of events to be set according to the purpose of the signal processing units 13a and 40.

[0183] 51 is a flowchart showing an example of processing by the event filter circuit 450. The event filter circuit 450 reads the signal values ​​(events) of the detection signals and determines whether each signal value (event) is a signal value within the imaging area (area of ​​interest) (step S70). If the filter circuit 450 determines that the signal value is within the imaging area (area of ​​interest) (Yes in step S70), it sets the event drop setting to low frequency (step S71).

[0184] On the other hand, if the filter circuit 450 determines that the signal value is outside the imaging area (area of ​​interest) (No in step S70), it sets the event drop setting to high frequency (step S72). Next, the filter circuit 450 randomly reduces the signal value in accordance with the set event drop setting (step S73). In this way, it is possible to reduce the number of events depending on whether the event is inside or outside the imaging area.

[0185] Tenth Embodiment An imaging device 20 of an electronic device 10 according to a tenth embodiment differs from the electronic device 10 according to the first embodiment in that, when reading out the signal value of a detection signal from the EVS pixel circuit 300, the imaging device 20 does not execute read control of a “0” signal that does not exceed the threshold value to the EVS pixel circuit 300. The differences from the electronic device 10 according to the first embodiment will be described below.

[0186] FIG. 52 is a diagram showing an example of a read operation when there is a row for which an event has not been issued. This shows an example in which a read operation (Read) is not performed on an EVS pixel circuit 300 for which a "0" signal does not exceed the threshold. For example, in a read example of an EVS pixel circuit 300, an event detection operation (Det1), an auto-zero operation (AZ1), and processing are performed, and then a read operation (RD1) is not performed on an EVS pixel circuit 300 for which a "0" signal does not exceed the threshold. On the other hand, in a read example of an imaging region, an event detection operation (Det2), an auto-zero operation (AZ2), and processing are performed, and then a read operation (RD2) is not performed on an EVS pixel circuit 300 for which a "0" signal does not exceed the threshold. In this way, the read operation from the EVS pixel circuit 300 is shortened.

[0187] Eleventh Embodiment An imaging device 20 of an electronic device 10 according to an eleventh embodiment differs from the electronic device 10 according to the first embodiment in that the signal processing unit 40 further includes an additional information generation unit 80 and a data transmission unit 84, and the control unit (AP) 13 includes an EVS / CIS-related data processing unit 88. The differences from the electronic device 10 according to the first embodiment will be described below.

[0188] 53 is a block diagram showing an example of the configuration of the signal processing unit 40 according to the 11th embodiment. The signal processing unit 40 further includes an additional information generation unit 80. The additional information generation unit 80 includes an event access unit 801, an event count unit 802, an event number analysis unit 803, an event frequency analysis unit 804, an attention calculation unit 805, an optical flow analysis unit 806, an exposure line detection unit 807, a data processing unit 808, a parameter setting unit 809, and an area detection unit 810.

[0189] The event signal processing unit 25 generates event raw data, the presence or absence of an event, the polarity of the event, and coordinates. The event access unit 801 generates a timestamp, the number of frames, and region information (event) based on information supplied from the event signal processing unit 25. The event counting unit 802 generates information related to the event count based on information supplied from the event signal processing unit 25. The event number analysis unit 803 generates the number of events based on information supplied from the event signal processing unit 25. The event frequency analysis unit 804 generates the presence or absence of flicker, the flicker occurrence position, flicker intensity, and flicker frequency based on information supplied from the event signal processing unit 25. The attention level calculation unit 805 generates the attention level of each pixel based on information supplied from the event signal processing unit 25.

[0190] The optical flow analysis unit 806 generates an optical flow value for each pixel based on information supplied from the event signal processing unit 25. The exposure row detection unit 807 generates information related to the exposure row based on information supplied from the control unit 23. The data processing unit 808 generates a classification value, a luminance change amount, and exposure frame information for each pixel based on information supplied from the event signal processing unit 25 and the exposure row detection unit 807.

[0191] The parameter setting unit 809 generates an exposure time, a gain, and an event detection threshold based on information supplied from the event signal processing unit 25 and the AD conversion unit 27, and information supplied from the control unit 13 via the AP data transmission unit 82. The area detection unit 810 generates area information (gradation) based on information supplied from the event signal processing unit 25 and the AD conversion unit 27, and information supplied from the control unit 13 via the AP data transmission unit 82. The control unit 23 is also supplied with the information generated by the parameter setting unit 809 and the area detection unit 810.

[0192] The data receiving unit 86 receives area information (gradation / event), timestamp, number of frames, event detection threshold, ROI information, presence or absence of flicker, flicker occurrence position, flicker intensity, flicker frequency, presence or absence of EVS movement, EVS movement direction, frame data amount, event raw data, exposure frame information, exposure time information, gain information, etc. transmitted from the imaging device 20.

[0193] 54 is a block diagram showing an example of the configuration of an EVS / CIS-related data processing unit 88 according to the 11th embodiment. The EVS / CIS-related data processing unit 88 performs various processes based on the information received by the data receiving unit 86.

[0194] The EVS / CIS-related data processing unit 88 has a region of interest (ROI) calculation unit 901, a recognition processing unit (Recognition) 902, a distance information generation unit (AE / AF) 903, a second distance information generation unit (VLC) 904, a movement amount information generation unit (SLAM) 905, a second movement amount information generation unit (OIS / EIS) 906, a subject presence / absence determination unit (Motion Detect) 906, a motion detection unit (Gesture) 907, a coordinate information generation unit (3DNR) 908, a subject movement amount information generation unit (Deblur) 909, a subject second movement amount information generation unit (VFI) 910, and a subject third movement amount information generation unit (Super-resolution) 911.

[0195] A region of interest (ROI) calculation unit 901 generates coordinate information of a region to be acquired. A recognition processing unit (Recognition) 902 generates the recognition result of the target region and coordinate information of the target object.

[0196] A distance information generation unit (AE / AF) 903 generates distance information to an object. A second distance information generation unit (VLC) 904 generates distance information to an object. A movement amount information generation unit (SLAM) 905 generates movement amount information per unit time. A second movement amount information generation unit (OIS / EIS) 906 generates movement amount information per unit time.

[0197] A subject presence / absence determination unit (Motion Detect) 906 generates information on the presence / absence of a subject on the screen. A motion detection unit (Gesture) 907 generates detection results of specific motions (e.g., waving, raising a hand, etc.). A coordinate information generation unit (3DNR) 908 generates coordinate information of a moving subject.

[0198] A subject movement amount information generation unit (Deblur) 909 generates information on the movement amount of the subject per unit time. A subject second movement amount information generation unit (VFI) 910 generates information on the movement amount of the subject per unit time. A subject third movement amount information generation unit (Super-resolution) 911 generates information on the movement amount of the subject per unit time.

[0199] Twelfth Embodiment An imaging device 20 of an electronic device 10 according to a twelfth embodiment differs from the electronic device 10 according to the first embodiment in that it has a stacked structure using TSVs (Through-Silicon Vias). The differences from the electronic device 10 according to the first embodiment will be described below.

[0200] 55 is a diagram showing an example of a three-layer structure. The imaging device 20 is stacked with a pixel chip 201, a middle (Mid) chip 202a, and a bottom (Btm) chip 202b. The pixel chip 201 has a photoelectric conversion element 200 arranged thereon. The middle (Mid) chip 202a has circuits other than the photoelectric conversion element 200 of the EVS pixel circuit 300, circuits other than the photoelectric conversion element 200 of the gradation pixel circuit 400, and a region control logic circuit arranged thereon.

[0201] The bottom (Btm) chip 202b is provided with a column AD conversion unit 27a, logic circuits, and peripheral circuits. In the AD conversion unit 27, a configuration having an AD converter 230 for each vertical signal line VSL1 is referred to as a column AD conversion unit 27a.

[0202] The pixel chip 201 and the mid chip 202a are connected by TSVs 211 and 212. The mid chip 202a and the bottom chip 202b are connected by TSVs 215 and 217.

[0203] Figure 56 is a diagram showing another example of a three-layer structure. The imaging device 20 is stacked with a pixel chip 201, a middle (Mid) chip 202a, and a bottom (Btm) chip 202b. The pixel chip 201 has a photoelectric conversion element 200 disposed thereon. This differs from Figure 55 in that an AD converter for each pixel circuit, an area AD converter, and a GS circuit are disposed on the middle (Mid) chip 202a, and the column AD conversion unit 27a is not disposed on the bottom (Btm) chip 202b. The pixel chip 201 and the middle (Mid) chip 202a are connected by Cu-Cu wiring.

[0204] FIG. 57 is a diagram showing an example of a two-layer structure. The imaging device 20 is stacked with a pixel chip 201 and a logic chip 202. The pixel chip 201 has a photoelectric conversion element 200 disposed thereon. The logic chip 202 has circuits of the EVS pixel circuit 300 excluding the photoelectric conversion element 200, circuits of the gradation pixel circuit 400 excluding the photoelectric conversion element 200, a region control logic circuit, and a RAM AD conversion unit 27a disposed thereon. The pixel chip 201 and the middle (Mid) chip 202a are connected by Cu-Cu wiring. The pixel chip 201 and the middle (Mid) chip 202a are also connected by TSVs 211 (216a) and 212 (217a).

[0205] An electronic device 10 according to a thirteenth embodiment differs from the electronic device 10 according to the first embodiment in that time information relating to the time interval between the base frame and each row of the sub-regions sub5 and sub6 is added to the data. The differences from the electronic device 10 according to the first embodiment will be described below.

[0206] FIG. 58 is a block diagram showing the configuration of an imaging device 20 according to the thirteenth embodiment. The imaging device 20 shown in FIG. 58 is a device capable of adding time information to data regarding the time intervals between the base frame and regions sub5 and sub6 for each row. The imaging device 20 according to the thirteenth embodiment differs from the imaging device 20 according to the first embodiment in that it further includes a time information generation unit 260 and an event processing unit 270, and does not include the event signal processing unit 25. Details of the time information generation unit 260 will be described later. In this embodiment, data such as regions sub1 to sub5 may be referred to as subframes or subregions. Furthermore, the base frame may be referred to as a main frame.

[0207] The control unit (AP) 13a includes, for example, an application processor (AP) and a signal processing unit 40b.

[0208] The signal processing unit 40b is capable of more advanced area recognition, more advanced imaging condition setting, etc. It is also capable of performing recognition processing that recognizes the category of an imaging object using a neural network or the like.

[0209] The control unit 23 controls the imaging operation of the imaging device 20. More specifically, the control unit 23 according to the present embodiment controls reading out gradation signals from at least two or more pixel signal acquisition circuits 30 in a predetermined row order, based on a detection signal indicating whether or not an output signal corresponding to the amount of incident light has changed beyond a predetermined threshold.

[0210] The access control circuit 220a is a circuit equivalent to the first access control circuit 22a (see FIG. 2) according to the first embodiment. That is, this access control circuit 220a controls the gradation pixel circuits 400 in the plurality of pixel signal acquisition circuits 30. The access control circuit 220a controls resetting of accumulated charge in each of the plurality of gradation pixel circuits 400, generation of gradation signals according to the accumulated amount of photoelectric conversion current, output of gradation signals, etc. Furthermore, as shown in FIG. 59 , the access control circuit 220a converts electrons accumulated by photoelectric conversion in each of the plurality of gradation pixel circuits 400 into voltage signals, and outputs the voltage signals to the AD converter 230 as gradation signals in sequence for each row.

[0211] 59 is a diagram showing an example of the operation of the gradation pixel circuit 400 according to the thirteenth embodiment. The horizontal axis represents time, and the vertical axis represents the row address of the pixel array unit 21. As shown in FIG. 59 , in the example of the imaging operation of the reference frame, the imaging process of the reference frame is repeated in accordance with the frame synchronization signal from the control unit 23.

[0212] 60 is a diagram showing an example configuration of the event processing unit 270. The event processing unit 270 has a storage unit 27a and an event determination unit 27b. The storage unit 27a stores image data for each reference frame. This storage unit 27a stores image data for at least the previous frame and the current frame. The event determination unit 27b calculates a difference value between the image data of the previous frame and the current frame at the pixel circuit end.

[0213] For example, if the difference value exceeds the upper threshold, the event determination unit 27b sets the signal value of the detection signal to "+1." If the difference value is below the lower threshold, the event determination unit 27b sets the signal value of the detection signal to "-1." If the difference value is within the range between the upper threshold and the lower threshold, the event determination unit 27b sets the signal value of the detection signal to "0."

[0214] The gradation signal processing unit 28a has functions in addition to the functions of the gradation signal processing unit 28 according to the first embodiment. That is, the gradation signal processing unit 28a has a signal processing unit 40a and a parameter setting unit 42a.

[0215] The signal processing unit 40a is a processing unit that can select a sub-region. The parameter setting unit 42a sets drive operation parameters for the sub-region.

[0216] Fig. 61 is a diagram showing an example of the processing result of the area detection unit 41a. As shown in Fig. 60, the area detection unit 41a selects sub-areas sub5 and sub6 in which moving objects are present based on the signal value of the detection signal.

[0217] Fig. 62 is a time chart showing an example of imaging operation in accordance with imaging parameters set for the sub-regions sub5 and sub6 and the entire region (reference frame) shown in Fig. 61. The horizontal axis represents time, and the vertical axis represents row addresses of the pixel array unit 21. The timing and rows for the shutter operation and read operation are different for sub-regions sub5 and sub6. In this way, the parameter setting unit 42a sets the timing and rows for the shutter operation and read operation for sub-regions sub5 and sub6.

[0218] This results in a difference in scan speed between sub-regions sub5 and sub6 and the reference frame, for example. Therefore, the time interval between the read time for each row in the reference frame and the read time for each row in sub-regions sub5 and sub6 differs for each row. Here, the scan speed is a speed based on the reciprocal of the time required to read out a row range including two or more gradation pixel circuits 400. For example, the scan speed is a value obtained by dividing the number of rows to be read out by the time required to read out.

[0219] Fig. 63 is a time chart showing an example of imaging operation different from the example of imaging operation shown in Fig. 62. Fig. 64 is a time chart showing an example of imaging operation different from the examples of imaging operation shown in Fig. 62 and Fig. 63. As shown in Fig. 63 and Fig. 64, the parameter setting unit 42a can change the imaging parameters of the reference frame that serves as a reference in accordance with changes in the imaging situation.

[0220] The time information generating unit 260 generates time information relating to the readout time for each row of two or more gradation pixel circuits 400. That is, the time information generating unit 260 generates information relating to the time interval based on the setting information of the parameter setting unit 42 a. The time information generating unit 260 generates time difference information between the target sub-regions sub5 and sub6 and the reference frame.

[0221] More specifically, the time information generating unit 260 generates time information indicating the time interval between the read (readout time) time in the reference frame and the read (readout time) time of the sub-region. Note that the target sub-regions sub5 and sub6 may also be referred to as target patches sub5 and sub6.

[0222] For example, the time information generating unit 260 sets the time difference information of the sub-region = readout time of the sub-region - (readout time of the first row of the reference frame + (row address of the reference frame x scan speed of the reference frame)). The scan speed of the reference frame is the value obtained by dividing the number of readout rows of the reference frame by the readout time of the reference frame.

[0223] The time information generating unit 260 can also generate time difference information for sub-regions on a row address basis. In this case, the read time at a row address in a sub-region corresponds to (the read time of the first row in the sub-region + (the row address of the sub-region x the scan speed of the sub-region)).

[0224] The readout time of a sub-region in row address units of the reference frame corresponds to (readout time of the first row of the reference frame + (row address of the reference frame × scan speed of the reference frame)). As can be seen from this, the signal processing units 40a, 40b can associate the readout time of the reference frame for each row with the readout time of the sub-region using information such as (readout time of the first row of the reference frame, row address of the reference frame, scan speed of the reference frame) and (readout time of the first row of the sub-region, row address of the reference frame, scan speed of the reference frame).

[0225] In this way, the time information generator 260 can also generate time difference information for each of the sub-regions sub5 and sub6. In this case, the time information generator 260 generates information on (read-out time of the first row of the reference frame, row address of the reference frame, scan speed of the reference frame) and (read-out time of the first row of the sub-region, row address of the reference frame, scan speed of the reference frame).

[0226] The read (readout time) time in the reference frame can also be calculated by the readout time of the last row of the reference frame - ((last row address of the reference frame - row address of the reference frame) x scan speed of the reference frame). Similarly, it can also be calculated by the time of the midpoint (center of exposure) of the read time of the reference frame - ((row address of the midpoint of the reference frame - row address of the reference frame) x scan speed of the reference frame). In this way, since the read (readout time) time changes linearly, any information that can express a line shown linearly will suffice.

[0227] Similarly, the time information generating unit 260 can calculate the read (readout time) time in a sub-area by the readout time of the last row in the sub-area - ((last row address of the sub-area - row address of the sub-area) x scanning speed of the sub-area). Similarly, it can also calculate the time of the midpoint (center of exposure) of the read time in the sub-area - ((row address of the midpoint of the sub-area - row address of the sub-area) x scanning speed of the sub-area). In this way, since the read (readout time) time changes linearly, any information that can express a line shown linearly will suffice.

[0228] As described above, the signal processing units 40a and 40b use information on the readout time of the first row of the sub-region, the scan speed of the sub-region, the readout time of the first row of the reference frame, and the scan speed of the reference frame to calculate the time interval between the read time for each row of the reference frame and the read time for each row of sub-regions sub5 and sub6. This allows the signal processing units 40a and 40b to perform, for example, rolling shutter distortion correction for sub-regions sub5 and sub6 relative to the reference frame.

[0229] Here, the frame format will be described with reference to Figures 20 to 29. As shown in Figure 21, the embedded data (SROI Embedded Data Packet) stored in the embedded data (EBD) stores the number of ROIs, which is the number of imaging regions; the number of ROI rows, which is the number of rows in the imaging regions; the coordinates of the origin of each imaging region; the vertical length H of the origin of each imaging region; and the horizontal length W. In addition to these, it is possible to store time difference information for each ROI. Also, for example, (readout time of the first row of the reference frame, the number of readout rows of the reference frame, and the readout time of the reference frame) and (the number of rows for each ROI, as well as the readout time of the first row and the readout time for each ROI) are stored.

[0230] Alternatively, as shown in FIG. 22 , the time difference information for each row of each ROI can be stored in the packet header PH. Alternatively, the time difference information for each row of each ROI can be stored in the packet footer PF. The time difference information can include time information regarding the time interval for each row between the reference frame and sub-regions sub5 and sub6. For example, this information may include (read-out time of the first row of the reference frame, row address of the reference frame, scan speed of the reference frame) and (read-out time of the first row of the sub-region, row address of the reference frame, scan speed of the reference frame).

[0231] FIG. 23 above is a diagram showing another example of a data format. FIG. 23 is a diagram showing four example imaging regions. As mentioned above, the imaging region is not limited to a rectangle, and any pixel circuit can be used as the imaging region. Also, as mentioned above, the minimum unit size of a pixel circuit is sometimes called a patch. For example, this is the minimum unit size for readout (16x16, etc.). In such a case, a rectangle larger than the minimum size can be treated as a collection of patches.

[0232] As explained with reference to Figures 24 to 27, the transmission data format of such data is configured with a packet header PH for each data. Time difference information for each ROI can be stored in these packet headers PH. Similarly, for example, it is possible to store the read time of the first row of the reference frame, the number of read rows of the reference frame, the read time of the reference frame, the read time of the first row for each ROI, the number of read rows, and the number of read rows.

[0233] FIG. 28 above is an example of a data format in which long packets are configured in units of lines. The line header (LH) stores data indicating the start of a line of image data. The line footer (LF) stores data indicating the end of a line of image data. In this way, image data can be transmitted in units of lines. In addition to this, the line header (LH) can also store time difference information for a line of image data. Alternatively, in addition to this, the line footer (LF) can also store time difference information for a line of image data.

[0234] FIG. 29 described above is an example of a data format configured in units of imaging areas (patches). The patch header (PH) stores data indicating the start of an imaging area. The line footer (LF) can store data indicating the end of a line of image data. In this way, image data can be transmitted in units of imaging areas (patches). In addition to this, the patch header (PH) can also store time difference information in units of imaging areas (patches). Alternatively, in addition to this, the line footer (LF) can also store time difference information in units of imaging areas (patches).

[0235] As described above, according to this embodiment, the time information generating unit 260 generates time information indicating the time interval between the read (readout time) time in the reference frame and the read (readout time) time in the sub-region. This enables the signal processing unit 40b to perform processing such as rolling shutter distortion correction using the read (readout time) time and the read (readout time) time in the sub-region.

[0236] Fourteenth Embodiment An electronic device 10 according to a fourteenth embodiment differs from the electronic device 10 according to the first embodiment in that it is possible to associate the time information of the EVS pixel circuit 300 with the time information of the gradation pixel circuits 400R, G, B (see FIG. 3 ). The differences from the electronic device 10 according to the first embodiment will be described below.

[0237] Fig. 65 is a block diagram showing the configuration of an imaging device 20 according to a fourteenth embodiment. The imaging device 20 shown in Fig. 65 is a device that can associate time information of an event with time information of gradation pixels (RGB). The imaging device 20 according to the fourteenth embodiment differs from the imaging device 20 according to the first embodiment in that it further includes a time information generation unit 260a.

[0238] In addition to the functions of the time stamp generation circuit 26 (see Figure 2), the time information generation unit 260a is capable of associating the time information of the EVS pixel circuit 300 with the time information of the gradation pixel circuits 400R, G, B (see Figure 3).

[0239] 66 is a time chart showing an example of imaging operation in accordance with imaging parameters set in, for example, the three regions sub1, sub2, and sub3 of FIG. 15. FIG. 66(a) is a diagram showing an example of operation of the gradation pixel circuit 400. The horizontal axis indicates time, and the vertical axis indicates row addresses of the pixel array section 21. FIG. 66(b) is a diagram showing an example of operation of the EVS pixel circuit 300. The horizontal axis indicates time, and the vertical axis indicates row addresses of the pixel array section 21.

[0240] 66(b), each EVS pixel circuit 300 repeats processing in a frame cycle, such as imaging processing t0 to t1, t2 to t3, t4 to t5, etc., for one frame, in accordance with a frame synchronization signal from the control unit 23. In each cycle, the event signal is read out at the Read timing.

[0241] The time information generating unit 260a according to this embodiment can generate time information indicating the time interval between the read (read time) time of the event frame and the read (read time) time of the sub-region. An event according to this embodiment is detected for all rows at once as the event detection time, and the data is transferred sequentially.

[0242] At this time, when the time information generating unit 260a adds RGB data as time difference information, the time difference information between the RGB row data and the event frame = (read time of the first row of the RGB frame + (row address of the RGB frame × scanning speed of the RGB frame)) - (event detection time).

[0243] Furthermore, when the time information generating unit 260a adds event data as time difference information, the time difference information between the event frame and the RGB row data becomes (event detection time) - (read time of the first row of the RGB frame + (row address of the RGB frame × scanning speed of the RGB frame)).

[0244] As can be seen from this, it is possible to match the readout times of the event frame, the three regions sub1, sub2, and sub3, and each row of the main frame. In other words, it is possible to calculate the time difference between the detection signal in each row in the event frame and the pixel signal in each row of the three regions sub1, sub2, and sub3 and the main frame.

[0245] 67 is a diagram showing an example of reading out row ranges of three areas sub1, sub2, and sub3. For example, the AD conversion unit 27 reads out only data in the row ranges of the three areas sub1, sub2, and sub3.

[0246] 68 is a diagram showing an example of data in area A65. The R, G, and B data in the gradation pixel circuits 400R, G, and B are shown as CIS data. As described above, the readout time for each row can be stored in the packet header PH and packet footer PF.

[0247] Alternatively, the embedded data (EBD) can store the following information: (read time of the first row of the main frame (reference frame), main row address, main scan speed) and (read time of the first row of the subframe, subframe row address, reference frame scan speed).

[0248] Similarly, the read time for each row can be stored in the packet header PH and packet footer PF of the mainframe. Alternatively, the embedded data (EBD) can store information such as (the read time of the first row of the event, the row address of the event, and the scan speed of the event) and (the read time of the first row of the mainframe, the row address of the mainframe, and the scan speed of the mainframe).

[0249] 69 shows data indicating an event signal corresponding to area A65. The read time for each row can be stored in at least one of the packet header PH and the packet footer PF.

[0250] It is also possible to store information such as the time difference information between the RGB row data and the event frame = (read time of the first row of the RGB frame + (row address of the RGB frame × scanning speed of the RGB frame)) - (event detection time) in the embedded data (EBD).

[0251] Furthermore, when event data is added to embedded data (EBD), it is also possible to store the following information: time difference information between the event frame and the RGB row data = (event detection time) - (read time of the first row of the RGB frame + (RGB frame row address x RGB frame scan speed)).

[0252] Fig. 70 is a diagram showing an example of reading out a patch area. As explained with reference to Figs. 24 to 27, the transmission data format of such data is configured with embedded data (EBD), a packet header PH, and a packet footer PF.

[0253] It is also possible to store information such as the time difference information between the RGB row data and the event frame = (read time of the first row of the RGB frame + (row address of the RGB frame × scanning speed of the RGB frame)) - (event detection time) in the embedded data (EBD).

[0254] Furthermore, when event data is added to the embedded data (EBD), it is also possible to store the following information: time difference information between the event frame and the RGB row data = (event detection time) - (read time of the first row of the RGB frame + (RGB frame row address x RGB frame scan speed)).The packet header PH and packet footer PH can store time difference information for each ROI.

[0255] FIG. 28 above is an example of a data format in which long packets are configured in units of lines. The line header (LH) stores data indicating the start of a line of image data. The line footer (LF) stores data indicating the end of a line of image data. In this way, image data can be transmitted in units of lines. In addition to this, the line header (LH) can also store time difference information for a line of image data. Alternatively, in addition to this, the line footer (LF) can also store time difference information for a line of image data.

[0256] For example, at least one of the packet header PH and the line footer (LF) can store (read time of the first row of the event signal, row address of the event, scan speed of the event), (read time of the first row of the subframe, row address of the subframe, scan speed of the subframe), and (read time of the first row of the mainframe, row address of the mainframe, scan speed of the mainframe).

[0257] 29 described above is an example of a data format configured in units of imaging areas (patches). Data indicating the start of an imaging area is stored in the patch header (PH). In addition, the patch header (PH) can also store time difference information in units of imaging areas (patches).

[0258] Furthermore, for example, it is also possible to store information in these packet headers PH such as the time difference information between the RGB row data and the event frame = (read time of the first row of the RGB frame + (row address of the RGB frame × scanning speed of the RGB frame)) - (event detection time).

[0259] Furthermore, when event data is added to the packet header PH, it is possible to store the following information: time difference information between the event frame and the RGB row data = (event detection time) - (read time of the first row of the RGB frame + (RGB frame row address x RGB frame scan speed)).

[0260] The signal processing unit 40b can perform rolling shutter distortion correction between the main frame and the sub-regions using this data.

[0261] As described above, according to this embodiment, the time information generator 260a generates time information indicating the time intervals between each row of the event frame, main frame, and subframe. This enables the signal processor 40b to perform rolling shutter distortion correction between the main frame (reference frame) and subframes sub1 and sub2. Furthermore, the signal processor 40b can associate the time information of the event signal of the event frame with processing such as recognition of subframes sub1 and sub2.

[0262] Fifteenth Embodiment An electronic device 10 according to a fifteenth embodiment differs from the electronic device 10 according to the thirteenth embodiment in that it is possible to determine the priority of reading out areas sub1, sub2, and sub3 based on the time information of areas sub1, sub2, and sub3. The differences from the electronic device 10 according to the thirteenth embodiment will be described below.

[0263] 71 is a diagram showing an example of the configuration of the grayscale signal processing unit 28 according to the fifteenth embodiment. The grayscale signal processing unit 28 according to the fifteenth embodiment has a storage unit 270 and a selection unit 271.

[0264] The storage unit 270 stores the time information of the regions sub1, sub2, and sub3 as a storage table. The selection unit 271 selects a range from which the pixel signal acquisition circuit 30 reads out, based on the time information of the regions sub1, sub2, and sub3 stored in the storage unit 270.

[0265] Fig. 72 is an image obtained by extracting a part of the range of Fig. 65. The signal processing unit 40b is equipped with a table that stores time information of read accesses to these pixels, and it is also possible to determine pixel access priorities based on the table information.

[0266] <Application to a Mobile Body> The technology according to the present disclosure (the present technology) can be applied to various products. For example, the technology according to the present disclosure may be realized as a device mounted on any type of mobile body, such as an automobile, an electric vehicle, a hybrid electric vehicle, a motorcycle, a bicycle, personal mobility, an airplane, a drone, a ship, or a robot.

[0267] FIG. 73 is a block diagram showing a schematic configuration example of a vehicle control system, which is an example of a mobile object control system to which the technology of the present disclosure can be applied.

[0268] The vehicle control system 12000 includes a plurality of electronic control units connected via a communication network 12001. In the example shown in Fig. 73, the vehicle control system 12000 includes a drive system control unit 12010, a body system control unit 12020, an outside-vehicle information detection unit 12030, an inside-vehicle information detection unit 12040, and an integrated control unit 12050. Also shown as functional components of the integrated control unit 12050 are a microcomputer 12051, an audio / video output unit 12052, and an in-vehicle network I / F (Interface) 12053.

[0269] The drivetrain control unit 12010 controls the operation of devices related to the drivetrain of the vehicle in accordance with various programs. For example, the drivetrain control unit 12010 functions as a control device for a drive force generating device for generating a drive force of the vehicle, such as an internal combustion engine or a drive motor, a drive force transmission mechanism for transmitting the drive force to the wheels, a steering mechanism for adjusting the steering angle of the vehicle, and a braking device for generating a braking force of the vehicle.

[0270] The body system control unit 12020 controls the operation of various devices equipped in the vehicle body according to various programs. For example, the body system control unit 12020 functions as a control device for a keyless entry system, a smart key system, a power window device, or various lamps such as headlamps, backup lamps, brake lamps, turn signals, and fog lamps. In this case, radio waves transmitted from a portable device that serves as a key or signals from various switches can be input to the body system control unit 12020. The body system control unit 12020 receives these radio waves or signals and controls the vehicle's door lock device, power window device, lamps, etc.

[0271] The outside-vehicle information detection unit 12030 detects information outside the vehicle equipped with the vehicle control system 12000. For example, an imaging unit 12031 is connected to the outside-vehicle information detection unit 12030. The outside-vehicle information detection unit 12030 causes the imaging unit 12031 to capture images outside the vehicle and receives the captured images. The outside-vehicle information detection unit 12030 may perform object detection processing or distance detection processing for people, cars, obstacles, signs, characters on the road surface, etc. based on the received images.

[0272] The imaging unit 12031 is an optical sensor that receives light and outputs an electrical signal corresponding to the amount of light received. The imaging unit 12031 can output the electrical signal as an image or as distance measurement information. The light received by the imaging unit 12031 may be visible light or invisible light such as infrared light.

[0273] The in-vehicle information detection unit 12040 detects information inside the vehicle. For example, a driver state detection unit 12041 that detects the state of the driver is connected to the in-vehicle information detection unit 12040. The driver state detection unit 12041 includes, for example, a camera that captures an image of the driver, and the in-vehicle information detection unit 12040 may calculate the degree of fatigue or concentration of the driver based on the detection information input from the driver state detection unit 12041, or may determine whether the driver is dozing off.

[0274] The microcomputer 12051 can calculate control target values ​​for the driving force generating device, steering mechanism, or braking device based on the information inside and outside the vehicle acquired by the outside-vehicle information detection unit 12030 or the inside-vehicle information detection unit 12040, and output control commands to the drive system control unit 12010. For example, the microcomputer 12051 can perform cooperative control aimed at realizing the functions of an ADAS (Advanced Driver Assistance System), including vehicle collision avoidance or impact mitigation, following driving based on the inter-vehicle distance, vehicle speed maintenance driving, vehicle collision warning, vehicle lane departure warning, etc.

[0275] In addition, the microcomputer 12051 can perform cooperative control for the purpose of autonomous driving, which allows the vehicle to travel autonomously without relying on driver operation, by controlling the driving force generating device, steering mechanism, braking device, etc. based on information about the surroundings of the vehicle obtained by the outside vehicle information detection unit 12030 or the inside vehicle information detection unit 12040.

[0276] Furthermore, the microcomputer 12051 can output a control command to the body system control unit 12020 based on the information outside the vehicle acquired by the outside information detection unit 12030. For example, the microcomputer 12051 can control the headlamps according to the position of a preceding vehicle or an oncoming vehicle detected by the outside information detection unit 12030, and perform cooperative control aimed at preventing glare, such as switching from high beams to low beams.

[0277] The audio / video output unit 12052 transmits at least one of audio and video output signals to an output device capable of visually or audibly notifying the passengers of the vehicle or the outside of the vehicle of information. In the example of Fig. 73, the output devices are exemplified by an audio speaker 12061, a display unit 12062, and an instrument panel 12063. The display unit 12062 may include, for example, at least one of an on-board display and a head-up display.

[0278] FIG. 74 is a diagram showing an example of the installation position of the imaging unit 12031.

[0279] In FIG. 74, the imaging unit 12031 includes imaging units 12101, 12102, 12103, 12104, and 12105.

[0280] The imaging units 12101, 12102, 12103, 12104, and 12105 are provided, for example, at positions such as the front nose, side mirrors, rear bumper, back door, and the top of the windshield inside the vehicle cabin of the vehicle 12100. The imaging unit 12101 provided on the front nose and the imaging unit 12105 provided on the top of the windshield inside the vehicle cabin mainly acquire images of the front of the vehicle 12100. The imaging units 12102 and 12103 provided on the side mirrors mainly acquire images of the sides of the vehicle 12100. The imaging unit 12104 provided on the rear bumper or back door mainly acquires images of the rear of the vehicle 12100. The imaging unit 12105 provided on the top of the windshield inside the vehicle cabin is mainly used to detect preceding vehicles, pedestrians, obstacles, traffic lights, traffic signs, lanes, etc.

[0281] 74 shows an example of the imaging ranges of the imaging units 12101 to 12104. Imaging range 12111 indicates the imaging range of the imaging unit 12101 provided on the front nose, imaging ranges 12112 and 12113 indicate the imaging ranges of the imaging units 12102 and 12103 provided on the side mirrors, respectively, and imaging range 12114 indicates the imaging range of the imaging unit 12104 provided on the rear bumper or back door. For example, by overlaying the image data captured by the imaging units 12101 to 12104, an overhead image of the vehicle 12100 viewed from above can be obtained.

[0282] At least one of the image capturing units 12101 to 12104 may have a function of acquiring distance information. For example, at least one of the image capturing units 12101 to 12104 may be a stereo camera made up of multiple image capturing elements, or may be an image capturing element having pixels for phase difference detection.

[0283] For example, based on the distance information obtained from the imaging units 12101 to 12104, the microcomputer 12051 can calculate the distance to each three-dimensional object within the imaging ranges 12111 to 12114 and the change in this distance over time (relative speed with respect to the vehicle 12100), thereby extracting as a preceding vehicle, in particular, the three-dimensional object that is the closest three-dimensional object on the path of the vehicle 12100 and traveling in approximately the same direction as the vehicle 12100 at a predetermined speed (e.g., 0 km / h or higher). Furthermore, the microcomputer 12051 can set a vehicle-to-vehicle distance to be maintained in advance in front of the preceding vehicle, and perform automatic braking control (including follow-up stop control), automatic acceleration control (including follow-up start control), etc. In this way, cooperative control can be performed for the purpose of autonomous driving, which runs autonomously without relying on driver operation.

[0284] For example, the microcomputer 12051 classifies and extracts three-dimensional object data regarding three-dimensional objects into two-wheeled vehicles, ordinary vehicles, large vehicles, pedestrians, utility poles, and other three-dimensional objects based on distance information obtained from the imaging units 12101 to 12104, and can use the data for automatic obstacle avoidance. For example, the microcomputer 12051 distinguishes obstacles around the vehicle 12100 into obstacles that are visible to the driver of the vehicle 12100 and obstacles that are difficult to see. The microcomputer 12051 then determines a collision risk that indicates the risk of collision with each obstacle, and when the collision risk is equal to or greater than a set value and a collision is possible, the microcomputer 12051 can provide driving assistance for collision avoidance by outputting an alarm to the driver via the audio speaker 12061 or the display unit 12062, or by performing forced deceleration or avoidance steering via the drive system control unit 12010.

[0285] At least one of the image capturing units 12101 to 12104 may be an infrared camera that detects infrared rays. For example, the microcomputer 12051 can recognize a pedestrian by determining whether or not a pedestrian is present in the images captured by the image capturing units 12101 to 12104. Such pedestrian recognition is performed, for example, by extracting feature points from the images captured by the image capturing units 12101 to 12104 as infrared cameras and performing pattern matching processing on a series of feature points that indicate the outline of an object to determine whether or not the object is a pedestrian. When the microcomputer 12051 determines that a pedestrian is present in the images captured by the image capturing units 12101 to 12104 and recognizes the pedestrian, the audio / image output unit 12052 controls the display unit 12062 to superimpose a rectangular outline on the recognized pedestrian for emphasis. The audio / image output unit 12052 may also control the display unit 12062 to display an icon or the like indicating the pedestrian at a desired position.

[0286] The above describes an example of a vehicle control system to which the technology according to the present disclosure can be applied. The technology according to the present disclosure can be applied to the imaging unit 12031 of the above-described configuration. Since a captured image that is easier to see can be obtained, it is possible to reduce driver fatigue.

[0287] The present technology can be configured as follows:

[0288] (1) An imaging device comprising: a pixel array unit having a plurality of first pixel circuits that output detection signals indicating whether an output signal corresponding to the amount of incident light has changed beyond a predetermined threshold, and a plurality of second pixel circuits that output gradation signals corresponding to the amount of incident light; and a control unit that controls imaging drive of the plurality of second pixel circuits based on the detection signals.

[0289] (2) The imaging device according to (1), wherein the plurality of first pixel circuits and the plurality of second pixel circuits are capable of simultaneously outputting the detection signal and the gradation signal.

[0290] (3) The imaging device described in (1) or (2), wherein each of the plurality of first pixel circuits comprises a first pixel having a first photoelectric conversion element that generates a photoelectric conversion signal on which the output signal is based in accordance with the amount of light, and each of the plurality of second pixel circuits comprises a second pixel having a second photoelectric conversion element that generates a photoelectric conversion signal on which the gradation signal is based in accordance with the amount of light, and the plurality of first pixels and the plurality of second pixels are arranged in a two-dimensional matrix on a light-receiving surface of the pixel array unit according to a predetermined rule, and the control unit selects the second pixel circuit that will perform the imaging drive from among the plurality of second pixel circuits based on the detection signal of each of the plurality of first pixel circuits.

[0291] (4) The imaging device according to any one of (1) to (3), further comprising a signal processing unit that selects the second pixel circuit that will perform the imaging drive based on the detection signal output by at least each of the plurality of first pixels, wherein the control unit selects the second pixel circuit that will perform the imaging drive based on information included in the output signal of the signal processing unit.

[0292] (5) The imaging device according to (4), wherein the control unit selects, as one or more regions, the second pixels that are to perform the imaging drive from among the plurality of second pixels, and performs different imaging drives on the second pixel circuits corresponding to the second pixels for each of the regions.

[0293] (6) The imaging device according to (5), wherein the control unit selects the region as one or more rectangular regions.

[0294] (7) The imaging device according to (5), wherein the control unit selects the region as a patch-unit region that is a combination of one or more of the second pixels.

[0295] (8) The imaging device described in any one of (4) to (7), wherein the signal processing unit selects the second pixels that perform the imaging drive as one or more regions, and based on the gradation signal, is capable of setting at least one of exposure time, frame rate, gain, and resolution as imaging conditions for the second pixel circuit for each region, and is capable of setting the threshold value of the first pixel circuit corresponding to the first pixel included in the region.

[0296] (9) The imaging device according to (8), wherein the plurality of first pixel circuits are repeatedly driven to output the detection signals in a predetermined cycle, and the signal processing unit changes the imaging conditions in response to the predetermined cycle.

[0297] (10) The imaging device according to (3), further comprising a plurality of AD converters corresponding to the plurality of second pixel circuits, respectively, wherein the control unit reduces power supplied to the AD converters corresponding to the second pixel circuits not selected as regions.

[0298] (11) The imaging device according to (4), wherein an external signal can be input to the signal processing unit.

[0299] (12) The imaging device described in (9), wherein the signal processing unit selects the second pixels to perform imaging driving as one or more regions based on information included in the external signal, and is capable of setting at least one of exposure time, frame rate, gain, and resolution as the imaging condition for each region, and is capable of setting the threshold value of the first pixel circuit corresponding to the first pixel included in the region.

[0300] (13) The imaging device according to (12), wherein the plurality of first pixel circuits are repeatedly driven to output the detection signals in a predetermined cycle, and the signal processing unit changes the imaging conditions in response to the predetermined cycle.

[0301] (14) The imaging device according to (13), wherein the external signal can be updated less frequently than the predetermined cycle.

[0302] (15) The imaging device according to (3), wherein the control unit is capable of selecting all of the second pixel circuits to be driven to perform the imaging drive and causing the second pixel circuits to perform the imaging drive.

[0303] (16) The imaging device according to (1), further comprising: an access control circuit that controls the imaging drive of the second pixel circuits based on control by the control unit.

[0304] (17) The imaging device according to (16), wherein the access control circuit controls the second pixel circuits arranged in a matrix via row direction control lines, and a column direction control line is connected to each of the second pixel circuits divided into a plurality of groups in the column direction.

[0305] (18) The imaging device according to (16), wherein the access control circuit includes: a first access control circuit that controls the second pixel circuits arranged in a matrix via a row-directional control line; and a second access control circuit that controls the second pixel circuits arranged in a matrix via a column-directional control line.

[0306] (19) The imaging device described in (18), wherein the second pixel circuit includes: a photoelectric conversion element that generates a photoelectric conversion signal in accordance with the amount of light; a first transfer transistor and a second transfer transistor connected in series between the photoelectric conversion element and a floating diffusion layer; a reset transistor having one end connected to the floating diffusion layer and the other end connected to a predetermined potential line; an amplification transistor having a gate connected to the floating diffusion layer and one end connected to the predetermined potential line; and a first selection transistor and a second selection transistor connected in series between the other end of the amplification transistor and a signal line, wherein one gate of the first transfer transistor and the second transfer transistor is connected to the first access control circuit and the other is connected to the second access control circuit; and one gate of the first selection transistor and the second selection transistor is connected to the first access control circuit and the other is connected to the second access control circuit.

[0307] (20) The imaging device according to (19), wherein the photoelectric conversion element is configured on a pixel chip, and the access control circuit is configured on a logic chip stacked on the pixel chip.

[0308] (21) The imaging device according to (17), wherein the column direction control lines are connected to the second pixel circuits arranged in a matrix in units of a plurality of columns.

[0309] (22) The imaging device according to (12), wherein information about the area and the imaging conditions are stored in output data.

[0310] (23) An electronic device comprising: an optical system that receives incident light; an imaging device that outputs a signal corresponding to the amount of light of the incident light that has passed through the optical system; a first control unit that can control the imaging device; and a recording unit that records the signal output by the imaging device, wherein the imaging device comprises: a pixel array unit having a plurality of first pixel circuits that output a detection signal indicating whether or not an output signal corresponding to the amount of light of the incident light has changed beyond a predetermined threshold, and a plurality of second pixel circuits that output a gradation signal corresponding to the amount of light of the incident light; and a second control unit that controls imaging driving of the plurality of second pixel circuits based on the detection signal.

[0311] (24) An imaging device comprising: a pixel array unit having at least a plurality of second pixel circuits that output gradation signals corresponding to the amount of incident light; a control unit that executes control to read out the gradation signals from at least two or more second pixel circuits among the plurality of second pixel circuits in a predetermined row order based on a detection signal that indicates whether or not the output signal corresponding to the amount of incident light has changed beyond a predetermined threshold; and a time information generation unit that generates time information regarding the readout time for each row of the two or more second pixel circuits.

[0312] (25) The imaging device according to (24), wherein the time information includes information on a scanning speed in the two or more second pixel circuits.

[0313] (26) The imaging device according to (24), wherein the scanning speed is a speed based on an inverse of a time required to read out a row range including the two or more second pixel circuits.

[0314] (27) The imaging device according to (25), wherein the time information includes at least one readout time within a row range in which the two or more second pixel circuits are read out.

[0315] (28) The imaging device according to (27), wherein the readout time is at least one of a readout start time, a readout end time, and a time between the readout start time and the readout end time in a row range in which the two or more second pixel circuits are read out.

[0316] (29) The imaging device according to (26), further comprising: an AD conversion unit that converts the gradation signal in a range where the second pixel circuit is read out into digital data; and a signal processing unit that generates the digital data and the time information in a predetermined data format.

[0317] (30) The imaging device according to (24), wherein the pixel array section further includes a plurality of first pixel circuits that output the detection signal indicating whether or not an output signal corresponding to the amount of incident light has changed beyond a predetermined threshold.

[0318] (31) The imaging device according to (30), wherein the time information generation unit is capable of generating time information relating to a readout time for each row of the first pixel circuits.

[0319] (32) The imaging device according to (24), further comprising a storage unit that stores the time information, wherein the control unit selects a range from which the second pixel circuits are read out based on the time information stored in the storage unit.

[0320] The aspects of the present disclosure are not limited to the individual embodiments described above, but include various modifications that may be conceived by those skilled in the art, and the effects of the present disclosure are not limited to the above-described contents. In other words, various additions, modifications, and partial deletions are possible within the scope of the conceptual idea and spirit of the present disclosure, which is derived from the contents defined in the claims and their equivalents.

[0321] 10: electronic device, 11: imaging lens (optical system), 12: recording unit, 13: control unit (AP), 13a: signal processing unit, 20: imaging device, 21: pixel array unit, 22: access control circuit, 23: control unit, 40: signal processing unit, 230: AD converter, 260: time information generation unit, 300: EVS pixel circuit, 400, 400R, 400G, 400B: gradation pixel circuits, 401: transfer transistor, 402: reset transistor, 403: amplification transistor, 404: selection transistor, 405: floating diffusion layer, 406: second transfer transistor, 407: second selection transistor, Main frame: entire image area, sub1, sub2, sub3: areas.

Claims

1. An imaging device comprising: a pixel array section having a plurality of first pixel circuits that output detection signals indicating whether an output signal corresponding to the amount of incident light has changed beyond a predetermined threshold, and a plurality of second pixel circuits that output gradation signals corresponding to the amount of incident light; and a control section that controls imaging drive of the plurality of second pixel circuits based on the detection signals.

2. The imaging device according to claim 1, wherein the plurality of first pixel circuits and the plurality of second pixel circuits are capable of simultaneously outputting the detection signal and the gradation signal.

3. The imaging device according to claim 1, wherein each of the plurality of first pixel circuits comprises a first pixel having a first photoelectric conversion element that generates a photoelectric conversion signal on which the output signal is based in accordance with the amount of light, and each of the plurality of second pixel circuits comprises a second pixel having a second photoelectric conversion element that generates a photoelectric conversion signal on which the gradation signal is based in accordance with the amount of light, and the plurality of first pixels and the plurality of second pixels are arranged in a two-dimensional matrix on a light-receiving surface of the pixel array unit according to a predetermined rule, and the control unit selects the second pixel circuit that will perform the imaging drive from among the plurality of second pixel circuits based on the detection signal of each of the plurality of first pixel circuits.

4. An imaging device as described in claim 3, further comprising a signal processing unit that selects the second pixel circuit that will perform the imaging drive based on the detection signal output by at least each of the plurality of first pixels, and the control unit selects the second pixel circuit that will perform the imaging drive based on information included in the output signal of the signal processing unit.

5. The imaging device described in claim 4, wherein the control unit selects one or more regions of the second pixels from among the plurality of second pixels to perform the imaging drive, and performs different imaging drives on the second pixel circuits corresponding to the second pixels for each region.

6. The imaging device according to claim 5, wherein the control unit selects the region as one or more rectangular regions.

7. The imaging device according to claim 5, wherein the control unit selects the region as a patch-unit region that is a combination of one or more of the second pixels.

8. The imaging device according to claim 4, wherein the signal processing unit selects the second pixels that will perform the imaging drive as one or more regions, and is capable of setting at least one of exposure time, frame rate, gain, and resolution as imaging conditions for the second pixel circuit for each region based on the gradation signal, and is capable of setting the threshold value of the first pixel circuit corresponding to the first pixels included in the region.

9. The imaging device according to claim 8, wherein the plurality of first pixel circuits are repeatedly driven to output the detection signals at a predetermined cycle, and the signal processing unit changes the imaging conditions in accordance with the predetermined cycle.

10. The imaging device of claim 3, further comprising a plurality of AD converters corresponding to the plurality of second pixel circuits, respectively, and wherein the control unit reduces the power supplied to the AD converters corresponding to the second pixel circuits not selected as areas.

11. The imaging device according to claim 4, wherein an external signal can be input to the signal processing section.

12. The imaging device described in claim 9, wherein the signal processing unit selects the second pixels to perform imaging drive as one or more regions based on information contained in the external signal, and is capable of setting at least one of exposure time, frame rate, gain, and resolution as the imaging conditions for each of the regions, and is capable of setting the threshold value of the first pixel circuit corresponding to the first pixels included in the region.

13. The imaging device according to claim 12, wherein the plurality of first pixel circuits are repeatedly driven to output the detection signals at a predetermined cycle, and the signal processing unit changes the imaging conditions in accordance with the predetermined cycle.

14. The imaging device according to claim 13, wherein the external signal can be updated less frequently than the predetermined cycle.

15. The imaging device according to claim 3, wherein the control unit is capable of selecting all of the plurality of second pixel circuits to be driven for imaging, and causing them to perform the imaging drive.

16. The imaging device according to claim 1, further comprising an access control circuit that controls the imaging drive of the plurality of second pixel circuits based on the control of the control unit.

17. The imaging device described in claim 16, wherein the access control circuit controls the plurality of second pixel circuits arranged in a matrix via row-direction control lines, and a column-direction control line is connected to each of the second pixel circuits divided into a plurality of groups in the column direction.

18. The imaging device described in claim 16, wherein the access control circuit comprises: a first access control circuit that controls the plurality of second pixel circuits arranged in a matrix via row-directional control lines; and a second access control circuit that controls the plurality of second pixel circuits arranged in a matrix via column-directional control lines.

19. The imaging device of claim 18, wherein the second pixel circuit comprises: a photoelectric conversion element that generates a photoelectric conversion signal in accordance with the amount of light; a first transfer transistor and a second transfer transistor connected in series between the photoelectric conversion element and a floating diffusion layer; a reset transistor having one end connected to the floating diffusion layer and the other end connected to a predetermined potential line; an amplifying transistor having a gate connected to the floating diffusion layer and one end connected to a predetermined potential line; and a first selection transistor and a second selection transistor connected in series between the other end of the amplifying transistor and a signal line, wherein one gate of the first transfer transistor and the second transfer transistor is connected to the first access control circuit and the other is connected to the second access control circuit; and one gate of the first selection transistor and the second selection transistor is connected to the first access control circuit and the other is connected to the second access control circuit.

20. The imaging device according to claim 19, wherein the photoelectric conversion elements are configured on a pixel chip, and the access control circuit is configured on a logic chip stacked on the pixel chip.

21. The imaging device according to claim 17, wherein the column direction control lines are connected in units of multiple columns of the second pixel circuits arranged in a matrix.

22. The imaging device according to claim 12, wherein information about the region and the imaging conditions are stored in output data.

23. An electronic device comprising: an optical system that receives incident light; an imaging device that outputs a signal corresponding to the amount of incident light that has passed through the optical system; a first control unit that can control the imaging device; and a recording unit that records the signal output by the imaging device, wherein the imaging device comprises: a pixel array unit having a plurality of first pixel circuits that output a detection signal indicating whether or not an output signal corresponding to the amount of incident light has changed beyond a predetermined threshold, and a plurality of second pixel circuits that output a gradation signal corresponding to the amount of incident light; and a second control unit that controls imaging driving of the plurality of second pixel circuits based on the detection signal.

24. An imaging device comprising: a pixel array unit having at least a plurality of second pixel circuits that output gradation signals corresponding to the amount of incident light; a control unit that executes control to read out the gradation signals from at least two or more second pixel circuits among the plurality of second pixel circuits in a predetermined row order based on a detection signal that indicates whether the output signal corresponding to the amount of incident light has changed beyond a predetermined threshold; and a time information generation unit that generates time information regarding the readout time for each row of the two or more second pixel circuits.

25. The imaging device according to claim 24, wherein the time information includes information on the scanning speed of the two or more second pixel circuits.

26. The imaging device according to claim 24, wherein the scanning speed is a speed based on the reciprocal of the time required to read out a row range including the two or more second pixel circuits.

27. The imaging device according to claim 25, wherein the time information includes at least one readout time within a row range for reading out the two or more second pixel circuits.

28. An imaging device as described in claim 27, wherein the readout time is at least one of a readout start time, a readout end time, and a time between the readout start time and the readout end time in a row range in which the two or more second pixel circuits are read out.

29. The imaging device described in claim 26, further comprising: an AD conversion unit that converts the gradation signal in the range that reads out the second pixel circuit into digital data; and a signal processing unit that generates the digital data and the time information in a predetermined data format.

30. An imaging device as described in claim 24, wherein the pixel array section further comprises a plurality of first pixel circuits that output the detection signal indicating whether or not an output signal corresponding to the amount of incident light has changed beyond a predetermined threshold value.

31. The imaging device according to claim 30, wherein the time information generating section is capable of generating time information relating to a readout time for each row of the first pixel circuits.

32. The imaging device according to claim 24, further comprising a storage unit that stores the time information, wherein the control unit selects a range for reading out the second pixel circuit based on the time information stored in the storage unit.

Images