Image sensor and image recognition system

Abstract

To suppress increase in power consumption, in arithmetic operation performed in an image sensor.SOLUTION: An image sensor for outputting a feature quantity for image recognition includes a plurality of pixel circuits, and a controller configured to execute first mode control which controls the pixel circuits. Each of the pixel circuits includes a photoelectric conversion element and a charge accumulation unit which receives charges transferred from the photoelectric conversion element, and the same is configured to output pixel signals in accordance with the amount of charges accumulated in the charge accumulation unit. The first mode control includes controlling transfer of the charges between the photoelectric conversion element and the charge accumulation unit for calculating the feature quantity.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present disclosure relates to an image sensor and an image recognition system. [Background technology]

[0002] In image recognition, feature values ​​of an image are extracted. For example, a machine learning model is used for image recognition. In the case of image recognition using a machine learning model, for example, feature values ​​of a high-resolution image as an input image are extracted, and the image is classified based on the feature values.

[0003] Non-Patent Document 1 discloses an image sensor that performs feature calculations (convolution calculations) inside a chip. The image sensor includes multiple pixels. Each pixel includes a photodiode, which is a photoelectric conversion element, and a readout circuit. The readout circuit for each pixel outputs a pixel signal, which is an electrical signal corresponding to the intensity of received light. The image sensor disclosed in Non-Patent Document 1 requires a special manufacturing process (IGZO) for accumulation and multiplication of the photoelectric conversion elements for performing the convolution calculations. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2016-219857 A [Non-patent literature]

[0005] [Non-Patent Document 1] Seiichi Yoneda et al., “Image Sensor Capable of Analog Convolution for Real-time Image Recognition System Using Crystalline Oxide Semiconductor FET”, International Image Sensor Workshop(IISW), 2019, pp.322-325. Summary of the Invention

[0006] Images that are the subject of image recognition have a large amount of data. Therefore, when extracting image features using an image recognizer, the calculation load on the image recognizer becomes large. Therefore, it is desirable to reduce the calculation load for extracting features in an image recognizer.

[0007] Here, when feature calculation is performed within the image sensor as in Non-Patent Document 1, it is expected that feature calculation in the image recognizer will become unnecessary or the calculation load will be reduced.

[0008] However, in Non-Patent Document 1, although feature calculation is performed inside the image sensor, it is performed by a calculation circuit using an operational amplifier, using pixel signals output from a readout circuit. When feature calculation is performed by the calculation circuit using an operational amplifier, a current corresponding to the magnitude of the pixel signal flows in the image sensor. This increases the power consumption of the image sensor.

[0009] Even when an image sensor performs a computation such as a feature computation, it is desirable to suppress an increase in power consumption. Here, the computation on the image is not limited to the feature computation, and other computations are also possible. Therefore, it is desirable to suppress an increase in power consumption in computations performed in the image sensor, not limited to the feature computation.

[0010] This problem is solved by performing a calculation using charges (photoelectric conversion charges) generated by photoelectric conversion. In an embodiment, the transfer of the photoelectric conversion charges is controlled for the calculation in a pixel circuit of an image sensor. Further details will be described as an embodiment below. [Brief description of the drawings]

[0011] [Figure 1] FIG. 1 is a configuration diagram of an image recognition system. [Diagram 2] FIG. 2 is a configuration diagram of a pixel unit. [Diagram 3] FIG. 3 is a circuit diagram of a pixel circuit. [Figure 4] FIG. 4 is an explanatory diagram of a filter and a calculation using the filter. [Diagram 5] FIG. 5 is a flowchart of the first mode control. [Figure 6] FIG. 6 is a timing chart of the first mode control. [Figure 7] FIG. 7 is a diagram illustrating the operation of the pixel circuit in the first mode control. [Figure 8] FIG. 8 is a diagram illustrating the operation of the pixel circuit in the first mode control. [Figure 9] FIG. 9 is a circuit diagram of a pixel circuit. [Figure 10] FIG. 10 is an explanatory diagram of the generation of a feature image. [Figure 11] FIG. 11 is a flowchart of the second mode control. [Figure 12] FIG. 12 is a timing chart of the second mode control. [Figure 13] FIG. 13 is a circuit diagram of a source follower circuit having a constant current source. [Figure 14] FIG. 14 is a timing chart showing the reading of pixel signals using a constant current source. [Figure 15] FIG. 15 is a circuit diagram of a source follower circuit having a switching circuit. [Figure 16] FIG. 16 is a timing chart showing the readout of pixel signals in a source follower circuit having a switching circuit. [Figure 17] FIG. 17 is a circuit diagram of a readout circuit including an analog-to-digital converter. [Figure 18] FIG. 18 is a timing chart showing the operation of the readout circuit during the second mode control. [Figure 19] FIG. 19 is a circuit diagram of a readout circuit including an analog-to-digital converter. [Figure 20] FIG. 20 is a timing chart showing the operation of the readout circuit during the first mode control. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] <1. Overview of image sensors and image recognition systems>

[0013] (1) An image sensor according to an embodiment outputs a feature quantity for image recognition. The image sensor according to an embodiment includes a plurality of pixel circuits and a controller configured to execute a first mode control for controlling the plurality of pixel circuits. Each of the plurality of pixel circuits includes a photoelectric conversion element and a charge accumulation unit that receives a charge transferred from the photoelectric conversion element, and is configured to output a pixel signal according to the amount of charge accumulated in the charge accumulation unit. The first mode control includes controlling the transfer of the charge between the photoelectric conversion element and the charge accumulation unit for a calculation to obtain the feature quantity. According to the image sensor according to an embodiment, a calculation is performed in the pixel circuit by controlling the transfer of the charge. In other words, the pixel circuit also functions as a calculation circuit.

[0014] (2) In the first mode control, it is preferable that controlling the transfer of the electric charges includes distributing electric charges generated in n or more (n is a natural number of 1 or more) photoelectric conversion elements to m or more (m is a natural number of 2 or more) charge accumulation units. In this case, a calculation is performed to divide the electric charges generated in the n photoelectric conversion elements by m.

[0015] (3) The plurality of pixel circuits preferably include a first pixel circuit and a second pixel circuit. The first pixel circuit preferably includes a first photoelectric conversion element and a first charge accumulation unit that receives a first charge transferred from the first photoelectric conversion element. The second pixel circuit preferably includes a second photoelectric conversion element and a second charge accumulation unit that receives a second charge transferred from the second photoelectric conversion element. Controlling the charge transfer in the first mode control preferably includes controlling the transfer of the first charge and the second charge between a plurality of photoelectric conversion elements including the first photoelectric conversion element and the second photoelectric conversion element and a plurality of charge accumulation units including the first charge accumulation unit and the second charge accumulation unit, for a calculation to obtain the feature amount.

[0016] (4) In the first mode control, it is preferable that the control of the transfer of the charges includes distributing the charges including the second charges to a plurality of charge accumulation units including the first charge accumulation unit and the second charge accumulation unit. In this case, division of the charges including the second charges is performed in a pixel circuit.

[0017] (5) It is preferable that controlling the transfer of the charges in the first mode control includes distributing charges including the second charges to a plurality of charge storage sections including the first charge storage section and the second charge storage section, resetting the charge in any one of the plurality of charge storage sections, and transferring the first charges to the reset charge storage section.

[0018] (6) It is preferable that the first mode control includes obtaining an output signal corresponding to a difference between a first voltage corresponding to the first charge and a second voltage corresponding to a charge stored in any one of the plurality of charge storage sections.

[0019] (7) The controller is configured to further execute a second mode control different from the first mode control, and the second mode control preferably includes controlling the transfer of the charge between the photoelectric conversion element and the charge storage unit so that a pixel signal corresponding to the amount of charge generated in the photoelectric conversion element is output. In the second mode control, a general image is obtained instead of a feature amount.

[0020] (8) It is preferable that the controller is configured to switch from the first mode control to the second mode control when a switching signal is given to the controller.

[0021] (9) The image sensor preferably includes a source follower circuit including a transistor that generates a voltage corresponding to the amount of charge in the charge storage section as a source voltage. The source follower circuit preferably has a switching circuit connected to the source of the transistor. In this case, power consumption in the source follower circuit can be reduced.

[0022] (10) The image sensor preferably includes an analog-digital converter that converts the pixel signal into a digital signal. The analog-digital converter preferably includes an amplifier that outputs a difference between the pixel signal and a reference signal, and a latch-type comparator to which the difference is provided. In the second mode control, the analog-digital converter preferably operates to output the difference using the amplifier and provide the difference to the latch-type comparator. In the first mode control, the analog-digital converter preferably operates to provide the pixel signal and the reference signal to the latch-type comparator without using the amplifier.

[0023] (11) It is preferable that the image recognition system according to the embodiment includes an image sensor according to any one of (1) to (10) and an image recognizer that performs image recognition based on features output from the image sensor.

[0024] (12) An image recognition system according to an embodiment preferably includes the image sensor according to any one of (8) to (10) and an image recognizer that detects an event by performing image recognition based on a feature output from the image sensor. The controller included in the image sensor is preferably configured to switch from the first mode control to the second mode control when the event is detected.

[0025] (13) An image sensor according to an embodiment preferably includes a plurality of pixel circuits and a controller configured to execute a first mode control for controlling the plurality of pixel circuits. Each of the plurality of pixel circuits preferably includes a photoelectric conversion element and a charge accumulation unit that receives charge transferred from the photoelectric conversion element, and is configured to output a pixel signal according to the amount of charge accumulated in the charge accumulation unit. The first mode control preferably includes distributing charge generated by photoelectric conversion in n or more photoelectric conversion elements (n is a natural number equal to or greater than 1) to m or more charge accumulation units (m is a natural number equal to or greater than 2).

[0026] (14) An image sensor according to an embodiment preferably includes a plurality of pixel circuits and a controller configured to execute a first mode control for controlling the plurality of pixel circuits. The plurality of pixel circuits preferably include a first pixel circuit and a second pixel circuit. The first pixel circuit preferably includes a first photoelectric conversion element and a first charge accumulation unit that receives a first charge transferred from the first photoelectric conversion element. The second pixel circuit preferably includes a second photoelectric conversion element and a second charge accumulation unit that receives a second charge transferred from the second photoelectric conversion element. The first mode control preferably includes controlling the transfer of the first charge and the second charge between a plurality of photoelectric conversion elements including the first photoelectric conversion element and the second photoelectric conversion element and a plurality of charge accumulation units including the first charge accumulation unit and the second charge accumulation unit.

[0027] (15) It is preferable that the first mode control includes obtaining an output voltage corresponding to a difference between a first voltage corresponding to the first charge and a second voltage corresponding to a charge stored in any one of the plurality of charge storage sections to which charge including the second charge is distributed.

[0028] <2. Example of an overview of an image sensor and image recognition system>

[0029] 1 shows a system 10 including an image sensor 100 according to an embodiment. As an example, the system 10 is an image recognition system 10, and includes an image recognizer 200. The system 10 may be another system in which the image sensor 100 is used. In the system 10, a device provided downstream of the image sensor 100 may be a device other than the image recognizer 200. The device other than the image recognizer 200 is, for example, an image processing device.

[0030] In a Trillion Sensor society, low power consumption is required. The image sensor 100 according to the embodiment consumes low power as described below, and the power consumption of the image recognizer 200 (artificial intelligence; AI) provided in the subsequent stage of the image sensor 100 can also be reduced. Therefore, the power consumption of the entire system 10 can be reduced, which is suitable for a Trillion Sensor society.

[0031] The image sensor 100 according to the embodiment is used for event detection. The detected event is, for example, an abnormality or change in a monitoring target. The monitoring target is a structure such as a building or a bridge, a living thing such as a person or an animal, or a vehicle such as a car, a motorcycle, a bicycle, or a train. The monitoring target is not particularly limited. The type of the event is also not particularly limited.

[0032] In order to detect an event, the image sensor 100 according to the embodiment outputs a feature image (first image) having a small amount of data with low power consumption during normal operation 121 (first mode control 121). The feature image is provided to the image recognizer 200.

[0033] The image recognizer 200 uses, for example, a machine learning model 210 for image recognition. The machine learning model 210 outputs an image recognition result (image classification result) from an input image. During normal operation 121, the image recognizer 200 performs image recognition using a feature image output from the image sensor 100 as an input image. As an example, the image recognizer 200 includes an event detector 220 that detects an event based on the image recognition result. When the event detector 220 detects the occurrence of an event, it outputs a switching signal (first switching signal) from the normal operation 121 to the event operation 122. The first switching signal is provided to the image sensor 100.

[0034] When the first switching signal is given, the image sensor 100 switches to the event operation 122 (second mode control 122). During the event operation 122 (second mode control 122), the image sensor 100 outputs a high-resolution image (second image) having a large amount of data. The high-resolution image here means a normal image that is not a feature image. In the high-resolution image, each pixel has a pixel value corresponding to the amount of charge generated in the photoelectric conversion element PD of the image sensor 100. In contrast, in the feature image, each pixel has a pixel value as a result of feature calculation using the charge generated in the photoelectric conversion element PD. The calculation using the charge will be described later.

[0035] As described above, during the event operation 122 (second mode control 122), the image sensor 100 outputs a high-resolution image. The high-resolution image is stored in a storage device, for example, to record an event that has occurred (anomaly or change in the monitored object). The high-resolution image may be provided to the image recognizer 200.

[0036] During the event operation 122, the image recognizer 200 performs image recognition using the high-resolution image output from the image sensor 100 as an input image. The event detector 220 can detect whether the event is continuing or has ended based on the image recognition result. When the event detector 220 detects that an event has not occurred, it outputs a switching signal (second switching signal) from the event operation 122 to the normal operation 121. The second switching signal is provided to the image sensor 100. When the second switching signal is provided, the image sensor 100 returns to the normal operation 121 (first mode control 121).

[0037] In image recognition using a general machine learning model, a high-resolution image is provided as an input image, features of the high-resolution image are extracted, and classification (classification) is performed based on the features. That is, the machine learning model includes a feature extraction unit and a classification unit. The feature extraction unit extracts features from the high-resolution image. The classification unit classifies the image based on the features. Since the high-resolution image provided as an input image to the machine learning model has a large amount of data, the computational load on the feature extraction unit in the machine learning model is large, and the power consumption is also large.

[0038] However, since the image sensor 100 according to the embodiment outputs a feature image during normal operation 121 (first mode control 121), the feature image is provided as an input image to the machine learning model 210. Therefore, the machine learning model 210 does not need to perform feature extraction calculation, and only needs to perform classification calculation from the feature image provided as the input image. Therefore, during normal operation 121, the feature extraction calculation in the machine learning model 210 can be omitted or reduced, and low power consumption in the image recognizer 200 can be achieved.

[0039] The image sensor 100 according to the embodiment is, for example, a CMOS (Complementary Metal Oxide Semiconductor) image sensor. The image sensor 100 includes a pixel unit 110 as an imaging unit and a controller 120. The image sensor 100 further includes a readout circuit 130, a vertical scanning circuit 140, and a horizontal scanning circuit 150. The readout circuit 130 reads out pixel signals from the pixel unit 110. The vertical scanning circuit 140 and the horizontal scanning circuit 150 provide the pixel unit 110 with a signal for selecting a target pixel circuit (target pixel) from which a pixel signal is to be read, among pixel circuits 111 (pixels) included in the pixel unit 110.

[0040] The controller 120 controls the operation of the pixel unit 110 via the vertical scanning circuit 140 and the horizontal scanning circuit 150. In other words, the "controller" may be a concept that includes not only the above-mentioned controller 120 but also the vertical scanning circuit 140 and the horizontal scanning circuit 150.

[0041] The controller 120 of the embodiment controls the pixel unit 110 to output a feature image from the image sensor 100 in the first mode control 121. Moreover, the controller 120 controls the image sensor 100 to output a high-resolution image in the second mode control 122. The controller 120 receives a switching signal from the outside and can switch the control by the controller 120 from the first mode control 121 to the second mode control 122 or from the second mode control 122 to the first mode control 121. That is, the controller 120 includes a switch 125 that switches the mode of the controller 120 between the first mode control 121 and the second mode control 122 based on the switching signal.

[0042] 2, the pixel section 110 includes a plurality of pixel circuits 111, 111A, 111B, and 111C arranged in a two-dimensional array. Note that the pixel circuits are also simply called pixels.

[0043] Fig. 3 shows an example of a pixel circuit included in the pixel unit 110. In Fig. 3, one pixel circuit 111B in the J-1th row, one pixel circuit 111A in the Jth row, and one pixel circuit 111C in the J+1th row are shown. The pixel circuits 111, 111A, 111B, and 111C included in the pixel unit 110 have a common structure. Note that in Fig. 3, the pixel circuit 111B and the pixel circuit 111C are depicted in a simplified manner compared to the pixel circuit 111A for convenience, but in reality, they have the same structure as the pixel circuit 111A.

[0044] The structure of the pixel circuit will be described below by taking the pixel circuit 111A in the Jth row as an example. The pixel circuit 111A includes a photoelectric conversion element PD. The photoelectric conversion element PD is, for example, a photodiode. The photoelectric conversion element PD accumulates an amount of charge according to the amount of light. The charge is, for example, an electron. The charge generated by photoelectric conversion is called a photoelectric conversion charge. Also, the electron generated by photoelectric conversion is called a photoelectric conversion electron.

[0045] The pixel circuit 111A includes a transfer gate TG connected to the photoelectric conversion element PD. The transfer gate TG transfers the charge accumulated in the photoelectric conversion element PD. The transfer gate TG is, for example, configured of a transistor (transfer transistor). The transistor is, for example, a MOSFET. Note that other transistors described below may also be MOSFETs. When a High signal is applied between the gate and source of the MOSFET, the drain and source are in a conductive state, and when a Low signal is applied between the gate and source, the drain and source are in a non-conductive state. In the following, application of a High signal between the gate and source is referred to as "on" and application of a Low signal is referred to as "off."

[0046] When the transfer gate TG is turned on, it becomes conductive and transfers the charge accumulated in the photoelectric conversion element PD to a charge accumulation section described below. When the transfer gate TG is turned off, it becomes non-conductive and prevents the transfer of the charge accumulated in the photoelectric conversion element PD.

[0047] The pixel circuit 111A includes a charge storage unit that receives charge transferred from the photoelectric conversion element PD via a transfer gate TG. In the embodiment, the charge storage unit has at least a floating diffusion FD1 (floating diffusion layer) that is a first capacitor connected to the transfer gate TG. The floating diffusion FD1 has a first capacitance C FD The floating diffusion FD1 is connected to a transfer gate TG so as to receive the charge transferred from the photoelectric conversion element PD.

[0048] In the embodiment, the charge storage section further includes a second switching gate SG2 which is a second capacitor connected to the floating diffusion FD1. The second switching gate SG2 is, for example, configured by a transistor (second switching transistor). When the second switching gate SG2 is turned on, it acts as a second capacitor and can store the transferred charge (electrons). The second switching gate SG2 is connected to a transfer gate TG together with the floating diffusion FD1 so as to receive the transfer of charge from the photoelectric conversion element PD.

[0049] The second switching gate SG2 acting as a second capacitor has a second capacitance [F]. The second capacitance is, for example, 0.5 C. FD When the second switching gate SG2 is turned on, the capacitance of the charge storage unit is the first capacitance plus the second capacitance. For example, C FD +0.5 CF =1.5C FD [F] When the second switching gate SG2 is turned on and charges are transferred, the charges are distributed and stored in the floating diffusion FD1 and the second switching gate SG2.

[0050] When the second switching gate SG2 is turned off, it does not store charge (electrons). Therefore, the capacitance of the charge storage section when the second switching gate SG2 is turned off is the same as the first capacitance, C FD [F].

[0051] In this way, the capacitance of the charge storage unit according to the embodiment is variable. That is, the charge storage unit according to the embodiment is a variable capacitor. By changing the capacitance of the charge storage unit, even if the amount of charge transferred from the photoelectric conversion element PD is the same, the voltage V FD In other words, by increasing the capacity of the charge storage section in which the charge is stored, the voltage V that appears in the floating diffusion FD1 can be changed. FD By reducing the capacitance, the voltage V that appears on the floating diffusion FD1 can be reduced. FD can be increased.

[0052] The pixel circuit 111A includes a reset gate RST connected to a charge accumulation unit. In the embodiment, the charge accumulation unit is connected to a floating diffusion FD1 and a second switching gate SG2. The reset gate RST discharges the charge accumulated in the charge accumulation unit. The reset gate RST is connected between the floating diffusion FD1 and a power supply (voltage: V DD ) is connected between

[0053] When the reset gate RST is turned on, it becomes conductive and discharges the charge stored in the floating diffusion FD1 or the second switching gate SG2, which is a charge storage unit, to the power supply. This discharge of charge is called resetting. By resetting, the potential of the charge storage unit is reduced by the power supply voltage V DD When the reset gate RST is turned off, it becomes non-conductive and prevents discharge of electric charge.

[0054] The pixel circuit 111A includes a transistor 20 that constitutes a source follower circuit 40 together with a constant current source CS described later. The transistor 20 is also called an amplification transistor or a source follower transistor. The transistor 20 has a gate terminal connected to a floating diffusion FD1 (charge storage unit) and a drain terminal connected to a power supply (voltage: V DDThe transistor 20 generates a voltage vx (pixel signal) at its source terminal according to the amount of charge stored in the charge storage section (and the capacitance of the charge storage section). The voltage vx (pixel signal) generated at the source terminal of the transistor 20 is a voltage V DD From the above, the voltage V at the floating diffusion FD1 FD and the drain-source voltage of the transistor, V GS The voltage drop (V DD -V FD -V GS ).

[0055] The pixel circuit 111A includes a selection gate SEL. The selection gate SEL is, for example, composed of a transistor (selection transistor). When the selection gate SEL is turned on, it becomes conductive and connects the source terminal of the transistor 20 to a signal line 135 for reading out a pixel signal. The selection gate SEL is an element for selecting a target pixel circuit (target pixel) from among the multiple pixel circuits 111A, 111B, 111C, and 111, which is a target for reading out a voltage in a charge accumulation section. The voltage in the charge accumulation section of the target pixel circuit is read out by the readout circuit 130. When the selection gate SEL is turned off, it becomes non-conductive and disconnects the transistor 20 from the signal line 135.

[0056] The pixel circuit 111A includes a first switching gate SG. The first switching gate SG is, for example, configured by a transistor (first switching transistor). When the first switching gate SG is turned on, it becomes conductive and connects the charge storage unit of the pixel circuit 111A to the charge storage unit of another pixel circuit 111C. When the first switching gate SG is turned off, it becomes non-conductive and disconnects the charge storage unit of the pixel circuit 111A from the charge storage unit of another pixel circuit 111C.

[0057] In other words, the first switching gate SG is an element for switching between connection / disconnection between a charge storage section provided in the pixel circuit 111A in which the first switching gate SG is provided and a charge storage section provided in another pixel circuit 111C adjacent to the pixel circuit 111A.

[0058] When the first switching gate SG of the pixel circuit 111A is brought into a conductive state, the charge accumulation portion (floating diffusion FD1) of the pixel circuit 111A and the charge accumulation portion (floating diffusion FD3) of the pixel circuit 111C are connected.

[0059] Furthermore, when the first switching gate SG of the pixel circuit 111B is brought into a conductive state, the charge accumulation portion (floating diffusion FD2) of the pixel circuit 111B and the charge accumulation portion (floating diffusion FD1) of the pixel circuit 111A are connected.

[0060] When both the first switching gate SG of pixel circuit 111A and the first switching gate SG of pixel circuit 111B are in a conductive state, the charge accumulation section (floating diffusion FD2) of pixel circuit 111B, the charge accumulation section (floating diffusion FD1) of pixel circuit 111A, and the charge accumulation section (floating diffusion FD3) of pixel circuit 111C are connected.

[0061] The read circuit 130 includes a constant current source CS which constitutes a source follower circuit 40 together with the above-mentioned transistor 20. The constant current source CS is connected to a signal line 135. The read circuit 130 includes a capacitor C CDS Capacitor C CDS is used for correlated double sampling, described below.

[0062] The read circuit 130 includes a clip gate CLP. The clip gate CLP is, for example, configured of a transistor (clip transistor). When the clip gate CLP is turned on, it becomes conductive and connects the signal line 135 and the ground GND. When the clip gate CLP is turned off, it becomes non-conductive and disconnects the signal line 135 from the ground GND. The operation of the clip gate CLP will be described later.

[0063] The above-mentioned gates are driven and controlled by a controller 120. The control of each gate will be described later.

[0064] 4 shows a filter K for a convolution operation for feature calculation realized by the pixel circuits 111A, 111B, and 111C according to the embodiment, and a convolution operation using the filter K. If the pixel value of a pixel of interest (target pixel) is I(1) and the pixel values ​​of pixels vertically adjacent to I(1) are I(2) and I(3), the feature I'(1) of the pixel of interest (target pixel) is calculated as shown in FIG. 4 using a horizontal stripe filter K. Here, the filter K=(-0.5, 1, -0.5). Generally, such feature extraction calculation is performed by an image recognizer 200 or the like, but in this embodiment, such feature calculation is performed inside the image sensor 100.

[0065] 5 and 6 show a procedure in which the controller 120 controls the pixel unit 110 for feature amount calculation during the first mode control 121. Here, a method for driving three pixel circuits, the pixel circuit 111B in the J-1th row, the pixel circuit 111A in the J-1th row, and the pixel circuit 111C in the J-1th row, will be described in order to perform calculations for three pixels corresponding to the horizontal stripe filter K shown in FIG. 4. By the driving method shown in FIG. 5 and FIG. 6, a calculation equivalent to the convolution calculation of the filter K shown in FIG. 4 is executed in the image sensor 100.

[0066] First, the controller 120 turns on the selection gate SEL(J) and clip gate CLP of the pixel circuit 111A in the Jth row (step S11). Here, the pixel circuit 111A in the Jth row is the target pixel circuit.

[0067] Next, the controller 120 turns on the reset gate RST(J-1) of the pixel circuit 111B in the J-1th row, the reset gate RST(J) of the pixel circuit 111A in the Jth row, and the reset gate RST(J+1) of the pixel circuit 111C in the J+1th row (step S12). This resets the floating diffusions FD1, FD2, and FD3 in the J-1th, Jth, and J+1th rows, respectively. The potentials of the floating diffusions FD1, FD2, and FD3 are set to the power supply voltage V DD become.

[0068] Then, the controller 120 turns on the first switching gate SG(J) of the pixel circuit 111A in the Jth row and the first switching gate SG(J-1) of the pixel circuit in the J-1th row, and turns on the transfer gate TG(J-1) of the pixel circuit 111B in the J-1th row and the transfer gate TG(J+1) of the pixel circuit 111C in the J+1th row (step S13).

[0069] The first switching gate SG(J) of the pixel circuit 111A in the Jth row and the first switching gate SG(J-1) of the pixel circuit in the J-1th row are turned on, so that the three floating diffusions FD1, FD2, and FD3 are connected to each other. This allows the three floating diffusions FD to cooperate with each other to store electric charges.

[0070] In addition, when the transfer gate TG(J-1) of the pixel circuit 111B in the J-1th row and the transfer gate TG(J+1) of the pixel circuit 111C in the J+1th row are turned on, the second charge Q(J-1) of the photoelectric conversion element PD of the pixel circuit 111B in the J-1th row and the third charge Q(J+1) of the photoelectric conversion element PD in the J+1th row are transferred.

[0071] 7, the second charge Q(J-1) and the third charge Q(J+1) are transferred to be distributed to the three floating diffusions FD1, FD2, and FD3 that are connected to each other. The sum {Q(J-1)+Q(J+1)} of the second charge Q(J-1) and the third charge Q(J+1) is evenly distributed to the three floating diffusions FD1, FD2, and FD3.

[0072] At the time of step S13, the second switching gate SG2(J) is turned off, so the capacitance of the charge storage section of each of the pixel circuits 111A, 111B, and 111C is the first capacitance of the floating diffusion FD=C FD It is.

[0073] Here, by distributing the electric charge to three floating diffusions FD1, FD2, and FD3 (first charge storage unit FD1, second charge storage unit FD2, and third charge storage unit FD3), an electric charge of {Q(J-1)+Q(J+1)} / 3 is stored in each of the floating diffusions FD1, FD2, and FD3. In other words, an electric charge of {Q(J-1)+Q(J+1)} / 3 is stored in the floating diffusion FD1 (see FIG. 7).

[0074] By distributing and storing the charge in three locations, the amount of charge equivalent to {Q(J-1)+Q(J+1)} divided by "3" is stored. In other words, division according to the number of distributions (3 in this case) is realized. In this way, the transfer of charge is controlled so that the charge is distributed to multiple floating diffusions FD (charge storage units), thereby realizing the division of the charge {Q(J-1)+Q(J+1)}.

[0075] Here, the charge generated in two (n=2) photoelectric conversion elements PD is distributed to three (m=3) charge accumulation units, but this is not limited to this. When implementing division, n may be a natural number equal to or greater than 1, and m may be a natural number equal to or greater than 2. For example, the charge generated in one (n=1) photoelectric conversion element PD may be distributed to m charge accumulation units. Also, the charge generated in three or more (m=3) photoelectric conversion elements may be distributed to m charge accumulation units. It is preferable that m is greater than n.

[0076] At the time of step S13, the clip gate CLP (and the selection gate SEL(J) of the pixel circuit 111A) is on, so the voltage V FD ={Q(J-1)+Q(J+1)} / 3C FD The potential difference according to the capacitor C CDS The resulting capacitor C CDS The potential difference between both ends is V DD -{Q(J-1)+Q(J+1)} / 3C FD -V GS It is.

[0077] Then, the controller 120 turns off the clip gate CLP (step S14). CDS The potential difference between both ends of the capacitor C CDS A signal corresponding to the division result {Q(J-1)+Q(J+1)} / 3 is held in the register.

[0078] Next, the controller 120 turns on the reset gate RST(J) of the pixel circuit 111A in the Jth row (step S15). This resets the floating diffusion FD of the pixel circuit 111A. Furthermore, prior to step S15, the second switching gate SG2(J) is turned on. By turning on the second switching gate SG2(J), the second switching gate SG2(J) as the second capacitor also constitutes the charge storage section of the pixel circuit 111A. In other words, at the time of step S15 (and the next step S16), the capacitance of the charge storage section of the pixel circuit 111A is expressed as follows: first capacitance+second capacitance=C FD +0.5 CF =1.5C FD It is [F]. In step S15, the second switching gate SG2(J) is also reset.

[0079] Then, the controller 120 turns on the transfer gate TG(J) of the pixel circuit 111A in the Jth row (step S16). By turning on the transfer gate TG, the first charge Q(J) accumulated in the photoelectric conversion element PD of the pixel circuit 111A is transferred to the charge accumulation section of the pixel circuit 111A. The first charge Q(J) is transferred to the floating diffusion FD1 and the second switching gate SG2 constituting the charge accumulation section so that the charge is distributed (see FIG. 8).

[0080] At the time of step S16, the first switching gate SG(J) of the pixel circuit 111A and the first switching gate SG(J-1) of the pixel circuit 111B are turned off, so that the first charge Q(J) is transferred only to the charge storage section of the pixel circuit 111A.

[0081] At the time of step S16, the capacitance of the charge storage unit of the pixel circuit 111A is 1.5C. FD [F], when the first charge Q(J) is accumulated in the charge storage section, the voltage V of the floating diffusion FD1 FD V FD =Q(J) / 1.5C FD become.

[0082] Therefore, the pixel signal vx = V DD -Q(J) / 1.5C FD -V GS (See FIG. 8.) The output signal vout of the read circuit 130 is CDS Since the difference between the potential difference (second voltage) between both ends of the pixel signal vx (first voltage) is the output signal vout=2{Q(J)-0.5Q(J-1)-0.5Q(J+1)} / 3C FD is obtained (see FIG. 8). That is, the output signal vout is a signal corresponding to the difference between a first voltage corresponding to the first charge amount Q(J) and a second voltage corresponding to the charge amount {Q(J-1)+Q(J+1)} / 3 stored in the charge storage unit of the pixel circuit 111A among the three charge storage units.

[0083] In the embodiment, as an example, a signal corresponding to {Q(J)-0.5Q(J-1)-0.5Q(J+1)} is obtained as the output signal vout. {Q(J)-0.5Q(J-1)-0.5Q(J+1)} is the sum of the first charge Q(J) multiplied by 1, the first charge Q(J-1) multiplied by -0.5, and the first charge Q(J+1) multiplied by -0.5. Therefore, {Q(J)-0.5Q(J-1)-0.5Q(J+1)} is equivalent to the result (feature) of multiplying Q(J), Q(J-1), and Q(J+1) by the filter K shown in FIG. 4. Therefore, the feature is obtained as the output signal vout obtained as a result of controlling the transfer of the charge. The output signal vout is also a pixel signal.

[0084] As described above, the feature amount calculation according to the embodiment is realized by controlling the transfer of electric charges. The control of the transfer of electric charges can be achieved by switching the gates included in the pixel circuit 111 or the like, and therefore consumes less power than the calculation using an operational amplifier as in Non-Patent Document 1. The image sensor 100 according to the embodiment can be manufactured by a standard CMOS image sensor manufacturing process, and does not require a special manufacturing process (IGZO) as in Non-Patent Document 1, and therefore the manufacturing cost is low.

[0085] In the above explanation, the calculation for three pixels in the vertical direction (pixel circuits 111A, 111B, 111C) has been described as an example, but the number of pixels to be calculated is not limited to three and may be, for example, five or more.

[0086] In addition, the pixels to be calculated are not limited to the pixels in the vertical direction, but may be the pixels in the horizontal direction. When the pixels in the horizontal direction are the pixels to be calculated, a vertical stripe filter operation is possible. Furthermore, the pixels to be calculated may be the pixels included in a two-dimensional range in the vertical and horizontal directions, in which case a Laplacian filter operation is possible.

[0087] To perform the calculation of the vertical stripe filter, it is necessary to connect a plurality of pixel circuits 111 (floating diffusions) in the horizontal direction in the above-mentioned step S13. Also, to perform the calculation of the Laplacian filter, it is necessary to connect a plurality of pixel circuits 111 (floating diffusions) in the vertical and horizontal directions in the above-mentioned step S13.

[0088] 9 shows the configuration of the pixel circuit 111 for connection in the vertical and horizontal directions. In FIG. 9, the first switching gate SG shown in FIG. 3 is the vertical first switching gate SG V The vertical first switching gate SG V 9, the pixel circuit 111A shown in FIG. 3 is provided with a horizontal first switching gate SG H The first horizontal switching gate SG H 9 is responsible for connection to the charge storage portion of another horizontally adjacent pixel circuit 111. In FIG. 9, the other configuration is the same as in FIG.

[0089] Horizontal first switching gate SG H The horizontal first switching gate SGH When turned on, the first horizontal switching gate SG H The horizontal direction first switching gate SG H When turned off, the horizontal first switching gate SG H The charge storage section of the pixel circuit 111 in which the pixel circuit 111 is provided is disconnected from the charge storage sections of other pixel circuits 111 in the horizontal direction.

[0090] Vertical first switching gate SG V When the first horizontal switching gate SG is turned on, multiple pixel circuits 111 in the vertical direction are connected, and a horizontal stripe filter can be calculated. H When the first vertical switching gate SG is turned on, a plurality of pixel circuits 111 in the horizontal direction are connected, and a vertical stripe filter can be calculated. V and the horizontal first switching gate SG H When turned on, a plurality of pixel circuits 111 in the vertical and horizontal directions are connected, making it possible to perform Laplacian filter calculations.

[0091] FIG. 10 shows an example of how to obtain a feature image. In the above explanation, the feature of one target pixel is 2{Q(J)-0.5Q(J-1)-0.5Q(J+1)} / 3C FD The feature image is obtained by calculating the above-mentioned feature amounts for all pixels.

[0092] In the embodiment, as shown in FIG. 10, one feature image is obtained in a total of three frames, the first frame, the second frame, and the third frame. As an example, as shown in FIG. 10, in the first frame, the selection gates SEL(2), SEL(5), SEL(8), etc. are turned on to read out the pixel circuits 111 in the second, fifth, eighth, etc. rows. In the second frame, the selection gates SEL(3), SEL(6), SEL(9) are turned on to read out the pixel circuits 111 in the third, sixth, ninth, etc. rows. In the third frame, the selection gates SEL(4), SEL(7), SEL(10), etc. are turned on to read out the pixel circuits 111 in the fourth, seventh, tenth, etc. rows. By reading out in three frames, one feature image is obtained.

[0093] Since the charges accumulated in the photoelectric conversion element PD are different in different frames (different times), there is a risk of motion blurring. However, motion blurring can be sufficiently reduced by operating the image sensor 100 at a sufficiently high speed. For example, if the image sensor 100 is operated at 30 frames / sec, the convolution operation of a horizontal stripe filter can be completed in 33 milliseconds. In addition, in the case of a 3×3 Laplacian filter, if the image sensor 100 is operated at 90 frames / sec, the convolution operation can be completed in 33 milliseconds. Furthermore, when extracting feature amounts, high-speed operation is possible by reading out a low-resolution signal as described later. Furthermore, operation with low power consumption is possible.

[0094] 11 and 12 show a procedure in which the controller 120 controls the pixel unit 110 to obtain a high-resolution image in the second mode control 122. In the second mode control 122, pixel signals are read out by a correlated double sampling method.

[0095] First, the controller 120 turns on the selection gate SEL(J) and clip gate CLP of the pixel circuit 111A in the Jth row, which is a target pixel circuit (step S21).

[0096] Next, the controller 120 turns on the reset gate RST(J) of the pixel circuit 111A in the Jth row (step S22). This resets the floating diffusion FD1 in the Jth row. The voltage vx of the pixel signal of the pixel circuit 111A at this time is set to a reset voltage V RST He said.

[0097] Then, the controller 120 turns off the clip gate CLP (step S23). CDS The potential difference between both ends of the reset voltage V RST is retained.

[0098] The controller 120 turns on the transfer gate TG(J) of the pixel circuit 111A in the Jth row (step S24). By turning on the transfer gate TG, the first charge Q(J) stored in the photoelectric conversion element PD of the pixel circuit 111A is transferred to the charge storage unit (floating diffusion FD1) of the pixel circuit 111A. The voltage vx of the pixel signal of the pixel circuit 111A is a signal voltage V corresponding to the charge stored in the floating diffusion FD1. SIG The output signal vout of the read circuit 130 is CDS Since it is the difference between the potential difference between both ends of the pixel signal vx, the output signal vout = V SIG and V RST The difference signal between V and V is obtained. SIG and V RST The difference signal between V and V has a magnitude according to the amount of charge generated in the photoelectric conversion element PD. SIG and V RST The difference signal between these indicates the value of one pixel that constitutes the high-resolution image.

[0099] Now, Fig. 13 shows the configuration of the constant current source CS included in the source follower circuit 40 shown in Fig. 3. The constant current source CS shown in Fig. 13 includes a DC power supply 31 (constant voltage source) and a transistor 32. The transistor 32 is connected to the source of the transistor 20 via a selection gate SEL. When a pixel signal vx for a high-resolution image is read out by correlated double sampling using the source follower circuit 40 including the constant current source CS, vx has a waveform as shown in Fig. 14. As described above, V SIG and V RST The difference signal between these becomes the pixel value.

[0100] 13, a high-resolution output can be obtained by reading out the pixel signal vx, which is advantageous for acquiring a high-resolution image during the event operation 122 (second mode control). However, because the constant current source CS is used, a direct current flows through the source follower circuit 40, resulting in high power consumption.

[0101] Therefore, as shown in Fig. 15, a clock CLK (pulse signal) is applied to the gate of transistor 32 instead of a constant voltage. Therefore, transistor 32 constitutes a switching circuit 50 connected to the source of transistor 20. By applying the clock CLK to transistor 32, power consumption can be reduced compared to the case where transistor 32 functions as a constant current source CS. The source follower circuit 40 shown in Fig. 15 is called a pre-charge type.

[0102] 16 shows a timing chart for the operation of the precharge type source follower circuit 40. First, the controller 120 turns on the reset gate RST while turning off the selection gate SEL of the pixel circuit 111. This resets the floating diffusion FD.

[0103] Then, the controller 120 sets the clock CLK to High and turns on the transistor 32 (step S32), which causes the transistor 32 to become conductive and vx to become the ground potential.

[0104] Next, the controller 120 sets the clock CLK to low and turns on the selection gate SEL (step S32). As a result, vx is disconnected from the ground and connected to the transistor 20, so that the potential of vx gradually rises from the ground potential. vx is the voltage V of the floating diffusion FD. FD The voltage V1 is, for example, a reset voltage V RST or the aforementioned voltage {V DD -{Q(J-1)+Q(J+1)} / 3C FD -V GS} (see step S13).

[0105] Furthermore, the controller 120 turns off the selection gate SEL, turns on the transfer gate TG, and sets the clock to High (step S33). This causes the charge stored in the photoelectric conversion element PD to be transferred to the charge storage unit (floating diffusion FD). Furthermore, by setting the clock CLK to High, vx becomes the ground potential.

[0106] The controller 120 sets the clock CLK to low and turns on the selection gate SEL (step S34). This causes the potential of vx to gradually rise from the ground potential. vx is the voltage V of the floating diffusion FD. FD vx rises to a voltage V2 according to the charge stored in the floating diffusion FD. The voltage V2 is, for example, a signal voltage V SIG or the aforementioned voltage {V DD -Q(J) / 1.5C FD -V GS}.

[0107] Since the source follower circuit 40 is configured as shown in FIG. 15, reading can be performed by a switching operation, and therefore power consumption can be reduced compared to a source follower circuit 40 that uses a constant current source CS.

[0108] In the case of the circuit shown in Fig. 15, the constant current source CS is not used, so the settling error becomes large. In other words, the error of V1 and V2 becomes large. However, since the feature does not require high-resolution output, the occurrence of the error is not a big problem.

[0109] During normal operation 121 (first mode control 121), the controller 120 can read out features with low power consumption by applying a clock CLK to the gate of the transistor 32. During event operation 122 (second mode control 122), the controller 120 applies a constant voltage to the gate of the transistor 32 to operate the transistor 32 as a constant current source CS, thereby enabling highly accurate reading of pixel values ​​for a high-resolution image.

[0110] Figures 17 and 19 show an analog-digital-analog converter 60 (ADC60) included in the readout circuit 130. Note that vy shown in Figures 17 and 19 corresponds to vout in Figure 3. The ADC 60 converts the pixel signal vy (output signal vout in Figure 3) into a digital signal.

[0111] The ADC 60 includes an amplifier 61 (preamplifier 61) and a comparator 62. The amplifier 61 receives the pixel signal vy and the reference signal V DAC This is an operational amplifier that amplifies and outputs the difference between the reference signal V DAC is output from a digital-to-analog converter 63 (DAC 63). The pixel signal vy is, for example, provided to an inverting input (-) of an amplifier 61, and a reference signal V DAC is provided to a non-inverting input (+) of an amplifier 61. The non-inverting input of the amplifier 61 and the output of the amplifier 61 are connected via a switching element AZ (transistor). The output of the amplifier 61 is provided as one input of a two-input comparator 62. The other input of the comparator 62 is connected to a DAC 63 via a switching element CTR (transistor). The comparator 62 is a latch-type comparator and can hold the comparison result.

[0112] When outputting a high-resolution image, the ADC 60 uses an amplifier 61 and a latch-type comparator 62 to convert the pixel signal vy and the reference signal V DAC The pixel signal vy is converted into a digital signal by comparing it with the

[0113] That is, in the event operation 122 (second mode control 122) for outputting a high-resolution image, the controller 120 first turns off the clip gate CLP and turns on the switching element AZ (step S41) as shown in Fig. 17 and Fig. 18. As a result, vx is read out and vy changes to a potential corresponding to vx.

[0114] Next, the controller 120 turns off the switching element AZ (step S42). This causes the amplifier 61 to operate as a comparator. That is, the amplifier 61 outputs a pixel signal vy and a reference signal V DAC The difference between the reference signal V and the reference signal V is output. The difference is provided to one input of the latch-type comparator 62 and is held by the latch-type comparator 62. DAC The pixel signal vy is converted into a digital signal by performing a binary search to adjust the magnitude of vy. When a high-resolution image is output, the amplifier 61 operates, and therefore the power consumption by the amplifier 61 increases.

[0115] In the above case, the switching element CTR is switched to the ground side, so that the other input of the latch-type comparator 62 is supplied with the ground potential.

[0116] On the other hand, since the feature image does not require high-resolution output, in normal operation 121 (first mode control 121) for outputting the feature image, the amplifier 61 is turned off and the ADC 60 operates as shown in FIGS. 19 and 20.

[0117] That is, in the normal operation 121 (first mode control 121) for outputting a feature image, the controller 120 first turns on the clip gate CLP (step S51). At this time, the switching element AZ is turned on. As a result, vx is read out, and vy rises to a potential corresponding to vx.

[0118] Next, the controller 120 turns off the clip gate CLP (step S51), thereby holding the pixel signal vy.

[0119] At this time, since the switching element AZ is on, the pixel signal vy is provided to one input of the latch-type comparator 62 via the switching element AZ. At this time, since the switching element CTR is connected to the DAC 63, the reference signal V DAC is given.

[0120] Therefore, in the normal operation 121 (first mode control 121), the latch-type comparator 62 detects the pixel signal vy and the reference signal V DAC The difference between the reference signal V DAC A binary search is performed to adjust the magnitude of the pixel signal vy, thereby converting the pixel signal vy into a digital signal. In the normal operation 121 (first mode control 121), the amplifier 61 is not used, so that the power supply to the amplifier 61 can be stopped. Therefore, the power consumption is reduced.

[0121] <3. Notes>

[0122] The present invention is not limited to the above-described embodiment, and various modifications are possible. [Explanation of symbols]

[0123] 10: Image recognition system 20: Transistor 31:DC power supply 32: Transistor 40: Source follower circuit 50: Switching circuit 60: Analog to digital to analog converter 61: Amplifier 62: Latch type comparator 63: Digital to Analog Converter 100: Image sensor 110: Pixel section 111: Pixel circuit 111A: Pixel circuit 111B: Pixel circuit 111C: Pixel circuit 120: Controller 121: First mode control 122: Second mode control 125: Switch 130: Readout circuit 135: Signal line 140: Vertical scanning circuit 150: Horizontal scanning circuit 200: Image recognizer 210: Machine learning models 220: Event detector AZ: Switching element C FD :1st capacity CLK: Clock CLP: Clip Gate CS: constant current source CTR: Switching element FD: Floating diffusion FD1: Floating diffusion FD2: Floating diffusion FD3: Third charge storage section GND: Ground K: Horizontal stripe filter PD: Photoelectric conversion element Q(J) :first charge Q(J-1): 2nd charge Q(J+1): 3rd charge RST: Reset gate SEL: Select gate SG: First switching gate SG2: Second switching gate SG H : First horizontal switching gate SG V : Vertical first switching gate TG: Transfer gate V1: Voltage V2: Voltage V DD : Power supply voltage V FD :Voltage V RST :Reset voltage V SIG : Signal voltage vout: Output signal (pixel signal) vx: pixel signal vy : pixel signal

Claims

1. An image sensor that outputs a feature amount for image recognition, A plurality of pixel circuits; a controller configured to perform a first mode control for controlling the plurality of pixel circuits; Equipped with Each of the plurality of pixel circuits A photoelectric conversion element; a charge storage section that receives charges transferred from the photoelectric conversion element; and configured to output a pixel signal according to an amount of charge stored in the charge storage unit; the first mode control performs a division of the charges by controlling transfer of the charges so as to distribute the charges generated in n or more (n is a natural number equal to or greater than 1) photoelectric conversion elements to m or more (m is a natural number equal to or greater than 2) charge accumulation units for a calculation to obtain the feature amount; and reading, as a result of the division, a pixel signal corresponding to the charge accumulated in any one of the charge accumulation sections selected from the m or more charge accumulation sections to which the charge is distributed. Image sensor.

2. An image sensor that outputs a feature amount for image recognition, A plurality of pixel circuits; a controller configured to perform a first mode control to control the plurality of pixel circuits; Equipped with Each of the plurality of pixel circuits A photoelectric conversion element; a charge storage section that receives charges transferred from the photoelectric conversion element; and configured to output a pixel signal according to an amount of charge stored in the charge storage unit; the first mode control includes controlling a transfer of the charge between the photoelectric conversion element and the charge accumulation unit for a calculation to obtain the feature amount, the plurality of pixel circuits include a first pixel circuit and a second pixel circuit, the first pixel circuit includes a first photoelectric conversion element and a first charge accumulation unit that receives a first charge transferred from the first photoelectric conversion element; the second pixel circuit includes a second photoelectric conversion element and a second charge accumulation unit that receives a second charge transferred from the second photoelectric conversion element; controlling the transfer of the charges in the first mode control includes controlling the transfer of the first charges and the second charges between a plurality of photoelectric conversion elements including the first photoelectric conversion element and the second photoelectric conversion element and a plurality of charge accumulation units including the first charge accumulation unit and the second charge accumulation unit, for a calculation to obtain the feature amount; The first mode control includes obtaining an output signal corresponding to a difference between a first voltage corresponding to the first charge and a second voltage corresponding to a charge stored in any one of the plurality of charge storage units. Image sensor.

3. Controlling the transfer of the charges in the first mode control includes distributing the charges including the second charges to a plurality of charge storage units including the first charge storage unit and the second charge storage unit.

3. The image sensor according to claim 2.

4. Controlling the transfer of the charges in the first mode control includes distributing charges including the second charges to a plurality of charge accumulation units including the first charge accumulation unit and the second charge accumulation unit; Resetting the charge in any one of the plurality of charge storage units; transferring the first charge to the reset charge storage section; 3. The image sensor of claim 2 .

5. the controller is configured to further perform a second mode control different from the first mode control; The second mode control includes controlling a transfer of the charge between the photoelectric conversion element and the charge accumulation unit so that a pixel signal corresponding to an amount of charge generated in the photoelectric conversion element is output. The image sensor according to claim 1 .

6. The controller is configured to switch from the first mode control to the second mode control when a switching signal is given.

6. The image sensor according to claim 5.

7. a source follower circuit including a transistor that generates a voltage according to an amount of charge in the charge storage unit as a source voltage; The source follower circuit includes a switching circuit connected to the source of the transistor. The image sensor according to claim 1 .

8. An image sensor that outputs a feature amount for image recognition, A plurality of pixel circuits; a controller configured to perform a first mode control for controlling the plurality of pixel circuits; an analog-to-digital converter that converts pixel signals output from the plurality of pixel circuits into digital signals; Each of the plurality of pixel circuits A photoelectric conversion element; a charge storage section that receives charges transferred from the photoelectric conversion element; and configured to output a pixel signal according to an amount of charge stored in the charge storage unit; the first mode control includes controlling a transfer of the charges between the photoelectric conversion element and the charge accumulation unit for a calculation to obtain the feature amount, the controller is configured to further perform a second mode control different from the first mode control; the second mode control includes controlling a transfer of the charge between the photoelectric conversion element and the charge accumulation unit so that a pixel signal corresponding to an amount of charge generated in the photoelectric conversion element is output; the analog-to-digital converter includes an amplifier that outputs a difference between the pixel signal and a reference signal, and a latch-type comparator to which the difference is applied; The analog-to-digital converter includes: In the second mode control, the amplifier is used to output the difference, and the difference is provided to the latch-type comparator; In the first mode control, the pixel signal and the reference signal are applied to the latch-type comparator without using the amplifier. It is configured as Image sensor.

9. An image sensor according to any one of claims 1 to 8; and an image recognizer that performs image recognition based on the feature amount output from the image sensor.

10. An image sensor according to claim 6 or claim 8; an image recognizer that detects an event by performing image recognition based on the feature amount output from the image sensor; Equipped with The controller of the image sensor is configured to switch from the first mode control to the second mode control when the event is detected. Image recognition system.

11. A plurality of pixel circuits; a controller configured to perform a first mode control for controlling the plurality of pixel circuits; Equipped with Each of the plurality of pixel circuits A photoelectric conversion element; a charge storage section that receives charges transferred from the photoelectric conversion element; and outputting a pixel signal according to an amount of charge stored in the charge storage unit, The first mode control includes controlling transfer of charges generated by photoelectric conversion in n or more photoelectric conversion elements (n is a natural number equal to or greater than 1) to distribute the charges to m or more charge accumulation sections (m is a natural number equal to or greater than 2), thereby performing division of the charges, and reading out, as a result of the division, a pixel signal according to the charges accumulated in any one of the m or more charge accumulation sections to which the charges are distributed. Image sensor.

12. A plurality of pixel circuits; a controller configured to perform a first mode control for controlling the plurality of pixel circuits; Equipped with the plurality of pixel circuits include a first pixel circuit and a second pixel circuit, the first pixel circuit includes a first photoelectric conversion element and a first charge accumulation unit that receives a first charge transferred from the first photoelectric conversion element; the second pixel circuit includes a second photoelectric conversion element and a second charge accumulation unit that receives a second charge transferred from the second photoelectric conversion element; the first mode control includes controlling transfer of the first charge and the second charge between a plurality of photoelectric conversion elements including the first photoelectric conversion element and the second photoelectric conversion element and a plurality of charge accumulation units including the first charge accumulation unit and the second charge accumulation unit; The first mode control includes obtaining an output voltage corresponding to a difference between a first voltage corresponding to the first charge and a second voltage corresponding to a charge stored in any one of the plurality of charge storage units to which the charge including the second charge is distributed. Image sensor.

Images