Solid-state imaging device and image recognition system

Abstract

While extracting image features, it acquires additional information necessary for exposure control without significantly increasing the amount of output data. [Solution] The controller 12 controls a plurality of pixels 21 in units of a pixel group that includes a predetermined number of adjacent pixels 21 from a plurality of pixels 21 connected to one of a plurality of signal lines 22. The controller 12 controls the plurality of pixels 21 and the readout circuit 15 to read the pixel values ​​of at least one pair of adjacent pixels 21 included in the pixel group and output at least one difference signal indicating the difference between those pixel values. The controller 12 controls the plurality of pixels 21 and the readout circuit 15 to read the pixel value of one pixel 21 included in the pixel group and output a brightness signal indicating that pixel value.

Description

Technical Field

[0001] The present disclosure relates to a solid-state imaging device and an image recognition system.

Background Art

[0002] Image recognition generally includes extracting feature amounts of an image and classifying the image based on the feature amounts. Generally, an image to be recognized has a large data amount. Therefore, when an image recognizer extracts the feature amounts of the image, the image recognizer has a large arithmetic load. Thus, in order to reduce the arithmetic load of the image recognizer, it has been proposed to calculate the feature amounts by an image sensor.

[0003] For example, Patent Document 1 discloses an image sensor that outputs feature amounts for image recognition. The image sensor of Patent Document 1 executes first mode control for obtaining feature amounts and second mode control for outputting pixel signals corresponding to the amount of charge generated in a photoelectric conversion element in order to control a plurality of pixel circuits.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] According to the image sensor described in Patent Document 1, in the first mode of control, the amount of data output from the sensor is small, resulting in low power consumption. However, since it mainly outputs lightweight edge signals, it is not possible to automatically control exposure. In the second mode of control, camera controls such as autofocus, automatic exposure, and automatic white balance can be performed by performing image processing similar to that of a conventional camera. However, the amount of data output from the sensor is large, resulting in high power consumption. Therefore, when extracting image features using an image sensor, there is a need to acquire additional information necessary for exposure control without significantly increasing the amount of data output.

[0006] The object of this disclosure is to provide a solid-state imaging device for extracting image features that can further acquire information necessary for exposure control without significantly increasing the amount of output data while extracting image features. Another object of this disclosure is to provide an image recognition system including such a solid-state imaging device. [Means for solving the problem]

[0007] A solid-state imaging device according to one aspect of this disclosure is: Multiple pixels arranged along multiple rows and multiple columns, Multiple signal lines connected to each pixel in the aforementioned row, A readout circuit that reads the pixel values ​​of the plurality of pixels via the plurality of signal lines, The system comprises the plurality of pixels and a control circuit that controls the readout circuit, The aforementioned control circuit is The plurality of pixels are controlled using a unit of a pixel group that includes a predetermined number of adjacent pixels among the plurality of pixels connected to one of the plurality of signal lines. The plurality of pixels and the readout circuit are controlled to read out the pixel values ​​of at least one pair of adjacent pixels included in the pixel group and output at least one difference signal indicating the difference between those pixel values. The plurality of pixels and the reading circuit are controlled to read the pixel value of one pixel included in the pixel group and output a brightness signal indicating that pixel value. [Effects of the Invention]

[0008] According to one aspect of the present disclosure, a solid-state imaging device can acquire additional information necessary for exposure control while extracting image features, without significantly increasing the amount of output data. [Brief explanation of the drawing]

[0009] [Figure 1] This is a block diagram showing the configuration of the image recognition system according to the first embodiment. [Figure 2] Figure 1 is a block diagram showing the configuration of the pixel array 11. [Figure 3] Figure 1 is a block diagram showing the configuration of the readout circuit 15. [Figure 4] Figure 2 is a circuit diagram showing a portion of the pixel array 11 and Figure 3 is a circuit diagram showing the configuration of the sample-and-hold circuit 41. [Figure 5] Figure 4 is a timing chart illustrating the operation of the pixel group 23 and the sample-and-hold circuit 41 when they are operating in feature mode. [Figure 6] Figure 4 is a timing chart illustrating the operation of the pixel group 23 and the sample-and-hold circuit 41 when they are operating in normal mode. [Figure 7] This is a circuit diagram showing the configuration of pixel 21A(J) according to a modified example of the first embodiment. [Figure 8] Figure 7 is a plan view showing the arrangement of the photoelectric conversion elements PD(J,R), PD(J,Gr), PD(J,B), and PD(J,Gb). [Figure 9] This is a circuit diagram showing the configuration of the pixel group 23B and the sample-and-hold circuit 41 according to the second embodiment. [Figure 10] This is a block diagram showing the configuration of the column reading circuit 31B according to the second embodiment. [Figure 11]A timing chart for explaining the operation when the pixel group 23B and the sample-and-hold circuit 41 in FIG. 9 operate in the feature mode. [Figure 12] A timing chart for explaining the operation when the pixel group 23B and the sample-and-hold circuit 41 in FIG. 9 operate in the normal mode. [Figure 13] A circuit diagram showing the configuration of the pixel 21C(J) according to the first modification of the second embodiment. [Figure 14] A circuit diagram showing the configuration of the pixel group 23D and the sample-and-hold circuit 41 according to the second modification of the second embodiment. [Figure 15A] A timing chart for explaining the first part of the operation when the pixel group 23D and the sample-and-hold circuit 41 in FIG. 14 operate in the feature mode. [Figure 15B] A timing chart for explaining the second part of the operation when the pixel group 23D and the sample-and-hold circuit 41 in FIG. 14 operate in the feature mode. [Figure 16] A circuit diagram showing the configuration of the pixel group 23E and the sample-and-hold circuit 41 according to the third modification of the second embodiment. [Figure 17] A timing chart for explaining the operation when the pixel group 23E and the sample-and-hold circuit 41 in FIG. 16 operate in the feature mode. [Figure 18] A circuit diagram showing the configuration of the pixel 21F(J) according to the fourth modification of the second embodiment. [Figure 19] A block diagram showing the configuration of the image recognition system according to the third embodiment. [Figure 20] A block diagram showing the configuration of the image recognition system according to the fourth embodiment. [Figure 21] A block diagram showing the configuration of the image recognition system according to the fifth embodiment.

Embodiments for Carrying Out the Invention

[0010] <Embodiments and modified examples relating to this disclosure will be described below with reference to the drawings. The same or similar components are denoted by the same reference numerals.

[0011] [First Embodiment] [Configuration of the first embodiment] Figure 1 is a block diagram showing the configuration of an image recognition system according to the first embodiment. The image recognition system in Figure 1 comprises a solid-state imaging device 10 and a computing device 20.

[0012] The solid-state imaging device 10 comprises a pixel array 11, a controller 12, a vertical scanning circuit 13, a horizontal scanning circuit 14, a readout circuit 15, and an interface (I / F) 16. The pixel array 11 comprises multiple pixels arranged along multiple rows and multiple columns, as described later. The vertical scanning circuit 13 selects a row from among the multiple rows of pixels in the pixel array 11 that contains the target pixel whose pixel value is to be read. The horizontal scanning circuit 14 selects a column from among the multiple columns of pixels in the pixel array 11 that contains the target pixel whose pixel value is to be read. The readout circuit 15 reads the pixel value of each pixel in the pixel array 11 as a pixel signal. The pixel signal includes a difference signal indicating the difference in pixel values ​​between adjacent pixels and a brightness signal indicating the pixel value of a particular pixel. The pixel signal is sent to the computing device 20 via the interface 16. The controller 12 controls each component of the solid-state imaging device 10, and in particular controls the pixel array 11 and the readout circuit 15 via the vertical scanning circuit 13 and the horizontal scanning circuit 14. The controller 12, vertical scanning circuit 13, and horizontal scanning circuit 14 are examples of control circuits for the solid-state imaging device 10.

[0013] The solid-state imaging device 10 is, for example, a CMOS (Complementary Metal Oxide Semiconductor) image sensor.

[0014] The computing device 20 may be a general-purpose computer that executes an application program for performing image recognition on images captured by the solid-state imaging device 10, or it may be a dedicated device for image recognition. The computing device 20 may be configured to process images using machine learning models such as a convolutional neural network (CNN) and / or a recurrent neural network (RNN). The computing device 20 may be provided separately from the solid-state imaging device 10, or it may be integrated with the solid-state imaging device 10.

[0015] Figure 2 is a block diagram showing the configuration of the pixel array 11 in Figure 1. The pixel array 11 comprises multiple pixels 21(1,1) to 21(M,N) arranged along multiple rows and multiple columns, and multiple signal lines 22(1) to 22(N) connected to each pixel 21 in each column. In this specification, pixels 21(1,1) to 21(M,N) are collectively referred to as pixels 21, and signal lines 22(1) to 22(N) are collectively referred to as signal lines 22.

[0016] Figure 3 is a block diagram showing the configuration of the read circuit 15 in Figure 1. The read circuit 15 comprises a plurality of column read circuits 31-1 to 31-N and a serializer 32.

[0017] Column readout circuits 31-1 to 31-N are connected to signal lines 22(1) to 22(N), respectively, and read out the pixel value of each pixel 21 via each signal line 22. Column readout circuit 31-1 includes a sample-and-hold circuit 41, an analog-to-digital converter 42, and a buffer memory 43. The sample-and-hold circuit 41 temporarily holds the voltage of the pixel value read out from each pixel 21. The analog-to-digital converter 42 samples and quantizes the analog pixel signal read out from the sample-and-hold circuit 41 with a predetermined resolution to generate a digital pixel signal. The buffer memory 43 temporarily stores the digital pixel signal. Column readout circuit 31-1 may further include a preamplifier before the analog-to-digital converter 42. Column readout circuits 31-2 to 31-N are also configured similarly to column readout circuit 31-1. In this specification, column readout circuits 31-1 to 31-N are collectively referred to as "column readout circuit 31".

[0018] The serializer 32 comprises multiple column selection circuits 51-1 to 51-N and a sense amplifier 52. Each of the column selection circuits 51-1 to 51-N is connected to the buffer memory 43 of the column reading circuits 31-1 to 31-N. Under the control of the horizontal scanning circuit 14, the column selection circuits 51-1 to 51-N sequentially read the pixel signals stored in these buffer memories 43 and send them to the sense amplifier 52. As a result, multiple parallel pixel signals corresponding to multiple columns of the pixel array 11 are serialized. The output wiring corresponding to the number of columns is bundled together, which increases the load and may reduce the amplitude of the pixel signal. Therefore, the sense amplifier 52 amplifies the pixel signal until it reaches the signal level required by the subsequent circuit.

[0019] As a result, the readout circuit 15 reads out the pixel values ​​of multiple pixels 21 via multiple signal lines 22.

[0020] Figure 4 is a circuit diagram showing a portion of the pixel array 11 in Figure 2 and the configuration of the sample-and-hold circuit 41 in Figure 3.

[0021] The controller 12 controls multiple pixels 21(1,1) to 21(M,N) as a unit, using a pixel group 23 consisting of a predetermined number of adjacent pixels from multiple pixels 21(1,1) to 21(M,N) connected to one of the multiple signal lines 22(1) to 22(N). In the example in Figure 4, the pixel group 23 includes three pixels 21(J,n), 21(J+1,n), and 21(J+2,n) connected to the signal line 22(n). Here, 1≦n≦N and 1≦J≦M-2. For the sake of simplicity, "n" is omitted, and the signal line 22(n) and pixels 21(J,n), 21(J+1,n), and 21(J+2,n) are shown as signal line 22 and pixels 21(J), 21(J+1), and 21(J+2).

[0022] Pixel 21(J) comprises a photoelectric conversion element PD(J), a transfer gate TG(J), a floating diffusion layer FD(J), a reset gate RST(J), an amplification transistor SF(J), a selection gate SEL(J), and a switching gate SG(J).

[0023] The transfer gate TG(J), reset gate RST(J), amplification transistor SF(J), selection gate SEL(J), and switching gate SG(J) are, for example, MOS transistors. A MOS transistor is turned on when a high-level signal is applied between the gate and source, causing the drain and source to conduct, and turned off when a low-level signal is applied between the gate and source, causing the drain and source to become non-conductive. The transfer gate TG(J), reset gate RST(J), amplification transistor SF(J), selection gate SEL(J), and switching gate SG(J) are controlled by the controller 12 and the vertical scanning circuit 13.

[0024] The photoelectric conversion element PD(J) converts the input light to pixel 21(J) into an amount of electric charge corresponding to the amount of light. The photoelectric conversion element PD(J) is, for example, a photodiode.

[0025] The transfer gate TG(J) connects the photoelectric conversion element PD(J) to the floating diffusion layer FD(J). When the transfer gate TG(J) is turned on, it transfers the charge stored in the photoelectric conversion element PD(J) to the floating diffusion layer FD(J), and when it is turned off, it stops the transfer of charge.

[0026] The floating diffusion layer FD(J) accumulates the charge transferred from the photoelectric conversion element PD(J) via the transfer gate TG(J). The floating diffusion layer FD(J) is also called "floating diffusion".

[0027] The reset gate RST(J) resets the potential of the floating diffusion layer FD(J) to a predetermined potential, such as the power supply potential Vdd. The reset gate RST(J) is connected between the floating diffusion layer FD(J) and the node with the power supply voltage Vdd. When the reset gate RST(J) is turned on, it discharges the charge accumulated in the floating diffusion layer FD(J) (or the switching gate SG(J)) to the power supply. This discharge of charge is called a reset. As a result of the reset, the potential of the floating diffusion layer FD(J) becomes the power supply voltage Vdd. When the reset gate RST(J) is turned off, it stops discharging charge.

[0028] The amplifying transistor SF(J) amplifies the potential of the floating diffusion layer FD(J). The gate of the amplifying transistor SF(J) is connected to the floating diffusion layer FD(J), and the drain of the amplifying transistor SF(J) is connected to the node of the power supply voltage Vdd. The amplifying transistor SF(J) generates a voltage at its source corresponding to the amount of charge accumulated in the floating diffusion layer FD(J). The voltage generated at the source of the amplifying transistor SF(J) is the voltage Vfd in the floating diffusion layer FD, reduced by the amount of the drain-source voltage Vgs of the amplifying transistor SF(J) (Vfd-Vgs). The amplifying transistor SF(J), together with the constant current source 61 of the sample-and-hold circuit 41 described later, constitutes a source follower circuit.

[0029] The selection gate SEL(J) outputs the potential of the floating diffusion layer FD(J), amplified by the amplification transistor SF(J), to the signal line 22. When the selection gate SEL(J) is turned on, it connects the source of the amplification transistor SF(J) to the signal line 22. The selection gate SEL(J) selects a target pixel from among the multiple pixels 21 connected to the signal line 22 for which to read the potential of the floating diffusion layer FD(J). The potential of the floating diffusion layer FD(J) of the target pixel is read by the readout circuit 15. When the selection gate SEL(J) is turned off, it disconnects the amplification transistor SF(J) from the signal line 22.

[0030] The switching gate SG(J) connects the floating diffuse layer FD(J) of the pixel 21(J) to the floating diffuse layer FD(J+1) of the adjacent pixel 21(J+1). When the switching gate SG(J) is turned on, it connects the floating diffuse layer FD(J) of pixel 21(J) to the floating diffuse layer FD(J+1) of the adjacent pixel 21(J+1), and when it is turned off, it disconnects the floating diffuse layer FD(J) of pixel 21(J) from the floating diffuse layer FD(J+1) of the adjacent pixel 21(J+1).

[0031] Pixels 21(J+1) and 21(J+2) are also configured in the same way as pixel 21(J).

[0032] The sample-and-hold circuit 41 comprises capacitors Ccds, a constant current source 61, and a clip gate CLP. Capacitors Ccds are connected in series with the signal line 22. The voltage Vx output from each pixel 21 is applied to one end of capacitors Ccds (the side connected to each pixel 21) via the signal line 22. This causes capacitors Ccds to temporarily hold the voltage of the pixel value read from each pixel 21. The constant current source 61 is connected to one end of capacitors Ccds (the side connected to each pixel 21). As mentioned above, the constant current source 61, together with the amplification transistors of each pixel 21, constitutes a source follower circuit. The voltage at the other end of capacitors Ccds (the side opposite to the side connected to each pixel 21) becomes the output voltage Vout of the sample-and-hold circuit 41. The clip gate CLP is connected to the node of the output voltage Vout. When the clip gate CLP is turned on, it connects the node of the output voltage Vout to ground, and when it is turned off, it disconnects the node of the output voltage Vout from ground. The clip gate CLP is controlled by controller 12.

[0033] [Operation of the first embodiment] The solid-state imaging device 10 according to this embodiment operates in either a normal mode, which outputs the pixel value of each pixel in the entire image captured by the pixel array 11, or a feature mode, which detects and outputs feature quantities of the image captured by the pixel array 11.

[0034] Figure 5 is a timing chart illustrating the operation of the pixel group 23 and the sample-and-hold circuit 41 in Figure 4 when they are operating in feature mode.

[0035] During period T1, the controller 12 controls multiple pixels 21 in units of a pixel group 23 consisting of three adjacent pixels 21(J) to 21(J+2) from among the multiple pixels 21 connected to the signal line 22. Period T1 includes partial periods T1a(1) and T1a(2). During partial period T1a(1), the controller 12 controls pixels 21(J), 21(J+1) and the readout circuit 15 so as to read out the pixel values ​​of pixels 21(J) and 21(J+1) and output a difference signal indicating the difference between those pixel values. During partial period T1a(2), the controller 12 controls pixel 21(J+2) and the readout circuit 15 so as to read out the pixel value of pixel 21(J+2) and output a brightness signal indicating that pixel value.

[0036] During partial period T1a(1), the controller 12 enables the reading of the pixel value from pixel 21(J) by turning on the selection gate SEL(J) and switching gate SG(J) of pixel 21(J), and connects the stray diffusion layers FD(J) and FD(J+1) of pixels 21(J) and 21(J+1) to each other. Next, the controller 12 grounds the node of the output voltage Vout by turning on the clip gate CLP, and resets the potential of the stray diffusion layer FD(J) of pixel 21(J) by turning on the reset gate RST(J) of pixel 21(J). The voltage Vx when the potential of the stray diffusion layer FD is reset is expressed as Vx=Vrst. At this time, the output voltage Vout becomes Vout=0. Subsequently, the controller 12 reads the pixel value of pixel 21(J) by turning on the transfer gate TG(J) of pixel 21(J). At this time, the voltage Vx becomes Vx=Vrst-Vsig(J). Here, Vsig(J) represents the voltage corresponding to the pixel value of pixel 21(J). The output voltage Vout remains Vout=0. Subsequently, controller 12 holds the voltage Vc=Vx-Vout=Vrst-Vsig(J) in capacitor Ccds by turning off the clip gate CLP. Next, controller 12 resets the potential of the stray diffusion layer FD(J) of pixel 21(J) by turning on the reset gate RST(J) of pixel 21(J). At this time, the voltage Vx becomes Vx=Vrst, and the output voltage Vout becomes Vout=V1=Vx-Vc=Vsig(J). Subsequently, controller 12 reads out the pixel value of pixel 21(J+1) by turning on the transfer gate TG(J+1) of pixel 21(J+1). At this time, the voltage Vx becomes Vx=Vrst-Vsig(J+1). Here, Vsig(J+1) represents the voltage corresponding to the pixel value of pixel 21(J+1). Also, the output voltage Vout is Vout=V2=Vx-Vc=Vsig(J)-Vsig(J+1). This gives the difference signal between the pixel values ​​of pixels 21(J) and 21(J+1).

[0037] During partial period T1a(2), controller 12 turns off the selection gate SEL(J) and switching gate SG(J) of pixel 21(J), and turns on the selection gate SEL(J+2) of pixel 21(J+2). Next, controller 12 turns on the clip gate CLP to ground the node of the output voltage Vout, and turns on the reset gate RST(J+2) of pixel 21(J+2) to reset the potential of the stray diffusion layer FD(J+2) of pixel 21(J+2). At this time, the voltage Vx becomes Vx=Vrst, and the output voltage Vout becomes Vout=0. Subsequently, controller 12 turns off the clip gate CLP to hold the voltage Vc=Vx-Vout=Vrst in capacitor Ccds. Next, controller 12 turns on the transfer gate TG(J+2) of pixel 21(J+2) to read out the pixel value of pixel 21(J+2). At this point, the voltage Vx becomes Vx = Vrst - Vsig(J+2). Here, Vsig(J+2) represents the voltage corresponding to the pixel value of pixel 21(J+2). Also, the output voltage Vout becomes Vout = V3 = Vx - Vc = -Vsig(J+2). This gives us the brightness signal of pixel 21(J+2) with the reset noise removed.

[0038] During partial period T1a(1), when reading the pixel values ​​of pixels 21(J) and 21(J+1) and outputting a difference signal, the controller 12 may turn on both the selection gates SEL(J) and SEL(J+1) for pixels 21(J) and 21(J+1). As a result, the pixel values ​​are read out via two amplification transistors SF(J) and SF(J+1), which reduces noise in the source follower circuit and improves driving force compared to when only one selection gate SEL(J) is turned on.

[0039] Furthermore, when reading the pixel values ​​of pixels 21(J) and 21(J+1) and outputting a difference signal during partial period T1a(1), the controller 12 may simultaneously turn on the reset gates RST(J) and RST(J+1) of pixels 21(J) and 21(J+1). As a result, the floating diffusion layer FD(J) and FD(J+1) are reset via the two reset gates RST(J) and RST(J+1), which reduces the on-resistance of the reset gates compared to when only one reset gate RST(J) is turned on.

[0040] Alternatively, the controller 12 may control pixels 21(J) to 21(J+2) and the readout circuit 15 so as to read the pixel value of pixel 21(J) and output a brightness signal indicating that pixel value, and then read the pixel values ​​of pixels 21(J+1) and 21(J+2) and output a single difference signal indicating the difference between those pixel values.

[0041] After the period T1 has elapsed, the controller 12 controls the pixels 21(J+3) to 21(J+5) and the readout circuit 15 to output a difference signal and a luminance signal for the next group of three pixels 21(J+3) to 21(J+5) connected to the signal line 22, similar to the case in Figure 5. Thereafter, the controller 12 continues to control multiple pixels 21 and the readout circuit 15 in units of groups of three adjacent pixels from among the multiple pixels 21 connected to the signal line 22.

[0042] The computing device 20 can detect horizontal edges in the image based on the difference signal output from the solid-state imaging device 10.

[0043] Figure 6 is a timing chart illustrating the operation of the pixel group 23 and the sample-and-hold circuit 41 shown in Figure 4 when they are operating in normal mode.

[0044] During period T2, controller 12 turns on the selection gate SEL(J) of pixel 21(J). Next, controller 12 turns on the clip gate CLP to ground the node of the output voltage Vout, and turns on the reset gate RST(J) of pixel 21(J) to reset the potential of the stray diffusion layer FD(J) of pixel 21(J). At this time, the voltage Vx becomes Vx=Vrst, and the output voltage Vout becomes Vout=0. Subsequently, controller 12 turns off the clip gate CLP to hold the voltage Vc=Vx-Vout=Vrst in capacitor Ccds. Next, controller 12 turns on the transfer gate TG(J) of pixel 21(J) to read out the pixel value of pixel 21(J). At this time, the voltage Vx becomes Vx=Vrst-Vsig(J), and the output voltage Vout becomes Vout=V3=Vx-Vc=-Vsig(J). This allows us to obtain the brightness signal of pixel 21(J) with reset noise removed.

[0045] After the period T2 has elapsed, the controller 12 controls the pixel 21(J+1) and the readout circuit 15 to output a brightness signal for the next pixel 21(J+1) connected to the signal line 22, in the same manner as in Figure 6. Thereafter, the controller 12 continues to control multiple pixels 21 and readout circuits 15 to output brightness signals in the same manner as in Figure 6.

[0046] In the embodiment described, when the clip gate CLP is turned on, the output voltage Vout node is grounded in both feature mode and normal mode. Alternatively, the output voltage Vout node may be connected to a node with a potential different from ground potential, depending on the input range of the circuit connected after the sample-and-hold circuit 41. Furthermore, the output voltage Vout node may be connected to nodes with different potentials in both feature mode and normal mode.

[0047] [Effects of the first embodiment] According to the solid-state imaging device 10 of this embodiment, the controller 12 controls a plurality of pixels 21 and a readout circuit 15 to read out the pixel values ​​of a pair of adjacent pixels 21 included in the pixel group 23 and output a difference signal indicating the difference between those pixel values. Furthermore, the controller 12 controls a plurality of pixels 21 and a readout circuit 15 to read out the pixel value of a single pixel 21 included in the pixel group 23 and output a brightness signal indicating that pixel value.

[0048] According to the solid-state imaging device 10 of this embodiment, both difference signals and luminance signals can be output from each pixel group 23.

[0049] According to the solid-state imaging device 10 of this embodiment, exposure can be automatically controlled even when operating in feature mode by outputting both a difference signal and a brightness signal.

[0050] Furthermore, according to the solid-state imaging device 10 of this embodiment, since one luminance signal is output from each pixel group 23, the amount of data output can be reduced compared to the case where the pixel values ​​of all pixels 21 are read out and a luminance signal indicating the pixel value is output. By reducing the amount of data output, the power consumption of the solid-state imaging device 10 can be reduced.

[0051] According to the solid-state imaging device 10 of this embodiment, it is possible to acquire further information necessary for exposure control while extracting image features without significantly increasing the amount of output data.

[0052] The solid-state imaging device 10 according to this embodiment can be realized at low cost without requiring significant additions to conventional solid-state imaging devices.

[0053] Furthermore, the accuracy of image recognition can be improved by having the computing device 20 perform image recognition using both the luminance signal and the luminance signal.

[0054] [Modified version of the first embodiment] Figure 7 is a circuit diagram showing the configuration of a pixel 21A(J) according to a modified example of the first embodiment. Pixel 21A(J) comprises four photoelectric conversion elements PD(J,R), PD(J,Gr), PD(J,B), PD(J,Gb) and four transfer gates TG(J,R), TG(J,Gr), TG(J,B), TG(J,Gb) in place of the one photoelectric conversion element PD(J) and one transfer gate TG(J) of pixel 21(J) in Figure 4.

[0055] Figure 8 is a plan view showing the arrangement of the photoelectric conversion elements PD(J,R), PD(J,Gr), PD(J,B), and PD(J,Gb) in Figure 7. Red (R), green (Gr), blue (B), and green (Gb) color filters (not shown) are placed above the photoelectric conversion elements PD(J,R), PD(J,Gr), PD(J,B), and PD(J,Gb). Figure 8 shows an example of a Bayer array. The photoelectric conversion elements PD(J,R), PD(J,Gr), PD(J,B), and PD(J,Gb) each convert input light of a specific wavelength, incident via the color filters, into charge in amounts corresponding to the amount of light.

[0056] The transfer gates TG(J,R), TG(J,Gr), TG(J,B), and TG(J,Gb) connect the photoelectric conversion elements PD(J,R), PD(J,Gr), PD(J,B), and PD(J,Gb) to a common floating diffusion layer FD(J), respectively. The floating diffusion layer FD(J), reset gate RST(J), amplifying transistor SF(J), selection gate SEL(J), and switching gate SG(J) are shared by the photoelectric conversion elements PD(J,R), PD(J,Gr), PD(J,B), and PD(J,Gb).

[0057] In feature mode, the controller 12 simultaneously turns on the transfer gates TG(J,R), TG(J,Gr), TG(J,B), and TG(J,Gb). By simultaneously turning on the transfer gates TG(J,R), TG(J,Gr), TG(J,B), and TG(J,Gb), multiple color components R, Gr, B, and Gb of the color image are mixed with each other in the floating diffuse layer FD(J) and converted to grayscale. Based on the grayscaled pixel value Y=(R+Gr+B+Gb) / 4, the controller 12 generates a difference signal and a luminance signal, as explained with reference to Figure 5.

[0058] Other pixels 21(J+1) to 21(J+(K-1)) in Figure 4 may also be equipped with four photoelectric elements and four transfer gates instead of one photoelectric element and one transfer gate, similar to pixel 21A(J) in Figure 7.

[0059] According to this modified version, since the color image is converted to grayscale before output, the amount of data output from the pixel array and the solid-state imaging device can be reduced.

[0060] [Second Embodiment] [Configuration of the second embodiment] Figure 9 is a circuit diagram showing the configuration of the pixel group 23B and the sample-and-hold circuit 41 according to the second embodiment.

[0061] The pixel array of the solid-state imaging device according to the second embodiment includes a plurality of pixels 21B configured as shown in Figure 9, instead of the plurality of pixels 21 configured as shown in Figure 4. The controller 12 controls the plurality of pixels 21 in units of a pixel group 23B consisting of a predetermined number of adjacent pixels K from the plurality of pixels 21B connected to the signal line 22. Here, K ≥ 2. In the example of Figure 9, the pixel group 23B includes the first to K pixels 21B(J), 21B(J+1), ..., 21B(J+(K-1)) arranged in order.

[0062] Pixel 21B(J) has a configuration similar to pixel 21(J) in Figure 4, but with the switching gate SG(J) removed. The floating diffusion layer FD(J) of pixel 21B(J) is not connected to the floating diffusion layer FD(J+1) of the adjacent pixel 21B(J+1). In other respects, pixel 21B(J) is configured similarly to pixel 21(J) in Figure 4.

[0063] Pixels 21B(J+1) to 21B(J+(K-1)) are also constructed in the same way as pixel 21B(J).

[0064] Figure 10 is a block diagram showing the configuration of the column readout circuit 31B according to the second embodiment. The readout circuit of the solid-state imaging device according to the second embodiment may include the column readout circuit 31B of Figure 10 instead of the column readout circuit 31 of Figure 3.

[0065] In this embodiment, when the pixel group 23B and the sample-and-hold circuit 41 operate in feature mode, one luminance signal is generated for each pixel group 23B, i.e., one signal is generated for every K rows, as will be described later with reference to Figure 11. If the number of luminance signals is insufficient for the desired image processing (e.g., automatic exposure control), the column readout circuit 31B in Figure 10 can be used to generate luminance signals for all pixels 21B included in the pixel group 23B based on one luminance signal and one or more difference signals.

[0066] The column readout circuit 31B includes a signal processing circuit 44 in addition to the components of the column readout circuit 31 shown in Figure 3. The analog-to-digital converter 42 samples and quantizes the analog pixel signal Vout(J+k) read from the sample-and-hold circuit 41 at a predetermined resolution to generate a digital pixel signal Dout(J+k) and sends it to the signal processing circuit 44. The signal processing circuit 44 includes an adder 71 and a delay unit 72. The adder 71 adds the past output signal Douta(J+(k-1)) of the adder 71, i.e., the pixel signal delayed by the delay unit 72, to the current pixel signal Dout(J+k). As a result, the signal processing circuit 44 accumulates one or more difference signals to generate a luminance signal Douta(J+k) that indicates the pixel value of the second to Kth pixels 21B.

[0067] The column readout circuit may be configured to select whether to output the difference signal as is (see Figure 3) or to generate a luminance signal by accumulating one or more difference signals, as explained with reference to Figure 10. For this reason, the column readout circuit may include a selection circuit to select whether to send the pixel signal Dout(J+k) output from the analog / digital converter 42 directly to the buffer memory or to the signal processing circuit 44. The column readout circuit may also be configured to selectively set the output signal of the delay unit 72 to zero.

[0068] Furthermore, the column readout circuit may include an additional buffer memory on the output side of the analog-to-digital converter 42. The column readout circuit may simultaneously output the luminance signal and the difference signal by outputting the pixel signal Dout(J+k) to the serializer 32 via the additional buffer memory.

[0069] [Operation of the second embodiment] Figure 11 is a timing chart illustrating the operation of the pixel group 23B and the sample-and-hold circuit 41 in Figure 9 when they are operating in feature mode.

[0070] During period T11, the controller 12 controls multiple pixels 21B in units of a pixel group 23B consisting of K adjacent pixels 21B(J) to 21B(J+(K-1)) from among the multiple pixels 21B connected to the signal line 22. Period T11 includes sub-periods T11a(1), T11a(2), ..., T11a(K-1). During sub-period T11a(1), the controller 12 controls the pixel 21B(J) and the readout circuit 15 so as to read the pixel value of the pixel 21B(J) and output a luminance signal indicating that pixel value. During sub-period T11a(2), the controller 12 controls the pixel 21B(J+1) and the readout circuit 15 so as to read the pixel value of the pixel 21B(J+1) and output a difference signal indicating the difference between the pixel values ​​of pixels 21B(J) and 21B(J+1). The process continues in the same manner thereafter, and during the partial period T11a(K-1), the controller 12 controls the pixel 21B(J+(K-1)) and the readout circuit 15 so as to read out the pixel value of pixel 21B(J+(K-1)) and output a difference signal indicating the difference between the pixel values ​​of pixels 21B(J+(K-2)) and 21B(J+(K-1)).

[0071] During partial period T11a(1), controller 12 turns on the selection gate SEL(J) of pixel 21B(J). Next, controller 12 turns on the clip gate CLP to ground the node of the output voltage Vout, and turns on the reset gate RST(J) of pixel 21B(J) to reset the potential of the stray diffusion layer FD(J) of pixel 21B(J). At this time, the voltage Vx becomes Vx=Vrst, and the output voltage Vout becomes Vout=0. Subsequently, controller 12 turns off the clip gate CLP to hold the voltage Vc=Vx-Vout=Vrst in capacitor Ccds. Next, controller 12 turns on the transfer gate TG(J) of pixel 21B(J) to read out the pixel value of pixel 21B(J). At this time, the voltage Vx becomes Vx=Vrst-Vsig(J), and the output voltage Vout becomes Vout=V11=Vx-Vc=-Vsig(J). This allows us to obtain the brightness signal of pixel 21B(J) with reset noise removed.

[0072] After reading the luminance signal, the controller 12 turns on the clip gate CLP to ground the node of the output voltage Vout, and then turns off the clip gate CLP to hold the voltage Vc = Vx - Vout = Vrst - Vsig(J) in the capacitor Ccds. Here, Vsig(J) represents the voltage corresponding to the pixel value of pixel 21B(J).

[0073] During partial period T11a(2), the controller 12 turns off the selection gate SEL(J) of pixel 21B(J) and turns on the selection gate SEL(J+1) of pixel 21B(J+1). Next, the controller 12 resets the potential of the stray diffusion layer FD(J+1) of pixel 21B(J+1) by turning on the reset gate RST(J+1) of pixel 21B(J+1). At this time, the voltage Vx becomes Vx=Vrst, and the output voltage Vout becomes Vout=V12=Vx-Vc=Vsig(J). Here, V12=-V11 holds. After that, the controller 12 reads out the pixel value of pixel 21B(J+1) by turning on the transfer gate TG(J+1) of pixel 21B(J+1). At this time, the voltage Vx becomes Vx=Vrst-Vsig(J+1). Here, Vsig(J+1) represents the voltage corresponding to the pixel value of pixel 21B(J+1). The output voltage Vout is Vout = V13 = Vx - Vc = Vsig(J) - Vsig(J+1). This yields the difference signal between the pixel values ​​of pixels 21B(J) and 21B(J+1).

[0074] During a partial period T11a(k) (1≦k≦K-1), the controller 12 controls the pixel 21B(J+k) and the readout circuit 15 so that it reads out the pixel value of pixel 21B(J+(k-1)) and outputs a difference signal indicating the difference between the pixel values ​​of pixels 21B(J+(k-1)) and 21B(J+k). The controller 12 continues to control each pixel 21B and the readout circuit 15 in the same manner thereafter to output a difference signal.

[0075] Furthermore, if luminance signals for pixels 21B(J+1) to 21B(J+(K-1)) are required, as described above, the signal processing circuit 44 may be used to generate the luminance signal for the current row based on the difference signal between the luminance signal of the previous row's pixels and the current row's luminance signal.

[0076] Furthermore, the output is not limited to outputting the luminance signal of the first pixel 21B(J) included in the pixel group 23B, but may also output the luminance signals of other pixels 21B(J+k) included in the pixel group 23B, and output the difference signals of each other pair of pixels.

[0077] Alternatively, the polarity of the difference signal may be inverted by a subsequent circuit.

[0078] After the period T11 has elapsed, the controller 12 controls the pixels 21B(J+K) to 21B(J+(2K-1)) and the readout circuit 15 to output a difference signal and a luminance signal for the next K pixel group 21B(J+K) to 21B(J+(2K-1)) connected to the signal line 22, similar to the case in Figure 11. Thereafter, the controller 12 continues to control multiple pixels 21B and the readout circuit 15 in the same manner, using pixel groups of K adjacent pixels from the multiple pixels 21B connected to the signal line 22 as units.

[0079] Figure 12 is a timing chart illustrating the operation of the pixel group 23B and sample-and-hold circuit 41 in Figure 9 when they are operating in normal mode. The operation in Figure 12 is the same as the operation in Figure 6, except that the switching gate SG(J) is removed.

[0080] [Effects of the second embodiment] In the solid-state imaging device according to this embodiment, the controller 12 controls a plurality of pixels 21B and the readout circuit 15 so as to read out the pixel values ​​of a pair of adjacent pixels 21B included in the pixel group 23B and output a difference signal indicating the difference between those pixel values. Furthermore, the controller 12 controls a plurality of pixels 21B and the readout circuit 15 so as to read out the pixel value of a single pixel 21B included in the pixel group 23B and output a brightness signal indicating that pixel value. In the solid-state imaging device according to this embodiment, both a difference signal and a brightness signal are output from each pixel group 23B, so that information necessary for exposure control can be further acquired without significantly increasing the amount of output data while extracting image features.

[0081] According to the solid-state imaging device of this embodiment, in addition to the effects of the solid-state imaging device of the first embodiment, it is possible to output a difference signal and a brightness signal in units of a pixel group consisting of at least two pixels.

[0082] [First modified example of the second embodiment] Figure 13 is a circuit diagram showing the configuration of a pixel 21C(J) according to the first modified example of the second embodiment. Pixel 21C(J) comprises four photoelectric conversion elements PD(J,R), PD(J,Gr), PD(J,B), PD(J,Gb) and four transfer gates TG(J,R), TG(J,Gr), TG(J,B), TG(J,Gb) in place of the one photoelectric conversion element PD(J) and one transfer gate TG(J) of pixel 21B(J) in Figure 9.

[0083] Other pixels 21B(J+1) to 21B(J+(K-1)) in Figure 9 may also be equipped with four photoelectric elements and four transfer gates instead of one photoelectric element and one transfer gate, similar to pixel 21C(J) in Figure 13.

[0084] Pixel 21C(J) has the same technical effect as described with reference to pixel 21A(J) in Figure 7.

[0085] [Second variation of the second embodiment] Figure 14 is a circuit diagram showing the configuration of the pixel group 23D and the sample-and-hold circuit 41 according to a second modified example of the second embodiment.

[0086] The pixel array of the solid-state imaging device according to this modified example comprises a plurality of pixels 21 configured in the same manner as in Figure 4. The controller 12 controls the plurality of pixels 21 in units of a pixel group 23D consisting of a predetermined number of adjacent pixels K from the plurality of pixels 21 connected to the signal line 22. Here, K ≥ 2. In the example of Figure 14, the pixel group 23D includes the first to K pixels 21(J), 21(J+1), ..., 21(J+(K-1)) connected to the signal line 22 and arranged in order.

[0087] Figure 15A is a timing chart illustrating the first part of the operation when the pixel group 23D and sample-and-hold circuit 41 in Figure 14 are operating in feature mode. Figure 15B is a timing chart illustrating the second part of the operation when the pixel group 23D and sample-and-hold circuit 41 in Figure 14 are operating in feature mode.

[0088] During period T21, the controller 12 controls multiple pixels 21 in units of a pixel group 23B consisting of K adjacent pixels 21(J) to 21(J+(K-1)) from among the multiple pixels 21 connected to the signal line 22. Period T21 includes partial periods T21a(1), T21a(2), ..., T21a(K-1). During partial period T21a(1), the controller 12 controls the pixel 21(J) and the readout circuit 15 so as to read the pixel value of pixel 21(J) and output a luminance signal indicating that pixel value. During partial period T21a(2), the controller 12 controls the pixel 21(J+1) and the readout circuit 15 so as to read the pixel value of pixel 21(J+1) and output a difference signal indicating the difference between the pixel values ​​of pixels 21(J) and 21(J+1). The process continues in the same manner thereafter, and during the partial period T21a(K-1), the controller 12 controls pixel 21(J+(K-1)) and the readout circuit 15 so as to read out the pixel value of pixel 21(J+(K-1)) and output a difference signal indicating the difference between the pixel values ​​of pixels 21(J+(K-2)) and 21(J+(K-1)).

[0089] During period T21, the controller 12 enables the reading of pixel values ​​from pixels 21(J) to 21(J+(K-1)) by turning on the selection gates SEL(J) to SEL(J+(K-1)) for all pixels 21(J) to 21(J+(K-1)) included in the pixel group 23D. The controller 12 also connects the floating diffuse layers FD(J) to FD(J+(K-1)) of all pixels 21(J) to 21(J+(K-1)) included in the pixel group 23D by turning on the switching gates SG(J) to SG(J+(K-2)) for pixels 21(J) to 21(J+(K-1)). In other respects, the controller 12 controls the multiple pixels 21 and the readout circuit 15 to output luminance signals and difference signals, similar to the operation shown in Figure 11.

[0090] When operating in normal mode, the pixel group 23D and the sample-and-hold circuit 41 operate in the same manner as described with reference to Figure 6.

[0091] According to this modified version, by turning on all selection gates SEL(J)~SEL(J+(K-1)) and connecting all floating diffusion layers FD(J)~FD(J+(K-1)) to each other, the gain does not vary when reading out the pixel value of each pixel 21, and the accuracy of the output pixel signal is improved.

[0092] Pixels 21(J) to 21(J+(K-1)) in Figure 14 may also be equipped with four photoelectric elements and four transfer gates instead of one photoelectric element and one transfer gate, similar to pixel 21A(J) in Figure 7.

[0093] [Third modified example of the second embodiment] Figure 16 is a circuit diagram showing the configuration of the pixel group 23E and the sample-and-hold circuit 41 according to a third modified example of the second embodiment.

[0094] The pixel array of the solid-state imaging device according to this modified example includes, instead of the multiple pixels 21 configured as shown in Figure 14, multiple pixels 21E(J), 21E(J+1), ..., 21E(J+(K-1)) configured as shown in Figure 16. The controller 12 controls the multiple pixels 21E as a unit, using a pixel group 23E consisting of a predetermined number of adjacent pixels K from the multiple pixels 21E connected to the signal line 22. In the example of Figure 16, the pixel group 23E includes the first to K pixels 21E(J), 21E(J+1), ..., 21E(J+(K-1)) connected to the signal line 22 and arranged in order.

[0095] Pixel 21E(J) has a configuration obtained by removing the switching gate SG(J) from pixel 21(J) in Figure 14. The floating diffusion layer FD(J) of pixel 21E(J) is directly connected to the floating diffusion layer FD(J+1) of the adjacent pixel 23E(J+1) included in the same pixel group 23E, without going through a switching gate. In other respects, pixel 21E(J) is configured the same as pixel 21(J) in Figure 14.

[0096] Pixel 21E(J+1) comprises only a photoelectric conversion element PD(J+1), a transfer gate TG(J+1), and a floating diffuse layer FD(J+1). The floating diffuse layer FD(J+1) of pixel 21E(J+1) is directly connected to the floating diffuse layers FD(J,FD(J+2)) of adjacent pixels 23E(J) and 23E(J+2) included in the same pixel group 23E, without the need for a switching gate.

[0097] Pixels 21E(J+2) to 21E(J+(K-1)) are also constructed in the same way as pixel 21E(J+1).

[0098] By connecting the floating diffusion layers FD(J) to FD(J+(K-1)), the photoelectric conversion elements PD(J) to 21E(J+(K-1)) share the reset gate RST(J), the amplification transistor SF(J), the selection gate SEL(J), and the switching gate SG(J) of the pixel 21E(J).

[0099] Figure 17 is a timing chart illustrating the operation of the pixel group 23E and the sample-and-hold circuit 41 in Figure 16 when they are operating in feature mode. The operation in Figure 17 is the same as that in Figures 15A and 15B, except that the switching gates SG(J)~SG(J+(K-1)) and the selection gates SEL(J+1)~SEL(J+(K-1)) are removed.

[0100] When operating in normal mode, the pixel group 23E and the sample-and-hold circuit 41 operate in the same manner as described with reference to Figure 12.

[0101] In the operation shown in Figures 15A and 15B, the selection gates SEL(J)~SEL(J+(K-1)) and switching gates SG(J)~SG(J+(K-2)) are turned on for almost the entire duration of period T21, and the reset gates RST(J)~RST(J+(K-1)) are turned on simultaneously. Therefore, according to the configuration in Figure 16, by linking the floating diffusion layers FD(J)~FD(J+(K-1)) together, the number of transistors in each pixel 21E can be reduced compared to the case in Figure 14.

[0102] [Fourth variation of the second embodiment] Figure 18 is a circuit diagram showing the configuration of a pixel 21F(J) according to a fourth modification of the second embodiment. Pixel 21F(J) comprises four photoelectric conversion elements PD(J,R), PD(J,Gr), PD(J,E), PD(J,Gb) and four transfer gates TG(J,R), TG(J,Gr), TG(J,E), TG(J,Gb) in place of the one photoelectric conversion element PD(J) and one transfer gate TG(J) of pixel 21E(J) in Figure 16.

[0103] Pixels 21E(J) to 21E(J+(K-1)) in Figure 16 may also be equipped with four photoelectric elements and four transfer gates instead of one photoelectric element and one transfer gate, similar to pixel 21F(J) in Figure 18.

[0104] This modified example has the same technical effect as described with reference to pixel 21A(J) in Figure 7.

[0105] [Third Embodiment] Figure 19 is a block diagram showing the configuration of an image recognition system according to a third embodiment. The image recognition system in Figure 19 includes a solid-state imaging device 10G instead of the solid-state imaging device 10 in Figure 1.

[0106] The solid-state imaging device 10G includes a readout circuit 15G and an interface 16G instead of the readout circuit 15 and interface 16 shown in Figure 1. The readout circuit 15G separates the difference signal and the luminance signal from each other. The interface 16G sends the separated difference signal and luminance signal to the computing device 20 via separate channels.

[0107] The solid-state imaging devices according to the first and second embodiments output difference signals and luminance signals alternately for each row. When these mixed difference signals and luminance signals are input to a machine learning model as a single-channel signal, the accuracy may decrease when extracting features from both the difference signals and the luminance signals. The image recognition system according to this embodiment separates the difference signals and luminance signals and processes them separately, thereby enabling high-precision extraction of features from both the difference signals and the luminance signals. As the number of signal features input to the machine learning model increases, the accuracy of image recognition improves.

[0108] [Fourth Embodiment] Figure 20 is a block diagram showing the configuration of an image recognition system according to the fourth embodiment. The image recognition system in Figure 20 includes a solid-state imaging device 10H instead of the solid-state imaging device 10G in Figure 19.

[0109] In addition to the components of the solid-state imaging device 10G shown in Figure 19, the solid-state imaging device 10H includes an integration circuit 17 that spatially integrates the luminance signal to generate an integrated luminance signal. The solid-state imaging device 10H sends only the integrated luminance signal to the computing device 20 instead of the luminance signal.

[0110] The computing device 20 may perform automatic exposure control based on the integrated luminance signal. When performing automatic exposure control of a camera or the like, the luminance of individual pixels in the image is not required; it is sufficient to know the average luminance of the entire image. By outputting an integrated luminance signal of the entire image instead of the luminance signal of each pixel, the amount of data output from the solid-state imaging device 10H can be reduced, and the power consumption of the interface 16G and the computing device 20 can be reduced.

[0111] [Fifth Embodiment] Figure 21 is a block diagram showing the configuration of an image recognition system according to the fifth embodiment. The image recognition system in Figure 21 includes a solid-state imaging device 10I instead of the solid-state imaging device 10H in Figure 20.

[0112] The solid-state imaging device 10I includes a controller 12I, a readout circuit 15I, and an interface 16I, instead of the controller 12, readout circuit 15, and interface 16G of the solid-state imaging device 10I shown in Figure 20. Furthermore, the solid-state imaging device 10I includes a convolutional neural network (CNN) 18 and a recurrent neural network (RNN) 19.

[0113] The convolutional neural network 18 and the recurrent neural network 19 are image classifiers that classify images captured by the solid-state imaging device 10I based on at least one of the difference signal and the luminance signal. The difference signal and / or luminance signal are input to the convolutional neural network 18 and the recurrent neural network 19 every few rows. Raw data may also be input to the convolutional neural network 18. The convolutional neural network 18 extracts features, and the recurrent neural network 19 outputs the classification result. It is also possible to obtain the features of the entire pixel array by feeding back the extracted features to the features extracted from the next few rows. The solid-state imaging device 10I may send only the classification result and the integrated luminance signal to the computing device 20 instead of the difference signal and luminance signal. This reduces the amount of data output from the solid-state imaging device 10I and reduces the power consumption of the interface 16I and the computing device 20.

[0114] The readout circuit 15I may include an analog-to-digital converter with variable resolution. In this case, the controller 12I may control the resolution of the analog-to-digital converter based on the image classification result by the image classifier. By reducing the resolution of the analog-to-digital converter of the readout circuit 15I, the power consumption of the readout circuit 15I can be reduced.

[0115] [Other embodiments] The solid-state imaging device may be configured, for example, in its readout circuit to acquire the difference in pixel values ​​between adjacent columns. The computing device may detect vertical edges in the image based on the difference in pixel values ​​between adjacent columns output from the solid-state imaging device.

[0116] In relation to the second embodiment, as described with reference to Figure 10, instead of the column selection circuit 31B accumulating one or more difference signals to generate a luminance signal, the computing device 20 may generate one or more difference signals to generate a luminance signal.

[0117] [Summary of Embodiments] The solid-state imaging device according to the first embodiment is: Multiple pixels arranged along multiple rows and multiple columns, Multiple signal lines connected to each pixel in the aforementioned row, A readout circuit that reads the pixel values ​​of the plurality of pixels via the plurality of signal lines, The system comprises the plurality of pixels and a control circuit that controls the readout circuit, The aforementioned control circuit is The plurality of pixels are controlled using a unit of a pixel group that includes a predetermined number of adjacent pixels among the plurality of pixels connected to one of the plurality of signal lines. The plurality of pixels and the readout circuit are controlled to read out the pixel values ​​of at least one pair of adjacent pixels included in the pixel group and output at least one difference signal indicating the difference between those pixel values. The plurality of pixels and the reading circuit are controlled to read the pixel value of one pixel included in the pixel group and output a brightness signal indicating that pixel value.

[0118] According to the second embodiment of the solid-state imaging device, in the solid-state imaging device according to the first embodiment, The aforementioned pixel group includes first to third pixels, and the first and second pixels are adjacent to each other. The aforementioned control circuit is The plurality of pixels and the readout circuit are controlled to read out the pixel values ​​of the first and second pixels and output a difference signal indicating the difference between the pixel values. The plurality of pixels and the readout circuit are controlled to read the pixel value of the third pixel and output a brightness signal indicating the pixel value.

[0119] According to the third embodiment of the solid-state imaging device, in the second embodiment of the solid-state imaging device, Each of the aforementioned plurality of pixels is A photoelectric conversion element that converts input light into electric charge, The floating diffusion layer, A transfer gate connecting the photoelectric conversion element to the floating diffusion layer, A reset gate that resets the potential of the floating diffusion layer to a predetermined potential, An amplifying transistor for amplifying the potential of the floating diffusion layer, A selection gate that outputs the potential of the floating diffusion layer, amplified by the amplification transistor, to the signal line, The system includes a switching gate that connects the floating diffusion layer of the pixel to the floating diffusion layer of an adjacent pixel, When the control circuit reads the pixel values ​​of the first and second pixels and outputs the difference signal, it turns on the selection gates of the first and second pixels and turns on the switching gate of the first or second pixel to connect the floating diffusion layers of the first and second pixels to each other.

[0120] According to the fourth embodiment of the solid-state imaging device, in the third embodiment of the solid-state imaging device, When the control circuit reads the pixel values ​​of the first and second pixels and outputs the difference signal, it simultaneously turns on the reset gates of the first and second pixels.

[0121] According to the solid-state imaging device of the fifth embodiment, in the solid-state imaging device of the third or fourth embodiment, Each of the aforementioned plurality of pixels is Multiple photoelectric conversion elements that convert input light into electric charge, The system comprises a plurality of transfer gates, each connecting the plurality of photoelectric conversion elements to the floating diffusion layer, The control circuit simultaneously turns on the multiple transfer gates in each of the multiple pixels.

[0122] According to the solid-state imaging device of the sixth embodiment, in the solid-state imaging device of the first embodiment, The aforementioned group of pixels includes the first to K pixels arranged in order, The aforementioned control circuit is The plurality of pixels and the reading circuit are controlled to read the pixel value of the first pixel and output a brightness signal indicating the pixel value. The plurality of pixels and the readout circuit are controlled to read out the pixel values ​​of the k-th and k+1th pixels (1≦k≦K-1) and output a single difference signal indicating the difference between those pixel values.

[0123] According to the solid-state imaging device of the seventh embodiment, in the solid-state imaging device of the sixth embodiment, The system further includes a signal processing circuit that accumulates one or more of the difference signals to generate a luminance signal indicating the pixel values ​​of the second to K pixels.

[0124] According to the solid-state imaging device of the eighth embodiment, in the solid-state imaging device of the sixth or seventh embodiment, Each of the aforementioned plurality of pixels is A photoelectric conversion element that converts input light into electric charge, The floating diffusion layer, A transfer gate connecting the photoelectric conversion element to the floating diffusion layer, A reset gate that resets the potential of the floating diffusion layer to a predetermined potential, An amplifying transistor for amplifying the potential of the floating diffusion layer, The system includes a selection gate that outputs the potential of the floating diffusion layer, amplified by the amplification transistor, to the signal line.

[0125] According to the solid-state imaging device of the ninth embodiment, in the solid-state imaging device of the eighth embodiment, Each of the aforementioned plurality of pixels is Multiple photoelectric conversion elements that convert input light into electric charge, The system comprises a plurality of transfer gates, each connecting the plurality of photoelectric conversion elements to the floating diffusion layer, The control circuit simultaneously turns on the multiple transfer gates in each of the multiple pixels.

[0126] According to the solid-state imaging device of the 10th embodiment, in the solid-state imaging device of the 8th or 9th embodiment, Each of the plurality of pixels further comprises a switching gate that connects the floating diffusion layer of the pixel to the floating diffusion layer of an adjacent pixel. The control circuit turns on the selection gates of the first to K pixels and turns on the switching gates of the first to K-1 pixels so that the floating diffusion layers of the first to K pixels are connected to each other.

[0127] According to the solid-state imaging device of the 11th embodiment, in the solid-state imaging device of the 10th embodiment, Each of the aforementioned plurality of pixels is Multiple photoelectric conversion elements that convert input light into electric charge, The system comprises a plurality of transfer gates, each connecting the plurality of photoelectric conversion elements to the floating diffusion layer, The control circuit simultaneously turns on the multiple transfer gates in each of the multiple pixels.

[0128] According to the solid-state imaging device of the 12th embodiment, in the solid-state imaging device of the 8th embodiment, Each of the aforementioned plurality of pixels is A photoelectric conversion element that converts input light into electric charge, The floating diffusion layer, The photoelectric conversion element is connected to a transfer gate that connects to the floating diffusion layer, The first pixel described above is A reset gate that resets the potential of the floating diffusion layer to a predetermined potential, An amplifying transistor for amplifying the potential of the floating diffusion layer, The system further comprises a selection gate that outputs the potential of the floating diffusion layer, amplified by the amplification transistor, to the signal line, The floating diffuse layers of the first to K pixels are connected to each other.

[0129] According to the solid-state imaging device of the 13th embodiment, in the solid-state imaging device of the 12th embodiment, Each of the aforementioned plurality of pixels is Multiple photoelectric conversion elements that convert input light into electric charge, The system comprises a plurality of transfer gates, each connecting the plurality of photoelectric conversion elements to the floating diffusion layer, The control circuit simultaneously turns on the multiple transfer gates in each of the multiple pixels.

[0130] According to the 14th embodiment of the solid-state imaging device, the solid-state imaging device according to one of the 1st to 13th embodiments is: The system further includes an integrating circuit that spatially integrates the aforementioned luminance signal to generate an integrated luminance signal.

[0131] According to the solid-state imaging device of the 15th embodiment, the solid-state imaging device of one of the first to 14 embodiments is: The system further includes an image classifier that classifies images captured by the solid-state imaging device based on at least one of the difference signal and the brightness signal.

[0132] According to the solid-state imaging apparatus of the 16th embodiment, in the solid-state imaging apparatus of the 15th embodiment, the readout circuit includes an analog / digital converter having variable resolution, The control circuit controls the resolution of the analog-to-digital converter based on the image classification result obtained by the image classifier.

[0133] The image recognition system according to the 17th aspect is: A solid-state imaging device according to one of the 1st to 16th embodiments, The system includes a computing device that recognizes an image captured by the solid-state imaging device based on the output signal of the solid-state imaging device. [Explanation of Symbols]

[0134] 10,10G~10I Solid State Imaging Device 11-pixel array 12, 12I Controller 13 Vertical scanning circuit 14. Horizontal scanning circuit 15,15G readout circuit 16, 16G Interface (I / F) 17 Integrating circuit 18. Convolutional Neural Networks (CNNs) 19. Recurrent Neural Networks (RNNs) 20 Computing equipment 21, 21A, 21B, 21C, 21E, 21F, pixels 22 signal lines 23, 23B, 23D, 23E, Pixel group 31,31B Column Readout Circuit 32 Serializer 41 Sample-and-Hold Circuit 42 Analog-to-Digital Converters 43 Buffer memory 44 Signal Processing Circuits 51-column selection circuit 52 SenseAmp 61 Constant current source 71 Adder 72 Delay Element Ccds capacitor CLP Clipgate FD (Floating Diffusion Layer) PD (Photoelectric Detector) RST Reset Gate SF Amplifying Transistor SEL Selection Gate SG Switching Gate TG Transfer Gate

Claims

1. Multiple pixels arranged along multiple rows and multiple columns, Multiple signal lines connected to each pixel in the aforementioned row, A readout circuit that reads the pixel values ​​of the plurality of pixels via the plurality of signal lines, The system comprises the plurality of pixels and a control circuit that controls the readout circuit, The aforementioned control circuit is The plurality of pixels are controlled using a unit that includes a predetermined number of adjacent pixels among the plurality of pixels connected to one of the plurality of signal lines. The plurality of pixels and the readout circuit are controlled to read out the pixel values ​​of at least one pair of adjacent pixels included in the pixel group and output at least one difference signal indicating the difference between those pixel values. The plurality of pixels and the reading circuit are controlled to read the pixel value of one pixel included in the pixel group and output a brightness signal indicating that pixel value. Solid-state imaging device.

2. The pixel group includes first to third pixels, and the first and second pixels are adjacent to each other. The aforementioned control circuit is The plurality of pixels and the readout circuit are controlled to read out the pixel values ​​of the first and second pixels and output a difference signal indicating the difference between the pixel values. The plurality of pixels and the reading circuit are controlled to read the pixel value of the third pixel and output a brightness signal indicating the pixel value. The solid-state imaging apparatus according to claim 1.

3. Each of the aforementioned plurality of pixels is A photoelectric conversion element that converts input light into electric charge, The floating diffusion layer, A transfer gate connecting the photoelectric conversion element to the floating diffusion layer, A reset gate that resets the potential of the floating diffusion layer to a predetermined potential, An amplifying transistor for amplifying the potential of the floating diffusion layer, A selection gate that outputs the potential of the floating diffusion layer, amplified by the amplification transistor, to the signal line, The system includes a switching gate that connects the floating diffusion layer of the pixel to the floating diffusion layer of an adjacent pixel, When the control circuit reads the pixel values ​​of the first and second pixels and outputs the difference signal, it turns on the selection gates of the first and second pixels and turns on the switching gate of the first or second pixel so as to connect the floating diffusion layers of the first and second pixels to each other. The solid-state imaging apparatus according to claim 2.

4. When the control circuit reads the pixel values ​​of the first and second pixels and outputs the difference signal, it simultaneously turns on the reset gates of the first and second pixels. The solid-state imaging apparatus according to claim 3.

5. Each of the aforementioned plurality of pixels is Multiple photoelectric conversion elements that convert input light into electric charge, The system comprises a plurality of transfer gates, each connecting the plurality of photoelectric conversion elements to the floating diffusion layer, The control circuit simultaneously turns on the multiple transfer gates in each of the multiple pixels. The solid-state imaging apparatus according to claim 3.

6. The aforementioned group of pixels includes first to K pixels arranged in order, The aforementioned control circuit is The plurality of pixels and the reading circuit are controlled to read the pixel value of the first pixel and output a brightness signal indicating the pixel value. The plurality of pixels and the readout circuit are controlled to read out the pixel values ​​of the kth and k+1th pixels (1 ≤ k ≤ K-1) and output a single difference signal indicating the difference between those pixel values. The solid-state imaging apparatus according to claim 1.

7. The system further includes a signal processing circuit that accumulates one or more of the difference signals to generate a brightness signal indicating the pixel values ​​of the second to K pixels. The solid-state imaging apparatus according to claim 6.

8. Each of the aforementioned plurality of pixels is A photoelectric conversion element that converts input light into electric charge, The floating diffusion layer, A transfer gate connecting the photoelectric conversion element to the floating diffusion layer, A reset gate that resets the potential of the floating diffusion layer to a predetermined potential, An amplifying transistor for amplifying the potential of the floating diffusion layer, The system includes a selection gate that outputs the potential of the floating diffusion layer, amplified by the amplification transistor, to the signal line. The solid-state imaging apparatus according to claim 6.

9. Each of the aforementioned plurality of pixels is Multiple photoelectric conversion elements that convert input light into electric charge, The system comprises a plurality of transfer gates, each connecting the plurality of photoelectric conversion elements to the floating diffusion layer, The control circuit simultaneously turns on the multiple transfer gates in each of the multiple pixels. The solid-state imaging apparatus according to claim 8.

10. Each of the plurality of pixels further comprises a switching gate that connects the floating diffusion layer of the pixel to the floating diffusion layer of an adjacent pixel. The control circuit turns on the selection gates of the first to K pixels and turns on the switching gates of the first to K-1 pixels so that the floating diffusion layers of the first to K pixels are connected to each other. The solid-state imaging apparatus according to claim 8.

11. Each of the aforementioned plurality of pixels is Multiple photoelectric conversion elements that convert input light into electric charge, The system comprises a plurality of transfer gates, each connecting the plurality of photoelectric conversion elements to the floating diffusion layer, The control circuit simultaneously turns on the multiple transfer gates in each of the multiple pixels. The solid-state imaging apparatus according to claim 10.

12. Each of the aforementioned plurality of pixels is A photoelectric conversion element that converts input light into electric charge, The floating diffusion layer, The photoelectric conversion element is connected to a transfer gate that connects to the floating diffusion layer, The first pixel is, A reset gate that resets the potential of the floating diffusion layer to a predetermined potential, An amplifying transistor for amplifying the potential of the floating diffusion layer, The system further comprises a selection gate that outputs the potential of the floating diffusion layer, amplified by the amplification transistor, to the signal line, The floating diffuse layers of the first to K pixels are connected to each other. The solid-state imaging apparatus according to claim 6.

13. Each of the aforementioned plurality of pixels is Multiple photoelectric conversion elements that convert input light into electric charge, The system comprises a plurality of transfer gates, each connecting the plurality of photoelectric conversion elements to the floating diffusion layer, The control circuit simultaneously turns on the multiple transfer gates in each of the multiple pixels. The solid-state imaging apparatus according to claim 12.

14. The system further includes an integration circuit that spatially integrates the aforementioned luminance signal to generate an integrated luminance signal. The solid-state imaging apparatus according to claim 1.

15. The system further includes an image classifier that classifies images captured by the solid-state imaging device based on at least one of the difference signal and the brightness signal. The solid-state imaging apparatus according to claim 1.

16. The readout circuit includes an analog-to-digital converter with variable resolution, The control circuit controls the resolution of the analog / digital converter based on the image classification result of the image classifier. The solid-state imaging apparatus according to claim 15.

17. A solid-state imaging device according to one of claims 1 to 16, The system comprises a computing device that recognizes an image captured by the solid-state imaging device based on the output signal of the solid-state imaging device, Image recognition system.

Images