Solid-state imaging devices, image processing systems, and electronic equipment

Abstract

This technology reduces power consumption when filtering output data from an image sensor using an image filter, and also allows for setting predetermined filter coefficients with a smaller circuit size and simpler adjustment method compared to conventional technologies. [Solution] The solid-state imaging device comprises a plurality of pixel circuits arranged in the row and column directions, each of which includes a photoelectric conversion element that converts an incident optical signal into a pixel signal and outputs a pixel signal; and a control circuit that controls the reading of each pixel signal from the plurality of pixel circuits in the row and column directions using a predetermined reading method. The control circuit controls the execution of image filtering on at least one of the pixel signals from the plurality of pixel circuits and the pixel signals within the plurality of pixel circuits.

Description

Technical Field

[0001] The present invention relates to a solid-state imaging device such as an image sensor, an image processing system including the solid-state imaging device, and an electronic device including the image processing system.

Background Art

[0002] In an image recognition system, feature amount extraction of an image is performed. For image recognition, for example, there is a method using a machine learning model. In the case of image recognition using a machine learning model, as an example, feature amounts of a high-resolution image as an input image are extracted, and image classification (clustering) is performed based on the feature amounts.

[0003] For example, Non-Patent Document 1 discloses an image sensor that performs feature amount calculation (convolution calculation) inside a chip. The image sensor includes a plurality of pixels. Each pixel includes a photodiode, which is a photoelectric conversion element, and a readout circuit. The readout circuit of each pixel outputs a pixel signal, which is an electrical signal corresponding to the received light intensity. The image sensor disclosed in Non-Patent Document 1 requires a special manufacturing process (IGZO) for accumulation and multiplication of photoelectric conversion elements for performing convolution calculation.

[0004] An image sensor for realizing feature amount calculation inside a chip with a CMOS process suitable for large-scale integration circuits is disclosed in, for example, Patent Document 1. This image sensor is an image sensor that outputs feature amounts for image recognition, and includes a plurality of pixel circuits and a controller configured to execute first mode control for controlling the plurality of pixel circuits. Each of the plurality of pixel circuits includes a photoelectric conversion element and a charge accumulation unit that receives transfer of charges from the photoelectric conversion element, and is configured to output a pixel signal corresponding to the amount of charges accumulated in the charge accumulation unit. The first mode control includes controlling the transfer of charges between the photoelectric conversion element and the charge accumulation unit for the calculation of obtaining the feature amounts.

Prior Art Documents

Patent Documents

[0005] [Patent Document 1] Japanese Patent Publication No. 2022-102604 [Patent Document 2] U.S. Patent No. 5465064 [Non-patent literature]

[0006] [Non-Patent Document 1] Seiichi Yoneda et al., "Image Sensor Capable of Analog Convolution for Real-time Image Recognition System Using Crystalline Oxide Semiconductor FET", International Image Sensor Workshop(IISW), 2019, pp.322-325. [Non-Patent Document 2] Kohei Yamamoto et al., "Image Classification Using Neural Networks for Feature Extraction-Capable CMOS Image Sensors," Technical Report of the Institute of Image Information and Television Engineers, Vol. 46, No. 29, pp. 21-24, published September 15, 2022. [Non-Patent Document 3] Shuhei Okumura et al., "Study on a Low-Cost CMOS Color Image Sensor with Feature Extraction Capabilities," IEICE Integrated Circuits Research Committee (ICD), Student / Young Researchers' Meeting, presented March 22, 2022. [Non-Patent Document 4] Yu Osuga et al., "Analysis of the Influence of Extractable Features in CMOS Image Sensors on Image Recognition," IEICE Integrated Circuits Research Committee (ICD), Student / Young Researchers' Meeting, March 22, 2022. [Overview of the Initiative] [Problems that the invention aims to solve]

[0007] In the conventional image sensors described above, there was a problem in that the filter coefficients of the image filter were fixed and could not be changed during the in-pixel calculations for feature extraction.

[0008] The object of the present invention is to solve the above-mentioned problems and provide a solid-state imaging device that can reduce power consumption when filtering pixel signals from a solid-state imaging device such as an image sensor using an image filter, and that has a smaller circuit size and allows predetermined filter coefficients to be set using a simple adjustment method compared to the conventional technology, an image processing system equipped with the solid-state imaging device, and an electronic device equipped with the image processing system. [Means for solving the problem]

[0009] A solid-state imaging device according to one aspect of this disclosure is: A plurality of pixel circuits arranged in the row and column directions, each of which includes a photoelectric conversion element that converts an incident optical signal into a pixel signal, A solid-state imaging device comprising a control circuit that controls the reading of each pixel signal from the plurality of pixel circuits in the row direction and column direction using a predetermined reading method, The control circuit controls the application of image filtering to at least one of the pixel signals from the plurality of pixel circuits and the pixel signals within the plurality of pixel circuits. [Effects of the Invention]

[0010] Accordingly, according to one aspect of the present disclosure, power consumption when filtering pixel signals from the solid-state imaging device can be reduced, and the circuit size is smaller and the predetermined filter coefficients can be set using a simple adjustment method compared to the prior art. [Brief explanation of the drawing]

[0011] [Figure 1A] This is a block diagram showing an example configuration of an image sensor 400, which is a solid-state imaging device according to Embodiment 1. [Figure 1B]This is a block diagram showing an example configuration of an image sensor 400, which is a solid-state imaging device according to a modified example of Embodiment 1. [Figure 2] Figure 1A is a circuit diagram showing an example configuration of pixel 11. [Figure 3] Figure 2 is a graph showing the number of electrons in the pixel circuit 111 with respect to (light intensity × storage time). [Figure 4A] Figure 1A is a block diagram showing an example of the configuration of a column-parallel readout circuit. [Figure 4B] Figure 1A is a block diagram showing an example of the configuration of a series-type readout circuit. [Figure 5] Figure 4A shows an example of row-direction exposure time control, including a block diagram and timing chart. [Figure 6A] Figure 4A shows the case where the filter coefficient Wr is set every three rows in row-direction exposure time control, with the filter coefficient Ki = Wri. [Figure 6B] Figure 6A shows the filter coefficients when Wr0=1, Wr1=2, and Wr1=1. [Figure 7] This is a circuit diagram showing an example of the configuration of a pixel with an addition function with row-directly adjacent pixels according to Embodiment 2. [Figure 8] Figure 7 is a circuit diagram showing an example configuration of the pixel 11A and the column readout circuit input section 19. [Figure 9] Figure 7 is a circuit diagram showing an example configuration of pixel 11A. [Figure 10] This is a timing chart showing the operation when reading data from the pixel area including pixel 11A in Figure 8. [Figure 11] Figure 8 is a flowchart showing the readout control process with in-pixel operations that is performed in the pixel circuit. [Figure 12] Figure 8 shows a block diagram and timing chart illustrating row-direction exposure time control in the pixel circuit. [Figure 13] This figure shows the calculation performed by controlling the exposure time in the row direction in the pixel circuit shown in Figure 8. [Figure 14]This is a circuit diagram showing an example configuration of the pixel circuit and readout circuit according to Embodiment 3. [Figure 15] This is a timing chart showing the operation of the circuit in Figure 14. [Figure 16] This figure shows an example of 2x2 filter operation in the pixel circuit shown in Figure 14. [Figure 17A] This is a photograph of the calculation result image related to the comparative example. [Figure 17B] This is a photograph of the calculation result image according to Embodiment 3. [Figure 18] This figure shows an example of the coefficients for a 2x2 filter that can be implemented with the pixel circuit shown in Figure 14. [Figure 19] This figure shows an example of the coefficients for a 3x3 filter that can be implemented with the pixel circuit shown in Figure 14. [Figure 20] Figure 14 shows an example of the coefficients for a 2x2 filter that can be implemented with the pixel circuit shown, which is achieved by combining horizontal edges, diagonal edges, and vertical edges. [Figure 21] Figure 14 shows an example of the coefficients for a 3x3 filter that can be implemented with the pixel circuit shown, which is achieved by combining horizontal and vertical edges. [Figure 22] This is a block diagram showing an example configuration of an image sensor 400B (digital type), which is a solid-state imaging device according to Embodiment 4. [Figure 23] This block diagram shows a first example configuration of the filter operation circuit shown in Figure 22. [Figure 24] This figure shows the input / output data and calculation formula for the filter operation circuit in Figure 23. [Figure 25] This block diagram shows a second example configuration of the filter operation circuit shown in Figure 22. [Figure 26] Figure 24 is a timing chart showing the input / output data, gain, slide, and calculation formula of the filter calculation circuit. [Figure 27] This is a timing chart and block diagram showing the operation of an image sensor according to a modified example 1 of Embodiment 4. [Figure 28]Figure 27 is a timing chart showing the input / output data, memory output, and calculation formula of the filter calculation circuit. [Figure 29] Figure 27 shows the calculation formula for the in-pixel operation. [Figure 30] This figure shows the calculation formula for the output filter of the readout circuit in Figure 27. [Figure 31] This is a block diagram showing an example configuration of an image sensor 400C (analog type), which is a solid-state imaging device according to a modified example 2 of Embodiment 4. [Figure 32] Figure 31 is a circuit diagram showing an example configuration of the weighted adder circuit 20A. [Figure 33] This is a block diagram showing an example configuration of the image sensor 400D, which is a solid-state imaging device according to Embodiment 5. [Figure 34] Figure 33 is a block diagram showing an example of the configuration of the arithmetic circuit 15B. [Figure 35] Figure 34 is a timing chart showing the input / output data, weighting coefficients, stride, and calculation formula of the arithmetic circuit 15B. [Figure 36] This is a block diagram showing an example configuration of the arithmetic circuit 15BA according to a modified example of Embodiment 5. [Figure 37] This is a block diagram showing an example of the configuration of an arithmetic circuit using the arithmetic circuit 15B shown in Figure 34. [Figure 38] Figure 37 is a timing chart showing the input / output data and stride of the arithmetic circuit. [Figure 39] Figure 38 shows the filter coefficients of a 3x3 filter. [Figure 40] This is a block diagram showing an example configuration of an image sensor 400E, which is a solid-state imaging device according to Embodiment 9. [Figure 41] This is a block diagram showing an example configuration of an image sensor 400F, which is a solid-state imaging device according to Embodiment 10. [Figure 42] This is a block diagram showing an example configuration of an image recognition system according to Embodiment 11, using the image sensor 400 according to Embodiment 1. [Figure 43]This is a block diagram showing an example configuration of an image recognition system according to Embodiment 12, using the image sensor 400 according to Embodiment 1. [Figure 44] This is a block diagram showing an example configuration of an image recognition system related to a comparative example. [Figure 45] Figure 44 is a block diagram showing an example of the configuration of the pixel section 110. [Figure 46] Figure 44 is a circuit diagram showing an example configuration of the pixel section 110 and the readout circuit 130. [Figure 47] This is a timing chart of each signal showing the operation of the circuit in Figure 46. [Modes for carrying out the invention]

[0012] Embodiments and modified examples of the present invention will be described below with reference to the drawings. The same or similar components are denoted by the same reference numerals.

[0013] (Inventor's insights) First, the image sensor 100 related to the comparative example disclosed in Patent Document 1 will be described below.

[0014] (Comparative example) Figure 44 is a block diagram showing an example configuration of an image recognition system according to a comparative example. In Figure 44, the image recognition system comprises an image sensor 100 and an image recognition unit 200. In a Trillion Sensor society, low power consumption is required. In this comparative example, the image sensor 100 has low power consumption, and the power consumption of the image recognition unit 200 (artificial intelligence; AI) located downstream of the image sensor 100 can also be reduced. Therefore, it is possible to reduce the power consumption of the entire image recognition system, making it suitable for a Trillion Sensor society.

[0015] The image sensor 100 is used for event detection. The detected events are, for example, abnormalities or changes in the monitored object. The monitored object may be a structure such as a building or bridge, a living being such as a person or animal, or a vehicle such as a car, motorcycle, bicycle, or train. In addition, for event detection, the image sensor 100 outputs a feature image (first image) with a small amount of data at low power consumption during normal operation in first mode control 121. The feature image is provided to the image recognition unit 200.

[0016] The image recognition unit 200 performs image recognition using, for example, a machine learning model 210 for image recognition. The machine learning model 210 outputs image recognition results (image classification (clustering) results) from the input image. In the first mode control 121 of normal operation, the image recognition unit 200 performs image recognition using the feature image output from the image sensor 100 as the input image. The image recognition unit 200 is equipped with an event detector 220 that detects events based on the image recognition results. When the event detector 220 detects the occurrence of an event, it outputs a switching signal (first switching signal) from the first mode control 121 of normal operation to the second mode control 122 of event operation. The first switching signal is supplied to the image sensor 100.

[0017] When the first switching signal is applied, the image sensor 100 switches to the second mode control 122 of event operation. In the second mode control 122 of event operation, the image sensor 100 outputs a high-resolution image (second image) with a large amount of data. Here, the high-resolution image refers to a normal image, not a feature image. In a high-resolution image, each pixel has a pixel value corresponding to the amount of charge generated in the photoelectric conversion element PD of the image sensor 100. In contrast, in a feature image, each pixel has a pixel value as a result of feature calculations performed using the charge generated in the photoelectric conversion element PD. The calculations using charge will be described later.

[0018] As described above, during the second mode control 122 of event operation, the image sensor 100 outputs a high-resolution image. The high-resolution image is stored in a memory device, for example, to record the event that occurred (an anomaly or change in the monitored object). During the second mode control 122 of event operation, the image recognition unit 200 uses the high-resolution image output from the image sensor 100 as an input image for image recognition. Based on the image recognition result, the event detector 220 can detect whether the event is continuing or has ended. When the event detector 220 detects that no event has occurred, it outputs a switching signal (second switching signal) from the second mode control 122 of event operation to the first mode control 121 of normal operation. The second switching signal is supplied to the image sensor 100. When the image sensor 100 receives the second switching signal, it returns to the first mode control 121 of normal operation.

[0019] In typical image recognition using machine learning models, high-resolution images are provided as input images, their features are extracted, and classification (classification or clustering) is performed based on these features. In other words, a machine learning model comprises a feature extraction unit and a classification unit. The feature extraction unit extracts features from high-resolution images. The classification unit classifies images based on these features. Because high-resolution images provided as input images to machine learning models have a large amount of data, the computational load on the feature extraction unit of the machine learning model is large, and power consumption is also high.

[0020] However, when the image sensor 100 is in the first mode control 121 of normal operation, it outputs a feature image, so the machine learning model 210 is given this feature image as an input image. Therefore, the machine learning model 210 does not need to perform feature extraction calculations and only needs to perform classification calculations from the feature image given as an input image. Consequently, when the first mode control 121 of normal operation is performed, the feature extraction calculations in the machine learning model 210 can be omitted or reduced, and the power consumption of the image recognition unit 200 can be reduced.

[0021] The image sensor 100 is, for example, a CMOS (Complementary Metal Oxide Semiconductor) image sensor and comprises a pixel unit 110 as an imaging unit and a controller 120. The image sensor 100 further comprises a readout circuit 130, a vertical scanning circuit 140, and a horizontal scanning circuit 150. The readout circuit 130 reads out pixel signals from the pixel unit 110. The vertical scanning circuit 140 provides a signal to the pixel unit 110 to select the target pixel circuit (target pixel) from among the pixel circuits 111 (pixels) included in the pixel unit 110 from which the pixel signal is read. The horizontal scanning circuit 150 provides a column selection signal to the readout circuit 130 to select the target pixel row from which the pixel signal is output.

[0022] The controller 120 controls the operation of the pixel unit 110 and the readout circuit 130 via the vertical scanning circuit 140 and the horizontal scanning circuit 150. That is, the term "controller" may include not only the aforementioned controller 120, but also the vertical scanning circuit 140 and the horizontal scanning circuit 150. By controlling the pixel unit 110, the controller 120 causes the image sensor 100 to output a feature image in the first mode control 121. In the second mode control 122, the controller 120 causes the image sensor 100 to output a high-resolution image. The controller 120 can receive a switching signal from an external source and switch the control by the controller 120 from the first mode control 121 to the second mode control 122, or from the second mode control 122 to the first mode control 121. That is, the controller 120 is equipped with a switch 125 that switches the mode of the controller 120 between the first mode control 121 and the second mode control 122 based on the switching signal.

[0023] Figure 45 is a block diagram showing an example configuration of the pixel unit 110 in Figure 44. In Figure 45, the pixel unit 110 comprises a plurality of pixel circuits 111, 111A, 111B, and 111C arranged in a two-dimensional array. Note that a pixel circuit is also simply called a pixel.

[0024] Figure 46 is a circuit diagram showing an example configuration of the pixel unit 110 and readout circuit 130 of Figure 44. In Figure 46, one pixel circuit 111B in row J-1 (where J is a natural number greater than or equal to 2), one pixel circuit 111A in row J, and one pixel circuit 111C in row J+1 are shown. The multiple pixel circuits 111, 111A, 111B, and 111C included in the pixel unit 110 have a common structure. Note that in Figure 46, pixel circuits 111B and 111C are depicted in a simplified manner compared to pixel circuit 111A for convenience, but in reality, they have the same structure as pixel circuit 111A.

[0025] The following describes the structure of a pixel circuit, using pixel circuit 111A in row J as an example. Pixel circuit 111A is equipped with a photoelectric conversion element PD, which is, for example, a photodiode. The photoelectric conversion element PD accumulates an amount of charge corresponding to the amount of light. This charge is, for example, electrons. The charge generated by photoelectric conversion is called the photoelectric conversion charge. The electrons generated by photoelectric conversion are called photoelectric conversion electrons.

[0026] The pixel circuit 111A includes a transfer gate TG connected to a photoelectric conversion element PD. The transfer gate TG transfers the charge stored in the photoelectric conversion element PD. The transfer gate TG is composed of, for example, a transistor (transfer transistor). The transistor is, for example, a MOSFET (hereinafter referred to as a MOS transistor). Other transistors described below may also be MOS transistors. A MOS transistor is turned on when a high-level signal is applied between the gate and source, causing conduction between the drain and source, and turned off when a low-level signal is applied between the gate and source, causing non-conduction between the drain and source.

[0027] Furthermore, when the transfer gate TG is turned on, it becomes conductive and transfers the charge stored in the photoelectric conversion element PD to the charge storage unit described later. When the transfer gate TG is turned off, it becomes non-conductive and prevents the transfer of charge stored in the photoelectric conversion element PD. The pixel circuit 111A includes a charge storage unit that receives charge transfer from the photoelectric conversion element PD via the transfer gate TG. The charge storage unit has at least a floating diffusion layer FD1, which is a first capacitor connected to the transfer gate TG. The floating diffusion layer FD1 is a first capacitor C FD [F] is present. The floating diffusion FD1 is connected to the transfer gate TG so that it can receive charge transfer from the photoelectric conversion element PD.

[0028] The charge storage unit further includes a second switching gate SG2, which is a second capacitor connected to the floating diffusion FD1. The second switching gate SG2 is composed of, for example, a transistor (second switching transistor). When the second switching gate SG2 is turned on, it acts as a second capacitor and can store the transferred charge (electrons). The second switching gate SG2, together with the floating diffusion FD1, is connected to a transfer gate TG so that it can receive charge transfer from the photoelectric conversion element PD.

[0029] The second switching gate SG2, which acts as a second capacitor, has a second capacitance [F]. The second capacitance is, for example, 0.5C. FD Therefore, when the second switching gate SG2 is turned on, the capacitance of the charge storage unit is the first capacitance + the second capacitance, and as an example, C FD +0.5C FD = 1.5C FD[F]. When the second switching gate SG2 is turned on and charges are transferred, the charges are distributed and stored in the floating diffusion FD1 and the second switching gate SG2. When the second switching gate SG2 is turned off, it does not store charges (electrons). Therefore, the capacitance of the charge storage unit when the second switching gate SG2 is turned off is the same as the first capacitance, C FD [F].

[0030] Note that on the left side of FIG. 46, ×(-0.5), ×(+1.0), ×(-0.5) indicate examples of horizontal stride filter coefficients for each row.

[0031] As described above, the capacitance of the charge storage unit is variable and is a variable capacitor. By changing the capacitance of the charge storage unit, even if the amount of charge transferred from the photoelectric conversion element PD is the same, the voltage V in the floating diffusion FD1 FD can be changed. That is, when the capacitance of the charge storage unit where charges are stored is increased, the voltage V FD appearing in the floating diffusion FD1 can be lowered, and when the capacitance is decreased, the voltage V FD appearing in the floating diffusion FD1 can be increased. [[ID=十七]]

[0032] The pixel circuit 111A includes a reset gate RST connected to the charge storage unit. In the embodiment, the charge storage unit is connected to the floating diffusion FD1 and the second switching gate SG2. The reset gate RST discharges the charges stored in the charge storage unit. The reset gate RST is connected between the floating diffusion FD1 and the power supply voltage V DD . When the reset gate RST is turned on, it becomes conductive and discharges the charges stored in the floating diffusion FD1 or the second switching gate SG2, which is the charge storage unit, to the power supply. Such discharge of charges is called reset. By resetting, the potential of the charge storage unit becomes the power supply voltage V DDWhen the reset gate RST is turned off, it becomes non-conductive, preventing charge discharge.

[0033] The pixel circuit 111A includes a MOS transistor 320 that, together with a constant current source CS, constitutes a source follower circuit (SF circuit) 340. The MOS transistor 320 is also called a source follower transistor, and its gate terminal is connected to a floating diffusion FD1 (charge storage unit), and its drain terminal is connected to the power supply voltage V DD It is connected to the source terminal. The MOS transistor 320 generates a voltage vx (pixel signal) at the source terminal corresponding to the amount of charge (and capacitance) stored in the charge storage unit. The voltage vx (pixel signal) generated at the source terminal of the MOS transistor 320 is connected to the voltage VFD in the floating diffusion FD1 and the drain-source voltage V of the transistor. GS The voltage (V) decreased by minutes. FD -V GS )

[0034] The pixel circuit 111A is equipped with a selection gate SEL, which is composed of, for example, a MOS transistor (selection transistor). When the selection gate SEL is turned on, it becomes conductive and connects the source terminal of the MOS transistor 320 to the signal line 135 for reading the pixel signal. The selection gate SEL is an element for selecting the target pixel circuit (target pixel) from among the multiple pixel circuits 111A, 111B, 111C, and 111 for which the voltage in the charge storage section will be read out. The voltage in the charge storage section of the target pixel circuit is read out by the readout circuit 130. When the selection gate SEL is turned off, it becomes non-conductive and disconnects the MOS transistor 320 from the signal line 135.

[0035] The pixel circuit 111A includes a first switching gate SG. The first switching gate SG is composed of, for example, a transistor (first switching transistor). When the first switching gate SG is turned on, it becomes conductive and connects the charge storage unit of the pixel circuit 111A to the charge storage unit of the other pixel circuit 111C. When the first switching gate SG is turned off, it becomes non-conductive and disconnects the charge storage unit of the pixel circuit 111A from the charge storage unit of the other pixel circuit 111C. In other words, the first switching gate SG is an element for switching the connection / disconnection between the charge storage unit provided in the pixel circuit 111A on which the first switching gate SG is located and the charge storage unit provided in the other pixel circuit 111C adjacent to the pixel circuit 111A.

[0036] When the first switching gate SG of pixel circuit 111A becomes conductive, the charge storage unit (floating diffusion FD1) of pixel circuit 111A and the charge storage unit (floating diffusion FD3) of pixel circuit 111C are connected. Also, when the first switching gate SG of pixel circuit 111B becomes conductive, the charge storage unit (floating diffusion FD2) of pixel circuit 111B and the charge storage unit (floating diffusion FD1) of pixel circuit 111A are connected. When both the first switching gate SG of pixel circuit 111A and the first switching gate SG of pixel circuit 111B become conductive, the charge storage unit (floating diffusion FD2) of pixel circuit 111B, the charge storage unit (floating diffusion FD1) of pixel circuit 111A, and the charge storage unit (floating diffusion FD3) of pixel circuit 111C are connected.

[0037] The readout circuit 130 includes a constant current source CS that, together with the MOS transistor 320, constitutes a source follower circuit 40. The constant current source CS is connected to the signal line 135. The readout circuit 130 also includes a CDS capacitor C, which is provided in parallel with the constant current source CS and interposed in the middle of the signal line 135. CDS It is equipped with a CDS capacitor C CDSThis is used for correlated double sampling (CDS). The readout circuit 130 is equipped with a clip gate CLP, which is composed of, for example, a transistor (clip transistor). When the clip gate CLP is turned on, it becomes conductive and connects the signal line 135 to ground (GND). When the clip gate CLP is turned off, it becomes non-conductive and disconnects the signal line 135 from ground (GND). Each of the aforementioned gates is driven and controlled by the controller 120.

[0038] Figure 47 is a timing chart of each signal showing the operation of the circuit in Figure 46. Here, Figure 47 shows the procedure by which the controller 120 controls the pixel unit 110 for feature calculation during first mode control 121. Here, the driving method of a total of three pixel circuits, the J-1 row pixel circuit 111B, the J row pixel circuit 111A, and the J-1 row pixel circuit 111C, will be described in order to perform calculations for three pixels corresponding to a predetermined horizontal stripe filter K. The calculation equivalent to the convolution operation of the horizontal stripe filter K is performed within the image sensor 100.

[0039] First, the controller 120 turns on the selection gate SEL(J) and clip gate CLP of the J-th row pixel circuit 111A (step S11). Here, the J-th row pixel circuit 111A is the target pixel circuit.

[0040] Next, the controller 120 turns on the reset gate RST(J-1) of the pixel circuit 111B in row J-1, the reset gate RST(J) of the pixel circuit 111A in row J, and the reset gate RST(J+1) of the pixel circuit 111C in row J+1 (step S12). This resets the floating diffusions FD1, FD2, and FD3 of rows J-1, J, and J+1, respectively. The potentials of the floating diffusions FD1, FD2, and FD3 are equal to the power supply voltage V DD It will become.

[0041] Next, the controller 120 turns on the first switching gate SG(J) of the pixel circuit 111A in row J and the first switching gate SG(J-1) of the pixel circuit in row J-1, and also turns on the transfer gate TG(J-1) of the pixel circuit 111B in row J-1 and the transfer gate TG(J+1) of the pixel circuit 111C in row J+1 (step S13).

[0042] When the first switching gate SG(J) of the pixel circuit 111A in row J and the first switching gate SG(J-1) of the pixel circuit in row J-1 are turned on, the three floating diffusion elements FD1, FD2, and FD3 are connected. This allows the three floating diffusion elements FD to work together to accumulate charge. Additionally, when the transfer gate TG(J-1) of the pixel circuit 111B in row J-1 and the transfer gate TG(J+1) of the pixel circuit 111C in row J+1 are turned on, the second charge Q(J-1) of the photoelectric conversion element PD in the pixel circuit 111B in row J-1 and the third charge Q(J+1) of the photoelectric conversion element PD in row J+1 are transferred.

[0043] By the way, the readout circuit 130 has the following three methods. (1) A serial method that reads out one pixel at a time. (2) A column-parallel method in which reading circuits are arranged according to the number of columns, and one row is read at a time. (3) A pixel-parallel method in which the number of readout circuits is arranged according to the number of pixels. Here, since (1) requires high-speed operation because one pixel at a time is read out, and since (3) requires as many readout circuits 130 as there are pixels, the column-parallel method shown in Figure 44 of (2) is generally used.

[0044] As explained with reference to Figure 44, high-resolution image data is supplied to the image recognition unit 200, which processes the image signal, from the image sensor 100, which is a solid-state imaging device. In the case of high resolution, a large amount of power is consumed for transmitting and receiving image data and for image signal processing. For this reason, Patent Document 1 provides a feature image output that reduces the amount of data by incorporating the necessary filtering processing for the image recognition unit 200, in addition to the normal high-resolution output.

[0045] In the image sensor 100 shown in Figure 44, the feature image is obtained by a convolution operation that weights and adds adjacent pixel signals. The pixel size used in the operation becomes the filter (kernel) size, and the addition weights become the filter (kernel) coefficients. As shown in Figure 46 and its timing chart, Figure 47, the charge generated by photoelectric conversion in the photoelectric conversion element PD, which is an in-pixel photodiode, in adjacent pixels is merged with the charge of the floating diffusion FD of the adjacent pixel by turning on the switching gate SG between pixel cells, and conversion is performed in the subtraction phase (row J) and addition phase (rows J+1, J-1) of correlated double sampling, respectively, thereby realizing the addition and subtraction of adjacent pixel signals. The capacitance value of the floating diffusion FD is changed by turning on / off the switching gate SG2 connected to the floating diffusion FD, thereby realizing the desired filter coefficients. In Figure 46, the filter size is 1 × 3, and the filter coefficients are fixed at (-0.5, 1, -0.5).

[0046] In the comparative example image sensor 100 shown in Figure 44, there was a problem in that the filter coefficients were fixed and could not be changed in the in-pixel calculations for feature calculation. The objective of this embodiment is to solve the above problem and provide an image sensor, etc., which is a solid-state imaging device that can reduce power consumption when filtering output data from the image sensor and can set predetermined filter coefficients with a smaller circuit size and simpler adjustment method compared to the conventional technology. Embodiments and modified examples will be described below.

[0047] (Embodiment 1) Figure 1A is a block diagram showing an example configuration of an image sensor 400, which is a solid-state imaging device according to Embodiment 1. Figure 2 is a circuit diagram showing an example configuration of pixel 11 in Figure 1A. The differences from the comparative example will be explained below.

[0048] In Figure 1A, the image sensor 400 comprises a pixel array 10 having pixel circuits or pixels 11 arranged in a grid shape, a vertical scanning circuit 12, a horizontal scanning circuit 13, and a readout circuit 14. The vertical scanning circuit 12 selects the exposure time in the row direction of the pixel array 10 and the readout target. The pixels 11 are pixel sensor units and, as shown in Figure 2, for example, have a configuration of three MOS transistors, but any pixel or pixel circuit configuration is acceptable as long as the electrical exposure time can be adjusted.

[0049] Each pixel 11 comprises a photoelectric conversion element PD, a reset gate RST, a selection gate SEL, and an SF transistor SF. Here, the photoelectric conversion element PD converts the incident optical signal into electrons corresponding to time. The pixel array 10 constitutes a two-dimensional sensor unit in which multiple pixels 11 are arranged in an array. The horizontal scanning circuit 13 selects a column to be read in the column direction, and the readout circuit 14 reads out the pixel signals from the pixel array 10. In Embodiment 1 of Figure 1A, the image sensor is shown in a column-parallel configuration, but other configurations may be used as will be described later.

[0050] Figure 3 is a graph showing the number of electrons in the pixel circuit 111 of Figure 2 as a function of (light intensity × storage time). In other words, Figure 3 shows the relationship between light intensity × storage time and the number of electrons emitted in photoelectric conversion, and the dotted line indicates the maximum number of electrons that the pixel can hold. Up to this maximum number of electrons, the number of electrons generated is proportional to the received light intensity × storage time. The reset gate RST resets the photoelectric conversion element PD and the SF input capacitance, and the exposure time is from the reset by the reset gate RST to the readout. The source follower transistor SF is a component of the source follower amplifier, and the charge generated in the photoelectric conversion element PD is converted into a voltage by the input capacitance of the source follower circuit, forming a buffer amplifier (with high input / low output impedance characteristics) that outputs that signal voltage. SEL is a selection gate that selects the pixel to be read out.

[0051] Figure 4A is a block diagram showing an example configuration of the column-parallel readout circuit 14 shown in Figure 1A. The readout circuit 14 in Figure 4A comprises a plurality of column readout circuits 15 arranged in parallel with respect to the pixel array 10, and the horizontal scanning circuit 13 selects one of the column addresses of each of these column readout circuits 15.

[0052] Figure 4B is a block diagram showing an example configuration of the serial readout circuit 14 of Figure 1A. In the readout circuit 14 of Figure 4B, multiple CDS circuits 16 are arranged in parallel with respect to the pixel array 10, and the horizontal scanning circuit 13 selects one of the column addresses of these CDS circuits 16. That is, in the serial configuration of Figure 4B, correlated double sampling (CDS) is performed by multiple CDS circuits 16 arranged row by row, and the target column specified by the horizontal scanning circuit 13 is selected to read out the pixel signal for each pixel.

[0053] Incidentally, in image recognition systems, filtering is applied to images depending on the situation and subject in order to extract features from the image. Filtering is performed by convolution, and it is necessary to take the sum of products of the pixel signal with a coefficient corresponding to the filter size. In order to reduce power consumption, it is desirable to perform the sum of products for filtering within the solid-state imaging device, in order to reduce the amount of communication data and computational load from the solid-state imaging device, which is the image sensor 100, to the image processing unit such as the image recognition device. Below, an example with 9x9 pixels and a filter size of 3x1 is shown. In Figure 5, the exposure time T EXPO By adjusting each row, the exposure time T EXPO Since the pixel signal is proportional to the exposure time T EXPO The filter coefficients can be set by their ratio.

[0054] In other words, Figure 5 is a block diagram and timing chart showing an example of row-direction exposure time control in Figure 4A. In Figure 5, the row-direction exposure time ratio is equal to the ratio of the filter coefficients, T EXPO The reference exposure time is set to , and Tn is the exposure time for the nth row, expressed as Nr mod(3). Here, Nr is the row number. The filter coefficients (weighting coefficients) Wr0, Wr1, Wr2 are expressed by the following equations.

[0055] Wr0 = T0 / T EXPO Wr1 = T1 / T EXPO Wr² = T² / T EXPO

[0056] First, the vertical scanning circuit 12 sends an H-level RST signal for each row, resetting the pixels 11. Next, the photoelectric conversion element PD converts the incident light into electrons corresponding to the exposure time. Then, the vertical scanning circuit 12 sends an H-level SEL signal, and a signal voltage is output from the pixels 11 of the selected row via the source follower transistor SF. Subsequently, the signal voltages of the pixel signals sent from the pixels 11 for each row are output sequentially by the readout circuit 14 according to the control signal of the horizontal scanning circuit 13.

[0057] Figure 6A shows the filter coefficients Ki=Wri when the filter coefficients Wr are set every three rows in the row-direction exposure time control shown in Figure 4A, and Figure 6B shows the filter coefficients when the filter coefficients Wr0=1, Wr1=2, and Wr1=1 in Figure 6A.

[0058] As described above, according to this embodiment, the exposure time T of the pixel 11 by the horizontal scanning circuit 13 EXPO The filter coefficient Wri can be changed simply by adjusting the [specific component]. In contrast, when changing the filter coefficient Wri using the readout circuit 14, it is necessary to change the gain for each row of readout, but in this embodiment, there is no increase in circuit size, and no additional circuit area or current consumption is required.

[0059] (Modified version of Embodiment 1) Figure 1B is a block diagram showing an example configuration of an image sensor 400A, which is a solid-state imaging device according to a modified embodiment of Embodiment 1. In Figure 1B, an example configuration of the image sensor 400A using a pixel-parallel method is shown, in contrast to the column-parallel method in Figure 1A. In Figure 1B, the image sensor 400A is configured to include a pixel array 10 having a plurality of pixels 11, a vertical scanning circuit 12, a horizontal scanning circuit 13, a readout circuit 14P, and a readout control circuit 14PC.

[0060] In the modified embodiment 1 configured as described above, the control of the readout target and exposure time adjustment for the pixel array 10 may be performed by the vertical scanning circuit 12, the horizontal scanning circuit 13, or both. Here, the readout control for the readout circuit 14P is performed, for example, by the readout control circuit 14PC, which outputs a pixel signal corresponding to the control. Therefore, the modified embodiment 1 has the same effects as embodiment 1.

[0061] (Embodiment 2) Figure 7 is a circuit diagram showing an example configuration of a pixel with an addition function with row-direction adjacent pixels according to Embodiment 2, and Figure 8 is a circuit diagram showing an example configuration of a plurality of adjacent pixels 11A and a column readout circuit input section 19 in Figure 7. Compared to the image sensor 400 according to Embodiment 1 in Figure 1A, the image sensor 2 is characterized in that the pixels 11 of the image sensor 400 are configured by removing the switching gate SG2 that adjusts the filter coefficients from the pixel circuit 111B with a feature calculation function in Figure 46 according to the comparative example, and performing addition and subtraction of signal charges in the row direction.

[0062] In Figures 7 and 8, the transfer gate TG transfers the charge from the photoelectric element PD to the floating diffusion layer FD. The charge generated in the photoelectric element PD is transferred to the floating diffusion layer FD via the transfer gate TG, and the FD capacitance C FDIt is transferred to. Also, the switching gate SG connects the floating diffusion FDs of row-direction adjacent cells to each other. Furthermore, the FDA is an amplifier of the floating diffusion FD, also called an FD amplifier, and uses a source follower transistor SF as in Embodiment 1.

[0063] The column readout circuit input section 19 in Figure 8 is comprised of a constant current source CS, a CDS capacitor CCDS, and a clip gate CLP. Here, the constant current source CS is a constant current source that loads the source follower transistor SF, and the clip gate CLP controls the input first voltage V1 by switching the clip gate CLP from on to off in correlated double sampling (CDS), thereby controlling the CDS capacitor C CDS The signal is held at the output terminal OUTc via the signal, and then the difference voltage (V2-V1) between the next input voltage V2 and the second voltage V1 is output to the output terminal OUTc.

[0064] Figure 9 is a circuit diagram showing an example configuration of pixel 11A in Figure 7, and Figure 10 is a timing chart showing the operation when reading data from the pixel section including pixel 11A in Figure 8. Figure 11 is a flowchart showing the read control process with in-pixel arithmetic performed in pixel 11A in Figure 8. Here, an example where the filter size is 3 × 1 (where the target of the arithmetic is a group of pixels including the i-1 row, i row, and i+1 row (hereinafter referred to as a cell)) is described below. Note that i is a natural number greater than or equal to 2.

[0065] In Figures 10 and 11, first, in step S1, the selection gate SEL <i-1>,CELL SEL<i+1> By turning this on, the three pixels in rows i-1, i, and i+1 of the target cell are selected so that the three pixel outputs of the target cell are added together and output. Also, the switching gate SG SG<iー1> By turning this on, the FD of three adjacent pixels is connected, and the clip gate CLP (Figure 8) is turned on.

[0066] Next, in step S2, reset gate RST <i-1>,RST RST<i+1> By turning this on, the charge of the floating diffusion FD of the three pixels is reset. Furthermore, in step S3, by turning on the transfer gate TG connected to the pixel to be subtracted (intrapixel operation polarity selection parameter p=-1) (a predetermined pixel to be subtracted among the three pixels), the charge from the PD of that pixel is transferred to the floating diffusion FD of the three adjacent pixels.

[0067] Next, in step S4, the clip gate CLP is turned off, thereby controlling the CDS capacitor C CDS The voltage V1 of the pixel signal to be subtracted is held. Then, in step S5, the reset gate RST <i-1>,RST RST<i+1> Turning this on resets the charge of the floating diffusion FD of the three pixels.

[0068] Next, in step S6, by turning on the transfer gate TG connected to the pixel to be added (in-pixel calculation polarity selection parameter p=1), charge is transferred from the photoelectric conversion element PD of that pixel to the floating diffusion FD of the three pixels. As a result, the voltage V2 of the signal of the pixel to be added is output to the output terminal OUT of that pixel, and the voltage V2 is transferred to the CDS capacitor C CDS The voltage is applied, and the differential voltage (V2-V1) is output to the output terminal OUTc of the column reading circuit input section 19 (Figure 8).

[0069] As explained above, by performing readout control processing with in-pixel calculations, the signal charge from the photoelectric conversion element PD is added by the switching gate SG in the pixel 11A, and the polarity of the addition and subtraction is set by the CDS capacitor C CDS By using a transfer gate TG to select whether to read out before or after the readout timing of correlated double sampling (CDS), it is possible to add or subtract the voltages of adjacent pixel signals. Furthermore, because the calculation is performed within the pixel array 10, the pixel data in the row direction for the adjacent pixels used in the calculation is compressed, reducing the number of readouts. As a result, the number of readouts is reduced, which can lower or speed up current consumption.

[0070] Furthermore, any filter can be configured by combining the above-mentioned pixel 11A with the filter coefficient adjustment by exposure time adjustment according to Embodiment 1 (see Figures 12 and 13). The calculation of the filter in Figure 13 is expressed by the following equation.

[0071] I' i,j =I i-1,j ·p i-1 ·Wr i-1 +I i,j ·p i Wr i +I i+1,j ·p i+1 ·Wr i+1

[0072] Here, I' i,j This is the calculated pixel data (pixel signal voltage) obtained from pixel 11A in row i and column j. Also, I i-1,j This is the pixel data (voltage of the pixel signal) obtained from pixel 11A in row i-1 and column j, and I i,j This is the pixel data (voltage of the pixel signal) obtained from pixel 11A in row i and column j, and I i+1,j This is the pixel data (pixel signal voltage) obtained from pixel 11A in row i+1 and column j. Furthermore, p i-1 is the pixel-in-pixel polarity selection parameter for row i-1, and p i is the pixel-in-pixel operation polarity selection parameter for row i, and p i+1 is the pixel-internal operation polarity selection parameter for row i+1, where the pixel-internal operation polarity selection parameter p i When it is +1, it indicates addition, and the pixel-level operation polarity selection parameter p i When it is -1, it indicates subtraction. Note that Wr i is the filter coefficient for the i-th row.

[0073] In the embodiments described above, a filter size of 3×1 is shown as an example, but any filter can be configured similarly even in the case of N×1 (where N is a natural number). Figure 12 is a block diagram and timing chart showing row-direction exposure time control in the pixel circuit of Figure 8, and Figure 13 is a diagram showing the calculations performed by row-direction exposure time control in the pixel circuit of Figure 8.

[0074] As explained above, according to this embodiment, compared to when the filtering calculation is performed after the readout circuit or in the digital section of an image processing circuit such as an image recognition device, a portion of the filtering calculation is performed earlier, and the pixel data is compressed, reducing the number of readouts and thus reducing current consumption or increasing speed. Also, compared to the comparative example, since the switching gate SG2 is not required, the pixel 11A can be made smaller, resulting in lower costs, and the capacitance C FD Because this can be reduced, the conversion gain can be increased.

[0075] (Embodiment 3) Figure 14 is a circuit diagram showing an example configuration of a pixel circuit and readout circuit according to Embodiment 3. The circuit in Figure 14 is the same as the circuit according to Embodiment 2, but with four pixels 11A, and by separating the control lines 501, 503 (for transfer gates TG1, TG3) for odd-numbered rows and the control lines 502, 504 (for transfer gates TG2, TG4) for even-numbered rows, it is possible to achieve the filter coefficients necessary for realizing edge filtering in both the horizontal and vertical directions. Alternatively, instead of separating the control lines 501, 503 for odd-numbered rows and the control lines 502, 504 for even-numbered rows as shown in Figure 14, three rows of pixels may be treated as a single pixel group, and the control lines (transfer gate TG) may be separated. Furthermore, since the number of wires increases, a back-illuminated image sensor structure that can reduce the circuit area by irradiating the optical signal from the back surface may be used.

[0076] Figure 15 is a timing chart showing the operation of the circuit in Figure 14.

[0077] In Figure 15, by turning on the transfer gate TG3 for the 2x2 pixel 11A in Figure 14, the voltage of the pixel signal from pixel 11A at (I,J+1) is read out, and that voltage is transferred to the CDS capacitor C in column I. CDS The signal is held in (hereinafter, (I,J) represents row I and column J). Simultaneously, by turning on the transfer gate TG4, the voltage of the pixel signal from pixel 11A at (I+1,J+1) is read out, and that voltage is stored in the CDS capacitor C in column I+1. CDS The image is held in this state. Next, by turning on the transfer gate TG1, which doubles the exposure time, the difference voltage between the pixel signal voltage from pixel 11A at (I,J) and the pixel signal voltage from pixel 11A at (I,J+1) is output. At this time, the transfer gate TG2 remains off, and the pixel signal voltage at (I+1,J) is not read out. A voltage that does not include the reset pixel signal (hereinafter referred to as the voltage due to the reset signal (RST)) is output from the FD amplifier output of the pixel 11A in question, and the difference voltage between this voltage due to the reset signal (RST) and the pixel signal voltage from pixel 11A at (I+1,J+1) is output. By adding these difference voltages in the horizontal direction (row direction), a composite filter of a horizontal edge filter and a diagonal edge filter can be realized.

[0078] Figure 16 shows that the 2x2 filter in the pixel circuit of Figure 14 is realized as a composite filter of a horizontal edge filter and a diagonal edge filter. In Figure 15, the first term of edge 2 is the output of column I in Figure 14, and the second term of edge 2 is the output of column I+1 in Figure 14, and the sum of these is equivalent to the term of edge 1. In Figure 16, the filter obtained by adding the 2x2 horizontal edge filter coefficients, which include the coefficient for taking the horizontal difference of the first term of edge 3, and the 2x2 diagonal edge filter coefficients, which include the coefficient for taking the diagonal difference of the second term of edge 3, is logically equivalent to the term of edge 1, and the 2x2 filter in the pixel circuit of Figure 14 can extract horizontal edges and diagonal edges.

[0079] Figure 17A is a photograph of the result image of the horizontal edge filter calculation, and Figure 17B is a photograph of the result image according to Embodiment 3. In Figure 17A, the calculation of a 2x2 filter [[1,1],[-1,-1]] that extracts horizontal edges is performed, and in Embodiment 4, Figure 17B shows the result image of the calculation of a 2x2 filter [[2,0],[-1,-1]]. As is clear from comparing Figure 17A and Figure 17B, in Figure 17A only the horizontal edge component can be extracted, while in Figure 17B both the horizontal edge component and the vertical edge component can be extracted.

[0080] Other feasible filter configurations and the logical meanings of representative filters are shown below.

[0081] Figure 18 shows an example of the coefficients for a 2x2 filter that can be realized using the pixel circuit in Figure 14, and Figure 19 shows an example of the coefficients for a 3x3 filter that can be realized using the pixel circuit in Figure 14. Furthermore, Figure 20 shows that the 2x2 filter that can be realized using the pixel circuit in Figure 14 is realized by a filter that combines horizontal edges, diagonal edges, and vertical edges, and Figure 21 shows that the 3x3 filter that can be realized using the pixel circuit in Figure 14 is realized by a filter that combines horizontal edges and vertical edges.

[0082] As described above, according to this embodiment, a 2x2 filter can realize a filter that combines horizontal edges, diagonal edges, and vertical edges, and a 3x3 filter can realize a filter that combines horizontal edges and vertical edges.

[0083] According to this embodiment, in addition to the effects of Embodiment 2, by realizing a composite filter of a horizontal edge filter and a diagonal edge filter for 2x2 pixels, it is possible to extract vertical edge component signals that cannot be detected by the horizontal edge filter alone, thereby increasing the number of features and improving the accuracy of image recognition. Compared to a conventional image with only horizontal edges (Figure 17A), the image with the composite filter according to this embodiment (Figure 17B) shows that the vertical edge component signals can be confirmed.

[0084] (Embodiment 4) Figure 22 is a block diagram showing an example configuration of an image sensor 400B (digital type), which is a solid-state imaging device according to Embodiment 4. The image sensor 400B in Figure 22 differs from the image sensor 400 in Figure 1A in the following ways. (1) An AD converter (ADC) 14A is provided at the final stage of the readout circuit 14. (2) A filter calculation circuit 20 that performs digital calculations for the filter is provided after the AD converter 14A. The differences are explained below.

[0085] In Figure 22, the AD converter 14A converts the pixel analog signal (pixel signal voltage) from the pixel array 10 into a digital signal (digital data) within the readout circuit 14, and then outputs it to the filter calculation circuit 20. The filter calculation circuit 20 is configured to perform weighted summation of the digital signal from the readout circuit 14 as shown in Figure 23 or Figure 25. Note that other configurations are acceptable as long as the output calculation result is the same as shown in Figures 23 and 25.

[0086] Figure 23 is a block diagram showing a first configuration example of the filter operation circuit in Figure 22, and Figure 24 is a diagram showing the input / output data and calculation formula of the filter operation circuit in Figure 23. The filter operation circuit 20a in Figure 23 is an example of a typical FIR filter, (1) Two delay units 21 and 22, each with one cycle, (2) A coefficient amplifier 30 having filter coefficients (weighting coefficients) Wc0, (3) A coefficient amplifier 31 having filter coefficients (weighting coefficients) Wc1, (4) A coefficient amplifier 32 having filter coefficients (weighting coefficients) Wc2, (5) Two adders 41, 42, It is configured to include the following. Note that one cycle is the update period of the output pixel data of the readout circuit 14, and the same applies hereafter.

[0087] In Figure 23, input pixel data is input to adder 41 via coefficient amplifier 30, input to adder 41 via delay unit 21 and coefficient amplifier 31, and pixel data from delay unit 21 is input to adder 41 via delay unit 22 and coefficient amplifier 32. Adder 41 adds the two input pixel data and then outputs the added pixel data to adder 42. Adder 42 adds the two input pixel data and then outputs the added pixel data as the output pixel data of filter operation circuit 20a.

[0088] As shown in Figure 24, the filter operation circuit 20a delays the output data by only one clock cycle relative to the input data, and the output pixel data (pixel signal voltage) is expressed by the following equation.

[0089] I' j =I j-1 ·Wc0+I j Wc1+I j+1 Wc2

[0090] Figure 25 is a block diagram showing a second configuration example of the filter operation circuit in Figure 22. The filter operation circuit 20b in Figure 22 is (1) A coefficient amplifier 51 having filter coefficients (weighting coefficients) Wc, (2) Adder 52, (3) Selector 53 and, (4) A 1-cycle delay circuit 54, (5) A control circuit 55 for setting coefficients for each column, (6) Filter calculation column direction stride setting control circuit 56, It is configured to include the following. Note that stride refers to the window movement size in the filter operation.

[0091] In Figure 25, the column-by-coefficient setting control circuit 55 sets the gain corresponding to the filter coefficient Wc in the coefficient amplifier 51. The gain is adjusted, for example, by bit shift setting, and the polarity is determined by inverting the sign bit to + / -1. Here, the column-by-coefficient setting control circuit 55 switches the gain setting according to the row data update period according to the filter size / coefficient of the input row data. In addition, the filter calculation column direction stride setting control circuit 56 selects whether to pass the input data through or output the sum of the input signal and the integrated value according to the stride setting. Here, when the selection instruction signal Thru=1, the input signal is passed through and the output integrated value is reset, thereby setting the stride. Also, when the selection instruction signal Thru=2, the sum of the input signal and the integrated value is output.

[0092] The input pixel data is input to the adder 52 and the second input terminal of the selector 53 via the coefficient amplifier 51. The adder 52 adds the pixel data from the coefficient amplifier 51 and the output pixel data of the delay unit 54, and then outputs the resulting pixel data to the first input terminal of the selector 53. Based on the selection instruction signal Thru, the selector 53 selects the pixel data to be input to the first or second input terminal and outputs it via the delay unit 54 as the output pixel data of the filter calculation circuit 20b.

[0093] Figure 26 is a timing chart showing the input / output data, gain, slide, and calculation formula of the filter operation circuit in Figure 24. As shown in Figure 26, the operation when the stride is 3 is shown, and the output pixel data is generated from the input pixel data as follows.

[0094] I' j =I j-1 ·Wc0+I j Wc1+I i+1 Wc2

[0095] In Figure 25, the filter calculation circuit 20b shows an example configuration that includes decimation in an FIR filter. The decimation period is determined by the selector setting period within the configuration of the filter calculation circuit 20b, and corresponds to the stride setting.

[0096] As described above, according to Embodiment 4, the output stage of the readout circuit 14 of Embodiment 3 is equipped with an AD converter 14A, and the output stage of the AD converter is equipped with a filter operation circuit 20, where the filter size and coefficients in the column direction are changed. As an example, the filter size is shown as 3 × 3, but any filter can be constructed in the same way even when it is N × M (where N and M are natural numbers).

[0097] Therefore, according to Embodiment 4, the pixel filter can be configured entirely within the image sensor 400B, which is a solid-state imaging device, and the filtering processing required by image processing systems such as image recognition devices can be reduced. By compressing the amount of data simultaneously with the filtering function, the interface speed between the image sensor 400B and the image recognition device (details described later) can be reduced, thereby reducing current consumption.

[0098] (Modification 1 of Embodiment 4) Figure 27 shows a timing chart and block diagram illustrating the operation of an image sensor according to a modified example 1 of Embodiment 4. The image sensor 400Ba in Figure 27 differs from the image sensor 400B in Figure 22 in the following respects. (1) Instead of the read circuit 14, a read circuit 14B is provided, and the read circuit 14B has a buffer memory 14m inserted between the AD converter 14A and the filter operation circuit 20. The differences are explained below.

[0099] In Figure 27, the buffer memory 14m is a buffer memory for pipeline processing of digital data from the AD converter 14A. Since the AD conversion of the next read row is performed during the filtering operation, the output data is transferred to this buffer memory 14m after the AD conversion. Although not shown in Figure 27 because the description includes the serial configuration, in the column-parallel configuration, a serializer (serialization circuit) is placed in parallel with the output of the buffer memory 14m to serialize the AD conversion output data.

[0100] Figure 28 is a timing chart showing the input / output data, memory output, and calculation formula of the filter calculation circuit in Figure 27. Figure 28 shows the operation of the AD converter 14A, the output data of the buffer memory 14m, and the output data of the filter calculation circuit 20. As shown in the timing chart of Figure 28, in the AD conversion of the pixel group of multiple rows divided into three sections, pipeline processing is performed as shown to obtain the output pixel data. Here, the output pixel data of the buffer memory 14m and the output pixel data of the filter calculation circuit 20 are shown below along with the calculation formulas.

[0101] I' j =I j-1 ·p i-1 ·Wr i-1+ I i,j ·p i Wr i +I i+1,j ·p i+1 ,Wr i+1 I" j =I' j-1 Wc j-1 +I' j Wc j +I' i+1 ·Wcj

[0102] Figure 29 shows the calculation formula for the in-pixel operation in Figure 27, and Figure 30 shows the calculation formula for the output filter of the readout circuit in Figure 27.

[0103] In Modification 1 of Embodiment 4, as an example, in a 9x9 pixel filter with a filter size of 3x3 and a stride of 3, as shown in Figures 27 to 28, the 3x3 pixel signals are multiplied and summed by row-by-row exposure time adjustment and in-pixel calculations in the row direction, and by a digital filter circuit in the column direction, and output as 1x1 data. Modification 1 of Embodiment 4 also has the same effects as Embodiment 4.

[0104] (Modification 2 of Embodiment 4) Figure 31 is a block diagram showing an example configuration of an image sensor 400C (analog type), which is a solid-state imaging device according to modification 2 of embodiment 4, and Figure 32 is a circuit diagram showing an example configuration of the weighted summing circuit 20A in Figure 31. The image sensor 400C in Figure 31 and the image sensor 400C in Figure 32 differ from the image sensor 400B in Figure 22 in the following respects.

[0105] (1) It does not have an AD converter 14A and processes with analog pixel signals. (2) Instead of the filter operation circuit 20, a weighted summing circuit 20A is provided. The differences are explained below.

[0106] The weighted summing circuit 20A performs a sample-and-hold operation on the output data of the readout circuit 14, for example, and then performs weighted summation using an add / subtract circuit that uses a switched capacitor circuit or an operational amplifier.

[0107] Referring to Figure 32, an image sensor according to a modified example 2 of Embodiment 4 will be described below.

[0108] In Figure 32, the multiplication circuit has switches SW0 to SW7 that selectively input analog data Vin, which is the voltage of the pixel signal. These switches SW0 to SW7 are controlled by the respective bits of digital data b0 to b7 from a control circuit (not shown). Switches SW0 to SW3 are connected to a first group of capacitors C0 to C3, and switches SW4 to SW7 are connected to a second group of capacitors C4 to C7. The former are coupled via a capacitive coupling unit CP1, and the latter are coupled via a capacitive coupling unit CP2. Capacitive coupling unit CP1 consists of capacitors C0 to C3, and capacitive coupling unit CP2 consists of capacitors C4 to C7. The capacitance of capacitors C0 to C3 changes in proportion to the weight of bits b0 to b3, and capacitors C4 to C7 have capacitances proportional to the weight of bits b4 to b7. Furthermore, capacitive coupling units CP1 and CP2 are grounded via capacitors C11 and C13. The output data from capacitive coupling units CP1 and CP2 are input to inverters INV1 and INV2, and the outputs of each inverter INV1 and INV2 are coupled by capacitive coupling unit CP3. The output data from capacitive coupling unit CP3 is output as analog data Vout, which is the pixel signal voltage, via inverter INV3. Capacitive coupling unit CP3 is grounded via capacitor C32. Inverters INV1 to INV3 are connected in series with each other, ensuring the output accuracy of each inverter INV1 to INV3. The output data from each inverter INV1 to INV3 is fed back to the input circuit side via capacitors C10, C12, and C31, respectively. The polarity selection and weighting coefficient can be determined by the ratio of capacitors Ci.

[0109] The weighted summing circuit 20A, configured as described above, inputs the output data of the capacitive coupling section to the series-connected inverters INV1 and INV2, and connects the grounded weighted capacitance to a capacitance that makes the closed-loop gains of inverters INV1 and INV2 substantially equal, as well as to the capacitive coupling connecting inverters INV1 and INV2. In this configuration, inverters INV1 and INV2 input the output of the capacitive coupling to the closed-loop gains of the series-connected inverters INV1 and INV2, and the grounded weighted capacitance is connected to the capacitive coupling. The closed-loop gains of inverters INV1 and INV2 are then balanced so as to minimize the effect of the bias voltage.

[0110] According to the modified example 2 of Embodiment 4 configured as described above, it has the same effects and advantages as Embodiment 4.

[0111] (Embodiment 5) Figure 33 is a block diagram showing an example configuration of an image sensor 400D, which is a solid-state imaging device according to Embodiment 5. The image sensor 400D in Figure 33 differs from the image sensor 400 in Figure 1 in the following ways. (1) In the reading circuit 14, a column reading circuit 15 is provided for each column, similar to Figure 4A. (2) Each column readout circuit 15 is configured with an AD converter (ADC) 15A, an arithmetic circuit 15B, and a buffer memory 15m. Here, the filter size is set for the arithmetic circuit 15B of each column from the filter size and coefficient control circuit 50, and the filter coefficients are controlled. (3) In the readout circuit 14, a column selector 61 is provided for each column to select column output data, and the output pixel signal is read out by a sense amplifier 62. Here, multiple column selectors 61 constitute a serializer 60. The serializer 60 serializes (converts into series) the output data of the parallel-arranged readout circuits 14 by sequentially selecting and reading column addresses from the horizontal scanning circuit to the column selectors 61 connected to the output data of the buffer memory 15m in the column readout circuit 15. Since the output wiring for the number of parallel circuits is bundled together, the load is large and the signal amplitude is small, so a sense amplifier 62 is connected to the output data of the column selector 61 to amplify the input signal to an amplitude that can be received in the subsequent stage.

[0112] The differences are explained below.

[0113] In the image sensors 400, 400A, and 400C according to Embodiments 1 to 4, the size and coefficients of the filter are changed by adjusting the exposure time, performing in-pixel calculations, and performing column-direction filter calculations. In contrast, Embodiment 5 is characterized by changing the size and coefficients of the filter by providing an arithmetic circuit 15B immediately after the output stage of the AD converter 15A in the column readout circuit 15. Here, the arithmetic circuit 15B is a circuit that calculates the calculation result data of a predetermined filter calculation formula based on the output data of the AD converter 15A, the output data of the buffer memory 15m, and the output data of the adjacent AD converter 15A. Figure 33 shows an embodiment in which, for example, the output data of the buffer memory 15m is input to the arithmetic circuit 15B.

[0114] Figure 34 is a block diagram showing an example configuration of the arithmetic circuit 15B in Figure 33. In Figure 34, the arithmetic circuit 15B is configured to include two coefficient amplifiers 71 and 72, an adder 73, a selector 74, a row direction coefficient setting control circuit 75, and a filter calculation row direction stride setting control circuit 76. Here, the row direction coefficient setting control circuit 75 sets and controls the filter coefficients (weighting coefficients) W1 and W0 by controlling the gain and polarity of the coefficient amplifiers 71 and 72. The filter calculation row direction stride setting control circuit 76 generates and outputs a selection instruction signal Thru to the selector 74 according to a predetermined calculation formula.

[0115] In Figure 34, the output data from the buffer memory 15m is output to the adder 73 via the coefficient amplifier 71. Meanwhile, the output data from the AD converter 15A is output to the adder 73 and the second input terminal of the selector 74 via the coefficient amplifier 72. The output data from the adder 73 is output to the first input terminal of the selector 74. Based on the selection instruction signal Thru, the selector 74 outputs either the data input to the first input terminal or the data input to the second input terminal as the output data of the arithmetic circuit 15B.

[0116] Figure 35 is a timing chart showing the input / output data, weighting coefficients, stride, and calculation formula of the arithmetic circuit 15B in Figure 34. In Figure 35, the output data of the AD converter 15A, the filter coefficient (weighting coefficient) W0, the selection instruction signal Thru, and the output data of the arithmetic circuit 15B are shown, and the calculation formula by the arithmetic circuit 15B is expressed by the following equation.

[0117] I' i =I i-1 ·Wr0 + I i ·Wr1+I i+1 ·Wr2

[0118] The arithmetic circuit 15B in Figure 35 uses the same method as the filter arithmetic circuit 20b in Figure 25, but is characterized by performing calculations in the row direction instead of the column direction.

[0119] In Figure 33, the voltage of the pixel signal is converted into a digital signal for each column by the AD converter 15A. Next, as shown in the example arithmetic circuit 15B, the size and coefficients of the filter are changed by performing calculations on the output data of the AD converter 15A, the output data of the buffer memory 15m, the output data of the adjacent AD converter 15A, the calculation results of these, or the output of the buffer memory 15m, or any combination thereof.

[0120] Generally, compared to imaging for normal human viewing, image recognition applications only require machine recognition, so high resolution is not required for the AD converter 15A. Therefore, the resolution of the AD converter 15A can be reduced, and the arithmetic circuit 15B can be simplified. By performing filtering and data compression in circuit blocks as close to the pixels as possible, the amount of data can be reduced, thereby lowering the power consumption of the subsequent buffer memory 15m, serializer 60, and interface.

[0121] As described above, according to Embodiment 4, the pixel filter can be configured entirely within the image sensor 400D, which is a solid-state imaging device, and the filtering processing required in image processing circuits such as image recognition devices can be reduced. By compressing the amount of data simultaneously with the filtering function, the current consumption can be reduced by slowing down the memory, serializer, and interface speed between the solid-state imaging device and the image recognition device in subsequent circuit blocks.

[0122] (Modified version of Embodiment 5) Figure 36 is a block diagram showing an example configuration of the arithmetic circuit 15BA according to a modified example of Embodiment 5. The arithmetic circuit 15BA in Figure 36 differs from the arithmetic circuit 15B in Figure 14 in the following respects. (1) The coefficient amplifiers 71 and 72 are replaced with coefficient amplifiers 81 and 82. (2) Adder 83 is provided instead of adder 73. (3) Instead of the row-direction coefficient setting control circuit 75, a column-by-coefficient setting control circuit 84 is provided to set the column-by-coefficient W0 and W1 of the coefficient amplifiers 81 and 82. (4) The filter calculation row direction stride setting control circuit 76 is not provided. The differences are explained below. Here, the column-by-coefficient setting control circuit 84 controls the gain and polarity of the coefficient amplifiers 81 and 82 to set and control the filter coefficients (weighting coefficients) W1 and W0.

[0123] In Figure 36, the output data from the AD converter 15A in column (j+1) is output to the adder 83 via the coefficient amplifier 81. On the other hand, the output data from the AD converter 15A in column j is output to the adder 83 via the coefficient amplifier 82. The output data from the adder 83 is output as the output data of the arithmetic circuit 15BA.

[0124] In the arithmetic circuit 15BA shown in Figure 36, column-direction addition is achieved by adding the output data of the adjacent AD converter 15A instead of the output of the buffer memory 15m.

[0125] Figure 37 is a block diagram showing an example configuration of an arithmetic circuit using the arithmetic circuit 15B of Figure 34, Figure 38 is a timing chart showing the input / output data and stride of the arithmetic circuit of Figure 37, and Figure 39 is a diagram showing the filter coefficients of the 3x3 filter of Figure 38. The arithmetic circuit of Figure 37 uses three columns of arithmetic circuits 15B to add the sum of the output data of adjacent AD converters 15A to the output of buffer memory 15m, changing the coefficients Wc0, Wc1, and Wc2 according to the filter size with each data update, thereby realizing a 3x3 filter with arbitrary coefficients.

[0126] According to the modified embodiment 5 configured as described above, it has the same effects and advantages as embodiment 5.

[0127] (Embodiment 6) The image sensor according to Embodiment 6 is an image sensor 400 in Figure 1 according to Embodiment 1, with the addition of a row-direction filter coefficient adjustment circuit by exposure time adjustment as described in Embodiment 1. It is characterized in that some or all of the row-direction filter coefficient adjustment is performed by exposure time adjustment, and the remaining filter calculation is performed by the image sensor configuration in Figures 7 to 13 of Embodiment 2. The image sensor according to Embodiment 6 has the same effects as the image sensor according to Embodiment 2.

[0128] (Embodiment 7) The image sensor according to Embodiment 7 adds row-direction filter coefficient adjustment by exposure time adjustment using the image sensor 400 in Figure 1 according to Embodiment 1, or the image sensor according to Embodiment 2, and an in-pixel calculation circuit according to Embodiment 2. In other words, in Embodiment 7, some or all of the row-direction filter coefficient adjustment is performed by exposure time adjustment, row-direction addition and subtraction are performed by in-pixel calculation, and the remaining column-direction filter calculation is performed in the same way as the image sensor according to Embodiment 2. The image sensor according to Embodiment 7 has the same effects as the image sensor according to Embodiment 2.

[0129] (Embodiment 8) The image sensor according to Embodiment 8 adds a row-direction filter coefficient adjustment circuit using exposure time adjustment with the image sensor 400 in Figure 1 according to Embodiment 1, or the image sensor 400D in Figure 33 according to Embodiment 5, and in-pixel calculations using the image sensor in Figure 14 according to Embodiment 3. That is, some or all of the row-direction filter coefficient adjustment is performed by exposure time adjustment, and row-direction addition and subtraction are performed by in-pixel calculations. Furthermore, the remaining column-direction filter calculations are performed with the configuration of the image sensor 400D in Figure 33 according to Embodiment 5. The image sensor according to Embodiment 8 has the same effects as the image sensor 400D in Figure 33 according to Embodiment 5. (Embodiment 9) Figure 40 is a block diagram showing an example configuration of an image sensor 400E, which is a solid-state imaging device according to Embodiment 9. The image sensor 400E in Figure 40 is characterized by having a readout control circuit 90 added to any of the image sensors 400 according to Embodiments 1 to 8.

[0130] Here, the readout control circuit 90 is a control circuit similar to the controller 120 in Figure 44 of the comparative example, and has a first control 121 for normal imaging and a second control 122 for filtering to extract features. When the first control 121 is active, the image sensor 400E performs normal imaging, and when the second control 122 is active, it performs filtering related to one of the image sensors from Embodiments 1 to 8 and outputs compressed data.

[0131] With the image sensor 400E configured as described above, the solid-state imaging device can be used not only for image recognition but also for normal imaging, and the cost when normal imaging is required in an image recognition system can be reduced compared to conventional technology.

[0132] (Embodiment 10) Figure 41 is a block diagram showing an example configuration of an image sensor 400F, which is a solid-state imaging device according to Embodiment 10. The image sensor 400F in Figure 41 is characterized by adding a resolution switching (adjustment) signal as a control signal for the AD converter 14A to the output of the readout control circuit 90 of the image sensor 400E according to Embodiment 9. Here, the resolution switching of the AD converter 14A may be adjusted by reducing the number of AD conversions in the case of successive approximation AD converters, pipeline AD converters, and cyclic AD converters, or by changing the AD conversion time in the case of single-slope AD converters, ΔΣ AD converters, and integral AD converters, or by simply reducing the output bit width by using only a portion of the output bits.

[0133] According to the image sensor 400E of Embodiment 10 configured as described above, during the first control 121, it is no different from the image sensor 400E of Embodiment 9, but during the second control 122, the resolution of the AD converter 14A is reduced, and filtering processing similar to that of the image sensors of Embodiments 1 to 8 is performed to output more compressed data. The filtered output data is for image recognition, and in some cases it is acceptable to reduce the resolution. For this reason, the resolution is reduced to reduce power consumption. Also, if the recognition rate from the image recognition device is poor, the resolution may be returned to the same as during the first control 121, or if there is no problem with the recognition rate, the resolution may be reduced further.

[0134] With the image sensor 400E configured as described above, the output bit width can be reduced by lowering the resolution of the AD converter 14A, thereby reducing the amount of data and thus enabling low power consumption. In addition, the bit width of the arithmetic circuit of the image sensor according to Embodiments 1 and 2 can also be reduced, thus reducing not only power consumption but also area.

[0135] (Embodiment 11) Figure 42 is a block diagram showing an example configuration of an image recognition system according to Embodiment 11, using the image sensor 400 according to Embodiment 1. The image recognition system in Figure 42 is configured to include the image sensor 400 of Figure 1, which can extract feature quantities such as horizontal edges, and an image recognition device 410 having a recognition detection circuit 411.

[0136] In Figure 42, the image sensor 400 outputs feature data image data, which includes edges (derivatives) used for edge extraction, to the recognition detection circuit 411 of the image recognition device 410. The recognition detection circuit 411 is configured, for example, using a machine learning model, and outputs the recognition result of the image data from the input feature data image data.

[0137] As described above, according to this embodiment, the pixel filter can be configured entirely within the image sensor 400, reducing the filtering processing required in image processing circuits such as the image recognition device 410. By compressing the amount of data simultaneously with the filtering function, the interface speed between the solid-state image sensor of the image sensor 400 and the image recognition device 410 can be reduced, thereby reducing current consumption.

[0138] In the image recognition system according to Embodiment 11 described above, the image sensor 400 shown in Figure 1 according to Embodiment 1 is used. However, the present invention is not limited to this, and modified versions of Embodiment 1 and Embodiments 2 to 8 and their modified versions may also be used.

[0139] (Embodiment 12) Figure 43 is a block diagram showing an example configuration of an image recognition system according to Embodiment 12, using the image sensor 400 according to Embodiment 1. The image recognition system in Figure 43 differs from the image recognition system in Figure 42 in the following ways. (1) Instead of the image recognition device 410, an image recognition device 410A is provided, which includes a pixel filter control circuit 412 and a recognition detection circuit 411. The differences are explained below.

[0140] In Figure 43, the pixel filter control circuit 412 is characterized by controlling the pixel filter settings of the image sensor 400, which is a solid-state imaging device, according to the output data of the recognition detection circuit 411 of the image recognition device 410A. In order to increase the image recognition rate of the image recognition system, the control circuit 412 determines, for example, whether to add averaging processing when there is a lot of noise, reduce the filter size to lower the data compression rate when the image recognition rate is poor, or whether it is preferable to change the vertical or horizontal direction of the differential filter depending on the object to be recognized and detected, and performs an operation to change the type of filter according to the object to be recognized. The filter setting may also be turned off and switched to normal imaging.

[0141] As described above, according to this embodiment, the filter size and coefficient can be changed on the image sensor 400 side, so that various image filtering functions necessary for image processing can be executed while maintaining the advantage of low current consumption in Embodiment 11. By changing and controlling the pixel filter according to the target of image recognition, the optimal pixel filter for the target of image recognition can be selected, thereby suppressing the increase in current consumption and improving the recognition rate.

[0142] (modified version) The image recognition system according to Embodiment 11 and the image recognition system according to Embodiment 12 are, for example, examples of image processing systems. The image recognition system or image processing system according to Embodiment 11 or 12 may be installed in, for example, an electronic device having the following known configuration. (1) Surveillance camera system for motion detection for security purposes, (2) Indoor human detection camera device, and (3) Fixed-point camera system for analyzing pedestrian flow.

[0143] In the embodiments and modifications described above, the horizontal scanning circuit, vertical scanning circuit, readout circuit, and readout control circuit are examples of control circuits for controlling the electrical exposure time according to the present invention. Here, when controlling the electrical exposure time according to the present invention, the scanning direction to be controlled is at least one of the horizontal and vertical directions. [Industrial applicability]

[0144] As detailed above, the solid-state imaging device, etc., according to the present invention reduces power consumption when filtering pixel signals from a solid-state imaging device such as an image sensor, and allows for setting predetermined filter coefficients with a smaller circuit size and simpler adjustment method compared to conventional technology. Specifically, the filter size and coefficients can be changed by adjusting the exposure time of the solid-state imaging device and by a simple calculation circuit and memory after the AD converter, enabling miniaturization of the circuit and filter modification by software. [Explanation of Symbols]

[0145] 10-pixel array 11,11A Pixel circuit (pixel) 12 Vertical scanning circuit 13 Horizontal scanning circuit 14, 14B, 14P Readout Circuit 14m buffer memory 14A AD Converter (ADC) 14PC Readout Control Circuit 15-column readout circuit 15A AD Converter (ADC) 15B, 15BA arithmetic circuit 15m buffer memory 16 CDS circuit 17 Column Switch 18 Readout Circuit 19 Column Readout Circuit Input Section 20, 20a, 20b Filter Calculation Circuit 20A Weighted Adder Circuit 21,22 Delay devices 30, 31, 32 coefficient amplifier 41,42 Adder 50 Filter size and coefficient control circuit 51 Coefficient Amplifier 52 Adder 53 Selector 54 Delay devices 55-column coefficient setting control circuit 56 Filter Calculation Column Direction Stride Setting Control Circuit 60 serializers 61 Column Selector 62 Sense Amplifier 71 Coefficient Amplifier 72 coefficient amplifier 73 Adder 74 Selector 75. Row direction coefficient setting control circuit 76 Filter Calculation Row Direction Stride Setting Control Circuit 81 Coefficient Amplifier 82 Coefficient Amplifier 83 Adder 84-column coefficient setting control circuit 90 Readout control circuit 100 Image Sensors 110 pixel section 111,111A~111C Pixel Circuit 120 controllers 121 First Mode Control 122 Second Mode Control 125 Switch 130 Readout Circuit 135 signal line 140 Vertical scanning circuit 150 Horizontal scanning circuit 200 Image Recognition Devices 210 Image Recognition Requires Machine Learning Models 220 Event Detectors 320 MOS transistors 340 Source Follower Circuit (SF Circuit) 400, 400A~400F, 400Ba Image Sensor (Solid State Imaging Device) 410, 410A Image Recognition Device 411 Recognition detection circuit 412 Pixel Filter Control Circuit 501-504 control lines C0~C32 Capacitors C CDS CDS capacitor COL Column Switch CLP Clipgate CS constant current source FD Floating Diffusion FDA Floating Diffusion Amplifier INV1~INV3 Inverter PD (Photoelectric Detector) RST Reset Gate SF Source Follower Transistor SEL Selection Gate SG Switching Gate SW0~SW7 switches TG, TG1~TG4 Transfer Gates

Claims

1. A plurality of pixel circuits arranged in the row and column directions, each of which includes a photoelectric conversion element that converts an incident optical signal into a pixel signal, A solid-state imaging device comprising a control circuit that controls the reading of each pixel signal from the plurality of pixel circuits in the row direction and column direction using a predetermined reading method, The control circuit controls the application of image filtering to at least one of the pixel signals from the plurality of pixel circuits and the pixel signals within the plurality of pixel circuits. Solid-state imaging device.

2. The control circuit controls the filter coefficients by controlling the exposure time when reading out each pixel signal from the plurality of pixel circuits in at least one of the row and column directions, based on the filter size and filter coefficients when performing the image filtering process. The solid-state imaging apparatus according to claim 1.

3. The aforementioned reading method is (1) A column-parallel readout method for reading pixel signals in parallel from a predetermined number of pixel circuits from a plurality of pixel circuits arranged side by side in the column direction, (2) A row-parallel readout method for reading pixel signals in parallel from a predetermined number of pixel circuits from a plurality of pixel circuits arranged side by side in the row direction, (3) A pixel-by-pixel readout method for reading out pixel signals from a predetermined number of pixel circuits from a plurality of pixel circuits, At least one of the following: The solid-state imaging apparatus according to claim 1.

4. The solid-state imaging apparatus according to claim 2, wherein the control circuit performs the image filtering process by performing a sum-of-products operation on each pixel signal based on the filter size and the filter coefficients.

5. The circuit further comprises an arithmetic circuit located after the readout circuit that reads out each of the aforementioned pixel signals, The control circuit performs the image filtering process by having the calculation circuit perform a sum-of-products operation on each pixel signal based on the filter size and the filter coefficients, according to claim 4.

6. The plurality of pixel circuits have an addition function with adjacent pixel circuits in the row direction. The solid-state imaging apparatus according to claim 1 or 2, wherein the control circuit performs the image filtering process by performing a sum-of-products operation on each pixel signal using the addition function, based on the filter size and filter coefficients when performing the image filtering process.

7. The solid-state imaging apparatus according to claim 1 or 2, wherein the control circuit divides the plurality of pixel circuits into a predetermined plurality of pixel circuit groups and performs the image filtering process by performing a sum-of-products operation on each pixel signal based on the filter size and filter coefficients when performing the image filtering process using each pixel signal from the plurality of pixel circuit groups for each plurality of pixel circuit groups.

8. Each of the multiple column readout circuits that read out each of the aforementioned pixel signals is further provided with a plurality of arithmetic circuits, The control circuit performs the image filtering process by having the calculation circuit perform a sum-of-products operation on each pixel signal based on the filter size and filter coefficients when performing the image filtering process, as described in claim 1.

9. A vertical scanning circuit that performs row-direction scanning on the plurality of pixel circuits, A horizontal scanning circuit that performs scanning in the column direction for the plurality of pixel circuits, A readout circuit that reads image data from the plurality of pixel circuits under the control of the horizontal scanning circuit, The system comprises the vertical scanning circuit, the horizontal scanning circuit, and a readout control circuit that controls the operation of the readout circuit, The readout circuit includes an AD converter in its output stage that converts each pixel signal from analog to digital. The solid-state imaging apparatus according to claim 1 or 2.

10. The readout control circuit controls the resolution of the AD converter. The solid-state imaging apparatus according to claim 9.

11. A solid-state imaging device according to claim 1, An image recognition device that performs image recognition processing based on image-filtered feature data from the solid-state imaging device, An image processing system equipped with the following features.

12. The image recognition device performs control to change the pixel filter settings according to the output data of the recognition detection circuit. The image processing system according to claim 11.

13. An electronic device comprising the image processing system according to claim 11 or 12.

Images