Weighting addition circuit and solid-state image pickup device

The weighted addition circuit in solid-state imaging devices manages the increase in signals by sharing readout wiring and capacitors, optimizing power consumption and reducing the number of wires, addressing the inefficiencies in deep neural network calculations.

JP2024158002AInactive Publication Date: 2024-11-08SONY SEMICON SOLUTIONS CORP
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
JP2023072751
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-04-26
Publication Date
2024-11-08
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

Deep neural networks require a large number of calculations and consume significant power for weighted addition, particularly in image sensors where signals from a pixel array are processed, leading to an increase in the number of readout lines and signals, which is not efficiently managed by existing technologies.

Method used

A weighted addition circuit that connects readout wiring with vertical signal lines and capacitors, using switches to sample and hold signals, allowing for sequential addition without increasing the number of wires, and incorporates a mechanism to share readout wiring, thereby reducing the need for additional capacitors and optimizing power consumption.

Benefits of technology

The solution effectively increases the number of weighted additions while suppressing the increase in readout lines, thereby optimizing power consumption and reducing the trade-off between the number of weighted additions and wires, enhancing the efficiency of signal processing in solid-state imaging devices.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024158002000001_ABST
    Figure 2024158002000001_ABST
Patent Text Reader

Abstract

To provide a weighting addition circuit and a solid-state image pickup device that can increase the number of weighted additions while suppressing an increase in the number of wirings of read lines.SOLUTION: A weighted addition circuit according to the present disclosure includes a readout wiring that receives input of a plurality of signals read out to a vertical signal line of a solid-state imaging device, a first switch that connects or disconnects the vertical signal line and the readout wiring, a plurality of capacitors that sample and hold the plurality of signals, and a plurality of second switches that connect or disconnect the readout wiring and the capacitors.SELECTED DRAWING: Figure 3
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present disclosure relates to a weighting addition circuit and a solid-state imaging device. [Background technology]

[0002] Deep Neural Networks (DNNs) are used as a technology to improve recognition performance, but the weighted addition used in this technology requires a large number of calculations and consumes a lot of power.

[0003] Meanwhile, with regard to weighted addition, a technology has been proposed that reduces power consumption by completing the calculation before performing AD (Analog to Digital) conversion, moving away from the conventional method of performing it using digital circuits. For example, in an image sensor, a technology has been proposed that performs a convolution calculation by performing weighted addition on the signals output from a pixel array. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2020-113809 A Summary of the Invention [Problem to be solved by the invention]

[0005] In the above-mentioned technology, signals are read out from the pixel array through multiple readout lines. In addition, weighted addition can be performed by adding signals through a capacitance corresponding to each readout line. However, the number of signals to be added simultaneously and the number of wirings of the readout lines are one to one, so if the number of additions is increased, the number of signals also increases accordingly.

[0006] In view of the above problems, the present disclosure provides a weighting addition circuit and a solid-state imaging device that can increase the number of weighting additions while suppressing an increase in the number of wirings of readout lines. [Means for solving the problem]

[0007] A weighted addition circuit according to a first aspect of the present disclosure includes a readout wiring that receives input of a plurality of signals read out to a vertical signal line of a solid-state imaging device, a first switch that connects or disconnects the vertical signal line and the readout wiring, a plurality of capacitors that sample and hold the plurality of signals, and a plurality of second switches that connect or disconnect the readout wiring and the capacitors. This allows the weighted addition circuit to share the readout wiring with each capacitor, thereby suppressing an increase in the number of wirings. Even if the number of signals to be added increases, the weighted addition circuit sequentially samples and holds the signals and performs weighted addition, thereby eliminating the trade-off between the number of weighted additions and the number of wirings.

[0008] Also, in this first aspect, the solid-state imaging device includes a cell array having a mechanism for holding the plurality of signals, the vertical signal lines for reading out the plurality of signals from the cell array, and an AD conversion unit for performing AD conversion of the signals weighted and added by the weighting addition circuit, the weighting addition circuit being connected to the cell array, the vertical signal lines, and the AD conversion unit. In this way, the weighting addition circuit can suppress an increase in the number of wirings by sharing the readout wirings with each capacitor. Even if the number of signals to be added increases, the weighting addition circuit sequentially samples and holds the signals and performs weighting addition, so that the trade-off between the number of weighting additions and the number of wirings can be eliminated.

[0009] In addition, in this first aspect, the weighted addition circuit sequentially samples and holds the multiple signals sequentially read out to the vertical signal lines, and performs weighted addition after completing the sample-and-hold of the multiple signals. In this way, the weighted addition circuit can suppress an increase in the number of wirings by sharing the readout wirings with each capacitor. Even if the number of signals to be added increases, the weighted addition circuit sequentially samples and holds the signals and performs weighted addition, so that it is possible to eliminate the trade-off between the number of weighted additions and the number of wirings.

[0010] In addition, in this first aspect, the cells included in the cell array are pixels including photodiodes or memories included in a memory array, whereby the weighted addition circuit can be connected not only to a pixel array unit but also to a cell array such as a memory array to perform weighted addition.

[0011] In the first aspect, the weighted addition circuit further includes a fourth switch that can switch a terminal voltage value at a terminal of the plurality of capacitors opposite to the terminal on the readout line side, whereby the weighted addition circuit can realize the function of a successive approximation type AD converter without adding a separate capacitor.

[0012] In addition, in the first aspect, the AD conversion unit performs a successive approximation operation between the DAC signal generated by the weighted addition circuit based on the switching of the fourth switch and the weighted-added signal, thereby enabling the weighted addition circuit to realize the function of a successive approximation type AD converter without adding any additional capacitance.

[0013] A solid-state imaging device according to a second aspect of the present disclosure includes a cell array having a mechanism for holding a plurality of signals, vertical signal lines for reading out the plurality of signals from the cell array, a weighting addition circuit for sampling and holding the plurality of signals read out to the vertical signal lines and performing weighting addition, and an AD conversion unit for performing AD conversion of the signals weighted and added by the weighting addition circuit, the weighting addition circuit including a readout wiring for receiving input of the plurality of signals read out to the vertical signal lines, a first switch for connecting or disconnecting the vertical signal line and the readout wiring, a plurality of capacitors for sampling and holding the plurality of signals, and a plurality of second switches for connecting or disconnecting the readout wiring and the capacitors. This allows the weighting addition circuit to share the readout wiring with each capacitor, thereby suppressing an increase in the number of wirings. Even if the number of signals to be added increases, the weighting addition circuit sequentially samples and holds the signals and performs weighting addition, thereby eliminating the trade-off between the number of weighting additions and the number of wirings.

[0014] In addition, in this second aspect, the weighted addition circuit sequentially samples and holds the multiple signals sequentially read out to the vertical signal lines, and performs weighted addition after completing the sample-and-hold of the multiple signals. In this way, the weighted addition circuit can suppress an increase in the number of wirings by sharing the readout wirings with each capacitor. Even if the number of signals to be added increases, the weighted addition circuit sequentially samples and holds the signals and performs weighted addition, so that it is possible to eliminate the trade-off between the number of weighted additions and the number of wirings.

[0015] In the second aspect, the cells included in the cell array are pixels including photodiodes or memories included in a memory array, whereby the weighted addition circuit can be connected not only to a pixel array unit but also to a cell array such as a memory array to perform weighted addition.

[0016] In the second aspect, the plurality of capacitances include a first capacitance and a second capacitance having different capacitances, whereby the weighted addition circuit can change the weighting of the input signal depending on the magnitude of the capacitance.

[0017] In the second aspect, a buffer circuit that provides a delay time for the input of the plurality of signals is connected between the vertical signal line and the weighted addition circuit, whereby the weighted addition circuit can provide a delay time when the signals are input to the weighted addition circuit by the buffer circuit, and can suppress deviations in charge accumulation time due to differences in the magnitude of the electrostatic capacitance of the capacitors.

[0018] In addition, in this second aspect, the weighted addition circuit further includes one or more dummy capacitances that adjust the difference in capacitance between the first capacitance and the second capacitance when sampling and holding, and one or more third switches that connect or disconnect the readout wiring and the dummy capacitance. This allows the weighted addition circuit to virtually adjust all capacitances to be constant when performing sampling and holding, and can suppress deviations in charge accumulation time due to differences in capacitance magnitude of the capacitances.

[0019] In the second aspect, the weighted addition circuit stores charge in one or more of the dummy capacitances based on the capacitance difference between the first capacitance and the second capacitance, whereby the weighted addition circuit can adjust all capacitances to be pseudo-constant when performing sample and hold, thereby suppressing deviations in charge accumulation time due to differences in capacitance size of the capacitances.

[0020] A solid-state imaging device according to a third aspect of the present disclosure includes a cell array having a mechanism for holding a plurality of signals, a vertical signal line for reading out the plurality of signals from the cell array, a first weighting circuit for sampling and holding the plurality of signals read out to the vertical signal line and performing weighting addition of positive values, a second weighting circuit for sampling and holding the plurality of signals read out to the vertical signal line and performing weighting addition of negative values, and an AD conversion unit for AD conversion of the signals weighted by the first weighting circuit and the signals weighted by the second weighting circuit, each of the first and second weighting circuits including a readout wiring for receiving an input of the plurality of signals read out to the vertical signal line, a first switch for connecting or disconnecting the vertical signal line and the readout wiring, a plurality of capacitances for sampling and holding the plurality of signals, and a plurality of second switches for connecting or disconnecting the readout wiring and the capacitance. This allows the weighting circuit to perform weighting addition of negative values ​​in addition to weighting addition of positive values.

[0021] In the third aspect, the cells included in the cell array are pixels including photodiodes or memories included in a memory array, whereby the weighted addition circuit can be connected not only to a pixel array unit but also to a cell array such as a memory array to perform weighted addition.

[0022] In this third aspect, each of the first and second weighted addition circuits sequentially samples and holds the multiple signals sequentially read out to the vertical signal lines, and performs weighted addition of positive values ​​and weighted addition of negative values ​​after completing the sample-and-hold of the multiple signals, thereby enabling the weighted addition circuit to perform weighted addition of negative values ​​in addition to weighted addition of positive values.

[0023] In the third aspect, the cells included in the cell array include a first selection transistor that switches the signals to the first weighted addition circuit and a second selection transistor that switches the signals to the second weighted addition circuit. This allows the cell to input signals to the first weighted addition circuit or the second weighted addition circuit by switching these transistors. This allows the weighted addition circuit to perform weighted addition of negative values ​​in addition to weighted addition of positive values. Furthermore, the weighted addition circuit allows the first weighted addition circuit to input a positive value signal and the second weighted addition circuit to input a negative value signal based on the opening and closing of the first selection transistor and the second selection transistor.

[0024] In the second aspect, the weighted addition circuit further includes a fourth switch that can switch a terminal voltage value at a terminal of the plurality of capacitors opposite to the terminal on the readout line side, thereby enabling the weighted addition circuit to realize the function of a successive approximation type AD converter without adding a separate capacitor.

[0025] In the second aspect, the AD conversion unit performs successive approximation between the DAC signal generated by the weighted addition circuit based on the switching of the fourth switch and the weighted-added signal, thereby enabling the weighted addition circuit to realize the function of a successive approximation type AD converter without adding any additional capacitance.

[0026] In this third aspect, each of the first and second weighted addition circuits further includes a fourth switch that can switch a terminal voltage value at a terminal of the plurality of capacitors opposite to the terminal on the readout line side, thereby enabling the weighted addition circuit to realize the function of a successive approximation type AD converter without adding a separate capacitor. [Brief description of the drawings]

[0027] [Figure 1] 1 is a block diagram showing a schematic configuration of a solid-state imaging device according to a first embodiment. [Diagram 2] 2 is a diagram showing an example of a circuit configuration of pixels arranged in a pixel array unit in the first embodiment. FIG. [Diagram 3] FIG. 4 is a diagram illustrating an example of a circuit configuration of a weighted addition circuit in the first embodiment. [Figure 4] 4 is an example of a timing chart of the weighted addition circuit in the first embodiment. [Diagram 5] 6 is another example of a timing chart of the weighted addition circuit and the like in the first embodiment. [Figure 6] FIG. 4 is a diagram showing a modified example of the pixel in the first embodiment. [Figure 7] 13 is a modified example of the pixel array in the first embodiment. [Figure 8] FIG. 11 is a circuit diagram of a weighting addition circuit in the second embodiment. [Figure 9] FIG. 13 is a circuit diagram of a weighting addition circuit in the third embodiment. [Figure 10] 13 is a timing chart of the weighting addition circuit in the third embodiment. [Figure 11] FIG. 13 is a circuit diagram of a weighting addition circuit and the like in a fourth embodiment. [Figure 12] 13 is a timing chart of a weighting addition circuit and the like in the fourth embodiment. [Figure 13] FIG. 13 is a circuit diagram of a weighting addition circuit and the like in the fifth embodiment. [Figure 14] FIG. 13 is a circuit diagram of a weighting addition circuit and the like in the sixth embodiment. [Figure 15] FIG. 23 is a circuit diagram of a modified example of the weighting addition circuit etc. in the sixth embodiment. [Figure 16] FIG. 1 is a block diagram showing an example of the configuration of a vehicle control system. [Figure 17] FIG. 2 is a diagram showing an example of a sensing region. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0028] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings.

[0029] (First embodiment) FIG. 1 is a block diagram showing a schematic configuration of a solid-state imaging device according to the first embodiment.

[0030] 1 includes a pixel array section 20 in which a plurality of pixels are arranged in a matrix, and a peripheral circuit section around the pixel array section 20. The peripheral circuit section includes a vertical drive section 12, an AD conversion section 13, a horizontal drive section 14, a control section 15, a signal processing circuit 16, a data storage section 17, an input / output section 18, and the like.

[0031] Each pixel arranged two-dimensionally in the pixel array unit 20 is composed of a photodiode as a photoelectric conversion unit and a plurality of pixel transistors, etc. The plurality of pixel transistors are, for example, MOS transistors such as a transfer transistor, an amplification transistor, a selection transistor, a reset transistor, etc. In each pixel of the pixel array unit 20, for example, red, green, or blue color filters are arranged in a Bayer array, and each pixel outputs a red, green, or blue pixel signal (hereinafter also simply referred to as a signal).

[0032] The vertical drive unit 12 is configured by, for example, a shift register, and drives the pixels in row units by supplying a drive pulse to each pixel of the pixel array unit 20 via pixel drive wiring (not shown). That is, the vertical drive unit 12 sequentially selects and scans each pixel of the pixel array unit 20 in the vertical direction in row units, and supplies a pixel signal based on a signal charge generated in the photodiode of each pixel according to the amount of incident light to a weighting addition circuit 19 through a vertical signal line provided in common for each column.

[0033] The weighted addition circuit 19 is provided, for example, for each pixel that shares a vertical signal line. The vertical signal line reads out a plurality of pixel signals for a predetermined weighted addition unit from each pixel of the pixel array unit 20, and sequentially supplies the signals to the weighted addition circuit 19. The weighted addition circuit 19 performs weighted addition on the signals received as input from the vertical signal line. This weighted addition unit is selected from among the pixels that share the vertical signal line. Details of the weighted addition circuit 19 will be described later.

[0034] An AD conversion unit (ADC) 13 performs a CDS (Correlated Double Sampling) process for removing pixel-specific fixed pattern noise and AD conversion on the signal output from the weighting addition circuit 19.

[0035] The horizontal drive unit 14 is composed of, for example, a shift register, and sequentially outputs horizontal scanning pulses to sequentially output the (digital) pixel signals after AD conversion of each pixel in a specified row held in the AD conversion unit 13 to the signal processing circuit 16.

[0036] The control unit 15 receives a clock signal input from the outside and data instructing an operation mode, etc., and controls the operation of the entire solid-state imaging device 10. For example, the control unit 15 generates a vertical synchronization signal, a horizontal synchronization signal, etc. based on the input clock signal, and supplies them to the vertical drive unit 12, the AD conversion unit 13, the horizontal drive unit 14, etc.

[0037] The signal processing circuit 16 performs various digital signal processing such as black level adjustment processing, column variation correction processing, and demosaic processing on the pixel signals supplied from the AD conversion unit 13 as necessary, and supplies the resulting signal to the input / output unit 18. Depending on the operation mode, the signal processing circuit 16 may only buffer the pixel signals and output the resulting signal. The data storage unit 17 stores data such as parameters required for the signal processing performed by the signal processing circuit 16. The data storage unit 17 also includes a frame memory for storing image signals in processing such as demosaic processing. The signal processing circuit 16 can store parameters and the like input from an external image processing device via the input / output unit 18 in the data storage unit 17, and can appropriately select and execute signal processing based on instructions from the external image processing device.

[0038] The input / output unit 18 outputs image signals sequentially input from the signal processing circuit 16 to an external image processing device, for example, a downstream ISP (Image Signal Processor) etc. The input / output unit 18 also supplies signals and parameters input from the external image processing device to the signal processing circuit 16 and the control unit 15.

[0039] The solid-state imaging device 10 is configured as described above, and is, for example, a CMOS image sensor of a so-called column AD type in which CDS processing and AD conversion processing are performed for each pixel column.

[0040] FIG. 2 is a diagram showing an example of a circuit configuration of pixels arranged in a pixel array unit in the first embodiment.

[0041] The pixel 100 in FIG. 2 includes a photodiode PD, a transfer transistor TRG, a floating diffusion FD, a reset transistor RST, an amplification transistor AMP, and a selection transistor SEL.

[0042] The photodiode PD receives the incident light, photoelectrically converts it, and accumulates the resulting charge. When the drive signal supplied to the gate of the transfer transistor TRG becomes Hi and the transfer transistor TRG is turned on, the charge accumulated in the photodiode PD is transferred to the floating diffusion FD via the transfer transistor TR.

[0043] The floating diffusion FD temporarily holds the charge transferred from the transfer transistor TR. The floating diffusion FD is also connected to a vertical signal line VSL via an amplifier transistor AMP and a selection transistor SEL. The vertical signal line VSL is connected to the weighted addition circuit 19 of the same column.

[0044] When the reset transistor RST is turned on by a reset signal, the charge held in the floating diffusion FD is discharged to the constant voltage source VDD, thereby resetting the potential of the floating diffusion FD.

[0045] The amplification transistor AMP outputs a pixel signal according to the potential of the floating diffusion FD. That is, the amplification transistor AMP forms a source follower circuit together with a load MOS (not shown) as a constant current source, and a pixel signal indicating a level according to the charge held in the floating diffusion FD is output from the amplification transistor AMP to the weighted addition circuit 19 via the selection transistor SEL.

[0046] The selection transistor SEL is turned on when the pixel 100 is selected by a selection signal, and outputs a pixel signal of the pixel 100 to the weighting addition circuit 19 via the vertical signal line VSL.

[0047] Note that the pixel circuit in Figure 2 is a rolling shutter type pixel circuit, but a charge holding section (memory section) that stores the charge transferred by the transfer transistor TRG and a transfer transistor that transfers the charge in the charge holding section to the floating diffusion FD may be further provided between the transfer transistor TRG and the floating diffusion FD, thereby enabling driving by a global shutter type.

[0048] FIG. 3 is a diagram illustrating an example of a circuit configuration of the weighting addition circuit in the first embodiment.

[0049] 3 includes readout wiring 130, a first switch 131, a plurality of capacitors 132, and a plurality of second switches 133. Furthermore, the input side of the weighted addition circuit 19 is connected to a plurality of pixels 100 via the vertical signal line VSL through the readout wiring 130, and the output side is connected to the AD conversion unit 13. That is, the weighted addition circuit 19 sequentially receives inputs of signals from the plurality of pixels 100 through the vertical signal line VSL, performs weighted addition, and then inputs these signals to the AD converter.

[0050] In FIG. 3, the multiple capacitances 132 have different capacitances. In this way, the weighted addition circuit 19 may have a first capacitance and a second capacitance with different capacitances for the capacitances 132. The weighted addition circuit 19 can change the weighting of the input signal depending on the magnitude of the capacitance. In the example of FIG. 3, the weighted addition circuit 19 includes four capacitances 132 and four second switches 133, and the capacitances of the capacitances 132 are C3, C2, C1, and C0 (C3>C2>C1>C0). In addition, the weighted addition circuit 19 performs weighted addition for the pixels 100 connected to the same vertical signal line VSL in units of the number of capacitances 132. For example, when twelve pixels 100 are connected to the same vertical signal line VSL and the weighted addition circuit 19 includes four capacitances 132, the twelve pixels 100 are subjected to weighted addition in three separate times.

[0051] The first switch 131 connects or disconnects the vertical signal line VSL and the readout wiring 130. When the first switch 131 is in an on state, the vertical signal line VSL and the readout wiring 130 are connected to each other, and the first switch 131 receives an input of a signal output from the pixel 100.

[0052] The second switch 133 connects or disconnects the readout wiring 130 and the capacitance 132. When the second switch 133 is turned on, the signal received by the weighted addition circuit 19 connects the readout wiring 130 to the corresponding capacitance 132 and stores charge in the capacitance 132. In this way, the weighted addition circuit 19 receives the signal input from the vertical signal line VSL and stores the signal charge in the corresponding capacitance 132, which is called sample and hold. The weighted addition circuit 19 can weight each signal based on the magnitude of the electrostatic capacitance of the capacitance 132.

[0053] After the sampling and holding is completed in each capacitor 132, the weighted addition circuit 19 turns on each second switch 133 and adds up all the signals sampled and held in the readout wiring 130. Adding up all the sampled and held signals in this manner is called weighted addition.

[0054] The weighted-added signal after weighting is output to AD conversion unit 13. AD conversion unit 13 receives the weighted-added signal and performs AD conversion processing. AD conversion unit 13 may also perform AD conversion processing on the weighted-added signal after performing CDS processing.

[0055] FIG. 4 is an example of a timing chart of the weighted addition circuit in the first embodiment.

[0056] FIG. 4 shows an example in which the weighting addition circuit 19 sequentially receives signal inputs from four pixels and performs weighting addition.

[0057] The timing chart of FIG. 4 shows an example in which a plurality of signals (signal voltages V0 to V3) read out from the pixel array unit 20 via the vertical signal lines VSL are sequentially sample-held by respective capacitors 132 having capacitance values of C0, C1, C2, and C3 (C0 < C1 < C2 < C3). Further, the timing chart of FIG. 4 shows an example in which weighted addition is performed on the readout wiring 130 after each capacitor 132 has performed sample-holding.

[0058] In this example, each second switch 133 connected to the capacitor 132 having capacitance values of C0, C1, C2, and C3 is represented as SW0, SW1, SW2, and SW3, respectively.

[0059] The weighted addition circuit 19 in the present embodiment shows an example in which the second switches 133 are turned on in the order of SW0 to SW3 for sample-holding. However, for example, the second switches 133 may be turned on in the order of SW3 to SW0, or may be turned on in other orders. Also, for the sake of explanation, in the timing chart of FIG. 4, the first switch 131 is denoted as SH. Also, the readout wiring 130 is denoted as PN.

[0060] First, the weighted addition circuit 19 turns on the first switch 131 to connect the vertical signal line VSL and the readout wiring 130, and receives an input of a signal from the pixel array unit 20. At this time, the voltage of SH becomes Hi, and the voltage state is maintained until the start of weighted addition.

[0061] Next, the weighted addition circuit 19 turns on SW0 among the second switches 133 to connect the readout wiring 130 and C0 among the capacitors 132. C0 receives an input of the signal of the signal voltage V0 and performs sample-holding. At this time, the voltage state of SW0 becomes Hi, and the voltage state is maintained until charge is stored in C0. Also, the charge Q0 stored in C0 is expressed as Equation (1). Q0 = V0 × C0 (1)

[0062] Next, the weighted addition circuit 19 turns on SW1 of the second switch 133, and connects the readout wiring 130 to C1 of the capacitor 132. C1 accepts the input of a signal of the signal voltage V1 and samples and holds it. At this time, the voltage state of SW1 becomes Hi, and the voltage state is maintained until a charge is stored in C1. The charge Q1 stored in C1 is expressed by equation (2). Q1 = V1 × C1 (2)

[0063] Next, the weighted addition circuit 19 turns on SW2 of the second switch 133, and connects the readout wiring 130 to C2 of the capacitor 132. C2 accepts the input of a signal of the signal voltage V2 and samples and holds it. At this time, the voltage state of SW2 becomes Hi, and the voltage state is maintained until a charge is stored in C2. The charge Q2 stored in C2 is expressed by equation (3). Q2 = V2 × C2 (3)

[0064] Next, the weighted addition circuit 19 turns on SW3 of the second switch 133, and connects the readout wiring 130 to C3 of the capacitor 132. C3 accepts the input of a signal of signal voltage V3 and samples and holds it. At this time, the voltage state of SW3 becomes Hi, and the voltage state is maintained until charge is stored in C3. Moreover, the charge Q3 stored in C3 is expressed by equation (4). Q3 = V3 × C3 (4)

[0065] After the sample and hold is completed in each capacitor 132, the weighted addition circuit 19 turns on all of the second switches 133 and performs addition in the read wiring 130. The signal voltage V of the read wiring 130 at this time is expressed by equation (5). V=(Q0+Q1+Q2+Q3) / (C0+C1+C2+C3) (5)

[0066] As described above, the weighted and added signal is input to the AD conversion unit 13. Also, the capacitance ratio of C3 to C0 is set to a power of 2, and each signal is added to perform a bit-serial calculation. In this way, the value of the capacitance 132 can be flexibly designed.

[0067] In this example, the weighting addition circuit 19 turns on SW0, SW1, SW2, and SW3 in this order, but this order is not limited, and the switches may be turned on in any order.

[0068] FIG. 5 is another example of a timing chart of the weighting addition circuit and the like in the first embodiment.

[0069] 5 shows an example in which signals are sequentially input from four pixels 100, sampled and held, and weighted addition is performed, similar to the example in FIG. 4. In addition, in the circuit in FIG. 5, a voltage source 140 for a dynamic source follower and a current source 141 for a source follower are connected to the vertical signal line VSL. The power supply voltage value of the voltage source 140 may be any voltage value including 0V. In addition, in the circuit in FIG. 5, only one of the voltage source 140 and the current source 141 may be connected to the vertical signal line VSL.

[0070] In this example, the AD conversion unit 13 performs a CDS operation for weighted addition. The CDS operation reads out a signal from the pixel 100 twice and obtains the difference between them. In this CDS operation, a signal obtained when the floating diffusion FD is initialized is read out as a P-phase level, and a signal obtained when a charge is transferred from the photodiode PD to the floating diffusion FD is read out as a D-phase level. Fixed pattern noise is removed by obtaining the difference between these P-phase level and D-phase level.

[0071] Specifically, using the timing chart of FIG. 5, first, to read out the P-phase level, the floating diffusion FD is initialized by the reset transistor RST0, the selection transistor SEL0 is turned on, and a reset level signal is sent to the weighted addition circuit 19. The weighted addition circuit 19 samples and holds the reset level signal by the corresponding capacitance C0. The weighted addition circuit 19 repeats this operation from C0 to C3, and then performs weighting and addition on the read wiring 130. The added signal is input to the AD conversion unit 13 as a reset signal, and AD conversion is performed.

[0072] Next, to read out the D-phase level, the transfer transistor TRG3 is turned on, and the selection transistor SEL is also turned on, thereby transmitting the charge accumulated in the photodiode PD to the weighted addition circuit 19. The weighted addition circuit 19 samples and holds the signal using the corresponding capacitance C0. The weighted addition circuit 19 repeats this operation from C0 to C3, and after completing the sampling and holding of the signal, performs weighted addition on the read wiring 130. The signal that has been subjected to the weighted addition is input as a data signal to the AD conversion unit 13, where AD conversion is performed.

[0073] The AD conversion unit 13 removes fixed pattern noise by determining the difference between the data signal and the reset signal.

[0074] FIG. 6 is a diagram showing a modification of the pixel in the first embodiment.

[0075] 6A shows an example of a so-called pixel-sharing circuit configuration in which a configuration including a photodiode PD, a first transfer transistor TRG', a second transfer transistor TRG'', and a memory MEM is regarded as one unit. In FIG. 6A, these two units share a floating diffusion FD, a reset transistor RST, and an amplification transistor AMP.

[0076] The first transfer transistor TRG' connected to the photodiode PD side transfers the charge stored in the photodiode PD to the memory MEM. The second transfer transistor TRG'' connected to the floating diffusion FD side transfers the charge stored in the memory MEM to the floating diffusion FD.

[0077] The circuit configuration of FIG. 6A differs from the pixel 100 of FIG. 2 in that it includes a memory MEM and a first transfer transistor TRG' that transfers charge to the memory MEM, in addition to a second transfer transistor TRG'' that transfers charge to the floating diffusion FD. The circuit configuration of FIG. 6A also differs in that it includes two photodiodes PD, etc.

[0078] In this circuit configuration, the number of circuits such as the floating diffusion FD shared by the photodiode PD, the first transfer transistor TRG', the second transfer transistor TRG'', and the memory MEM is not limited to two. Any number of photodiodes PD, etc. can share the circuit such as the floating diffusion FD.

[0079] Fig. 6B shows an example of a pixel sharing circuit configuration in which a configuration including an organic photoelectric conversion film or an inorganic photoelectric conversion film (hereinafter simply referred to as photoelectric conversion film 42), a transparent electrode 43, a lower electrode 44, and a transfer transistor TRG is regarded as one unit. In Fig. 6B, these three units share a floating diffusion FD, a reset transistor RST, and an amplification transistor AMP. In the circuit configuration of Fig. 5B, VC is a power supply voltage connected to the transparent electrode 43, and VR is a power supply voltage connected to the reset transistor RST.

[0080] The circuit configuration of Figure 6B differs from the pixel 100 of Figure 2 in that the photodiode PD of Figure 2 has been changed to a three-part configuration including a photoelectric conversion film 42, a transparent electrode 43, and a lower electrode 44, and in that the selection transistor SEL has been removed.

[0081] In this circuit configuration, the number of circuits such as floating diffusion FD shared by the photoelectric conversion film 42, the transparent electrode 43, and the lower electrode 44 is not limited to 3. Any number of photoelectric conversion films 42, etc. can share circuits such as floating diffusion FD.

[0082] Without being limited to the above example, the pixel 100 may have another configuration including the photodiode PD or the photoelectric conversion film 42. The pixel 100 may also include a memory MEM that temporarily stores the charge generated in the photodiode PD or the photoelectric conversion film 42.

[0083] FIG. 7 shows a modification of the pixel array in the first embodiment.

[0084] FIG. 7 shows an example of a computation in memory (CiM) having an SRAM (Static Random Access Memory) structure including six transistors M1 to M6. The SRAM structure includes, for example, two sets of inverters, each of which is composed of two transistors that hold a signal and is cross-connected, and two transistors that read and write to each inverter. In the example of FIG. 7, the SRAM structure includes an inverter composed of a transistor M1 and a transistor M3, and an inverter composed of a transistor M2 and a transistor M4, and each inverter is cross-connected. In addition, a transistor M5 and a transistor M6 that write and read are connected to each inverter. In addition, in this example, the computation in memory includes a transistor M7 connected to the source of the transistor M2 and the drain of the transistor M4. In addition, in this example, the computation in memory includes a transistor M8 connected to the source of the transistor M1 and the drain of the transistor M3.

[0085] For example, in a deep neural network, when performing a product-sum operation of signals, a memory may be used as a mechanism for holding the above-mentioned signals. The memory is composed of elements for holding signals, such as transistors and capacitors. In addition to the SRAM structure illustrated in FIG. 7, a circuit including an array structure of a memory such as a resistive random access memory (ReRAM), also called a resistive memory, a ferroelectric random access memory (FeRAM), also called a ferroelectric memory, or a magnetoresistive random access memory (MRAM), also called a magnetoresistive memory, may be connected to the input side of the weighting circuit 19. A structure in which memories are arranged in an array is called a memory array. This memory array may be a one-dimensional structure instead of a two-dimensional structure. A capacitive ladder circuit may be connected to the input side of the weighting circuit 19.

[0086] A mechanism for holding a signal, such as the pixel 100 including the photodiode PD described above and a memory included in a memory array, is called a cell. An array structure of cells, such as the pixel array unit 20 described above and a memory array, is called a cell array.

[0087] The cell array and the weighted addition circuit 19 may be mounted on different substrates. For example, the cell array may be mounted on a first substrate, and the weighted addition circuit 19 and the AD conversion unit 13 may be mounted on a second substrate. In addition to these configurations, a DRAM may be mounted on a third substrate. These substrates are connected by, for example, TSV, Cu-Cu connections, or microbumps.

[0088] According to this embodiment, weighted addition circuit 19 can suppress an increase in the number of wirings by sharing read wiring 130 among capacitors 132. Even if the number of signals to be added increases, weighted addition circuit 19 sequentially samples and holds the signals and performs weighted addition, so that it is possible to eliminate the trade-off between the number of weighted additions and the number of wirings.

[0089] Furthermore, according to this embodiment, the weighted addition circuit 19 inputs signals from the cell array via common vertical signal lines VSL. In this embodiment, the ratio of the number of vertical signal lines VSL to the number of weighted addition circuits 19 is 1:1. This allows the number of circuit vertical signal lines VSL to be reduced.

[0090] Furthermore, according to this embodiment, the AD conversion section 13 receives separate inputs of a reset signal and a data signal from the weighting addition circuit 19, thereby enabling the CDS operation to be realized in addition to the weighting addition.

[0091] Furthermore, according to this embodiment, the weighted addition circuit 19 can perform weighted addition not only by connecting to the pixel array section 20 but also to a cell array such as a memory array. This allows the weighted addition circuit 19 to realize weighted addition in a mechanism for holding signals other than the pixels 100.

[0092] Furthermore, according to this embodiment, by connecting a cell array having a pixel-sharing circuit configuration to the input side, the weighted addition circuit 19 can realize weighting in addition to the weighting in the cell array. This allows the weighted addition circuit 19 to supplement the weighting when the weighting in the cell array is insufficient. Second embodiment

[0093] FIG. 8 is a circuit diagram of a weighting addition circuit in the second embodiment.

[0094] In the weighted addition circuit 19, when the capacitances of the capacitances 132 for sample and hold are different, the time until the accumulation of charge in the capacitance 132 is completed may change. For example, when the capacitance 132 has a small capacitance, the time until the accumulation of charge is completed is short, and when the capacitance 132 has a large capacitance, the time until the accumulation of charge is completed is long. In this embodiment, the time until the accumulation of charge is completed is also called the settling time. In particular, in addition to the charge share used in this embodiment, in the case of other capacitive load readouts (dynamic source follower, current addition, etc.), it is more preferable to suppress the fluctuation of the capacitance value.

[0095] The weighted addition circuit 19 in this embodiment includes a buffer circuit 142 between the vertical signal line VSL and the weighted addition circuit 19, and provides an input delay time for a signal input to the weighted addition circuit 19 according to the size of the capacitance 132. The buffer circuit 142 may be, for example, a source follower circuit or a unity gain buffer circuit.

[0096] For example, when sampling and holding in C0 having a small capacitance 132, the buffer circuit 142 provides a delay time in the input of the signal to the weighted addition circuit 19. When sampling and holding in C3 having a large capacitance, the buffer circuit 142 does not provide a delay in the input to the weighted addition circuit 19.

[0097] In this embodiment, similarly to the first embodiment, not only the pixel array section 20 but also a cell array such as a memory array may be connected to the input side of the weighted addition circuit 19.

[0098] According to this embodiment, the weighted addition circuit 19 can provide a delay time when a signal is input to the weighted addition circuit 19 by the buffer circuit 142, and can suppress the difference in the charge accumulation time due to differences in the magnitude of the capacitance of the capacitor 132.

[0099] Furthermore, according to this embodiment, the weighted addition circuit 19 can perform weighted addition not only by connecting to the pixel array section 20 but also to a cell array such as a memory array. This allows the weighted addition circuit 19 to suppress deviations in charge accumulation time due to differences in the magnitude of the capacitance of the capacitor 132 for input from a mechanism that holds a signal.

[0100] Third embodiment FIG. 9 is a circuit diagram of a weighting addition circuit in the third embodiment.

[0101] The weighted addition circuit 19 in this embodiment includes, in addition to the above-mentioned capacitance 132, a dummy capacitance 134 and a third switch 135. The third switch 135 connects or disconnects the readout wiring 130 and the dummy capacitance 134.

[0102] The weighted addition circuit 19 stores charge in the dummy capacitance 134 based on the difference in capacitance between the first capacitance and the second capacitance. When sampling and holding in the capacitances 132 having different capacitances, the weighted addition circuit 19 connects the dummy capacitance 134 to the readout wiring 130 and adjusts the difference in capacitance so that the total value of the capacitances seen from the input side is constant. For example, when sampling and holding in the first capacitance having a large capacitance, the weighted addition circuit 19 does not store charge in the dummy capacitance 132, but when sampling and holding in the second capacitance having a small capacitance, the weighted addition circuit 19 also accumulates charge in the dummy capacitance 134. The charge stored in the dummy capacitance 134 is reset without being added.

[0103] 9, the weighted addition circuit 19 includes three dummy capacitances 134 in addition to the four capacitances 132. Each dummy capacitance 134 is connected to the readout wiring 130 via a third switch 135. The capacitances of the four capacitances 132 are C3, C2, C1, and C0 from the input side, respectively. The capacitances of the three dummy capacitances 134 are DC2, DC1, and DC0 from the input side, respectively.

[0104] Also, it is assumed that Equation (6) holds for each capacitance 132 and the dummy capacitance 134. C3 = C2 + DC1 + DC0 = C1 + DC2 + DC0 = C0 + DC2 + DC1 (7)

[0105] Equation (7) is an example, and the value and number of the capacitance of the dummy capacitance 134 connected together with the capacitance 132 depend on the values of the capacitances C3 to C0. This combination is a design parameter. Also, in the weighted addition circuit 19, the number of capacitances 132 and the number of dummy capacitances 134 do not necessarily have to match, and the capacitance values of the respective dummy capacitances 134 may be the same value or different values.

[0106] Since the dummy capacitance 134 may always be floating, for example, an SW for fixing to a predetermined voltage such as GND or VDD may be provided between the third switch 135 and the dummy capacitance 134. Hereinafter, for simplicity of explanation, it will be described as if these configurations do not exist.

[0107] FIG. 10 is a timing chart of the weighted addition circuit in the third embodiment.

[0108] The timing chart of FIG. 10 shows an example in which a plurality of signals (signal voltages V0 to V3) read from the pixel array unit 20 via the vertical signal lines VSL are sample-held by the respective capacitances 132 having capacitances of C0, C1, C2, and C3 (C0 < C1 < C2 < C3). Also, the timing chart of FIG. 10 shows an example in which weighted addition is performed on the readout wiring 130 after each capacitance 132 has sample-held.

[0109] Also, in the timing chart of FIG. 10, when sample-holding the capacitances 132, the weighted addition circuit 19 opens and closes the third switch 135 so that the respective capacitances 132 are the same, and charges are also accumulated in the respective dummy capacitances 134 having capacitances of DC0, DC1, and DC2 (DC0 < DC1 < DC2).

[0110] In this example, the second switches 133 connected to the capacitances 132 having capacitances of C0, C1, C2, and C3 are represented as SW0, SW1, SW2, and SW3, respectively. The third switches 135 connected to the dummy capacitances 134 having capacitances of DC0, DC1, and DC2 are represented as DSW0, DSW1, and DSW2, respectively. The capacitance relation between the capacitances 132 and the dummy capacitances 134 is represented as Equation (7).

[0111] In this embodiment, the weighted addition circuit 19 shows an example in which the second switches 133 are turned on in the order of SW0 to SW3 and sampled and held, but for example, the second switches 133 may be turned on in the order of SW3 to SW0, or may be turned on in another order.

[0112] First, the weighted addition circuit 19 turns on the first switch 131, connects the vertical signal line VSL to the readout wiring 130, and receives a signal input from the pixel array unit 20. At this time, the voltage of SH becomes Hi, and this voltage state is maintained until the weighted addition is completed.

[0113] Next, the weighted addition circuit 19 turns on SW0 of the second switch 133, and connects the readout wiring 130 to C0 of the capacitance 132. C0 accepts an input signal of signal voltage V0 and samples and holds it. At this time, the voltage state of SW0 becomes Hi, and the voltage state is maintained until charge is stored in C0. In addition, the weighted addition circuit 19 turns on DSW1 and DSW2 of the third switch 135, and connects the readout wiring 130 to DC1 and DC2 of the dummy capacitance 134. In addition, the charge Q0 stored in C0 is expressed as equation (1).

[0114] Next, the weighted addition circuit 19 turns on SW1 of the second switch 133, and connects the readout wiring 130 to C1 of the capacitance 132. C1 accepts an input signal of signal voltage V1 and samples and holds it. At this time, the voltage state of SW1 becomes Hi, and the voltage state is maintained until charge is stored in C1. In addition, the weighted addition circuit 19 turns on DSW0 and DSW2 of the third switch 135, and connects the readout wiring 130 to DC0 and DC2 of the dummy capacitance 134. In addition, the charge Q1 stored in C1 is expressed as equation (2).

[0115] Next, the weighted addition circuit 19 turns on SW2 of the second switch 133, and connects the readout wiring 130 to C2 of the capacitance 132. C2 accepts the input of a signal of the signal voltage V2 and samples and holds it. At this time, the voltage state of SW2 becomes Hi, and the voltage state is maintained until a charge is stored in C2. In addition, the weighted addition circuit 19 turns on DSW0 and DSW1 of the third switch 135, and connects the readout wiring 130 to DC0 and DC1 of the dummy capacitance 134. In addition, the charge Q2 stored in C2 is expressed as equation (3).

[0116] Next, the weighted addition circuit 19 turns on SW3 of the second switch 133, and connects the readout wiring 130 to C3 of the capacitor 132. C3 accepts the input of a signal with a signal voltage V3 and samples and holds it. At this time, the voltage state of SW3 becomes Hi, and the voltage state is maintained until charge is stored in C3. The charge Q3 stored in C3 is expressed by equation (4). The voltage of the readout wiring 130 when charge is stored in C3 becomes V3.

[0117] After the sample-and-hold operation is completed in each capacitance 132, the weighted addition circuit 19 turns on all of the second switches 133 and performs addition in the readout wiring 130. The signal voltage V of the readout wiring 130 at this time is expressed by equation (5). Note that the charge stored in the dummy capacitance 134 is reset without being AD converted.

[0118] As described above, the signal that has been subjected to the weighted addition is input to the AD conversion unit 13. Furthermore, by making the capacitance ratio of C3 to C0 a power of 2 and adding up the respective signals, it is possible to perform a bit-serial operation.

[0119] In this example, the weighting addition circuit 19 turns on SW0, SW1, SW2, and SW3 in this order, but this order is not limited, and the switches may be turned on in any order.

[0120] In this embodiment, similarly to the first embodiment, not only the pixel array section 20 but also a cell array such as a memory array may be connected to the input side of the weighted addition circuit 19.

[0121] According to this embodiment, the weighted addition circuit 19 includes a dummy capacitance 134 in addition to the capacitance 132. Furthermore, when performing sample-and-hold in each capacitance 132, the weighted addition circuit 19 turns on a third switch 135 to store charge in the dummy capacitance 134 as well. This allows the weighted addition circuit 19 to virtually adjust all capacitances 132 to be constant when performing sample-and-hold, thereby suppressing deviations in charge accumulation time due to differences in the magnitude of the electrostatic capacitance of the capacitance 132.

[0122] Furthermore, according to this embodiment, the weighted addition circuit 19 can perform weighted addition not only by connecting to the pixel array section 20 but also to a cell array such as a memory array. This allows the weighted addition circuit 19 to suppress deviations in charge accumulation time due to differences in the size of the capacitance 132 for inputs from a mechanism that holds signals.

[0123] (Fourth embodiment) FIG. 11 is a circuit diagram of a weighting addition circuit and the like in the fourth embodiment.

[0124] In the first to third embodiments, an example has been described in which the weighting addition circuit 19 performs weighting addition of a positive value. For example, when performing a convolution operation, the weighting addition circuit 19 may perform weighting addition of a negative value.

[0125] In this embodiment, the weighted addition circuit 19 includes a first weighted addition circuit 19' that performs weighted addition of positive values, and a second weighted addition circuit 19'' that performs weighted addition of negative values. In this configuration, the first weighted addition circuit 19' and the second weighted addition circuit 19'' input signals from the vertical signal line VSL by switching the respective first switches 131, and perform weighted addition for positive values ​​and negative values. In this example, the first switch 131 of the first weighted addition circuit 19' is represented as SH, and the first switch 131 of the second weighted addition circuit 19'' is represented as nSH. The configurations of these weighted addition circuits 19' and 19'' are similar to those in FIG. 3.

[0126] In other words, by turning SH on and nSH off, the first weighting addition circuit 19' can perform weighting addition of positive values, and by turning SH off and nSH on, the second weighting addition circuit 19'' can perform weighting addition of negative values.

[0127] The weighted addition circuit 19 may input the positive weighted addition result and the negative weighted addition result separately to the AD conversion unit 13, perform AD conversion, and then calculate the positive and negative values ​​by taking the difference between these values ​​using a subtraction circuit (not shown) or the like. Alternatively, after the weighted addition circuit 19 takes the difference between the results of the first weighted addition circuit 19' and the second weighted addition circuit 19'' using a subtraction circuit or the like, the AD conversion unit 13 may AD convert the difference signal.

[0128] Furthermore, the weighted addition circuit 19 may reset the circuits that do not perform sample and hold to an initial voltage before weighted addition. Furthermore, the weighted addition circuit 19 may turn on the second switches 133 associated with all the capacitors 132 during weighted addition, or may turn on only the second switches 133 associated with the capacitors 132 that have performed sample and hold. Since only the weighting coefficient changes depending on the on and off states of the second switches 133, the open / closed state of the second switches 133 may be designed arbitrarily.

[0129] FIG. 12 is a timing chart of the weighting addition circuit and the like in the fourth embodiment.

[0130] 12 shows an example in which signals are sequentially input from four pixels 100 and sample-hold and weighted addition are performed. In this example, two of the four pixels 100 input signals to a first weighted addition circuit 19' in order to perform weighted addition of positive values. The remaining two pixels 100 input signals to a second weighted addition circuit 19'' in order to perform weighted addition of negative values.

[0131] 5, this example shows an example in which the AD conversion unit 13 performs a CDS operation before performing weighted addition. Also, this example shows an example in which sample-hold is not performed on C2, C3, nC0, and nC1.

[0132] Specifically, using the timing chart of Fig. 12, the pixel 100 first initializes the floating diffusion FD by the reset transistor RST0 to read out the P-phase level, turns on the selection transistor SEL0, and transmits a reset level signal to the weighted addition circuit 19. The weighted addition circuit 19 samples and holds the reset level signal by C0, which is the corresponding capacitance 132. The weighted addition circuit 19 repeats this operation with C0, C1, nC2, and nC3, and then performs weighting and addition on the read wiring 130. The added signal is input to the AD conversion unit 13 as a reset signal, and AD conversion is performed.

[0133] Next, the pixel 100 turns on the transfer transistor TRG3 as a readout of the D-phase level, and further turns on the selection transistor SEL to transmit the charge accumulated in the photodiode PD to the weighted addition circuit 19. The weighted addition circuit 19 samples and holds the signal with C0, which is the corresponding capacitance 132. The weighted addition circuit 19 repeats this operation with C0, C1, nC2, and nC3 to sample and hold the signal, and then performs weighted addition with the readout wiring 130. In this example, positive addition is performed for C0 and C1, and negative addition is performed for nC2 and nC3. The weighted addition results of the positive value and the weighted addition results of the negative value are input to the AD conversion unit 13 as data signals and are AD converted. In this example, after AD conversion, a subtraction circuit takes the difference between these values ​​to perform a positive value calculation and a negative value calculation. In addition, the subtraction circuit calculates the difference between the difference signal of this data signal and the reset signal to remove fixed pattern noise.

[0134] In this embodiment, the weighting addition circuit 19 may be connected not only to the pixel array section 20 but also to a cell array such as a memory array to perform weighting addition.

[0135] According to the present embodiment, the weighting addition circuit 19 includes a second weighting addition circuit 19'' in addition to a first weighting addition circuit 19'. This enables the weighting addition circuit 19 to perform weighting addition of negative values ​​in addition to weighting addition of positive values.

[0136] Furthermore, according to this embodiment, the AD conversion unit 13 can realize CDS operation in addition to weighted addition by receiving separate inputs of a reset signal and a data signal from the first weighted addition circuit 19' and the second weighted addition circuit 19''.

[0137] Furthermore, according to this embodiment, the weighted addition circuit 19 can perform weighted addition not only by connecting to the pixel array section 20 but also by connecting to a cell array such as a memory array. This makes it possible to realize weighted addition not only in the pixel 100 but also in a mechanism for holding a signal.

[0138] Furthermore, according to this embodiment, the first weighted addition circuit 19' and the second weighted addition circuit 19'' receive signals from the cell array via a common vertical signal line VSL. In this embodiment, the ratio of the number of vertical signal lines VSL to the number of weighted addition circuits 19 is 1:2. This makes it possible to reduce the number of circuit vertical signal lines VSL.

[0139] Fifth embodiment FIG. 13 is a circuit diagram of a weighting addition circuit and the like in the fifth embodiment.

[0140] In this embodiment, each pixel 100 includes a first selection transistor SEL' that inputs a signal to a first weighting addition circuit 19' via a first vertical signal line VSL'. Also, each pixel 100 includes a second selection transistor nSEL that inputs a signal to a second weighting addition circuit 19'' via a second vertical signal line nVSL.

[0141] In this embodiment, for the sake of explanation, an example in which the pixel 100 includes a first selection transistor SEL' and a second selection transistor nSEL is taken up. However, the present invention is not limited to this example, and any configuration may be used as long as the cell of the memory or the like described above includes the first selection transistor SEL' and the second selection transistor nSEL.

[0142] When performing weighted addition of a positive value, the pixel 100 turns on the first selection transistor SEL' and turns off the second selection transistor nSEL to input a signal to the first weighted addition circuit 19'. When performing weighted addition of a negative value, the pixel 100 turns on the second selection transistor nSEL and turns off the first selection transistor SEL' to input a signal to the second weighted addition circuit 19''. In this way, the pixel 100 switches the first selection transistor SEL' and the second selection transistor nSEL to input a signal to the first weighted addition circuit 19' or the second weighted addition circuit 19''.

[0143] The first weighting addition circuit 19' sequentially receives and samples and holds signals from the pixels 100 whose first selection transistors SEL' are on. After the sampling and holding is completed for all signals, the first weighting addition circuit 19' performs weighting addition with a positive value.

[0144] The second weighted addition circuit 19'' sequentially receives and samples and holds signals from the pixels 100 whose second selection transistors nSEL are on. After the sampling and holding is completed for all signals, the second weighted addition circuit 19'' performs weighted addition of negative values.

[0145] Weighted addition circuit 19 may input the positive weighted addition result and the negative weighted addition result separately to AD conversion unit 13, and after AD conversion, may take the difference between these values ​​using a subtraction circuit (not shown) or the like to perform positive value calculation and negative value calculation. Alternatively, weighted addition circuit 19 may include a subtraction circuit or the like, and after taking the difference between the first weighted addition circuit 19' and the second weighted addition circuit 19'', AD conversion unit 13 may AD convert the difference signal.

[0146] According to this embodiment, the cell includes a first selection transistor SEL' and a second selection transistor nSEL. The cell can input a signal to the first weighting addition circuit 19' or the second weighting addition circuit 19'' by switching these transistors. This allows the weighting addition circuit 19 to perform weighting addition of negative values ​​in addition to weighting addition of positive values.

[0147] Furthermore, in the present embodiment, based on the opening and closing of the first selection transistor SEL' and the second selection transistor nSEL, the first weighted addition circuit 19' can input a positive value signal, and at the same time, the second weighted addition circuit 19'' can input a negative value signal.

[0148] Furthermore, according to this embodiment, the first weighted addition circuit 19' and the second weighted addition circuit 19'' receive signals from the cell array via the first vertical signal line VSL' and the second vertical signal line nVSL, respectively. In this embodiment, the ratio of the number of vertical signal lines to the number of weighted addition circuits 19 is 1:1. This makes it possible to reduce the number of circuit vertical signal lines VSL.

[0149] Sixth embodiment FIG. 14 is a circuit diagram of a weighting addition circuit and the like in the sixth embodiment.

[0150] The weighted addition circuit in this embodiment includes a fourth switch 136 that is connected to the terminal of each capacitor 132 opposite to the terminal on the readout line 130 side and that can switch the terminal voltage between a plurality of values.

[0151] 14A shows a configuration in which the fourth switch 136 can set the terminal voltage on the opposite side to the readout line 130 to two values ​​for each capacitor 132. For example, the two values ​​can be set to values ​​such as GND or VDD.

[0152] 14A also illustrates a comparator 33 included in the AD conversion unit 13. A data signal is input to one input of the comparator 33, and a reference voltage Ref is input to the other input.

[0153] 14B shows a configuration in which the fourth switch 136 can set the terminal voltage on the opposite side to the readout line 130 to three values ​​for each capacitor 132. In this example, the three values ​​are represented as Ref1, Ref2, and Ref3.

[0154] The weighted addition circuit 19 samples and holds a signal in each capacitor 132, and fixes the fourth switch 136 to a predetermined value until weighted addition is performed. In FIG. 4A, for example, the fourth switch 136 may fix the voltage value to a GND voltage. After weighted addition, the weighted addition circuit 19 inputs a data signal to the AD conversion unit 13. During AD conversion, the weighted addition circuit 19 successively switches the fourth switch 136 to generate a DAC signal, and inputs the DAC signal to the AD conversion unit 13. The AD conversion unit 13 can realize successive approximation type (also called SAR (Successive Approximation Register) type) AD conversion by performing a successive approximation operation between the data signal and the DAC signal.

[0155] FIG. 15 is a circuit diagram of a modification of the weighting addition circuit etc. in the sixth embodiment.

[0156] The weighted addition circuit 19 shown in FIG. 15 includes a first weighted addition circuit 19' that performs weighted addition of positive values, and a second weighted addition circuit 19'' that performs weighted addition of negative values. Furthermore, the first weighted addition circuit 19' and the second weighted addition circuit 19'' each include a fourth switch 136 that enables the terminal voltage on the opposite side to the readout wiring 130 for each capacitor 132 to be switched between multiple values. FIG. 15 shows a configuration in which the fourth switch 136 can set the terminal voltage on the opposite side to the readout wiring 130 for each capacitor 132 to two values.

[0157] The first weighting addition circuit 19' and the second weighting addition circuit 19'' in this modification sample and hold the signal in each capacitance 132, and fix the fourth switch 136 to a predetermined value until weighting addition is performed. For example, the fourth switch 136 may fix the voltage value to the GND voltage. The first weighting addition circuit 19' and the second weighting addition circuit 19'' input the data signal to the AD conversion unit 13. The first weighting addition circuit 19' and the second weighting addition circuit 19'' input the DAC signal to the AD conversion unit 13 by successively switching the fourth switch 136 during AD conversion. The AD conversion unit 13 can realize successive approximation type AD conversion by successively comparing the data signal and the DAC signal even for the weighting addition result of a positive value and the weighting addition result of a negative value.

[0158] According to the present embodiment, the weighted addition circuit 19 includes a fourth switch 136 that enables the terminal voltage on the opposite side to the readout wiring 130 to be switched between multiple values ​​for each capacitor 132. This allows the weighted addition circuit 19 to achieve the function of a successive approximation type AD converter without adding a separate capacitor 132.

[0159] <<Example of vehicle control system configuration>> FIG. 16 is a block diagram showing an example of the configuration of a vehicle control system 11, which is an example of a mobility device control system to which the present technology is applied.

[0160] The vehicle control system 11 is provided in the vehicle 1 and performs processing related to driving assistance and automatic driving of the vehicle 1.

[0161] The vehicle control system 11 includes a vehicle control ECU (Electronic Control Unit) 21, a communication unit 22, a map information storage unit 23, a position information acquisition unit 24, an external recognition sensor 25, an in-vehicle sensor 26, a vehicle sensor 27, a memory unit 28, a driving assistance / automatic driving control unit 29, a DMS (Driver Monitoring System) 30, an HMI (Human Machine Interface) 31, and a vehicle control unit 32.

[0162] The vehicle control ECU 21, the communication unit 22, the map information storage unit 23, the position information acquisition unit 24, the external recognition sensor 25, the in-vehicle sensor 26, the vehicle sensor 27, the storage unit 28, the driving support / automatic driving control unit 29, the driver monitoring system (DMS) 30, the human machine interface (HMI) 31, and the vehicle control unit 32 are connected to each other so as to be able to communicate with each other via a communication network 41. The communication network 41 is configured by an in-vehicle communication network or a bus conforming to a digital two-way communication standard such as CAN (Controller Area Network), LIN (Local Interconnect Network), LAN (Local Area Network), FlexRay (registered trademark), or Ethernet (registered trademark). The communication network 41 may be used differently depending on the type of data to be transmitted. For example, CAN may be applied to data related to vehicle control, and Ethernet may be applied to large-volume data. In addition, each part of the vehicle control system 11 may be directly connected to each other without going through the communication network 41, using wireless communication intended for communication over relatively short distances, such as near field communication (NFC) or Bluetooth (registered trademark).

[0163] In the following description, when each unit of the vehicle control system 11 communicates with each other via the communication network 41, the description of the communication network 41 will be omitted. For example, when the vehicle control ECU 21 and the communication unit 22 communicate with each other via the communication network 41, it will be simply described that the vehicle control ECU 21 and the communication unit 22 communicate with each other.

[0164] The vehicle control ECU 21 is configured with various processors such as a CPU (Central Processing Unit), an MPU (Micro Processing Unit), etc. The vehicle control ECU 21 controls the entire or part of the functions of the vehicle control system 11.

[0165] The communication unit 22 communicates with various devices inside and outside the vehicle, other vehicles, servers, base stations, etc., and transmits and receives various data. At this time, the communication unit 22 can communicate using a plurality of communication methods.

[0166] The following provides an overview of communication with the outside of the vehicle that can be performed by the communication unit 22. The communication unit 22 communicates with a server (hereinafter referred to as an external server) on an external network via a base station or an access point using a wireless communication method such as 5G (fifth generation mobile communication system), LTE (Long Term Evolution), or DSRC (Dedicated Short Range Communications). The external network with which the communication unit 22 communicates is, for example, the Internet, a cloud network, or a network specific to a carrier. The communication method that the communication unit 22 uses with the external network is not particularly limited as long as it is a wireless communication method that allows digital two-way communication at a communication speed equal to or higher than a predetermined distance.

[0167] Also, for example, the communication unit 22 can communicate with a terminal existing near the vehicle by using a P2P (Peer To Peer) technique. The terminal existing near the vehicle can be, for example, a terminal attached to a moving body moving at a relatively low speed such as a pedestrian or a bicycle, a terminal installed at a fixed position in a store, or an MTC (Machine Type Communication) terminal. Furthermore, the communication unit 22 can also perform V2X communication. The V2X communication refers to communication between the vehicle and others, such as vehicle-to-vehicle communication with another vehicle, vehicle-to-infrastructure communication with a roadside device, vehicle-to-home communication with a home, and vehicle-to-pedestrian communication with a terminal carried by a pedestrian.

[0168] The communication unit 22 can receive, for example, a program for updating software that controls the operation of the vehicle control system 11 from the outside (Over The Air). The communication unit 22 can further receive map information, traffic information, information about the surroundings of the vehicle 1, and the like from the outside. For example, the communication unit 22 can transmit information about the vehicle 1 and information about the surroundings of the vehicle 1 to the outside. Information about the vehicle 1 that the communication unit 22 transmits to the outside includes, for example, data indicating the state of the vehicle 1, a recognition result by the recognition unit 73, and the like. For example, the communication unit 22 also performs communication corresponding to a vehicle emergency call system such as e-Call.

[0169] For example, the communication unit 22 receives electromagnetic waves transmitted by a road traffic information and communication system (VICS (Vehicle Information and Communication System) (registered trademark)) such as a radio beacon, an optical beacon, or FM multiplex broadcasting.

[0170] The communication with the inside of the vehicle that can be performed by the communication unit 22 will be briefly described. The communication unit 22 can communicate with each device in the vehicle using, for example, wireless communication. The communication unit 22 can wirelessly communicate with each device in the vehicle using a communication method that allows digital two-way communication at a communication speed of a predetermined speed or higher by wireless communication, such as wireless LAN, Bluetooth, NFC, or WUSB (Wireless USB). Not limited to this, the communication unit 22 can also communicate with each device in the vehicle using wired communication. For example, the communication unit 22 can communicate with each device in the vehicle using wired communication via a cable connected to a connection terminal not shown. The communication unit 22 can communicate with each device in the vehicle using a communication method that allows digital two-way communication at a communication speed of a predetermined speed or higher by wired communication, such as USB (Universal Serial Bus), HDMI (High-Definition Multimedia Interface) (registered trademark), or MHL (Mobile High-definition Link).

[0171] Here, the in-vehicle device refers to, for example, a device in the vehicle that is not connected to the communication network 41. Assumed in-vehicle devices include, for example, a mobile device or a wearable device carried by a passenger such as a driver, and an information device brought into the vehicle and temporarily installed therein.

[0172] The map information storage unit 23 stores one or both of a map acquired from an external source and a map created by the vehicle 1. For example, the map information storage unit 23 stores a three-dimensional high-precision map, a global map that has lower precision than a high-precision map and covers a wide area, and the like.

[0173] The high-precision map is, for example, a dynamic map, a point cloud map, a vector map, etc. The dynamic map is, for example, a map consisting of four layers of dynamic information, semi-dynamic information, semi-static information, and static information, and is provided to the vehicle 1 from an external server or the like. The point cloud map is a map composed of a point cloud (point cloud data). The vector map is, for example, a map in which traffic information such as the positions of lanes and traffic lights is associated with the point cloud map, and is adapted to an ADAS (Advanced Driver Assistance System) or AD (Autonomous Driving).

[0174] The point cloud map and the vector map may be provided, for example, from an external server or the like, or may be created by the vehicle 1 as a map for matching with a local map described later based on sensing results from the camera 51, the radar 52, the LiDAR 53, etc., and stored in the map information storage unit 23. In addition, when a high-precision map is provided from an external server or the like, map data of, for example, an area of ​​several hundred meters square regarding the planned route along which the vehicle 1 will travel is acquired from the external server or the like in order to reduce communication capacity.

[0175] The position information acquisition unit 24 receives GNSS signals from Global Navigation Satellite System (GNSS) satellites and acquires position information of the vehicle 1. The acquired position information is supplied to the driving assistance / automated driving control unit 29. Note that the method used by the position information acquisition unit 24 is not limited to a method using GNSS signals, and for example, the position information may be acquired using a beacon.

[0176] The external recognition sensor 25 includes various sensors used to recognize the situation outside the vehicle 1, and supplies sensor data from each sensor to each unit of the vehicle control system 11. The type and number of sensors included in the external recognition sensor 25 are arbitrary.

[0177] For example, the external recognition sensor 25 includes a camera 51, a radar 52, a LiDAR (Light Detection and Ranging, Laser Imaging Detection and Ranging) 53, and an ultrasonic sensor 54. Not limited to this, the external recognition sensor 25 may include one or more types of sensors among the camera 51, the radar 52, the LiDAR 53, and the ultrasonic sensor 54. The number of the cameras 51, the radar 52, the LiDAR 53, and the ultrasonic sensor 54 is not particularly limited as long as it is a number that can be practically installed in the vehicle 1. In addition, the type of the sensor included in the external recognition sensor 25 is not limited to this example, and the external recognition sensor 25 may include other types of sensors. Examples of sensing areas of each sensor included in the external recognition sensor 25 will be described later.

[0178] The imaging method of the camera 51 is not particularly limited. For example, cameras of various imaging methods, such as a ToF (Time Of Flight) camera, a stereo camera, a monocular camera, and an infrared camera, which are imaging methods capable of distance measurement, can be applied to the camera 51 as necessary. However, the camera 51 may be one simply for acquiring a photographed image, regardless of distance measurement.

[0179] Furthermore, for example, the external recognition sensor 25 may include an environmental sensor for detecting the environment for the vehicle 1. The environmental sensor is a sensor for detecting the environment such as the weather, climate, and brightness, and may include various sensors such as a raindrop sensor, a fog sensor, a sunlight sensor, a snow sensor, and an illuminance sensor.

[0180] Furthermore, for example, the external recognition sensor 25 includes a microphone used to detect sounds around the vehicle 1 and the positions of sound sources.

[0181] The in-vehicle sensor 26 includes various sensors for detecting information inside the vehicle, and supplies sensor data from each sensor to each unit of the vehicle control system 11. The types and number of the various sensors included in the in-vehicle sensor 26 are not particularly limited as long as they are of the types and number that can be realistically installed in the vehicle 1.

[0182] For example, the in-vehicle sensor 26 may include one or more types of sensors selected from a camera, a radar, a seating sensor, a steering wheel sensor, a microphone, and a biosensor. The camera included in the in-vehicle sensor 26 may be a camera using various imaging methods capable of measuring distances, such as a ToF camera, a stereo camera, a monocular camera, and an infrared camera. The camera included in the in-vehicle sensor 26 may be a camera simply for acquiring captured images, regardless of distance measurement. The biosensor included in the in-vehicle sensor 26 is provided, for example, on a seat, a steering wheel, or the like, and detects various types of bioinformation of a passenger such as a driver.

[0183] The vehicle sensor 27 includes various sensors for detecting the state of the vehicle 1, and supplies sensor data from each sensor to each unit of the vehicle control system 11. The types and number of the various sensors included in the vehicle sensor 27 are not particularly limited as long as they are types and numbers that can be realistically installed in the vehicle 1.

[0184] For example, the vehicle sensor 27 includes a speed sensor, an acceleration sensor, an angular velocity sensor (gyro sensor), and an inertial measurement unit (IMU) that integrates these. For example, the vehicle sensor 27 includes a steering angle sensor that detects the steering angle of the steering wheel, a yaw rate sensor, an accelerator sensor that detects the amount of operation of an accelerator pedal, and a brake sensor that detects the amount of operation of a brake pedal. For example, the vehicle sensor 27 includes a rotation sensor that detects the number of rotations of the engine or the motor, an air pressure sensor that detects the air pressure of the tires, a slip ratio sensor that detects the slip ratio of the tires, and a wheel speed sensor that detects the rotation speed of the wheels. For example, the vehicle sensor 27 includes a battery sensor that detects the remaining charge and temperature of the battery, and an impact sensor that detects an impact from the outside.

[0185] The storage unit 28 includes at least one of a non-volatile storage medium and a volatile storage medium, and stores data and programs. The storage unit 28 is used, for example, as an EEPROM (Electrically Erasable Programmable Read Only Memory) and a RAM (Random Access Memory), and the storage medium may be a magnetic storage device such as a hard disk drive (HDD), a semiconductor storage device, an optical storage device, or a magneto-optical storage device. The storage unit 28 stores various programs and data used by each part of the vehicle control system 11. For example, the storage unit 28 includes an EDR (Event Data Recorder) and a DSSAD (Data Storage System for Automated Driving), and stores information on the vehicle 1 before and after an event such as an accident, and information acquired by the in-vehicle sensor 26.

[0186] The driving support / automatic driving control unit 29 controls driving support and automatic driving of the vehicle 1. For example, the driving support / automatic driving control unit 29 includes an analysis unit 61, an action planning unit 62, and an operation control unit 63.

[0187] The analysis unit 61 performs an analysis process of the vehicle 1 and the surrounding situation. The analysis unit 61 includes a self-position estimation unit 71, a sensor fusion unit 72, and a recognition unit 73.

[0188] The self-position estimation unit 71 estimates the self-position of the vehicle 1 based on the sensor data from the external recognition sensor 25 and the high-precision map stored in the map information storage unit 23. For example, the self-position estimation unit 71 generates a local map based on the sensor data from the external recognition sensor 25, and estimates the self-position of the vehicle 1 by matching the local map with the high-precision map. The position of the vehicle 1 is based on, for example, the center of the rear wheel pair axle.

[0189] The local map is, for example, a three-dimensional high-precision map or an occupancy grid map created using a technique such as SLAM (Simultaneous Localization and Mapping). The three-dimensional high-precision map is, for example, the above-mentioned point cloud map. The occupancy grid map is a map in which the three-dimensional or two-dimensional space around the vehicle 1 is divided into grids of a predetermined size, and the occupancy state of an object is indicated on a grid-by-grid basis. The occupancy state of an object is indicated, for example, by the presence or absence of an object and the probability of its existence. The local map is also used, for example, in detection processing and recognition processing of the external situation of the vehicle 1 by the recognition unit 73.

[0190] The self-position estimation unit 71 may estimate the self-position of the vehicle 1 based on the position information acquired by the position information acquisition unit 24 and the sensor data from the vehicle sensor 27.

[0191] The sensor fusion unit 72 performs a sensor fusion process to obtain new information by combining a plurality of different types of sensor data (for example, image data supplied from the camera 51 and sensor data supplied from the radar 52). Methods for combining different types of sensor data include integration, fusion, and association.

[0192] The recognition unit 73 executes a detection process for detecting the situation outside the vehicle 1 and a recognition process for recognizing the situation outside the vehicle 1.

[0193] For example, the recognition unit 73 performs detection processing and recognition processing of the situation outside the vehicle 1 based on information from the external recognition sensor 25, information from the self-position estimation unit 71, information from the sensor fusion unit 72, and the like.

[0194] Specifically, for example, the recognition unit 73 performs detection processing and recognition processing of objects around the vehicle 1. The object detection processing is, for example, processing to detect the presence or absence, size, shape, position, movement, etc. of an object. The object recognition processing is, for example, processing to recognize attributes such as the type of object, or to identify a specific object. However, the detection processing and the recognition processing are not necessarily clearly separated, and may overlap.

[0195] For example, the recognition unit 73 detects objects around the vehicle 1 by performing clustering to classify a point cloud based on sensor data from the radar 52, the LiDAR 53, or the like into clusters of points. This allows the presence or absence, size, shape, and position of objects around the vehicle 1 to be detected.

[0196] For example, the recognition unit 73 performs tracking to follow the movement of the clusters of the point cloud classified by clustering, thereby detecting the movement of the objects around the vehicle 1. As a result, the speed and traveling direction (movement vector) of the objects around the vehicle 1 are detected.

[0197] For example, the recognition unit 73 detects or recognizes vehicles, people, bicycles, obstacles, structures, roads, traffic lights, traffic signs, road markings, and the like, based on image data supplied from the camera 51. Furthermore, the recognition unit 73 may recognize the types of objects around the vehicle 1 by performing recognition processing such as semantic segmentation.

[0198] For example, the recognition unit 73 can perform recognition processing of traffic rules around the vehicle 1 based on the map stored in the map information storage unit 23, the estimation result of the self-position by the self-position estimation unit 71, and the recognition result of the objects around the vehicle 1 by the recognition unit 73. Through this processing, the recognition unit 73 can recognize the positions and states of traffic lights, the contents of traffic signs and road markings, the contents of traffic regulations, and lanes on which travel is possible.

[0199] For example, the recognition unit 73 can perform a recognition process of the environment around the vehicle 1. The surrounding environment to be recognized by the recognition unit 73 may include weather, temperature, humidity, brightness, and road surface conditions.

[0200] The behavior planning unit 62 creates a behavior plan for the vehicle 1. For example, the behavior planning unit 62 creates the behavior plan by performing route planning and route following processing.

[0201] Global path planning is a process for planning a rough route from the start to the goal. This route planning also includes a process for local path planning, which is called trajectory planning and allows the vehicle 1 to proceed safely and smoothly in the vicinity of the vehicle 1, taking into account the motion characteristics of the vehicle 1 on the planned route.

[0202] Path following is a process for planning an operation for safely and accurately traveling along a route planned by a route plan within a planned time. The action planning unit 62 can calculate a target speed and a target angular velocity of the vehicle 1, for example, based on the result of the path following process.

[0203] The operation control unit 63 controls the operation of the vehicle 1 in order to realize the action plan created by the action planning unit 62.

[0204] For example, the operation control unit 63 controls a steering control unit 81, a brake control unit 82, and a drive control unit 83 included in a vehicle control unit 32 described later, and performs acceleration / deceleration control and direction control so that the vehicle 1 proceeds along a trajectory calculated by a trajectory plan. For example, the operation control unit 63 performs cooperative control for the purpose of realizing ADAS functions such as collision avoidance or impact mitigation, following driving, vehicle speed maintenance driving, host vehicle collision warning, host vehicle lane departure warning, etc. For example, the operation control unit 63 performs cooperative control for the purpose of automatic driving that autonomously drives without the driver's operation.

[0205] The DMS 30 performs authentication processing of the driver and recognition processing of the driver's state based on the sensor data from the in-vehicle sensor 26 and input data input to the HMI 31 described later. Examples of the driver's state to be recognized include physical condition, alertness, concentration, fatigue, line of sight, drunkenness, driving operation, posture, and the like.

[0206] The DMS 30 may perform authentication processing of passengers other than the driver and recognition processing of the conditions of the passengers. For example, the DMS 30 may perform recognition processing of the conditions inside the vehicle based on sensor data from the in-vehicle sensor 26. Conditions inside the vehicle to be recognized include, for example, temperature, humidity, brightness, odor, and the like.

[0207] The HMI 31 inputs various data and instructions, and presents various data to the driver, etc.

[0208] The input of data by the HMI 31 will be briefly described. The HMI 31 includes an input device for a person to input data. The HMI 31 generates an input signal based on data, instructions, etc. input by the input device, and supplies the input signal to each part of the vehicle control system 11. The HMI 31 includes, as input devices, operators such as a touch panel, a button, a switch, and a lever. Without being limited to this, the HMI 31 may further include an input device that allows information to be input by a method other than manual operation, such as voice or gesture. Furthermore, the HMI 31 may use, as an input device, an externally connected device such as a remote control device using infrared rays or radio waves, or a mobile device or wearable device compatible with the operation of the vehicle control system 11.

[0209] The presentation of data by the HMI 31 will be briefly described. The HMI 31 generates visual information, auditory information, and tactile information for the passenger or the outside of the vehicle. The HMI 31 also performs output control to control the output, output contents, output timing, output method, etc. of each generated information. The HMI 31 generates and outputs, as visual information, information indicated by images or lights, such as an operation screen, a status display of the vehicle 1, a warning display, and a monitor image indicating the situation around the vehicle 1. The HMI 31 also generates and outputs, as auditory information, information indicated by sounds, such as voice guidance, warning sounds, and warning messages. The HMI 31 also generates and outputs, as tactile information, information given to the passenger's sense of touch by, for example, force, vibration, movement, etc.

[0210] Examples of the output device to which the HMI 31 outputs visual information include a display device that displays an image to present visual information, and a projector device that projects an image to present visual information. The display device may be a device that displays visual information within the field of vision of the passenger, such as a head-up display, a transmissive display, or a wearable device with an AR (Augmented Reality) function, in addition to a display device having a normal display. The HMI 31 may also use display devices provided in a navigation device, an instrument panel, a CMS (Camera Monitoring System), an electronic mirror, a lamp, and the like provided in the vehicle 1 as output devices that output visual information.

[0211] As an output device to which the HMI 31 outputs auditory information, for example, an audio speaker, a headphone, or an earphone can be applied.

[0212] For example, a haptic element using haptic technology can be applied as an output device to which the HMI 31 outputs haptic information. The haptic element is provided on a part of the vehicle 1 that an occupant touches, such as a steering wheel or a seat.

[0213] The vehicle control unit 32 controls each part of the vehicle 1. The vehicle control unit 32 includes a steering control unit 81, a brake control unit 82, a drive control unit 83, a body system control unit 84, a light control unit 85, and a horn control unit 86.

[0214] The steering control unit 81 detects and controls the state of the steering system of the vehicle 1. The steering system includes, for example, a steering mechanism including a steering wheel, an electric power steering, etc. The steering control unit 81 includes, for example, a steering ECU that controls the steering system, an actuator that drives the steering system, etc.

[0215] The brake control unit 82 detects and controls the state of the brake system of the vehicle 1. The brake system includes, for example, a brake mechanism including a brake pedal, an ABS (Antilock Brake System), a regenerative brake mechanism, etc. The brake control unit 82 includes, for example, a brake ECU that controls the brake system, an actuator that drives the brake system, etc.

[0216] The drive control unit 83 detects and controls the state of the drive system of the vehicle 1. The drive system includes, for example, an accelerator pedal, a drive force generating device for generating drive force such as an internal combustion engine or a drive motor, a drive force transmission mechanism for transmitting the drive force to the wheels, etc. The drive control unit 83 includes, for example, a drive ECU for controlling the drive system, an actuator for driving the drive system, etc.

[0217] The body system control unit 84 detects and controls the states of the body system systems of the vehicle 1. The body system systems include, for example, a keyless entry system, a smart key system, a power window device, a power seat, an air conditioning system, an airbag, a seat belt, a shift lever, etc. The body system control unit 84 includes, for example, a body system ECU that controls the body system systems, an actuator that drives the body system systems, etc.

[0218] The light control unit 85 detects and controls the states of various lights of the vehicle 1. Examples of lights to be controlled include headlights, backlights, fog lights, turn signals, brake lights, projections, and bumper displays. The light control unit 85 includes a light ECU that controls the lights, an actuator that drives the lights, and the like.

[0219] The horn control unit 86 detects and controls the state of the car horn of the vehicle 1. The horn control unit 86 includes, for example, a horn ECU that controls the car horn, an actuator that drives the car horn, and the like.

[0220] Fig. 17 is a diagram showing an example of a sensing area by the camera 51, the radar 52, the LiDAR 53, and the ultrasonic sensor 54 of the external recognition sensor 25 in Fig. 16. Note that Fig. 17 shows a schematic view of the vehicle 1 from above, with the left end side being the front end of the vehicle 1 and the right end side being the rear end of the vehicle 1.

[0221] Sensing area 101F and sensing area 101B show examples of sensing areas of the ultrasonic sensors 54. The sensing area 101F covers the periphery of the front end of the vehicle 1 by the multiple ultrasonic sensors 54. The sensing area 101B covers the periphery of the rear end of the vehicle 1 by the multiple ultrasonic sensors 54.

[0222] The sensing results in the sensing area 101F and the sensing area 101B are used for, for example, parking assistance for the vehicle 1.

[0223] Sensing area 102F to sensing area 102B show examples of sensing areas of a short-range or medium-range radar 52. Sensing area 102F covers a position farther in front of the vehicle 1 than sensing area 101F. Sensing area 102B covers a position farther in the rear of the vehicle 1 than sensing area 101B. Sensing area 102L covers the rear periphery of the left side of the vehicle 1. Sensing area 102R covers the rear periphery of the right side of the vehicle 1.

[0224] The sensing results in the sensing area 102F are used, for example, to detect vehicles, pedestrians, and the like that exist in front of the vehicle 1. The sensing results in the sensing area 102B are used, for example, for a collision prevention function behind the vehicle 1. The sensing results in the sensing area 102L and the sensing area 102R are used, for example, to detect objects in blind spots on the sides of the vehicle 1.

[0225] Sensing area 103F to sensing area 103B show examples of sensing areas by camera 51. Sensing area 103F covers a position farther in front of vehicle 1 than sensing area 102F. Sensing area 103B covers a position farther in the rear of vehicle 1 than sensing area 102B. Sensing area 103L covers the periphery of the left side of vehicle 1. Sensing area 103R covers the periphery of the right side of vehicle 1.

[0226] The sensing results in the sensing area 103F can be used for, for example, recognition of traffic lights and traffic signs, lane departure prevention support systems, and automatic headlight control systems. The sensing results in the sensing area 103B can be used for, for example, parking support and surround view systems. The sensing results in the sensing areas 103L and 103R can be used for, for example, surround view systems.

[0227] Sensing area 104 illustrates an example of the sensing area of ​​the LiDAR 53. Sensing area 104 covers a position farther forward than sensing area 103F in front of the vehicle 1. On the other hand, sensing area 104 has a narrower range in the left-right direction than sensing area 103F.

[0228] The sensing results in the sensing area 104 are used to detect objects such as surrounding vehicles, for example.

[0229] Sensing area 105 shows an example of the sensing area of ​​the long-range radar 52 . The sensing area 105 covers a position farther in front of the vehicle 1 than the sensing area 104. On the other hand, the sensing area 105 has a narrower range than the sensing area 104 in the left-right direction.

[0230] The sensing results in the sensing area 105 are used for, for example, ACC (Adaptive Cruise Control), emergency braking, collision avoidance, and the like.

[0231] The sensing areas of the cameras 51, the radar 52, the LiDAR 53, and the ultrasonic sensor 54 included in the external recognition sensor 25 may have various configurations other than those shown in FIG. 17. Specifically, the ultrasonic sensor 54 may also sense the sides of the vehicle 1, and the LiDAR 53 may sense the rear of the vehicle 1. The installation positions of the sensors are not limited to the above-mentioned examples. The number of sensors may be one or more.

[0232] The present disclosure has been described above by giving embodiments and their modifications, application examples, and application examples, but the present disclosure is not limited to the above-mentioned embodiments, etc., and various modifications are possible. Note that the effects described in this specification are merely examples. The effects of the present disclosure are not limited to the effects described in this specification. The present disclosure may have effects other than those described in this specification.

[0233] Furthermore, for example, the present disclosure can have the following configuration.

[0234] (1) a readout wiring for receiving an input of a plurality of signals read out to vertical signal lines of the solid-state imaging device; a first switch that connects or disconnects the vertical signal line and the readout wiring; A plurality of capacitances for sampling and holding the plurality of signals; a plurality of second switches for connecting or disconnecting the readout wiring and the capacitance; Weighted summation circuit.

[0235] (2) The solid-state imaging device includes: a cell array having a mechanism for holding the plurality of signals; the vertical signal lines for reading out the signals from the cell array; an AD conversion unit that performs AD conversion of the signal weighted and added by the weighting and adding circuit, the weighted addition circuit is connected to the cell array, the vertical signal line, and the AD conversion unit; The weighting addition circuit according to (1).

[0236] (3) The weighting addition circuit includes: Sequentially sampling and holding the plurality of signals sequentially read out to the vertical signal lines; After completing the sample-holding of the plurality of signals, a weighted sum is performed. The weighting addition circuit according to (1).

[0237] (4) The weighted addition circuit according to (2), wherein the cells included in the cell array are pixels including a photodiode or memories included in a memory array.

[0238] (5) The weighted addition circuit according to (2), further including a fourth switch that can switch a terminal voltage value at a terminal of the plurality of capacitors opposite to the terminal on the readout line side.

[0239] (6) The weighted addition circuit according to (5), wherein the AD conversion unit performs a successive comparison operation between a DAC signal generated by the weighted addition circuit based on switching of the fourth switch and the weighted-added signal.

[0240] (7) a cell array having a mechanism for holding a plurality of signals; vertical signal lines for reading out the signals from the cell array; a weighted addition circuit that samples and holds the plurality of signals read out to the vertical signal lines and performs weighted addition; an AD conversion unit that performs AD conversion of the signal weighted and added by the weighting and adding circuit, The weighting addition circuit includes: a readout wiring that receives an input of the plurality of signals read out to the vertical signal lines; a first switch that connects or disconnects the vertical signal line and the readout wiring; A plurality of capacitances for sampling and holding the plurality of signals; a plurality of second switches that connect or disconnect the readout wiring and the capacitor; Solid-state imaging device.

[0241] (8) The weighting addition circuit includes: Sequentially sampling and holding the plurality of signals sequentially read out to the vertical signal lines; After completing the sample-holding of the plurality of signals, a weighted sum is performed. A solid-state imaging device according to (7).

[0242] (9) The solid-state imaging device according to (7), wherein the cells included in the cell array are pixels including a photodiode or memories included in a memory array.

[0243] (10) The solid-state imaging device according to (7), wherein the plurality of capacitors include a first capacitor and a second capacitor having different capacitances.

[0244] (11) The solid-state imaging device according to (7), wherein a buffer circuit that provides a delay time for inputting the plurality of signals is connected between the vertical signal line and the weighted addition circuit.

[0245] (12) The weighting addition circuit includes: one or more dummy capacitances for adjusting a difference in capacitance when the first capacitance and the second capacitance sample and hold; and one or more third switches for connecting or disconnecting the readout wiring and the dummy capacitance. A solid-state imaging device according to (10).

[0246] (13) The solid-state imaging device according to (12), wherein the weighting addition circuit stores electric charge in one or more of the dummy capacitances based on a capacitance difference between the first capacitance and the second capacitance.

[0247] (14) a cell array having a mechanism for holding a plurality of signals; vertical signal lines for reading out the signals from the cell array; a first weighting addition circuit that samples and holds the plurality of signals read out to the vertical signal lines and performs weighting addition of positive values; a second weighting addition circuit that samples and holds the plurality of signals read out to the vertical signal lines and performs weighting addition of negative values; an AD conversion unit that performs AD conversion on the signal weighted and added by the first weighting and addition circuit and the signal weighted and added by the second weighting and addition circuit, Each of the first and second weighting and adding circuits comprises: a readout wiring that receives an input of the plurality of signals read out to the vertical signal lines; a first switch that connects or disconnects the vertical signal line and the readout wiring; A plurality of capacitances for sampling and holding the plurality of signals; a plurality of second switches that connect or disconnect the readout wiring and the capacitor; Solid-state imaging device.

[0248] (15) The solid-state imaging device according to (14), wherein the cells included in the cell array are pixels including a photodiode or memories included in a memory array.

[0249] (16) Each of the first and second weighting and adding circuits comprises: Sequentially sampling and holding the plurality of signals sequentially read out to the vertical signal lines; After completing the sample-and-hold of the plurality of signals, a weighted addition of positive values ​​and a weighted addition of negative values ​​are performed. A solid-state imaging device according to (14).

[0250] (17) The plurality of cells included in the cell array include a first selection transistor that switches and inputs the plurality of signals to the first weighted addition circuit; a second selection transistor that switches and inputs the plurality of signals to the second weighted addition circuit; A solid-state imaging device according to (14).

[0251] (18) The solid-state imaging device according to (8), wherein the weighted addition circuit further includes a fourth switch capable of switching a terminal voltage value at a terminal of the plurality of capacitors opposite to the terminal on the readout line side.

[0252] (19) The solid-state imaging device according to (18), wherein the AD conversion unit performs successive comparison of the DAC signal generated by the weighted addition circuit based on the switching of the fourth switch and the weighted-added signal.

[0253] (20) A solid-state imaging device as described in (14), wherein each of the first and second weighted addition circuits further includes a fourth switch that can switch the value of a terminal voltage at a terminal of the plurality of capacitors opposite to the terminal on the readout wiring side. [Explanation of symbols]

[0254] 1: vehicle, 10: solid-state imaging device, 11: vehicle control system, 12: vertical drive unit, 13: AD conversion unit, 14: horizontal drive unit, 15: control unit, 16: signal processing circuit, 17: data storage unit, 18: input / output unit, 19: weighted addition circuit, 19': weighted addition circuit, 19'': weighted addition circuit, 20: pixel array unit, 21: vehicle control ECU, 22: communication unit, 23: map information storage unit, 24: location information acquisition unit, 25: external recognition sensor, 26: in-vehicle sensor, 27: Vehicle sensor, 28: Memory unit, 29: Driving assistance / automatic driving control unit, 30: DMS, 31: HMI, 32: vehicle control unit, 33: comparator, 41: communication network, 42: photoelectric conversion film, 43: transparent electrode, 44: lower electrode, 51: camera, 52: radar, 53: LiDAR, 54: ultrasonic sensor, 61: analysis unit, 62: action planning unit, 63: motion control unit, 71: self-position estimation unit, 72: sensor fusion unit, 73: recognition unit, 81: steering control unit, 82: brake control unit, 83: drive control unit, 84: body system control unit, 85: light control unit, 86: horn control unit, 100: pixel, 130: readout wiring, 131: first switch, 132: capacitance, 133: second switch, 134: dummy capacitance, 135: third switch, 136: fourth switch 140: voltage source, 141: current source, 142: buffer circuit

Claims

1. a readout wiring for receiving an input of a plurality of signals read out to vertical signal lines of the solid-state imaging device; a first switch that connects or disconnects the vertical signal line and the readout wiring; A plurality of capacitances for sampling and holding the plurality of signals; a plurality of second switches for connecting or disconnecting the readout wiring and the capacitor; Weighted summation circuit.

2. The solid-state imaging device includes: a cell array having a mechanism for holding the plurality of signals; the vertical signal lines for reading out the signals from the cell array; an AD conversion unit that performs AD conversion of the signal weighted and added by the weighting and adding circuit, the weighted addition circuit is connected to the cell array, the vertical signal line, and the AD conversion unit; 2. The weighted summation circuit according to claim 1.

3. The weighting addition circuit includes: Sequentially sampling and holding the plurality of signals sequentially read out to the vertical signal lines; After completing the sample-holding of the plurality of signals, a weighted sum is performed.

2. The weighted summation circuit according to claim 1.

4. 3. The weighted addition circuit according to claim 2, wherein the cells included in the cell array are pixels including a photodiode or memories included in a memory array.

5. 3. The weighted addition circuit according to claim 2, further comprising a fourth switch capable of switching a terminal voltage value at terminals of the plurality of capacitors opposite to the terminals on the readout line side.

6. 6. The weighting and addition circuit according to claim 5, wherein the AD conversion section performs a successive approximation operation between the DAC signal generated by the weighting and addition circuit based on switching of the fourth switch and the weighted-added signal.

7. a cell array having a mechanism for holding a plurality of signals; vertical signal lines for reading out the signals from the cell array; a weighted addition circuit that samples and holds the plurality of signals read out to the vertical signal lines and performs weighted addition; an AD conversion unit that performs AD conversion of the signal weighted and added by the weighting and adding circuit, The weighting addition circuit includes: a readout wiring that receives an input of the plurality of signals read out to the vertical signal lines; a first switch that connects or disconnects the vertical signal line and the readout wiring; A plurality of capacitances for sampling and holding the plurality of signals; a plurality of second switches for connecting or disconnecting the readout wiring and the capacitor; Solid-state imaging device.

8. The weighting addition circuit includes: Sequentially sampling and holding the plurality of signals sequentially read out to the vertical signal lines; After completing the sample-holding of the plurality of signals, a weighted sum is performed. The solid-state imaging device according to claim 7 .

9. The solid-state imaging device according to claim 7 , wherein the cells included in the cell array are pixels including a photodiode or memories included in a memory array.

10. The solid-state imaging device according to claim 7 , wherein the plurality of capacitors include a first capacitor and a second capacitor having different electrostatic capacitances.

11. 8. The solid-state imaging device according to claim 7, further comprising a buffer circuit for providing a delay time for inputting the plurality of signals, the buffer circuit being connected between the vertical signal line and the weighted addition circuit.

12. The weighting addition circuit includes: one or more dummy capacitances that adjust a difference in capacitance when the first capacitance and the second capacitance sample and hold; and one or more third switches for connecting or disconnecting the readout wiring and the dummy capacitance. The solid-state imaging device according to claim 10.

13. 13. The solid-state imaging device according to claim 12, wherein the weighting addition circuit stores electric charge in one or more of the dummy capacitances based on a difference in electrostatic capacitance between the first capacitance and the second capacitance.

14. a cell array having a mechanism for holding a plurality of signals; vertical signal lines for reading out the signals from the cell array; a first weighting addition circuit that samples and holds the plurality of signals read out to the vertical signal lines and performs weighting addition of positive values; a second weighting addition circuit that samples and holds the plurality of signals read out to the vertical signal lines and performs weighting addition of negative values; an AD conversion unit that performs AD conversion on the signal weighted and added by the first weighting and addition circuit and the signal weighted and added by the second weighting and addition circuit, Each of the first and second weighting and adding circuits comprises: a readout wiring that receives an input of the plurality of signals read out to the vertical signal lines; a first switch that connects or disconnects the vertical signal line and the readout wiring; A plurality of capacitances for sampling and holding the plurality of signals; a plurality of second switches for connecting or disconnecting the readout wiring and the capacitor; Solid-state imaging device.

15. The solid-state imaging device according to claim 14 , wherein the cells included in the cell array are pixels including a photodiode or memories included in a memory array.

16. Each of the first and second weighting and adding circuits comprises: Sequentially sampling and holding the plurality of signals sequentially read out to the vertical signal lines; After completing the sample-and-hold of the plurality of signals, a weighted addition of positive values ​​and a weighted addition of negative values ​​are performed. The solid-state imaging device according to claim 14.

17. The plurality of cells included in the cell array include a first selection transistor that switches the plurality of signals to be input to the first weighted addition circuit; a second selection transistor that switches the plurality of signals to the second weighted addition circuit; The solid-state imaging device according to claim 14.

18. 9. The solid-state imaging device according to claim 8, wherein the weighted addition circuit further includes a fourth switch capable of switching a value of a terminal voltage at a terminal of the plurality of capacitors opposite to the terminal on the readout line side.

19. 20. The solid-state imaging device according to claim 18, wherein the AD conversion section performs successive approximation of the DAC signal generated by the weighting and addition circuit based on switching of the fourth switch and the weighted and added signal.

20. 15. The solid-state imaging device according to claim 14, wherein each of the first and second weighted addition circuits further includes a fourth switch that can switch a terminal voltage value at a terminal of the plurality of capacitors opposite to the terminal on the readout wiring side.

Citation Information

Patent Citations

  • Low-capacitance chip varistor and its manufacture

    JP1999003809A