Imaging device

Abstract

Provided is an imaging device having a motion detection function and an image processing function. The imaging device can detect a difference between a frame image to be referenced and a frame image to be compared, and can be switched from a motion detection mode to a general imaging mode when a significant difference is detected. In the motion detection mode, the imaging device is operated at a low frame rate, so that power consumption can be suppressed. Further, the imaging device has an image recognition function and combines the motion detection function, and can thus be switched from the motion detection mode to the general imaging mode when a specific image is recognized.

Description

[Technical field]

[0001] One aspect of the present invention relates to an imaging device.

[0002] Note that one embodiment of the present invention is not limited to the above technical field. The technical field of one embodiment of the invention disclosed in this specification etc. relates to an object, a method, or a manufacturing method. Alternatively, one embodiment of the present invention relates to a process, a machine, a manufacture, or a composition of matter. Therefore, more specifically, examples of the technical field of one embodiment of the present invention disclosed in this specification include a semiconductor device, a display device, a liquid crystal display device, a light-emitting device, a lighting device, a power storage device, a memory device, an imaging device, an operation method thereof, or a manufacturing method thereof.

[0003] Note that in this specification and the like, a semiconductor device refers to any device that can function by utilizing semiconductor characteristics. A transistor and a semiconductor circuit are one embodiment of a semiconductor device. Further, a memory device, a display device, an imaging device, and an electronic device may include a semiconductor device. [Background technology]

[0004] A technology for forming a transistor using an oxide semiconductor thin film formed over a substrate has attracted attention. For example, Patent Document 1 discloses an imaging device in which a transistor that includes an oxide semiconductor and has an extremely low off-state current is used in a pixel circuit.

[0005] Moreover, Patent Document 2 discloses a technique for adding a calculation function to an imaging device. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] JP 2011-119711 A [Patent Document 2] JP 2016-123087 A Summary of the Invention [Problem to be solved by the invention]

[0007] With the advancement of technology, imaging devices equipped with solid-state imaging elements such as CMOS image sensors can easily capture high-quality images. In the next generation, imaging devices will be required to be equipped with even more intelligent functions.

[0008] Currently, image data compression and image recognition are performed by converting image data (analog data) into digital data and taking it out for processing. If this processing could be performed within the imaging device, linking with external devices would be faster, improving user convenience. In addition, the load and power consumption of peripheral devices could be reduced. Furthermore, if complex data processing could be performed on analog data, the time required for data conversion could be shortened.

[0009] In addition, surveillance cameras and the like are constantly in operation and consume a lot of power. For example, if it were possible to simplify the operation when no event occurs and transition to normal imaging operation when an event occurs, it would be possible to significantly reduce power consumption. It is also preferable to be able to reduce power consumption by power gating when no event occurs.

[0010] Therefore, one object of one embodiment of the present invention is to provide an imaging device capable of performing image processing. Another object is to provide an imaging device capable of detecting a change in a subject. Another object is to provide an imaging device with low power consumption. Another object is to provide a small imaging device. Another object is to provide an imaging device with high reliability. Another object is to provide a novel imaging device or the like. Another object is to provide a method for operating the imaging device. Another object is to provide a novel semiconductor device or the like.

[0011] Note that the description of these problems does not preclude the existence of other problems. Note that one embodiment of the present invention does not necessarily solve all of these problems. Note that problems other than these will become apparent from the description of the specification, drawings, claims, etc., and it is possible to extract problems other than these from the description of the specification, drawings, claims, etc. [Means for solving the problem]

[0012] One embodiment of the present invention relates to an imaging device that has a motion detection function and an image processing function and operates with low power consumption, or to an operation method thereof.

[0013] One embodiment of the present invention is an imaging device having a pixel, a first circuit, and a second circuit, in which the first circuit has a function of supplying a first potential to the pixel, the pixel has a function of acquiring the first data and the second data, the pixel has a function of generating third data that is a difference between the first data and the second data, the pixel has a function of generating fourth data by adding a potential based on the first potential to the third data, and the second circuit has a function of generating fifth data that corresponds to the difference between the third data output by the pixel and the fourth data.

[0014] Another embodiment of the present invention is an imaging device having a pixel block, a first circuit, and a second circuit. The pixel block has a plurality of pixels arranged in a matrix. The first circuit has a function of supplying a first potential to the pixels. The pixels have a function of acquiring first data and second data. The pixels have a function of generating third data that is a difference between the first data and the second data. The pixels have a function of generating fourth data by adding a potential based on the first potential to the third data. The second circuit has a function of generating fifth data that corresponds to a difference between a sum of the third data output by the plurality of pixels in the pixel block and a sum of the fourth data output by the plurality of pixels in the pixel block.

[0015] The pixel has a photoelectric conversion device, a first transistor, a second transistor, a third transistor, a fourth transistor, a fifth transistor, a sixth transistor, a first capacitor, and a second capacitor, one electrode of the photoelectric conversion device is electrically connected to one of a source or a drain of the first transistor, the other of the source or the drain of the first transistor is electrically connected to one of a source or a drain of the second transistor and one electrode of the first capacitor, and the other electrode of the first capacitor is electrically connected to one of a source or a drain of the third transistor. one of the source or drain of the fourth transistor, one electrode of the second capacitor, and the gate of the fourth transistor, one of the source or drain of the fourth transistor is electrically connected to one of the source or drain of the fifth transistor, the other electrode of the second capacitor is electrically connected to one of the source or drain of the sixth transistor, the other of the source or drain of the fifth transistor is electrically connected to the second circuit, and the other of the source or drain of the sixth transistor is electrically connected to the first circuit.

[0016] The second circuit may be a correlated double sampling circuit.

[0017] It is preferable that the transistor included in the pixel has a metal oxide in a channel formation region, and the metal oxide has In, Zn, and M (M is one or more of Al, Ti, Ga, Ge, Sn, Y, Zr, La, Ce, Nd, and Hf).

[0018] Another aspect of the present invention is a method for operating an imaging device, which performs a first step of starting an imaging operation at a first frame rate, a second step of capturing and storing a reference image, a third step of capturing a comparison image, a fourth step of obtaining difference data between the reference image and the comparison image, a fifth step of extracting a second feature from the difference data, and a sixth step of comparing a predetermined first feature with the second feature, in the above order, and switches to a second frame rate to perform imaging operation when it is determined that the first feature has been detected from the second feature, and returns to the third step when it is determined that the first feature has not been detected from the second feature.

[0019] The second frame rate is preferably higher than the first frame rate. For example, the first frame rate is preferably 0.1 fps or more and 10 fps or less, and the second frame rate is preferably 15 fps or more and 240 fps or less.

[0020] During the period in which the imaging operation is performed at the first frame rate, power gating may be performed. Effect of the Invention

[0021] By using one embodiment of the present invention, it is possible to provide an imaging device capable of performing image processing. Or, it is possible to provide an imaging device capable of detecting a change in a subject. Or, it is possible to provide an imaging device with low power consumption. Or, it is possible to provide a small-sized imaging device. Or, it is possible to provide an imaging device with high reliability. Or, it is possible to provide a novel imaging device or the like. Or, it is possible to provide a method for operating the imaging device. Or, it is possible to provide a novel semiconductor device or the like. [Brief description of the drawings]

[0022] FIG. 1 is a block diagram illustrating an imaging device. FIG. 2 is a diagram illustrating a pixel block 200 and a circuit 201. As shown in FIG. 3A and 3B are diagrams illustrating a pixel 100. FIG. 4A and 4B are diagrams illustrating a pixel 100. FIG. FIG. 5 is a timing chart illustrating a normal imaging operation. FIG. 6 is a timing chart illustrating a motion detection operation (without difference) and a product-sum operation. FIG. 7 is a timing chart illustrating the motion detection operation (with difference). FIG. 8 is a diagram illustrating the circuit 304. FIG. 9 is a timing chart illustrating the operation of the circuit 304. As shown in FIG. FIG. 10 is a timing chart illustrating the operation of the circuit 304. 11A and 11B are diagrams illustrating a circuit 301 and a circuit 302. FIG. FIG. 12 is a diagram illustrating a memory cell. 13A and 13B are diagrams showing examples of the configuration of a neural network. FIG. 14 is a flowchart illustrating the operation of the imaging device. 15A and 15B are diagrams illustrating the operation of the imaging device. 16A to 16D are diagrams for explaining the configuration of a pixel of an imaging device. 17A to 17C are diagrams illustrating the configuration of a photoelectric conversion device. FIG. 18 is a cross-sectional view illustrating a pixel. 19A to 19C are diagrams illustrating a Si transistor. FIG. 20 is a cross-sectional view illustrating a pixel. FIG. 21 is a cross-sectional view illustrating a pixel. 22A to 22D illustrate OS transistors. FIG. 23 is a cross-sectional view illustrating a pixel. 24A1 to 24A3 and 24B1 to 24B3 are perspective views of a package and a module that house an imaging device. Fig. 25A is a block diagram illustrating a memory circuit, and Fig. 25B to Fig. 25E are circuit diagrams illustrating a memory cell. 26A and 26B are block diagrams showing configuration examples of a semiconductor device. 27A to 27D are diagrams for explaining an example of the operation of power management of a semiconductor device. FIG. 28 is a flowchart showing an example of an operation of power management of a semiconductor device. 29A and 29B are block diagrams showing configuration examples of a semiconductor device. FIG. 30 is a block diagram showing an example of the configuration of a processor core. FIG. 31 is a circuit diagram showing a configuration example of the memory circuit. FIG. 32 is a timing chart illustrating an example of the operation of the memory circuit. FIG. 33 is a circuit diagram showing an example of the configuration of a memory cell of a cache. FIG. 34 is a timing chart illustrating an example of the operation of a memory cell. 35A to 35F are diagrams illustrating an electronic device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0023] The embodiments will be described in detail with reference to the drawings. However, the present invention is not limited to the following description, and it is easily understood by those skilled in the art that the form and details of the present invention can be modified in various ways without departing from the spirit and scope of the present invention. Therefore, the present invention is not interpreted as being limited to the description of the embodiments shown below. In the configuration of the invention described below, the same reference numerals are used in common between different drawings for the same parts or parts having similar functions, and repeated explanations may be omitted. In addition, hatching of the same elements constituting the drawings may be omitted or changed as appropriate between different drawings.

[0024] In addition, even if a circuit diagram shows a single element, the element may be configured as a plurality of elements as long as there is no functional problem. For example, a plurality of transistors operating as a switch may be connected in series or parallel. A capacitor may also be divided and placed in multiple positions.

[0025] In addition, one conductor may have multiple functions such as wiring, an electrode, and a terminal, and in this specification, multiple names may be used for the same element. Even if elements are shown as being directly connected to each other on a circuit diagram, the elements may actually be connected to each other via one or more conductors, and in this specification, such a configuration is also included in the category of direct connection.

[0026] (Embodiment 1) In this embodiment, an imaging device which is one embodiment of the present invention will be described with reference to drawings.

[0027] One aspect of the present invention is an imaging device with additional functions such as image recognition. The imaging device stores analog data (image data) acquired during imaging in pixels, and can extract data obtained by multiplying the analog data by an arbitrary weighting factor.

[0028] By inputting the data into a neural network, it is possible to perform image recognition and other processing. Since a huge amount of image data can be stored in the pixels as analog data, processing can be performed efficiently.

[0029] In addition, the imaging device according to one embodiment of the present invention has a motion detection function. The imaging device can detect a difference between a reference frame image and a frame image to be compared, and can switch from the motion detection mode to the normal imaging mode when a significant difference is detected. By operating at a low frame rate in the motion detection mode, power consumption can be reduced. Furthermore, by operating at a high frame rate in the normal imaging mode, high-quality image data can be obtained.

[0030] In addition, the imaging device according to one embodiment of the present invention can switch from the motion detection mode to the normal imaging mode when a specific image is recognized by combining the image recognition function and the motion detection function described above. Therefore, for example, it can distinguish between people, animals, plants, etc., and can also respond to defects in products, changes over time, and abnormalities such as natural disasters.

[0031] <Imaging device> 1 is a block diagram illustrating an imaging device of one embodiment of the present invention. The imaging device includes a pixel array 300, a circuit 201, a circuit 302, a circuit 303, a circuit 304, and a circuit 305. Note that the circuit 201 and the circuits 301 to 305 are not limited to a single circuit configuration and may be configured with a plurality of circuits. Alternatively, any of a plurality of the above circuits may be integrated. Furthermore, a circuit other than the above may be connected.

[0032] The pixel array 300 has an imaging function and a calculation function. The circuits 201 and 301 have a calculation function. The circuit 302 has a calculation function or a data conversion function. The circuits 303 and 304 have a selection function. The circuit 305 has a function of supplying a potential for a product-sum calculation to the pixel. A shift register, a decoder, or the like can be used as the circuit having the selection function. Note that the circuits 301 and 302 may be provided externally.

[0033] The pixel array 300 has a plurality of pixel blocks 200. As shown in Fig. 2, the pixel block 200 has a plurality of pixels 100 arranged in a matrix, and each pixel 100 is electrically connected to a circuit 201. Note that the circuit 201 may be provided within the pixel block 200.

[0034] The pixel 100 can generate image data and data obtained by applying a weighting factor to the image data. In FIG. 2, the number of pixels in the pixel block 200 is 2×2 as an example, but is not limited to this. For example, it can be 3×3, 4×4, etc. Alternatively, the number of pixels in the horizontal direction and the vertical direction may be different. Also, some pixels may be shared between adjacent pixel blocks.

[0035] The pixel block 200 operates as a product-sum calculation circuit. Moreover, a circuit 201 electrically connected to the pixel block 200 has a function of extracting the product of image data and a weighting coefficient from the pixel 100.

[0036] <Pixel circuit> Pixel 100 may include a photoelectric conversion device 101, a transistor 102, a transistor 103, a capacitor 104, a transistor 105, a capacitor 106, a transistor 107, a transistor 108, and a transistor 109, as shown in FIG. 3A.

[0037] One electrode of the photoelectric conversion device 101 is electrically connected to one of the source or drain of the transistor 102. The other of the source or drain of the transistor 102 is electrically connected to one of the source or drain of the transistor 103 and one electrode of the capacitor 104. The other electrode of the capacitor 104 is electrically connected to one of the source or drain of the transistor 105, one electrode of the capacitor 106, and the gate of the transistor 107. One of the source or drain of the transistor 107 is electrically connected to one of the source or drain of the transistor 108. The other electrode of the capacitor 106 is electrically connected to one of the source or drain of the transistor 109.

[0038] The other electrode of the photoelectric conversion device 101 is electrically connected to a wiring 113. The other of the source or drain of the transistor 103 is electrically connected to a wiring 114. The other of the source or drain of the transistor 105 is electrically connected to a wiring 115. The other of the source or drain of the transistor 107 is electrically connected to a GND wiring or the like. The other of the source or drain of the transistor 108 is electrically connected to a wiring 112. The other of the source or drain of the transistor 109 is electrically connected to a wiring 111.

[0039] A gate of the transistor 102 is electrically connected to a wiring 121. A gate of the transistor 103 is electrically connected to a wiring 122. A gate of the transistor 105 is electrically connected to a wiring 123. A gate of the transistor 108 is electrically connected to a wiring 125. A gate of the transistor 109 is electrically connected to a wiring 124.

[0040] Here, an electrical connection point (wiring) between the other of the source or drain of transistor 102, the other of the source or drain of transistor 103, and one electrode of capacitor 104 is defined as node N1. Also, an electrical connection point (wiring) between the other electrode of capacitor 104, one of the source or drain of transistor 105, one electrode of capacitor 106, and the gate of transistor 107 is defined as node N2.

[0041] The wirings 113, 114, and 115 can function as power supply lines. For example, the wirings 114 and 115 can function as high-potential power supply lines, and the wiring 113 can function as a low-potential power supply line. The wirings 121, 122, 123, 124, and 125 can function as signal lines that control the conduction of each transistor. The wiring 111 can function as a wiring that supplies a potential corresponding to a weighting coefficient to the pixel 100. The wiring 112 can function as a wiring that electrically connects the pixel 100 and the circuit 201.

[0042] The wiring 112 may be electrically connected to an amplifier circuit or a gain adjustment circuit.

[0043] A photodiode can be used as the photoelectric conversion device 101. Any type of photodiode can be used, such as a Si photodiode having silicon in the photoelectric conversion layer, or an organic photodiode having an organic photoconductive film in the photoelectric conversion layer. Note that, when it is desired to increase the light detection sensitivity at low illuminance, it is preferable to use an avalanche photodiode.

[0044] The transistor 102 can have a function of controlling the potential of the node N1. The transistor 103 can have a function of initializing the potential of the node N1. The transistor 105 can have a function of initializing the potential of the node N2. The transistor 107 can have a function of controlling a current flowing from the circuit 201 in accordance with the potential of the node N2. The transistor 108 can have a function of selecting a pixel. The transistor 109 can have a function of supplying a potential corresponding to a weighting coefficient to the node N2.

[0045] As shown in FIG. 3B, the transistors 107 and 108 may be configured such that one of the source or drain of the transistor 107 is electrically connected to one of the source or drain of the transistor 108, the other of the source or drain of the transistor 107 is connected to a wiring 112, and the other of the source or drain of the transistor 108 is electrically connected to a GND wiring or the like.

[0046] 4A, a second gate may be provided to the transistor 107, and one of the source and the drain of the transistor 109 may be electrically connected to the second gate. A capacitor 151 may be provided in which the second gate is connected to one electrode of the transistor 109. The capacitor 151 functions as a storage capacitor. Note that the capacitor 151 may not be provided.

[0047] 4B, the connection direction of the photoelectric conversion device 101 may be reversed. In this case, the wirings 114 and 115 may function as low-potential power supply lines, and the wiring 113 may function as a high-potential power supply line. Note that, similarly to the configuration shown in FIG. 3B, one of the source or drain of the transistor 107 may be electrically connected to one of the source or drain of the transistor 108, the other of the source or drain of the transistor 107 may be connected to the wiring 112, and the other of the source or drain of the transistor 108 may be electrically connected to a GND wiring or the like.

[0048] When an avalanche photodiode is used for the photoelectric conversion device 101, a high voltage may be applied, and therefore a high-voltage transistor is preferably used for the transistor connected to the photoelectric conversion device 101. For example, a transistor using a metal oxide in a channel formation region (hereinafter, referred to as an OS transistor) can be used for the high-voltage transistor. Specifically, an OS transistor is preferably used for the transistor 102.

[0049] In addition, the OS transistor has a characteristic of having an extremely low off-state current. By using OS transistors for the transistors 102, 103, 105, and 109, the period during which charges can be held at the nodes N1 and N2 can be made extremely long. Therefore, a global shutter system in which charges are accumulated simultaneously in all pixels can be applied without complicating the circuit configuration or operation method. In addition, while image data is held in the node N2, multiple calculations can be performed using the image data.

[0050] On the other hand, there are cases where it is desirable for the transistor 107 to have excellent amplification characteristics. In addition, since the transistor 108 may be frequently turned on and off, it may be preferable to use a transistor with high mobility that can operate at high speed. Therefore, a transistor using silicon for a channel formation region (hereinafter, referred to as a Si transistor) may be used as the transistors 107 and 108.

[0051] Note that the present invention is not limited to the above, and any combination of OS transistors and Si transistors may be used. All the transistors may be OS transistors. Alternatively, all the transistors may be Si transistors. Examples of Si transistors include transistors having amorphous silicon and transistors having crystalline silicon (microcrystalline silicon, low-temperature polysilicon, and single crystal silicon).

[0052] The potential of the node N2 in the pixel 100 is determined by capacitive coupling between a potential obtained by capacitively coupling a fluctuation in the potential of the node N1 (image data) to a reset potential supplied from a wiring 115, and a potential corresponding to a weighting coefficient supplied from a wiring 111. That is, the gate of the transistor 107 becomes a potential obtained by adding an arbitrary weighting coefficient to the image data, and a current including a term representing the product of the image data and the arbitrary weighting coefficient flows through the transistor 107.

[0053] <Circuit 201> 2, the pixels 100 are electrically connected to each other through wirings 112. The circuit 201 can perform calculations using the sum of currents flowing through the transistors 107 of the pixels 100.

[0054] The circuit 201 includes a capacitor 202 , a transistor 203 , a transistor 204 , a transistor 205 , a transistor 206 , and a resistor 207 .

[0055] One electrode of the capacitor 202 is electrically connected to one of the source or drain of the transistor 203. One of the source or drain of the transistor 203 is electrically connected to the gate of the transistor 204. One of the source or drain of the transistor 204 is electrically connected to one of the source or drain of the transistor 205. One of the source or drain of the transistor 205 is electrically connected to one of the source or drain of the transistor 206. One electrode of the resistor 207 is electrically connected to the other electrode of the capacitor 202.

[0056] The other electrode of the capacitor 202 is electrically connected to the wiring 112. The other of the source and the drain of the transistor 203 is electrically connected to a wiring 218. The other of the source and the drain of the transistor 204 is electrically connected to a wiring 219. The other of the source and the drain of the transistor 205 is electrically connected to a reference power supply line such as a GND wiring. The other of the source and the drain of the transistor 206 is electrically connected to a wiring 212. The other electrode of the resistor 207 is electrically connected to a wiring 217.

[0057] The wirings 217, 218, and 219 can function as power supply lines. For example, the wiring 218 can function as a wiring that supplies a reset potential for reading. The wirings 217 and 219 can function as high-potential power supply lines. The wirings 213, 215, and 216 can function as signal lines that control the conduction of each transistor. The wiring 212 is an output line and can be electrically connected to, for example, the circuit 301 shown in FIG. 1.

[0058] The transistor 203 can have a function of resetting the potential of the wiring 211 to the potential of the wiring 218. The transistors 204 and 205 can have a function as source follower circuits. The transistor 206 can have a function of controlling reading. Note that the circuit 201 has a function as a correlated double sampling circuit (CDS circuit) and can be replaced with a circuit having the same function.

[0059] In one embodiment of the present invention, offset components other than the product of image data (X) and weighting coefficient (W) are removed to extract the target WX. WX can be calculated using data with and without imaging for the same pixel, and data when weights are applied to each of the data.

[0060] The current (I p ) is the sum of kΣ(XV th ) 2, the current (I p ) is kΣ(W+XV th ) 2 In addition, the current (I ref ) is the sum of kΣ(0-V th ) 2 , the current (I ref ) is the sum of kΣ(WV th ) 2 Here, k is a constant, V th is the threshold voltage of transistor 107.

[0061] First, calculate the difference (data A) between the captured data and the weighted data. kΣ((XV th ) 2 -(W+XV th ) 2 )=kΣ(-W 2 -2W·X+2W·V th )

[0062] Next, the difference between the no-image data and the weighted data (data B) is calculated. kΣ((0-V th ) 2 -(WV th ) 2 )=kΣ(-W 2 +2W·V th )

[0063] Then, take the difference between data A and data B. kΣ(-W 2 -2W·X+2W·V th -(-W 2 +2W·V th ))=kΣ(-2W·X). In other words, it is possible to remove offset components other than the product of the image data (X) and the weighting coefficient (W).

[0064] The circuit 201 can read out data A and data B. Note that the difference between data A and data B can be calculated by the circuit 301, for example.

[0065] The pixel 100 shown in Figures 3A, 3B, 4A and 4B has a normal imaging function and a motion detection function, and also has a part of a product-sum calculation function.

[0066] <Normal shooting mode> First, the normal imaging mode will be described with reference to the timing chart shown in Fig. 5. Note that in the normal imaging mode, since a product-sum operation is not performed, it is preferable that the potential of the wiring 111 is always "L", the potential of the wiring 124 is always "H", and the potential of the other electrode of the capacitor 106 is fixed. Also, here, the operation of the pixel 100 having the configuration of Fig. 3A or Fig. 3B will be described.

[0067] First, at time T1, the potential of the wiring 121 is set to "H", the potential of the wiring 122 is set to "H", and the potential of the wiring 123 is set to "H". Then, the transistors 102 and 103 are turned on, and the potential of the node N1 becomes the reset potential (the potential of the wiring 114) "V RES1 In addition, the transistor 105 is turned on, and the potential of the node N2 becomes the reset potential (the potential of the wiring 115) “V RES2 " Note that "V RES1 " and "V RES2 " may be at the same potential.

[0068] When the potential of the wiring 125 is set to "H" and the potential of the wiring 216 is set to "H", the transistor 203 in the circuit 201 is turned on, and the potential of the wiring 211 becomes the potential "Vr" of the wiring 218. That is, the output potential of the pixel 100 in the reset state is initialized to the potential "Vr".

[0069] At time T2, when the potential of the wiring 121 is set to “H”, the potential of the wiring 122 is set to “L”, the potential of the wiring 123 is set to “L”, the potential of the wiring 125 is set to “L”, and the potential of the wiring 216 is set to “L”, the transistors 103, 105, and 108 are turned off, and the potential of the node N1 becomes “V RES1 -V A ". In addition, the potential of the node N2 also changes to "V RES2 -V BIf the capacitance of the capacitor 104 is sufficiently larger than the capacitance of the node N2, V A and V B In addition, the transistor 203 is turned off, and the potential of the wiring 221 is held at "Vr."

[0070] When the potential of the wiring 121 is set to "L" at time T3, the transistor 102 is turned off, and the potentials of the nodes N1 and N2 are held.

[0071] At time T4, the potential of the wiring 125 is set to “H”, the potential of the wiring 213 is set to “H”, the potential of the wiring 215 is set to “V bias When an appropriate analog potential such as "Vr" is applied, the transistor 108 becomes conductive, and a current according to the potential of the node N2 flows from the wiring 112 to the transistor 107. Here, the potential of the other electrode of the capacitor 202 changes according to the current flowing through the wiring 112, and the change Y is added to the potential "Vr" of the wiring 211 by capacitive coupling.

[0072] Therefore, the potential of the wiring 211 becomes "Vr+Y." If Vr=0, Y becomes the difference between the image data and the data (noise) at the time of reset. In other words, image data from which the noise component has been removed can be obtained.

[0073] Furthermore, when the transistors 205 and 206 are turned on, image data can be output to the wiring 212 due to the source follower operation of the transistor 204 .

[0074] <Motion detection mode> Next, the motion detection function of the pixel 100 and the product-sum operation by the pixel block 200 and the circuit 201 will be described with reference to the timing chart shown in FIG.

[0075] First, the operation of the motion detection mode will be described with reference to a timing chart showing times T1 to T7. Note that Fig. 6 shows a case where there is no change between the reference image and the comparative image.

[0076] The period from time T1 to time T7 is a period for determining the potential of node N2. Since a weight (W) will be added to node N2 later by the capacitive coupling of capacitor 106, at least during this period, the potential of the other electrode of capacitor 106 is set to a potential corresponding to a weight coefficient of 0. Therefore, during this period, the potential of wiring 111 is set to a potential corresponding to a weight coefficient of 0 (for example, 0 V), and the potential of wiring 124 is set to "H."

[0077] At time T1, when the potential of the wiring 121 is set to "H", the potential of the wiring 122 is set to "H", and the potential of the wiring 123 is set to "H", the transistors 102 and 103 are turned on, and the potential of the node N1 becomes the reset potential (the potential of the wiring 114) "V RES1 In addition, the transistor 105 is turned on, and the potential of the node N2 becomes the reset potential (the potential of the wiring 115) “V RES2 " Note that "V RES1 " and "V RES2 " may be at the same potential.

[0078] At time T2, when the potential of the wiring 121 is set to “H”, the potential of the wiring 122 is set to “L”, and the potential of the wiring 123 is set to “H”, the transistor 103 is turned off, and the potential of the node N1 is set to “V RES1 -V A At this time, since the transistor 105 is conductive, the potential of the node N2 changes to "V RES2 " will be maintained.

[0079] When the potential of the wiring 121 is set to "L" at time T3, the transistor 102 is turned off and the potential of the node N1 is held. In addition, since the transistor 105 is turned on, the potential of the node N2 is set to "V RES2 " will be maintained.

[0080] When the potential of the wiring 123 is set to "L" at time T4, the transistor 105 is turned off, and the potential of the node N2 is held. The operations from time T1 up to this point correspond to the operation of obtaining and holding a reference image.

[0081] At time T5, when the potential of the wiring 121 is set to “H” and the potential of the wiring 122 is set to “H”, the transistors 102 and 103 are turned on, and the potential of the node N1 becomes “V RES1 -V A " to "V RES1 " In other words, the potential of the node N1 becomes "V A At this time, the change in the potential of the node N2 is capacitively coupled to the change in the potential of the node N1, so that the potential of the node N2 increases by "V RES2 +V B " Here, as explained above, V A and V B is almost the same value.

[0082] At time T6, when the potential of the wiring 121 is set to “H” and the potential of the wiring 122 is set to “L”, the transistor 103 is turned off, and the potential of the node N1 is set to “V RES1 -V A ". Also, the change in node N1, "-V A " is capacitively coupled to node N2, so the potential of node N2 is "V RES2 "

[0083] When the potential of the wiring 121 is set to "L" at time T7, the transistor 102 is turned off, and the potentials of the nodes N1 and N2 are held. The operations from time T5 to this point correspond to the operation of obtaining and holding a comparative image.

[0084] That is, if there is no change between the reference image and the comparison image, the potential of node N1 will be the same as the reference image held before time T5, and the potential of node N2 will be the reset potential, indicating that there is no difference between the reference image and the comparison image.

[0085] If there is a change between the reference image and the comparative image, as shown in the timing chart of FIG. 7, at time T7, the potential of the node N1 becomes “V RES1 -V C " and the potential of node N2 becomes "V RES2 +V B -V D " Here, "V C" is "V A " is a different value from "V D " is "V B " is a value different from the reset potential. In other words, the potential of node N2 is a potential different from the reset potential, indicating that there is a difference between the reference image and the comparative image.

[0086] Next, the product-sum operation will be described using the timing chart shown at times T8 to T11 in Fig. 6. First, the operation of calculating the difference (data A) between the captured data and the weighted data will be described. Note that the presence or absence of a difference between the reference image and the comparative image is not involved in the product-sum operation described here.

[0087] When the potential of the wiring 125 is set to "H" and the potential of the wiring 216 is set to "H" at time T8, the transistor 203 in the circuit 201 is turned on, and the potential of the wiring 211 becomes the potential "Vr" of the wiring 218. That is, the output potential of the pixel 100 in the reset state is initialized to "Vr." The operations up to the period T7 correspond to the acquisition of image-captured data, and the data is represented as the potential "Vr" of the wiring 211.

[0088] At time T9, when the potential of the wiring 111 is set to a potential corresponding to the weighting factor W, the potential of the wiring 124 is set to "H", the potential of the wiring 125 is set to "L", and the potential of the wiring 216 is set to "L", the transistor 109 becomes conductive, and the potential of the other electrode of the capacitor 106 changes from "0" to "W". Therefore, the difference "W" is added to the node N2. In other words, if the potential of the node N2 determined by the image data acquisition operation is set to "X", the potential of the node N2 when the weighting factor W is added becomes "X+W".

[0089] At time T10, when the potential of wiring 124 is set to “L”, the potential of wiring 125 is set to “H”, the potential of wiring 213 is set to “H”, and the potential of wiring 215 is set to “H”, the potential of the other electrode of capacitor 106 and the potential of node N2 are maintained, transistor 108 becomes conductive, and a current corresponding to the potential “X+W” of node N2 flows from wiring 112 to transistor 107.

[0090] Here, the potential of the other electrode of capacitor 202 changes according to the current flowing through wiring 112, and the change Z is added to the potential Vr of wiring 211 by capacitive coupling. Therefore, the potential of wiring 211 becomes "Vr+Z." Here, if Vr=0 is considered, Z is the difference itself, and data A has been calculated.

[0091] In addition, the potential of the wiring 213 is set to “H” and the potential of the wiring 215 is set to “V bias By setting an appropriate analog potential such as "data A", the circuit 201 can output a signal potential according to the data A by a source follower operation.

[0092] In a similar step, the difference (data B) between data without imaging and data with a weight added thereto can be calculated. Note that since no imaging is performed, an operation is performed without providing an accumulation period. For example, when the wiring 121 is at "H", the wiring 122 is also at "H", so that the node N1 can be maintained at the reset potential. The operations from time T5 to time T8 may be omitted, and it is sufficient that the nodes N1 and N2 are at the reset potential before time T8.

[0093] Although one pixel has been described here, a plurality of pixels are connected in parallel to the circuit 201, and the above operation (product-sum operation) is performed for a plurality of pixels.

[0094] The data A and data B output from the circuit 201 by the above operation are input to the circuit 301. The circuit 301 performs an arithmetic operation to obtain the difference between the data A and the data B, and can remove unnecessary offset components other than the product of the image data (potential X) and the weighting coefficient (potential W). The circuit 301 may have a configuration including an arithmetic circuit like the circuit 201, or may have a configuration in which a memory circuit (also referred to as a storage circuit) and software processing are used to obtain the difference.

[0095] 1 to the wiring 111, and the weighting coefficient is preferably rewritten at least once within a frame period. A decoder can be used as the circuit 305. The circuit 305 may include a D / A converter or an SRAM.

[0096] A signal can be output to the wiring 112 that selects the pixel 100 to which the weighting coefficient is to be input from the circuit 303. The circuit 303 can be a decoder or a shift register.

[0097] A signal can be output from the circuit 304 to the wiring 125 or the like connected to the gate of the transistor 108 of the pixel 100. The circuit 304 can be a decoder or a shift register.

[0098] Although the processing of captured image data has been described above, the imaging device of one embodiment of the present invention can extract image data without processing it.

[0099] In the multiply-and-accumulate operation, it is preferable to simultaneously select pixels from multiple rows. On the other hand, when extracting only imaging data, it is preferable to extract data from pixels from one row. In one embodiment of the present invention, the circuit 304 for selecting the pixels 100 has a function of switching the number of rows to be selected.

[0100] <Shift register> 8 is an example of a circuit that can be used for the circuit 304. This circuit is a shift register circuit, in which a plurality of logic circuits (SR) are electrically connected. Signal lines such as wiring RES, wiring VSS_RDRS, wiring RPWC_SE[0:3], wiring RCLK[0:3], and wiring RSP are connected to each logic circuit (SR), and by inputting an appropriate signal potential to each signal line, selection signal potentials can be sequentially output from the logic circuit (SR).

[0101] The logic circuit (SR) is electrically connected to a circuit 170. The circuit 170 is provided with a plurality of transistors, and is connected to signal lines such as wirings SE_SW[0:2] and wirings SX[0:2]. The conduction of the transistors is controlled by inputting an appropriate signal potential to each signal line. The number of rows of pixels to be selected can be switched by controlling the circuit 170.

[0102] One of the source and the drain of one transistor is electrically connected to an output terminal of one logic circuit (SR), and the other of the source and the drain of the transistor is connected to a wiring SE. The wiring SE is electrically connected to a wiring 122 that selects the pixel 100.

[0103] A signal potential supplied from the wiring SE_SW[0] can be input to the gate of a transistor connected to the wiring SE[0]. A signal potential supplied from the wiring SE_SW[1] can be input to the gate of a transistor connected to the wiring SE[1]. A signal potential supplied from the wiring SE_SW[2] can be input to the gate of a transistor connected to the wiring SE[2]. A signal potential supplied from one of the wirings SE_SW[0:2] can be input to the gate of a transistor connected to the wiring SE[3] and subsequent wirings in the same order.

[0104] Adjacent wirings SE are electrically connected via one transistor, and the wiring SE[0] is electrically connected to the power supply line (VSS) via one transistor.

[0105] A signal potential supplied from the wiring SX[0] can be input to the gate of a transistor that electrically connects the power supply line (VSS) and the wiring SE[0]. A signal potential supplied from the wiring SX[1] can be input to the gate of a transistor that electrically connects the wiring SE[0] and the wiring SE[1]. A signal potential supplied from the wiring SX[2] can be input to the gate of a transistor that electrically connects the wiring SE[1] and the wiring SE[2]. Any of the signal potentials supplied from the wirings SX[0:2] can be input to the gates of transistors that electrically connect the subsequent wirings SE in the same order.

[0106] 9 is a timing chart illustrating an operation of simultaneously selecting a plurality of rows (three rows) by the circuit illustrated in FIG 8. (0) to (161) correspond to the timing at which the logic circuit (SR) outputs a signal potential to the wiring SE.

[0107] At timing (0), when the potential of the wire SX[0] becomes "L", the potential of the wire SX[1] becomes "H", the potential of the wire SX[2] becomes "H", the potential of the wire SE_SW[0] becomes "H", the potential of the wire SE_SW[1] becomes "L", and the potential of the wire SE_SW[2] becomes "L", the conduction of each transistor is controlled, and "H" is output to the wire SE[0], "H" to the wire SE[1], and "H" to the wire SE[2]. "L" is output to the other wires SE.

[0108] Therefore, three rows can be selected simultaneously, and a product-sum operation can be performed on, for example, pixels in three rows and three columns.

[0109] At timing (1), when the potential of the wire SX[0] becomes "H", the potential of the wire SX[1] becomes "L", the potential of the wire SX[2] becomes "H", the potential of the wire SE_SW[0] becomes "L", the potential of the wire SE_SW[1] becomes "H", and the potential of the wire SE_SW[2] becomes "L", the conduction of each transistor is controlled, and "L" is output to the wire SE[0], "H" to the wire SE[1], "H" to the wire SE[2], and "H" to the wire SE[3]. "L" is output to the other wires SE.

[0110] In other words, at timing (1), a multiply-and-accumulate operation with a stride of 1, which is shifted by one row from timing (0), is possible.

[0111] FIG. 10 is a timing chart illustrating the operation of selecting one row by the circuit shown in FIG.

[0112] In the operation according to the timing chart, the potential of the wirings SE_SW[0:2] is always “H” and the potential of the wirings SX[0:2] is always “L.” Therefore, the output of the logic circuit (SR) appears directly on each wiring SE, making it possible to select each row.

[0113] <Circuits 301, 302> 11A is a diagram for explaining circuits 301 and 302 connected to circuit 201. Data resulting from product-sum calculation output from circuit 201 is sequentially input to circuit 301. Circuit 301 may have various calculation functions in addition to the function of calculating the difference between data A and data B described above. For example, circuit 301 may have the same configuration as circuit 201. Alternatively, the function of circuit 301 may be replaced by software processing.

[0114] The circuit 301 may also have a circuit that performs an activation function calculation. For example, a comparator circuit can be used for the circuit. The comparator circuit compares input data with a set threshold value and outputs the result as binary data. That is, the pixel block 200 and the circuit 301 can act as a part of a neural network.

[0115] The circuit 301 may also have an A / D converter. When image data is to be output to the outside without performing a multiply-and-accumulate operation, the circuit 301 can convert analog data into digital data.

[0116] Furthermore, the data output by the pixel block 200 corresponds to multi-bit image data, but if the circuit 301 can binarize the data, it can be said that the image data is compressed.

[0117] Data output from the circuit 301 is input sequentially to the circuit 302. The circuit 302 can have a configuration including, for example, a latch circuit and a shift register. This configuration enables parallel-serial conversion to be performed, and data input in parallel can be output as serial data to the wiring 311. There is no limitation on the connection destination of the wiring 311. For example, the wiring 311 can be connected to a neural network, a storage device, a communication device, or the like.

[0118] 11B, the circuit 302 may have a neural network. The neural network has memory cells arranged in a matrix, and each memory cell holds a weighting factor. Data output from the circuit 301 is input to each of the memory cells 320, and a multiply-and-accumulate operation can be performed. Note that the number of memory cells shown in FIG. 11B is an example and is not limited.

[0119] The neural network shown in FIG. 11B includes memory cells 320 and reference memory cells 325 arranged in a matrix, a circuit 330, a circuit 350, a circuit 360, and a circuit 370.

[0120] 12 shows an example of a memory cell 320 and a reference memory cell 325. The reference memory cell 325 is provided in an arbitrary column. The memory cell 320 and the reference memory cell 325 have the same configuration, and include a transistor 161, a transistor 162, and a capacitor 163.

[0121] One of the source or drain of the transistor 161 is electrically connected to the gate of the transistor 162. The gate of the transistor 162 is electrically connected to one electrode of the capacitor 163. Here, a point where one of the source or drain of the transistor 161, the gate of the transistor 162, and one electrode of the capacitor 163 are connected is referred to as a node NM.

[0122] A gate of the transistor 161 is electrically connected to the wiring WL. The other electrode of the capacitor 163 is electrically connected to the wiring RW. One of the source and the drain of the transistor 162 is electrically connected to a reference potential wiring such as a GND wiring.

[0123] In the memory cell 320, the other of the source and the drain of the transistor 161 is electrically connected to a wiring WD. The other of the source and the drain of the transistor 162 is electrically connected to a wiring BL.

[0124] In the reference memory cell 325, the other of the source and the drain of the transistor 161 is electrically connected to a wiring WDref. The other of the source and the drain of the transistor 162 is electrically connected to a wiring BLref.

[0125] The wiring WL is electrically connected to the circuit 330. The circuit 330 can be a decoder, a shift register, or the like.

[0126] The wiring RW is electrically connected to the circuit 301. Binary data output from the circuit 301 is written to each memory cell. Note that a sequential circuit such as a shift register may be provided between the circuit 301 and each memory cell.

[0127] The wiring WD and the wiring WDref are electrically connected to the circuit 350. The circuit 350 can be a decoder, a shift register, or the like. The circuit 350 may also include a D / A converter or an SRAM. The circuit 350 can output the weighting coefficient written to the node NM.

[0128] The wiring BL and the wiring BLref are electrically connected to the circuit 360. The circuit 360 can have a structure similar to that of the circuit 201. The circuit 360 can provide a signal obtained by removing the offset component from the product-sum operation result.

[0129] The circuit 360 is electrically connected to the circuit 370. The circuit 370 can also be called an activation function circuit. The activation function circuit has a function of performing an operation for converting a signal input from the circuit 360 according to a predefined activation function. As the activation function, for example, a sigmoid function, a tanh function, a softmax function, a ReLU function, a threshold function, or the like can be used. The signal converted by the activation function circuit is output to the outside as output data.

[0130] As shown in FIG. 13A, the neural network NN can be configured with an input layer IL, an output layer OL, and an intermediate layer (hidden layer) HL. The input layer IL, the output layer OL, and the intermediate layer HL each have one or more neurons (units). The intermediate layer HL may be one layer or two or more layers. A neural network having two or more intermediate layers HL can also be called a DNN (deep neural network). Learning using a deep neural network can also be called deep learning.

[0131] Input data is input to each neuron in the input layer IL. An output signal from a neuron in the previous or next layer is input to each neuron in the hidden layer HL. An output signal from a neuron in the previous layer is input to each neuron in the output layer OL. Each neuron may be connected to all neurons in the previous or next layer (full connection), or may be connected to only a portion of the neurons.

[0132] FIG. 13B shows an example of a computation by a neuron. Here, neuron N and two neurons in the previous layer that output signals to neuron N are shown. Neuron N receives the output x1 and the output of the previous layer neuron x 2 is input. Then, in neuron N, the output is x 1 and weight w 1 The multiplication result (x 1 w 1 ) and output x 2 and weight w 2 The multiplication result (x 2 w 2 ) x 1 w 1 +x 2 w 2 After is calculated, a bias b is added if necessary to get the value a=x 1 w 1 +x 2 w 2 +b is obtained. Then, the value a is transformed by the activation function h, and the output signal y=ah is output from the neuron N.

[0133] In this way, the operations performed by neurons include summing the products of the outputs of neurons in the previous layer and the weights, that is, multiplication and addition operations (x 1 w 1 +x 2 w 2 This multiplication and accumulation operation may be performed by software using a program, or may be performed by hardware.

[0134] In one embodiment of the present invention, a multiply-and-accumulate circuit is used as hardware. When an analog circuit is used for the multiply-and-accumulate circuit, the circuit scale of the multiply-and-accumulate circuit can be reduced, or the number of accesses to a memory can be reduced, leading to improved processing speed and reduced power consumption.

[0135] The product-sum circuit preferably includes an OS transistor. Since an OS transistor has an extremely small off-state current, it is suitable as a transistor constituting an analog memory of the product-sum circuit. Note that the product-sum circuit may be configured using both a Si transistor and an OS transistor.

[0136] <Operation of the imaging device> Next, an example of a method for operating the imaging device of one embodiment of the present invention will be described with reference to the flowchart in FIG.

[0137] First, an imaging operation is selected (S1). If normal imaging is to be performed, the process proceeds to setting of the normal imaging mode (S11).

[0138] The normal imaging mode is a high-speed video imaging mode, and the frame rate is set to, for example, 15 fps to 240 fps, preferably 30 fps to 120 fps, typically 60 fps. Then, a continuous operation or a timer operation is performed under the set conditions (S12). The imaging operation ends after the timer operation or at the user's discretion.

[0139] If motion detection is to be performed, the process proceeds to setting the motion detection mode (S2).

[0140] The motion detection mode is a low-speed video capture mode, and the frame rate is set to, for example, 0.1 fps to 10 fps, typically 1 fps. By setting a low frame rate, power consumption can be reduced. Note that if a change occurs in the subject being captured and it takes a short time for the subject to return to normal, a relatively high frame rate may be set.

[0141] Next, a reference image is captured (S3). The operation of capturing the reference image corresponds to the operations at times T1 to T4 in the timing chart of FIG.

[0142] Next, a comparison image is captured and an intra-pixel difference calculation is performed (S4). This operation corresponds to the operation from time T5 to time T7 in the timing chart of FIG.

[0143] Next, feature B extraction operation 1 is performed (S5). This operation corresponds to the acquisition of data A by the operation from time T8 to T11 in the timing chart of Fig. 6, as well as the acquisition of data B in an operation without imaging. Note that the weighting applied to the image data here corresponds to the convolution filter of a convolutional neural network (CNN).

[0144] Next, feature B extraction operation 2 is performed (S6). This operation can be performed, for example, by the circuit 301 or an external circuit, and a difference between data A and data B is calculated. Pooling processing may also be performed. By obtaining the difference between data A and data B, it is possible to remove offset components other than the product of the image data (X) and the weighting coefficient (W). In other words, feature B is a feature extracted from the captured image data.

[0145] Next, feature quantities A and B are compared (S7, S8). This operation can be performed, for example, by an external circuit. Note that at some point prior to this, a reference feature quantity A is set (S0). For feature quantity A, for example, a feature quantity such as the shape or pattern of the target object can be used. Therefore, it is preferable to select a convolution filter according to feature quantity A.

[0146] When feature A is detected from feature B by comparing feature B with feature A extracted in S6, the process proceeds to S11, where imaging is performed in normal imaging mode. When feature A is not detected, the process returns to S3 or S4 via counter operations (S9, S10) and the like. The threshold value for determining whether feature A has been detected can be set arbitrarily.

[0147] When feature A is not detected, the reference image is stored in the pixels, so there is no problem in the short term by returning to S4, but if there is a change in the illuminance of natural light or a change over time in the subject, this may affect the feature comparison operation. Therefore, it is preferable to use a counter or the like to return to S3 and capture the reference image again when a set value reaches, for example, 1 minute, 10 minutes, 1 hour, 6 hours, etc. in terms of time. Alternatively, a timer or the like may be used for time control.

[0148] Note that there may be multiple feature amounts A. In this case, it is also possible to set conditions such as whether all of feature amount A has been detected, whether a part of feature amount A has been detected, etc. Alternatively, even if feature amount A is not detected, the operation proceeding to S11 may be performed if there is a change between the reference image and the comparative image.

[0149] 15A and 15B are diagrams for explaining a specific operation in the motion detection mode.

[0150] 15A is a diagram for explaining the operation when feature amount A is detected, and illustrates changes in frame images on the time axis. Note that the subject is a landscape, and feature amount A is data including the features of a bird.

[0151] First, a low frame rate is set, and a reference image is captured in the first frame (corresponding to S3). Note that in the first frame of Fig. 15A, an image of a landscape is illustrated by a dashed line, but in reality, the reference image is held within the pixels, and image data is not output from the imaging device.

[0152] Following the capture of the reference image, a comparison image is captured and a pixel difference calculation (corresponding to S4) is performed for each frame to determine the detection of feature A (corresponding to S8). The nxth frame (n and x are natural numbers, and n>x is a value greater than 1) shown in FIG. 15A shows a case where there is no change in the scenery. The image data that is output is the difference data between the reference image and the comparison image, and image data without change corresponds to all white or all black. If there is no change in the scenery, a frame similar to the nxth frame is repeated.

[0153] Next, if a bird is captured in the nth frame, it is determined that the captured image matches feature amount A including the bird's features, and the system switches to a normal imaging mode with a high frame rate from the n+1th frame onward (S11). The above is the operation when feature amount A is detected.

[0154] Fig. 15B is a diagram for explaining the operation when the feature amount A is not detected. When imaging is started under the same conditions as Fig. 15A, and the feature amount A (bird) is not captured even if a change appears in the scenery, the motion detection mode continues and the mode does not switch to the normal imaging mode. Fig. 15B shows a case where an airship is captured in the nth frame, but since it is determined that the feature amount does not match A, the motion detection mode continues from the n+1th frame onwards.

[0155] In this way, the imaging device according to one embodiment of the present invention can determine the presence or absence of detection by limiting the objects, and therefore, when the imaging device is used in a security camera or the like, the mode switching can be performed only on people, not on dogs, cats, plants, etc.

[0156] This embodiment mode can be appropriately combined with the descriptions of other embodiment modes.

[0157] (Embodiment 2) In this embodiment, a structural example of an imaging device according to one embodiment of the present invention will be described.

[0158] <Structure example> FIG. 16A is a diagram showing an example of the structure of a pixel of an imaging device, which can have a stacked structure of a layer 561 and a layer 563.

[0159] The layer 561 includes the photoelectric conversion device 101. The photoelectric conversion device 101 can include a layer 565a and a layer 565b as shown in Fig. 17A. Note that the layer may be referred to as a region in some cases.

[0160] 17A is a pn junction photodiode, and may use, for example, a p-type semiconductor for the layer 565a and an n-type semiconductor for the layer 565b. Alternatively, the layer 565a may be an n-type semiconductor, and the layer 565b may be a p-type semiconductor.

[0161] The pn junction photodiode can be typically formed using single crystal silicon.

[0162] Furthermore, the photoelectric conversion device 101 included in the layer 561 may be a laminate of layers 566a, 566b, 566c, and 566d as shown in Fig. 17B. The photoelectric conversion device 101 shown in Fig. 17B is an example of an avalanche photodiode, in which the layers 566a and 566d correspond to electrodes, and the layers 566b and 566c correspond to a photoelectric conversion unit.

[0163] Layer 566a is preferably a low resistance metal layer, such as aluminum, titanium, tungsten, tantalum, silver, or a laminate of these.

[0164] The layer 566d is preferably a conductive layer having a high light-transmitting property to visible light. For example, indium oxide, tin oxide, zinc oxide, indium-tin oxide, gallium-zinc oxide, indium-gallium-zinc oxide, graphene, or the like can be used. Note that the layer 566d may be omitted.

[0165] The layers 566b and 566c of the photoelectric conversion unit can be configured as a pn junction photodiode with a photoelectric conversion layer made of, for example, a selenium-based material. It is preferable that the layer 566b is made of a selenium-based material that is a p-type semiconductor, and the layer 566c is made of gallium oxide or the like that is an n-type semiconductor.

[0166] Photoelectric conversion devices using selenium-based materials have the characteristic of high external quantum efficiency for visible light. In such photoelectric conversion devices, avalanche multiplication can be used to increase the amplification of electrons relative to the amount of incident light. In addition, selenium-based materials have a high optical absorption coefficient, which offers the advantage of production in that the photoelectric conversion layer can be made as a thin film. Thin films of selenium-based materials can be formed using vacuum deposition or sputtering methods.

[0167] As the selenium-based material, crystalline selenium such as single crystal selenium or polycrystalline selenium, amorphous selenium, a compound of copper, indium, and selenium (CIS), or a compound of copper, indium, gallium, and selenium (CIGS) can be used.

[0168] The n-type semiconductor is preferably made of a material that has a wide band gap and is transparent to visible light. For example, zinc oxide, gallium oxide, indium oxide, tin oxide, or a mixture of these oxides can be used. These materials also function as a hole injection blocking layer, and can reduce dark current.

[0169] Furthermore, the photoelectric conversion device 101 included in the layer 561 may be a laminate of layers 567a, 567b, 567c, 567d, and 567e as shown in Fig. 17C. The photoelectric conversion device 101 shown in Fig. 17C is an example of an organic photoconductive film, in which the layer 567a is a lower electrode, the layer 567e is a light-transmitting upper electrode, and the layers 567b, 567c, and 567d correspond to photoelectric conversion units.

[0170] One of the layers 567b and 567d of the photoelectric conversion portion can be a hole transport layer, and the other can be an electron transport layer. The layer 567c can be a photoelectric conversion layer.

[0171] For the hole transport layer, for example, molybdenum oxide can be used. For the electron transport layer, for example, C 60 , C 70 or derivatives thereof can be used.

[0172] As the photoelectric conversion layer, a mixed layer (bulk heterojunction structure) of an n-type organic semiconductor and a p-type organic semiconductor can be used.

[0173] For example, a silicon substrate can be used as the layer 563 shown in FIG. 16A. The silicon substrate has Si transistors and the like. Using the Si transistors, in addition to pixel circuits, circuits for driving the pixel circuits, image signal readout circuits, image processing circuits, neural networks, communication circuits, and the like can be formed. Also, memory circuits such as DRAMs (Dynamic Random Access Memory), CPUs (Central Processing Units), MCUs (Micro Controller Units), and the like may be formed. In this embodiment, the above circuits excluding the pixel circuits are called functional circuits.

[0174] For example, some or all of the transistors included in the pixel circuit (pixel 100) and the functional circuits (circuits 201, 301, 302, 303, 304, 305, and the like) described in Embodiment 1 can be provided in the layer 563.

[0175] Also, the layer 563 may be a laminate of multiple layers as shown in FIG. 16B. Although three layers, 563a, 563b, and 563c, are illustrated in FIG. 16B, two layers may be used. Alternatively, the layer 563 may be a laminate of four or more layers. These layers can be laminated using, for example, a bonding process. With this configuration, the pixel circuits and the functional circuits can be distributed among multiple layers and the pixel circuits and the functional circuits can be stacked, so that a small-sized, highly functional imaging device can be manufactured.

[0176] Alternatively, the pixel may have a stacked structure of a layer 561, a layer 562, and a layer 563 as shown in FIG. 16C.

[0177] The layer 562 may include OS transistors. One or more of the functional circuits described above may be formed using OS transistors. Alternatively, one or more of the functional circuits may be formed using the Si transistors in the layer 563 and the OS transistors in the layer 562.

[0178] For example, a normally-off CPU (also referred to as an Noff-CPU) can be realized by using OS transistors and Si transistors. Note that an Noff-CPU is an integrated circuit including normally-off transistors that are in a non-conducting state (also referred to as an off state) even when the gate voltage is 0 V.

[0179] The Noff-CPU can stop the power supply to circuits in the Noff-CPU that are not required to operate, and put the circuits into a standby state. When the power supply is stopped and the circuits are in a standby state, no power is consumed. Thus, the Noff-CPU can minimize power consumption. The Noff-CPU can also retain information necessary for operation, such as setting conditions, for a long period of time even if the power supply is stopped. To return from the standby state, it is only necessary to resume the power supply to the circuit, and there is no need to rewrite the setting conditions, etc. In other words, high-speed return from the standby state is possible. In this way, the Noff-CPU can reduce power consumption without significantly reducing the operating speed.

[0180] Also, layer 562 may be a laminate of multiple layers as shown in Fig. 16D. Although two layers, layers 562a and 563b, are illustrated in Fig. 16D, layer 562 may be a laminate of three or more layers. These layers may be formed by stacking them on layer 563, for example. Alternatively, layer 562 may be formed by bonding a layer formed on layer 563 and a layer formed on layer 561.

[0181] As a semiconductor material used for an OS transistor, a metal oxide having an energy gap of 2 eV or more, preferably 2.5 eV or more, more preferably 3 eV or more can be used. A typical example is an oxide semiconductor containing indium, for example, CAAC-OS or CAC-OS described later can be used. CAAC-OS has stable atoms constituting the crystal, and is suitable for transistors that emphasize reliability. In addition, CAC-OS has high mobility characteristics, and is suitable for transistors that operate at high speed.

[0182] OS transistors have a large energy gap in the semiconductor layer, and therefore exhibit extremely low off-current characteristics of a few yA / μm (current value per 1 μm of channel width). OS transistors also have characteristics different from Si transistors, such as the absence of impact ionization, avalanche breakdown, and short channel effects, and can form highly reliable circuits with high voltage resistance. OS transistors are also less susceptible to variations in electrical characteristics caused by non-uniformity in crystallinity, which is a problem with Si transistors.

[0183] The semiconductor layer of the OS transistor can be, for example, a film represented by an In-M-Zn oxide containing indium, zinc, and M (one or more selected from metals such as aluminum, titanium, gallium, germanium, yttrium, zirconium, lanthanum, cerium, tin, neodymium, and hafnium). The In-M-Zn oxide can be typically formed by a sputtering method. Alternatively, it may be formed by an ALD (atomic layer deposition) method.

[0184] The atomic ratio of the metal elements of the sputtering target used to form the In-M-Zn oxide by the sputtering method preferably satisfies In≧M and Zn≧M. The atomic ratio of the metal elements of such a sputtering target is preferably In:M:Zn=1:1:1, In:M:Zn=1:1:1.2, In:M:Zn=3:1:2, In:M:Zn=4:2:3, In:M:Zn=4:2:4.1, In:M:Zn=5:1:6, In:M:Zn=5:1:7, In:M:Zn=5:1:8, etc. The atomic ratio of the semiconductor layer to be formed includes a variation of ±40% of the atomic ratio of the metal elements contained in the above sputtering target.

[0185] The semiconductor layer is made of an oxide semiconductor with a low carrier density. For example, the semiconductor layer has a carrier density of 1×10 17 / cm 3 Less than or equal to 1×10 15 / cm 3Less than 1×10, more preferably 13 / cm 3 Less than or equal to 1×10 11 / cm 3 Less than 1×10, more preferably 10 / cm 3 Less than 1 x 10 -9 / cm 3 An oxide semiconductor having a carrier density of 1000 or more can be used. Such an oxide semiconductor is called a high-purity intrinsic or substantially high-purity intrinsic oxide semiconductor. The oxide semiconductor has a low density of defect states and is stable in characteristics.

[0186] Note that the composition is not limited to these, and may be appropriate depending on the required semiconductor characteristics and electrical characteristics (field-effect mobility, threshold voltage, etc.) of the transistor. In order to obtain the required semiconductor characteristics of the transistor, it is preferable to appropriately set the carrier density, impurity concentration, defect density, atomic ratio of metal element to oxygen, interatomic distance, density, and the like of the semiconductor layer.

[0187] When silicon or carbon, which is one of the group 14 elements, is contained in the oxide semiconductor constituting the semiconductor layer, oxygen vacancies increase and the semiconductor layer becomes n-type. For this reason, the concentration of silicon or carbon in the semiconductor layer (concentration obtained by secondary ion mass spectrometry) is set to 2×10 18 atoms / cm 3 Less than or equal to 2×10 17 atoms / cm 3 The following applies.

[0188] In addition, when an alkali metal or an alkaline earth metal is bonded to an oxide semiconductor, carriers may be generated, which may increase the off-state current of a transistor. For this reason, the concentration of an alkali metal or an alkaline earth metal in a semiconductor layer (concentration obtained by secondary ion mass spectrometry) is set to 1×10 18 atoms / cm 3 Less than or equal to 2×10 16 atoms / cm 3 To the following:

[0189] In addition, when nitrogen is contained in the oxide semiconductor constituting the semiconductor layer, electrons that act as carriers are generated, which increases the carrier density and makes the semiconductor layer more likely to be n-type. As a result, a transistor using an oxide semiconductor that contains nitrogen is likely to have normally-on characteristics. For this reason, the nitrogen concentration in the semiconductor layer (concentration obtained by secondary ion mass spectrometry) is 5×10 18 atoms / cm 3 It is preferable to do the following:

[0190] Furthermore, when hydrogen is contained in an oxide semiconductor constituting a semiconductor layer, it reacts with oxygen bonded to a metal atom to form water, which may form oxygen vacancies in the oxide semiconductor. When oxygen vacancies are present in a channel formation region in an oxide semiconductor, the transistor may have normally-on characteristics. Furthermore, defects in which hydrogen has entered the oxygen vacancies may function as donors and generate electrons that serve as carriers. In addition, some of the hydrogen may bond with oxygen that is bonded to a metal atom to generate electrons that serve as carriers. Therefore, a transistor using an oxide semiconductor that contains a large amount of hydrogen is likely to have normally-on characteristics.

[0191] A defect in which hydrogen has entered an oxygen vacancy can function as a donor for an oxide semiconductor. However, it is difficult to quantitatively evaluate the defect. Thus, an oxide semiconductor may be evaluated by its carrier concentration instead of its donor concentration. Thus, in this specification and the like, a carrier concentration assuming a state in which no electric field is applied may be used as a parameter of an oxide semiconductor instead of the donor concentration. In other words, the "carrier concentration" described in this specification and the like may be rephrased as the "donor concentration".

[0192] Therefore, it is preferable that hydrogen in the oxide semiconductor be reduced as much as possible. Specifically, the hydrogen concentration in the oxide semiconductor measured by secondary ion mass spectrometry (SIMS) is 1×10 20 atoms / cm 3Less than 1 x 10 19 atoms / cm 3 less than 5×10 18 atoms / cm 3 less than 1×10 18 atoms / cm 3 When an oxide semiconductor in which impurities such as hydrogen are sufficiently reduced is used for a channel formation region of a transistor, stable electrical characteristics can be obtained.

[0193] The semiconductor layer may have a non-single crystal structure. The non-single crystal structure includes, for example, a C-Axis Aligned Crystalline Oxide Semiconductor (CAAC-OS) having crystals oriented along the c-axis, a polycrystalline structure, a microcrystalline structure, or an amorphous structure. Among the non-single crystal structures, the amorphous structure has the highest density of defect states, and the CAAC-OS has the lowest density of defect states.

[0194] An oxide semiconductor film having an amorphous structure has, for example, a disordered atomic arrangement and does not include a crystalline component, or an oxide film having an amorphous structure has, for example, a completely amorphous structure and does not include a crystalline portion.

[0195] The semiconductor layer may be a mixed film having two or more of an amorphous structure region, a microcrystalline structure region, a polycrystalline structure region, a CAAC-OS region, and a single crystal structure region. The mixed film may have a single layer structure or a multilayer structure including two or more of the above-mentioned regions.

[0196] The following describes the structure of a cloud-aligned composite (CAC)-OS, which is one type of non-single crystal semiconductor layer.

[0197] CAC-OS is, for example, a material in which elements constituting an oxide semiconductor are unevenly distributed with a size of 0.5 nm to 10 nm, preferably 1 nm to 2 nm, or thereabouts. Note that, hereinafter, a state in which one or more metal elements are unevenly distributed in an oxide semiconductor and regions containing the metal elements are mixed with a size of 0.5 nm to 10 nm, preferably 1 nm to 2 nm, or thereabouts, is also referred to as a mosaic or patch shape.

[0198] The oxide semiconductor preferably contains at least indium, particularly indium and zinc, and may further contain one or more elements selected from aluminum, gallium, yttrium, copper, vanadium, beryllium, boron, silicon, titanium, iron, nickel, germanium, zirconium, molybdenum, lanthanum, cerium, neodymium, hafnium, tantalum, tungsten, magnesium, and the like.

[0199] For example, the CAC-OS in In-Ga-Zn oxide (In-Ga-Zn oxide among CAC-OS may be specifically referred to as CAC-IGZO) is an indium oxide (hereinafter, InO X1 (X1 is a real number greater than 0).) or indium zinc oxide (hereinafter, In X2 Zinc Y2 O Z2 (X2, Y2, and Z2 are real numbers greater than 0.) and gallium oxide (GaO X3 (X3 is a real number greater than 0). ) or gallium zinc oxide (Ga X4 Zinc Y4 O Z4 (X4, Y4, and Z4 are real numbers greater than 0).) The material is separated into a mosaic shape, and the mosaic shape of InO X1 , or In X2 Zinc Y2 O Z2 However, the structure is such that the particles are uniformly distributed in the film (hereinafter, also referred to as a cloud-like structure).

[0200] In other words, CAC-OS is X3 The region where In is the main component and X2 Zinc Y2 O Z2 , or InO X1 In this specification, for example, when the atomic ratio of In to element M in the first region is greater than the atomic ratio of In to element M in the second region, the first region is said to have a higher In concentration than the second region.

[0201] IGZO is a common name and may refer to a compound made of In, Ga, Zn, and O. A typical example is InGaO 3 (ZnO) m1 (m1 is a natural number), or In (1+x0) Ga (1-x0) O 3 (ZnO) m0 (-1≦x0≦1, m0 is an arbitrary number).

[0202] The crystalline compound has a single crystal structure, a polycrystalline structure, or a CAAC structure, where multiple IGZO nanocrystals have a c-axis orientation and are connected without being oriented in the ab plane.

[0203] On the other hand, CAC-OS is a material composition of oxide semiconductor. CAC-OS is a material composition containing In, Ga, Zn, and O, in which some regions observed as nanoparticles mainly composed of Ga and some regions observed as nanoparticles mainly composed of In are randomly distributed in a mosaic pattern. Therefore, in CAC-OS, the crystal structure is a secondary element.

[0204] Note that the CAC-OS does not include a laminated structure of two or more films with different compositions, such as a two-layer structure consisting of a film mainly containing In and a film mainly containing Ga.

[0205] In addition, GaO X3 The region where In is the main component and X2 Zinc Y2 O Z2 , or InO X1 In some cases, it may be difficult to observe a clear boundary between the region in which the main component is the

[0206] In addition, when one or more elements selected from aluminum, yttrium, copper, vanadium, beryllium, boron, silicon, titanium, iron, nickel, germanium, zirconium, molybdenum, lanthanum, cerium, neodymium, hafnium, tantalum, tungsten, magnesium, etc. are contained instead of gallium, the CAC-OS has a configuration in which some regions observed to be in the form of nanoparticles mainly composed of the metal element and some regions observed to be in the form of nanoparticles mainly composed of In are randomly dispersed in a mosaic pattern.

[0207] CAC-OS can be formed, for example, by a sputtering method under conditions where the substrate is not intentionally heated. When CAC-OS is formed by a sputtering method, any one or more selected from an inert gas (typically argon), oxygen gas, and nitrogen gas may be used as a deposition gas. The lower the flow rate ratio of oxygen gas to the total flow rate of deposition gas during deposition, the more preferable it is, for example, 0% or more and less than 30%, preferably 0% or more and 10% or less.

[0208] CAC-OS has the characteristic that no clear peaks are observed when it is measured using the θ / 2θ scan by the out-of-plane method, which is one of the X-ray diffraction (XRD) measurement methods. In other words, the X-ray diffraction measurement shows that there is no orientation in the ab plane direction or the c axis direction in the measurement area.

[0209] In addition, in the electron beam diffraction pattern obtained by irradiating CAC-OS with an electron beam (also called nano-beam electron beam) with a probe diameter of 1 nm, a ring-shaped region of high brightness (ring region) and multiple bright spots are observed in the ring region. Therefore, the electron beam diffraction pattern shows that the crystal structure of CAC-OS has an nc (nano-crystal) structure that does not have orientation in the planar direction and cross-sectional direction.

[0210] For example, in the case of CAC-OS of In-Ga-Zn oxide, EDX mapping obtained using EDX (Energy Dispersive X-ray spectroscopy) revealed that GaO X3 The region where In is the main component and X2 Zinc Y2 O Z2 , or InO X1 It can be seen that the region in which the main component is the crystalline silicon is unevenly distributed and mixed.

[0211] CAC-OS has a different structure from IGZO compounds in which metal elements are uniformly distributed, and has different properties from IGZO compounds. X3 The main components are In X2 Zinc Y2 O Z2 , or InO X1 The structure is such that the regions having each element as the main component are separated from each other in a mosaic pattern.

[0212] Here, In X2 Zinc Y2 O Z2 , or InO X1 The area where GaO is the main component X3 This region has a higher electrical conductivity than the region where In is the main component. X2 Zinc Y2 O Z2 , or InO X1 When carriers flow through the region where In is the main component, the conductivity of the oxide semiconductor is expressed.X2 Zinc Y2 O Z2 , or InO X1 When the region mainly composed of is distributed in a cloud shape in the oxide semiconductor, high field-effect mobility (μ) can be achieved.

[0213] On the other hand, GaO X3 The region where the main components are In X2 Zinc Y2 O Z2 , or InO X1 This region has higher insulating properties than the region where GaO is the main component. X3 When a region containing these as main components is distributed in an oxide semiconductor, leakage current can be suppressed and favorable switching operation can be achieved.

[0214] Therefore, when CAC-OS is used in a semiconductor device, GaO X3 Insulation caused by X2 Zinc Y2 O Z2 , or InO X1 The conductivity caused by the MOSFET and the MOSFET's resistance to the MOSFET's resistance work in a complementary manner, resulting in a high on-state current (I on ), and high field-effect mobility (μ).

[0215] In addition, semiconductor elements using CAC-OS have high reliability, making CAC-OS suitable as a constituent material for various semiconductor devices.

[0216] <Laminated structure 1> Next, the laminated structure of the imaging device will be described using a cross-sectional view. Note that the elements such as the insulating layer and the conductive layer shown below are examples, and other elements may be further included. Alternatively, some of the elements shown below may be omitted. Also, the laminated structure shown below can be formed using a bonding process, a polishing process, etc., as necessary.

[0217] FIG. 18 is an example of a cross-sectional view of a laminate having layers 560, 561, and a layer 563, with a bonding surface between layers 563a and 563b that constitute layer 563.

[0218] <layer 563b> The layer 563b has a functional circuit provided on the silicon substrate 611. Here, a capacitor 202, a transistor 203, and a transistor 204 included in the circuit 201 are shown as parts of the functional circuit. One electrode of the capacitor 202, one of the source or drain of the transistor 203, and the gate of the transistor 204 are electrically connected to each other.

[0219] The layer 563b includes a silicon substrate 611 and insulating layers 612, 613, 614, 615, 616, 617, and 618. The insulating layer 612 functions as a protective film. The insulating layers 613, 613, 616, and 617 function as an interlayer insulating film and a planarizing film. The insulating layer 615 functions as a dielectric layer of the capacitor 202. The insulating layer 618 and the conductive layer 619 function as bonding layers. The conductive layer 619 is electrically connected to one electrode of the capacitor 202.

[0220] The protective film may be, for example, a silicon nitride film, a silicon oxide film, or an aluminum oxide film. The interlayer insulating film and the planarizing film may be, for example, an inorganic insulating film such as a silicon oxide film, or an organic insulating film such as an acrylic resin or a polyimide resin. The dielectric layer of the capacitor may be, for example, a silicon nitride film, a silicon oxide film, or an aluminum oxide film. The lamination layer will be described later.

[0221] The conductor that can be used as wiring, electrodes, and plugs for electrical connection between devices may be a metal element selected from aluminum, chromium, copper, silver, gold, platinum, tantalum, nickel, titanium, molybdenum, tungsten, hafnium, vanadium, niobium, manganese, magnesium, zirconium, beryllium, indium, ruthenium, iridium, strontium, lanthanum, etc., or an alloy containing the above-mentioned metal elements as components, or an alloy combining the above-mentioned metal elements, etc. The conductor is not limited to a single layer, and may be multiple layers composed of different materials.

[0222] <Layer 563a> The layer 563a includes elements of the pixel 100. Here, the transistor 102 and the transistor 108 are shown as some of the elements of the pixel 100. In the cross-sectional view shown in FIG. 18, the electrical connection between them is not shown.

[0223] The layer 563a is provided with a silicon substrate 632, insulating layers 631, 633, 634, 635, 637, and 638, and conductive layers 636 and 639.

[0224] The insulating layer 631 and the conductive layer 639 function as bonding layers. The insulating layers 634, 635, and 637 function as an interlayer insulating film and a planarizing film. The insulating layer 633 functions as a protective film. The insulating layer 638 functions to insulate the silicon substrate 632 and the conductive layer 639. The insulating layer 638 can be formed of the same material as the other insulating layers. The insulating layer 638 may be formed of the same material as the insulating layer 631.

[0225] The conductive layer 639 is electrically connected to the other of the source and the drain of the transistor 108 and the conductive layer 619. In addition, the conductive layer 636 is electrically connected to the wiring 113 (see FIG. 3A).

[0226] The Si transistor shown in Fig. 18 is a fin type having a channel formation region in a silicon substrate (silicon substrates 611 and 632). A cross section in the channel width direction (a cross section of A1-A2 shown in layer 563a in Fig. 18) is shown in Fig. 19A. The Si transistor may be a planar type as shown in Fig. 19B.

[0227] 19C, it may be a transistor having a silicon thin-film semiconductor layer 545. The semiconductor layer 545 may be, for example, single crystal silicon (SOI (Silicon on Insulator)) formed on an insulating layer 546 on a silicon substrate 611.

[0228] <layer 561> The layer 561 has a photoelectric conversion device 101. The photoelectric conversion device 101 can be formed on the layer 563a. Fig. 18 shows a configuration in which the organic photoconductive film shown in Fig. 17C is used as the photoelectric conversion layer for the photoelectric conversion device 101. Here, the layer 567a is a cathode, and the layer 567e is an anode.

[0229] Layer 561 is provided with insulating layers 651, 652, 653, 654 and a conductive layer 655.

[0230] The insulating layers 651, 653, and 654 function as an interlayer insulating film and a planarizing film. The insulating layer 654 is provided to cover the end of the photoelectric conversion device 101, and also functions to prevent a short circuit between the layer 567e and the layer 567a. The insulating layer 652 functions as an element isolation layer. It is preferable to use an organic insulating film or the like as the element isolation layer.

[0231] The layer 567a corresponding to the cathode of the photoelectric conversion device 101 is electrically connected to one of the source and drain of the transistor 102 included in the layer 563a. The layer 567e corresponding to the anode of the photoelectric conversion device 101 is electrically connected to the conductive layer 636 included in the layer 563a through the conductive layer 655.

[0232] <layer 560> The layer 560 is formed on the layer 561. The layer 560 includes a light-shielding layer 671, an optical conversion layer 672, and a microlens array 673.

[0233] The light-shielding layer 671 can suppress the inflow of light into adjacent pixels. A metal layer such as aluminum or tungsten can be used for the light-shielding layer 671. In addition, the metal layer and a dielectric film having a function as an anti-reflection film may be laminated.

[0234] A color filter can be used for the optical conversion layer 672. A color image can be obtained by assigning colors such as red (red), green (G), blue (B), yellow (Y), cyan (C), and magenta (M) to the color filter for each pixel.

[0235] Furthermore, if a wavelength cut filter is used in the optical conversion layer 672, an imaging device capable of obtaining images in various wavelength regions can be obtained.

[0236] For example, an infrared imaging device can be formed by using a filter that blocks light with wavelengths equal to or shorter than visible light in the optical conversion layer 672. A far-infrared imaging device can be formed by using a filter that blocks light with wavelengths equal to or shorter than near-infrared light in the optical conversion layer 672. An ultraviolet imaging device can be formed by using a filter that blocks light with wavelengths equal to or longer than visible light in the optical conversion layer 672.

[0237] Furthermore, if a scintillator is used for the optical conversion layer 672, an imaging device can be provided that obtains an image that visualizes the intensity of radiation used in an X-ray imaging device or the like. When radiation such as X-rays that has passed through a subject is incident on the scintillator, it is converted into light (fluorescence) such as visible light or ultraviolet light by the photoluminescence phenomenon. Then, image data is obtained by detecting the light with the photoelectric conversion device 101. An imaging device having such a configuration may also be used for a radiation detector or the like.

[0238] Scintillators contain materials that absorb the energy of radiation such as X-rays or gamma rays and emit visible or ultraviolet light when irradiated with the radiation. For example, Gd 2 O 2 S: Tb, Gd 2 O 2 S: Pr, Gd 2 O 2 S:Eu, BaFCl:Eu, NaI, CsI, CaF 2 , BaF 2 , CeF 3 , LiF, LiI, ZnO, etc. dispersed in resin or ceramics can be used.

[0239] A microlens array 673 is provided on the optical conversion layer 672. Light passing through each lens of the microlens array 673 passes through the optical conversion layer 672 directly below, and is irradiated onto the photoelectric conversion device 101. By providing the microlens array 673, concentrated light can be incident on the photoelectric conversion device 101, so that photoelectric conversion can be performed efficiently. The microlens array 673 is preferably formed of a resin or glass that is highly translucent to visible light.

[0240] <Lamination> Next, the bonding of the layer 563b and the layer 563a will be described.

[0241] The layer 563b is provided with an insulating layer 618 and a conductive layer 619. The conductive layer 619 has a region buried in the insulating layer 618. The surfaces of the insulating layer 618 and the conductive layer 619 are flattened so that they are at the same height.

[0242] The layer 563a is provided with an insulating layer 631 and a conductive layer 639. The conductive layer 639 has a region buried in the insulating layer 631. The surfaces of the insulating layer 631 and the conductive layer 639 are flattened so that they are at the same height.

[0243] Here, the conductive layer 619 and the conductive layer 639 preferably contain the same metal element as a main component. The insulating layer 618 and the insulating layer 631 preferably contain the same component.

[0244] For example, Cu, Al, Sn, Zn, W, Ag, Pt, or Au can be used for the conductive layers 619 and 639. Cu, Al, W, or Au is preferably used because of ease of bonding. Silicon oxide, silicon oxynitride, silicon nitride oxide, silicon nitride, titanium nitride, or the like can be used for the insulating layers 618 and 631.

[0245] That is, the same metal material as described above is preferably used for the conductive layer 619 and the conductive layer 639. The same insulating material as described above is preferably used for the insulating layer 618 and the insulating layer 631. With this structure, the layer 563b and the layer 563a can be bonded to each other at a boundary between them.

[0246] The conductive layers 619 and 639 may have a multi-layer structure with multiple layers, in which case the surface layers (joint surfaces) may be made of the same metal material. The insulating layers 618 and 631 may also have a multi-layer structure with multiple layers, in which case the surface layers (joint surfaces) may be made of the same insulating material.

[0247] This bonding makes it possible to obtain electrical connection between the conductive layer 619 and the conductive layer 639. Also, it makes it possible to obtain a connection between the insulating layer 618 and the insulating layer 631 that has sufficient mechanical strength.

[0248] To bond metal layers together, a surface activation bonding method can be used, in which oxide films and adsorbed layers of impurities on the surfaces are removed by sputtering or other methods, and cleaned and activated surfaces are brought into contact with each other to bond them. Alternatively, a diffusion bonding method can be used, in which surfaces are bonded together using a combination of temperature and pressure. Both methods involve bonding at the atomic level, resulting in excellent bonding not only electrically but also mechanically.

[0249] In addition, for bonding insulating layers, a hydrophilic bonding method can be used in which high flatness is achieved by polishing, etc., and then the surfaces that have been hydrophilically treated with oxygen plasma or the like are brought into contact with each other to form a temporary bond, and then the final bond is achieved by dehydrating them through heat treatment. Hydrophilic bonding also produces bonds at the atomic level, and therefore can provide mechanically excellent bonds.

[0250] When bonding the layer 563b and the layer 563a, an insulating layer and a metal layer are mixed on each bonding surface, so that, for example, a surface activated bonding method and a hydrophilic bonding method may be combined.

[0251] For example, a method of cleaning the surface after polishing, subjecting the surface of the metal layer to an anti-oxidation treatment, and then subjecting it to a hydrophilic treatment and bonding can be used. The surface of the metal layer may be made of a resistant metal such as Au and then subjected to a hydrophilic treatment. Note that bonding methods other than the above-mentioned methods may also be used.

[0252] By the above-mentioned bonding, the circuit 201 included in the layer 563b can be electrically connected to the elements of the pixel 100 included in the layer 563a.

[0253] <Modification of Laminate Structure 1> FIG. 20 shows a modified example of the stacked structure shown in FIG. 18, in which the configuration of the photoelectric conversion device 101 in the layer 561 and a portion of the configuration of the layer 563a are different, and a bonding surface is also provided between the layer 561 and the layer 563a.

[0254] Layer 561 includes photovoltaic device 101, insulating layers 661, 662, 664, 665 and conductive layers 135, 136.

[0255] The photoelectric conversion device 101 is a pn junction type photodiode formed on a silicon substrate, and has a layer 565b corresponding to a p-type region and a layer 565a corresponding to an n-type region. The photoelectric conversion device 101 is a buried type photodiode, and a thin p-type region (part of the layer 565b) provided on the front side (current extraction side) of the layer 565a can suppress dark current and reduce noise.

[0256] The insulating layer 661 and the conductive layers 135 and 136 function as bonding layers. The insulating layer 662 functions as an interlayer insulating film and a planarizing film. The insulating layer 664 functions as an element isolation layer. The insulating layer 665 functions to suppress the outflow of carriers.

[0257] The silicon substrate is provided with openings for separating pixels, and the insulating layer 665 is provided on the upper surface of the silicon substrate and in the openings. By providing the insulating layer 665, it is possible to prevent carriers generated in the photoelectric conversion device 101 from flowing out to adjacent pixels. The insulating layer 665 also has a function of preventing the intrusion of stray light. Therefore, the insulating layer 665 can prevent color mixing. An anti-reflection film may be provided between the upper surface of the silicon substrate and the insulating layer 665.

[0258] The element isolation layer can be formed by using a LOCOS (LOCal Oxidation of Silicon) method. Alternatively, it may be formed by using a STI (Shallow Trench Isolation) method or the like. For example, an inorganic insulating film such as silicon oxide or silicon nitride, or an organic insulating film such as polyimide or acrylic can be used as the insulating layer 665. The insulating layer 665 may have a multi-layer structure.

[0259] A layer 565a (n-type region, corresponding to a cathode) of the photoelectric conversion device 101 is electrically connected to the conductive layer 135. A layer 565b (p-type region, corresponding to an anode) is electrically connected to the conductive layer 136. The conductive layers 135 and 136 have regions buried in an insulating layer 661. The surfaces of the insulating layer 661 and the conductive layers 135 and 136 are flattened so that they are at the same height.

[0260] In the layer 563a, an insulating layer 638 is formed over the insulating layer 637. In addition, a conductive layer 133 electrically connected to one of the source and the drain of the transistor 102, and a conductive layer 134 electrically connected to the conductive layer 636 are formed.

[0261] The insulating layer 638 and the conductive layers 133 and 134 function as bonding layers. The conductive layers 133 and 134 have a region buried in the insulating layer 638. The surfaces of the insulating layer 638 and the conductive layers 133 and 134 are flattened so that they are at the same height.

[0262] Here, the conductive layers 133, 134, 135, and 136 are the same bonding layers as the above-mentioned conductive layers 619 and 639. The insulating layers 638 and 661 are the same bonding layers as the above-mentioned insulating layers 618 and 631.

[0263] Therefore, by bonding the conductive layer 133 and the conductive layer 135 together, it is possible to electrically connect the layer 565a (n-type region, corresponding to the cathode) of the photoelectric conversion device to one of the source or drain of the transistor 102. Also, by bonding the conductive layer 134 and the conductive layer 136 together, it is possible to electrically connect the layer 565b (p-type region, corresponding to the anode) of the photoelectric conversion device to the wiring 113 (see FIG. 3). Also, by bonding the insulating layer 638 and the insulating layer 661 together, it is possible to electrically and mechanically bond the layer 561 to the layer 563a.

[0264] <Laminated structure 2> 21 is an example of a cross-sectional view of a stack having layers 560, 561, 562, and 563 and no bonding surface. A Si transistor is provided in layer 563. An OS transistor is provided in layer 562. Here, an example is described in which components of a memory circuit are provided in layers 562 and 563, and a driver circuit for the memory circuit is provided in layer 563. Note that the configurations of layers 561 and 560 are the same as those shown in FIG. 18, and therefore description thereof is omitted here.

[0265] <layer 563> The layer 563 has functional circuits provided over a silicon substrate 611. Here, a transistor 251 included in a driver circuit of a memory circuit and transistors 252 and 253 included in the memory circuit are shown as part of the functional circuits.

[0266] <layer 562b> A layer 562b is formed over the layer 563. The layer 562b includes an OS transistor, shown here as transistor 254, as part of a memory circuit.

[0267] The layer 562b is provided with insulating layers 621, 622, 623, 624, 625, 626, 628, and 629. In addition, a conductive layer 627 is provided. The conductive layer 627 can be electrically connected to the wiring 113 (see FIG. 3).

[0268] The insulating layer 621 functions as a blocking layer. The insulating layers 622, 623, 625, 626, 628, and 629 function as interlayer insulating films and planarizing films. The insulating layer 624 functions as a protective film.

[0269] The blocking layer is preferably a film having a function of preventing hydrogen diffusion. In a Si device, hydrogen is required to terminate dangling bonds, but hydrogen near an OS transistor is one of the factors that generate carriers in an oxide semiconductor layer, thereby reducing reliability. Therefore, a hydrogen blocking film is preferably provided between a layer in which a Si device is formed and a layer in which an OS transistor is formed.

[0270] As the blocking film, for example, aluminum oxide, aluminum oxynitride, gallium oxide, gallium oxynitride, yttrium oxide, yttrium oxynitride, hafnium oxide, hafnium oxynitride, yttria stabilized zirconia (YSZ), or the like can be used.

[0271] Here, the memory circuit included in the layer 563 and the layer 562b has a memory cell including a transistor 254, a transistor 253, and a transistor 252. One of the source and the drain of the transistor 254 is electrically connected to the gate of the transistor 253. The gate of the transistor 254 is electrically connected to a transistor 251 included in a driver circuit of the memory circuit.

[0272] The memory cell has a gate of the transistor 253 as a data storage portion, and data is written by the transistor 254. The memory cell is read by turning on the transistor 252. The data storage portion can be extended by using an OS transistor with low off-state current as the transistor 254 connected to the data storage portion. For details, refer to the description of NOSRAM and the like in the following embodiment.

[0273] Fig. 22A shows details of an OS transistor. The OS transistor shown in Fig. 22A has a self-aligned structure in which an insulating layer is provided over a stack of an oxide semiconductor layer and a conductive layer, and a source electrode 705 and a drain electrode 706 are formed by providing openings that reach the oxide semiconductor layer.

[0274] The OS transistor can have a structure including a channel formation region, a source region 703, and a drain region 704 formed in the oxide semiconductor layer, as well as a gate electrode 701 and a gate insulating film 702. At least the gate insulating film 702 and the gate electrode 701 are provided in the opening. An oxide semiconductor layer 707 may be further provided in the groove.

[0275] As shown in FIG. 22B, the OS transistor may have a self-aligned structure in which a source region 703 and a drain region 704 are formed in a semiconductor layer using a gate electrode 701 as a mask.

[0276] Alternatively, as shown in FIG. 22C, it may be a non-self-aligned top-gate transistor having a region where the source electrode 705 or the drain electrode 706 overlaps with the gate electrode 701.

[0277] Although the OS transistor has a structure having a back gate 535, it may have a structure without a back gate. The back gate 535 may be electrically connected to the front gate of the transistor provided opposite to the back gate, as shown in the cross-sectional view of the transistor in the channel width direction in Fig. 22D. Note that Fig. 22D shows the cross section of the transistor taken along B1-B2 in Fig. 22A as an example, but the same applies to transistors having other structures. In addition, a fixed potential different from that of the front gate may be supplied to the back gate 535.

[0278] <Layer 562a> The layer 562a is formed on the layer 562b. The layer 562a includes elements of the pixel 100 including OS transistors. Here, the transistors 102 and 103 are shown as some of the elements of the pixel 100.

[0279] The layer 562a is provided with insulating layers 641, 642, 643, 644, 645, and 647. In addition, a conductive layer 646 is provided.

[0280] The insulating layers 641, 642, 644, 645, and 647 function as interlayer insulating films and planarizing films. The insulating layer 643 functions as a protective film.

[0281] One of the source and the drain of the transistor 102 is electrically connected to the cathode of the photoelectric conversion device 101 included in the layer 561. The conductive layer 646 is electrically connected to the anode of the photoelectric conversion device 101 included in the layer 561 and the conductive layer 627 included in the layer 562b.

[0282] 21, the pixel circuit included in the layer 562a can be electrically connected to the circuit 201 included in the layer 563. In addition, the circuit 201 can be electrically connected to other functional circuits.

[0283] <Modification of Laminate Structure 2> FIG. 23 shows a modified example of the laminated structure shown in FIG. 22, in which the configuration of the photoelectric conversion device 101 included in the layer 561 and a portion of the configuration of the layer 562a are different, and a bonding surface is provided between the layer 561 and the layer 562a.

[0284] The photoelectric conversion device 101 included in the layer 561 is a pn junction type photodiode formed on a silicon substrate, and has the same configuration as that shown in FIG.

[0285] In the layer 562a, an insulating layer 648 is formed over the insulating layer 647. In addition, a conductive layer 138 electrically connected to one of the source and the drain of the transistor 102, and a conductive layer 139 electrically connected to the conductive layer 646 are formed.

[0286] The insulating layer 648 and the conductive layers 138 and 139 function as bonding layers. The conductive layers 138 and 139 have a region buried in the insulating layer 648. The surfaces of the insulating layer 648 and the conductive layers 133 and 134 are flattened so that they are at the same height.

[0287] Here, the conductive layers 138 and 139 are the same bonding layer as the above-mentioned conductive layers 619 and 639. The insulating layer 648 is the same bonding layer as the above-mentioned insulating layers 618 and 631.

[0288] Therefore, by bonding the conductive layer 138 and the conductive layer 135 together, it is possible to electrically connect the layer 565a (n-type region, corresponding to the cathode) of the photoelectric conversion device to one of the source or drain of the transistor 102. Also, by bonding the conductive layer 139 and the conductive layer 136 together, it is possible to electrically connect the layer 565b (p-type region, corresponding to the anode) of the photoelectric conversion device to the wiring 113 (see FIG. 3). Also, by bonding the insulating layer 648 and the insulating layer 661 together, it is possible to electrically and mechanically bond the layer 561 to the layer 562a.

[0289] When stacking multiple Si devices, polishing and bonding processes are required multiple times. This poses issues such as a large number of steps, the need for dedicated equipment, low yields, and high manufacturing costs. OS transistors can be formed by stacking on a silicon substrate on which devices are already formed, eliminating the need for bonding processes.

[0290] Fig. 24A1 is a perspective view showing the appearance of the upper surface of a package containing an image sensor chip. The package includes a package substrate 410 for fixing an image sensor chip 450 (see Fig. 24A3), a cover glass 420, and an adhesive 430 for bonding the two together.

[0291] 24A2 is an external perspective view of the bottom surface of the package. The bottom surface of the package has a BGA (Ball grid array) with solder balls as bumps 440. Note that the package is not limited to a BGA, and may have an LGA (Land grid array) or a PGA (Pin Grid Array), etc.

[0292] 26A3 is a perspective view of the package with a portion of the cover glass 420 and the adhesive 430 omitted. Electrode pads 460 are formed on the package substrate 410, and the electrode pads 460 and the bumps 440 are electrically connected via through holes. The electrode pads 460 are electrically connected to the image sensor chip 450 by wires 470.

[0293] Fig. 24B1 is a perspective view showing the appearance of the upper surface of a camera module in which an image sensor chip is housed in a lens-integrated package. The camera module includes a package substrate 411 for fixing an image sensor chip 451 (Fig. 24B3), a lens cover 421, and a lens 435. An IC chip 490 (Fig. 24B3) having functions such as a drive circuit for an imaging device and a signal conversion circuit is also provided between the package substrate 411 and the image sensor chip 451, and the camera module is configured as a SiP (System in Package).

[0294] 24B2 is a perspective view of the exterior of the bottom surface of the camera module. The bottom surface and side surfaces of the package substrate 411 are provided with lands 441 for mounting, forming a QFN (Quad flat no-lead package) configuration. Note that this configuration is merely an example, and a QFP (Quad flat package) or the above-mentioned BGA may also be provided.

[0295] 24B3 is a perspective view of the module in which a portion of lens cover 421 and lens 435 is omitted. Land 441 is electrically connected to electrode pad 461, and electrode pad 461 is electrically connected to image sensor chip 451 or IC chip 490 by wire 471.

[0296] By housing the image sensor chip in a package of the above-mentioned type, mounting on a printed circuit board or the like becomes easy, and the image sensor chip can be incorporated into various semiconductor devices and electronic devices.

[0297] This embodiment mode can be appropriately combined with the descriptions of other embodiment modes.

[0298] (Embodiment 3) In this embodiment mode, a memory circuit which can be used as the functional circuit described in Embodiment Mode 2 and can be stacked with a pixel circuit will be described.

[0299] The memory circuit can store image data captured by the pixel circuit or the results of product-sum calculations, and therefore an imaging device having a memory circuit can perform high-speed imaging and calculations.

[0300] 25A is a diagram showing the connections of elements (memory cells 321a, row drivers 312, and column drivers 313) included in the memory circuit 321. It is preferable to use OS transistors as transistors forming the memory cells 321a.

[0301] The memory circuit 321 has m (m is an integer of 1 or more) memory cells 321a in one column and n (n is an integer of 1 or more) memory cells 321a in one row, totaling m×n memory cells 321a arranged in a matrix. In FIG. 25A, the addresses of the memory cells 321a are also indicated. For example, [1,1] indicates the memory cell 321a located at the address in the first row and the first column, and [i,j] (i is an integer of 1 to m, j is an integer of 1 to n) indicates the memory cell 321a located at the address in the i-th row and the j-th column. The number of wirings connecting the memory circuit 321 and the row driver 312 is determined by the configuration of the memory cells 321a, the number of memory cells 321a included in one column, and the like. The number of wirings connecting the memory circuit 321 and the column driver 313 is determined by the configuration of the memory cells 321a, the number of memory cells 321a included in one row, and the like.

[0302] 25B to 25E are diagrams for explaining memory cells 321aA to 321aD that can be applied to the memory cell 321a. In the following description, the bit lines can be connected to a column driver 313. The word lines can be connected to a row driver 312.

[0303] For example, a decoder or a shift register can be used for the row driver 312 and the column driver 313. Note that a plurality of row drivers 312 and a plurality of column drivers 313 may be provided.

[0304] [DOSRAM] 25B shows an example of a circuit configuration of a DRAM-type memory cell 321aA. In this specification and the like, a DRAM using an OS transistor is called a dynamic oxide semiconductor random access memory (DOSRAM). The memory cell 321aA includes a transistor M11 and a capacitor Cs.

[0305] A first terminal of the transistor M11 is connected to a first terminal of the capacitor Cs, a second terminal of the transistor M11 is connected to a wiring BIL, a gate of the transistor M11 is connected to a wiring WL, and a back gate of the transistor M11 is connected to a wiring BGL. A second terminal of the capacitor Cs is connected to a wiring GNDL. The wiring GNDL is a wiring that provides a low-level potential (reference potential).

[0306] The wiring BIL functions as a bit line. The wiring WL functions as a word line. The wiring BGL functions as a wiring for applying a potential to the back gate of the transistor M11. By applying an arbitrary potential to the wiring BGL, the threshold voltage of the transistor M11 can be increased or decreased.

[0307] Data is written and read by applying a high-level potential to the wiring WL, turning on the transistor M11, and electrically connecting the wiring BIL and the first terminal of the capacitor Cs.

[0308] The transistor M11 is preferably an OS transistor. In addition, an oxide semiconductor containing any one of indium, an element M (the element M is one or more of aluminum, gallium, yttrium, and tin), and zinc is preferably used for a semiconductor layer of the OS transistor. In particular, an oxide semiconductor containing indium, gallium, or zinc is preferably used.

[0309] An OS transistor using an oxide semiconductor containing indium, gallium, or zinc has a characteristic of having an extremely small off-state current. By using an OS transistor as the transistor M11, the leakage current of the transistor M11 can be made extremely low. In other words, written data can be held by the transistor M11 for a long time, so that the frequency of refreshing the memory cell can be reduced. In addition, the refresh operation of the memory cell can be made unnecessary.

[0310] [NOSRAM] 25C shows an example of a circuit configuration of a gain cell type (also called "2Tr1C type") memory cell 321aB having two transistors and one capacitor. The memory cell 321aB has a transistor M11, a transistor M3, and a capacitor Cs.

[0311] A first terminal of the transistor M11 is connected to a first terminal of the capacitor Cs, a second terminal of the transistor M11 is connected to the wiring WBL, a gate of the transistor M11 is connected to the wiring WL, and a back gate of the transistor M11 is connected to the wiring BGL. A second terminal of the capacitor Cs is connected to the wiring RL. A first terminal of the transistor M3 is connected to the wiring RBL, a second terminal of the transistor M3 is connected to the wiring SL, and a gate of the transistor M3 is connected to the first terminal of the capacitor Cs.

[0312] The line WBL functions as a write bit line. The line RBL functions as a read bit line. The line WL functions as a word line. The line RL functions as a line for applying a predetermined potential to the second terminal of the capacitor Cs. When writing data and while the data is being held, it is preferable to apply a reference potential to the line RL.

[0313] The wiring BGL functions as a wiring for applying a potential to the back gate of the transistor M11. By applying an arbitrary potential to the wiring BGL, the threshold voltage of the transistor M11 can be increased or decreased.

[0314] Data is written by applying a high-level potential to the wiring WL, turning on the transistor M11, and electrically connecting the wiring WBL and the first terminal of the capacitor Cs. Specifically, when the transistor M11 is in a conductive state, a potential corresponding to the information to be recorded is applied to the wiring WBL, and the potential is written to the first terminal of the capacitor Cs and the gate of the transistor M3. After that, a low-level potential is applied to the wiring WL, turning off the transistor M11, thereby holding the potential of the first terminal of the capacitor Cs and the potential of the gate of the transistor M3.

[0315] Data is read by applying a predetermined potential to the wiring RL and the wiring SL. The current flowing between the source and drain of the transistor M3 and the potential of the first terminal of the transistor M3 are determined by the potential of the gate of the transistor M3 and the potential of the second terminal of the transistor M3, so the potential held in the first terminal of the capacitor Cs (or the gate of the transistor M3) can be read by reading the potential of the wiring RBL connected to the first terminal of the transistor M3. In other words, the information written in this memory cell can be read from the potential held in the first terminal of the capacitor Cs (or the gate of the transistor M3). Or, it can be known whether or not information is written in this memory cell.

[0316] Also, as shown in Fig. 25D, the wiring WBL and the wiring RBL may be combined into a single wiring BIL. The memory cell 321aC shown in Fig. 25D is configured such that the wiring WBL and the wiring RBL of the memory cell 321aB are combined into a single wiring BIL, and the second terminal of the transistor M11 and the first terminal of the transistor M3 are connected to the wiring BIL. In other words, the memory cell 321aC is configured to operate the write bit line and the read bit line as a single wiring BIL.

[0317] Note that it is also preferable to use an OS transistor for the transistor M11 in the memory cell 321aB and the memory cell 321aC. A storage device using an OS transistor for the transistor M11 and using a 2Tr1C type memory cell such as the memory cell 321aB and the memory cell 321aC is called a non-volatile oxide semiconductor random access memory (NOSRAM).

[0318] 25E shows an example of a circuit configuration of a 3-transistor, 1-capacitor gain cell type (also called "3Tr1C type") memory cell 321aD. The memory cell 321aD has a transistor M11, a transistor M5, a transistor M6, and a capacitor Cs.

[0319] A first terminal of the transistor M11 is connected to a first terminal of the capacitor Cs, a second terminal of the transistor M11 is connected to a wiring BIL, a gate of the transistor M11 is connected to a wiring WL, and a back gate of the transistor M11 is electrically connected to a wiring BGL. A second terminal of the capacitor Cs is electrically connected to a first terminal of the transistor M5 and a wiring GNDL. A second terminal of the transistor M5 is connected to a first terminal of the transistor M6, and a gate of the transistor M5 is connected to a first terminal of the capacitor Cs. A second terminal of the transistor M6 is connected to a wiring BIL, and a gate of the transistor M6 is connected to a wiring RL.

[0320] The wiring BIL functions as a bit line, the wiring WL functions as a write word line, and the wiring RL functions as a read word line.

[0321] The wiring BGL functions as a wiring for applying a potential to the back gate of the transistor M11. By applying an arbitrary potential to the wiring BGL, the threshold voltage of the transistor M11 can be increased or decreased.

[0322] Data is written by applying a high-level potential to the wiring WL, turning on the transistor M11, and connecting the wiring BIL to the first terminal of the capacitor Cs. Specifically, when the transistor M11 is in a conductive state, a potential corresponding to the information to be recorded is applied to the wiring BIL, and the potential is written to the first terminal of the capacitor Cs and the gate of the transistor M5. After that, a low-level potential is applied to the wiring WL, turning off the transistor M11, thereby holding the potential of the first terminal of the capacitor Cs and the potential of the gate of the transistor M5.

[0323] Data is read by precharging the wiring BIL with a predetermined potential, then putting the wiring BIL in an electrically floating state, and applying a high-level potential to the wiring RL. Since the wiring RL is at a high-level potential, the transistor M6 is in a conductive state, and the wiring BIL and the second terminal of the transistor M5 are electrically connected. At this time, the potential of the wiring BIL is applied to the second terminal of the transistor M5, and the potential of the second terminal of the transistor M5 and the potential of the wiring BIL change depending on the potential held in the first terminal of the capacitor Cs (or the gate of the transistor M5). Here, by reading the potential of the wiring BIL, the potential held in the first terminal of the capacitor Cs (or the gate of the transistor M5) can be read. In other words, the information written in this memory cell can be read from the potential held in the first terminal of the capacitor Cs (or the gate of the transistor M5). Or, the presence or absence of information written in this memory cell can be known.

[0324] In addition, it is preferable to use an OS transistor for the transistor M11 in the memory cell 321aD as well. The 3Tr1C type memory cell 321aD in which an OS transistor is used as the transistor M11 is one aspect of the above-mentioned NOSRAM. The circuit configuration of the memory cell can be changed as appropriate. Also, Si transistors can be used for the transistors constituting the memory cell.

[0325] (Embodiment 4) In this embodiment mode, a semiconductor device which can be used as the functional circuit described in Embodiment 2 and can be stacked with a pixel circuit will be described.

[0326] The semiconductor device described in this embodiment includes a processor and has a function of controlling the operation of an imaging device. The semiconductor device described in this embodiment is an example of a basic configuration of a processor and its peripheral circuits, and a circuit including the configuration can be called a CPU, MCU, or the like.

[0327] The semiconductor device is a normally-off type, and can perform imaging timing control for the pixel circuits and the driving circuits. A circuit or power supply circuit capable of power gating in the semiconductor device can be switched to a pause state by performing power gating. In addition, only a specific circuit and power supply circuit can be switched to a power-on mode as necessary, and an imaging permission signal can be output to the pixel circuits and the driving circuits. Note that, if there are other necessary processes in the semiconductor device, these can also be executed.

[0328] In an imaging device according to one embodiment of the present invention including the above semiconductor device, power gating can be performed in a motion detection mode, for example, so that power consumption can be reduced.

[0329] The configuration of the semiconductor device and its power management mechanism will be described.

[0330] <Configuration Example 1 of Semiconductor Device> A semiconductor device and its power management will be described with reference to FIG. 26A and FIG. 26B. The semiconductor device shown in FIG. 26A has a power circuit 10 and a processing unit (PU) 20. The PU 20 is a circuit having a function of executing instructions. The PU 20 has a plurality of functional circuits integrated on one chip. The PU 20 has a processor core 30, a power management unit (PMU) 60, a clock control circuit 65, a power switch (PSW) 70, and terminals 80 to 83. FIG. 26A shows an example in which the power circuit 10 is provided on a chip different from the PU 20. The terminal 80 is a terminal to which a power supply potential VDD is input from the power circuit 10. The terminal 81 is a terminal to which a reference clock signal CLKM is input from the outside. The terminal 82 is a terminal to which a signal INT is input from the outside. The signal INT is an interrupt signal that requests interrupt processing. The signal INT is input to the PU 20 and the PMU 60. The terminal 83 is a terminal to which the control signal generated by the PMU 60 is output, and is electrically connected to the power supply circuit 10.

[0331] <Processor core 30, memory circuit 31> The processor core 30 is a circuit capable of processing instructions, and can be called an arithmetic processing circuit. It has a memory circuit 31, a combinational circuit 32, etc., which constitute various functional circuits. For example, the memory circuit 31 is included in a register. D represents an input signal, and Q represents an output signal.

[0332] As shown in FIG. 26B, the memory circuit 31 has a circuit MemC1 and a circuit BKC1. The circuit MemC1 has a function of holding data generated by the processor core 30, and can be configured with, for example, a flip-flop circuit, a latch circuit, or the like. The circuit BKC1 can function as a backup circuit for the circuit MemC1, and is a circuit capable of holding data for a long period of time even if the power supply is cut off or the clock signal is cut off. By having such a memory circuit 31, it becomes possible to perform power gating of the processor core 30. This is because, by saving the data of the circuit MemC1 to the circuit BKC1 in the memory circuit 31 before cutting off the power supply, the state of the processor core 30 at the time of power cut off can be held. When the power supply is resumed, the data held in the circuit BKC1 is written to the circuit MemC1, so that the processor core 30 can be restored to the state at the time of power cut off. Therefore, after the power supply is resumed, the PU 20 can immediately perform normal processing operations.

[0333] The circuit BKC1 has at least a holding circuit having one transistor (MW1) and one capacitance element (CB1). The holding circuit shown in FIG. 26B has a circuit configuration similar to that of a 1T1C (one transistor, one capacitance element) type memory cell of a standard DRAM (dynamic random access memory), and can perform write and read operations in the same way. The charging and discharging of the capacitance element CB1 is controlled by controlling the conductive state of the transistor MW1. By turning off the transistor MW1, the node FN1 is electrically floating. By making the drain current (off current) of the transistor MW1 in the off state extremely small, the fluctuation of the potential of the node FN1 can be suppressed, and therefore the data holding time of the circuit BKC1 can be extended. The data holding time of the circuit BKC1 is determined by the leakage current of the transistor MW1, the capacitance of the capacitance element CB1, and the like. By making the transistor MW1 a transistor with an extremely small off current, it is not necessary to refresh the circuit BKC1 while the PU21 is operating. Therefore, the circuit BKC1 can be used substantially as a nonvolatile storage circuit.

[0334] An OS transistor is preferably used as the transistor MW1. Since an OS transistor has an extremely small off-state current, the circuit BKC1 can substantially function as a nonvolatile memory circuit while the PU20 is in operation.

[0335] The circuit BKC1 writes data using voltage, so it can reduce the write power compared to MRAM (Magnetoresistive RAM), which writes data using current. Also, because data is held by the load capacitance of node FN1, there is no limit to the number of times data can be rewritten, as there is with flash memory.

[0336] In the circuit BKC1, the energy required to write data corresponds to the energy associated with charging and discharging electric charge to the capacitive element CB1. On the other hand, in a storage circuit using a two-terminal storage element such as an MRAM, the energy required to write data corresponds to the energy consumed when a current flows through the storage element. In the MRAM, a current continues to flow during the data writing period, so the energy required to write data is high. Compared to such an MRAM, the circuit BKC1 can reduce the energy consumed in writing data. Therefore, compared to a storage circuit in which the backup circuit is configured with an MRAM, the storage circuit 31 has more opportunities to perform voltage scaling and power gating that can reduce the energy consumed, and therefore can reduce the power consumption of the PU 20.

[0337] <Power management> The PMU 60 has a function of controlling a power gating operation, a clock gating operation, a voltage scaling operation, and the like. More specifically, the PMU 60 has a function of controlling the power supply circuit 10, a function of controlling the memory circuit 31, a function of controlling the clock control circuit 65, and a function of controlling the PSW 70. Therefore, the PMU 60 has a function of generating control signals for controlling these circuits (10, 31, 65, 70). The PMU 60 has a circuit 61. The circuit 61 has a function of measuring time. The PMU 60 has a function of managing the power supply based on data related to time obtained by the circuit 61.

[0338] The PSW70 has a function of controlling the supply of a power supply potential MVDD to the PU20 in accordance with a control signal from the PMU60. Here, the power supply potential supplied to the PU20 via the PSW70 is referred to as the power supply potential VDD. The processor core 30 may have a plurality of power supply domains. In this case, it is sufficient that the PSW70 is capable of independently controlling the power supply to the plurality of power supply domains. The processor core 30 may also have a power supply domain for which power gating is not required. In this case, the power supply potential may be supplied to this power supply domain without going through the PSW70.

[0339] The clock control circuit 65 has a function of receiving the reference clock signal CLKM, generating and outputting a gated clock signal. The clock control circuit 65 has a function of being able to cut off the clock signal to the processor core 30 in accordance with a control signal from the PMU 60. The power supply circuit 10 has a function of being able to change the magnitude of the potential of VDD in accordance with a control signal from the PMU 60.

[0340] The signal SLP output from the processor core 30 to the PMU 60 is a signal that triggers the transition of the processor core 30 to a sleep state. When the signal SLP is input, the PMU 60 generates a control signal for transitioning to a sleep state and outputs it to the functional circuit to be controlled. The power supply circuit 10 makes the MVDD lower than during normal operation based on the control signal of the PMU 60. When a certain period of time has passed in the sleep state, the PMU 60 controls the PSW 70 to cut off the power supply to the processor core 30. When the processor core 30 transitions from the normal state to the sleep state, the PMU 60 performs a voltage scaling operation to lower the power supply potential VDD of the processor core 30. When the period of the sleep state exceeds a set time, the PMU 60 performs a power gating operation to stop the supply of VDD to the processor core 30 in order to further reduce the power consumption of the processor core 30. Next, the power management of the semiconductor device shown in FIG. 26A and FIG. 26B will be described.

[0341] 27A to 27D are diagrams showing a schematic representation of changes in the potential of the power line. The power line is a wiring to which a power supply potential VDD is supplied via a PSW 70. The horizontal axis of the diagram represents the elapsed time from the normal state to the pause state, with t0, t1, etc. representing time. FIG. 27A is an example in which only power gating is performed in the pause state, and FIG. 27B is an example in which only voltage scaling is performed in the pause state. FIG. 27C and FIG. 27D are examples in which voltage scaling and power gating are performed. In the normal state, the magnitude of the power supply potential MVDD supplied from the power supply circuit 10 is VH1.

[0342] In the following, the power supply modes of the PU 20 are classified into three modes: a power-on mode, a power-off mode, and a low power mode. The power-on mode is a mode in which a power supply potential VDD that enables normal processing is supplied to the PU 20. The power-off mode is a mode in which the supply of VDD is stopped by the PSW 70. The low power mode is a mode in which a power supply potential VDD lower than that of the power-on mode is supplied.

[0343] An example of FIG. 27A will be described. At time t0, the processor core 30 starts a process of transitioning to a hibernation state. For example, the memory circuit 31 is backed up. The PMU 60 controls the PSW 70 to cut off the power supply to the processor core 30 at time t1. The power line 35 naturally discharges, and its potential drops to 0V. This allows the leakage current of the processor core 30 in the hibernation state to be significantly reduced, and the power consumption in the hibernation state (hereinafter, sometimes referred to as standby power) can be reduced. When returning to the normal state due to an external interrupt request or the like, the PMU 60 controls the PSW 70 to resume the supply of VDD. Here, the supply of VDD is resumed at time t4. The potential of the power line 35 rises and becomes VH1 at time t6.

[0344] 27B, in order to perform voltage scaling, at time t1, PMU 60 controls power supply circuit 10 to lower the MVDD potential to VH2. The potential of power supply line 35 eventually becomes VH2. At time t4, when power supply potential MVDD returns from VH2 to VH1, the potential of power supply line 35 rises and becomes VH1 at time t5.

[0345] In the example of Fig. 27A, the time required to return from the hibernation state to the normal state (overhead time) is the time required for the potential of the power line 35 to rise from 0V to VH1, and the energy overhead required for return is the energy required to charge the load capacitance of the power line 35 from 0V to VH1. If the period (t1-t4) of the power-off mode is sufficiently long, power gating is effective in reducing the standby power of the PU 20. On the other hand, if the period (t1-t4) is short, the power required to return to the normal state becomes greater than the power that can be reduced by cutting off the power supply, and the effect of power gating cannot be obtained.

[0346] In the voltage scaling example shown in FIG. 27B, the potential of the power line 35 is VH2 in the pause state, so the amount of reduction in standby power is smaller than in the power gating example shown in FIG. 27A. On the other hand, in the example shown in FIG. 27B, the potential of the power line 35 fluctuates less, so the time required to return to the normal state is shorter than in the example shown in FIG. 27A, and the energy required for the return is less. Therefore, in the semiconductor device shown in FIG. 26, in order to reduce the standby power of the PU 20 more efficiently, power management that combines power gating and voltage scaling is possible. Examples of power management are shown in FIG. 27C and FIG. 27D.

[0347] As shown in FIG. 27C, first, in the hibernation state, a voltage scaling operation is performed, and the mode transitions from the power-on mode to the low power mode. As in FIG. 27B, at time t1, the PMU 60 controls the power supply circuit 10 to lower the potential of MVDD to VH2, so that the potential of the power supply line 35 eventually becomes VH2. After a certain period (t1-t3) has elapsed since the transition to the low power mode, the PMU 60 controls the PSW 70 to switch to the power-off mode. The period (t3-t4) is a period during which it is possible to reduce power by cutting off the power supply to the PU 20 by power gating, even if it includes the power consumed to return to the normal state, rather than supplying VH2 to the PU 20.

[0348] For example, the potential VH2 is a power supply potential having a magnitude capable of holding data in the circuit MemC1 of the memory circuit 31, and the potential VH3 is a potential at which the data in the circuit MemC1 is lost. In the PU20 of FIG. 26A, the circuit BKC1 is a circuit capable of holding data even during a period when the supply of power is stopped. By saving the data in the memory circuit 31 to the circuit BKC1 during the period (t0-t1), it is possible to lower the VDD to the potential VH3 at which the data in the circuit MemC1 is lost in the low power mode. This allows the standby power of the PU20 to be further reduced.

[0349] The PMU 60 has a function that can return the PU 20 to the normal state based on an interrupt request or the like. The PMU 60 controls the power supply circuit 10 to boost the magnitude of MVDD to VH1, and also controls the PSW 70 to resume the supply of VDD to the PU 20. The power-on mode is in effect from time t4 onwards. At time t6, the potential of the power supply line 35 becomes stable, and the PU 20 becomes able to operate normally from time t6 onwards.

[0350] 27D shows an example in which an interrupt request to return to normal operation occurs before time t3. From time t2 onwards, the power-on mode is active. At time t2, the PMU 60 controls the power supply circuit 10 to change the magnitude of MVDD to the potential VH1 of the power-on mode. At time t3, the potential of the power line 35 rises to VH1.

[0351] 27C and 27D, in the hibernation state, the time required to return the potential of the power line 35 to VH1 is longer when returning from the power off mode to the power on mode than when returning from the low power mode to the power on mode. Therefore, the PMU 60 has a function to adjust the timing of the operation of returning the processor core 30 from the hibernation state to the normal state according to the power mode. This makes it possible to return the processor core 30 from the hibernation state to the normal state in the shortest time.

[0352] In addition, in the hibernation state, the transition from the low power mode to the power off mode can be made by measuring time with the circuit 61 provided in the PMU 60. When the signal SLP is input from the PU 20, the PMU 60 starts measuring time with the circuit 61. When a predetermined time has elapsed since the low power mode was entered, the PMU 60 transitions to the power off mode. The PSW 70 is turned off by a control signal from the PMU 60, and the supply of VDD is cut off. In this way, it is possible to transition from the low power mode to the power off mode by an interrupt request based on the measurement data of the circuit 61. An example of the power management operation of the PMU 60 will be described below with reference to the flowchart shown in FIG. 28.

[0353] The PU 20 is performing a normal operation. The power supply mode is the power-on mode, and the PMU 60 is in an idle state (S10). The PMU 60 is in the idle state until the signal SLP is input, and executes a save sequence using the input of the signal SLP as a trigger (S11). In the example of the save sequence in FIG. 28, first, the PMU 60 outputs a control signal to the clock control circuit 65 to stop outputting the clock signal (S12). Next, it outputs a control signal to the memory circuit 31 to save data (S13). In the memory circuit 31, the data held in the circuit MemC1 is saved to the circuit BKC1 according to the control signal of the PMU 60. Finally, the PMU 60 controls the power supply circuit 10 to lower the MVDD. These operations cause the power supply mode to transition to the low power supply mode (S14). When the signal SLP is input, the PMU 60 controls the built-in circuit 61 to measure the time Ta of the low power supply mode (S15). The timing for operating the circuit 61 can be any timing while the save sequence is being executed, such as when the signal SLP is input, when a control signal is output to the clock control circuit 65, when data save is started, when data save is completed, when a control signal is output to the power supply circuit 10, etc.

[0354] After the execution of the save sequence, the PMU 60 goes into an idle state (S16) and monitors the input of the signal INT and the measurement time Ta of the clock control circuit 65. When the signal INT is input, the PMU 60 transitions to a restore sequence (S17). vs The PMU 60 determines whether the time Ta exceeds the time T vs If the time T exceeds the time T, the power supply mode is switched to the power off mode (S19), and if the time T does not exceed the time T, the idle state is maintained (S16). vs The time may be set so that the standby power consumption of the processor core 30 can be reduced more by putting the processor core 30 in the power off mode than by putting the processor core 30 in the low power mode.

[0355] In S19, the PMU 60 outputs a control signal to the PSW 70 to cut off the power supply to the processor core 30. After entering the power off mode, the PMU 60 again enters the idle state (S20) and monitors the input of the signal INT (S21). When the signal INT is input, the PMU 60 executes a recovery sequence.

[0356] In the recovery sequence, first, the PMU 60 transitions from the power off mode to the power on mode (S22). The PMU 60 controls the power supply circuit 10 to output a power supply potential for normal operation. The PMU 60 also controls the PSW 70 to resume the supply of VDD to the processor core 30. Next, a control signal is output to the memory circuit 31 to restore the data in the memory circuit 31 (S23). In response to the control signal from the PMU 60, the memory circuit 31 writes back the data held in the circuit BKC1 to the circuit MemC1. The PMU 60 outputs a control signal to the clock control circuit 65 to output a clock signal (S24). In response to the control signal from the PMU 60, the clock control circuit 65 resumes outputting the clock signal.

[0357] When the return sequence is executed from the determination process of S17, the mode returns from the low power mode to the power on mode, and the potential of the power line 35 can be stabilized more quickly than when the return sequence is executed from the determination process of S21. Therefore, in the PMU 60, when transitioning from S17 to the return sequence, the timing of executing S23 is set earlier than when transitioning from S21 to the return sequence. This makes it possible to shorten the time required for the processor core 30 to return from the sleep state to the normal state.

[0358] 26A and 26B, when the PU 20 enters a sleep state, first, a voltage scaling operation is performed to lower the power supply potential supplied to the processor core 30, thereby reducing the leakage current and suppressing the time and energy overhead of the process of returning from the sleep state to the normal state. When the sleep state continues for a certain period of time, a power gating operation is performed to suppress the leakage current of the processor core 30 as much as possible. This makes it possible to reduce the power consumption of the PU 20 in the sleep state without reducing the processing capacity of the PU 20.

[0359] <Configuration Example 2 of Semiconductor Device> FIG. 29A shows a modified example of the semiconductor device of FIG. 26A. The processing unit (PU) 21 shown in FIG. 29A has a configuration in which a cache 40 and a power switch (PSW) 71 are added to the PU 20. The cache 40 is capable of power gating and voltage scaling, similar to the PU 20, and the power mode of the cache 40 changes in conjunction with the power mode of the PU 21. The PSW 71 is a circuit that controls the supply of a power supply potential MVDD to the cache 40, and is controlled by the PMU 60. Here, the power supply potential input to the cache 40 via the PSW 71 is VDD_MEM. The cache 40 receives a control signal from the PMU 60 and a gated clock signal from the clock control circuit 65, similar to the processor core 30.

[0360] <Cache 40> The cache 40 is a storage device having a function of temporarily storing frequently used data. The cache 40 has a memory array 41, a peripheral circuit 42, and a control circuit 43. The memory array 41 has a plurality of memory cells 45. The control circuit 43 controls the operation of the cache 40 in accordance with a request from the processor core 30. For example, it controls the write operation and read operation of the memory array 41. The peripheral circuit 42 has a function of generating a signal to drive the memory array 41 in accordance with a control signal from the control circuit 43. The memory array 41 has memory cells 45 that hold data.

[0361] As shown in FIG. 29B, the memory cell 45 has a circuit MemC2 and a circuit BKC2. The circuit MemC2 is a memory cell that is accessed in normal operation. For example, a memory cell of an SRAM (static random access memory) may be applied. The circuit BKC2 can function as a backup circuit for the circuit MemC2, and is a circuit that can hold data for a long period of time even if the power supply is cut off or the clock signal is cut off. By providing such a memory cell 45, it is possible to perform power gating of the cache 40. Before cutting off the power supply, the memory cell 45 saves data of the circuit MemC2 to BKC2. After the power supply is resumed, the data held in the circuit BKC2 is written back to the circuit MemC2, so that the PU21 can be quickly restored to the state before the power supply was cut off.

[0362] The circuit BKC2 of the memory cell 45 has at least a retention circuit having one transistor (MW2) and one capacitance element (CB2) like the circuit BKC1 of FIG. 26B. That is, the circuit BKC2 has a retention circuit with a configuration similar to that of a 1T1C memory cell of a standard DRAM. An OS transistor with an extremely low off-state current may be used for the transistor MW2 like the transistor MW1. With this configuration, the circuit BKC2 can also suppress the fluctuation of the potential of the node FN2, which is in an electrically floating state, and therefore the circuit BKC2 can retain data for a long period of time. The data retention time of the circuit BKC2 is determined by the leakage current of the transistor MW2, the capacitance of the capacitance element CB2, and the like. By using a transistor with an extremely low off-state current as the transistor MW2, the circuit BKC2 can be used as a nonvolatile memory circuit that does not require a refresh operation.

[0363] In the PU21 shown in Fig. 29A, the PMU60 performs power management in the same manner as in the PU20 (see Fig. 28). In S13 of Fig. 28, a data save operation is performed for the memory circuit 31 and the cache 40. In S19, the PSW70 and PSW71 are controlled to stop the power supply to the processor core 30 and the cache 40. In S22, the PSW70 and PSW71 are controlled to resume the power supply to the processor core 30 and the cache 40. In S23, a data restore operation is performed for the memory circuit 31 and the cache 40.

[0364] Therefore, like the semiconductor device shown in FIG. 26, the semiconductor device shown in FIG. 29 also performs power management that combines voltage scaling and power gating, making it possible to reduce the power consumed by PU21 when it is in a quiescent state without reducing the processing capability of PU21.

[0365] <Processor core configuration example> 30 is an example of a circuit that can be used as the processor core 30. The processor core 180 has a control unit 181, a program counter 182, a pipeline register 183, a pipeline register 184, a register file 185, an arithmetic logic unit (ALU) 186, and a data bus 187. Data is exchanged between the processor core 180 and peripheral circuits such as a PMU and a cache via the data bus 187.

[0366] The control device 181 has a function of decoding and executing instructions included in a program such as an input application by comprehensively controlling the operations of the program counter 182, pipeline register 183, pipeline register 184, register file 185, ALU 186, and data bus 187. The ALU 186 has a function of performing various arithmetic processing such as arithmetic operations and logical operations. The program counter 182 is a register that has a function of storing the address of the next instruction to be executed.

[0367] Pipeline register 183 is a register having a function of temporarily storing instruction data. Register file 185 has a plurality of registers including general-purpose registers, and can store data read from the main memory, data obtained as a result of the arithmetic processing of ALU 186, etc. Pipeline register 184 is a register having a function of temporarily storing data used in the arithmetic processing of ALU 186, data obtained by the arithmetic processing of ALU 186, etc.

[0368] The storage circuit 31 in FIG. 26B is used for a register included in the processor core 180.

[0369] <Example of memory circuit configuration> A more specific example of the configuration of the memory circuit 31 shown in Fig. 26B will be described. Fig. 31 is a circuit diagram showing an example of the configuration of the memory circuit. A memory circuit 190 shown in Fig. 31 functions as a flip-flop circuit.

[0370] A standard flip-flop circuit can be applied to the circuit MemC1, for example, a master-slave type flip-flop circuit. An example of such a configuration is shown in FIG. 31. The flip-flop circuit 110 has transmission gates (TG1, TG2, TG3, TG4, TG5), inverter circuits (INV1, INV2, INV3, INV4), and NAND circuits (NAND1, NAND2). The signal RESET and the signal OSR are control signals output from the PMU 60. The signal OSR and its inverted signal are input to TG5. The clock signal CLK and its inverted signal are input to TG1 to TG4. A clocked inverter circuit may be provided instead of TG1 and INV1. A clocked NAND circuit may be provided instead of TG2 and NAND2. A clocked inverter circuit may be provided instead of TG3 and INV3. TG5 functions as a switch that controls the conduction state between the output node of NAND1 and the node NR1. The node NB1 is electrically connected to an input node of the circuit BKC10, and the node NR1 is electrically connected to an output node of the circuit BKC10.

[0371] The circuit BKC10 shown in FIG. 31 functions as a backup circuit for the flip-flop circuit 110. The circuit BKC10 has a circuit RTC10 and a circuit PCC10. Signals (OSG, OSC, OSR) input to the circuit BKC10 are control signals output from the PMU 60. The power supply potential VSS is a low power supply potential, and may be, for example, a ground potential (GND) or 0V. The power supply potential VSS and the power supply potential VDD are input to the flip-flop circuit 110 as in the circuit BKC1. In the memory circuit 190, the supply of VDD is managed by the PMU 60.

[0372] The circuit RTC10 has transistors MW1, MA1, and MR1, a node FN1, and a node NK1. The circuit RTC10 has a function of retaining data, and is configured here as a memory circuit with a 3T-type gain cell structure. The transistor MW1 is a write transistor and an OS transistor. The transistor MR1 is a read transistor, and the transistor MA1 is an amplifying transistor and a read transistor. The data is retained at the node FN1. The node NK1 is a data input node. The node NR1 is a data output node of the circuit RTC10.

[0373] 31 shows a configuration example in which the circuit BKC10 reads data from the slave latch circuit of the flip-flop circuit 110 in a save operation, and writes the held data back to the master latch circuit in a restore operation. The data to be saved may be data from the master latch circuit. Also, the data may be restored to the slave latch circuit. In this case, TG5 may be provided in the slave latch circuit.

[0374] Moreover, the transistors MR1 ​​and MA1 of the circuit RTC10 may be either n-type or p-type, and the potential of the signal OSR and the level of the power supply potential supplied to the transistor MA1 may be changed depending on the conductivity type of the transistors MR1 ​​and MA1. Also, the logic circuit of the flip-flop circuit 110 may be set appropriately. For example, when the transistors MR1 ​​and MA1 are p-type transistors, NAND1 and INV3 may be swapped in the master latch circuit, and INV2 and NAND2 may be swapped in the slave latch circuit. Also, VDD may be input to the transistor MA1 instead of VSS.

[0375] The circuit BKC10 writes data using voltage, so it can reduce the write power compared to MRAM, which writes data using current. Also, since data is held by the load capacitance of the node FN1, there is no limit to the number of times data can be rewritten, as there is in flash memory.

[0376] In the circuit RTC10, the energy required to write data corresponds to the energy associated with charging and discharging electric charge to the capacitive element CB1. On the other hand, in a memory circuit using a two-terminal memory element such as MRAM, the energy required to write data corresponds to the energy consumed when a current flows through the memory element. Therefore, compared to the case of using MRAM or the like in which a current continues to flow during the data writing period, the circuit BKC10 can reduce the energy consumed by saving data. Therefore, by providing the circuit BKC10 in the backup circuit, it is possible to shorten the BET (Break Even Time) compared to the case of providing MRAM. As a result, the opportunities for power gating that can reduce the energy consumed are increased, and the power consumption of the semiconductor device can be reduced.

[0377] The circuit PCC10 includes a transistor MC1 and a transistor MC2. The circuit PCC10 has a function of precharging the node FN1. The circuit PCC10 does not necessarily have to be provided. As will be described later, by providing the circuit PCC10, the data save time of the circuit BKC10 can be shortened.

[0378] <Memory circuit operation example> FIG. 32 is a timing chart showing an example of the operation of the memory circuit 190, and shows the waveforms of the control signals (SLP, RESET, CLK, OSG, OSC, and OSR) and changes in the power supply potential VDD and the potentials of the node FN1 and the node NR1.

[0379] [Normal operation] A power supply potential VDD and a signal CLK are supplied to the memory circuit 190. The flip-flop circuit 110 functions as a sequential circuit. Since the signal RESET is maintained at a high level, NAND1 and NAND2 function as inverter circuits. In the circuit BKC10, the transistor MC1 is in an off state, and the transistors MC2 and MW1 are in an on state, so that the potential of the node FN1 is precharged to a high level.

[0380] [Data evacuation] First, the clock signal CLK is stopped. This stops rewriting the data at the node NB1. In the example of FIG. 32, the potential level of the node NB1 is low ("0") when the potential of the node NR1 is high ("1"), and is high ("1") when the potential is low ("0"). During the period when the signal OSC is high, the data at the node NB1 is saved to the node FN1. Specifically, since the transistors MC1 and MW1 are on, the nodes FN1 and NB1 are electrically connected. By setting the signal OSG to low and turning off the transistor MW1, the node FN1 is electrically floating, and the circuit BKC10 is in a data holding state. The potential of the node FN1 is high when the node NR1 is low ("0"), and is low when the node NR1 is high ("1").

[0381] The data evacuation ends when the signal OSG is set to low level, so the voltage scaling operation of the PU20 can be performed immediately after the signal OSG is set to low level. Also, because the node FN1 is precharged to high level by the transistor MC2 during normal operation, the data evacuation operation that sets the node FN1 to high level does not involve the movement of charge at the node FN1. This allows the circuit BKC10 to complete the evacuation operation in a short time.

[0382] In the data save operation, it is sufficient that the signal CLK is inactive. In the example of FIG. 32, the potential of the signal CLK is set to a low level, but it may also be set to a high level.

[0383] [Voltage scaling, low power mode] In conjunction with the falling edge of the signal OSC, the PMU 60 performs a voltage scaling operation, which causes the storage circuit 190 to transition to a low power mode.

[0384] [Power gating, power off mode] When a certain period of time has elapsed since the transition to the low power mode, the PMU 60 performs a power gating operation and puts the storage circuit 190 into the power off mode.

[0385] [Power on mode] In response to the interrupt request, the PMU 60 returns the memory circuit 190 to the power-on mode. In the example of Fig. 32, when the potential of the power supply line that supplies VDD becomes stable, the signal CLK becomes high level.

[0386] [Data recovery] A data recovery operation is performed while the signal OSR is at a high level. By setting the signal RESET to a high level, the potential of the node NR1 is precharged to a high level ("1"). By setting the signal OSR to a high level, TG5 is set to a high impedance state and the transistor MR1 is set to a conductive state. The conductive state of the transistor MA1 is determined by the potential of the node FN1. If the node FN1 is at a high level, the transistor MA1 is in a conductive state, so that the potential of the node NR1 drops to a low level ("0"). If the node FN1 is at a low level, the potential of the node NR1 is maintained at a high level. In other words, the state of the flip-flop circuit 110 is restored to the state before the transition to the pause state.

[0387] As described above, the rising edges of the signals RESET and OSR allow high-level data to be written back to the node NR1, thereby enabling the storage circuit 190 to shorten the recovery operation period.

[0388] 32 shows an example of returning from the power-off mode to the power-on mode. When returning from the low power mode to the power-on mode, a period T on In this case, it is better to make the signal OSR rise earlier than when returning from the power off mode.

[0389] [Normal operation] By resuming the supply of the signal CLK, the normal operation is resumed. By making the signal OSG high, the node FN1 is precharged by the circuit PCC10 and goes to high.

[0390] <cache> An example in which the cache 40 is configured using an SRAM will be described below.

[0391] <Memory cell configuration example> An example of the configuration of a cache memory cell is shown in Fig. 33. The memory cell 120 shown in Fig. 33 has a circuit SMC20 and a circuit BKC20. The circuit SMC20 may have a circuit configuration similar to that of a standard SRAM memory cell. The circuit SMC20 shown in Fig. 33 has an inverter circuit INV21, an inverter circuit INV22, a transistor M21, and a transistor M22.

[0392] The circuit BKC20 functions as a backup circuit for the circuit SMC20. The circuit BKC20 has a transistor MW11, a transistor MW12, and a capacitance element CB11 and a capacitance element CB12. The transistors MW11 and MW12 are OS transistors. The circuit SMC20 has two 1T1C type holding circuits, and data is held at nodes SN1 and SN2, respectively. The holding circuit made up of the transistor MW11 and the capacitance element CB11 has a function of backing up data at node NET1. The holding circuit made up of the transistor MW12 and the capacitance element CB12 has a function of backing up data at node NET2.

[0393] The memory cell 120 is supplied with power supply potentials VDDMC and VSS. The memory cell 120 is electrically connected to wirings (WL, BL, BLB, BRL). A signal SLC is input to the wiring WL. When writing data, a data signal D and a data signal DB are input to the wirings BL and BLB. Data is read by detecting the potentials of the wirings BL and BLB. A signal OSS is input to the wiring BRL. The signal OSS is a signal input from the PMU 60.

[0394] <Memory cell operation example> The following describes an example of the operation of the memory cell 120. FIG.

[0395] [Normal operation] An access request is made to the circuit MemC20, and data is written or read. In the circuit BKC20, the signal OSS is at a low level, so that the nodes SN1 and SN2 are in an electrically floating state and are in a data holding state. In the example of FIG. 34, the potential of the node SN1 is at a low level ("0"), and the potential of the node SN2 is at a high level ("1").

[0396] [Data evacuation] When the signal OSS goes high, the transistors MW11 and MW12 are turned on, and the nodes SN1 and SN2 are at the same potential levels as the nodes NET1 and NET2, respectively. In the example of Fig. 34, the potentials of the nodes SN1 and SN2 are high and low, respectively. The signal OSS goes low, the circuit BKC20 goes into a data holding state, and the data save operation ends.

[0397] [Voltage scaling, low power mode] In conjunction with the falling edge of the signal OSS, the PMU 60 performs a voltage scaling operation, which causes the cache 40 to transition to a low power mode.

[0398] [Power gating, power off mode] When a certain period of time has elapsed since the transition to the low power mode, the PMU 60 performs a power gating operation and puts the cache 40 into the power off mode.

[0399] [Data recovery, power on mode] In response to the interrupt request, the PMU 60 restores the cache 40 to the normal state. The signal OSS is set to high level, and the data held in the circuit BKC 20 is written back to the circuit SMC 20. During the period when the signal OSS is at high level, the PMU 60 performs a voltage scaling operation and a power gating operation, and restores the memory circuit 190 to the power-on mode. In the example of FIG. 32, when the potential of the power line supplying VDD becomes stable, the signal CLK becomes high level. When the potential of the power line supplying VDDMC becomes stable, the signal OSS is returned to low level, and the data restoration operation is terminated. The states of the nodes SN1 and SN2 are restored to the states they were in immediately before entering the hibernation state.

[0400] [Normal operation] When the supply of VDDMC is resumed, the circuit SMC20 returns to the normal mode in which normal operation is possible.

[0401] As described above, by using OS transistors, a backup circuit capable of retaining data for a long period of time even when the power supply is cut off can be configured. Providing this backup circuit enables power gating of the processor core and cache. In addition, by performing power management in hibernation that combines voltage scaling and power gating, it is possible to reduce the energy and time overhead required for the process of returning from the hibernation state to the normal state. Therefore, it is possible to efficiently reduce power without reducing the processing capacity of the processing device.

[0402] This embodiment mode can be appropriately combined with the descriptions of other embodiment modes.

[0403] (Embodiment 5) Examples of electronic devices that can use the imaging device according to one embodiment of the present invention include display devices, personal computers, image storage devices or image playback devices equipped with a recording medium, mobile phones, game machines including portable types, portable data terminals, electronic book terminals, cameras such as video cameras and digital still cameras, goggle-type displays (head-mounted displays), navigation systems, audio playback devices (car audio, digital audio players, etc.), copiers, facsimiles, printers, printer-combined machines, automated teller machines (ATMs), vending machines, etc. Specific examples of these electronic devices are shown in Figures 35A to 35F.

[0404] 35A illustrates an example of a mobile phone, which includes a housing 981, a display portion 982, operation buttons 983, an external connection port 984, a speaker 985, a microphone 986, a camera 987, and the like. The mobile phone includes a touch sensor in the display portion 982. Any operation, such as making a call or inputting characters, can be performed by touching the display portion 982 with a finger, a stylus, or the like. The imaging device and the operating method thereof according to one embodiment of the present invention can be applied to the mobile phone.

[0405] 35B shows a portable data terminal, which includes a housing 911, a display portion 912, a speaker 913, a camera 919, and the like. Information can be input and output using a touch panel function of the display portion 912. Characters and the like can be recognized from an image acquired by the camera 919, and the characters can be output as voice through the speaker 913. The imaging device and the operating method thereof according to one embodiment of the present invention can be applied to the portable data terminal.

[0406] FIG. 35C shows a surveillance camera, which includes a support base 951, a camera unit 952, a protective cover 953, and the like. The camera unit 952 is provided with a rotation mechanism and the like, and can capture images of the entire periphery by installing it on the ceiling. The imaging device and its operating method according to one embodiment of the present invention can be applied to the elements for acquiring images in the camera unit. Note that the term "surveillance camera" is a common name and is not intended to limit the use. For example, a device having a function as a surveillance camera is also called a camera or a video camera.

[0407] 35D shows a video camera, which has a first housing 971, a second housing 972, a display unit 973, operation keys 974, a lens 975, a connection unit 976, a speaker 977, a microphone 978, and the like. The operation keys 974 and the lens 975 are provided in the first housing 971, and the display unit 973 is provided in the second housing 972. An imaging device and an operation method thereof according to one embodiment of the present invention can be applied to the video camera.

[0408] 35E shows a digital camera, which includes a housing 961, a shutter button 962, a microphone 963, a light-emitting portion 967, a lens 965, and the like. The imaging device and the operating method thereof according to one embodiment of the present invention can be applied to this digital camera.

[0409] 35F shows a wristwatch-type information terminal, which includes a display unit 932, a housing / wristband 933, a camera 939, and the like. The display unit 932 includes a touch panel for operating the information terminal. The display unit 932 and the housing / wristband 933 are flexible and have excellent wearability on the body. The imaging device and the operating method thereof according to one embodiment of the present invention can be applied to this information terminal.

[0410] This embodiment mode can be appropriately combined with the descriptions of other embodiment modes. [Explanation of symbols]

[0411] 10: power supply circuit, 12: MW, 20: PU, 21: PU, 30: processor core, 31: memory circuit, 32: circuit, 35: power supply line, 40: cache, 41: memory array, 42: peripheral circuit, 43: control circuit, 45: memory cell, 60: PMU, 61: circuit, 65: clock control circuit, 70: PSW, 71: PSW, 80: terminal, 81: terminal, 82: terminal, 83: terminal, 100: pixel, 101: photoelectric conversion device, 102: transistor, 103: transistor, 104: capacitor, 105: transistor, 106: capacitor, 107: transistor , 108: transistor, 109: transistor, 110: flip-flop circuit, 111: wiring, 112: wiring, 113: wiring, 114: wiring, 115: wiring, 120: memory cell, 121: wiring, 122: wiring, 123: wiring, 124: wiring, 125: wiring, 133: conductive layer, 134: conductive layer, 135: conductive layer, 136: conductive layer, 138: conductive layer, 139: conductive layer, 151: capacitor, 161: transistor, 162: transistor, 163: capacitor, 170: circuit, 180: processor core, 181: control device, 182: program counter ,183: Pipeline register, 184: Pipeline register, 185: Register file, 186: ALU, 187: Data bus, 190: Memory circuit, 200: Pixel block, 201: Circuit, 202: Capacitor, 203: Transistor, 204: Transistor, 205: Transistor, 206: Transistor, 207: Resistor, 211: Wiring, 212: Wiring, 213: Wiring, 215: Wiring, 216: Wiring, 217: Wiring, 218: Wiring, 219: Wiring, 221: Wiring, 251: Transistor, 252: Transistor, 253: Transistor, 254: Transistor, 255: Transistor, 256: Transistor, 257: Transistor, 258: Transistor, 259: Transistor, 260: Transistor, 261: Transistor, 262: Transistor, 263: Transistor, 264: Transistor, 265: Transistor, 266: Transistor, 267: Transistor, 268: Transistor, 269: Transistor, 270: Transistor, 271: Transistor, 272: Transistor, 273: Transistor, 274: Transistor, 275: Transistor, 276: Transistor, 277: Transistor, 278: Transistor, 279: Transistor, 280: Transistor, 281: Transistor, 282: Transistor, 283: Transistor, 284: Transistor, 285: Transistor, 286: Transistor, 287: Transistor, 288: Transistor, 289: Transistor, 290: Transistor, 2 4: transistor, 300: pixel array, 301: circuit, 302: circuit, 303: circuit, 304: circuit, 305: circuit, 311: wiring, 312: row driver, 313: column driver, 320: memory cell, 321: memory circuit, 321a: memory cell, 321aA: memory cell, 321aB: memory cell, 321aC: memory cell, 321aD: memory cell, 325: reference memory cell, 330: circuit, 350: circuit, 360: circuit, 370: circuit, 410: package substrate, 411: package substrate, 420: cover glass, 421: lens cover,430: adhesive, 435: lens, 440: bump, 441: land, 450: image sensor chip, 451: image sensor chip, 460: electrode pad, 461: electrode pad, 470: wire, 471: wire, 490: IC chip, 535: back gate, 545: semiconductor layer, 546: insulating layer, 560: layer, 561: layer, 562: layer, 562a: layer, 562b: layer, 563: layer, 563a: layer, 563b: layer, 563c: layer, 565a: layer, 565b: layer, 566a: layer, 566b: layer, 566c: layer, 566d: layer, 567a: layer, 567b: layer , 567c: layer, 567d: layer, 567e: layer, 611: silicon substrate, 612: insulating layer, 613: insulating layer, 614: insulating layer, 615: insulating layer, 616: insulating layer, 617: insulating layer, 618: insulating layer, 619: conductive layer, 621: insulating layer, 622: insulating layer, 623: insulating layer, 624: insulating layer, 625: insulating layer, 626: insulating layer, 627: conductive layer, 628: insulating layer, 629: insulating layer, 631: insulating layer, 632: silicon substrate, 633: insulating layer, 634: insulating layer, 635: insulating layer, 636: conductive layer, 637: insulating layer, 638: insulating layer, 639: conductive layer, 641: insulating layer, 642: insulating layer, 643: insulating layer, 644: insulating layer, 645: insulating layer, 646: conductive layer, 647: insulating layer, 648: insulating layer, 651: insulating layer, 652: insulating layer, 653: insulating layer, 654: insulating layer, 655: conductive layer, 661: insulating layer, 662: insulating layer, 664: insulating layer, 665: insulating layer, 671: light shielding layer, 672: optical conversion layer, 673: microlens array, 701: gate electrode, 702: gate insulating film, 703: source region, 704: drain region, 705: source electrode, 706: drain electrode, 707: oxide semiconductor layer, 911: housing, 912: surface display unit, 913: speaker, 919: camera, 932: display unit, 933: housing / wristband, 939: camera, 951: support stand, 952: camera unit, 953: protective cover, 961: housing, 962: shutter button, 963: microphone, 965: lens, 967: light emitting unit, 971: housing, 972: housing, 973: display unit, 974: operation keys, 975: lens, 976: connection unit, 977: speaker, 978: microphone, 981: housing, 982: display unit, 983: operation button, 984: external connection port, 985: speaker, 986: microphone, 987: camera,

Claims

1. An imaging device having a pixel, a first circuit, and a second circuit, the first circuit has a function of supplying a first potential to the pixel; The pixel has a function of acquiring first data and second data, the pixel has a function of generating third data by adding a potential based on the first potential to the first data, and a function of generating fourth data by adding a potential based on the first potential to the second data, The second circuit has a function of reading out a difference between the first data and the third data, and a difference between the second data and the fourth data.

2. In claim 1, the first potential is a potential for a multiply-accumulate operation, The first data is data obtained when an image is captured, An imaging device, wherein the second data is data for a case where imaging is not performed.

3. In claim 1 or 2, the pixel includes a photoelectric conversion device, a first transistor, a second transistor, a third transistor, a fourth transistor, a fifth transistor, a sixth transistor, a first capacitor, and a second capacitor; one electrode of the photoelectric conversion device is electrically connected to one of the source and drain of the first transistor; the other of the source and the drain of the first transistor is electrically connected to one of the source and the drain of the second transistor and one electrode of the first capacitor; the other electrode of the first capacitor is electrically connected to one of the source and the drain of the third transistor, one electrode of the second capacitor, and a gate of the fourth transistor; one of a source and a drain of the fourth transistor is electrically connected to one of a source and a drain of the fifth transistor; the other electrode of the second capacitor is electrically connected to one of the source and the drain of the sixth transistor; the other of the source and the drain of the fifth transistor is electrically connected to the second circuit; the other of the source and the drain of the sixth transistor is electrically connected to the first circuit.

4. In any one of claims 1 to 3, The imaging device, wherein the second circuit is a correlated double sampling circuit.

5. In any one of claims 1 to 4, an imaging device, wherein a transistor included in the pixel has a metal oxide in a channel formation region, the metal oxide including In, Zn, and M (M is one or more of Al, Ti, Ga, Ge, Sn, Y, Zr, La, Ce, Nd, and Hf).

Images