Imaging systems and electronic devices

Abstract

Provided is an image capturing system which has image processing functionality and can generate interpolation images. The image capturing system comprises additional functionality such as image processing and can generate the interpolation images by using image data output by an image capturing device. The image capturing device can carry out a filtering process in parallel during an exposure period, and therefore a large number of computations are possible and high-quality interpolation images can be generated. In particular, a computation count can be further increased in image capturing in dark places having a long exposure time. Accordingly, the frame rate can be increased substantially and outstanding video image data can be generated.

Description

[Technical Field]

[0001] One aspect of the present invention relates to an imaging system.

[0002] Note that one embodiment of the present invention is not limited to the above-mentioned technical field. The technical field of one embodiment of the invention disclosed in this specification relates to an object, a method, or a manufacturing method. Alternatively, one embodiment of the present invention relates to a process, a machine, a manufacture, or a composition of matter. Therefore, more specific examples of the technical field of one embodiment of the present invention disclosed in this specification include semiconductor devices, display devices, liquid crystal display devices, light-emitting devices, lighting devices, power storage devices, memory devices, imaging devices, and operation methods thereof or manufacturing methods thereof.

[0003] Note that in this specification and the like, a semiconductor device refers to any device that can function by utilizing semiconductor characteristics. A transistor and a semiconductor circuit are examples of a semiconductor device. In addition, a memory device, a display device, an imaging device, an electronic device, and a system including any of them may include a semiconductor device. [Background technology]

[0004] A technique for forming a transistor using an oxide semiconductor thin film formed over a substrate has attracted attention. For example, Patent Document 1 discloses an imaging device having a pixel circuit that uses a transistor that includes an oxide semiconductor and has extremely low off-state current.

[0005] Furthermore, Patent Document 2 discloses a technique for adding a calculation function to an imaging device. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Japanese Patent Application Laid-Open No. 2011-119711 [Patent Document 2] Japanese Patent Application Laid-Open No. 2016-123087 Summary of the Invention [Problem to be solved by the invention]

[0007] While imaging devices equipped with solid-state imaging elements such as CMOS image sensors can easily capture high-quality images, the next generation of imaging devices will be required to incorporate even more intelligent functions.

[0008] Image data (analog data) acquired by an imaging device is converted to digital data, and after being output externally, image processing is performed as needed. If this processing can be performed within the imaging device, cooperation with external devices will be faster, improving user convenience. It will also reduce the load and power consumption of peripheral devices. Furthermore, if complex data processing can be performed in the analog data state, the time required for data conversion can also be shortened.

[0009] Furthermore, in order to express smooth movement in moving images, it is preferable to increase the frame rate. However, the greater the number of pixels, the more difficult it becomes to secure a horizontal period, so it is not possible to simply increase the frame rate. For this reason, technology is sometimes used to generate images that interpolate between frames of actual images (interpolated images). However, if the frame rate at which actual images are acquired is relatively high, it can be difficult to secure the time to generate the interpolated images.

[0010] Therefore, an object of one embodiment of the present invention is to provide an imaging system capable of performing image processing. Another object is to provide an imaging system capable of generating an interpolated image. Another object is to provide an imaging system with low power consumption. Another object is to provide an imaging system with high reliability. Another object is to provide a novel imaging device or imaging system. Another object is to provide a driving method of the imaging device or imaging system. Another object is to provide a novel semiconductor device or the like.

[0011] Note that the description of these problems does not preclude the existence of other problems. Note that one embodiment of the present invention does not necessarily solve all of these problems. Note that problems other than these will become apparent from the description of the specification, drawings, claims, etc., and it is possible to extract other problems from the description of the specification, drawings, claims, etc. [Means for solving the problem]

[0012] One aspect of the present invention relates to an imaging system that has an image processing function and is capable of generating an interpolated image.

[0013] One embodiment of the present invention is an imaging system including an imaging device, a first semiconductor device, a second semiconductor device, and a third semiconductor device. The imaging device has a function of acquiring first data and second data in each frame period. The imaging device has a function of filtering the first data to generate third data and a function of filtering the second data to generate fourth data. The first semiconductor device has a function of generating fifth data by performing an operation using the third data and fourth data acquired in an nth frame (n is a natural number). The first semiconductor device has a function of generating sixth data by performing an operation using the third data and fourth data acquired in an n+1th frame. The second semiconductor device has a function of generating an interpolated image by using the fifth data and the sixth data. The third semiconductor device has a function of generating moving image data having the first data acquired in the nth frame, the interpolated image, and the first data acquired in the n+1th frame in this order.

[0014] The generation of the third data can occur during the acquisition of the second data.

[0015] Another embodiment of the present invention is an imaging system including an imaging device, a first semiconductor device, a second semiconductor device, and a third semiconductor device. The imaging device has a function of acquiring first data and second data, and the first data is acquired in each frame period. The imaging device has a function of filtering the first data to generate third data and a function of filtering the second data to generate fourth data. The first semiconductor device has a function of generating fifth data by performing an operation using the third data and fourth data acquired in an nth frame (n is a natural number). The first semiconductor device has a function of generating sixth data by performing an operation using the third data and fourth data acquired in an n+1th frame. The second semiconductor device has a function of generating an interpolated image by using the fifth data and the sixth data. The third semiconductor device has a function of generating moving image data having the first data acquired in the nth frame, the interpolated image, and the first data acquired in the n+1th frame in this order.

[0016] The filtering of the first data acquired in the nth frame can be performed in the n+1th frame.

[0017] The imaging device can have a pixel block, a first circuit, and a second circuit, where the pixel block has a plurality of pixels, the first circuit has a function of supplying a first potential or a second potential to each of the pixels, each of the pixels has a function of acquiring first data, each of the pixels has a function of adding the first potential to the first data to generate second data, each of the pixels has a function of adding the second potential to the first data to generate third data, and the second circuit has a function of generating fourth data corresponding to the difference between the sum of the second data output by the plurality of pixels and the sum of the third data output by the plurality of pixels.

[0018] Each of the pixels has a first transistor, a second transistor, a third transistor, a fourth transistor, a fifth transistor, a sixth transistor, a photoelectric conversion device, and a capacitor, and one electrode of the photoelectric conversion device is electrically connected to one of the source or drain of the first transistor and one of the source or drain of the second transistor, the other of the source or drain of the first transistor is electrically connected to one of the source or drain of the third transistor, the gate of the fourth transistor, and one electrode of the capacitor, one of the source or drain of the fourth transistor is electrically connected to one of the source or drain of the fifth transistor, and the other electrode of the capacitor is electrically connected to one of the source or drain of the sixth transistor.

[0019] The first circuit can be electrically connected to the other of the source or the drain of the sixth transistor.

[0020] The second circuit has a function of a correlated double sampling circuit, and the second circuit can be electrically connected to the other of the source and the drain of the fifth transistor.

[0021] It is preferable that one or more of the first to sixth transistors have a metal oxide in a channel formation region, and the metal oxide contains In, Zn, and M (M is one or more of Al, Ti, Ga, Ge, Sn, Y, Zr, La, Ce, Nd, and Hf). [Effects of the Invention]

[0022] By using one embodiment of the present invention, an imaging system capable of performing image processing can be provided. Alternatively, an imaging system capable of generating an interpolated image can be provided. Alternatively, an imaging system with low power consumption can be provided. Alternatively, an imaging system with high reliability can be provided. Alternatively, a novel imaging device or imaging system can be provided. Alternatively, a driving method of the imaging device or imaging system can be provided. Alternatively, a novel semiconductor device or the like can be provided. [Brief explanation of the drawings]

[0023] FIG. 1 is a block diagram illustrating an imaging system. FIG. 2 is a diagram illustrating a pixel block and a circuit 201. As shown in FIG. 3A and 3B are diagrams illustrating a pixel 100. FIG. 4A and 4B are diagrams illustrating a pixel 100. FIG. FIG. 5 is a diagram illustrating an imaging system. FIG. 6 is a timing chart illustrating the operation of the imaging device. Fig. 7A is a timing chart illustrating the operation of the imaging device, and Fig. 7B is a diagram illustrating filters supplied to pixel blocks. Fig. 8A is a timing chart illustrating the operation of the imaging device, and Fig. 8B is a diagram illustrating filters supplied to pixel blocks. FIG. 9 is a diagram illustrating an imaging system. FIG. 10 is a diagram illustrating the circuit 312. FIG. 11 is a timing chart illustrating the operation of the circuit 312. FIG. 12 is a timing chart illustrating the operation of the circuit 312. 13A and 13B are diagrams illustrating the circuit 301 and the circuit 302. FIG. FIG. 14 is a diagram illustrating a memory cell. 15A and 15B are diagrams showing examples of the configuration of a neural network. 16A to 16D are diagrams illustrating the configuration of a pixel in an imaging device. 17A to 17C are diagrams illustrating the configuration of a photoelectric conversion device. FIG. 18 is a cross-sectional view illustrating a pixel. 19A to 19C are diagrams illustrating a Si transistor. FIG. 20 is a cross-sectional view illustrating a pixel. FIG. 21 is a cross-sectional view illustrating a pixel. 22A to 22D illustrate an OS transistor. FIG. 23 is a cross-sectional view illustrating a pixel. 24A1 to 24A3 and 24B1 to 24B3 are perspective views of a package and a module that house an imaging device. 25A to 25F are diagrams illustrating an electronic device. DETAILED DESCRIPTION OF THE INVENTION

[0024] The embodiments will be described in detail with reference to the drawings. However, the present invention is not limited to the following description, and it will be readily understood by those skilled in the art that various modifications in form and detail may be made without departing from the spirit and scope of the present invention. Therefore, the present invention should not be interpreted as being limited to the description of the embodiments shown below. In the configuration of the invention described below, the same parts or parts having similar functions will be designated by the same reference numerals in different drawings, and repeated description thereof may be omitted. In addition, hatching of the same elements constituting the drawings may be omitted or changed as appropriate in different drawings.

[0025] Furthermore, even if a circuit diagram shows a single element, that element may be configured as multiple elements as long as there is no functional problem. For example, multiple transistors operating as switches may be connected in series or parallel. Also, a capacitor may be divided and placed in multiple locations.

[0026] Furthermore, a single conductor may have multiple functions such as wiring, an electrode, and a terminal, and in this specification, multiple names may be used for the same element. Also, even when elements are shown as being directly connected to each other on a circuit diagram, in reality, the elements may be connected via one or more conductors, and in this specification, such a configuration is also included in the category of direct connection.

[0027] (Embodiment 1) In this embodiment, an imaging system which is one embodiment of the present invention will be described with reference to drawings.

[0028] One aspect of the present invention is an imaging system having an imaging device with additional functions such as image processing, etc. The imaging device stores analog data (image data) acquired during imaging in pixels and can extract data obtained by multiplying the analog data by an arbitrary weighting coefficient.

[0029] By inputting this data into a neural network, it is possible to perform processing such as image recognition. Because huge amounts of image data can be stored in pixels as analog data, processing can be performed efficiently.

[0030] Furthermore, an imaging system according to one embodiment of the present invention can generate an interpolated image using image data output by an imaging device. The imaging device can perform filtering in parallel during an exposure period, enabling multiple calculations and generating a high-quality interpolated image. In particular, the number of calculations can be further increased when imaging in a dark place with a long exposure time. Therefore, the frame rate can be substantially increased, enabling the generation of high-quality moving image data.

[0031] <Imaging system> 1 is a block diagram illustrating an imaging system of one embodiment of the present invention. The imaging system includes an imaging device 300 and a plurality of semiconductor devices. The imaging device 300 includes a pixel array 310, a circuit 311, a circuit 312, a circuit 313, and a circuit 201. The semiconductor devices include the circuit 301, a circuit 302, a circuit 303, a circuit 304, and a circuit 305. The imaging system may further include a display device 306.

[0032] In this embodiment, the imaging device 300 has the above-described configuration, but the imaging device 300 may include one or more of the above semiconductor devices. The number of semiconductor devices is not limited to the above. For example, a semiconductor device in which some of the above semiconductor devices are integrated may be used. One or more functions of the above semiconductor devices may be replaced by software operations. Semiconductor devices other than those described above may also be provided.

[0033] The pixel array 310 has an imaging function and a calculation function. The circuits 311 and 312 have a selection function. The circuit 313 has a function of supplying potentials for product-sum calculations to pixels and a selection function. The circuits 201 and 301 have a calculation function. The circuit 302 has a calculation function or a data conversion function. The circuit 303 has a function of generating an image. The circuit 304 has a function of integrating multiple image data. The circuit 305 has a function of storing image data. The display device 306 can display image data output by the circuit 304 or the circuit 305. Note that a shift register, a decoder, or the like can be used as the circuit having a selection function.

[0034] The pixel array 310 has a plurality of pixel blocks 200. As shown in FIG. 2, the pixel block 200 has a plurality of pixels 100 arranged in a matrix, and each pixel 100 is electrically connected to a circuit 201 via a wiring 112. Note that the circuit 201 can also be provided within the pixel block 200.

[0035] The pixel 100 can acquire image data and generate data by adding the image data and a weighting factor. While the pixel block 200 in FIG. 2 has 3×3 pixels as an example, this is not limiting. For example, it can have 2×2 or 4×4 pixels. Alternatively, the number of pixels in the horizontal and vertical directions may be different. Furthermore, some pixels may be shared between adjacent pixel blocks.

[0036] The pixel block 200 and the circuit 201 can be operated as a multiply-accumulate circuit.

[0037] <Pixel circuit> Pixel 100 may include a photoelectric conversion device 101, a transistor 102, a transistor 103, a transistor 104, a transistor 105, a transistor 106, a transistor 107, and a capacitor 108, as shown in FIG. 3A.

[0038] One electrode of the photoelectric conversion device 101 is electrically connected to one of the source or drain of the transistor 102 and one of the source or drain of the transistor 103. The other of the source or drain of the transistor 102 is electrically connected to one of the source or drain of the transistor 104, the gate of the transistor 105, and one electrode of the capacitor 108. One of the source or drain of the transistor 105 is electrically connected to one of the source or drain of the transistor 106. The other electrode of the capacitor 108 is electrically connected to one of the source or drain of the transistor 107.

[0039] The other electrode of the photoelectric conversion device 101 is electrically connected to a wiring 114. The other of the source and the drain of the transistor 103 is electrically connected to a wiring 115. The other of the source and the drain of the transistor 104 is electrically connected to a wiring 116. The other of the source and the drain of the transistor 106 is electrically connected to a wiring 112. The other of the source and the drain of the transistor 105 is electrically connected to a GND wiring or the like. The other of the source and the drain of the transistor 107 is electrically connected to a wiring 111.

[0040] A gate of the transistor 102 is electrically connected to a wiring 121. A gate of the transistor 103 is electrically connected to a wiring 122. A gate of the transistor 104 is electrically connected to a wiring 123. A gate of the transistor 106 is electrically connected to a wiring 124. A gate of the transistor 107 is electrically connected to a wiring 125.

[0041] Here, the electrical connection point between one electrode of the photoelectric conversion device 101, one of the source or drain of the transistor 102, and one of the source or drain of the transistor 103 is referred to as a node FDP. Also, the electrical connection point between the other of the source or drain of the transistor 102, one of the source or drain of the transistor 104, one electrode of the capacitor 108, and the gate of the transistor 105 is referred to as a node FD.

[0042] The wirings 114, 115, and 116 can function as power supply lines. For example, the wiring 114 can function as a high-potential power supply line, and the wirings 115 and 116 can function as low-potential power supply lines. Note that the wirings 115 and 116 may be electrically connected. The wirings 121, 122, 123, 124, and 125 can function as signal lines that control the conduction of each transistor. The wiring 111 can function as a wiring that supplies a potential corresponding to a weighting coefficient to the pixel 100. The wiring 112 can function as a wiring that electrically connects the pixel 100 and the circuit 201.

[0043] The wiring 112 may be electrically connected to an amplifier circuit or a gain adjustment circuit.

[0044] A photodiode can be used as the photoelectric conversion device 101. Any type of photodiode can be used, such as a Si photodiode having silicon in the photoelectric conversion layer or an organic photodiode having an organic photoconductive film in the photoelectric conversion layer. Note that, if you want to increase the light detection sensitivity at low illuminance, it is preferable to use an avalanche photodiode.

[0045] The transistor 102 can have a function of controlling the potential of the node FD. The transistor 103 can have a function of initializing the potential of the node FDP. The transistor 104 can have a function of initializing the potential of the node FD. The transistor 105 can have a function of controlling the current flowing from the circuit 201 in accordance with the potential of the node FD. The transistor 106 can have a function of selecting a pixel. The transistor 107 can have a function of supplying a potential corresponding to a weighting coefficient to the other electrode of the capacitor 108.

[0046] As shown in Figure 3B, one of the source and drain of the transistor 105 may be electrically connected to one of the source and drain of the transistor 106, the other of the source and drain of the transistor 105 may be electrically connected to a wiring 112, and the other of the source and drain of the transistor 106 may be electrically connected to a GND wiring or the like.

[0047] 4A and 4B, the connection direction of the photoelectric conversion device 101 may be reversed. In this case, the wiring 114 may function as a low-potential power supply line, and the wirings 115 and 116 may function as high-potential power supply lines.

[0048] When an avalanche photodiode is used for the photoelectric conversion device 101, a high voltage may be applied, and therefore a high-voltage transistor is preferably used as the transistor connected to the photoelectric conversion device 101. For example, a transistor using a metal oxide in a channel formation region (hereinafter referred to as an OS transistor) can be used as the high-voltage transistor. Specifically, it is preferable to use an OS transistor as the transistors 102, 103, and 104.

[0049] In addition, OS transistors have an extremely low off-state current. By using OS transistors for the transistors 102, 103, 104, and 107, the period during which charge can be held at the nodes FD and FDP can be significantly extended. Therefore, a global shutter system in which charge is accumulated simultaneously in all pixels can be applied without complicating the circuit configuration or operation method. Furthermore, while image data is held at the node FD, multiple calculations can be performed using the image data.

[0050] On the other hand, it may be desirable for the transistor 105 to have excellent amplification characteristics. Also, it may be preferable to use a transistor with high mobility that can operate at high speed as the transistor 107. Therefore, the transistors 105 and 107 may be transistors that use silicon in their channel formation regions (hereinafter referred to as Si transistors).

[0051] Note that the present invention is not limited to the above, and any combination of OS transistors and Si transistors may be used. Furthermore, all transistors may be OS transistors. Alternatively, all transistors may be Si transistors. Examples of Si transistors include transistors containing amorphous silicon and transistors containing crystalline silicon (microcrystalline silicon, low-temperature polysilicon, and single-crystal silicon).

[0052] The potential of the node FD in the pixel 100 is determined by the sum of a reset potential supplied from the wiring 116 and a potential (image data) generated by photoelectric conversion by the photoelectric conversion device 101. Alternatively, the potential is determined by capacitively coupling a potential corresponding to a weighting factor supplied from the wiring 111. Therefore, the transistor 105 can pass a current corresponding to the data in which an arbitrary weighting factor has been added to the image data.

[0053] Furthermore, pixel 100 is provided with separate transistors for resetting the potentials of node FD and node FDP. Therefore, the exposure operation (storage operation) and readout operation can be performed in parallel. Although detailed operations will be described later, for example, the following operation can be performed. After the exposure operation, the charge of node FDP is transferred to node FD. Then, node FDP is reset, and the next exposure operation begins. During this exposure operation, node FD is read, and after the readout is completed, node FD is reset. By performing this operation continuously, sufficient exposure time can be secured, and the frame frequency can be easily increased.

[0054] Note that this is just an example of the circuit configuration of the pixel 100 described in this embodiment, and other circuit configurations related to photoelectric conversion may be used.

[0055] <Circuit 201> 2, the pixels 100 are electrically connected to each other through wirings 112. The circuit 201 can perform calculations using the sum of currents flowing through the transistors 105 of the pixels 100.

[0056] The circuit 201 includes a capacitor 202, a transistor 203, a transistor 204, a transistor 205, a transistor 206, and a transistor 207 as a voltage conversion circuit.

[0057] One electrode of the capacitor 202 is electrically connected to one of the source and drain of the transistor 203. One of the source and drain of the transistor 203 is electrically connected to the gate of the transistor 204. One of the source and drain of the transistor 204 is electrically connected to one of the source and drain of the transistor 205. One of the source and drain of the transistor 205 is electrically connected to one of the source and drain of the transistor 206. One of the source and drain of the transistor 207 is electrically connected to the other electrode of the capacitor 202.

[0058] The other electrode of the capacitor 202 is electrically connected to a wiring 112. The other of the source and the drain of the transistor 203 is electrically connected to a wiring 218. The other of the source and the drain of the transistor 204 is electrically connected to a wiring 219. The other of the source and the drain of the transistor 205 is electrically connected to a reference power supply line such as a GND wiring. The other of the source and the drain of the transistor 206 is electrically connected to a wiring 212. The other of the source and the drain of the transistor 207 is electrically connected to a wiring 217. The gate of the transistor 203 is electrically connected to a wiring 216. The gate of the transistor 205 is electrically connected to a wiring 215. The gate of the transistor 206 is electrically connected to a wiring 213.

[0059] The wirings 217, 218, and 219 can function as power supply lines. For example, the wiring 218 can function as a wiring that supplies a dedicated potential for readout. The wirings 217 and 219 can function as high-potential power supply lines. The wirings 213, 215, and 216 can function as signal lines that control the conduction of each transistor. The wiring 212 is an output line and can be electrically connected to, for example, the circuit 301 shown in FIG. 1.

[0060] The transistor 203 can have a function of resetting the potential of the wiring 211 to the potential of the wiring 218. The transistors 204 and 205 can function as source follower circuits. The transistor 206 can have a function of controlling readout. Note that the circuit 201 has a function as a correlated double sampling circuit (CDS circuit) and can be replaced with a circuit having the same function. Note that the wiring 211 is a wiring that electrically connects one electrode of the capacitor 202, one of the source or drain of the transistor 203, and the gate of the transistor 204.

[0061] In one embodiment of the present invention, offset components other than the product of image data (X) and a weighting coefficient (W) are removed to extract the target WX. WX can be calculated using data acquired from the same pixel with exposure (with imaging) and without exposure (without imaging), and data obtained by adding weights to each of these.

[0062] The current (I p ) is the sum of kΣ(XV th ) 2 , the current (I p ) is the sum of kΣ(W+XV th ) 2 In addition, the current (I ref ) is the sum of kΣ(0-V th ) 2 , the current (I ref ) is the sum of kΣ(WV th ) 2 where k is a constant, V th is the threshold voltage of transistor 105.

[0063] First, calculate the difference (data A) between the data with exposure and the data with weighting added to the data with exposure. th ) 2 -(W+XV th ) 2 )=kΣ(-W 2-2W·X+2W·V th )

[0064] Next, calculate the difference (data B) between the data without exposure and the weighted data. kΣ((0-V th ) 2 -(WV th ) 2 )=kΣ(-W 2 +2W·V th )

[0065] Then, take the difference between data A and data B. kΣ(-W 2 -2W·X+2W·V th -(-W 2 +2W·V th ))=kΣ(-2W·X). In other words, it is possible to remove offset components other than the product of the image data (X) and the weighting coefficient (W).

[0066] The circuit 201 can read out data A and data B. Note that the difference between data A and data B can be calculated by the circuit 301, for example.

[0067] <Operation of the imaging system> 5 is a diagram illustrating the operation of generating moving image data using an imaging system according to one embodiment of the present invention. As described above, WX can be calculated using data acquired with and without exposure from the same pixel. Therefore, data acquired with and without exposure are acquired within one frame period. In FIG. 5, exposure operations corresponding to no exposure are represented by Exp. A, and exposure operations corresponding to exposure are represented by Exp. B.

[0068] Furthermore, the pixel 100 can perform exposure and readout operations in parallel. Therefore, as shown in Figure 5, data acquired in an Exp. A operation can be read out during the next Exp. B operation. Data acquired in an Exp. B operation can be read out during the next Exp. A operation.

[0069] The readout operation involves adding weights to the data acquired by the pixels and reading the data, and also reading the data without adding weights to the data. The former corresponds to filtering, and can be performed using, for example, a convolution filter of a convolutional neural network (CNN). Furthermore, it is preferable to perform multiple filter operations as filtering. The operations up to this point can be performed by the imaging device 300.

[0070] The filtered data output from the imaging device 300 is input to a semiconductor device having circuits 301 and 302, where it is subjected to processes such as difference calculation, filter calculation, pooling, etc. Note that, in order to clarify the flow of data, multiple circuits 301, 302, and 303 are shown in Fig. 5, but in reality, each circuit can perform multiple processes.

[0071] Then, a plurality of data are output from the semiconductor device having the circuits 301 and 302 and input to the semiconductor device having the circuit 303. The circuit 303 can generate an interpolated image using the input data. Note that an interpolated image is an image equivalent to an intermediate image between a certain frame image and the next frame image, and a moving image including the interpolated image can express smooth movement. It can also be said that the frame frequency is increased.

[0072] For example, as shown in Fig. 5, an interpolated image a can be generated using data a1 generated based on data acquired in the n-2th frame and data a2 generated based on data acquired in the n-1th frame. Similarly, an interpolated image b can be generated using data b1 generated based on data acquired in the n-1th frame and data b2 generated based on data acquired in the nth frame. Note that while the above is an example in which one interpolated image is generated between two actually acquired frame images, two or more interpolated images may be generated.

[0073] The normally read data that has not been filtered and the interpolated image generated by the circuit 303 are input to a semiconductor device having a circuit 304. The circuit 304 can generate new moving image data by connecting these pieces of data. The moving image data is input to and stored in a memory device, such as a circuit 305. Alternatively, the moving image data may be input to a display device 306 shown in FIG. 1 and displayed.

[0074] <Image capture operation> Next, the Exp. B (with exposure) operation of the n-1th frame and the Exp. A (without exposure) operation of the nth frame shown in FIG. 5 will be specifically described using the timing chart shown in FIG. 6. The pixel 100 described here has the configuration shown in FIG. 3A or FIG. 3B. It is also assumed that a predetermined constant potential is supplied to the power supply line, etc. The weight supply operation (filtering process) will be omitted in the description of FIG. 6.

[0075] First, the operation of reading out data acquired in the exposure operation (Exp. A, without exposure) of the n-1th frame during the exposure operation (Exp. B, with exposure) of the n-1th frame from time T1 to T3 will be described.

[0076] Note that before time T1, the potential of the wiring 121 is set to "H" to turn on the transistor 102, and data acquired in the Exp. A (no exposure) operation of the (n-1)th frame is transferred to the node FD.

[0077] At time T1, when the potential of the wiring 121 is set to "L" and the potential of the wiring 122 is set to "H," the transistor 102 is turned off, and the potential of the node FD is maintained. In addition, the transistor 103 is turned on, and the potential of the node FDP is set to the reset potential (the potential of the wiring 115) "V RES "

[0078] When the potential of the wiring 122 is set to “L” at time T2, the transistor 103 becomes non-conductive, and the potential of the node FDP changes in accordance with the operation of the photoelectric conversion device 101. This operation corresponds to the Exp. B (with exposure) operation of the (n−1)th frame.

[0079] At time T3, when the potential of the wiring 121 is set to "H", the transistor 102 is turned on and the potential of the node FDP is transferred to the node FD.

[0080] Here, between time T2 and time T3, an operation is performed in which the wirings 123[1] to [m] (the number in parentheses is the row number of the pixel block 200, m is a natural number) and 124[1] to [m] are sequentially set to "H", and the data acquired in the Exp. A (no exposure) operation of the n-1th frame is read out. Furthermore, at the end of reading out all rows, the potentials of the wirings 123[1] to [m] are set to "H", causing the transistor 104 to conduct and resetting the potential of the node FD. The transfer of the potential of the node FDP to the node FD at time T3 is performed after this reset operation. Note that this operation can be omitted.

[0081] Next, the operation of reading out data acquired in the exposure operation (Exp. B, with exposure) of the (n-1)th frame during the exposure operation (Exp. A, without exposure) of the nth frame from time T4 to T6 will be described.

[0082] When the potential of the wiring 122 is set to “H” at time T4, the transistor 103 is turned on, and the potential of the node FDP becomes the reset potential (the potential of the wiring 115) “V RES "

[0083] After time T5, the potential of the wiring 122 is kept at “H” to turn on the transistor 103. That is, although it is the exposure operation period, the reset potential continues to be supplied, so the potential of the node FDP is kept at “V RES This operation corresponds to the Exp. A (no exposure) operation of the nth frame.

[0084] At time T6, when the potential of the wiring 121 is set to "H", the transistor 102 is turned on and the potential of the node FDP is transferred to the node FD.

[0085] Here, before time T5 to time T6, an operation is performed in which the wirings 123[1] to [m] and 124[1] to [m] are sequentially set to "H," and data acquired in the Exp. B (with exposure) operation of the n-1th frame is read out. Furthermore, at the end of reading out all rows, the potentials of the wirings 123[1] to [m] are set to "H," causing the transistor 104 to conduct and resetting the potential of the node FD. The potential of the node FDP is transferred to the node FD at time T6 after this reset operation. This operation can also be omitted.

[0086] By performing the above operations, the exposure operation and the readout operation can be performed in parallel.

[0087] Next, the read operation accompanied by the weight supply (filtering) and the product-sum operation will be described with reference to the timing chart shown in Fig. 7A. The operation shown in Fig. 7A corresponds to the operation when the same weight is supplied to one column of pixels 100 included in pixel block 200 as shown in Fig. 7B.

[0088] 7A is an enlarged timing chart of a part (wiring 123[1], wiring 124[1]) of times T5 to T6 in FIG. 6. It also shows the potentials of the wiring 125[1], the node FD, the wiring 111, and the wirings 213, 215, and 216 in the circuit 201 at the same time. The operation described next is the operation of reading out data with exposure and data with weighted data added to the exposed data. The circuit 201 can calculate the difference between these data, and can read out the aforementioned data A.

[0089] It is assumed that before time T11 (corresponding to time T5 in FIG. 6), data (ΔX) acquired in the Exp. B (with exposure) operation of the (n-1)th frame is transferred to and held at node FD. Furthermore, at this time, transistor 107 is in a conductive state, and the other electrode of capacitor 108 is supplied with the potential "L (=0)" of wiring 111. It is also assumed that an appropriate analog potential is supplied to the gate of transistor 207 (see FIG. 2).

[0090] At time T11, when a weight W1 is supplied to the wiring 111 and the potential of the wiring 125 is set to "H", the transistor 107 becomes conductive and the potential "W1" is written to the other electrode of the capacitor 108. Also, due to the capacitive coupling of the capacitor 108, the change in the potential of the other electrode of the capacitor 108 ("ΔW1") is added to the node FD, and the potential of the node FD changes to "ΔX+ΔW1'". Note that if the capacitance of the capacitor 108 is sufficiently larger than the capacitance of the node FD, "ΔW1" and "ΔW1'" will be approximately the same value.

[0091] Furthermore, when the potentials of the wiring 124[1], the wiring 125[1], and the wiring 216 (see FIG. 2) are set to "H" at time T11, the transistor 106 is turned on, and a current according to the potential of the node FD flows from the wiring 112 to the transistor 105. Furthermore, in the circuit 201, the transistor 203 is turned on, and the potential of the wiring 211 becomes the potential "Vr" of the wiring 218. In other words, when the potential of the other electrode of the capacitor 202 is the output potential obtained when a weight is added to the image data acquired by the pixel 100, the potential of one electrode of the capacitor 202 is initialized to the potential "Vr."

[0092] Next, the potential of the wiring 111 is set to "L (=0)", the potential of the wiring 216 is set to "L", the potential of the wiring 213 is set to "H", the potential of the wiring 215 is set to "V bias " is set to an appropriate analog potential such as "ΔX", a current corresponding to the potential of node FD ("ΔX") flows from wiring 112 to transistor 105. Here, the potential of the other electrode of capacitor 202 changes according to the current flowing through wiring 112, and the change Y is added to the potential "Vr" of wiring 211 by capacitive coupling.

[0093] Therefore, the potential of the wiring 211 becomes "Vr+Y." If Vr=0, then Y is the difference itself, and data A described in the explanation of the circuit 201 is calculated. Furthermore, the circuit 201 can output a signal potential corresponding to data A by source follower operation. The processing up to this point is the filter processing (1) shown in FIG. 7.

[0094] By performing the same processing as above using weights W2 to W(n) (n is a natural number) up until time T12, it is possible to perform filtering processing up to (n).

[0095] Between time T12 and time T13, in preparation for the next read from the node FDP, the potential of the wiring 123[1] is set to "H" to perform a reset operation on the node FD.

[0096] Also, before and after time T12, a normal read operation without adding weights is performed. In this operation, image data is read without adding weights before time T12, and then the node FD is reset at time T12 and read again. This operation corresponds to an operation of subtracting noise at the time of reset from the image data, and image data with less noise can be obtained.

[0097] Similarly, at times T2 to T3 in Fig. 6, it is possible to calculate data B, which is the difference between the no-exposure data and the weighted data. Note that the normal read operation described above is not required during the period when the no-exposure data is read.

[0098] Next, an operation when the weights of one column of the pixels 100 in the pixel block 200 are not the same will be described with reference to the timing chart shown in Fig. 8A. Note that Fig. 8A shows an operation corresponding to one type of filtering processing.

[0099] For example, as shown in FIG. 8B, if the weights supplied to a column of pixels 100 in a pixel block 200 are not the same, the potentials of wirings 125[1_1] to 125[1_3] (the numbers in parentheses indicate "row number of pixel block_row number of pixel 100") are set to "H" at different times, and different weights (W1_1, W1_2, W1_3) are supplied to each pixel 100.

[0100] As a result of the above operation, the potential of FD[1] (node ​​FD of pixel 100 in the first row in pixel block 200) becomes ΔX+ΔW1_1'. The potential of FD[2] (node ​​FD of pixel 100 in the second row in pixel block 200) becomes ΔX+ΔW1_2'. The potential of FD[3] (node ​​FD of pixel 100 in the third row in pixel block 200) becomes ΔX+ΔW1_3'.

[0101] In the above state, by setting the potential of the wiring 216 to "H", the sum of the weighted data is converted to the potential "Vr" of the wiring 211 in the circuit 201, similar to the above-described operation. Finally, the weight added to the node FD is set to 0, and a difference calculation is performed using capacitive coupling, thereby allowing data A or data B to be calculated.

[0102] The data A and data B output from the circuit 201 by the above operation are input to the circuit 301. The circuit 301 performs an operation to obtain the difference between the data A and the data B, thereby making it possible to remove unnecessary offset components other than the product of the image data and the weighting coefficient. The circuit 301 may have a configuration including an arithmetic circuit like the circuit 201, or may have a configuration in which a memory circuit (also referred to as a storage circuit) and software processing are used to obtain the difference.

[0103] 1 to the wiring 111, and the weighting coefficient is preferably rewritten at least once within a frame period. A decoder can be used as the circuit 313. The circuit 313 may also include a D / A converter or an SRAM.

[0104] A signal can be output to the wiring 125 that selects the pixel 100 to which the weighting coefficient is to be input from the circuit 311. The circuit 311 can be a decoder or a shift register.

[0105] A signal can be output from the circuit 312 to the wiring 124 or the like connected to the gate of the transistor 106 of the pixel 100. The circuit 312 can be a decoder or a shift register.

[0106] The above is a description of the operation of acquiring data A and data B for each frame and performing a calculation to find the difference between them in circuit 301. Here, if there is no change in the weight and no degradation in the acquired image data, data B will always be the same data. Therefore, if circuit 301 is provided with a memory function and data B is stored therein, the operation of acquiring data B for each frame can be omitted. Note that multiple pieces of data B corresponding to multiple weights may be stored in the memory.

[0107] Figure 9 is a diagram explaining this operation. An exposure operation (Exp. A) equivalent to no exposure is performed in the initial (Init) frame, and exposure operations (Exp. B) equivalent to exposure are performed consecutively in the subsequent frames. Other operations are the same as those explained in Figures 5 to 8. By performing this operation, the frame frequency can be further increased.

[0108] Note that although the processing of captured image data has been described above, the imaging device of one embodiment of the present invention can also extract image data without processing it.

[0109] In a product-sum operation, it is preferable to simultaneously select pixels from multiple rows. On the other hand, when extracting only image data, it is preferable to extract data from pixels from one row. In one embodiment of the present invention, the circuit 312 for selecting the pixel 100 has a function of switching the number of rows to be selected. Note that to select one pixel, for example, a negative weight or the like may be applied to pixels other than the pixel to be selected, so that the transistor 105 is not turned on.

[0110] <Shift register> 10 is an example of a circuit that can be used for the circuit 312. This circuit is a shift register circuit, in which a plurality of logic circuits (SR) are electrically connected. Signal lines such as wiring RES, wiring VSS_RDRS, wiring RPWC_SE[0:3], wiring RCLK[0:3], and wiring RSP are connected to each logic circuit (SR), and by inputting an appropriate signal potential to each signal line, selection signal potentials can be sequentially output from the logic circuit (SR).

[0111] The logic circuit (SR) is electrically connected to a circuit 170. The circuit 170 includes a plurality of transistors, and signal lines such as wirings SE_SW[0:2] and SX[0:2] are connected to the circuit 170. The conduction of the transistors is controlled by inputting an appropriate signal potential to each signal line. The number of rows of pixels to be selected can be changed by controlling the circuit 170.

[0112] The output terminal of one logic circuit (SR) is electrically connected to one of the source and drain of one transistor, and the other of the source and drain of the transistor is connected to a wiring SE, which is electrically connected to a wiring 124 that selects the pixel 100.

[0113] A signal potential supplied from the wiring SE_SW[0] can be input to the gate of a transistor connected to the wiring SE[0]. A signal potential supplied from the wiring SE_SW[1] can be input to the gate of a transistor connected to the wiring SE[1]. A signal potential supplied from the wiring SE_SW[2] can be input to the gate of a transistor connected to the wiring SE[2]. A signal potential supplied from one of the wirings SE_SW[0:2] can be input to the gate of a transistor connected to the wiring SE[3] and subsequent wirings in the same order.

[0114] Adjacent wirings SE are electrically connected via one transistor, and wiring SE[0] is electrically connected to the power supply line (VSS) via one transistor.

[0115] A signal potential supplied from the wiring SX[0] can be input to the gate of a transistor electrically connecting the power supply line (VSS) and the wiring SE[0]. A signal potential supplied from the wiring SX[1] can be input to the gate of a transistor electrically connecting the wiring SE[0] and the wiring SE[1]. A signal potential supplied from the wiring SX[2] can be input to the gate of a transistor electrically connecting the wiring SE[1] and the wiring SE[2]. Any of the signal potentials supplied from the wirings SX[0:2] can be input to the gates of the transistors electrically connecting the subsequent wirings SE in the same order.

[0116] 11 is a timing chart illustrating the operation of simultaneously selecting multiple rows (three rows) using the circuit shown in FIG. 10. (0) to (161) correspond to the timing at which the logic circuit (SR) outputs a signal potential to the wiring SE.

[0117] At timing (0), when the potential of the wiring SX[0] becomes "L", the potential of the wiring SX[1] becomes "H", the potential of the wiring SX[2] becomes "H", the potential of the wiring SE_SW[0] becomes "H", the potential of the wiring SE_SW[1] becomes "L", and the potential of the wiring SE_SW[2] becomes "L", the conduction of each transistor is controlled, and "H" is output to the wiring SE[0], "H" to the wiring SE[1], and "H" to the wiring SE[2]. "L" is output to the other wirings SE.

[0118] Therefore, three rows can be selected simultaneously, and a product-sum operation can be performed on pixels in, for example, three rows and three columns.

[0119] At timing (1), when the potential of the wiring SX[0] becomes "H", the potential of the wiring SX[1] becomes "L", the potential of the wiring SX[2] becomes "H", the potential of the wiring SE_SW[0] becomes "L", the potential of the wiring SE_SW[1] becomes "H", and the potential of the wiring SE_SW[2] becomes "L", the conduction of each transistor is controlled, and "L" is output to the wiring SE[0], "H" to the wiring SE[1], "H" to the wiring SE[2], and "H" to the wiring SE[3]. "L" is output to the other wirings SE.

[0120] In other words, at timing (1), a multiply-and-accumulate operation with a stride of 1, which is shifted by one row from timing (0), is possible.

[0121] FIG. 12 is a timing chart illustrating the operation of selecting one row by the circuit shown in FIG.

[0122] In the operation according to the timing chart, the potential of the wiring SE_SW[0:2] is always “H” and the potential of the wiring SX[0:2] is always “L.” Therefore, the output of the logic circuit (SR) appears directly on each wiring SE, making it possible to select each row individually.

[0123] <Circuits 301, 302> 13A is a diagram illustrating circuits 301 and 302 connected to circuit 201. Data resulting from a product-sum operation output from circuit 201 is sequentially input to circuit 301. Circuit 301 may have various other operation functions in addition to the function of calculating the difference between data A and data B described above. For example, circuit 301 may have the same configuration as circuit 201. Alternatively, the function of circuit 301 may be replaced by software processing.

[0124] The circuit 301 may also include a circuit that performs activation function calculations. For example, a comparator circuit can be used for this circuit. The comparator circuit compares input data with a set threshold value and outputs the result as binary data. In other words, the pixel block 200 and the circuit 301 can function as part of a neural network.

[0125] The circuit 301 may also have an A / D converter. When image data is to be output to the outside without performing a product-sum operation or the like, the circuit 301 can convert analog data into digital data.

[0126] Furthermore, the data output by the pixel block 200 corresponds to multi-bit image data, but if the circuit 301 can binarize it, it can also be said that the image data is compressed.

[0127] Data output from the circuit 301 is input sequentially to the circuit 302. The circuit 302 can have a configuration including, for example, a latch circuit and a shift register. This configuration enables parallel-serial conversion, and data input in parallel can be output as serial data to the wiring 315. There are no limitations on the connection destination of the wiring 315. For example, the wiring 315 can be connected to a neural network, a storage device, a communication device, or the like.

[0128] 13B, the circuit 302 may have a neural network configuration. The neural network has memory cells arranged in a matrix, each holding a weighting coefficient. Data output from the circuit 301 is input to memory cells 320, where a product-sum operation can be performed. Note that the number of memory cells shown in FIG. 12B is an example and is not limited thereto.

[0129] The neural network shown in FIG. 13B includes memory cells 320 and reference memory cells 325 arranged in a matrix, a circuit 330, a circuit 350, a circuit 360, and a circuit 370.

[0130] 14 shows an example of a memory cell 320 and a reference memory cell 325. The reference memory cell 325 is provided in any one column. The memory cell 320 and the reference memory cell 325 have the same configuration, and each include a transistor 161, a transistor 162, and a capacitor 163.

[0131] One of the source or drain of the transistor 161 is electrically connected to the gate of the transistor 162. The gate of the transistor 162 is electrically connected to one electrode of the capacitor 163. Here, the point where one of the source or drain of the transistor 161, the gate of the transistor 162, and one electrode of the capacitor 163 are connected is referred to as a node NM.

[0132] A gate of the transistor 161 is electrically connected to a wiring WL. The other electrode of the capacitor 163 is electrically connected to a wiring RW. One of the source and the drain of the transistor 162 is electrically connected to a reference potential wiring such as a GND wiring.

[0133] In the memory cell 320, the other of the source and the drain of the transistor 161 is electrically connected to a wiring WD. The other of the source and the drain of the transistor 162 is electrically connected to a wiring BL.

[0134] In the reference memory cell 325, the other of the source and the drain of the transistor 161 is electrically connected to a wiring WDref. The other of the source and the drain of the transistor 162 is electrically connected to a wiring BLref.

[0135] The wiring WL is electrically connected to the circuit 330. The circuit 330 can be a decoder, a shift register, or the like.

[0136] The wiring RW is electrically connected to the circuit 301. Binary data output from the circuit 301 is written to each memory cell. Note that a sequential circuit such as a shift register may be provided between the circuit 301 and each memory cell.

[0137] The wiring WD and the wiring WDref are electrically connected to the circuit 350. The circuit 350 may be a decoder, a shift register, or the like. The circuit 350 may also include a D / A converter and an SRAM. The circuit 350 can output the weighting coefficient written to the node NM.

[0138] The wiring BL and the wiring BLref are electrically connected to the circuit 360. The circuit 360 can have the same structure as the circuit 201. The circuit 360 can obtain a signal obtained by removing the offset component from the result of the product-sum operation.

[0139] Circuit 360 is electrically connected to circuit 370. Circuit 370 can also be described as an activation function circuit. The activation function circuit has a function of performing calculations to convert the signal input from circuit 360 according to a predefined activation function. Examples of the activation function that can be used include a sigmoid function, a tanh function, a softmax function, a ReLU function, and a threshold function. The signal converted by the activation function circuit is output to the outside as output data.

[0140] As shown in FIG. 15A, a neural network NN can be configured with an input layer IL, an output layer OL, and an intermediate layer (hidden layer) HL. The input layer IL, output layer OL, and intermediate layer HL each have one or more neurons (units). The intermediate layer HL may have one layer or two or more layers. A neural network with two or more intermediate layers HL can also be called a DNN (deep neural network). Learning using a deep neural network can also be called deep learning.

[0141] Input data is input to each neuron in the input layer IL. An output signal from a neuron in the previous or next layer is input to each neuron in the hidden layer HL. An output signal from a neuron in the previous layer is input to each neuron in the output layer OL. Each neuron may be connected to all neurons in the previous or next layer (fully connected), or may be connected to only a portion of the neurons in the previous or next layer.

[0142] Figure 15B shows an example of a neuron's operation. It shows neuron N and two neurons in the previous layer that output signals to neuron N. Neuron N receives the output x1 of a neuron in the previous layer and the output x2 of a neuron in the previous layer. Neuron N then multiplies the output x1 by a weight w1 (x1w1) and the output x2 by a weight w2 (x2w2), calculating the sum x1w1+x2w2. After this, a bias b is added as necessary, resulting in a value a = x1w1+x2w2+b. The value a is then transformed by the activation function h, and neuron N outputs an output signal y = ah.

[0143] In this way, the computation by a neuron includes the sum of the product of the output of the neuron in the previous layer and the weight, i.e., the sum-of-products computation (x1w1+x2w2 above). This sum-of-products computation can be performed by software using a program, or by hardware.

[0144] In one embodiment of the present invention, a product-sum operation is performed using an analog circuit as hardware. When an analog circuit is used for the product-sum operation circuit, the circuit scale of the product-sum operation circuit can be reduced, or the number of accesses to a memory can be reduced, thereby improving the processing speed and reducing power consumption.

[0145] The product-sum circuit preferably includes an OS transistor. Since the off-state current of an OS transistor is extremely small, the OS transistor is suitable as a transistor constituting an analog memory of the product-sum circuit. Note that the product-sum circuit may be configured using both a Si transistor and an OS transistor.

[0146] This embodiment mode can be combined with the descriptions of other embodiment modes as appropriate.

[0147] (Embodiment 2) In this embodiment, a structural example of an imaging device according to one embodiment of the present invention will be described.

[0148] <Structure example> FIG. 16A is a diagram showing an example of the structure of a pixel of an imaging device, which can have a stacked structure of a layer 561 and a layer 563.

[0149] The layer 561 includes the photoelectric conversion device 101. The photoelectric conversion device 101 can include a layer 565a and a layer 565b as shown in Fig. 17A. Note that the term "layer" may be replaced with the term "region" in some cases.

[0150] 17A is a pn junction photodiode, and for example, a p-type semiconductor may be used for the layer 565a and an n-type semiconductor may be used for the layer 565b. Alternatively, an n-type semiconductor may be used for the layer 565a and a p-type semiconductor may be used for the layer 565b.

[0151] The pn junction photodiode can be typically formed using single crystal silicon.

[0152] 17B, ​​the photoelectric conversion device 101 included in the layer 561 may be a stack of layers 566a, 566b, 566c, and 566d. The photoelectric conversion device 101 shown in FIG. 17B is an example of an avalanche photodiode, in which the layers 566a and 566d correspond to electrodes, and the layers 566b and 566c correspond to a photoelectric conversion unit.

[0153] The layer 566a is preferably a low-resistance metal layer, such as aluminum, titanium, tungsten, tantalum, silver, or a laminate of these.

[0154] The layer 566d is preferably a conductive layer that has a high light-transmitting property to visible light. For example, indium oxide, tin oxide, zinc oxide, indium-tin oxide, gallium-zinc oxide, indium-gallium-zinc oxide, graphene, or the like can be used. Note that the layer 566d may be omitted.

[0155] The layers 566b and 566c of the photoelectric conversion unit can be configured as a pn junction photodiode with a photoelectric conversion layer made of, for example, a selenium-based material. It is preferable that the layer 566b is made of a selenium-based material, which is a p-type semiconductor, and the layer 566c is made of an n-type semiconductor such as gallium oxide.

[0156] Photoelectric conversion devices using selenium-based materials have the characteristic of high external quantum efficiency for visible light. In these photoelectric conversion devices, avalanche multiplication can be used to increase the amplification of electrons relative to the amount of incident light. Furthermore, selenium-based materials have a high optical absorption coefficient, which offers the advantage of production, such as the ability to fabricate thin-film photoelectric conversion layers. Thin films of selenium-based materials can be formed using vacuum deposition or sputtering.

[0157] As the selenium-based material, crystalline selenium such as single crystal selenium and polycrystalline selenium, amorphous selenium, a compound of copper, indium, and selenium (CIS), or a compound of copper, indium, gallium, and selenium (CIGS) can be used.

[0158] The n-type semiconductor is preferably formed from a material that has a wide band gap and is transparent to visible light. For example, zinc oxide, gallium oxide, indium oxide, tin oxide, or a mixture of these oxides can be used. These materials also function as a hole injection blocking layer and can reduce dark current.

[0159] 17C, the photoelectric conversion device 101 included in the layer 561 may be a laminate of layers 567a, 567b, 567c, 567d, and 567e. The photoelectric conversion device 101 shown in FIG. 17C is an example using an organic photoconductive film, in which the layer 567a is a lower electrode, the layer 567e is a light-transmitting upper electrode, and the layers 567b, 567c, and 567d correspond to photoelectric conversion units.

[0160] One of the layers 567b and 567d of the photoelectric conversion portion can be a hole transport layer, and the other can be an electron transport layer. The layer 567c can be a photoelectric conversion layer.

[0161] For example, molybdenum oxide can be used as the hole transport layer. For example, C 60 , C 70 or derivatives thereof can be used.

[0162] The photoelectric conversion layer may be a mixed layer (bulk heterojunction structure) of an n-type organic semiconductor and a p-type organic semiconductor.

[0163] A silicon substrate, for example, can be used as the layer 563 shown in FIG. 16A. The silicon substrate has Si transistors and the like. Using the Si transistors, in addition to pixel circuits, circuits for driving the pixel circuits, image signal readout circuits, image processing circuits, neural networks, communication circuits, and the like can be formed. Furthermore, memory circuits such as DRAMs (Dynamic Random Access Memory), CPUs (Central Processing Units), MCUs (Micro Controller Units), and the like may also be formed. In this embodiment, the above circuits excluding the pixel circuits are referred to as functional circuits.

[0164] For example, some or all of the transistors included in the pixel circuit (pixel 100) and the functional circuits (circuits 201, 301, 302, 303, 304, 305, 311, 312, 313, and the like) described in Embodiment 1 can be provided in the layer 563.

[0165] Furthermore, the layer 563 may be a laminate of multiple layers as shown in FIG. 16B. Although FIG. 16B illustrates three layers, 563a, 563b, and 563c, two layers may be used. Alternatively, the layer 563 may be a laminate of four or more layers. These layers can be laminated using, for example, a bonding process. With this configuration, the pixel circuits and functional circuits can be distributed across multiple layers and can be stacked on top of each other, thereby enabling the manufacture of a compact, highly functional imaging device.

[0166] Alternatively, the pixel may have a stacked structure of layers 561, 562, and 563 as shown in FIG. 16C.

[0167] The layer 562 can include OS transistors. One or more of the functional circuits described above may be formed using OS transistors. Alternatively, one or more of the functional circuits may be formed using Si transistors included in the layer 563 and OS transistors included in the layer 562. Alternatively, the layer 563 may be used as a support substrate such as a glass substrate, and the functional circuits may be formed using OS transistors included in the layer 562.

[0168] For example, a normally-off CPU (also referred to as an "Noff-CPU") can be realized using OS transistors and Si transistors. Note that an Noff-CPU is an integrated circuit including normally-off transistors that are off (off) even when the gate voltage is 0 V.

[0169] The Noff-CPU can stop the power supply to circuits within the Noff-CPU that are not in operation, putting those circuits into a standby state. When the power supply is stopped and the circuit is in a standby state, no power is consumed. Therefore, the Noff-CPU can minimize power consumption. Furthermore, the Noff-CPU can retain information necessary for operation, such as setting conditions, for a long period of time even if the power supply is stopped. To return from the standby state, it is only necessary to resume the power supply to the circuit, and there is no need to rewrite setting conditions, etc. In other words, it is possible to quickly return from the standby state. In this way, the Noff-CPU can reduce power consumption without significantly reducing operating speed.

[0170] Furthermore, layer 562 may be a laminate of multiple layers as shown in FIG. 16D. While FIG. 16D illustrates two layers, layers 562a and 562b, layer 562 may be a laminate of three or more layers. These layers may be formed by stacking them on layer 563, for example. Alternatively, layer 562 may be formed by bonding a layer formed on layer 563 with a layer formed on layer 561.

[0171] The semiconductor material used for an OS transistor can be a metal oxide with an energy gap of 2 eV or more, preferably 2.5 eV or more, and more preferably 3 eV or more. A typical example is an oxide semiconductor containing indium, such as CAAC-OS or CAC-OS, which will be described later. CAAC-OS has stable atoms constituting the crystal, making it suitable for transistors that prioritize reliability. Furthermore, CAC-OS exhibits high mobility, making it suitable for transistors that operate at high speed.

[0172] Because of the large energy gap of the semiconductor layer, OS transistors exhibit extremely low off-state currents of a few yA / μm (current value per 1 μm of channel width). Furthermore, unlike Si transistors, OS transistors have characteristics such as the absence of impact ionization, avalanche breakdown, and short-channel effects, making them suitable for the formation of high-voltage, highly reliable circuits. Furthermore, OS transistors are less susceptible to variations in electrical characteristics due to non-uniformity in crystallinity, which is a problem with Si transistors.

[0173] The semiconductor layer of the OS transistor can be, for example, a film represented by an In-M-Zn oxide containing indium, zinc, and M (one or more metals selected from aluminum, titanium, gallium, germanium, yttrium, zirconium, lanthanum, cerium, tin, neodymium, hafnium, etc.). The In-M-Zn oxide can be formed by, for example, a sputtering method, an atomic layer deposition (ALD) method, or a metal organic chemical vapor deposition (MOCVD) method.

[0174] When forming an In-M-Zn-based oxide by sputtering, the atomic ratio of the metal elements in the sputtering target preferably satisfies In≧M and Zn≧M. The atomic ratios of the metal elements in such sputtering targets are preferably In:M:Zn=1:1:1, In:M:Zn=1:1:1.2, In:M:Zn=3:1:2, In:M:Zn=4:2:3, In:M:Zn=4:2:4.1, In:M:Zn=5:1:6, In:M:Zn=5:1:7, In:M:Zn=5:1:8, etc. The atomic ratios of the semiconductor layer to be formed may vary within ±40% of the atomic ratios of the metal elements contained in the sputtering target.

[0175] The semiconductor layer is made of an oxide semiconductor with a low carrier density. For example, the semiconductor layer has a carrier density of 1×10 17 / cm 3 Less than 1 × 1015 / cm 3 or less, more preferably 1 × 10 13 / cm 3 Less than or equal to 1×10 11 / cm 3 or less, more preferably 1 × 10 10 / cm 3 Less than 1 x 10 -9 / cm 3 An oxide semiconductor having a carrier density above or equal to this can be used. Such an oxide semiconductor is called a highly purified intrinsic or substantially highly purified intrinsic oxide semiconductor. The oxide semiconductor has a low density of defect states and stable characteristics.

[0176] Note that the present invention is not limited to these, and an appropriate composition may be used depending on the required semiconductor characteristics and electrical characteristics (field-effect mobility, threshold voltage, etc.) of the transistor. In order to obtain the required semiconductor characteristics of the transistor, it is preferable to appropriately set the carrier density, impurity concentration, defect density, atomic ratio of metal element to oxygen, interatomic distance, density, and the like of the semiconductor layer.

[0177] If silicon or carbon, which is one of the group 14 elements, is contained in the oxide semiconductor that constitutes the semiconductor layer, oxygen vacancies increase, resulting in n-type conductivity. For this reason, the concentration of silicon or carbon in the semiconductor layer (concentration obtained by secondary ion mass spectrometry) is set to 2×10 18 atoms / cm 3 Less than or equal to 2 x 10 17 atoms / cm 3 The following applies.

[0178] In addition, when an alkali metal or alkaline earth metal is bonded to an oxide semiconductor, it may generate carriers, which may increase the off-state current of a transistor. Therefore, the concentration of the alkali metal or alkaline earth metal in the semiconductor layer (concentration obtained by secondary ion mass spectrometry) is set to 1×10 18 atoms / cm 3 Less than or equal to 2 x 10 16 atoms / cm 3Do the following:

[0179] Furthermore, if nitrogen is contained in the oxide semiconductor that constitutes the semiconductor layer, electrons acting as carriers are generated, increasing the carrier density and making the semiconductor layer more likely to be n-type. As a result, transistors using oxide semiconductors that contain nitrogen tend to have normally-on characteristics. Therefore, the nitrogen concentration in the semiconductor layer (concentration obtained by secondary ion mass spectrometry) is 5×10 18 atoms / cm 3 It is preferable to do the following:

[0180] Furthermore, if hydrogen is contained in an oxide semiconductor constituting a semiconductor layer, it may react with oxygen bonded to metal atoms to form water, which may form oxygen vacancies in the oxide semiconductor. If oxygen vacancies are present in the channel formation region of an oxide semiconductor, the transistor may exhibit normally-on characteristics. Furthermore, defects in which hydrogen enters the oxygen vacancies may function as donors and generate electrons that serve as carriers. Furthermore, some of the hydrogen may bond with oxygen that is bonded to metal atoms to generate electrons that serve as carriers. Therefore, a transistor using an oxide semiconductor containing a large amount of hydrogen is likely to exhibit normally-on characteristics.

[0181] A defect in which hydrogen is introduced into an oxygen vacancy can function as a donor in an oxide semiconductor. However, it is difficult to quantitatively evaluate such defects. Therefore, oxide semiconductors are sometimes evaluated using carrier concentration instead of donor concentration. Therefore, in this specification and the like, a carrier concentration assuming a state in which no electric field is applied may be used as a parameter of an oxide semiconductor instead of donor concentration. In other words, the "carrier concentration" described in this specification and the like may be rephrased as "donor concentration."

[0182] Therefore, it is preferable that the hydrogen concentration in the oxide semiconductor be reduced as much as possible. Specifically, the hydrogen concentration in the oxide semiconductor measured by secondary ion mass spectrometry (SIMS) is 1×10 20 atoms / cm3 Less than 1 x 10 19 atoms / cm 3 less than 5 × 10 18 atoms / cm 3 less than 1×10 18 atoms / cm 3 When an oxide semiconductor in which impurities such as hydrogen are sufficiently reduced is used for a channel formation region of a transistor, stable electrical characteristics can be obtained.

[0183] The semiconductor layer may also have a non-single-crystal structure. Examples of the non-single-crystal structure include a c-axis aligned crystalline oxide semiconductor (CAAC-OS) having crystals oriented along the c-axis, a polycrystalline structure, a microcrystalline structure, and an amorphous structure. Among non-single-crystal structures, the amorphous structure has the highest density of defect states, and the CAAC-OS has the lowest density of defect states.

[0184] An amorphous oxide semiconductor film has, for example, a disordered atomic arrangement and does not contain any crystalline components, or an amorphous oxide film has, for example, a completely amorphous structure and does not contain any crystalline parts.

[0185] The semiconductor layer may be a mixed film having two or more of an amorphous structure region, a microcrystalline structure region, a polycrystalline structure region, a CAAC-OS region, and a single-crystal structure region. The mixed film may have a single layer structure or a multilayer structure including two or more of the above-mentioned regions.

[0186] The following describes the structure of a cloud-aligned composite (CAC)-OS, which is one type of non-single-crystal semiconductor layer.

[0187] CAC-OS is a material in which, for example, elements constituting an oxide semiconductor are unevenly distributed in a size range of 0.5 nm to 10 nm, preferably 1 nm to 2 nm, or in the vicinity thereof. Note that hereinafter, a state in which one or more metal elements are unevenly distributed in an oxide semiconductor and regions containing the metal elements are mixed in a size range of 0.5 nm to 10 nm, preferably 1 nm to 2 nm, or in the vicinity thereof, is also referred to as a mosaic or patch state.

[0188] The oxide semiconductor preferably contains at least indium, particularly indium and zinc, and may further contain one or more elements selected from aluminum, gallium, yttrium, copper, vanadium, beryllium, boron, silicon, titanium, iron, nickel, germanium, zirconium, molybdenum, lanthanum, cerium, neodymium, hafnium, tantalum, tungsten, magnesium, and the like.

[0189] For example, CAC-OS in In-Ga-Zn oxide (In-Ga-Zn oxide among CAC-OS may be particularly referred to as CAC-IGZO) is an indium oxide (hereinafter referred to as InO X1 (X1 is a real number greater than 0) or indium zinc oxide (hereinafter referred to as In X2 Zn Y2 O Z2 (X2, Y2, and Z2 are real numbers greater than 0.) and gallium oxide (hereinafter referred to as GaO X3 (X3 is a real number greater than 0) or gallium zinc oxide (Ga X4 Zn Y4 O Z4 (X4, Y4, and Z4 are real numbers greater than 0).) The material is separated into mosaics, and the mosaic InO X1 , or In X2 Zn Y2 O Z2 However, the structure is such that the particles are uniformly distributed in the film (hereinafter also referred to as a cloud-like structure).

[0190] In other words, CAC-OS is X3 The region where In is the main component and X2 Zn Y2 O Z2 , or InO X1 In this specification, for example, when the atomic ratio of In to element M in the first region is greater than the atomic ratio of In to element M in the second region, the first region is said to have a higher In concentration than the second region.

[0191] IGZO is a common name and may refer to a compound made of In, Ga, Zn, and O. A typical example is InGaO3(ZnO). m1 (m1 is a natural number), or In (1+x0) Ga (1-x0) O3(ZnO) m0 (-1≦x0≦1, m0 is an arbitrary number).

[0192] The crystalline compounds have a single crystal structure, a polycrystalline structure, or a CAAC structure, where multiple IGZO nanocrystals are connected together with their c-axis orientation and no orientation in the ab plane.

[0193] On the other hand, CAC-OS refers to the material structure of an oxide semiconductor. CAC-OS is a material structure containing In, Ga, Zn, and O, in which some regions observed as nanoparticles mainly composed of Ga and some regions observed as nanoparticles mainly composed of In are randomly dispersed in a mosaic pattern. Therefore, the crystal structure is a secondary element in CAC-OS.

[0194] Note that CAC-OS does not include a stacked structure of two or more films with different compositions, such as a two-layer structure consisting of a film mainly containing In and a film mainly containing Ga.

[0195] In addition, GaO X3The region where In is the main component and X2 Zn Y2 O Z2 , or InO X1 In some cases, a clear boundary between the region where the main component is the chromatic aberration and the region where the chromatic aberration is the main component may not be observed.

[0196] When one or more elements selected from aluminum, yttrium, copper, vanadium, beryllium, boron, silicon, titanium, iron, nickel, germanium, zirconium, molybdenum, lanthanum, cerium, neodymium, hafnium, tantalum, tungsten, and magnesium are contained instead of gallium, the CAC-OS has a structure in which some regions observed to be nanoparticles containing the metal element as the main component and some regions observed to be nanoparticles containing In as the main component are randomly dispersed in a mosaic pattern.

[0197] The CAC-OS can be formed, for example, by a sputtering method under conditions where the substrate is not intentionally heated. When the CAC-OS is formed by a sputtering method, one or more of an inert gas (typically argon), oxygen gas, and nitrogen gas may be used as the deposition gas. The lower the flow rate ratio of oxygen gas to the total flow rate of deposition gas during deposition, the better. For example, the flow rate ratio of oxygen gas is preferably 0% or more and less than 30%, and more preferably 0% or more and 10% or less.

[0198] CAC-OS has the characteristic that no clear peaks are observed when measured using the θ / 2θ scan by the out-of-plane X-ray diffraction (XRD) method, which indicates that the ab-plane and c-axis orientations of the measured region are not observed.

[0199] In addition, in the electron beam diffraction pattern obtained by irradiating CAC-OS with an electron beam (also called nanobeam electron beam) with a probe diameter of 1 nm, a ring-shaped region of high brightness (ring region) and multiple bright spots are observed in the ring region. Therefore, the electron beam diffraction pattern indicates that the crystal structure of CAC-OS has an nc (nano-crystal) structure that does not have orientation in the planar and cross-sectional directions.

[0200] For example, in the case of CAC-OS made of In-Ga-Zn oxide, EDX mapping obtained using energy dispersive X-ray spectroscopy (EDX) revealed that GaO X3 The region where In is the main component and X2 Zn Y2 O Z2 , or InO X1 It can be seen that the region where the main component is the crystalline silicon is unevenly distributed and mixed.

[0201] CAC-OS has a different structure from IGZO compounds, in which metal elements are uniformly distributed, and has different properties from IGZO compounds. X3 The region where In is the main component. X2 Zn Y2 O Z2 , or InO X1 The structure is such that the regions are separated into a mosaic of regions each containing one of the elements as the main component and a region each containing one of the elements as the main component.

[0202] Here, In X2 Zn Y2 O Z2 , or InO X1 The region where is the main component is GaO X3 This region has higher conductivity than the region where In is the main component. X2 Zn Y2 O Z2 , or InO X1 When carriers flow through the region where In is the main component, the conductivity of the oxide semiconductor is exhibited.X2 Zn Y2 O Z2 , or InO X1 When the region mainly composed of is distributed in a cloud-like shape in the oxide semiconductor, a high field-effect mobility (μ) can be achieved.

[0203] On the other hand, GaO X3 The region where the main components are In X2 Zn Y2 O Z2 , or InO X1 This region has higher insulating properties than the region where GaO is the main component. X3 When a region containing the above as a main component is distributed in the oxide semiconductor, leakage current can be suppressed and good switching operation can be achieved.

[0204] Therefore, when CAC-OS is used in a semiconductor device, GaO X3 Insulation due to X2 Zn Y2 O Z2 , or InO X1 The conductivity due to the gate insulating layer and the gate insulating layer work in a complementary manner, resulting in a high on-state current (I on ), and high field-effect mobility (μ) can be achieved.

[0205] Furthermore, semiconductor elements using CAC-OS have high reliability, making CAC-OS suitable as a constituent material for various semiconductor devices.

[0206] <Laminated structure 1> Next, the layered structure of the imaging device will be described using cross-sectional views. Note that the elements such as the insulating layer and conductive layer shown below are examples, and other elements may be included. Alternatively, some of the elements shown below may be omitted. Furthermore, the layered structure shown below can be formed using a bonding process, a polishing process, or the like, as necessary.

[0207] FIG. 18 is an example of a cross-sectional view of a laminate having layers 560, 561, and 563, with a bonding surface between layers 563a and 563b that constitute layer 563.

[0208] <layer 563b> The layer 563b has a functional circuit provided on the silicon substrate 611. Here, a capacitor 202, a transistor 203, and a transistor 204 included in the circuit 201 are shown as parts of the functional circuit. One electrode of the capacitor 202, one of the source or drain of the transistor 203, and the gate of the transistor 204 are electrically connected.

[0209] The layer 563b includes a silicon substrate 611, insulating layers 612, 613, 614, 615, 616, 617, and 618, and a conductive layer 619. The insulating layer 612 functions as a protective film. The insulating layers 613, 614, 616, and 617 function as an interlayer insulating film and a planarizing film. The insulating layer 615 functions as a dielectric layer of the capacitor 202. The insulating layer 618 and the conductive layer 619 function as bonding layers. The conductive layer 619 is electrically connected to one electrode of the capacitor 202.

[0210] The protective film may be, for example, a silicon nitride film, a silicon oxide film, or an aluminum oxide film. The interlayer insulating film and the planarizing film may be, for example, an inorganic insulating film such as a silicon oxide film, or an organic insulating film such as an acrylic resin or a polyimide resin. The dielectric layer of the capacitor may be, for example, a silicon nitride film, a silicon oxide film, or an aluminum oxide film. The lamination layer will be described later.

[0211] Conductors that can be used as wiring, electrodes, and plugs for electrical connection between devices may be made of a metal element selected from aluminum, chromium, copper, silver, gold, platinum, tantalum, nickel, titanium, molybdenum, tungsten, hafnium, vanadium, niobium, manganese, magnesium, zirconium, beryllium, indium, ruthenium, iridium, strontium, lanthanum, etc., or an alloy containing the above-mentioned metal elements as a component, or an alloy combining the above-mentioned metal elements, etc. The conductor is not limited to a single layer, and may also be made of multiple layers composed of different materials.

[0212] <Layer 563a> The layer 563a includes elements of the pixel 100. Here, the transistor 102 and the transistor 105 are shown as some of the elements of the pixel 100. In the cross-sectional view shown in Figure 18, the electrical connection between them is not shown.

[0213] The layer 563a is provided with a silicon substrate 632, insulating layers 631, 633, 634, 635, 637, and 638, and conductive layers 636 and 639.

[0214] The insulating layer 631 and the conductive layer 639 function as bonding layers. The insulating layers 634, 635, and 637 function as interlayer insulating films and planarizing films. The insulating layer 633 functions as a protective film. The insulating layer 638 functions to insulate the silicon substrate 632 from the conductive layer 639. The insulating layer 638 can be formed of the same material as the other insulating layers. Alternatively, the insulating layer 638 may be formed of the same material as the insulating layer 631.

[0215] The conductive layer 639 is electrically connected to the other of the source and the drain of the transistor 105 and the conductive layer 619. The conductive layer 636 is electrically connected to the wiring 114 (see FIG. 3A).

[0216] The Si transistor shown in Fig. 18 is a fin type having a channel formation region in a silicon substrate (silicon substrates 611 and 632). A cross section in the channel width direction (a cross section taken along A1-A2 in layer 563a in Fig. 18) is shown in Fig. 19A. The Si transistor may also be a planar type, as shown in Fig. 19B.

[0217] 19C, the transistor may have a silicon thin-film semiconductor layer 545. The semiconductor layer 545 may be, for example, single-crystal silicon (SOI (Silicon on Insulator)) formed on an insulating layer 546 on a silicon substrate 632.

[0218] <layer 561> The layer 561 has a photoelectric conversion device 101. The photoelectric conversion device 101 can be formed on the layer 563a. Fig. 18 shows a configuration in which the organic photoconductive film shown in Fig. 17C is used as the photoelectric conversion layer for the photoelectric conversion device 101. Here, the layer 567a is the cathode, and the layer 567e is the anode.

[0219] Layer 561 is provided with insulating layers 651, 652, 653, 654 and a conductive layer 655.

[0220] The insulating layers 651, 653, and 654 function as an interlayer insulating film and a planarizing film. The insulating layer 654 is provided to cover the end of the photoelectric conversion device 101 and also functions to prevent a short circuit between the layer 567e and the layer 567a. The insulating layer 652 functions as an element isolation layer. An organic insulating film or the like is preferably used as the element isolation layer.

[0221] The layer 567a, which corresponds to the cathode of the photoelectric conversion device 101, is electrically connected to one of the source and drain of the transistor 102 included in the layer 563a. The layer 567e, which corresponds to the anode of the photoelectric conversion device 101, is electrically connected to the conductive layer 636 included in the layer 563a via the conductive layer 655.

[0222] <layer 560> The layer 560 is formed on the layer 561. The layer 560 includes a light-shielding layer 671, an optical conversion layer 672, and a microlens array 673.

[0223] The light-shielding layer 671 can prevent light from flowing into adjacent pixels. A metal layer such as aluminum or tungsten can be used for the light-shielding layer 671. The metal layer may also be stacked with a dielectric film that functions as an anti-reflection film.

[0224] A color filter can be used for the optical conversion layer 672. A color image can be obtained by assigning colors such as red (red), green (G), blue (B), yellow (Y), cyan (C), and magenta (M) to the color filter for each pixel.

[0225] Furthermore, if a wavelength cut filter is used in the optical conversion layer 672, an imaging device that can obtain images in various wavelength regions can be obtained.

[0226] For example, if a filter that blocks light with wavelengths equal to or shorter than visible light is used in the optical conversion layer 672, it can be used as an infrared imaging device. Also, if a filter that blocks light with wavelengths equal to or shorter than near-infrared light is used in the optical conversion layer 672, it can be used as a far-infrared imaging device. Also, if a filter that blocks light with wavelengths equal to or longer than visible light is used in the optical conversion layer 672, it can be used as an ultraviolet imaging device.

[0227] Furthermore, if a scintillator is used for the optical conversion layer 672, an imaging device can be provided that obtains an image that visualizes the intensity of radiation, such as for use in an X-ray imaging device. When radiation such as X-rays that has passed through a subject is incident on the scintillator, it is converted into light (fluorescence) such as visible light or ultraviolet light by the photoluminescence phenomenon. Then, image data is obtained by detecting this light with the photoelectric conversion device 101. An imaging device having this configuration may also be used for a radiation detector or the like.

[0228] Scintillators contain materials that absorb the energy of radiation such as X-rays or gamma rays and emit visible or ultraviolet light when irradiated with such radiation. For example, materials such as Gd2O2S:Tb, Gd2O2S:Pr, Gd2O2S:Eu, BaFCl:Eu, NaI, CsI, CaF2, BaF2, CeF3, LiF, LiI, and ZnO dispersed in resin or ceramics can be used.

[0229] A microlens array 673 is provided on the optical conversion layer 672. Light passing through each lens of the microlens array 673 passes through the optical conversion layer 672 directly below and is irradiated onto the photoelectric conversion device 101. By providing the microlens array 673, concentrated light can be incident on the photoelectric conversion device 101, thereby enabling efficient photoelectric conversion. The microlens array 673 is preferably formed from a resin or glass that is highly translucent to visible light.

[0230] <Laminating> Next, the bonding of the layer 563b and the layer 563a will be described.

[0231] The layer 563b is provided with an insulating layer 618 and a conductive layer 619. The conductive layer 619 has a region buried in the insulating layer 618. The surfaces of the insulating layer 618 and the conductive layer 619 are flattened so that they are at the same height.

[0232] The layer 563a is provided with an insulating layer 631 and a conductive layer 639. The conductive layer 639 has a region buried in the insulating layer 631. The surfaces of the insulating layer 631 and the conductive layer 639 are flattened so that they are at the same height.

[0233] Here, the conductive layer 619 and the conductive layer 639 preferably contain the same metal element as a main component. The insulating layer 618 and the insulating layer 631 preferably contain the same component.

[0234] For example, Cu, Al, Sn, Zn, W, Ag, Pt, or Au can be used for the conductive layers 619 and 639. Cu, Al, W, or Au is preferred for ease of bonding. Silicon oxide, silicon oxynitride, silicon nitride oxide, silicon nitride, titanium nitride, or the like can be used for the insulating layers 618 and 631.

[0235] That is, the same metal material as described above is preferably used for the conductive layer 619 and the conductive layer 639. The same insulating material as described above is preferably used for the insulating layer 618 and the insulating layer 631. With this structure, the layer 563b and the layer 563a can be bonded together at the boundary between them.

[0236] The conductive layers 619 and 639 may have a multilayer structure with multiple layers, in which case the surface layers (joint surfaces) may be made of the same metal material. The insulating layers 618 and 631 may also have a multilayer structure with multiple layers, in which case the surface layers (joint surfaces) may be made of the same insulating material.

[0237] This bonding makes it possible to obtain electrical connection between the conductive layer 619 and the conductive layer 639. Furthermore, it is possible to obtain connection between the insulating layer 618 and the insulating layer 631 with sufficient mechanical strength.

[0238] To bond metal layers together, surface activated bonding can be used, in which oxide films and impurity adsorption layers on the surfaces are removed by sputtering or other methods, and the cleaned and activated surfaces are then brought into contact and bonded. Alternatively, diffusion bonding can be used, in which surfaces are bonded using a combination of temperature and pressure. Both methods create bonds at the atomic level, resulting in excellent bonding not only electrically but also mechanically.

[0239] Furthermore, to bond insulating layers together, a hydrophilic bonding method can be used, in which high flatness is achieved by polishing or other methods, then surfaces that have been hydrophilically treated with oxygen plasma or other methods are brought into contact with each other to form a temporary bond, and the final bond is then achieved by dehydrating them through heat treatment.Hydrophilic bonding also creates bonds at the atomic level, so it is possible to obtain mechanically excellent bonds.

[0240] When bonding the layer 563b and the layer 563a, an insulating layer and a metal layer are mixed on each bonding surface, so that, for example, a surface activated bonding method and a hydrophilic bonding method may be combined.

[0241] For example, a method can be used in which the surface is polished, cleaned, the surface of the metal layer is subjected to an anti-oxidation treatment, and then a hydrophilic treatment is performed before bonding. Alternatively, the surface of the metal layer may be made of a resistant metal such as Au and then subjected to a hydrophilic treatment. Note that bonding methods other than those described above may also be used.

[0242] By the above-described bonding, the circuit 201 included in the layer 563b can be electrically connected to the elements of the pixel 100 included in the layer 563a.

[0243] <Modification of laminate structure 1> FIG. 20 is a modified example of the stacked structure shown in FIG. 18, in which the configuration of the photoelectric conversion device 101 in the layer 561 and a portion of the configuration of the layer 563a are different, and there is also a bonding surface between the layer 561 and the layer 563a.

[0244] Layer 561 includes photovoltaic device 101, insulating layers 661, 662, 664, 665 and conductive layers 135, 136.

[0245] The photoelectric conversion device 101 is a pn junction photodiode formed on a silicon substrate, and has a layer 565b corresponding to a p-type region and a layer 565a corresponding to an n-type region. The photoelectric conversion device 101 is a buried photodiode, and a thin p-type region (part of the layer 565b) provided on the surface side (current extraction side) of the layer 565a can suppress dark current and reduce noise.

[0246] The insulating layer 661 and the conductive layers 135 and 136 function as bonding layers. The insulating layer 662 functions as an interlayer insulating film and a planarizing film. The insulating layer 664 functions as an element isolation layer. The insulating layer 665 functions to suppress the outflow of carriers.

[0247] Grooves that separate pixels are provided in the silicon substrate, and an insulating layer 665 is provided on the upper surface of the silicon substrate and in the grooves. The insulating layer 665 can prevent carriers generated in the photoelectric conversion device 101 from flowing into adjacent pixels. The insulating layer 665 also has the function of preventing stray light from entering. Therefore, the insulating layer 665 can prevent color mixing. An anti-reflection film may be provided between the upper surface of the silicon substrate and the insulating layer 665.

[0248] The element isolation layer can be formed by using a LOCOS (LOCal Oxidation of Silicon) method. Alternatively, it may be formed by using a STI (Shallow Trench Isolation) method or the like. The insulating layer 665 may be, for example, an inorganic insulating film such as silicon oxide or silicon nitride, or an organic insulating film such as polyimide or acrylic. The insulating layer 665 may have a multi-layer structure.

[0249] A layer 565a (n-type region, corresponding to the cathode) of the photoelectric conversion device 101 is electrically connected to the conductive layer 135. A layer 565b (p-type region, corresponding to the anode) is electrically connected to the conductive layer 136. The conductive layers 135 and 136 have regions buried in an insulating layer 661. The surfaces of the insulating layer 661 and the conductive layers 135 and 136 are flattened so that they are at the same height.

[0250] In the layer 563a, an insulating layer 638 is formed over the insulating layer 637. A conductive layer 133 electrically connected to one of the source and the drain of the transistor 102, and a conductive layer 134 electrically connected to the conductive layer 636 are formed.

[0251] The insulating layer 638 and the conductive layers 133 and 134 function as bonding layers. The conductive layers 133 and 134 have regions buried in the insulating layer 638. The surfaces of the insulating layer 638 and the conductive layers 133 and 134 are flattened so that they are at the same height.

[0252] Here, the conductive layers 133, 134, 135, and 136 are the same bonding layers as the above-described conductive layers 619 and 639. The insulating layers 638 and 661 are the same bonding layers as the above-described insulating layers 618 and 631.

[0253] Therefore, by bonding the conductive layer 133 and the conductive layer 135 together, it is possible to electrically connect the layer 565a (n-type region, corresponding to the cathode) of the photoelectric conversion device to either the source or the drain of the transistor 102. Also, by bonding the conductive layer 134 and the conductive layer 136 together, it is possible to electrically connect the layer 565b (p-type region, corresponding to the anode) of the photoelectric conversion device to the wiring 114 (see FIG. 3A). Also, by bonding the insulating layer 638 and the insulating layer 661 together, it is possible to electrically and mechanically bond the layer 561 to the layer 563a.

[0254] <Laminated structure 2> 21 is an example cross-sectional view of a stack having layers 560, 561, 562, and 563 and no bonding surface. A Si transistor is provided in the layer 563. An OS transistor is provided in the layer 562. Note that the structures of the layers 563, 561, and 560 are the same as those shown in FIG. 18, and therefore description thereof will be omitted here.

[0255] <layer 562> Layer 562 is formed on layer 563. Layer 562 includes an OS transistor. Here, transistor 102 and transistor 105 are shown as part of the elements of pixel 100. In the cross-sectional view shown in FIG. 21, the electrical connection between them is not shown.

[0256] The layer 562 includes insulating layers 621, 622, 623, 624, 625, 626, and 628. A conductive layer 627 is also provided. The conductive layer 627 can be electrically connected to the wiring 114 (see FIG. 3A).

[0257] The insulating layer 621 functions as a blocking layer. The insulating layers 622, 623, 625, 626, and 628 function as an interlayer insulating film and a planarizing film. The insulating layer 624 functions as a protective film.

[0258] The blocking layer is preferably a film that has a function of preventing hydrogen diffusion. In Si devices, hydrogen is required to terminate dangling bonds. However, hydrogen near an OS transistor can generate carriers in the oxide semiconductor layer, reducing reliability. Therefore, a hydrogen blocking film is preferably provided between the layer where the Si device is formed and the layer where the OS transistor is formed.

[0259] The blocking film may be made of, for example, aluminum oxide, aluminum oxynitride, gallium oxide, gallium oxynitride, yttrium oxide, yttrium oxynitride, hafnium oxide, hafnium oxynitride, yttria-stabilized zirconia (YSZ), or the like.

[0260] The other of the source and the drain of the transistor 105 is electrically connected to one electrode of the capacitor 202 through a plug. The conductive layer 627 is electrically connected to the wiring 114 (see FIG. 3A).

[0261] One of the source and drain of the transistor 102 is electrically connected to the cathode of the photoelectric conversion device 101 included in the layer 561. The conductive layer 627 is electrically connected to the anode of the photoelectric conversion device 101 included in the layer 561.

[0262] 22A shows details of an OS transistor. The OS transistor shown in FIG. 22A has a self-aligned structure in which an insulating layer is provided over a stack of an oxide semiconductor layer and a conductive layer, and a source electrode 705 and a drain electrode 706 are formed by providing openings that reach the oxide semiconductor layer.

[0263] The OS transistor can have a structure including a channel formation region, a source region 703, and a drain region 704 formed in an oxide semiconductor layer, as well as a gate electrode 701 and a gate insulating film 702. At least the gate insulating film 702 and the gate electrode 701 are provided in the opening. An oxide semiconductor layer 707 may be further provided in the opening.

[0264] As shown in FIG. 22B, the OS transistor may have a self-aligned structure in which a source region 703 and a drain region 704 are formed in a semiconductor layer using a gate electrode 701 as a mask.

[0265] Alternatively, as shown in FIG. 22C, it may be a non-self-aligned top-gate transistor having a region where the source electrode 705 or the drain electrode 706 overlaps with the gate electrode 701.

[0266] Although the OS transistor has a back gate 535, it may not have a back gate. The back gate 535 may be electrically connected to the front gate of the transistor provided opposite to it, as shown in the cross-sectional view of the transistor in the channel width direction in FIG. 22D. Note that FIG. 22D shows the cross section of the transistor taken along line B1-B2 in FIG. 22A as an example, but the same applies to transistors with other structures. Furthermore, a fixed potential different from that of the front gate may be supplied to the back gate 535.

[0267] <Modification of laminate structure 2> 23 is a modified example of the stacked structure shown in FIG. 21, in which the configuration of the photoelectric conversion device 101 in the layer 561 and a portion of the configuration of the layer 562 are different, and a bonding surface is provided between the layer 561 and the layer 562.

[0268] The photoelectric conversion device 101 included in the layer 561 is a pn junction photodiode formed on a silicon substrate, and has the same configuration as that shown in Fig. 20. The insulating layer 663 functions as a blocking layer similar to the insulating layer 621.

[0269] In the layer 562, an insulating layer 648 is formed over the insulating layer 628. In addition, a conductive layer 138 electrically connected to one of the source and the drain of the transistor 102, and a conductive layer 139 electrically connected to the conductive layer 627 are formed.

[0270] The insulating layer 648 and the conductive layers 138 and 139 function as bonding layers. The conductive layers 138 and 139 have regions buried in the insulating layer 648. The surfaces of the insulating layer 648 and the conductive layers 138 and 139 are flattened so that they are at the same height.

[0271] Here, the conductive layers 138 and 139 are the same bonding layer as the above-described conductive layers 619 and 639. The insulating layer 648 is the same bonding layer as the above-described insulating layers 618 and 631.

[0272] Therefore, by bonding the conductive layer 138 and the conductive layer 135 together, it is possible to electrically connect the layer 565a (n-type region, corresponding to the cathode) of the photoelectric conversion device to either the source or the drain of the transistor 102. Also, by bonding the conductive layer 139 and the conductive layer 136 together, it is possible to electrically connect the layer 565b (p-type region, corresponding to the anode) of the photoelectric conversion device to the wiring 114 (see FIG. 3A). Also, by bonding the insulating layer 648 and the insulating layer 661 together, it is possible to electrically and mechanically bond the layer 561 and the layer 562.

[0273] When stacking multiple Si devices, multiple polishing and bonding processes are required. This poses challenges such as a large number of steps, the need for specialized equipment, low yields, and high manufacturing costs. OS transistors can be formed by stacking them on a silicon substrate on which devices are already formed, eliminating the need for bonding processes.

[0274] <Package, Module> 24A1 is a perspective view of the top surface of a package containing an image sensor chip. The package includes a package substrate 410 for fixing an image sensor chip 450 (see FIG. 24A3), a cover glass 420, and an adhesive 430 for bonding the two together.

[0275] 24A2 is a perspective view of the underside of the package. The underside of the package has a BGA (Ball Grid Array) with solder balls as bumps 440. Note that the package is not limited to a BGA, and may have an LGA (Land Grid Array) or a PGA (Pin Grid Array), etc.

[0276] 24A3 is a perspective view of the package, with the cover glass 420 and part of the adhesive 430 omitted. Electrode pads 460 are formed on the package substrate 410, and the electrode pads 460 and bumps 440 are electrically connected via through holes. The electrode pads 460 are electrically connected to the image sensor chip 450 by wires 470.

[0277] 24B1 is a perspective view of the top surface of a camera module in which an image sensor chip is housed in a lens-integrated package. The camera module includes a package substrate 411 for fixing an image sensor chip 451 (see FIG. 24B3), a lens cover 421, and a lens 435. An IC chip 490 (see FIG. 24B3) having functions such as a drive circuit and a signal conversion circuit for the imaging device is also provided between the package substrate 411 and the image sensor chip 451, forming a SiP (System in Package) configuration.

[0278] 24B2 is a perspective view of the appearance of the bottom side of the camera module. The bottom and side surfaces of the package substrate 411 have a QFN (quad flat no-lead package) configuration with mounting lands 441 provided. Note that this configuration is just one example, and a QFP (quad flat package) or the aforementioned BGA may also be provided.

[0279] 24B3 is a perspective view of the module, omitting a portion of lens cover 421 and lens 435. Land 441 is electrically connected to electrode pad 461, and electrode pad 461 is electrically connected to image sensor chip 451 or IC chip 490 by wire 471.

[0280] By housing the image sensor chip in a package of the above-described type, it becomes easy to mount the image sensor chip on a printed circuit board or the like, and the image sensor chip can be incorporated into various semiconductor devices and electronic devices.

[0281] This embodiment mode can be combined with the descriptions of other embodiment modes as appropriate.

[0282] (Embodiment 3) Examples of electronic devices that can use the imaging device according to one embodiment of the present invention include display devices, personal computers, image storage devices or image playback devices equipped with a recording medium, mobile phones, game consoles including portable types, portable data terminals, e-book terminals, cameras such as video cameras and digital still cameras, goggle-type displays (head-mounted displays), navigation systems, audio playback devices (car audio, digital audio players, etc.), copiers, facsimiles, printers, printer-combined machines, automated teller machines (ATMs), vending machines, etc. Specific examples of these electronic devices are shown in FIGS.

[0283] 25A illustrates an example of a mobile phone, which includes a housing 981, a display portion 982, operation buttons 983, an external connection port 984, a speaker 985, a microphone 986, a camera 987, and the like. The mobile phone includes a touch sensor in the display portion 982. Any operation, such as making a call or inputting characters, can be performed by touching the display portion 982 with a finger, a stylus, or the like. The imaging device and the operating method thereof according to one embodiment of the present invention can be applied to the mobile phone.

[0284] 25B shows a portable data terminal including a housing 911, a display portion 912, a speaker 913, a camera 919, and the like. Information can be input and output using a touch panel function of the display portion 912. Characters and the like can be recognized from an image acquired by the camera 919 and output as voice through the speaker 913. The imaging device and the operating method thereof according to one embodiment of the present invention can be applied to the portable data terminal.

[0285] FIG. 25C shows a surveillance camera, which includes a support base 951, a camera unit 952, a protective cover 953, and the like. The camera unit 952 is provided with a rotation mechanism and is installed on the ceiling, enabling it to capture images of the entire periphery. The imaging device and its operating method according to one embodiment of the present invention can be applied to the elements for acquiring images in the camera unit. Note that the term "surveillance camera" is a common name and is not intended to limit the application. For example, a device having the function of a surveillance camera is also called a camera or a video camera.

[0286] 25D shows a video camera including a first housing 971, a second housing 972, a display unit 973, operation keys 974, a lens 975, a connection unit 976, a speaker 977, a microphone 978, and the like. The operation keys 974 and the lens 975 are provided in the first housing 971, and the display unit 973 is provided in the second housing 972. An imaging device and an operation method thereof according to one embodiment of the present invention can be applied to the video camera.

[0287] 25E shows a digital camera including a housing 961, a shutter button 962, a microphone 963, a light-emitting portion 967, a lens 965, etc. The imaging device and the operating method thereof according to one embodiment of the present invention can be applied to this digital camera.

[0288] 25F shows a wristwatch-type information terminal including a display portion 932, a housing / wristband 933, a camera 939, and the like. The display portion 932 includes a touch panel for operating the information terminal. The display portion 932 and the housing / wristband 933 are flexible and therefore easily worn on the body. The imaging device and the operating method thereof according to one embodiment of the present invention can be applied to this information terminal.

[0289] This embodiment mode can be combined with the descriptions of other embodiment modes as appropriate. [Explanation of symbols]

[0290] 100: pixel, 101: photoelectric conversion device, 102: transistor, 103: transistor, 104: transistor, 105: transistor, 106: transistor, 107: transistor, 108: capacitor, 111: wiring, 112: wiring, 114: wiring, 115: wiring, 116: wiring, 121: wiring, 122: wiring, 123: wiring, 124: wiring, 125: wiring, 133: conductive layer, 134: conductive layer, 135: conductive layer, 136: conductive layer, 138: conductive layer, 139: conductive layer, 161: transistor, 162: transistor, 163: capacitor, 170 : Circuit, 200: Pixel block, 201: Circuit, 202: Capacitor, 203: Transistor, 204: Transistor, 205: Transistor, 206: Transistor, 207: Transistor, 211: Wiring, 212: Wiring, 213: Wiring, 215: Wiring, 216: Wiring, 217: Wiring, 218: Wiring, 219: Wiring, 300: Imaging device, 301: Circuit, 302: Circuit, 303: Circuit, 304: Circuit, 305: Circuit, 306: Display device, 310: Pixel array, 311: Circuit, 312: Circuit, 313: Circuit, 315: Wiring, 320: Memory cell, 325: Reference Memory cell, 330: circuit, 350: circuit, 360: circuit, 370: circuit, 410: package substrate, 411: package substrate, 420: cover glass, 421: lens cover, 430: adhesive, 435: lens, 440: bump, 441: land, 450: image sensor chip, 451: image sensor chip, 460: electrode pad, 461: electrode pad, 470: wire, 471: wire, 490: IC chip, 535: back gate, 545: semiconductor layer, 546: insulating layer, 560: layer, 561: layer, 562: layer, 562a: layer, 562b: layer, 5 63: layer, 563a: layer, 563b: layer, 563c: layer, 565a: layer, 565b: layer, 566a: layer, 566b: layer, 566c: layer, 566d: layer, 567a: layer, 567b: layer, 567c: layer, 567d: layer, 567e: layer, 611: silicon substrate, 612: insulating layer, 613: insulating layer, 614: insulating layer, 615: insulating layer, 616: insulating layer, 617: insulating layer, 618: insulating layer, 619: conductive layer, 621: insulating layer, 622: insulating layer, 623: insulating layer, 624: insulating layer, 625: insulating layer, 626: insulating layer, 627: conductive layer, 628: insulating layer, 631: insulating layer,632: silicon substrate, 633: insulating layer, 634: insulating layer, 635: insulating layer, 636: conductive layer, 637: insulating layer, 638: insulating layer, 639: conductive layer, 648: insulating layer, 651: insulating layer, 652: insulating layer, 653: insulating layer, 654: insulating layer, 655: conductive layer, 661: insulating layer, 662: insulating layer, 663: insulating layer, 664: insulating layer, 665: insulating layer, 671: light-shielding layer, 672: optical conversion layer, 673: microlens array, 701: gate electrode, 702: gate insulating film, 703: source region, 704: drain region, 705: source electrode, 706: drain electrode, 707: oxide semiconductor layer , 911: housing, 912: display unit, 913: speaker, 919: camera, 932: display unit, 933: housing and wristband, 939: camera, 951: support stand, 952: camera unit, 953: protective cover, 961: housing, 962: shutter button, 963: microphone, 965: lens, 967: light emitting unit, 971: housing, 972: housing, 973: display unit, 974: operation keys, 975: lens, 976: connection unit, 977: speaker, 978: microphone, 981: housing, 982: display unit, 983: operation button, 984: external connection port, 985: speaker, 986: microphone, 987: camera,

Claims

1. an imaging device, a first semiconductor device, a second semiconductor device, and a third semiconductor device; the imaging device has a function of acquiring first data and second data in each frame period; the imaging device has a function of filtering the first data to generate third data and a function of filtering the second data to generate fourth data; the first semiconductor device has a function of generating fifth data by calculation using third data acquired in an (n-1)th frame (n is a natural number) and fourth data acquired in an nth frame; the first semiconductor device has a function of generating sixth data by calculation using third data acquired in an (n-1)th frame and fourth data acquired in an nth frame; the second semiconductor device has a function of generating an interpolated image using the fifth data and the sixth data; The third semiconductor device is an imaging system having a function of generating moving image data having the order of first data acquired in the nth frame, the interpolated image, and first data acquired in the n+1th frame.

2. In claim 1, An imaging system in which the third data is generated while the second data is being acquired.

3. an imaging device, a first semiconductor device, a second semiconductor device, and a third semiconductor device; the imaging device has a function of acquiring first data and second data; the first data is acquired in each frame period; the imaging device has a function of filtering the first data to generate third data and a function of filtering the second data to generate fourth data; the first semiconductor device has a function of generating fifth data by calculation using third data acquired in an (n-1)th frame (n is a natural number) and fourth data acquired in an nth frame; the first semiconductor device has a function of generating sixth data by calculation using third data acquired in an (n-1)th frame and fourth data acquired in an nth frame; the second semiconductor device has a function of generating an interpolated image using the fifth data and the sixth data; The third semiconductor device is an imaging system having a function of generating moving image data having the order of first data acquired in the nth frame, the interpolated image, and first data acquired in the n+1th frame.

4. In claim 3, An imaging system in which filtering of the first data acquired in the nth frame is performed in the n+1th frame.

5. In any one of claims 1 to 4, The imaging device includes a pixel block, a first circuit, and a second circuit, The pixel block has a plurality of pixels, the first circuit has a function of supplying a first potential or a second potential to each of the pixels; Each of the pixels has a function of acquiring the first data, each of the pixels has a function of adding the first potential to the first data to generate the second data; each of the pixels has a function of adding the second potential to the first data to generate the third data; an imaging system, wherein the second circuit has a function of generating the fourth data corresponding to the difference between the sum of the second data output by the plurality of pixels and the sum of the third data output by the plurality of pixels.

6. In claim 5, each of the pixels includes a first transistor, a second transistor, a third transistor, a fourth transistor, a fifth transistor, a sixth transistor, a photoelectric conversion device, and a capacitor; one electrode of the photoelectric conversion device is electrically connected to one of a source or a drain of the first transistor and one of a source or a drain of the second transistor, and the other of the source or the drain of the first transistor is electrically connected to one of a source or a drain of the third transistor, a gate of the fourth transistor, and one electrode of the capacitor; one of a source or a drain of the fourth transistor is electrically connected to one of a source or a drain of the fifth transistor; The other electrode of the capacitor is electrically connected to one of the source and drain of the sixth transistor.

7. In claim 6, The imaging system, wherein the first circuit is electrically connected to the other of the source and the drain of the sixth transistor.

8. In claim 6 or 7, the second circuit has a function of a correlated double sampling circuit; The imaging system, wherein the second circuit is electrically connected to the other of the source and the drain of the fifth transistor.

9. In any one of claims 6 to 8, one or more of the first to sixth transistors have a metal oxide in a channel formation region, the metal oxide including In, Zn, and M (M is one or more of Al, Ti, Ga, Ge, Sn, Y, Zr, La, Ce, Nd, and Hf).

10. An electronic device comprising: the imaging system according to claim 1; and a display device.

Images