Imaging element and imaging device

Abstract

The imaging element includes: a pixel unit having a plurality of pixels comprising a photoelectric conversion unit that converts light into charge; an exposure processing unit that calculates the accumulation time for accumulating the charge converted by the photoelectric conversion unit; and a replacement unit that replaces a portion of first data related to a signal read from the pixels with second data related to the accumulation time calculated by the exposure processing unit.

Description

Technical Field

[0001] This application claims priority to Japanese Patent Application No. 2024-43884, filed on March 19, 2024 (Reiwa 6), and incorporates it hereby by reference to its contents.

[0002] This invention relates to imaging elements and imaging devices. Background Technology

[0003] A solid-state imaging device with multiple pixel units is known (e.g., Patent Document 1). There has been a persistent pursuit of increasing the dynamic range. Existing technical documents Patent documents

[0004] Patent document 1: Japanese Patent Application Publication No. 2014-75767. Summary of the Invention

[0005] The imaging element of the first disclosure technology includes: a pixel unit having a plurality of pixels, the plurality of pixels including a photoelectric conversion unit that converts light into charge; an exposure processing unit that calculates an accumulation time for accumulating the charge converted by the photoelectric conversion unit; and a replacement unit that replaces a portion of first data related to a signal read from the pixels with second data related to the accumulation time calculated by the exposure processing unit.

[0006] The camera device of the second disclosure technology includes the camera element of the first disclosure technology. Attached Figure Description

[0007] Figure 1 This is an exploded stereoscopic view showing an example of a camera element. Figure 2 This is an explanatory diagram showing an example of the specific structure of a pixel. Figure 3 This is a circuit diagram representing an example of the circuit structure of a pixel. Figure 4 This is an explanatory diagram showing an example of the specific structure of the control circuit section. Figure 5 This is an explanatory diagram showing an example of the internal structure of a control block. Figure 6 This is an explanatory diagram illustrating an example of signal transmission between the first semiconductor substrate and the second semiconductor substrate in an imaging element. Figure 7 This is an explanatory diagram showing an example of the XZ direction cross-section of the imaging element involved in this embodiment. Figure 8 This is a timing diagram representing the camera action of the camera element, as shown in Example 1. Figure 9This is a timing diagram representing the camera action of the camera element, as shown in Example 2. Figure 10 This is a time diagram showing the recording action of the camera element involved in the comparative example. Figure 11 This is an explanatory diagram showing an example of a subject captured by a camera element. Figure 12 It means Figure 11 The exposure time time diagrams for regions 1 to 5 are shown. Figure 13 This is a plan view showing an example layout of multiple control blocks. Figure 14 This is another example of a circuit diagram representing the circuit structure of a pixel. Figure 15 This is a timing diagram representing the camera action of the camera element, as shown in Example 3. Figure 16 This is an exploded stereoscopic view showing another example of a camera element. Figure 17 This is an explanatory diagram showing another example of the specific structure of the control circuit section. Figure 18 This is an explanatory diagram showing the connection relationship between the first semiconductor substrate and the second semiconductor substrate in the imaging element. Figure 19 This is an explanatory diagram illustrating an example of signal transmission between the first semiconductor substrate and the second semiconductor substrate in an imaging element. Figure 20 This is an explanatory diagram showing the connection relationship between the ADC unit and the pixel block. Figure 21 It is a time map representing the camera action within the pixel block of the camera element. Figure 22 This is an illustrative diagram showing an example of the exposure timing for each pixel block. Figure 23 This is a block diagram representing a structural example of autonomous exposure control mode 1. Figure 24 This is a block diagram representing a structural example of autonomous exposure control mode 2. Figure 25 This is a block diagram representing a structural example of autonomous exposure control mode 3. Figure 26 This is an explanatory diagram (Figure 1) showing the image data output from the camera element. Figure 27 This is an explanatory diagram (Figure 2) showing the image data output from the camera element. Figure 28 It is a chart showing the relationship between exposure time and exposure value. Figure 29 This is an illustration of embedding example 1, which shows the embedding of exposure values ​​into block image data. Figure 30 This is an illustration of embedding exposure values ​​into block image data, as shown in Example 2-1. Figure 31 This is an illustration of embedding exposure values ​​into block image data, as shown in Example 2-2. Figure 32 This is an illustration of embedding example 3, which shows the embedding of exposure values ​​into block image data. Figure 33 This is an illustration of embedding example 4, which shows the embedding of exposure values ​​into block image data. Figure 34 This is a block diagram illustrating a structural example of the camera device involved in the embodiment. Detailed Implementation

[0008] The present invention will now be described through embodiments thereof, but these embodiments do not limit the invention as defined in the claims. Furthermore, not all combinations of features described in the embodiments are necessary for the solutions provided in the invention.

[0009] In this specification, the X-axis and Y-axis are orthogonal to each other, and the Z-axis is orthogonal to the XY plane. The XYZ axes form a right-handed system. Sometimes, the direction parallel to the Z-axis is referred to as the stacking direction of the imaging element. In this specification, the terms "up" and "down" are not limited to the vertical direction in the direction of gravity. These terms refer only to the relative direction in the Z-axis direction. Furthermore, although this specification uses "row" for arrangement in the X-axis direction and "column" for arrangement in the Y-axis direction, the row and column directions are not limited to these.

[0010] <Structure of camera element> First, use Figures 1 to 22 The structure of the camera element is described. The camera element can be constructed as either a back-illuminated type or a surface-illuminated type.

[0011] Figure 1 This is an exploded perspective view showing an example of an image sensor 100A. The image sensor 100A captures a subject. The image sensor 100A generates image data of the captured subject. The image sensor 100A includes a first semiconductor substrate 110, a second semiconductor substrate 120, and a third semiconductor substrate 130. Figure 1 As shown, the first semiconductor substrate 110 is stacked on the second semiconductor substrate 120, and the second semiconductor substrate 120 is stacked on the third semiconductor substrate 130.

[0012] The first semiconductor substrate 110 has a pixel section 101. The pixel section 101 outputs a pixel signal obtained based on the incident light.

[0013] The second semiconductor substrate 120 includes a control circuit section 102 and a peripheral circuit section 121. Furthermore, the peripheral circuit section 121 allows for separate circuit configuration on the first and second semiconductor substrates. This enables efficient circuit configuration and reduces chip area.

[0014] The control circuit unit 102 receives a pixel signal output from the first semiconductor substrate 110. The control circuit unit 102 processes the received pixel signal. The control circuit unit 102 is disposed in the second semiconductor substrate 120 at a position opposite to the pixel unit 101. For example, the control circuit unit 102 is configured to overlap with the pixel unit 101 in the direction in which the first semiconductor substrate 110 and the second semiconductor substrate 120 are stacked. The control circuit unit 102 can output control signals for controlling the drive of the pixel unit 101 to the pixel unit 101.

[0015] The peripheral circuit section 121 controls the driving of the control circuit section 102. The peripheral circuit section 121 is disposed on the periphery of the control circuit section 102 in the second semiconductor substrate 120. Specifically, the peripheral circuit section 121 is disposed in a region of the second semiconductor substrate 120 that is outside the region where the control circuit section 102 is disposed. Alternatively, the peripheral circuit section 121 may be electrically connected to the first semiconductor substrate 110 to control the driving of the pixel section 101. The peripheral circuit section 121 is disposed along two edges of the second semiconductor substrate 120, but the arrangement method of the peripheral circuit section 121 is not limited to this example.

[0016] The third semiconductor substrate 130 has a data processing unit 103. The data processing unit 103 uses digital data output from the second semiconductor substrate 120 to perform addition processing, interval elimination processing, and other image processing.

[0017] Figure 2 This is an explanatory diagram showing an example of the specific structure of the pixel section 101. The pixel section 101 has a plurality of pixel blocks 200. The plurality of pixel blocks 200 are arranged in the pixel section 101 along the row and column directions. Specifically, the plurality of pixel blocks 200 has M×N (M and N are natural numbers) pixel blocks 200 arranged in the pixel section 101 along the row and column directions. Although the case where M and N are equal is illustrated, M and N can also be different.

[0018] Pixel block 200 has a plurality of pixels 201. The plurality of pixels 201 are arranged in the pixel block 200 along both row and column directions. Pixel block 200 has m × n pixels 201 (where m and n are natural numbers) arranged along both row and column directions. For example, pixel block 200 has 16 × 16 pixels 201 arranged along both row and column directions. The number of pixels 201 corresponding to pixel block 200 is not limited to this. Although the illustration shows the case where m and n are equal, m can also be different from n.

[0019] Pixel block 200 has a plurality of pixels 201 connected in the row direction to a common control line (e.g., transmission control line 311 and discharge control line 312 described later). For example, each pixel 201 of pixel block 200 is connected to the aforementioned common control line to be set to the same exposure time. Specifically, for example, each of the n pixels 201 arranged in the row direction is connected by the aforementioned common control line.

[0020] Alternatively, between different pixel blocks 200, one pixel block 200 may be set to an exposure time different from that of another pixel block 200. For example, when one pixel block 200 and another pixel block 200 are arranged in a row direction, the pixels 201 of one pixel block 200 and the pixels 201 of another pixel block 200 are connected by different control lines. The pixels 201 in the m-th row of one pixel block 200 are connected by control lines different from the common control lines connecting the pixels 201 in the m-th row of the other pixel block 200. Furthermore, when one pixel block 200 and another pixel block 200 are arranged in a column direction, the pixels 201 of one pixel block 200 and the pixels 201 of another pixel block 200 are connected by different control lines. The pixels 201 in the m-th row of one pixel block 200 are connected by control lines different from the common control lines connecting the pixels 201 in the m-th row of the other pixel block 200.

[0021] Additionally, for example, when one pixel block 200 and another pixel block 200 are arranged in a row direction, the plurality of pixels 201 in one pixel block 200 are connected to the plurality of pixels 201 in the other pixel block 200 by different signal lines 202. The plurality of pixels 201 in the nth column of one pixel block 200 are connected by signal lines 202 that are different from the common signal lines 202 connecting the plurality of pixels 201 in the nth column of the other pixel block 200. Furthermore, when one pixel block 200 and another pixel block 200 are arranged in a column direction, the plurality of pixels 201 in one pixel block 200 and the plurality of pixels 201 in the other pixel block 200 are connected by different signal lines 202. The plurality of pixels 201 in the nth column of one pixel block 200 are connected by signal lines 202 that are different from the common signal lines 202 connecting the plurality of pixels 201 in the nth column of the other pixel block 200.

[0022] Pixel block 200 and control blocks 400A and 400B (described later) (see reference) Figure 4 and Figure 17 The corresponding configuration is as follows: That is, a pixel block 200 is configured relative to a control block 400A and 400B.

[0023] Alternatively, multiple pixel blocks 200 can be configured relative to a single control block 400A, 400B. Even when multiple pixel blocks 200 are configured relative to a single control block 400A, 400B, each pixel block 200 can be set to a different exposure time. When two pixel blocks 200 arranged in a column direction are configured relative to a single control block, control blocks 400A, 400B control 2m × n pixels 201. Specifically, for example, control blocks 400A, 400B control 32 × 16 pixels 201. The number of pixels 201 corresponding to control blocks 400A, 400B is not limited to this.

[0024] Figure 3 This is a circuit diagram illustrating an example of the circuit structure of pixel 201. Pixel 201 includes a photoelectric conversion unit 300 and a readout unit 310. The readout unit 310 includes a transmission unit 301, an emission unit 302, an FD (floating diffusion) unit 303, a reset unit 304, and a pixel output unit 305, and reads out the pixel signal obtained based on the charge converted by the photoelectric conversion unit 300 to the signal line 202. The pixel output unit 305 includes an amplification unit 351 and a selection unit 352. The transmission unit 301, emission unit 302, FD 303, reset unit 304, amplification unit 351, and selection unit 352 are referred to as the readout unit 310. The readout unit 310 is described as an N-channel FET, but the type of transistor is not limited to this.

[0025] The photoelectric conversion unit 300 has a photoelectric conversion function that converts light into electrical charge. The photoelectric conversion unit 300 stores the charge obtained from the photoelectric conversion. The photoelectric conversion unit 300 is, for example, composed of a photodiode.

[0026] The transmission unit 301 transfers the charge from the photoelectric conversion unit 300 to the FD 303. The transmission unit 301 controls the electrical connection between the photoelectric conversion unit 300 and the FD 303. The transmission unit 301 is, for example, constructed from a transistor. Alternatively, the transmission unit 301 may be a component having at least a gate terminal and forming part of a transistor with a portion of the photoelectric conversion unit 300 as the source terminal and a portion of the FD 303 as the drain terminal. The gate terminal of the transmission unit 301 is connected to the input transmission control signal. The TX transmission control line 311 is connected. The transmission control line 311 will be explained later.

[0027] The discharge section 302 discharges the charge accumulated in the photoelectric conversion section 300 to the power supply wiring supplied with the power supply voltage VDD. The discharge section 302 controls the connection between the photoelectric conversion section 300 and the power supply wiring. The discharge section 302 is, for example, constructed from a transistor. Alternatively, the discharge section 302 may be a component having at least a gate terminal and forming part of a transistor with a portion of the photoelectric conversion section 300 as the source terminal and a portion of the diffusion region connected to the power supply wiring as the drain terminal. The gate terminal of the discharge section 302 is connected to a discharge control signal. The discharge control line 312 of PDRST is connected. It should be noted that although it is described that the discharge section 302 discharges the charge of the photoelectric conversion section 300 to the power supply line supplied with power supply voltage VDD, it can also discharge to the power supply line supplied with a power supply voltage different from the power supply voltage VDD.

[0028] FD303 stores the charge transferred from photoelectric conversion unit 300 by transmission unit 301.

[0029] The reset unit 304 discharges the charge accumulated in FD303 to the power supply wiring supplied with the power supply voltage VDD. The reset unit 304 resets the potential of FD303 to the reference potential, i.e., the power supply voltage VDD. The reset unit 304 controls the electrical connection between FD303 and the power supply wiring. The reset unit 304 is, for example, constructed from a transistor. Alternatively, the reset unit 304 may also be a component having at least a gate terminal and forming part of a transistor with a portion of FD303 as the source terminal and a portion of the diffusion region connected to the power supply wiring as the drain terminal. The gate terminal of the reset unit 304 is connected to a source terminal for inputting a reset control signal. The reset control line 313 for RST is connected. The reset control line 313 will be explained later.

[0030] The pixel output unit 305 outputs a pixel signal obtained based on the potential of FD303 to the signal line 202. The pixel output unit 305 has an amplification unit 351 and a selection unit 352. The amplification unit 351 is composed of a transistor. The gate terminal of the amplification unit 351 is connected to FD303, the drain terminal is connected to the power supply wiring supplied with the power supply voltage VDD, and the source terminal is connected to the drain terminal of the selection unit 352.

[0031] The selection unit 352 controls the electrical connection between the pixel 201 and the signal line 202. When the pixel 201 is electrically connected to the signal line 202 via the selection unit 352, a pixel signal is output from the pixel 201 to the signal line 202. The selection unit 352 is constructed of a transistor. Alternatively, the selection unit 352 may be an element having at least a gate terminal and constituting part of a transistor with a portion of the amplification unit 351 as the source terminal and a portion of the diffusion region connected to the signal line 202 as the drain terminal. The gate terminal of the selection unit 352 is connected to a selection control signal input. The selection control line 314 of the SEL is connected to the multiple pixel blocks 200. The source terminal of the selection unit 352 is connected to the load current source 306.

[0032] The load current source 306 is connected to the signal line 202 and supplies current for reading pixel signals from the pixel 201. This stabilizes the operation of the amplification unit 351. The load current source 306 can be located on either the first semiconductor substrate 110 or the second semiconductor substrate 120.

[0033] Alternatively, FD303 and pixel output unit 305 can be shared with other pixels 201. For example, FD303 and pixel output unit 305 can be shared among multiple pixels 201 arranged along the row or column direction. In addition, pixel 201 can also be composed of multiple photoelectric conversion units 300 and transmission units 301.

[0034] Figure 4 This is an explanatory diagram showing an example of the specific structure of the control circuit section 102. The control circuit section 102 has a plurality of control blocks 400A. The plurality of control blocks 400A are arranged in the control circuit section 102 along the row direction and the column direction. Specifically, the control circuit section 102 has M×N control blocks 400A. When a pixel block 200 is arranged relative to one control block 400A, the control circuit section 102 has a control block 400A directly below the pixel block 200. The pixel block 200 and the control block 400A have approximately the same shape and the same size. In addition, when a plurality of pixel blocks 200 arranged along the column direction are arranged relative to one control block 400A, the control circuit section 102 has a control block 400A directly below the plurality of pixel blocks 200 arranged along the column direction.

[0035] Control block 400A is configured correspondingly to pixel block 200. As an example of the correspondence between control block and pixel block, control block 400A is located directly below pixel block 200 in the stacking direction (lamination direction) of the first semiconductor substrate 110 and the second semiconductor substrate 120. Furthermore, control block 400A is electrically connected to pixel block 200 via signal line 202, transmission control line 311, and discharge control line 312. Specifically, control block 400A, located directly below pixel block 200 in the stacking direction, is electrically connected to pixel block 200 (hereinafter referred to as corresponding pixel block 200) located directly above it in the stacking direction via local control lines such as transmission control line 311 and discharge control line 312. Additionally, control block 400A receives pixel signals output from pixel 201 of corresponding pixel block 200 via signal line 202.

[0036] Control block 400A controls the driving of the corresponding pixel block 200. For example, control block 400A controls the exposure time of the pixels 201 included in the corresponding pixel block 200. Furthermore, control block 400A has a signal processing unit 402 that processes the input signals and processes the pixel signals output from the pixels 201 included in the corresponding pixel block 200. For example, control block 400A converts the analog pixel signals output from the pixels 201 included in the corresponding pixel block 200 into digital signals.

[0037] The control block 400A includes a pixel control unit 401 and a signal processing unit 402. The pixel control unit 401 includes an autonomous exposure processing unit 411, an exposure control unit 412, and a pixel driving unit 413, and controls the pixels 201 of the pixel unit 101. The signal processing unit 402 includes a signal input unit 421, a signal conversion unit 422, and a signal output unit 423, and converts the analog pixel signals from the pixel unit 101 into digital signals and transmits them to the pixel control unit 401 and the data processing unit 103.

[0038] The autonomous exposure processing unit 411 is a circuit that calculates the exposure time of pixels 201 contained in the corresponding pixel block 200 based on pixel signals converted into digital signals by the signal processing unit 402. Details regarding the autonomous exposure processing unit 411 will be explained later.

[0039] The exposure control unit 412 is a circuit that controls the exposure of pixels 201 included in a corresponding pixel block 200 based on the exposure time calculated by the autonomous exposure processing unit 411. Specifically, the exposure control unit 412 generates a control signal for controlling the exposure time (charge accumulation time of the photoelectric conversion unit 300) of the pixels 201 included in the corresponding pixel block 200. For example, the exposure control unit 412 adjusts the start timing or end timing of the exposure of the pixels 201 included in the corresponding pixel block 200 to control the exposure time of each pixel block 200. The exposure control unit 412 is provided extending along the row direction in the control block 400A.

[0040] The pixel driving unit 413 outputs the control signal generated by the exposure control unit 412 to the pixels 201 included in the corresponding pixel block 200. The pixel driving unit 413 is a driving circuit that drives the pixels 201 included in the corresponding pixel block 200. The pixel driving unit 413 drives the pixels 201 of the pixel row selected from the pixels 201 included in the corresponding pixel block 200. The pixel driving unit 413 extends along the column direction. Thus, the pixel driving unit 413 is positioned at a position corresponding to the m pixels 201 arranged along the column direction. Since the pixel driving unit 413 extends along the column direction and the autonomous exposure processing unit 411 and the exposure control unit 412 extend along the row direction, the autonomous exposure processing unit 411, the exposure control unit 412, and the pixel driving unit 413 are arranged in an L-shape in the control block 400A.

[0041] The signal input unit 421 inputs pixel signals output from the pixels 201 included in the corresponding pixel block 200. The signal input unit 421 outputs the input pixel signals to the signal conversion unit 422. The signal input unit 421 may be configured for each of the n pixels 201 arranged along the row direction in the corresponding pixel block 200. The signal input unit 421 may also include signal processing circuitry for performing noise reduction or other signal processing on the pixel signals output from the first semiconductor substrate 110. Furthermore, the signal input unit 421 may include a voltage adjustment circuit that adjusts the voltage of the signal line 202 connected to the pixels 201 included in the corresponding pixel block 200 to prevent it from falling below a predetermined value. The load current source 306 may also be configured in the signal input unit 421 included in the corresponding control block 400A.

[0042] The signal conversion unit 422 converts the pixel signals output from the signal input unit 421 into digital signals. The signal conversion unit 422 sequentially converts the pixel signals output from each of the m pixels 201 arranged along the column direction in the corresponding pixel block 200 into digital signals. The signal conversion unit 422 also converts the pixel signals output from the pixels 201 arranged in n columns along the row direction in the corresponding pixel block 200 into digital signals in parallel.

[0043] The signal output unit 423 stores the pixel signal converted into a digital signal by the signal conversion unit 422. The signal output unit 423 may also have a latching circuit for storing the digital signal. The signal output unit 423 is disposed in the column direction between the signal conversion unit 422 and the autonomous exposure processing unit 411. The signal output unit 423 outputs the pixel signal converted into a digital signal to the outside of the control circuit unit 102. The signal output unit 423 extends along the row direction in the control block 400A. The signal output unit 423 is disposed in the column direction between the signal conversion unit 422 and the autonomous exposure processing unit 411.

[0044] Figure 5 This is an explanatory diagram showing an example of the internal structure of the control block 400A. The signal conversion unit 422 includes n comparators 501 and n storage units 502. The exposure control unit 412 includes a pixel block control unit 503 and a level shifting unit 504. A comparator 501 and a storage unit 502 connected to the comparator 501 are combined to form an analog-to-digital converter (ADC) 500.

[0045] Comparator 501 is provided in control block 400A extending along the column direction. n comparators 501 are arranged along the row direction. The comparator 501 is configured for each of the m pixels 201 arranged along the column direction in the corresponding pixel block 200. The comparator 501 reads out the pixel signals of the m pixels 201 arranged along the column direction in the corresponding pixel block 200 in sequence and converts them into digital signals.

[0046] The storage unit 502 stores the pixel signals converted into digital signals by the comparator 501. The storage unit 502 is located on the negative side of the Y-axis direction compared to the comparator 501 in the signal conversion unit 422. For example, the storage unit 502 includes a latching circuit. The storage unit 502 may also include a memory composed of SRAM or the like.

[0047] The pixel block control unit 503 controls the operation of the transmission unit 301 and the discharge unit 302 of the pixels 201 included in the corresponding pixel block 200. Specifically, the pixel block control unit 503 outputs a transmission control signal for controlling the transmission unit 301 of the pixels 201 included in the corresponding pixel block 200. TX, and discharge control signal for controlling the discharge section 302 of the pixel 201 included in the corresponding pixel block 200. PDRST. The pixel block control unit 503 extends along the row direction in the control block 400A. The pixel block control unit 503 is disposed in the column direction between the level shifting unit 504 and the autonomous exposure processing unit 411.

[0048] The level shifting unit 504 adjusts the voltage level of the control signal output from the pixel block control unit 503. Specifically, the level shifting unit 504 adjusts the voltage level of the transmission control signal output from the pixel block control unit 503. The voltage level of TX is boosted. Additionally, the level shifting unit 504 will take the output control signal from the pixel block control unit 503. The voltage level of PDRST is boosted.

[0049] The transmission unit 301 receives the transmission control signal boosted by the level shifter 504 via the transmission control line 311. TX. The discharge unit 302 receives the discharge control signal boosted by the pixel block control unit 503 via the discharge control line 312. PDRST.

[0050] In this way, the level shifting unit 504 transmits the control signal in a manner that corresponds to the voltage level used in the transmission unit 301 and the discharge unit 302 of the readout unit 310 of the pixel 201. TX and discharge control signals PDRST boost. The level shifting unit 504 extends along the row direction in the control block 400A.

[0051] The level shifting unit 504 is located on the outer periphery of the control block 400A, compared to the pixel block control unit 503. The positive end of the level shifting unit 504 in the X-axis direction and the negative end in the Y-axis direction are located on the outermost side of the control block 400A. The negative end of the level shifting unit 504 in the X-axis direction is connected to the pixel driving unit 413.

[0052] The level shifting unit 504 and the pixel driving unit 413 process the level-shifted signal. On the other hand, the autonomous exposure processing unit 411, the pixel block control unit 503, the level shifting unit 504, and the pixel driving unit 413 process the pixel signal output from the first semiconductor substrate 110.

[0053] Here, the various structures of the control block 400A are formed in well regions provided on the second semiconductor substrate 120. The well regions are separated according to the voltage level of the signal to be processed. The well regions are also separated depending on whether the power supply used is digital or analog. Furthermore, even when using the same analog power supply, the signal conversion unit 422 may sometimes be separated from other areas using analog power supplies for noise reasons. The separation of the well regions requires well separation areas spaced according to manufacturing process regulations.

[0054] The control block 400A separates the well regions used to form the level shifting section 504 and the pixel driving section 413 from other well regions. For example, the level shifting section 504 and the pixel driving section 413 can share the well regions of the level shifting section 504 and the pixel driving section 413 by being configured in an L-shape. By sharing the well regions, the well separation area can be omitted, thus improving layout efficiency.

[0055] Furthermore, by mirroring the circuit layout in each adjacent block, even adjacent blocks can share well regions, eliminating the need for well separation areas. This improves layout efficiency. Details will follow... Figure 13 This will be discussed later.

[0056] The L-shaped pixel control section 401 forms part of the outer periphery of the control block 400A. Therefore, it can share the well region with other adjacent control blocks 400A in both the row and column directions.

[0057] Figure 6 This is an explanatory diagram showing an example of signal transmission between the first semiconductor substrate 110 and the second semiconductor substrate 120 in the imaging element 100A. The global drive unit 600 is provided at both ends of the peripheral circuit unit 121, which is arranged sandwiching the control circuit unit 102.

[0058] Transmission control line 311a and discharge control line 312a are respectively connected to the pixels 201 included in pixel block 200a. Transmission control line 311a is connected to the gate terminal of the transmission section 301 of the pixel 201 included in pixel block 200a, and discharge control line 312a is connected to the gate terminal of the discharge section 302 of the pixel 201 included in pixel block 200a. Transmission control line 311a receives the transmission control signal output from control block 400Aa. TX is supplied to the transmission unit 301 of the pixels 201 included in pixel block 200a. The discharge control line 312a receives the discharge control signal output from control block 400Aa. PDRST is supplied to the discharge section 302 of the pixel 201 included in the pixel block 200a.

[0059] Similarly, transmission control line 311b and discharge control line 312b are respectively connected to the pixels 201 included in pixel block 200b. Transmission control line 311b is connected to the gate terminal of the transmission section 301 of the pixel 201 included in pixel block 200b, and discharge control line 312b is connected to the gate terminal of the discharge section 302 of the pixel 201 included in pixel block 200b. Transmission control line 311b receives the transmission control signal output from control block 400Ab. TX is supplied to the transmission unit 301 of the pixels 201 included in pixel block 200b. The discharge control line 312b receives the discharge control signal output from control block 400Ab. PDRST is supplied to the discharge section 302 of the pixel 201 included in the pixel block 200b.

[0060] Without distinguishing between transmission control lines 311a and 311b, it is referred to as transmission control line 311. Without distinguishing between discharge control lines 312a and 312b, it is referred to as discharge control line 312.

[0061] The transmission control line 311 and the discharge control line 312 are examples of local control lines connected to the first pixel of the pixel block 200. It should be noted that the transmission control line 311 and the discharge control line 312 are connected together with respect to n pixels 201 arranged in the row direction in the pixel block 200.

[0062] The global driver unit 600 outputs a reset control signal. RST, Select Control Signal SEL and transmission selection control signal TXSEL. The global drive unit 600 is connected to the reset control line 313, the selection control line 314, and the transmission selection control line 603, which output control signals to each pixel block 200.

[0063] The global drive unit 600 supplies reset control signals to multiple pixel blocks 200 via reset control line 313 and selection control line 314. RST and selection control signals SEL. The global driver unit 600 supplies transmission selection control signals to multiple control blocks 400A via the transmission selection control line 603. TXSEL.

[0064] Transmission selection control signal The TXSEL is supplied from the global drive unit 600 to the control block 400A to control the exposure time of each pixel block 200. A transmission selection control signal is also supplied. TXSEL's control block 400A will transmit the selection control signal. The TXSEL output is sent to the corresponding pixel block 200. Control block 400A determines whether to transmit the selection control signal. TXSEL as a transmission control signal TX or discharge control signal PDRST input is sent to pixel 201. This allows the transmission of control signals to be skipped. TX or discharge control signal PDRST is the input to pixel 201.

[0065] For example, in transmitting control signals When TX determines the end time of the exposure, control block 400A transmits control signals. TX jumps to extend the exposure time. Additionally, in transmitting control signals... When TX determines the start time of exposure, control block 400A can transmit control signals. TX skips to shorten the exposure time. This is achieved by transmitting a selection control signal. TXSEL can adjust the exposure time of pixel block 200. (This is followed by a seemingly unrelated sentence about output control signals.) The same applies when PDRST determines the start or end time of the exposure.

[0066] The reset control line 313, selection control line 314, and transmission selection control line 603 are shared by multiple pixel blocks 200. The reset control line 313, selection control line 314, and transmission selection control line 603 are routed transversely across the first semiconductor substrate 110 in the row direction. Alternatively, the reset control line 313, selection control line 314, and transmission selection control line 603 may be routed transversely across the first semiconductor substrate 110 in the column direction.

[0067] For example, the reset control line 313 is connected to the gate terminal of the reset section 304 of the pixel 201 within the pixel block 200, and supplies a reset control signal. RST. The selection control line 314 is connected to the gate terminal of the selection section 352 of the pixel 201 within the pixel block 200, and supplies a selection control signal. SEL. The transmission selection control line 603 is connected to multiple control blocks 400A respectively, and supplies transmission selection control signals to the pixel control unit 401. TXSEL.

[0068] It should be noted that the global drive unit 600 transmits the selection control signal from the second semiconductor substrate 120 via the first semiconductor substrate 110. The TXSEL output is sent to the control block 400A, but the selection control signal can also be output to the control block 400A without going through the first semiconductor substrate 110. TXSEL. In this case, the transmission selection control line 603 is provided on the second semiconductor substrate 120.

[0069] A bonding portion 610 is provided on the bonding surface where the first semiconductor substrate 110 and the second semiconductor substrate 120 are bonded together. The bonding portion 610 aligns the transmission control line 311, the discharge control line 312, and the transmission selection control line 603 between the first semiconductor substrate 110 and the second semiconductor substrate 120. Each bonding portion 610 is composed of a pair of conductive bonding pads, which are bonded and electrically connected by pressure treatment of the first semiconductor substrate 110 and the second semiconductor substrate 120, etc.

[0070] The imaging element 100A controls the exposure time for each pixel block 200 by using local control lines such as the transmission control line 311 and the discharge control line 312 to change the timing of at least one of the transmission section 301 and the discharge section 302. By combining local control lines such as the transmission control line 311 and the discharge control line 312 with global control lines such as the reset control line 313, the selection control line 314, and the transmission selection control line 603, the imaging element 100A can achieve exposure time control with fewer control lines.

[0071] Figure 7 This is an explanatory diagram showing an example of a cross-section in the XZ direction of the imaging element 100A according to this embodiment. Figure 7The image shows a back-illuminated imaging element 100A, but the imaging element 100A is not limited to the back-illuminated type. The imaging element 100A includes a microlens layer 700, a color filter layer 702, a first semiconductor substrate 110, a second semiconductor substrate 120, and a third semiconductor substrate 130. It should be noted that, as shown in the figure, light from the subject is incident in the direction indicated by the blank arrow (the negative Z-axis direction in the figure). In the first semiconductor substrate 110, the side on which light is incident (the positive Z-axis side in the figure) is sometimes referred to as the surface, and the opposite side (the negative Z-axis side in the figure) is referred to as the back surface.

[0072] The microlens layer 700 has a plurality of microlenses 701. The plurality of microlenses 701 are stacked on the positive side of the Z-axis compared to the color filter layer 702. Light is incident on the microlenses 701. The microlenses 701 focus the incident light onto the photoelectric conversion unit 300. Microlenses 701 can be provided for each photoelectric conversion unit 300. The optical axis L of the microlenses 701 is the stacking direction (parallel to the Z-axis) of the first semiconductor substrate 110, the second semiconductor substrate 120, and the third semiconductor substrate 130.

[0073] Alternatively, the position of the microlens 701 can be shifted towards the center of the pixel portion 101 in the peripheral area of ​​pixel 201, a process known as microlens shifting. Generally, a camera's imaging lens has a pupil position where light is perpendicularly incident at the center of the optical axis, but obliquely incident at the periphery of the optical axis. Therefore, by performing microlens shifting, peripheral light reduction can be suppressed.

[0074] The color filter layer 702 has multiple color filters 703 and a passivation film 704. The color filter layer 702 is stacked on the positive Z-axis side compared to the first semiconductor layer 711. The color filters 703 are optical filters that allow light of a specific wavelength range to pass through. The color filters 703 are optical filters with specific spectral dispersion characteristics. The multiple color filters 703 have different spectral dispersion characteristics and allow light of different wavelength ranges to pass through. The multiple color filters 703 are arranged in a specific configuration (e.g., a Bayer array).

[0075] One example of the first semiconductor substrate 110 is a back-illuminated CMOS image sensor. The first semiconductor substrate 110 has a first semiconductor layer 711 and a first wiring layer 712. The first semiconductor layer 711 is located on the positive side of the Z-axis compared to the first wiring layer 712. The first semiconductor layer 711 has a plurality of pixel blocks 200 arranged in a two-dimensional configuration in both row and column directions. The first semiconductor layer 711 also has a plurality of pixels 201 arranged in a two-dimensional configuration in both row and column directions. Each of the plurality of pixels 201 has a plurality of photoelectric conversion units 300 that accumulate charge based on incident light, and a plurality of readout units 310.

[0076] The first wiring layer 712 is disposed on the side of the second semiconductor substrate 120 (negative Z-axis side in the figure) compared to the first semiconductor layer 711. The first wiring layer 712 has a plurality of wirings 713, a plurality of bonding pads 714, and an insulating film (insulating layer) made of a conductor film (metal film).

[0077] The first wiring layer 712 has a plurality of wirings 713 electrically connected to power supplies or circuits. Specifically, in the first semiconductor substrate 110, the wirings 713 include, for example, power supply wirings supplied with a predetermined power supply voltage, signal lines 202 transmitting pixel signals from the first semiconductor substrate 110 (pixels) to the second semiconductor substrate 120, transmission control lines 311 transmitting control signals from the second semiconductor substrate 120 to the first semiconductor substrate 110 (pixels), discharge control lines 312, reset control lines 313, selection control lines 314, and transmission selection control lines 603. The first wiring layer 712 may also be multilayered, and may also include passive and active components.

[0078] Bonding pads 714 are provided on the surface (the negative Z-axis side) of the first wiring layer 712 and are connected to the wiring 713. Additionally, as described later, bonding pads 714 also assist in the connection between layers. Bonding pads 714 are formed, for example, of a conductive material such as copper. It should be noted that bonding pads 714 can also be formed of gold, silver, or aluminum. Insulating layers (insulating films) are formed between the plurality of wirings 713 and between the plurality of bonding pads 714.

[0079] The second semiconductor substrate 120 has a second semiconductor layer 721, a second wiring layer 722, and a wiring layer 723. The second wiring layer 722 is disposed on the side of the first semiconductor substrate 110 (positive Z-axis side in the figure) compared to the second semiconductor layer 721. The wiring layer 723 is disposed on the side of the third semiconductor substrate 130 (negative Z-axis side in the figure) compared to the second semiconductor layer 721, and is located between the second semiconductor layer 721 and the third semiconductor substrate 130. The second semiconductor layer 721 has a control circuit section 102 and a peripheral circuit section 121. The control circuit section 102 has a plurality of control blocks 400A arranged in a two-dimensional configuration in the row and column directions.

[0080] The second semiconductor substrate 120, like the first semiconductor substrate 110, has a plurality of wirings 713 disposed on the second wiring layer 722, a plurality of bonding pads 714 disposed on the second wiring layer 722 and wiring layer 723, and an insulating film (insulating layer) disposed on the second wiring layer 722 and wiring layer 723.

[0081] The second wiring layer 722 has multiple wirings 713 for electrical connection to power supply or circuits, for transmitting signals from pixel unit 101 to control circuit unit 102, and for transmitting signals from control circuit unit 102 to pixel unit 101, as well as bonding pads 714. Specifically, in the second semiconductor substrate 120, the wirings 713 include, for example, power supply wiring supplied with a predetermined power supply voltage, signal lines 202 for transmitting pixel signals from first semiconductor substrate 110 (pixel) to second semiconductor substrate 120, transmission control lines 311 for transmitting control signals from second semiconductor substrate 120 to first semiconductor substrate 110 (pixel), discharge control lines 312, reset control lines 313, selection control lines 314, and transmission selection control lines 603. The second wiring layer 722 may also be multilayered, and may also include passive and active components. The wirings 713 and bonding pads 714 may also be provided in the wiring layer 723.

[0082] The second semiconductor substrate 120 also has a TSV (through-silicon electrode) 724 that interconnects circuits respectively disposed on the back side of the surface. The TSV 724 is preferably disposed in the peripheral area. The TSV 724 transmits image data, etc. generated by the data processing unit 103 to the first semiconductor substrate 110. The TSV 724 may also be disposed on the first semiconductor substrate 110 and the third semiconductor substrate 130.

[0083] The third semiconductor substrate 130 has a third semiconductor layer 731 and a third wiring layer 732, on which a data processing unit 103 is provided. The third wiring layer 732 is disposed between the third semiconductor layer 731 and the second semiconductor substrate 120.

[0084] The third semiconductor substrate 130, like the first semiconductor substrate 110, has wiring 713 and a plurality of bonding pads 714 disposed on the third wiring layer 732. The third wiring layer 732 has a plurality of wiring 713 and bonding pads 714 for electrical connection to power supply or circuit, for transmitting signals from control circuit section 102 to data processing section 103, and for transmitting signals from data processing section 103 to control circuit section 102 of second semiconductor substrate 120.

[0085] It should be noted that the first semiconductor substrate 110, the second semiconductor substrate 120 and the third semiconductor substrate 130 are stacked together by electrical connection between each other through bonding pads 714 provided on each layer and bonding between each other through wiring layers (insulating layers).

[0086] When the first semiconductor substrate 110 and the second semiconductor substrate 120 are stacked, a boundary surface 720 is formed by the negative Z-axis surface of the first wiring layer 712 and the positive Z-axis surface of the second wiring layer 722. Similarly, when the second semiconductor substrate 120 and the third semiconductor substrate 130 are stacked, a boundary surface 730 is formed by the negative Z-axis surface of the wiring layer 723 and the positive Z-axis surface of the third wiring layer 732. A plurality of bonding pads 714 are disposed on the boundary surfaces 720 and 730. Specifically, by aligning the corresponding bonding pads 714 with each other and stacking the two layers, the aligned joints are electrically connected.

[0087] The first semiconductor substrate 110, the second semiconductor substrate 120, and the third semiconductor substrate 130 can be formed by stacking them in a wafer state before chip fabrication and then cutting the stacked wafers (monolithization), or by stacking them after cutting each wafer of the first semiconductor substrate 110, the second semiconductor substrate 120, and the third semiconductor substrate 130.

[0088] Figure 8 This is a timing diagram representing the camera action example 1 of the camera element 100A. Figure 8 It utilizes transmission control signals TX, discharge control signal PDRST, Reset Control Signal RST and selection control signals Example of camera operation where SEL controls the driving of camera element 100A. In Figure 8 In the middle, discharge control signal PDRST is controlled locally, transmitting control signals. TX, reset control signal RST and selection control signals SEL is globally controlled. It should be noted that the signals at the end of the left side... <1> , <2> ... <m>This indicates the row number of pixel 201 within the pixel block.

[0089] Discharge control signal PDRST controls the timing of the start of exposure. The start timing of exposure and the discharge control signal are related. The falling edge of PDRST is timed (e.g., at time T1). That is, the control signal is emitted before the start time T1 of the exposure. PDRST turns on the discharge section 302 to discharge the charge accumulated in the photoelectric conversion section 300, and discharges the charge in the discharge control signal. Exposure begins on the falling edge of PDRST. Due to the discharge control signal... PDRST is controlled locally, so the exposure time can be adjusted for each pixel block by 200.

[0090] Transmission control signals TX controls the timing of ending the exposure. At time T3, a control signal is transmitted. TX transfers the charge stored in the photoelectric conversion unit 300 to the FD 303 by turning on the transmission unit 301. Exposure end timing and transmission control signal. The timing of the falling edge of TX (e.g., time T4) corresponds to this. This is due to the transmission control signal. TX is the signal after global control, so the timing for ending the exposure is the same in each pixel block 200.

[0091] Reset control signal RST controls the timing of discharging the charge accumulated in FD303. At time T2, the reset control signal... RST discharges the charge from FD303 by turning on the reset unit 304. By discharging the charge from FD303 before the end timing of the exposure, the influence of residual charge in FD303 can be suppressed when transferring charge from the photoelectric conversion unit 300.

[0092] Select control signal SEL is the signal used to select any pixel 201. Selection control signal. The SEL control selector 352 is switched on and off. At time T2, the selection control signal is... SEL is set to high. At time T3, the selection control signal is... Pixel 201, with SEL set to high level, is controlled by the transmission control signal. When TX is turned on, a pixel signal is output to signal line 202. On the other hand, the selection control signal... In pixel 201 where SEL is not set to high, no pixel signal is output.

[0093] The imaging element 100A controls the discharge signal. PDRST allows for localized control, enabling the adjustment of the exposure start timing for each pixel block 200, thereby controlling the exposure time for each pixel block 200. Additionally, the image sensor 100A can also control the transmission of control signals. TX performs local control to control the end timing of exposure for each pixel block 200. Furthermore, the image sensor 100A can also transmit control signals. TX and discharge control signals PDRST uses both local control to control the start and end timings of exposure for each pixel block 200.

[0094] Figure 9 This is a timing diagram representing the camera action of the camera element 100A, Example 2. Figure 9 It utilizes transmission control signals TX, reset control signal RST and selection control signals Example of using SEL to control the driving of the imaging element 100A for imaging operation. The imaging element 100A utilizes the transmission control signal. TX controls the start timing of exposure and is similar to Figure 8 The situations are different. Regarding... Figure 8 The differences will be explained in detail.

[0095] Transmission control signals TX controls the timing of the start and end of exposure. In frame (n), exposure begins at time T5 and ends at time T7.

[0096] At the start of exposure, T5, due to the transmission of control signals Exposure begins when TX decreases. That is, control signals are transmitted before the start of exposure, T5. TX via reset control signal When RST is on, the transmission unit 301 is turned on to discharge the charge accumulated in the photoelectric conversion unit 300, and the transmission control signal is transmitted. Exposure begins on the falling edge of TX. Due to the transmission control signal... TX is a signal after local control, so it can change the timing of the start of exposure in each pixel block 200. However, it can also make the timing of the start of exposure consistent in each pixel block 200.

[0097] Additionally, at the end of the exposure, T7, due to the transmission of control signals... The exposure ends when TX decreases. That is, the control signal is transmitted before the end of the exposure, T7. TX via reset control signal With RST disconnected, the transmission unit 301 is turned on to transfer the charge stored in the photoelectric conversion unit 300 to the FD303, and a transmission control signal is transmitted. Exposure ends on the falling edge of TX. This is due to the transmission control signal. TX is a signal after local control, so it can change the timing of the end of exposure in each pixel block 200. However, it can also make the timing of the end of exposure consistent in each pixel block 200.

[0098] Select control signal SEL is the signal used to select any pixel 201. At time T6, the selection control signal... Pixel 201, with SEL set to high, outputs a pixel signal to signal line 202.

[0099] Reset control signal RST controls the timing of discharging the charge accumulated in FD303. Reset control signal. RST can be a signal used after global control. (This is due to the reset control signal.) RST is always on except during the readout timing, so no charge will accumulate in FD303. On the other hand, by setting the reset control signal during the readout timing... RST is disconnected and the transmission control signal is activated. TX is activated, thereby transferring charge from the photoelectric conversion unit 300 to the FD303. (Reset control signal) In RST, because the switching timing is the same during reading, it can be synchronized with the selection control signal. SEL pulse commonization.

[0100] The camera element 100A transmits control signals. TX performs local control, allowing the start or end timing of exposure to be changed for each pixel block 200, thereby controlling the exposure time for each pixel block 200. Additionally, the image sensor 100A resets the control signal. RST and selection control signal The pulse commonality of SEL can further simplify the control circuit.

[0101] Figure 10 This is a timing diagram showing the imaging action of the imaging element in the comparative example. Figure 10 It utilizes transmission control signals TX, reset control signal RST and selection control signals The SEL is used to control the driving of the camera element in the camera action example, but the exposure time is not controlled for each pixel block 200.

[0102] In the comparative example, transmission control signals are used. TX and reset control signals RST is used to control the start of exposure. The exposure start timing is achieved by transmitting the control signal. TX and reset control signals The timing of the falling edge of RST (time t1). The timing of the end of exposure is the transmission control signal. Timing of the falling edge of TX (time t2). In the comparative example, the start and end timings of the exposure are globally controlled, and the exposure time is not controlled for each pixel block 200.

[0103] Figure 11 This is an explanatory diagram showing an example of a subject captured by the image sensor 100A. Figure 11 In the process, the camera element 100A controls the exposure time for each pixel block 200 under the condition of sunset outside the tunnel.

[0104] Zones 1 through 5 are five zones divided according to brightness. Zones 1 through 5 are numbered in order of brightness. Zone 1 is the brightest zone where the sunset can be directly seen. Zone 2 corresponds to the tunnel exit and is darker than Zone 1. Zone 3 is the zone inside the tunnel where the sunset is reflected and is darker than Zone 2. Zone 4 is the zone inside the tunnel where the sunset shines into the tunnel from the exit and is darker than Zone 3. Zone 5 is the darkest zone inside the tunnel where the sunset does not shine into the tunnel from the exit.

[0105] The imaging element 100A controls the exposure time for each pixel block 200 based on the brightness of each area. The imaging element 100A controls the exposure time by setting a shorter exposure time for pixel blocks 200 in brighter areas. The exposure time for area 1 is set to the shortest, and the exposure time for area 5 is set to the longest. For example, the exposure times for areas 1 through 5 are 1 / 15360s, 1 / 1920s, 1 / 960s, 1 / 240s, and 1 / 120s, respectively.

[0106] Figure 12 It means Figure 11 The exposure time time graphs for regions 1-5 are shown. Figure 12 In the middle, the 100A camera element is designed for Figure 11 Each pixel block 200 in regions 1 to 5 controls the exposure time. The interval from time T11 to time T19 corresponds to the frame rate of the dynamic image.

[0107] In region 1, control block 400A controls the drive to make the exposure time in pixel block 200 a predetermined exposure time ET1. Control block 400A utilizes an output control signal. PDRST controls the start of exposure and utilizes transmission control signals. TX controls the end of exposure. In region 1, exposure ends at times T12 to T19 respectively.

[0108] In region 2, control block 400A controls the drive to make the exposure time in pixel block 200 a longer exposure time ET2 than ET1. Control block 400A makes the exposure start time in region 2 earlier than in region 1, and makes the exposure end time consistent with that in region 1. Therefore, in region 2, the exposure ends at times T12 to T19 respectively. The exposure time ET2 in region 2 is shorter than the period of the sensor rate.

[0109] In region 3, control block 400A controls the drive to make the exposure time in pixel block 200 a longer exposure time ET3 than ET2. Control block 400A makes the exposure start time in region 3 earlier than in region 2, and makes the exposure end time consistent with that in region 2. Therefore, in region 3, the exposure ends at times T12 to T19 respectively. The exposure time ET3 in region 3 is set to be the same as the period of the sensor rate.

[0110] In region 4, control block 400A controls the drive to make the exposure time in pixel block 200 a longer exposure time ET4 than ET3. Control block 400A sets region 4 to the same exposure start time as region 3, and on the other hand, utilizes a transmission selection control signal. TXSEL skips the end of the exposure. Control block 400A utilizes the transmission selection control signal. The TXSEL skipped three times, achieving four times the exposure time for region 3. In region 4, transmission selection control signals were supplied at various times from T12 to T14. TXSEL.

[0111] In region 5, control block 400A controls the drive to make the exposure time in pixel block 200 a longer exposure time ET5 than ET4. Control block 400A sets region 5 to have the same exposure start time as region 4, and on the other hand, it adds a transmission selection control signal. The number of times TXSEL skips the end of the exposure. Control block 400A selects the control signal by utilizing the transmission selection signal. TXSEL skips seven times, achieving twice the exposure time for region 4. The exposure time ET5 for region 5 is set to match the period of the moving image frame rate. In region 5, transmission selection control signals are supplied at various times from T12 to T18. TXSEL.

[0112] The camera element 100A transmits control signals. TX and discharge control signal Short exposure times are achieved by closely spaced PDRST intervals. Additionally, the image sensor 100A utilizes transmission selection control signals... TXSEL skips transmission control signals Long exposures are achieved through TX control. This, in turn, expands the dynamic range.

[0113] Figure 13 This is a plan view showing an example layout of multiple control blocks 400A. In the multiple control blocks 400A, adjacent control blocks 400A are arranged in a flipped configuration. Figure 13 Example 12 of the multiple control blocks 400A provided in the control circuit section 102.

[0114] The flip configuration means that the forming areas of each structure of the control block 400A (e.g., exposure control unit 412, pixel driving unit 413, signal input unit 421, signal conversion unit 422, and signal output unit 423) are mirror-flipped about the boundary lines between the control blocks 400A (arranged symmetrically). The circuitry of each structure of the control block 400A may not need to be flipped. Furthermore, the readout order of the pixels in the control block 400A is not limited to flipped readout.

[0115] For example, when multiple control blocks 400A arranged adjacent to each other in the row direction are flipped, since the structures of each control block 400A are flipped in the row direction, each pixel driving unit 413 is arranged adjacent to each other at the boundary between two control blocks 400A. As a result, multiple pixel driving units 413 arranged adjacent to each other in the row direction can be laid out as a single pixel driving unit 413, thereby improving the layout efficiency of the control blocks 400A.

[0116] Similarly, when multiple control blocks 400A arranged adjacent to each other in the column direction are flipped together, since the structures of each control block 400A are flipped together in the column direction, the same structures are arranged adjacent to each other at the boundary of two control blocks 400A. As a result, multiple signal input sections 421 arranged adjacent to each other in the column direction can be laid out as a single signal input section 421, thereby improving the layout efficiency of the control blocks 400A.

[0117] Control blocks 400A are flipped together with adjacent control blocks 400A. All control blocks 400A are flipped together in both the row and column directions, but they can also be flipped together in only one of the row and column directions. For example, the signal conversion unit 422 of control block 400A is flipped together with the signal conversion units 422 of control blocks 400A adjacent to it in the row direction. Furthermore, the signal conversion unit 422 of control block 400A is also flipped together with the signal conversion units 422 of control blocks 400A adjacent to it in the column direction.

[0118] Control blocks 400Aa and 400Ab are arranged adjacent to each other in the row direction. Control blocks 400Aa and 400Ab are flipped together. The level shifting section 504 of control block 400Aa is located in the same well region as the level shifting section 504 of control block 400Ab. Similarly, the pixel block control section 503, the storage section 502, and the signal output section 423 are located in the same well region in control blocks 400Aa and 400Ab.

[0119] Control blocks 400Ab and 400Ac are arranged adjacent to each other in the row direction. Control blocks 400Ab and 400Ac are flipped together. The pixel driving section 413 of control block 400Ab is located in the same well region as the pixel driving section 413 of control block 400Ac. The well region of the pixel driving section 413 may also be shared with the well region of the level shifting section 504.

[0120] Control blocks 400Aa and 400Ad are arranged adjacent to each other in the column direction. Control blocks 400Aa and 400Ad are flipped together. The pixel driving section 413 of control block 400Aa is located in the same well region as the pixel driving section 413 of control block 400Ad. In addition, the signal conversion section 422 of control block 400Aa is located in the same well region as the signal conversion section 422 of control block 400Ad.

[0121] Control blocks 400Ad and 400Ae are arranged adjacent to each other in the column direction. Control blocks 400Ad and 400Ae are configured in an inverted manner. The pixel driving section 413 and the level shifting section 504 of control block 400Ad are located in the same well region as the pixel driving section 413 and the level shifting section 504 of control block 400Ae.

[0122] By flipping the configuration of the control blocks 400A, the imaging element 100A improves layout efficiency even when signal processing is performed side-by-side for each control block 400A. By flipping the configuration of multiple control blocks 400A in the XY plane, the imaging element 100A can share a well region among adjacent control blocks 400A. This reduces the number of well region switching operations and improves area efficiency.

[0123] Figure 14 This is a circuit diagram illustrating another example of the circuit structure of pixel 201. In pixel 201, for... Figure 3 The same structural elements are labeled with the same reference numerals and descriptions are omitted. In pixel 201, there is no discharge section 302. When discharging the charge accumulated in the photoelectric conversion section 300 to the power supply wiring supplied with power voltage VDD, a transmission control signal is input to the gate terminal of the transmission section 301. TX, and inputs a reset control signal to the gate terminal of the reset unit 304. RST.

[0124] Figure 15 This is a timing diagram representing the imaging action of the imaging element 100A, as shown in Example 3. Figure 15 In the middle, it is used Figure 14 The pixel 201 shown utilizes the transmission control signal TX, reset control signal RST and selection control signals Example of using SEL to control the driving of the imaging element 100A for imaging operation. The imaging element 100A utilizes the transmission control signal. TX controls the start timing of exposure and is similar to Figure 12 The situations are different. Regarding... Figure 12 The differences will be specifically explained.

[0125] Transmission control signals TX controls the timing of the start and end of exposure. In frame (n), exposure begins at time T5 and ends at time T7.

[0126] At the start of exposure, T5, due to the transmission of control signals Exposure begins when TX decreases. That is, control signals are transmitted before the start of exposure, T5. TX via reset control signal When RST is on, the transmission unit 301 is turned on to discharge the charge accumulated in the photoelectric conversion unit 300, and the transmission control signal is transmitted. Exposure begins on the falling edge of TX. Due to the transmission control signal... TX is a signal after local control, so it can also change the timing of the start of exposure in each pixel block 200.

[0127] Additionally, at the end of the exposure, T7, due to the transmission of control signals... The exposure ends when TX decreases. That is, the control signal is transmitted before the end of the exposure, T7. TX via reset control signal With RST disconnected, the transmission unit 301 is turned on to transfer the charge stored in the photoelectric conversion unit 300 to the FD303, and a transmission control signal is transmitted. Exposure ends on the falling edge of TX. This is due to the transmission control signal. TX is a signal after local control, so it can also change the timing of the end of exposure in each pixel block 200.

[0128] Select control signal SEL is the signal used to select any pixel 201. At time T6, the selection control signal... Pixel 201, with SEL set to high, outputs a pixel signal to signal line 202.

[0129] Reset control signal RST controls the timing of discharging the charge accumulated in FD303. Reset control signal. RST can also be a signal used after global control. (This is due to the reset control signal.) RST is always on except during the readout timing, so no charge will accumulate in FD303. On the other hand, by setting the reset control signal during the readout timing... RST is disconnected and the transmission control signal is activated. TX is activated, thereby transferring charge from the photoelectric conversion unit 300 to the FD303. (Reset control signal) In RST, the switching timing during reading is the same, therefore it can be synchronized with the selection control signal. SEL pulse commonization.

[0130] Thus, according to Figures 1-15 The structure of the image sensor 100A shown allows for exposure in units of pixel blocks 200, each consisting of multiple pixels 201, and for reading out pixel signals from pixel blocks 200 in units of control blocks 400A corresponding to each pixel block 200, converting analog signals into digital signals. Furthermore, the image sensor 100A reads out pixel signals side-by-side for each pixel block 200 via control blocks 400A provided for each pixel block 200. Therefore, because the image sensor 100A can set the exposure time for each pixel block 200 according to the intensity of the incident light, it can expand the dynamic range.

[0131] Next, use Figures 16-22 The structure of the camera element 100B is described. Although the camera element 100B exposes in units of 200 pixel blocks, it reads out the pixel signals sequentially by pixel row and performs AD conversion by pixel column.

[0132] Figure 16 This is an exploded perspective view showing other examples of an imaging element. The imaging element 100B includes a first semiconductor substrate 110, a second semiconductor substrate 120, and a third semiconductor substrate 130. For example... Figure 16 As shown, the first semiconductor substrate 110 is stacked on the second semiconductor substrate 120, and the second semiconductor substrate 120 is stacked on the third semiconductor substrate 130.

[0133] The first semiconductor substrate 110 has a pixel portion 101 and a connection region 1601. The pixel portion 101 outputs a pixel signal obtained based on incident light. The connection region 1601 is disposed around the periphery of the pixel portion 101. Figure 16 In the example, a pair of connection regions 1601 are arranged near and deep to the pixel portion 101 along two opposite sides of the first semiconductor substrate 110.

[0134] The second semiconductor substrate 120 includes a control circuit section 102, a peripheral circuit section 121, and a signal processing section 1602. Furthermore, the signal processing section 1602 can be configured to separately house the circuitry on the first semiconductor substrate 110 and the second semiconductor substrate 120. In this case, the connection region 1601 is also included in the signal processing section 1602, becoming part of the signal processing section 1602's area. This allows for efficient circuit configuration and reduces chip area.

[0135] The control circuit unit 102 outputs control signals for controlling the driving of the pixel unit 101 to the pixel unit 101. The control circuit unit 102 is disposed in the second semiconductor substrate 120 at a position opposite to the pixel unit 101.

[0136] The peripheral circuit section 121 controls the driving of the control circuit section 102. The peripheral circuit section 121 is disposed on the periphery of the control circuit section 102 in the second semiconductor substrate 120. Alternatively, the peripheral circuit section 121 may be electrically connected to the first semiconductor substrate 110 and control the driving of the pixel section 101. The peripheral circuit section 121 is disposed along two opposite edges of the second semiconductor substrate 120, but the arrangement method of the peripheral circuit section 121 is not limited to this example.

[0137] An analog pixel signal output from the first semiconductor substrate 110 is input to the signal processing unit 1602. The signal processing unit 1602 performs signal processing on the pixel signal. For example, the signal processing unit 1602 performs processing to convert the analog pixel signal into a digital signal. The signal processing unit 1602 may also perform other signal processing. Examples of other signal processing include noise reduction processing such as analog or digital CDS (correlated double sampling). The signal processing unit 1602 is disposed on the periphery, i.e., the outer side, of the control circuit unit 102. Figure 16 In this example, a pair of signal processing units 1602 are arranged near and deep within the control circuit unit 102 along two opposite edges of the second semiconductor substrate 120. The signal processing unit 1602 may also be a circuit included in the peripheral circuit unit 121.

[0138] The third semiconductor substrate 130 has a data processing unit 103. The data processing unit 103 uses digital data output from the second semiconductor substrate 120 to perform addition processing, interval elimination processing, and other image processing.

[0139] Figure 17 These are explanatory diagrams showing other examples of the specific structure of the control circuit section 102. Figure 17 In the control block 400B, there is a pixel control unit 401 (autonomous exposure processing unit 411, exposure control unit 412, pixel driving unit 413), but no signal processing unit 402.

[0140] Alternatively, instead of setting one control block 400B for one pixel block 200, one control block 400B can be set for N (N is a natural number greater than 2) pixel blocks 200. Sometimes, the N pixel blocks 200 corresponding to one pixel block are called a pixel block group. For example, two pixel blocks 200 arranged along a column direction can also be set as a pixel block group and a single control block 400B can be set. In this case, the control block 400B can also control the exposure time for each pixel block 200.

[0141] Incidentally, the control block 400B is electrically connected to at least one pixel block 200 and can be considered the smallest unit of circuitry that controls the exposure of pixels 201 of the at least one pixel block 200.

[0142] Figure 18 This is an explanatory diagram showing the connection relationship between the first semiconductor substrate 110 and the second semiconductor substrate 120 in the imaging element 100B. The first semiconductor substrate 110 has connection regions 1801 and 1601 provided around the pixel portion 101 and electrically connected to the pixel portion 101. The second semiconductor substrate 120 has connection regions 1802 and 1803 provided around the control circuit portion 102 and electrically connected to the control circuit portion 102. Furthermore, the global driving unit 600 can separately configure the circuitry on the first semiconductor substrate 110 and the second semiconductor substrate 120. This allows for efficient circuit configuration and reduces chip area.

[0143] A pair of connection regions 1801 are connected to a pair of connection regions 1802 located at opposite positions. The interconnected connection regions 1801 and 1802 use global control lines to input control signals from the global drive unit 600 to the pixel unit 101.

[0144] A pair of connection regions 1601 are respectively connected to a pair of connection regions 1803 located at opposite positions. The interconnected connection regions 1601 and 1803 use a common signal line to input the pixel signal from the pixel unit 101 to the corresponding ADC unit 1820 and ADC unit 1830.

[0145] Figure 19 This is an explanatory diagram showing an example of signal transmission between the first semiconductor substrate 110 and the second semiconductor substrate 120 in the imaging element 100B. The global drive unit 600 outputs a reset control signal. RST, Select Control Signal SEL and transmission selection control signal TXSEL. The global drive unit 600 is connected to the reset control line 1903 and the selection control line 1904, which output signals to each pixel block 200. The global drive unit 600 supplies reset control signals to multiple pixel blocks 200 via the reset control line 1903. RST, and provides selection control signal via selection control line 1904. SEL. The global driver unit 600 supplies transmission selection control signals to multiple control blocks 400B via the transmission selection control line 1905. TXSEL.

[0146] Transmission selection control signal The TXSEL is supplied from the global drive unit 600 to the control block 400B to control the exposure time of each pixel block 200. A transmission selection control signal is also supplied. TXSEL's control block 400B will transmit the selection control signal. The TXSEL output is sent to the corresponding pixel block 200. Pixel block 200 determines whether to transmit the selection control signal. TXSEL as a transmission control signal TX or discharge control signal PDRST input is sent to pixel 201. This allows the transmission control signal to be skipped. TX or discharge control signal PDRST is the input to pixel 201.

[0147] For example, in transmitting control signals When TX determines the end time of the exposure, control block 400B skips the transmission control signal. TX is used to extend the exposure time. Additionally, control signals are transmitted. If TX determines the start time of exposure, control block 400B can skip transmitting control signals. TX is used to shorten the exposure time. This is achieved by transmitting a selection control signal. TXSEL can adjust the exposure time of pixel block 200. (This is followed by a seemingly unrelated sentence about output control signals.) The same applies when PDRST determines the start or end time of the exposure.

[0148] The reset control line 1903, selection control line 1904, and transmission selection control line 1905 are globally routed, meaning they are shared across multiple pixel blocks 200. The reset control line 1903, selection control line 1904, and transmission selection control line 1905 are routed across the pixel section 101 in the row direction. Alternatively, the reset control line 1903, selection control line 1904, and transmission selection control line 1905 can be routed across the pixel section 101 in the column direction.

[0149] For example, the reset control line 1903 is connected to the gate terminal of the reset section 304 of the pixel block 200 and supplies a reset control signal. RST. The selection control line 1904 is connected to the gate terminal of the selection section 352 of the pixel block 200 and supplies a selection control signal. SEL. Additionally, the transmission selection control line 1905 is connected to multiple control blocks 400B respectively, and supplies transmission selection control signals to the pixel control unit 401. TXSEL.

[0150] It should be noted that the global drive unit 600 outputs a transmission selection control signal from the second semiconductor substrate 120 to the first semiconductor substrate 110. TXSEL, but it is also possible to output a transmission selection control signal to the control block 400B instead of supplying it to the first semiconductor substrate 110. TXSEL. In this case, the transmission selection control line 1905 is provided on the second semiconductor substrate 120.

[0151] On the other hand, the transmission control line 1901a and the discharge control line 1902a are connected to the pixel block 200a. The transmission control line 1901a is connected to the gate terminal of the transmission section 301 located in the pixel block 200a. The transmission control line 1901a receives the transmission control signal output from the control block 400Ba. TX is supplied to pixel block 200a. The discharge control line 1902a is connected to the gate terminal of the discharge section 302 located in pixel block 200a. The discharge control line 1902a receives the discharge control signal output from control block 400Ba. PDRST is supplied to pixel block 200a.

[0152] Transmission control line 1901b and discharge control line 1902b are connected to pixel block 200b. Transmission control line 1901b is connected to the gate terminal of transmission section 301 located in pixel block 200b. Transmission control line 1901b receives transmission control signals output from control block 400Bb. TX is supplied to pixel block 200b. The discharge control line 1902b is connected to the gate terminal of the discharge section 302 located in pixel block 200b. The discharge control line 1902b receives the discharge control signal output from control block 400Bb. PDRST is supplied to pixel block 200b.

[0153] Multiple bonding portions 610 are provided on the bonding surfaces where the first semiconductor substrate 110 and the second semiconductor substrate 120 are bonded together. The bonding portions 610 of the first semiconductor substrate 110 are aligned with the bonding portions 610 of the second semiconductor substrate 120. The opposing multiple bonding portions 610 are bonded and electrically connected by pressing or other processes on the first semiconductor substrate 110 and the second semiconductor substrate 120. In this case, the bonding portion 610 of the global control line can be located either below the corresponding pixel block 200 or in the connection region 1801 or connection region 1802. On the other hand, the bonding portion 610 of the local control line is provided below the corresponding pixel block 200 (which is also on the control block 400B).

[0154] The imaging element 100B controls the exposure time for each pixel block 200 by changing the timing of at least one of the transmission section 301 and the discharge section 302 using local control lines. By combining local control lines with global control lines, the imaging element 100B can achieve exposure time control with fewer control lines.

[0155] Figure 20 This is an explanatory diagram showing the connection relationship between the ADC unit and the pixel block. For example... Figure 20 As shown, a common signal line 202 extending along the column direction is arranged in columns inside pixel block 200c. Moreover, this signal line 202 is also common to multiple pixel blocks 200c and 200d arranged along the column direction. Therefore, in this example, m×M pixels 201 arranged in a column are connected to a single signal line 202, and pixel signals from these pixels 201 are output.

[0156] On the signal lines 202, ADCs 2000 are connected to the second semiconductor substrate 120 side via junctions 610. The multiple ADCs 2000 corresponding to the multiple signal lines 202 constitute the ADC section 1820.

[0157] exist Figure 20 In the example, the ADC2000 corresponding to the odd-numbered pixel blocks 200c and 200d is located in the ADC section 1820, and the ADC2000 corresponding to the even-numbered pixel blocks 200e and 200f is located in the ADC section 1830. However, the configuration relationship between pixel blocks 200c, etc., and their corresponding ADC sections 1820, etc., is not limited to this.

[0158] According to the above structure, each ADC 2000 converts the pixel signals sequentially output from a column of m×M pixels 201 into digital signals and outputs them. In this case, the ADC units 1820 and 1830 as a whole convert the pixel signals from the pixels 201 arranged in n×N columns along the row direction into digital signals in parallel. From this point of view, this digital conversion can also be considered a type of column ADC. It should be noted that while a single-slope ADC can be cited as an example of an ADC, other digital conversion methods can also be used. Furthermore, the connection positions of each pixel 201 and the signal line 202 are not limited to... Figure 20 The shape shown, as another example, can also be located within each pixel block 200c, etc.

[0159] Figure 21 This is a timeline representing the imaging action within pixel block 200 of the imaging element 100B. It utilizes transmission control signals. TX, discharge control signal PDRST, Reset Control Signal RST and selection control signals SEL is used to control the driving of pixel block 200.

[0160] Discharge control signal PDRST controls the timing of the start of exposure. The start timing of exposure and the discharge control signal are related. The falling edge of PDRST is timed (e.g., at time T1). That is, the control signal is emitted before the start time T1 of the exposure. PDRST activates the discharge section 302 to discharge the charge accumulated in the photoelectric conversion section 300, and discharges the charge via a discharge control signal. Exposure begins on the falling edge of PDRST. Due to the discharge control signal... PDRST is controlled locally, so the exposure time can be adjusted for each pixel block by 200.

[0161] Transmission control signals TX controls the timing of ending the exposure. At time T3, a control signal is transmitted. TX transfers the charge stored in the photoelectric conversion unit 300 to the FD303 by turning on the transmission unit 301. Exposure end timing and transmission control signal. The timing of the falling edge of TX (e.g., time T4) corresponds to this.

[0162] Reset control signal RST controls the timing of discharging the charge accumulated in FD303. At time T2, the reset control signal... RST discharges the charge from FD303 by turning on the reset unit 304. By discharging the charge from FD303 before the end timing of the exposure, the influence of residual charge in FD303 can be suppressed when transferring charge from the photoelectric conversion unit 300.

[0163] Select control signal SEL is the signal used to select any pixel 201. Selection control signal. The SEL control selector 352 is switched on and off. At time T2, the selection control signal is... SEL is set to high. At time T3, the selection control signal is... Pixel 201, with SEL set to high level, is controlled by the transmission control signal. When TX is turned on, a pixel signal is output to signal line 202. On the other hand, the selection control signal... In pixel 201 where SEL is not set to high, no pixel signal is output.

[0164] The camera element 100B controls the discharge signal. PDRST allows for localized control, enabling the adjustment of the exposure start timing for each pixel block 200, thereby controlling the exposure time for each pixel block 200. Additionally, the image sensor 100B can also control the transmission of control signals. TX performs local control to control the end timing of exposure for each pixel block 200. Furthermore, the image sensor 100B can also transmit control signals. TX and discharge control signals PDRST uses these two methods for local control to control the start and end timings of exposure for each pixel block 200.

[0165] Each pixel signal of pixel 201 corresponds to the amount of charge accumulated in the photoelectric conversion unit 300. Therefore, controlling the timing of exposure of pixel 201 can also be described as controlling the timing of charge accumulation in the photoelectric conversion unit 300. More specifically, controlling the timing of exposure of pixel 201 can be described as controlling the timing and length of charge accumulation time from charge discharge to transmission.

[0166] Figure 22 This is an explanatory diagram illustrating an example of the exposure timing for each pixel block 200. For three pixel blocks 200 arranged in a row, the exposure time is controlled for each pixel block 200. Here, the imaging element 100B changes the exposure by staggering the times at which the pixels are reset for each pixel block 200.

[0167] On the other hand, the timing of reading out the pixel signals is arranged sequentially starting from the upper pixel block 200. That is, the pixel signal is read out from the pixel 201 of "pixel block 1", then from the pixel 201 of "pixel block 2", and then from the pixel 201 of "pixel block 3".

[0168] Additionally, within pixel block 200, as in Figure 21 As explained, pixel signals are read sequentially starting from the pixels 201 in the upper row. Therefore, when viewed as a whole, pixel signals are read sequentially starting from the upper row of the m×M pixels 201 in the same column connected to the common signal line 202. In other words, the global drive unit 600 reads the selection control signal row by row from the 1st row to the m×Mth row, traversing multiple pixel blocks 200 arranged in a column. SEL is set to high level.

[0169] In this case, such as in Figure 20 As explained, for multiple pixel blocks 200 arranged in a row, there is a common selection control line 1904 connecting them to n×N pixel blocks arranged in the same row. Therefore, from the selection control signal... With SEL set to high, n×N pixels 201 in a row-connected array are read out in parallel. This allows for the output of one frame's worth of pixel signals.

[0170] These pixel signals are as follows Figure 20 As described, digital conversion is performed by ADC units 1820 and 252. The digitally converted pixel signals are output to subsequent image processing to form a frame of image.

[0171] As described above, from the perspective of sequentially reading out pixel signals from the top row of the same column among multiple pixel blocks 200, the readout method of this embodiment, as a whole, can also be described as a so-called rolling shutter method. However, it should be noted that even in this case, a different exposure time can be set for each pixel block 200.

[0172] so, Figures 16-22 Although the image sensor 100B shown exposes in units of pixel blocks 200, it reads out pixel signals sequentially row by row and performs AD conversion column by column. Specifically, the image sensor 100B reads out pixel signals from pixels 201 of the upper pixel block 200 in a row of multiple pixel blocks 200 arranged in a column, and then reads out pixel signals from pixels 201 of the lower pixel block 200. Therefore, image distortion caused by the readout order in the case of shooting a moving subject becomes smoother, reducing the viewer's sense of disharmony with the image. More specifically, when a moving subject is read out from multiple pixel blocks 200 arranged in a column side by side, multiple jagged stair-step differences corresponding to the pixel blocks 200 appear in the vertical direction of the image (i.e., corresponding to the column direction of the pixels), which can cause a sense of disharmony for the viewer. In contrast, according to Figures 16-22 The multiple step differences shown in the image sensor 100B will not appear in the image.

[0173] in addition, Figures 16-22 The image sensor 100B shown does not have an ADC (Analog-to-Digital Converter) unit within the control block 400B; instead, a signal processing unit 1602 is positioned outside the control circuit unit 102. Therefore, the area of ​​the control block 400B can be reduced, and the size of the pixel blocks 200 positioned corresponding to the control block 400B can be reduced, meaning exposure control of the control block 400B can be performed on a smaller number of pixels. This allows for precise exposure time control within the image, making the boundaries of the pixel blocks 200 less noticeable on the image. Furthermore, since digital conversion is not performed directly below the pixel 201, the impact of noise caused by heat on the pixel 201 can be suppressed.

[0174] It should be noted that the signal processing unit 1602 may not be provided in multiple separate areas, or it may be provided in one area relative to the entire pixel unit 101.

[0175] As described above, from the viewpoint that the result is the sequential readout of pixel signals from the top row of the same column among multiple pixel blocks 200, just like with the image sensor 100A, the readout method of the image sensor 100B as a whole can also be described as a so-called rolling shutter method. However, even in this case, different exposure times can be set for each pixel block 200, just like with the image sensor 100A. Therefore, in the image sensor 100B, just like with the image sensor 100A, the image distortion caused by the readout order when shooting a moving subject becomes smoother, and the sense of incongruity in the image perceived by the viewer can be reduced.

[0176] [Autonomous Exposure Processing Department 411] Next, the details of the aforementioned autonomous exposure processing unit 411 will be explained. It should be noted that in the following explanation, the image sensor 100A and 100B will be referred to as image sensor 100 without distinguishing between them, and the control block 400A and 400B will be referred to as control block 400 without distinguishing between them.

[0177] like Figure 4 and Figure 17 As shown, the autonomous exposure processing unit 411 is installed within the control block 400. Alternatively, the autonomous exposure processing unit 411 can also be installed within the peripheral circuit section 121 instead of the control block 400; or it can be installed in both the control block 400 and the peripheral circuit section 121. Hereinafter, using... Figures 23-25 These three modes will be explained.

[0178] Figure 23 This is a block diagram illustrating a structural example of autonomous exposure control method 1. Autonomous exposure control method 1 is a structural example in which the autonomous exposure processing unit 411 is installed within the control block 400. By adding the autonomous exposure processing unit 411 into the control block 400, although the circuit size of the control block 400 increases, correspondingly, the individual pixels 201 of the pixel block 200 sometimes also increase in size, thus expanding the light-receiving area.

[0179] exist Figure 23 The following example illustrates the concept of control block 400A. Figure 25 Similarly). Control block 400A includes a signal conversion unit 422, a signal output unit 423, an autonomous exposure processing unit 411, an exposure control unit 412, and a pixel driving unit 413. For ease of explanation, the signal input unit 421 is omitted. It should be noted that if it is control block 400B, the signal input unit 421, signal conversion unit 422, and signal output unit 423 are not included in control block 400B, but are disposed as signal processing unit 1602 on the second semiconductor substrate 120. Figure 25 (The same applies).

[0180] The signal conversion unit 422 has n ADCs 500. The n ADCs 500 convert analog pixel signals from m pixels 201 in the connected column direction into digital signals. Each ADC 500 is composed of a comparator 501 and a storage unit 502.

[0181] The column selection circuit 2301 is included in the signal output unit 423. Whenever a column selection signal is read from an external input, the column selection circuit 2301 sequentially selects the columns of the pixel block 200. Whenever a horizontal transmission clock is input from an external input, the column selection circuit 2301 outputs the digital pixel signals from the m pixels 201 of the selected column to the peripheral circuit unit 121 via the replacement unit 2340 and the horizontal transmission line 2300, and also outputs them to the autonomous exposure processing unit 411.

[0182] The autonomous exposure processing unit 411 calculates an exposure value representing the exposure time of the pixel block 200. Specifically, for example, the autonomous exposure processing unit 411 includes a preprocessing unit 2311, a controller 2312, and an exposure value calculation unit 2313.

[0183] The preprocessing unit 2311 acquires digital pixel signals for each pixel column of the pixel block 200 from the column selection circuit 2301. Then, the preprocessing unit 2311 calculates statistical values ​​(e.g., average, median, maximum, or minimum) of the acquired pixel signals. The preprocessing unit 2311 outputs the calculation result to the exposure value calculation unit 2313.

[0184] The controller 2312 inputs a reset signal to the preprocessing unit 2311, causing the preprocessing based on the preprocessing unit 2311 to reset. As a result, the preprocessing unit 2311 calculates statistical values ​​of the pixel signals from the pixel block 200 each time it is reset, that is, for each frame.

[0185] The exposure value calculation unit 2313 determines the next exposure value based on the calculation result (statistical value of pixel signal) from the preprocessing unit 2311. Specifically, for example, the exposure value calculation unit 2313 determines the next exposure value in a way that will not result in underexposure or overexposure based on the calculation result. For example, the exposure value calculation unit 2313 maintains a first threshold and a second threshold. The first threshold is a threshold used to determine whether the calculation result results in underexposure. The second threshold is a threshold larger than the first threshold and is a threshold used to determine whether the calculation result results in overexposure.

[0186] The exposure value calculation unit 2313 maintains the exposure value of the previous frame. The exposure value calculation unit 2313 determines whether the calculation result is above a first threshold and below a second threshold. If the calculation result is above the first threshold and below the second threshold, the exposure value calculation unit outputs an exposure value with the same exposure time as the previous frame to the latching circuit 2321 of the exposure control unit 412. Alternatively, if the calculation result is less than the first threshold, the exposure value calculation unit 2313 outputs an exposure value with one exposure time longer than the previous frame to the latching circuit 2321 of the exposure control unit 412. Alternatively, if the calculation result exceeds the second threshold, the exposure value calculation unit outputs an exposure value with one exposure time shorter than the previous frame to the latching circuit 2321 of the exposure control unit 412.

[0187] In addition, the exposure value calculation unit 2313 can also maintain multiple threshold levels. In this case, if the calculation result is above the first threshold and below the second threshold, the exposure value calculation unit 2313 will output the exposure value with the same exposure time as the previous frame to the latch circuit 2321 of the exposure control unit 412.

[0188] Furthermore, if the calculated result is less than the first threshold, the exposure value calculation unit 2313 adjusts the degree of exposure time change to +1, +2, or +3 stops compared to the previous frame, depending on the degree of underexposure, and outputs the exposure value to the latching circuit 2321 of the exposure control unit 412. Conversely, if the calculated result exceeds the second threshold, the exposure value calculation unit 2313 adjusts the degree of exposure time change to -1, -2, or -3 stops compared to the previous frame, depending on the degree of overexposure, and outputs the exposure value to the latching circuit 2321 of the exposure control unit 412.

[0189] The exposure control unit 412 includes, for example, a latch circuit 2321, a shift register 2322, a pixel block control unit 503, and a level shifting unit 504. The latch circuit 2321 holds the exposure value from the autonomous exposure processing unit. Whenever a latch pulse is input from an external source, the latch circuit 2321 outputs the held exposure value to the pixel block control unit 503 and the shift register 2322.

[0190] The shift register 2322 converts the exposure value from the latch circuit 2321 from parallel to serial, and outputs the exposure value as a serial signal to the data processing unit 103 and the replacement unit 2340.

[0191] If the exposure time is calculated in an external system outside the image sensor 100 and the calculation result is fed back to the image sensor 100, it will take time to reflect the exposure time to the image sensor 100, and the power consumption will increase. In contrast, by providing an autonomous exposure processing unit 411 in the control block 400, it is possible to improve the response speed of reflecting the exposure time to the pixel block 200 and reduce the power consumption.

[0192] Furthermore, the setting unit 2330 is located outside the control circuit unit 102, for example, within the peripheral circuit unit 121 or the data processing unit 103. The setting unit 2330 sets the embedding position for each control block 400A, in which the exposure value is embedded into the m-row, n-column digital pixel signal output from the column selection circuit 2301 to the peripheral circuit unit 121. The embedding position of the exposure value set by the setting unit 2330 will be discussed later. Figures 26-33 This will be discussed later.

[0193] Additionally, the control block 400A has a replacement unit 2340. The replacement unit 2340 replaces a portion of the first data related to the pixel signal read from the pixel 201 with second data related to the accumulation time calculated by the exposure value calculation unit 2313, which calculates the accumulation time for accumulating the charge converted by the photoelectric conversion unit 300.

[0194] Specifically, for example, the replacement unit 2340 receives an m-row, n-column digital pixel signal from the column selection circuit 2301, an exposure value from the shift register 2322, and an embedding position of the exposure value from the setting unit 2330. Based on the embedding position of the exposure value from the setting unit 2330, the replacement unit 2340 selects an embedding target pixel signal within the m-row, n-column digital pixel signal and replaces the value of the selected embedding target pixel signal with the bit value contained in the exposure value. Then, the replacement unit 2340 outputs the digital pixel signal with the embedded exposure value to the peripheral circuit unit 121 via the horizontal transmission line 2300.

[0195] It should be noted that, in Figure 23 The description illustrates the case where exposure control is performed on one pixel block 200 using a single control block 400. However, when multiple pixel blocks 200 are exposed using a single control block 400, the autonomous exposure processing unit 411 can also sequentially select one pixel block 200 from the multiple pixel blocks 200 to calculate the exposure value, synchronized with the reset signal. A selector is provided on the output side of the exposure value calculation unit 2313, and the controller 2312 outputs a selection signal for selecting one pixel block 200 from the multiple pixel blocks 200 to the selector.

[0196] Furthermore, in this case, the exposure control unit 412 has a latching circuit 2321 and a shift register 2322 for each pixel block 200. The latching circuit 2321 is connected to a selector (not shown) in the autonomous exposure processing unit 411. When an exposure value is input from the selector, a latching pulse is input, and each time this occurs, the held exposure value is output to the pixel block control unit 503 and the shift register 2322. Thus, autonomous exposure can be achieved even when multiple pixel blocks 200 are exposed using a single control block 400.

[0197] Figure 24 This is a block diagram illustrating a structural example of autonomous exposure control method 2. Autonomous exposure control method 2 is a structural example in which the autonomous exposure processing unit 411 is installed within the peripheral circuit section 121. The autonomous exposure processing unit 411 is installed within the peripheral circuit section 121 rather than within the control block. Therefore, compared to... Figure 23 Compared to the previous situation, this allows for a reduction in the circuit size of the control block 400.

[0198] The peripheral circuit section 121 is connected to the pixel section 101 via the horizontal transmission section 2410. The horizontal transmission section 2410 is connected to the pixel blocks 200 arranged in the row direction (hereinafter referred to as pixel block rows) and transmits pixel signals to the peripheral circuit section 121 according to the pixel block rows. Since the pixel section 101 is a collection of M rows and N columns of pixel blocks 200, the horizontal transmission section 2410 transmits pixel signals to the peripheral circuit section 121 for each of the M pixel block rows.

[0199] The peripheral circuit section 121 has row-direction autonomous exposure processing unit groups 2400-1 to 2400-M for each pixel block row (hereinafter referred to as row-direction autonomous exposure processing unit group 2400 without distinction). The row-direction autonomous exposure processing unit group 2400 includes a data sampling unit 2411 and autonomous exposure processing units 411 (pre-processing unit 2311, controller 2312, and exposure value calculation unit 2313) with a number corresponding to the number of columns N of the pixel blocks. Figure 24 Since N=4, four pre-processing units 2311, controllers 2312, and exposure value calculation units 2313 are installed.

[0200] The data sampling unit 2411 divides the pixel signal column N from the pixel block row of the horizontal transmission unit 2410 into equal parts and samples them. The data sampling unit 2411 outputs the sampled pixel signal columns to the corresponding preprocessing unit 2311.

[0201] As described above, the preprocessing unit 2311 calculates statistical values ​​of the pixel signals from the corresponding pixel block 200. Furthermore, since the peripheral circuit unit 121 can have a larger circuit size than the control block 400, the preprocessing unit 2311 can perform other processing besides calculating the statistical values ​​of the pixel signals.

[0202] For example, the preprocessing unit 2311 has a memory that stores the pixel numbers of defective pixels in the corresponding pixel block 200 during manufacturing. When the data sampling unit 2411 samples the pixel signal with the pixel number, the preprocessing unit 2311 does not use the pixel signal for the calculation of statistical values. As a result, it is possible to achieve high precision in the calculation of statistical values ​​of pixel signals.

[0203] Alternatively, the preprocessing unit 2311 can also obtain calculation results from other preprocessing units 2311 responsible for pixel blocks 200 adjacent to the corresponding pixel block 200, and calculate statistical values ​​of the pixel signals from the corresponding pixel block 200 based on the calculation results obtained from the other preprocessing units 2311. This allows for smoothing of the exposure step difference between adjacent pixel blocks 200.

[0204] Furthermore, while a first threshold and a second threshold are set in the exposure value calculation unit 2313, at least one of the first threshold and the second threshold can be changed according to the shooting mode in the camera device with the image sensor 100 installed. Thus, optimal exposure calculation can be performed based on the shooting mode.

[0205] In addition, the peripheral circuit section 121 has a latch circuit 2321 and a shift register 2322 for each exposure value calculation section 2313. The shift register 2322 performs parallel-to-serial conversion on the exposure value from the latch circuit 2321, and outputs the exposure value as a serial signal to the data processing section 103, and outputs the exposure value to the exposure control section 412 in the control block 400 corresponding to the pixel block 200.

[0206] Furthermore, the setting unit 2330 is located outside the control circuit unit 102, for example, within the peripheral circuit unit 121 or the data processing unit 103. The setting unit 2330 sets, for each control block 400, the embedding position of the exposure value into the m rows and n columns of digital pixel signals output from the column selection circuit 2301 to the peripheral circuit unit 121. The embedding position of the exposure value set by the setting unit 2330 will be discussed later. Figures 26-33 This will be discussed later.

[0207] The replacement unit 2340 is disposed on the second semiconductor substrate 120 or the third semiconductor substrate 130. Specifically, for each control block 400, the replacement unit 2340 receives, for example, an m-row n-column digital pixel signal from the horizontal transmission unit 2410, an exposure value from the shift register 2322, and an embedding position of the exposure value from the setting unit 2330. Based on the embedding position of the exposure value from the setting unit 2330, the replacement unit 2340 selects an embedding target pixel signal within the m-row n-column digital pixel signal and replaces the value of the selected embedding target pixel signal with the bit value contained in the exposure value. Then, the replacement unit 2340 outputs a digital pixel signal with the embedded exposure value for each control block 400.

[0208] according to Figure 24 The structure shown is similar to Figure 23 Compared to the previous approach, this allows for a reduction in the circuit size of the control block 400, which in turn reduces the size of the corresponding pixel block 200. Therefore, the increased number of pixel blocks enables extremely precise autonomous exposure control. Alternatively, the exposure control unit 412 and the pixel driving unit 413 can be mounted in the peripheral circuit unit 121. This further reduces the circuit size of the control block 400 and the size of the corresponding pixel block 200.

[0209] Figure 25 This is a block diagram illustrating a structural example of autonomous exposure control method 3. Autonomous exposure control method 3 is a structural example in which the autonomous exposure processing unit 411 is installed both within the control block 400A and the peripheral circuit unit 121. When automatic exposure control is performed within the control block 400A, data transmission such as sending pixel signals from the control block 400A to the peripheral circuit unit 121 or sending exposure values ​​from the peripheral circuit unit 121 to the pixel block 200 is no longer required. Therefore, compared to the case where it is performed within the peripheral circuit unit 121, the feedback to the corresponding pixel block 200 is faster.

[0210] On the other hand, since the area of ​​the control block 400A is constrained by the area of ​​the corresponding pixel block 200, installing it in the peripheral circuit section 121 can increase the circuit size of the autonomous exposure processing unit 411 compared to installing it in the control block 400A. Therefore, installing it in the peripheral circuit section 121 allows for the installation of more advanced functions (e.g., in autonomous exposure control) for autonomous exposure control. Figure 24 The document describes the removal of pixel signals from defective pixels, control of exposure step difference with adjacent pixel blocks 200, and calculation of optimal exposure corresponding to the shooting mode.

[0211] Therefore, in autonomous exposure control mode 3, the camera element 100 performs autonomous exposure control according to the situation. When performing high-performance calculations related to autonomous exposure control, autonomous exposure control is performed within the peripheral circuit section 121; when performing high-speed exposure value feedback, autonomous exposure control is performed within the control block 400A. Figure 25 In the example of autonomous exposure control mode 3, autonomous exposure control is performed by the row direction autonomous exposure processing unit group 2400 in the peripheral circuit unit 121. However, when a certain trigger is provided to the control circuit unit 102, the camera element 100 performs autonomous exposure control for each control block 400A.

[0212] For example, the imaging element 100 performs autonomous exposure control based on user operation. If high-performance computation related to autonomous exposure control is selected, autonomous exposure control is performed within the peripheral circuit section 121; if high-speed execution of exposure value feedback is selected, autonomous exposure control is performed within the control block 400A. Alternatively, if the battery level is below a specified amount, the imaging element 100 may select and perform low-power processing from both high-performance computation related to autonomous exposure control and high-speed execution of exposure value feedback.

[0213] Due to the horizontal direction autonomous exposure processing unit group 2400 installed in the peripheral circuit section 121 and Figure 24 The structures shown are the same, so in Figure 25 It is omitted in the text.

[0214] The column selection circuit 2301 outputs an n-bit digital pixel signal to n OR circuits 2501. The autonomous exposure processing unit 2500 in the control block 400A, in addition to the controller 2312, also has n OR circuits 2501, an output data latch circuit 2502, and an n-bit AND circuit 2503.

[0215] When an n-bit signal is output from the output data latch circuit 2502, the controller 2312 inputs a reset signal to the output data latch circuit 2502.

[0216] The OR circuit 2501 is a 2-input, 1-output logic circuit. One input of the OR circuit 2501 is connected to a column select circuit, and the other input is connected to the output of an n-bit AND circuit 2503.

[0217] n OR circuits 2501 are connected to the input of the output data latch circuit 2502. The output data latch circuit 2502 holds an n-bit signal from the n OR circuits 2501. When a horizontal transmission clock is input, the output data latch circuit 2502 outputs an n-bit signal to the n-bit AND circuit 2503. Additionally, when a reset signal is input from the controller 2312, the output data latch circuit 2502 resets the held n-bit signal and outputs an n-bit signal with at least one bit set to 0 to the n-bit AND circuit 2503.

[0218] The n-bit AND circuit 2503 is an n-input, 1-output AND circuit. The output of the output data latch circuit 2502 is connected to the input of the n-bit AND circuit 2503. The output of the n-bit AND circuit 2503 is connected to the selector 2512 of the exposure control unit 412 and the inputs of each OR circuit 2501. If the output from the n-bit AND circuit 2503 is "0", it indicates that the pixel column that outputs the n-bit digital pixel signal is not saturated. If the output from the n-bit AND circuit 2503 is "1", it indicates that the pixel column that outputs the n-bit digital pixel signal is saturated. Hereinafter, the 1-bit signal with an output of "1" from the n-bit AND circuit 2503 will be referred to as the saturation detection signal.

[0219] If the value of the digital pixel signal from pixel 201 in the pixel column is "1", it indicates that pixel 201 is saturated. If all the values ​​of the n-bit signal from the column selection circuit 2301 are "1", it indicates that the entire pixel column is saturated. In this case, since all inputs to one input of each OR circuit 2501 are "1", each OR circuit 2501 outputs a 1-bit signal with a value of "1" to the output data latch circuit 2502.

[0220] The output data latch circuit 2502 holds the n-bit signal whose values ​​are all "1" and outputs the held n-bit signal to the n-bit AND circuit 2503 when the horizontal transmission clock is input.

[0221] When an n-bit signal consisting entirely of "1"s is input, the n-bit AND circuit 2503 outputs a saturation detection signal of "1" to the selector 2512 and each OR circuit 2501. Consequently, the output data latch circuit 2502 outputs an n-bit signal consisting entirely of "1"s to the n-bit AND circuit 2503 before receiving a reset signal. Therefore, the n-bit AND circuit 2503 outputs a saturation detection signal before receiving a reset signal from the controller 2312 to the output data latch circuit 2502.

[0222] Exposure Control Department 412 Figure 24 In addition to the structure shown, there is a shift register 2511 and a selector 2512. The shift register 2511 performs serial-to-parallel conversion on the exposure value from the peripheral circuit section 121 and outputs it to the level shift section 504 and the selector 2512.

[0223] Selector 2512 receives the exposure value and the set exposure value from shift register 2511. Selector 2512 selects one of the exposure value and the set exposure value from shift register 2511 based on the output signal from n-bit AND circuit 2503, and outputs the selected exposure value to latch circuit 2321. The set exposure value is the exposure time corresponding to the degree of desaturation of pixel 201, for example, the exposure value set to the shortest exposure time.

[0224] The exposure value is calculated and set by an external system other than the control block 400A. The exposure value can be a fixed value or selected from the external system. The external system is, for example, the peripheral circuit section 121 in the image sensor 100, the data processing section 103 of the third semiconductor substrate 130, or the image processing section 3411 connected to the image sensor 100 in the imaging device having the image sensor 100.

[0225] Specifically, selector 2512 selects the exposure value from shift register 2511 and outputs it to latch circuit 2321 when the output signal from n-bit AND circuit 2503 is not a saturation detection signal. On the other hand, selector 2512 selects the set exposure value and outputs it to latch circuit 2321 when the output signal from n-bit AND circuit 2503 is a saturation detection signal.

[0226] exist Figure 25 In this circuit, the setting unit 2330 is located within the peripheral circuit section 121. The control block 400A includes a replacement unit 2340. Specifically, the replacement unit 2340 receives, for example, an m-row, n-column digital pixel signal from the column selection circuit 2301, an exposure value from the shift register 2322, and an embedding position of the exposure value from the setting unit 2330. Based on the embedding position of the exposure value from the setting unit 2330 within the peripheral circuit section 121, the replacement unit 2340 selects an embedding target pixel signal from the m-row, n-column digital pixel signal and replaces the value of the selected embedding target pixel signal with the bit value contained in the exposure value. Then, the replacement unit 2340 outputs the digital pixel signal with the embedded exposure value to the peripheral circuit section 121 via the horizontal transmission line 2300.

[0227] Within the control block 400A, the autonomous exposure processing unit 2500 and the exposure control unit 412 use the exposure value from the peripheral circuit unit 121 to perform autonomous exposure control until saturation is detected in the control block 400A. When saturation is detected in the control block 400A, the set exposure value within the exposure control unit 412 is used to perform autonomous exposure control.

[0228] Therefore, it is possible to select the following processing: a process that sets a high-precision exposure value for a pixel column in a non-saturated state based on the exposure value from the peripheral circuit section 121; and a process that changes the exposure value of a pixel column in a saturated state to a non-saturated state, which allows for simple and high-speed feedback.

[0229] Additionally, the autonomous exposure processing unit 2500 within the control block 400 can also be... Figure 23 The autonomous exposure processing unit 411 is shown. In this case, for example, the autonomous exposure processing unit 411 in the peripheral circuit section 121 and the autonomous exposure processing unit 411 in the control block 400 can also be selected by user settings.

[0230] For example, a camera device equipped with an image sensor 100 can select between the autonomous exposure processing unit 411 in the peripheral circuit section 121 and the autonomous exposure processing unit 411 in the control block 400 based on the remaining battery level. In this case, if the remaining battery level is above a predetermined value, the camera device selects autonomous exposure control based on the autonomous exposure processing unit 411 in the peripheral circuit section 121; if the remaining battery level is below the predetermined value, the camera device selects autonomous exposure control based on the autonomous exposure processing unit 411 in the control block 400. Furthermore, if high-quality video recording is desired, the user can select the autonomous exposure processing unit 411 in the peripheral circuit section 121; if reduced power consumption is desired, the user can select the autonomous exposure processing unit 411 in the control block 400.

[0231] <Example of embedding exposure values ​​into image data> Next, an example of embedding exposure values ​​into image data will be explained.

[0232] Figure 26 This is an explanatory diagram (Figure 1) showing the image data output from the imaging element 100. The image data IMG output from the imaging element 100 includes M×N block image data IMG(1,1) to IMG(M,N). The image data IMG is output from the imaging element 100 in the order of block image data IMG(1,1) to IMG(M,N). The block image data IMG(i,j) (where i is an integer satisfying 1≤i≤M and j is an integer satisfying 1≤j≤N) is the image data output from the pixel block 200 (hereinafter referred to as pixel block 200(i,j)) in row i and column j of the imaging element 100, that is, the digital pixel signal from column selection circuit 2301 in row m and column n.

[0233] Pixel block 200(i,j) Figure 2 and Figure 4 As shown, let's define a set of 201 pixels in m rows and n columns, but... Figure 26 In this example, we use m=n=16 for explanation. The output data from a single pixel 201 is designated as pixel data 2600. Pixel data 2600 is multi-bit, for example, 12-bit data. The pixel X-coordinate represents m×n (=256) numbered pixel data 2600s. The set of pixel data 2600s with pixel X-coordinate values ​​from 1 to 16 constitutes the permutation of pixel data 2600 with m=1. When using the pixel X-coordinate value within pixel data 2600, branch numbers are used. For example, the pixel data 2600s with pixel X-coordinate values ​​from 1 to 256 are denoted as pixel data 2600-1 to 2600-256.

[0234] Figure 27 This is an explanatory diagram (Figure 2) showing the image data output from the camera element 100. Figure 27 The image data IMG represents the configuration of pixel data 2600 corresponding to each pixel block 200(i,j) of the imaging element 100. In pixel block 200(i,j), the arrangement of the pixel data 2600 in the first row constitutes... Figure 26 The arrangement of pixel data in columns 2600-1 to 2600-16, and the pixel data in the second row of 2600. Figure 26 The columns containing pixel data 2600-17 to 2600-32. The same process continues below, with the final row, row 16, consisting of the arrangement of pixel data 2600. Figure 26 The pixel data is in columns 2600-241 to 2600-256.

[0235] Figure 28 This is a graph showing the correspondence between exposure time and exposure value. In graph 2800, TV value 2801, exposure time 2802, and exposure value 2803 are interconnected. TV value 2801 is the unique identification information that determines the combination of exposure time 2802 and exposure value 2803. Exposure time 2802 is the time it takes for the imaging element 100 to perform photoelectric conversion on light and store charge. Exposure value 2803 is data representing exposure time 2802 using 4 bits, calculated by exposure value calculation unit 2313.

[0236] Next, use Figures 29-33 An example will be described where the setting unit 2330 sets the embedding position, the replacement unit 2340 selects the pixel signal at the embedding position set by the setting unit 2330, and embeds the exposure value.

[0237] (Embedded Example 1) Figure 29 This is an explanatory diagram illustrating an embedding example 1 where an exposure value of 2803 is embedded into block image data IMG(i,j). The embedding target pixel data group 2900 is a set of pixel data 2600-1 to 2600-4 into which the exposure value 2803 is embedded. The replacement unit 2340 embeds the exposure value 2803 into the least significant bit of each of the pixel data 2600-1 to 2600-4, totaling 4 bits. Pixel data 2600-1 to 2600-4 are output in the block image data IMG(i,j) before the subsequent pixel data 2600-5 to 2600-256. Therefore, compared to embedding the exposure value 2803 into subsequent pixel data 2600-5 to 2600-256, the output destination of the block image data IMG(i,j) (e.g., peripheral circuit unit 121, data processing unit 103, and image processing unit 3411 outside the camera element 100 described later) can read out the exposure value 2803 earlier.

[0238] The least significant bit (1 bit) is used to embed the pixel data 2600-1 to 2600-4 for the exposure value 2803, which is essentially equivalent to 11 bits. However, the lost data is only the least significant bit, hidden in the noise component, and the adverse effect is not easily apparent. Furthermore, within the block image data IMG(i,j), the number of pixel data embedded for the exposure value 2803 is only 4, and the adverse effect is still not easily apparent, reaching a level that is difficult to visually confirm.

[0239] Furthermore, in the output destination of the block image data IMG(i,j), after reading the exposure value 2803 from the least significant bits of pixel data 2600-1, 2600-2, 2600-3, and 2600-4, the value of each least significant bit is set to 0. Thus, pixel data 2600-1, 2600-2, 2600-3, and 2600-4 effectively become 11 bits.

[0240] (Embedded Example 2) Figure 30 This is an explanatory diagram of embedding example 2-1, which shows the embedding of exposure value 2803 into block image data IMG(i,j). The embedding target pixel data group 3000 is a set of pixel data 2600-1, 2600-3, 2600-5, and 2600-7 into which exposure value 2803 is embedded. Through the replacement unit 2340, exposure value 2803 is embedded in a total of 4 bits, consisting of the least significant bits of each of the pixel data 2600-1, 2600-3, 2600-5, and 2600-7.

[0241] exist Figure 29 In the data, the exposure value 2803 is embedded in the least significant bits of the consecutively arranged pixel data 2600-1 to 2600-4, totaling 4 bits. Figure 30 In this configuration, the exposure value 2803 is embedded in discretely configured pixel data 2600-1, 2600-3, 2600-5, and 2600-7. The pixel data 2600 embedded with the exposure value 2803 are not limited to pixel data 2600-1, 2600-3, 2600-5, and 2600-7, as long as they are not configured consecutively. Furthermore, the discrete configuration interval of the pixel data 2600 with the embedded exposure value 2803 is set to one pixel, but it can also be two or more pixel intervals.

[0242] Furthermore, in the output destination of the block image data IMG(i,j), after reading the exposure value 2803 from the least significant bits of pixel data 2600-1, 2600-3, 2600-5, and 2600-7, the values ​​of these least significant bits are set to 0. Thus, pixel data 2600-1, 2600-3, 2600-5, and 2600-7 effectively become 11 bits.

[0243] Figure 31 This is an illustration of embedding example 2-2, which shows the embedding of exposure value 2803 into block image data IMG(i,j). Figure 31 This is an example where the discrete configuration interval of the pixel data 2600 embedded with an exposure value of 2803 is set to 3 pixel data points. Specifically, for example, in Figure 31 In this context, the embedded object pixel data group 3100 is a set of pixel data 2600-1, 2600-5, 2600-9, and 2600-13 into which the exposure value 2803 is embedded. Through the replacement unit 2340, the exposure value 2803 is embedded into a total of 4 bits, representing the least significant bit of each of the pixel data 2600-1, 2600-5, 2600-9, and 2600-13.

[0244] Furthermore, in the output destination of the block image data IMG(i,j), after reading the exposure value 2803 from the least significant bits of pixel data 2600-1, 2600-5, 2600-7, and 2600-13, the values ​​of these least significant bits are set to 0. Thus, pixel data 2600-1, 2600-5, 2600-7, and 2600-13 effectively become 11 bits.

[0245] (Embedded Example 3) Figure 32 This is an illustration of embedding example 3, which shows the embedding of exposure value 2803 into the block image data IMG(i,j). Figure 32 Not like Figures 29-31 Instead of embedding the exposure value 2803 into pixel data groups 2900, 3000, and 3100 that are at the edges of the block image data IMG(i,j), this example embeds it into pixel data group 3200 that is inside the edges of the block image data IMG(i,j). That is, in Figures 29-31 In the diagram, pixel data groups 2900, 3000, and 3100 are an arrangement of one-dimensional pixel data 2600, but... Figure 32 In the pixel data group 3200, the pixel data 2600 with the embedded exposure value 2803 is dispersed in two dimensions. Therefore, the adverse effects of the embedded data are lower than in the case of embedding example 1, and it is also more difficult to visually confirm in the image.

[0246] Specifically, for example, the embedded object pixel data group 3200 is a set of pixel data 2600-68, 2600-76, 2600-192, and 2600-200 into which the exposure value 2803 is embedded. Through the replacement unit 2340, the exposure value 2803 is embedded in a total of 4 bits, representing the least significant bits of each of the pixel data 2600-68, 2600-76, 2600-192, and 2600-200.

[0247] In the output destination of the block image data IMG(i,j), after reading the exposure value 2803 from the least significant bits of pixel data 2600-68, 2600-76, 2600-192, and 2600-200, the values ​​of the least significant bits are set to 0. Therefore, pixel data 2600-68, 2600-76, 2600-192, and 2600-200 effectively become 11 bits.

[0248] in addition, Figures 29-32 (Embedding Examples 1-3) were illustrated with 4 bits of embedded data, but it is not limited to this and more bits can be embedded.

[0249] (Embedded Example 4) Figure 33 This is an illustrative diagram representing embedding example 4, which embeds the exposure value 2803 into the block image data IMG(i,j). Figures 29-32 In this process, the 4-bit exposure value 2803 is distributed across the least significant bit of the 4 pixel data 2600, while... Figure 33 The example shown is where the 4-bit exposure value 2803 is concentrated into the lower 4 bits of a single pixel data 2600. Figure 33 In this example, by means of replacement part 2340, exposure value 2803 is embedded in the lower 4 bits of pixel data 2600-35.

[0250] Regarding pixel data 2600-35, which is actually 8 bits, there are concerns about its impact on image quality. Therefore, after reading the exposure value 2803 from the output destination of pixel data 2600-35, this pixel data 2600-35 can be treated as a defective pixel. Thus, the output destination of pixel data 2600-35 can discard pixel data 2600-35 and supplement it with surrounding pixel data. Furthermore, when treating pixel data 2600-35 as defective pixel data, the embedded exposure value 2803 is not limited to 4 bits; all 12 bits can also be used as the exposure value 2803.

[0251] In order to enable interpolation processing to be performed within the block image data IMG(i,j) at its output destination, the pixel data 2600 embedded with the exposure value 2803 is positioned inside the edge of the block image data IMG(i,j). For example, when the block image data IMG(i,j) is output from pixel block 200(i,j) of the Bayer array, the pixel data 2600 embedded with the exposure value 2803 needs to be pixel data in the following manner: pixel data of the same color that is separated from the pixel data 2600 by two pixels vertically, horizontally, and vertically exists within the block image data IMG(i,j).

[0252] In the case of pixel data 2600-35, since the four pixel data that are separated by two pixel data in the top, bottom, left and right are located in the block image data IMG(i,j), the above conditions are satisfied.

[0253] Furthermore, as shown in the embedding examples 1 to 4 above, the exposure value 2803 is embedded at the same embedding position in each block of image data IMG(i,j) output in the time direction. Alternatively, the setting unit 2330 can set different embedding positions in each block of image data IMG(i,j) output in the time direction.

[0254] Specifically, for example, the setting unit 2330 includes a pseudo-random number generation circuit. The pseudo-random number generation circuit generates a pseudo-random number each time block image data IMG(i,j) is output, and adds the pseudo-random number to the embedding position (at least one of the X coordinate value and the Y coordinate value). The replacement unit 2340 embeds the exposure value 2803 into the block image data IMG(i,j) at the embedding position after adding the pseudo-random number.

[0255] By employing this method, since the spatial embedding position of the exposure value 2803 within the block image data IMG(i,j) differs from each other in the temporal direction among the block image data IMG(i,j), the influence of the exposure value 2803 at the same embedding position can be dispersed. Furthermore, while a pseudo-random number generation circuit was used in the above description, a counter that increments by a predetermined value each time could also be used.

[0256] Figure 34 This is a block diagram illustrating a structural example of the camera device 3400 according to the embodiment. The camera device 3400 includes an imaging element 100, a system control unit 3401, a drive unit 3402, a light metering unit 3403, a working memory 3404, a recording unit 3405, a display unit 3406, an operation unit 3408, a drive unit 3414, and a photographic lens 3420.

[0257] The photographic lens 3420 guides the subject beam incident along the optical axis OA to the imaging element 100. The photographic lens 3420 consists of multiple optical lens groups, enabling the subject beam from the scene to be imaged near its focal plane. The photographic lens 3420 can also be a replaceable lens that can be attached and detached from the imaging device 3400. It should be noted that... Figure 34 In the image, the photographic lens 3420 is represented by an imaginary lens positioned near the pupil.

[0258] The drive unit 3414 drives the photographic lens 3420. For example, the drive unit 3414 moves the optical lens group of the photographic lens 3420 to change the focus position. In addition, the drive unit 3414 drives the variable aperture in the photographic lens 3420 to control the amount of light of the subject beam incident on the imaging element 100.

[0259] The drive unit 3402 has a control circuit that performs charge accumulation control such as timing control and area control of the imaging element 100 according to instructions from the system control unit 3401. Additionally, the operation unit 3408 receives instructions from the photographer via a release button or the like.

[0260] The imaging element 100 transmits pixel signals to the image processing unit 3411 of the system control unit 3401. The image processing unit 3411 uses the working memory 3404 as a working area to generate image data obtained by performing various image processing operations. For example, when generating image data in JPEG file format, compression processing is performed after generating a color image signal based on the signal obtained using the Bayer array. The generated image data is recorded in the recording unit 3405 and converted into a display signal, which is then displayed on the display unit 3406 for a preset time period.

[0261] The metering unit 3403 detects the brightness distribution of the scene before a series of photographic processes used to generate image data. The metering unit 3403 may include, for example, a 1-megapixel AE sensor. The arithmetic unit 3412 of the system control unit 3401 receives the output of the metering unit 6603 and calculates the brightness of each area of ​​the scene.

[0262] The arithmetic unit 3412 determines the shutter speed, aperture value, and ISO sensitivity based on the calculated brightness distribution. The metering unit 3403 can also be used in the image sensor 100. It should be noted that the arithmetic unit 3412 also performs various calculations to operate the image sensor 3400. The drive unit 3402 can also be partially or entirely mounted in the image sensor 100. A portion of the system control unit 3401 can also be mounted in the image sensor 100.

[0263] As described above, the exposure value 2803 relative to the pixel block 200 is not provided by the camera element 100 from outside the camera element 100, but is calculated within the control block 400 corresponding to the pixel block 200 within the camera element 100. In order to perform image processing outside the camera element 100, in addition to the block image data IMG(i,j) output for each pixel block 200, the camera element 100 also needs to output the exposure value 2803 calculated by each control block 400.

[0264] Therefore, the camera element 100 does not set the exposure value 2803 at the head or tail of the block image data IMG(i,j), but embeds the exposure value 2803 into the block image data IMG(i,j) itself.

[0265] Therefore, in order to set the exposure value 2803, it is not necessary to set the header and tail in the block image data IMG(i,j). That is, at least the output circuit section (not shown) connected to each data output line only needs to output the block image data IMG(i,j), and the circuit that performs data rearrangement, assigns headers and tails, and serially transmits the data from the data output line can be directly used.

[0266] In addition, even when a head and tail are required, the size increase of the head and tail can be suppressed because the exposure value of 2803 is not set.

[0267] Furthermore, in the image processing unit 3411 outside the imaging element 100, image processing can be performed directly using the block image data IMG(i,j) embedded with the exposure value 2803, without needing to obtain the exposure value 2803 from the head or tail of the block image data IMG(i,j). That is, by pre-obtaining the embedding position of the exposure value 2803 set by the setting unit 2330, the image processing unit 3411 can read the exposure value 2803 from the block image data IMG(i,j) and perform image processing.

[0268] Additionally, the image processing unit 3411 performs the following processing, for example. In the block image data IMG(i,j), the exposure value 2803 is read from the pixel data 2600 embedded with the exposure value 2803. Then, the image processing unit 3411 sets the least significant bit of the pixel data 2600 from which the exposure value 2803 has been read to 0 (in the cases of embedding examples 1 to 3). Alternatively, the image processing unit 3411 treats this pixel data 2600 as defective pixel data and interpolates and fills it with surrounding pixel data.

[0269] At this point in time, since the exposure time of each pixel block 200 is different, a step difference can be seen at the boundary of the pixel block 200. Therefore, the image processing unit 3411 applies a digital gain based on the read exposure value 2803. A larger digital gain is applied to pixel blocks 200 with shorter exposure times, and a lower digital gain is applied to pixel blocks 200 with longer exposure times. The digital gain is inversely proportional to the exposure time, using a reference exposure time as the digital gain multiplied by 1.

[0270] Or, such as Figure 12 As with regions 5, T12 to T18, sometimes the image is not output. In this case, the image processing unit 3411 discards the block image data IMG(i,j) from that pixel block 200 as invalid data and only outputs the valid block image data IMG(i,j) of T11 and T19.

[0271] Alternatively, the system control unit 3401 may also have the same pseudo-random number circuit as the imaging element 100. In this case, it is only necessary to make the pseudo-random number circuit in the imaging element 100 and the pseudo-random number circuit in the system control unit 3401 operate at the same timing. As a result, since each pseudo-random number circuit outputs exactly the same pseudo-random number, it is not necessary to output information from the imaging element 100, such as which coordinate position of the exposure value 2803 is embedded in the block image data IMG(i,j). Thus, the readout of exposure value 2803, the correction of block image data IMG(i,j), and the block step difference correction via digital gain are automatically performed in the system control unit 3401. Using the corrected block image data IMG(i,j), recording and image display are possible.

[0272] It should be noted that the present invention is not limited to the above-described contents, and these contents can be combined in any way. Furthermore, other embodiments considered within the scope of the technical concept of the present invention are also included within the scope of the present invention. Explanation of reference numerals in the attached figures

[0273] 100, 100A, 100B... Imaging element, 101... Pixel unit, 102... Control circuit unit, 103... Data processing unit, 110... First semiconductor substrate, 120... Second semiconductor substrate, 121... Peripheral circuit unit, 130... Third semiconductor substrate, 200... Pixel block, 201... Pixel, 202... Signal line, 210... Pixel group, 300... Photoelectric conversion unit, 301... Transmission unit, 302... Discharge unit, 304... Reset unit, 305... Pixel output unit, 306... Load circuit Stream source, 310...readout unit, 351...magnification unit, 352...selection unit, 400, 400A, 400B...control blocks, 401...pixel control unit, 402...signal transmission unit, 411...auto-exposure processing unit, 412...exposure control unit, 413...pixel drive unit, 421...signal input unit, 422...signal processing unit, 423...signal output unit, 2330...setting unit, 2340...replacement unit, IMG...image data, IMG(i,j)...block image data, 2803...exposure value.< / m>

Claims

1. A camera element, characterized in that, have: A pixel unit, which is configured with a plurality of pixels, the plurality of pixels including a photoelectric conversion unit that converts light into electrical charge; The exposure processing unit calculates the accumulation time of the charge converted by the photoelectric conversion unit; as well as The replacement unit replaces a portion of the first data related to the signal read from the pixel with second data related to the accumulation time calculated by the exposure processing unit.

2. The imaging element according to claim 1, characterized in that, The replacement part replaces a portion of the first data of the first pixel among the plurality of pixels with the second data.

3. The imaging element according to claim 2, characterized in that, The first pixel is configured in multiple units within the pixel portion.

4. The imaging element according to claim 3, characterized in that, In the pixel portion, pixel blocks having at least one first pixel are arranged along a first direction and a second direction intersecting the first direction. The replacement part replaces a portion of the first data of the first pixel in the pixel block with the second data.

5. The imaging element according to claim 4, characterized in that, The pixel block has a plurality of the first pixels.

6. The imaging element according to claim 5, characterized in that, The first pixel is arranged in the pixel block along the first direction or the second direction.

7. The imaging element according to claim 6, characterized in that, The first pixel is arranged adjacent to each other in the pixel block along the first direction or the second direction.

8. The imaging element according to claim 4, characterized in that, The pixel portion is disposed on the first semiconductor substrate. The exposure processing unit is disposed on a second semiconductor substrate connected to the first semiconductor substrate.

9. The imaging element according to claim 8, characterized in that, In the second semiconductor substrate, the exposure processing unit is disposed in the control circuit unit located at a position opposite to the pixel unit.

10. The imaging element according to claim 9, characterized in that, The control circuit section has multiple exposure processing sections.

11. The imaging element according to claim 10, characterized in that, In the control circuit section, control blocks, each having the exposure processing section, are arranged along the first direction and the second direction. The exposure processing unit calculates the accumulation time of the photoelectric conversion unit contained in the pixel of the corresponding pixel block.

12. The imaging element according to claim 11, characterized in that, Each of the control blocks has the replacement part. The replacement unit replaces a portion of the first data of the first pixel in the corresponding pixel block with the second data.

13. The imaging element according to claim 12, characterized in that, The control block has an exposure control unit. The exposure control unit controls the accumulation time of the charge converted by the photoelectric conversion unit of the pixel in the corresponding pixel block based on the calculation results of the exposure processing unit.

14. The imaging element according to claim 13, characterized in that, The control block has a pixel driving unit, which is used to output the control signal generated by the exposure control unit to the pixel.

15. The imaging element according to claim 14, characterized in that, The control block has a level shifting unit that adjusts the voltage level of the control signal generated by the exposure control unit. The pixel driving unit outputs the control signal, whose voltage level has been adjusted by the level shifting unit, to the pixel.

16. The imaging element according to claim 15, characterized in that, The pixel has a transmission section that transmits the charge of the photoelectric conversion section. The pixel driver outputs a transmission control signal for controlling the transmission unit.

17. The imaging element according to claim 16, characterized in that, The pixel has a discharge section that discharges the charge from the photoelectric conversion section. The pixel driver outputs a discharge control signal for controlling the discharge unit.

18. The imaging element according to claim 15, characterized in that, The pixel has a discharge section that discharges the charge from the photoelectric conversion section. The pixel driver outputs a discharge control signal for controlling the discharge unit.

19. The imaging element according to claim 12, characterized in that, The control block has a signal processing unit that performs signal processing on the signal read from the pixel of the corresponding pixel block.

20. The imaging element according to claim 19, characterized in that, The signal processing unit has a conversion unit that converts the signal into a digital signal.

21. The imaging element according to claim 20, characterized in that, The replacement part replaces the least significant bit of the first data with the second data.

22. The imaging element according to claim 8, characterized in that, The second semiconductor substrate has a control circuit section disposed at a position opposite to the pixel section, and a peripheral circuit section disposed outside the control circuit section. The exposure processing unit is located within the peripheral circuit unit.

23. The imaging element according to claim 22, characterized in that, The peripheral circuit section has multiple exposure processing sections.

24. The imaging element according to claim 23, characterized in that, The exposure processing unit calculates the accumulation time of the photoelectric conversion unit contained in the pixel of the corresponding pixel block.

25. The imaging element according to claim 24, characterized in that, The peripheral circuit section has the replacement section. The replacement unit replaces a portion of the first data of the first pixel in the corresponding pixel block with the second data.

26. The imaging element according to claim 25, characterized in that, The control circuit section has a plurality of control blocks arranged along the first direction and the second direction. Each of the control blocks has an exposure control unit, which controls the accumulation time of the charge converted by the photoelectric conversion unit of the pixel in the corresponding pixel block based on the calculation result of the exposure processing unit.

27. The imaging element according to claim 26, characterized in that, The control block has a pixel driving unit, which is used to output the control signal generated by the exposure control unit to the pixel of the corresponding pixel block.

28. The imaging element according to claim 27, characterized in that, The control block has a level shifting unit that adjusts the voltage level of the control signal generated by the exposure control unit. The pixel driving unit outputs the control signal, whose voltage level has been adjusted by the level shifting unit, to the pixel.

29. The imaging element according to claim 28, characterized in that, The pixel has a transmission section that transmits the charge of the photoelectric conversion section. The pixel driver outputs a transmission control signal for controlling the transmission unit.

30. The imaging element according to claim 29, characterized in that, The pixel has a discharge section that discharges the charge from the photoelectric conversion section. The pixel driver outputs a discharge control signal for controlling the discharge unit.

31. The imaging element according to claim 28, characterized in that, The pixel has a discharge section that discharges the charge from the photoelectric conversion section. The pixel driver outputs a discharge control signal for controlling the discharge unit.

32. The imaging element according to claim 26, characterized in that, The control block has a signal processing unit that performs signal processing on the signal read from the pixel of the corresponding pixel block.

33. The imaging element according to claim 32, characterized in that, The signal processing unit has a conversion unit that converts the signal into a digital signal.

34. The imaging element according to claim 33, characterized in that, The replacement part replaces the least significant bit of the first data with the second data.

35. A camera device, which is a camera device according to any one of claims 1 to 34.

Images